In this paper , I'll talk about some of these mistakes and the lessons I've learned from trying to mitigate them in future projects . This is by no means a discussion of the ideal solution , It's just that I use Go The experience of learning and developing ideas ：
1. Goroutines
in my opinion ,Go What's so fascinating about being a language （ Besides its simplicity and proximity C Performance of ） It's easy to write concurrent code .Goroutines It's about writing concurrent code Go The way .goroutine It's a lightweight thread , Or green thread , Yes , It's not a kernel thread .Goroutines It's the green thread , Because their scheduling is completely controlled by Go Runtime rather than operating system management .Go The scheduler is responsible for goroutine Multiplexing to real kernel threads , It's good for you goroutine Very lightweight in terms of startup time and memory requirements , So as to allow go Applications run millions of goroutine！
In my submission Go The way concurrency is handled is unique , A common mistake is to deal with any other language （ for example Java） It's handled in a concurrent way Go Concurrent .Go How different is the way concurrency is handled , One of the most famous examples can be summed up as ：
Don't communicate through shared memory , Share memory through communication .
A very common situation is that the application will have multiple goroutine Access shared memory blocks . therefore , for example , We are implementing a connection pool , There is an array of available connections , Every goroutine You can get or release connections . The most common method is to use mutex, It's only allowed to hold Of goroutinemutex Exclusive access to the connection array at a given time . So the code looks like this （ Note that some details are abstracted to keep the code concise ）：
type ConnectionPool struct {
Mu sync.Mutex
Connections []Connection
}
func (pool *ConnectionPool) Acquire() Connection {
pool.Mu.Lock()
defer pool.Mu.Unlock()
//acquire and return a connection
}
func (pool *ConnectionPool) Release(c Connection) {
pool.Mu.Lock()
defer pool.Mu.Unlock()
//release the connection c
}
|
That seems reasonable , But what if we forget to implement locking logic ？ What if we do implement it and forget to lock in one of the many features ？ If we don't forget to lock , It's about forgetting to unlock ？ If we only lock in part of the critical area （ Under lock ） What will happen ？ Or if we lock in parts that don't belong to the critical zone （ Over locking ） What do I do ？ It seems error prone , And it's usually not Go The way concurrency is handled .
This brings us back to Go The mantra of “ Don't communicate through shared memory , Sharing memory through communication ”. Understand what this means , We first need to understand what is Go passageway channel？ The channel is to achieve goroutine Communication between Go The way . It's essentially a thread safe data pipeline , allow goroutine Send or receive data between them , Without accessing the shared memory block .Go Channels can also be buffered , This allows it to control the number of simultaneous calls , Effectively acting as a semaphore ！
therefore , Re modify our code to share it through communication rather than locking , We get a code that looks like this （ Note that some details are abstracted to keep the code concise ）：
type ConnectionPool struct {
Connections chan Connection
}
func NewConnectionPool(limit int) *ConnectionPool {
connections := make(chan Connection, limit)
return &{ Connections: connections }
}
func (pool *ConnectionPool) Acquire() Connection {
<- pool.Connections
//acquire and return a connection
}
func (pool *ConnectionPool) Release(c Connection) {
pool.Connections <- c
//release the connection c
}
Use Go Channels not only reduce the size and overall complexity of the code , It also abstracts the need to explicitly implement thread safety . So now the data structure itself is essentially thread safe , So even if we forget that , It still works .
There are many advantages to using channels , This example just touches the surface , But the lesson here is not to write in any other language Go Write concurrent code in .
2. If you can take a single example , That's a single example
Go Applications may have to access Database or Cache etc. , They are examples of resources with connection pools , This means that there is a limit on the number of concurrent connections to the resource . According to my experience ,Go Most of the connection objects in （ database 、 Cache, etc ） Are built as thread safe connection pools , There can be more than one goroutine Use at the same time , Not a single connection .
therefore , Suppose we have a Go Applications , it mysql Through one *sql.DB Object is accessed as a database , The object is essentially a pool of connections to the database . If there are many applications goroutines, So create a new *sql.DB The object is meaningless , In fact, this can cause the connection pool to run out （ Note that not closing the connection after use can also lead to this situation ）. therefore *sql.DB It makes sense that objects representing connection pools must be singletons , So even goroutine Trying to create a new object , It also returns the same object , Thus, multiple connection pools are not allowed .
It's usually a good practice to create objects that can be shared over the life cycle of an application singleton , Because it encapsulates this logic and prevents code from not following this policy . A common pitfall is that the implementation itself is not a thread safe creation logic singleton . for example , Consider the following code （ Note that some details are abstracted to keep the code concise ）：
var dbInstance *DB
func DBConnection() *DB {
if dbInstance != nil {
return dbInstance
}
dbInstance = &sql.Open(...)
return dbInstance
}
|
The previous code checks if the singleton object is not nil（ That means it was created before ）, In this case, it returns it , But if it is ,nil Then it creates a new connection , Assign it to the singleton object and return it . In principle, , This should only create a database connection , But it's not thread safe .
consider 2 individual goroutine Call the function at the same time DBConnection() The situation of . Maybe the first one goroutine Read dbInstance And find its value nil Then continue to create a new connection , But before the newly created instance is assigned to the singleton object , the second goroutine Perform the same check as well , Come to the same conclusion and continue to create a new connection , Leave us 2 A connection instead of 1 individual .
This problem can be dealt with using the locks discussed in the previous section , But it's not Go The way .Go Supports default thread safe atomic operations , So if we can use something that's thread safe instead of explicitly implementing it , So let's do this ！
therefore , Revisit our code to make it thread safe , The code we get looks like this （ Note that some details are abstracted to keep the code concise ）：
var dbOnce sync.Once
var dbInstance *DB
func DBConnection() *DB {
dbOnce.Do(func() {
dbInstance = &sql.Open(...)
}
return dbInstance
}
|
This code uses a called Go structure sync.Once, It allows us to write a function that only executes once . such , Even if there are more than one goroutine Try to execute at the same time , The connection creation segment is also guaranteed to run only once .
3. Be careful of blocking the code
Sometimes your Go The application performs blocking calls , It could be a request for an external service that might not respond , Or it could be calling functions that are blocked under certain conditions . Based on experience , Never assume that the call will return at the right time , Because sometimes they don't .
The usual way to deal with this situation is to set a timeout , After that, the call will be cancelled and can continue . Also recommended （ If possible ） Block calls to individual routines , Instead of blocking the main routine .
So let's think about it Go Application needs to pass http. Gohttp The client will not time out by default , But we can set the timeout as follows ：
&http.Client{Timeout: time.Minute}
|
Although it's a good job , But it's very limited , Because now we're on the same client （ This could be a singleton object ） All requests executed have the same timeout .Go There is one named contex package , It allows us to request the value of the scope 、 Cancellation signals and timeouts are passed to all involved in processing the request goroutine. We can use it like this ：
ctx, cancel := context.WithTimeout(context.Background(), time.Minute) defer cancel() req := req.WithContext(ctx) res, err := c.Do(req) The previous code allows you to set the timeout for each request , You can also cancel the request or pass the value of each request as needed . If the request is sent goroutine Derived other goroutine, And these goroutine You need to exit when the request times out , So it's going to be particularly useful . |
In this case , We are very lucky ：http Packages support context , But what if we're dealing with an unsupported blocking function ？ There is a temporary way to use Go Of select Statement and Go channels（ Again ！） To implement the timeout logic . Consider the following code ：
func SendValue(){
sendChan := make(chan bool, 1)
go func() {
sendChan <- Send()
}()
select {
case <-sendChan:
case <-time.After(time.Minute):
return
}
//continue logic in case send didn't timeout
}
|
We have one called SendValue（） Function of , It calls a call called Send（） The blocking function of , This function may block forever . therefore , We initialize a buffered Boolean channel , And in a separate goroutine Executing blocking functions on , When it's done, send a signal in the channel , And in the Lord goroutine in , We block the waiting channel to produce a value （ The request is successful ） Or wait for 1 Return in minutes . Please note that , Code Missing cancel Send（） Function to terminate goroutine The logic of （ Otherwise, it will lead to goroutine leak ）.
4. Graceful termination and cleanup
If you are writing a long running process , for example Web Server or background workers, etc , You are at risk of sudden termination . This termination may be due to the process being terminated by the scheduler , Or even if you're releasing new code and launching it .
Usually , A long-running process may contain data in its memory , If the process is to terminate , This data will be lost , Or it may hold resources that need to be released back to the resource pool . So when a process is killed , We need to be able to perform graceful termination . To terminate the process gracefully means that we intercept kill signal And execute application specific closing logic , To make sure everything is OK before the actual termination .
therefore , Suppose we're building a Web The server , We usually want it to run like this ：
server := NewServer() server.Run() |
The key idea here is Run() Functions that run infinitely , In a sense , If we were main Call it at the end of the function , The process does not exit , Only in Run() The function exits when it exits .
You can implement the server logic to check the shutdown signal And just signal Exit on receipt of closure , As shown below ：
func (server *Server) Run() {
for {
//infinite loop
select {
case <- server.shutdown:
return
default:
//do work
}
}
}
|
The server circulates to see if the signal is sent through the close channel , In this case, it exits the server loop , Otherwise, continue to carry out its work .
Now? , The only missing part of this puzzle is the ability to intercept operating system interrupts , for example （SIGKILL or SIGTERM）, And call Server.Shutdown（）, It performs the closing logic （ Flush memory to disk 、 Release resources 、 Clean up, etc ）, And send a shutdown signal to terminate the server loop . We can do it in the following ways ：
func main() {
signals := make(chan os.Signal, 1)
signal.Notify(signals, os.Interrupt)
server := NewServer()
server.Run()
select {
case <-signals:
server.Shutdown()
}
}
|
Created a os.Signal Type of buffer channel , And it's happening os interrupt Send a signal to the channel when it is interrupted .main The rest of the function runs the server and prevents waiting on the channel . When a signal is received , This means that an operating system interrupt has occurred , Instead of quitting immediately , It calls the server shutdown logic , Give it a chance to end gracefully .
5. Go modular FTW
Your Go It's common for applications to have external dependencies , for example , You are using mysql Driver or redis Driver or any other package . When you first build an application , The build process will get the latest version of each dependency , It's great . Now you've built your binaries , You tested it and it works , So you're in the production phase .
After a month , You need to add new features or patches , This may not require new dependencies , But you need to rebuild the application to generate new binaries . The build process will also get the latest version of each required package , But this version may be different from the one you got when you first built it , And may contain significant changes that cause disruption to the application itself . So obviously , Unless you explicitly choose upgrade , Otherwise, we need to manage this problem by always getting the same version of each dependency .
Go.11 Introduced go.mod This is Go A new way to deal with dependency version control in . When you initialize a Go mod And then when you build it , It will automatically generate a go.mod Documents and a go.sum file .mod The file looks like this ：
module github.com/org/module_name go 1.14 require ( github.com/go-sql-driver/mysql v1.5.0 github.com/onsi/ginkgo v1.12.3 // indirect gopkg.in/redis.v5 v5.2.9 gopkg.in/yaml.v2 v2.3.0 ) |
It's locked in Go Version and the version used for each dependency , So for example redis v5.2.9, Even if redis The repository v5.3.0 Released as its latest version to ensure stability , We will also always get every build . Please note that , A second dependency marked as indirect means that the dependency is not imported directly by your application , Instead, one of its dependencies imports , And it also locks its version .
Use Go mods There are many other benefits , for example ：