Five lessons learned in two years of using go - hashnode

Jiedao jdon 2021-06-18 11:34:27
lessons learned years using hashnode

In this paper , I'll talk about some of these mistakes and the lessons I've learned from trying to mitigate them in future projects . This is by no means a discussion of the ideal solution , It's just that I use Go The experience of learning and developing ideas :


1. Goroutines

in my opinion ,Go What's so fascinating about being a language ( Besides its simplicity and proximity C Performance of ) It's easy to write concurrent code .Goroutines It's about writing concurrent code Go The way .goroutine It's a lightweight thread , Or green thread , Yes , It's not a kernel thread .Goroutines It's the green thread , Because their scheduling is completely controlled by Go Runtime rather than operating system management .Go The scheduler is responsible for goroutine Multiplexing to real kernel threads , It's good for you goroutine Very lightweight in terms of startup time and memory requirements , So as to allow go Applications run millions of goroutine!

In my submission Go The way concurrency is handled is unique , A common mistake is to deal with any other language ( for example Java) It's handled in a concurrent way Go Concurrent .Go How different is the way concurrency is handled , One of the most famous examples can be summed up as :

Don't communicate through shared memory , Share memory through communication .

A very common situation is that the application will have multiple goroutine Access shared memory blocks . therefore , for example , We are implementing a connection pool , There is an array of available connections , Every goroutine You can get or release connections . The most common method is to use mutex, It's only allowed to hold Of goroutinemutex Exclusive access to the connection array at a given time . So the code looks like this ( Note that some details are abstracted to keep the code concise ):

type ConnectionPool struct {
  Mu          sync.Mutex
  Connections []Connection
func (pool *ConnectionPool) Acquire() Connection {
  defer pool.Mu.Unlock()
  //acquire and return a connection
func (pool *ConnectionPool) Release(c Connection) {
  defer pool.Mu.Unlock()
  //release the connection c

That seems reasonable , But what if we forget to implement locking logic ? What if we do implement it and forget to lock in one of the many features ? If we don't forget to lock , It's about forgetting to unlock ? If we only lock in part of the critical area ( Under lock ) What will happen ? Or if we lock in parts that don't belong to the critical zone ( Over locking ) What do I do ? It seems error prone , And it's usually not Go The way concurrency is handled .

This brings us back to Go The mantra of “ Don't communicate through shared memory , Sharing memory through communication ”. Understand what this means , We first need to understand what is Go passageway channel? The channel is to achieve goroutine Communication between Go The way . It's essentially a thread safe data pipeline , allow goroutine Send or receive data between them , Without accessing the shared memory block .Go Channels can also be buffered , This allows it to control the number of simultaneous calls , Effectively acting as a semaphore !

therefore , Re modify our code to share it through communication rather than locking , We get a code that looks like this ( Note that some details are abstracted to keep the code concise ):

type ConnectionPool struct {

  Connections chan Connection


func NewConnectionPool(limit int) *ConnectionPool {

  connections := make(chan Connection, limit)

  return &{ Connections: connections }


func (pool *ConnectionPool) Acquire() Connection {

  <- pool.Connections

  //acquire and return a connection


func (pool *ConnectionPool) Release(c Connection) {

  pool.Connections <- c

  //release the connection c


Use Go Channels not only reduce the size and overall complexity of the code , It also abstracts the need to explicitly implement thread safety . So now the data structure itself is essentially thread safe , So even if we forget that , It still works .

There are many advantages to using channels , This example just touches the surface , But the lesson here is not to write in any other language Go Write concurrent code in .


2. If you can take a single example , That's a single example

Go Applications may have to access Database or Cache etc. , They are examples of resources with connection pools , This means that there is a limit on the number of concurrent connections to the resource . According to my experience ,Go Most of the connection objects in ( database 、 Cache, etc ) Are built as thread safe connection pools , There can be more than one goroutine Use at the same time , Not a single connection .

therefore , Suppose we have a Go Applications , it mysql Through one *sql.DB Object is accessed as a database , The object is essentially a pool of connections to the database . If there are many applications goroutines, So create a new *sql.DB The object is meaningless , In fact, this can cause the connection pool to run out ( Note that not closing the connection after use can also lead to this situation ). therefore *sql.DB It makes sense that objects representing connection pools must be singletons , So even goroutine Trying to create a new object , It also returns the same object , Thus, multiple connection pools are not allowed .

It's usually a good practice to create objects that can be shared over the life cycle of an application singleton , Because it encapsulates this logic and prevents code from not following this policy . A common pitfall is that the implementation itself is not a thread safe creation logic singleton . for example , Consider the following code ( Note that some details are abstracted to keep the code concise ):

var dbInstance *DB
func DBConnection() *DB {
  if dbInstance != nil {
    return dbInstance
  dbInstance = &sql.Open(...)
  return dbInstance

The previous code checks if the singleton object is not nil( That means it was created before ), In this case, it returns it , But if it is ,nil Then it creates a new connection , Assign it to the singleton object and return it . In principle, , This should only create a database connection , But it's not thread safe .

consider 2 individual goroutine Call the function at the same time DBConnection() The situation of . Maybe the first one goroutine Read dbInstance And find its value nil Then continue to create a new connection , But before the newly created instance is assigned to the singleton object , the second goroutine Perform the same check as well , Come to the same conclusion and continue to create a new connection , Leave us 2 A connection instead of 1 individual .

This problem can be dealt with using the locks discussed in the previous section , But it's not Go The way .Go Supports default thread safe atomic operations , So if we can use something that's thread safe instead of explicitly implementing it , So let's do this !

therefore , Revisit our code to make it thread safe , The code we get looks like this ( Note that some details are abstracted to keep the code concise ):

var dbOnce sync.Once
var dbInstance *DB
func DBConnection() *DB {
  dbOnce.Do(func() {
    dbInstance = &sql.Open(...)
  return dbInstance

This code uses a called Go structure sync.Once, It allows us to write a function that only executes once . such , Even if there are more than one goroutine Try to execute at the same time , The connection creation segment is also guaranteed to run only once .


3. Be careful of blocking the code

Sometimes your Go The application performs blocking calls , It could be a request for an external service that might not respond , Or it could be calling functions that are blocked under certain conditions . Based on experience , Never assume that the call will return at the right time , Because sometimes they don't .

The usual way to deal with this situation is to set a timeout , After that, the call will be cancelled and can continue . Also recommended ( If possible ) Block calls to individual routines , Instead of blocking the main routine .

So let's think about it Go Application needs to pass http. Gohttp The client will not time out by default , But we can set the timeout as follows :

&http.Client{Timeout: time.Minute}

Although it's a good job , But it's very limited , Because now we're on the same client ( This could be a singleton object ) All requests executed have the same timeout .Go There is one named contex package , It allows us to request the value of the scope 、 Cancellation signals and timeouts are passed to all involved in processing the request goroutine. We can use it like this :

ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
req := req.WithContext(ctx)
res, err := c.Do(req)
The previous code allows you to set the timeout for each request , You can also cancel the request or pass the value of each request as needed . If the request is sent goroutine Derived other goroutine, And these goroutine You need to exit when the request times out , So it's going to be particularly useful .

In this case , We are very lucky :http Packages support context , But what if we're dealing with an unsupported blocking function ? There is a temporary way to use Go Of select Statement and Go channels( Again !) To implement the timeout logic . Consider the following code :

func SendValue(){
  sendChan := make(chan bool, 1)
  go func() {
    sendChan <- Send()
  select {
    case <-sendChan:
    case <-time.After(time.Minute):
  //continue logic in case send didn't timeout

We have one called SendValue() Function of , It calls a call called Send() The blocking function of , This function may block forever . therefore , We initialize a buffered Boolean channel , And in a separate goroutine Executing blocking functions on , When it's done, send a signal in the channel , And in the Lord goroutine in , We block the waiting channel to produce a value ( The request is successful ) Or wait for 1 Return in minutes . Please note that , Code Missing cancel Send() Function to terminate goroutine The logic of ( Otherwise, it will lead to goroutine leak ).


4. Graceful termination and cleanup

If you are writing a long running process , for example Web Server or background workers, etc , You are at risk of sudden termination . This termination may be due to the process being terminated by the scheduler , Or even if you're releasing new code and launching it .

Usually , A long-running process may contain data in its memory , If the process is to terminate , This data will be lost , Or it may hold resources that need to be released back to the resource pool . So when a process is killed , We need to be able to perform graceful termination . To terminate the process gracefully means that we intercept kill signal And execute application specific closing logic , To make sure everything is OK before the actual termination .

therefore , Suppose we're building a Web The server , We usually want it to run like this :

server := NewServer()

The key idea here is Run() Functions that run infinitely , In a sense , If we were main Call it at the end of the function , The process does not exit , Only in Run() The function exits when it exits .

You can implement the server logic to check the shutdown signal And just signal Exit on receipt of closure , As shown below :

func (server *Server) Run() {
  for {
    //infinite loop
    select {
      case <- server.shutdown:
        //do work

The server circulates to see if the signal is sent through the close channel , In this case, it exits the server loop , Otherwise, continue to carry out its work .

Now? , The only missing part of this puzzle is the ability to intercept operating system interrupts , for example (SIGKILL or SIGTERM), And call Server.Shutdown(), It performs the closing logic ( Flush memory to disk 、 Release resources 、 Clean up, etc ), And send a shutdown signal to terminate the server loop . We can do it in the following ways :

func main() {
  signals := make(chan os.Signal, 1)
  signal.Notify(signals, os.Interrupt)
  server := NewServer()
  select {
    case <-signals:

Created a os.Signal Type of buffer channel , And it's happening os interrupt Send a signal to the channel when it is interrupted .main The rest of the function runs the server and prevents waiting on the channel . When a signal is received , This means that an operating system interrupt has occurred , Instead of quitting immediately , It calls the server shutdown logic , Give it a chance to end gracefully .


5. Go modular FTW

Your Go It's common for applications to have external dependencies , for example , You are using mysql Driver or redis Driver or any other package . When you first build an application , The build process will get the latest version of each dependency , It's great . Now you've built your binaries , You tested it and it works , So you're in the production phase .

After a month , You need to add new features or patches , This may not require new dependencies , But you need to rebuild the application to generate new binaries . The build process will also get the latest version of each required package , But this version may be different from the one you got when you first built it , And may contain significant changes that cause disruption to the application itself . So obviously , Unless you explicitly choose upgrade , Otherwise, we need to manage this problem by always getting the same version of each dependency .

Go.11 Introduced go.mod This is Go A new way to deal with dependency version control in . When you initialize a Go mod And then when you build it , It will automatically generate a go.mod Documents and a go.sum file .mod The file looks like this :

go 1.14
require ( v1.5.0 v1.12.3 // indirect v5.2.9 v2.3.0

It's locked in Go Version and the version used for each dependency , So for example redis v5.2.9, Even if redis The repository v5.3.0 Released as its latest version to ensure stability , We will also always get every build . Please note that , A second dependency marked as indirect means that the dependency is not imported directly by your application , Instead, one of its dependencies imports , And it also locks its version .

Use Go mods There are many other benefits , for example :

  1. It will run automatically go mod tidy, To remove any unnecessary dependencies .
  2. It allows you to run code from any directory ( stay go mods Before ,go The project must be placed in a specific directory ).
  3. It wraps the entire application into a module Applications that can be imported into new projects . If you decide to wrap some logic in one package ( For example, the recorder package ) And import it into all other projects , In order to use this logging function in multiple code bases , This will be particularly useful .


本文为[Jiedao jdon]所创,转载请带上原文链接,感谢

  1. HTML + CSS + JavaScript to achieve cool Fireworks (cloud like particle text 3D opening)
  2. HTML + CSS + JavaScript realizes 520 advertising love tree (including music), which is necessary for programmers to express themselves
  3. Solve the problem of Web front-end deployment server (it can be deployed online without a server)
  4. HTML + CSS + JS make wedding countdown web page template (520 / Tanabata Valentine's Day / programmer advertisement)
  5. What else can driverless minibus do besides "Park connection"?
  6. Cloud native leads the era of all cloud development
  7. NRM mirror source management tool
  8. Bring it to you, flex Jiugong
  9. Lolstyle UI component development practice (II) -- button group component
  10. Deconstruction assignment in ES6
  11. Luo 2 peerless Tang clan was officially launched. The official gave a key point, and the broadcast time was implied
  12. 20初识前端HTML(1)
  13. 当新零售遇上 Serverless
  14. 20 initial knowledge of front-end HTML (1)
  15. When new retail meets serverless
  16. [golang] - go into go language lesson 5 type conversion
  17. [golang] - go into go language lesson 6 conditional expression
  18. HTML5(八)——SVG 之 path 详解
  19. HTML5 (8) -- detailed explanation of SVG path
  20. 需要开通VIP以后页面内容才能复制怎么办?控制台禁用javascript即可
  21. Web前端|CSS入门教程(超详细的CSS使用讲解,适合前端初学者)
  22. 实践积累 —— 用Vue3简单写一个单行横向滚动组件
  23. Serverless 全能选手,再下一城
  24. What if you need to open a VIP to copy the page content? Just disable JavaScript on the console
  25. Web front end | CSS introductory tutorial (super detailed CSS explanation, suitable for front-end beginners)
  26. Practice accumulation - write a single line horizontal scroll component simply with vue3
  27. Dili Reba is thin again. She looks elegant and high in a strapless hollow skirt, and her "palm waist" is beautiful to a new height
  28. Serverless all-round player, next city
  29. The difference between MySQL semi synchronous replication and lossless semi synchronous replication
  30. Vue表单设计器的终极解决方案
  31. The ultimate solution for Vue form designer
  32. Nginx从理论到实践超详细笔记
  33. Yu Shuxin's red backless swimsuit is split to the waist and tail, with a concave convex figure and excessive color matching, and his face is white to dazzling
  34. Nginx ultra detailed notes from theory to practice
  35. 【动画消消乐|CSS】086.炫酷水波浪Loading过渡动画
  36. typecho全站启用https
  37. CCTV has another popular employee. The off-site interpretation is very professional, and the appearance ability is no less than that of Wang Bingbing
  38. [animation Xiaole | CSS] 086. Cool water wave loading transition animation
  39. Enable HTTPS in Typecho
  40. 50天用JavaScript完成50个web项目,我学到了什么?
  41. 根据JavaScript中原生的XMLHttpRequest实现jQuery的Ajax
  42. What have I learned from completing 50 web projects with JavaScript in 50 days?
  43. "My neighbor doesn't grow up" has hit the whole network. There are countless horse music circles, and actor Zhou Xiaochuan has successfully made a circle
  44. 根据JavaScript中原生的XMLHttpRequest实现jQuery的Ajax
  45. Implement the Ajax of jQuery according to the native XMLHttpRequest in JavaScript
  46. Implement the Ajax of jQuery according to the native XMLHttpRequest in JavaScript
  47. 30 + women still wear less T-shirts and jeans. If they wear them like stars, they will lose weight
  48. 数栈技术分享前端篇:TS,看你哪里逃~
  49. Several stack technology sharing front end: TS, see where you escape~
  50. 舍弃Kong和Nginx,Apache APISIX 在趣链科技 BaaS 平台的落地实践
  51. Abandon the landing practice of Kong and nginx, Apache apisik on the baas platform of fun chain technology
  52. 浪迹天涯king教你用elementui做复杂的表格,去处理报表数据(合并表头,合并表体行和列)
  53. 前端HTML两万字图文大总结,快来看看你会多少!【️熬夜整理&建议收藏️】
  54. Wandering around the world king teaches you to use elementui to make complex tables and process report data (merge header, merge table body rows and columns)
  55. 路由刷新数据丢失 - vuex数据读取的问题
  56. Front end HTML 20000 word graphic summary, come and see how much you can【 Stay up late to sort out & suggestions]
  57. Route refresh data loss - vuex data reading problem
  58. Systemctl系统启动Nginx服务脚本
  59. Systemctl system startup nginx service script
  60. sleepless