Synchronization

This chapter is from the book

Go Fundamentals: Gopher Guides

Learn More Buy

This chapter is from the book

This chapter is from the book

Go Fundamentals: Gopher Guides

Learn More Buy

Error Management with Error Groups

One of the downsides of thesync.WaitGroupis that it has no error management built in to capture errors that occur in the goroutines. It also has an API that requires exact implementation; otherwise, it panics.

To help address some of these issues, thegolang.org/x/sync/errgroup,¹⁰Listing 13.15, package was introduced, providing a simpler API as well as built-in error management.

Listing 13.15 Thegolang.org/x/sync/errgroupPackage

$ go doc golang.org/x/sync/errgroup package errgroup // import "golang.org/x/sync/errgroup" Package errgroup provides synchronization, error propagation, and Contextcancelation for groups of goroutines working on subtasks of a common task. type Group struct{ ... } func WithContext(ctx context.Context) (*Group, context.Context)

Go Version: go1.19

The Problem

Consider Listing 13.16. In the example, a number of goroutines are launched to call thegenerateThumbnailfunctions. Async.WaitGroupis used to wait for all the goroutines to finish.

Listing 13.16 Managing Goroutines withsync.WaitGroup

funcTest_ThumbnailGenerator(t*testing.T) {t.Parallel()// image that we need thumbnails forconstimage="foo.png"varwg sync.WaitGroup// start 5 goroutines to generate thumbnailsfori:=0;i<5;i++ {wg.Add(1)// start a new goroutine for each thumbnailgogenerateThumbnail(&wg,image,i+1)}fmt.打印ln("Waiting for thumbnails to be generated")// wait for all goroutines to finishwg.Wait()fmt.打印ln("Finished generate all thumbnails") }

Inside thegenerateThumbnailfunction, Listing 13.17, we see that there is an error that occurs if thesizeargument is divisible by 5, which causes a panic, and the application crashes.

Listing 13.17 ThegenerateThumbnailFunction

funcgenerateThumbnail(wg*sync.WaitGroup,imagestring,sizeint) {deferwg.Done()// error if the size is divisible by 5ifsize%5==0{// how do we return this error back to the main // goroutine without panicking?err:=fmt.Errorf("%d is divisible by 5",size)panic(err) }// thumbnail to be generatedthumb:=fmt.Sprintf("%s@%dx.png",image,size)fmt.打印ln("Generating thumbnail:",thumb)// wait for the thumbnail to be readytime.Sleep(time.Millisecond*time.Duration(size))fmt.打印ln("Finished generating thumbnail:",thumb) }

$ go test -v === RUN Test_ThumbnailGenerator === PAUSE Test_ThumbnailGenerator === CONT Test_ThumbnailGenerator Waiting for thumbnails to be generated panic: 5 is divisible by 5 goroutine 9 [running]: demo.generateThumbnail(0x0?, {0x10268ac77?, 0x0?}, 0x0?) ./demo_test.go:47 +0x21c created by demo.Test_ThumbnailGenerator ./demo_test.go:24 +0x4c exit status 2 FAIL demo 0.619s

Go Version: go1.19

The Error Group

Theerrgroup.Group¹¹type provides a minimal API when compared to thesync.WaitGrouptype. There are only two methods:errgroup.Group.Go¹²anderrgroup.Group.Wait.¹³

Theerrgroup.Grouptype, Listing 13.18, manages the counter for you, so there is no need for counter management as there is with thesync.WaitGrouptype.

Listing 13.18 Theerrgroup.GroupType

$ go doc golang.org/x/sync/errgroup.Group package errgroup // import "golang.org/x/sync/errgroup" type Group struct { // Has unexported fields. } A Group is a collection of goroutines working on subtasks that are partof the same overall task. A zero Group is valid and does not cancel on error. func WithContext(ctx context.Context) (*Group, context.Context) func (g *Group) Go(f func() error) func (g *Group) Wait() error

Go Version: go1.19

TheGoMethod

You use theerrgroup.Group.Gomethod, Listing 13.19, to launch a goroutine for the providedfunc() errorfunction provided. Thefunc() errorfunction is executed in a goroutine. If the function returns an error, theerrgroup.Grouptype captures the error, cancels the other goroutines, and returns the error to the caller from theerrgroup.Group.Waitmethod.

Listing 13.19 Theerrgroup.Group.GoMethod

$ go doc golang.org/x/sync/errgroup.Group.Go package errgroup // import "golang.org/x/sync/errgroup" func (g *Group) Go(f func() error) Go calls the given function in a new goroutine. The first call to return a non-nil error cancels the group; its errorwill be returned by Wait.

Go Version: go1.19

TheWaitMethod

You useerrgroup.Group.Waitmethod, Listing 13.20, to wait for all the goroutines to finish. If any of the goroutines return an error, theerrgroup.Grouptype returns the error to the caller.

Listing 13.20 Theerrgroup.Group.WaitMethod

$ go doc golang.org/x/sync/errgroup.Group.Wait package errgroup // import "golang.org/x/sync/errgroup" func (g *Group) Wait() error Wait blocks until all function calls from the Go method have returned,then returns the first non-nil error (if any) from them.

Go Version: go1.19

It is important to understand theerrgroup.Group.Waitmethod returnsonlythefirsterror that occurs. Any other errors that occur are ignored. In Listing 13.21, we are calling theGomethod on anerrorgroup.Group10 times. Each time we pass a function that sleeps for a random time, prints a messages, and then returns an error. From the output of the test, you can see that the error returned by theWaitmethod was returned from function number 4. The other nine errors are discarded.

Listing 13.21 Theerrgroup.Group.WaitMethod Returns Only the First Error

funcTest_ErrorGroup_Multiple_Errors(t*testing.T) {t.Parallel()varwg errgroup.Groupfori:=0;i<10;i++ {i:=i+1wg.Go(func()error{time.Sleep(time.Millisecond*time.Duration(rand.Intn(10)))fmt.打印f("about to error from %d\n",i)returnfmt.Errorf("error %d",i)}) }err:=wg.Wait()iferr!=零{t.Fatal(err) }}

$ go test -v === RUN Test_ErrorGroup_Multiple_Errors === PAUSE Test_ErrorGroup_Multiple_Errors === CONT Test_ErrorGroup_Multiple_Errors about to error from 4 about to error from 6 about to error from 3 about to error from 1 about to error from 9 about to error from 10 about to error from 8 about to error from 5 about to error from 2 about to error from 7 demo_test.go:38: error 4 --- FAIL: Test_ErrorGroup_Multiple_Errors (0.01s) FAIL exit status 1 FAIL demo 0.263s

Go Version: go1.19

Listening for Error Group Cancellation

When launching a number of goroutines, it can often be useful to let others know the tasks have all completed. Theerrgroup.WithContext¹⁴function, Listing 13.22, returns a newerrgroup.Group, as well as acontext.Contextthat can be listened to for cancellation.

Listing 13.22 Theerrgroup.WithContextFunction

funcTest_ErrorGroup_Context(t*testing.T) {t.Parallel()// create a new error group // and a context that will be canceled // when the group is donewg,ctx:=errgroup.WithContext(context.Background())// create a quit channel for the goroutine // waiting for the context to be canceled // can close to signal the goroutine has finishedquit:=make(陈结构{})// launch a goroutine that will // wait for the errgroup context to finishgo func() {fmt.打印ln("waiting for context to cancel")// wait for the context to be canceled<-ctx.Done()fmt.打印ln("context canceled")// close the quit channel so the test // will finishclose(quit)}()// add a task to the errgroupwg.Go(func()error{time.Sleep(time.Millisecond*5)return零})// wait for the errgroup to finisherr:=wg.Wait()iferr!=零{t.Fatal(err) }// wait for the context goroutine to finish<-quit}

$ go test -v === RUN Test_ErrorGroup_Context === PAUSE Test_ErrorGroup_Context === CONT Test_ErrorGroup_Context waiting for context to cancel context canceled --- PASS: Test_ErrorGroup_Context (0.01s) PASS ok demo 0.725s

Go Version: go1.19

Wrapping Up Error Groups

Theerrgroup.Grouptype provides a simpler API than thesync.WaitGrouptype. It also provides built-in error management. This makes managing goroutines with error handling much easier. The downside is that it is not as flexible as thesync.WaitGrouptype, where you are in charge of managing the counter.

Which type you choose to use will vary from situation to situation, so it is important to understand the tradeoffs and benefits of each before deciding which type to use.

Using aerrgroup.Groupallows you to clean up your code significantly to make it easier to understand and to manage errors.

As you can see from Listing 13.23, thegenerateThumbnailfunction no longer needs to take async.WaitGroupas an argument.

Listing 13.23 Update thegenerateThumbnailFunction to useerrgroup.Group

funcgenerateThumbnail(imagestring,sizeint)error{// error if the size is divisible by 5ifsize%5==0{returnfmt.Errorf("%d is divisible by 5",size) }// thumbnail to be generatedthumb:=fmt.Sprintf("%s@%dx.png",image,size)fmt.打印ln("Generating thumbnail:",thumb)// wait for the thumbnail to be readytime.Sleep(time.Millisecond*time.Duration(size))fmt.打印ln("Finished generating thumbnail:",thumb)return零}

Being able to return an error from the function means the function no longer needs to panic.

As you can see from the output, Listing 13.24, thegenerateThumbnailfunction no longer panics, and the test is now able to exit properly.

Listing 13.24 Using theerrgroup.GroupType

funcTest_ThumbnailGenerator(t*testing.T) {t.Parallel()// image that we need thumbnails forconstimage="foo.png"// create a new error groupvarwg errgroup.Group// start 5 goroutines to generate thumbnailsfori:=0;i<5;i++ {// capture the i to the current scopei:=i// start a new goroutine for each thumbnailwg.Go(func()error{/ /返回generateThumbn的结果ailreturngenerateThumbnail(image,i) })}fmt.打印ln("Waiting for thumbnails to be generated")// wait for all goroutines to finisherr:=wg.Wait()// check for any errorsiferr!=零{t.Fatal(err) }fmt.打印ln("Finished generate all thumbnails") }

$ go test -v === RUN Test_ThumbnailGenerator === PAUSE Test_ThumbnailGenerator === CONT Test_ThumbnailGenerator Waiting for thumbnails to be generated Generating thumbnail: foo.png@4x.png Generating thumbnail: foo.png@3x.png Generating thumbnail: foo.png@1x.png Generating thumbnail: foo.png@2x.png Finished generating thumbnail: foo.png@1x.png Finished generating thumbnail: foo.png@2x.png Finished generating thumbnail: foo.png@3x.png Finished generating thumbnail: foo.png@4x.png demo_test.go:43: 0 is divisible by 5 --- FAIL: Test_ThumbnailGenerator (0.00s) FAIL exit status 1 FAIL demo 0.570s

Go Version: go1.19