Go Concurrency Basics and Patterns

Key Takeaways:

Goroutines are cheap (~2 KB initial stack) and multiplex onto OS threads by the Go runtime scheduler; channels are the idiomatic way to communicate between them.

Unbuffered channels synchronize senders and receivers directly; buffered channels decouple them but can hide backpressure problems.

select with a time.After case is the standard timeout pattern; context.Context propagates cancellation and deadlines across goroutine trees.

The race detector (go test -race) catches data races but only along code paths exercised by your tests; it is not a proof of correctness.

For production, prefer errgroup for error propagation and singleflight to collapse duplicate concurrent work.

Goroutines and Channels: The Two Primitives

Imagine launching thousands of lightweight tasks to fetch data from multiple APIs simultaneously. Each task runs as a goroutine, started with the go keyword. According to the Go FAQ, goroutines start with a tiny stack (a few kilobytes) that grows as needed, and the runtime multiplexes them onto a limited number of OS threads. Channels, on the other hand, are typed conduits created with make(chan T). They facilitate communication between goroutines, using the <- operator to send and receive.

package main

import (
 "fmt"
 "net/http"
 "time"
)

// fetchAll issues N concurrent HTTP GETs and returns their status codes.
// Note: production use should add a client timeout and a bounded worker
// pool; spawning one goroutine per URL is fine for dozens, not millions.
func fetchAll(urls []string) []int {
 results := make(chan int, len(urls))
 for _, u := range urls {
 go func(u string) {
 resp, err := http.Get(u)
 if err != nil {
 results <- 0
 return
 }
 results <- resp.StatusCode
 resp.Body.Close()
 }(u)
 }
 codes := make([]int, 0, len(urls))
 for range urls {
 codes = append(codes, <-results)
 }
 return codes
}

func main() {
 urls := []string{
 "https://go.dev",
 "https://pkg.go.dev",
 "https://golang.org",
 }
 fmt.Println(fetchAll(urls)) // e.g. [200 200 200]
}

The key point here is the loop variable capture. Passing u as a parameter ensures each goroutine captures its own copy. Since Go 1.22, this pattern is safer, but explicit passing remains idiomatic and compatible with older versions.

Select, Timeouts, and Cancellation

The select statement is a powerful tool that lets a goroutine wait on multiple channel operations simultaneously. It's the backbone of timeout handling. The common pattern pairs a blocking receive with time.After to prevent goroutines from leaking if the other side stalls.

package main

import (
 "context"
 "fmt"
 "time"
)

// fetchWithTimeout runs work against a context deadline.
// The context carries cancellation across the goroutine tree.
func fetchWithTimeout(ctx context.Context, work func() string) (string, error) {
 ch := make(chan string, 1)
 go func() { ch <- work() }()

 select {
 case result := <-ch:
 return result, nil
 case <-ctx.Done():
 return "", ctx.Err() // "context deadline exceeded"
 }
}

func main() {
 ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
 defer cancel()

 out, err := fetchWithTimeout(ctx, func() string {
 time.Sleep(3 * time.Second) // simulates a slow backend
 return "done"
 })
 fmt.Println(out, err) // " context deadline exceeded"
}

The Go blog's context article explains how each request runs in its own goroutine, often spawning more for database or RPC calls. When canceled or timed out, all these goroutines must exit promptly. The context.Context object carries a deadline and cancellation signals, and passing it as the first argument to I/O functions is standard practice.

A subtle bug occurs if the buffered channel in fetchWithTimeout isn't used carefully. If the select chooses the ctx.Done() branch while work() is still running, the goroutine might block forever on a send. Using a buffer of size 1 allows the goroutine to deposit its result and exit, avoiding leaks.

Pipelines, Fan-Out, and Fan-In

A pipeline chains stages via channels, each stage transforming data before passing it along. The Go blog's pipelines article describes this pattern: fan-out involves multiple goroutines reading from one channel to parallelize work; fan-in merges multiple channels into one.

package main

import (
 "fmt"
 "sync"
)

// gen produces a stream of integers.
func gen(nums ...int) <-chan int {
 out := make(chan int)
 go func() {
 for _, n := range nums {
 out <- n
 }
 close(out) // signals no more values
 }()
 return out
}

// square is a pipeline stage: one value in, one value out.
func square(in <-chan int) <-chan int {
 out := make(chan int)
 go func() {
 for n := range in {
 out <- n * n
 }
 close(out)
 }()
 return out
}

// merge is fan-in: multiple channels into one.
func merge(cs ...<-chan int) <-chan int {
 out := make(chan int)
 var wg sync.WaitGroup
 for _, c := range cs {
 wg.Add(1)
 go func(c <-chan int) {
 defer wg.Done()
 for n := range c {
 out <- n
 }
 }(c)
 }
 go func() {
 wg.Wait()
 close(out)
 }()
 return out
}

func main() {
 in := gen(2, 3, 4)
 c1 := square(in)
 c2 := square(in)
 for n := range merge(c1, c2) {
 fmt.Println(n) // 4, 9, 16 (order may vary)
 }
}

The core idea is that the sender closes the channel to signal completion. This allows range loops to terminate cleanly. Closing from the receiver side or sending on a closed channel causes runtime panics, which only surface during execution.

Early termination is trickier. If a downstream stage fails, upstream stages should not block forever. The solution pairs a done channel with select in each stage, propagating cancellation backward. This pattern underpins the context package, which automates this process.

Data Races and the Race Detector

Go's primitives make it easy to write concurrent code, but they don't prevent data races, situations where goroutines read and write shared variables without synchronization. The race detector, integrated since Go 1.1, instruments memory accesses at runtime to find unsynchronized shared variable access.

Enable it with go test -race or go run -race. It tracks the last goroutine to access each memory location and flags any access that violates the happens-before relationship. The trade-off is roughly a tenfold slowdown and increased memory use, so it's mainly a testing tool, not for production.

Note that the race detector only finds races on code paths your tests cover. An untested race might still exist in error branches or rare interleavings. The post about the detector emphasizes it's a tool for discovery, not proof of correctness. Running under realistic load and concurrency is recommended for thorough detection.

Performance considerations matter too. The Effective Go section on concurrency and the Uber style guide advise: share memory by communicating, not by sharing variables directly. Using channels or sync.Mutex guards is safer and clearer than relying on subtle ordering assumptions that the race detector might miss.

sync.WaitGroup, errgroup, and singleflight

Beyond raw goroutines, three packages from the standard library and extended ecosystem address common production concurrency patterns. sync.WaitGroup waits for a set of goroutines to finish; errgroup (from golang.org/x/sync) adds error propagation and context cancellation; and singleflight collapses duplicate requests into a single call, preventing cache stampedes.

The Encore blog's advanced concurrency post covers both errgroup and singleflight with concrete examples. errgroup is ideal for running multiple tasks that should fail fast on the first error, canceling remaining goroutines. singleflight handles scenarios where many requests for the same data arrive simultaneously, ensuring only one fetch occurs.

Primitive	Source	Error handling	Best fit
`sync.WaitGroup`	standard library	manual (collect errors yourself)	fire-and-forget fan-out where failures are logged, not propagated
`errgroup.Group`	golang.org/x/sync	first error cancels the context	parallel tasks that must fail fast as a unit

The choice between WaitGroup and errgroup boils down to error semantics. WaitGroup requires you to handle errors manually, while errgroup automates error collection and cancellation. Both do not enforce correctness; they coordinate goroutines, but shared data safety remains the programmer's responsibility. The race detector, proper use of context, and communication patterns are what ensure reliable, correct concurrent programs.

More in-depth coverage from this blog on closely related topics:

Sources and References

Sources cited while researching and writing this article: