Project: edge-cache · Branch: feat/swr · Mode: default · Model: claude-sonnet-4-6

Summary

Implemented stale-while-revalidate in the Go caching proxy: stale entries are served immediately while a single background refresh per key runs under singleflight, so a hot key never stampedes the origin.

Conversation

Turn 1 — User

Add stale-while-revalidate. Origin must not get a thundering herd.

Turn 1 — Assistant

Serve stale within the SWR window, trigger one refresh per key through singleflight.Group, and cap refresh concurrency with a semaphore.

func (c Cache) serve(w http.ResponseWriter, r http.Request, key string) {
	entry, ok := c.store.Get(key)
	if ok && entry.Fresh(time.Now()) {
		entry.WriteTo(w)
		return
	}
	if ok && entry.WithinSWR(time.Now()) {
		entry.WriteTo(w) // serve stale now
		go c.group.Do(key, func() (any, error) { return c.refresh(key, r) })
		return
	}
	c.fetchAndStore(w, r, key)
}

Tools used in this turn:

- Edit: cache/handler.go
- Write: cache/refresh.go
- Bash: go test ./... -race — ok

Connections

- edge-cache
- Go
- Singleflight
- StaleWhileRevalidate