The One and the Many

How Go caches test results

I recently updated our CI system at work to use a GOCACHEPROG program to change where Go stores its cache. As part of this, I investigated why test results were not cached. I thought I'd write about how test result caching works for future reference.

What it is

The go tool provides a way to run tests via go test. If you have many tests, running them can be slow. As an optimization, go skips running tests if nothing changed.

For example, if you're working on a program with several packages, you can run go test ./... to run all the program's tests. However, your change might only affect one package, so re-running tests for unrelated packages each time you make a change is not necessary. With test result caching, go test will only re-run tests related to code you've changed.

How does it work?

At a high level, go test first checks if there are cached test results for the test binary. If there are, it checks whether anything the test depends on changed (such as files it reads). If nothing changed, it prints out the cached test results instead of executing the tests. If it needs to execute the tests, it stores test results in the cache.

To know exactly how this works, we can look at the code. All references are current as of Go 1.25.5.

tryCacheWithID() is responsible for finding cached test results. It has a comment that explains it well. Expanding on that:

  1. go takes the linker inputs hash or the test binary hash and looks in the cache.
    • The cache key is the sha256 of a string like "test binary...".
    • We call this the testID.
    • The value associated with the key in the cache is a log of which environment variables and files the test used when it ran.
  2. If it finds a log for the testID, then it knows this test binary ran before. However, it needs to check if anything changed since that run.
    • To do this, go parses the log it found of environment variables etc using computeTestInputsID() and records their current values.
    • It creates a new sha256 out of these values.
    • We call this the testInputsID.
  3. go creates a cache key that depends on both the binary (testID) and the re-calculated test inputs (testInputsID).
    • This cache key is the result of testAndInputKey().
    • Specifically, the cache key is the sha256 of subkey:<testID>inputs:<testInputsID>.
    • The value associated with the key in the cache is the test output.
  4. If there's a value in the cache for the subkey, then we can use the cache. go prints the cached output along with (cached) instead of executing the tests.

So the final cache key we check depends on both the test binary and the test inputs. To use test results from the cache, the test binary must be the same and the test inputs must be the same.

Debugging why test results aren't cached

Using our knowledge from above, we can figure out why a particular test isn't being cached even though we'd like it to be. Let's look at an example.

Create a main_test.go:

package main

import (
        "os"
        "testing"
)

func TestHelloWorld(t *testing.T) {
        value := os.Getenv("FOO")
        if value != "bar" {
                t.Fatal("FOO is not bar")
        }
}

Let's start with a clear cache:

$ go clean -cache -testcache -modcache

Run the test with GODEBUG=gocachetest=1. This will show output about cache actions:

$ FOO=bar GODEBUG=gocachetest=1 go test -v ./...
testcache: github.com/horgh/example-test-results: test ID 7a39326d484f6b76305769706c656a6c64716255 => abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4
testcache: github.com/horgh/example-test-results: input list not found: cache entry not found: open /home/will/.cache/go-build/ab/abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4-a: no such file or directory
testcache: github.com/horgh/example-test-results: test ID 324f615f46666c4d3451694d4230375256504741 => bfc42cd86a62a6c3265c1271a58479370e41f04b2a45a249b95820472467f3b8
testcache: github.com/horgh/example-test-results: input list not found: cache entry not found: open /home/will/.cache/go-build/bf/bfc42cd86a62a6c3265c1271a58479370e41f04b2a45a249b95820472467f3b8-a: no such file or directory
=== RUN   TestHelloWorld
--- PASS: TestHelloWorld (0.00s)
PASS
ok      github.com/horgh/example-test-results   0.002s
testcache: github.com/horgh/example-test-results: save test ID abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4 => input ID 5738ca7325a0f415f2ca98938e4b70ae37a792b6582cb5970d0b9cd0e898da8f => 34060fe8ae395418f79ac680fd490c1d6c0cf0c9a5c030b37da3da0c023b5d33
testcache: github.com/horgh/example-test-results: save test ID bfc42cd86a62a6c3265c1271a58479370e41f04b2a45a249b95820472467f3b8 => input ID 5738ca7325a0f415f2ca98938e4b70ae37a792b6582cb5970d0b9cd0e898da8f => 3da80fdb98f7d16f886f88f620f684019e526aa1187d29a80afe214ba39171ee

First we can see that we look for the linker input and then the test binary testIDs in the cache. That's what these lines are about:

testcache: github.com/horgh/example-test-results: test ID 7a39326d484f6b76305769706c656a6c64716255 => abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4

7a39... is the linker input hash and abea... is the calculated testID. Since we cleared the cache, neither of these is in the cache, so we run the test.

At the end, we can see that we save the results to the cache under both the linker input hash and the test binary hash with the two lines that look like this:

testcache: github.com/horgh/example-test-results: save test ID abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4 => input ID 5738ca7325a0f415f2ca98938e4b70ae37a792b6582cb5970d0b9cd0e898da8f => 34060fe8ae395418f79ac680fd490c1d6c0cf0c9a5c030b37da3da0c023b5d33

This is saying that we have testID abea... and testInputsID (the hash of our environment variable) 5738.... We store the test results under the key 3406... which is the combination of the testID and testInputsID.

Let's run it again:

$ FOO=bar GODEBUG=gocachetest=1 go test -v ./...
testcache: github.com/horgh/example-test-results: test ID 7a39326d484f6b76305769706c656a6c64716255 => abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4
testcache: github.com/horgh/example-test-results: test ID abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4 => input ID 5738ca7325a0f415f2ca98938e4b70ae37a792b6582cb5970d0b9cd0e898da8f => 34060fe8ae395418f79ac680fd490c1d6c0cf0c9a5c030b37da3da0c023b5d33
=== RUN   TestHelloWorld
--- PASS: TestHelloWorld (0.00s)
PASS
ok      github.com/horgh/example-test-results   (cached)

We can see that we got the test results from the cache.

This line shows that we find the log of test inputs:

testcache: github.com/horgh/example-test-results: test ID 7a39326d484f6b76305769706c656a6c64716255 => abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4

The testID again is abea... since we didn't change any code.

go re-calculates the test inputs. Specifically, it looks up the environment variable and gets the same value as before. Using the testID and the testInputsID, it finds the cache key 3406... which we saved results under before.

What happens if we change the environment variable's value?

$ FOO=baz GODEBUG=gocachetest=1 go test -v ./...
testcache: github.com/horgh/example-test-results: test ID 7a39326d484f6b76305769706c656a6c64716255 => abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4
testcache: github.com/horgh/example-test-results: test ID abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4 => input ID 15620d07b0719e6e08d64dbacfdfa531736a9914750cbf80e840819b025b57b1 => cb572f117e96eea262fde5cf338cf782a2a96a06404145c8f6a3cad451e42242
testcache: github.com/horgh/example-test-results: test output not found: cache entry not found: open /home/will/.cache/go-build/cb/cb572f117e96eea262fde5cf338cf782a2a96a06404145c8f6a3cad451e42242-a: no such file or directory
testcache: github.com/horgh/example-test-results: test ID 324f615f46666c4d3451694d4230375256504741 => bfc42cd86a62a6c3265c1271a58479370e41f04b2a45a249b95820472467f3b8
testcache: github.com/horgh/example-test-results: test ID bfc42cd86a62a6c3265c1271a58479370e41f04b2a45a249b95820472467f3b8 => input ID 15620d07b0719e6e08d64dbacfdfa531736a9914750cbf80e840819b025b57b1 => 117b122a95db13c194ae9594226410c442b4ddf8d078125ed87754afd1fe4f39
testcache: github.com/horgh/example-test-results: test output not found: cache entry not found: open /home/will/.cache/go-build/11/117b122a95db13c194ae9594226410c442b4ddf8d078125ed87754afd1fe4f39-a: no such file or directory
=== RUN   TestHelloWorld
    main_test.go:11: FOO is not bar
--- FAIL: TestHelloWorld (0.00s)
FAIL
FAIL    github.com/horgh/example-test-results   0.002s
FAIL

We can see that we have the same testID abea.... This finds the log of test inputs including the environment variable FOO. In this case, the environment variable value is different, so we calculate a different testInputsID 1562..., yielding the combined cache key of testID and testInputsID cb57.... This is different from before and isn't in our cache, so we run the test again.

We can enable additional debug output using gocachehash=1 to see what's going into the testInputsID calculation:

$ FOO=baz GODEBUG=gocachetest=1,gocachehash=1 go test -v ./...

This produces a lot of output, so I won't show all of it.

First we can see what goes into calculating the testInputsID hash (which is now 456c... since changing GODEBUG changes the test inputs):

HASH[testInputs]
HASH[testInputs]: "go1.25.5"
HASH[getenv]
HASH[getenv]: "go1.25.5"
HASH[getenv]: "\x01"
HASH[getenv]: "gocachetest=1,gocachehash=1"
HASH[getenv]: c0217c9fec3a3cad31c47a569baec70cc108c13f49b83e4ef41dae0e685e7440
HASH[testInputs]: "env GODEBUG c0217c9fec3a3cad31c47a569baec70cc108c13f49b83e4ef41dae0e685e7440\n"
HASH[getenv]
HASH[getenv]: "go1.25.5"
HASH[getenv]: "\x01"
HASH[getenv]: "baz"
HASH[getenv]: 7543df3f3b756567771d69f9b28e4e5f89c595f53337393a869aa00039244b09
HASH[testInputs]: "env FOO 7543df3f3b756567771d69f9b28e4e5f89c595f53337393a869aa00039244b09\n"
HASH[testInputs]: 456c469a46807a4c4a49a61c5dc0436d9f9aabd78225ef0bea903dbba3cd717d

Then we use that to make the combined testID and testInputsID cache key 76ca... and look it up:

HASH subkey abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4 "inputs:456c469a46807a4c4a49a61c5dc0436d9f9aabd78225ef0bea903dbba3cd717d" = 76caa4ba07e78f762677ddd18eb0998933ea739d8e564555dd628c03afbf9552
testcache: github.com/horgh/example-test-results: test ID abeacc6b535984f66e5ee7476e1078562411dae318e3f098d6d6956606cd7ee4 => input ID 456c469a46807a4c4a49a61c5dc0436d9f9aabd78225ef0bea903dbba3cd717d => 76caa4ba07e78f762677ddd18eb0998933ea739d8e564555dd628c03afbf9552
testcache: github.com/horgh/example-test-results: test output not found: cache entry not found: open /home/will/.cache/go-build/76/76caa4ba07e78f762677ddd18eb0998933ea739d8e564555dd628c03afbf9552-a: no such file or directory

Using our knowledge of how test results caching works along with use of GODEBUG=gocachetest=1,gocachehash=1, we can debug why any test is not being cached.