Robolectric and the case of the missing Gradle task caching

Haze’s CI uses a self-hosted Gradle Build Cache Node behind Caddy and Cloudflare. Pushes to main write to it, and pull requests only read from it. In theory, a PR that touches nothing but a workflow file should be almost free: every test task has already run on main with the same inputs, so its result should come straight from the cache.

For the Desktop screenshot tests, that’s exactly what happened. PR #1781 only changed workflow files, and the Desktop screenshot job took about two minutes:

> Task :haze-screenshot-tests:jvmTest FROM-CACHE
> Task :sample:screenshot-tests:jvmTest FROM-CACHE
BUILD SUCCESSFUL in 1m 27s

The Android screenshot job in the same run took over ten minutes, because testAndroidHostTest ran in full:

> Task :haze-screenshot-tests:compileAndroidHostTest FROM-CACHE
> Task :haze-screenshot-tests:testAndroidHostTest
BUILD SUCCESSFUL in 10m 1s

It didn’t just happen on that PR. The Android tests ran in full every time.

I handed the investigation to a coding agent, and much of what follows is what it dug up. I made the infrastructure changes.

The clue

#

The Android job wasn’t missing the cache entirely. Every compile task in the module came from the cache, including compileAndroidHostTest. Only the test task itself ran.

The PR logs couldn’t explain that, because PRs don’t write to the cache. The agent found the answer in the main push builds, which do. Here’s the Android job from the push that merged that PR:

> Task :haze-screenshot-tests:testAndroidHostTest
Could not store entry a3bc28e0… in remote build cache: Storing entry at '…' response status 413: Payload Too Large
...
BUILD SUCCESSFUL in 8m 50s
The remote build cache was disabled during the build due to errors.

That last line matters. In these Gradle 9.9.0 and 9.9.1 builds, the rejected upload disabled the remote cache for the rest of the build.¹ So testAndroidHostTest ran and passed, but its result was never stored remotely. The next build had nothing to load, so the test ran again.

Is the test task the problem?

#

The obvious suspect was that testAndroidHostTest wasn’t cacheable, or that its cache key changed on every build.

The agent checked that locally with -Dorg.gradle.caching.debug=true. The task was cacheable, and its key was stable. After deleting the module’s build/ directory and running it again, it came back FROM-CACHE. That ruled out an inherently uncacheable test, but didn’t identify the rejected entry.

A 150 MB cache entry

#

The suspect was JavaResCompressionTransform in the Android Gradle Plugin (AGP), which recompresses the Java-resource JARs on the test runtime classpath. The published AGP 9.5.1 sources mark it as cacheable and enable it by default, so its outputs go into the build cache too.

Most of the JARs on that classpath are small. One isn’t. Robolectric pulls in org.robolectric:nativeruntime-dist-compat:1.0.19, which contains Robolectric’s native runtime. The JAR is 159 MB on Maven Central, and the transformed output came to 152.7 MB as a cache entry locally.

Disabling the transform removed all cache entries over 10 MB while leaving the tests passing. That and the oversized local output pointed to the transform as the rejected upload; the CI warning alone doesn’t name it.

The Desktop jvmTest task was never affected because it doesn’t use Robolectric, so it doesn’t need the oversized native runtime JAR.

The false lead

#

A 413 means the server refused the request body because it was too large, and the cache node has its own limit for that, maxArtifactSize. Its default is 100 MiB, and the docs warn that raising it “may require further configuration changes in network components”. I raised it to 200 MiB.

It didn’t help. The next push to main hit the same 413.

So the limit was somewhere else. Caddy, the reverse proxy in front of the cache node, didn’t have a request_body limit configured. That left Cloudflare. When a hostname is proxied through Cloudflare, Cloudflare caps request bodies based on the plan: 100 MB on Free and Pro, 200 MB on Business, and up to 5 GB on Enterprise. Anything bigger gets a 413.

That’s the limit the 150 MB upload was hitting, and no setting on the cache node was going to change it.

The fix

#

I switched the cache hostname to DNS-only in Cloudflare, which turns off the proxy. CI uploads now go straight to Caddy and the cache node, and the cache node’s own limit is the one that applies. The trade-off is that the hostname now resolves to the server’s own IP address, and Cloudflare’s proxy no longer sits in front of it.

There’s another option if you’d rather keep the proxy: set this in gradle.properties:

android.experimental.enableJavaResourceOptimizations=false

This stops AGP registering the transform. I didn’t go with it: the flag is experimental, applies to the whole build, and I haven’t established what disabling the optimisation costs. AGP’s source also marks it for removal in AGP 10, when the optimisation will always be enabled.

Did it work?

#

The first main push after the change ran testAndroidHostTest in full, but this time there was no store error and no “disabled” message. It took 11m 21s.

Then I re-ran that same job, on the same commit:

> Task :haze-screenshot-tests:compileAndroidHostTest FROM-CACHE
> Task :haze-screenshot-tests:testAndroidHostTest FROM-CACHE
BUILD SUCCESSFUL in 49s

The rerun could only load that result because the first attempt had stored it, so the upload got through. The whole job took 1m 33s, down from over ten minutes.

Haze sets roborazzi.test.verify=true, which makes Roborazzi’s Gradle plugin register the baseline images as test inputs. Changing a baseline changes the cache key, so a baseline-only PR can’t reuse a pass for different images.

If your builds keep missing the remote cache, search the logs for remote build cache was disabled, including after BUILD SUCCESSFUL. For a rejected upload, check the limit at every hop: CDN, reverse proxy and cache node. The smallest one wins.