Claus Ibsen created CAMEL-24459:
-----------------------------------
Summary: Develocity local build cache hides failing/stale Failsafe
integration test results
Key: CAMEL-24459
URL: https://issues.apache.org/jira/browse/CAMEL-24459
Project: Camel
Issue Type: Task
Components: build system
Reporter: Claus Ibsen
h2. Problem
{{.mvn/develocity.xml}} enables the Develocity Maven extension's *local* build
cache whenever the {{GITHUB_ACTIONS}} env var is unset:
{code:xml}
<buildCache>
<local><enabled>#{isFalse(env['GITHUB_ACTIONS'])}</enabled></local>
<remote><enabled>false</enabled></remote>
</buildCache>
{code}
This means the local cache is always ON on developer machines and always OFF in
CI. In practice this causes {{mvn verify}} to silently replay a cached
"success" for the Failsafe {{integration-test}}/{{verify}} goals instead of
actually re-running integration tests against a live service (e.g.
Testcontainers), whenever Maven's declared inputs (sources/poms/deps) look
unchanged since a previous local build -- even though nothing about the
*runtime environment* (Docker state, external service state) is tracked as a
build input.
Console output makes this easy to miss: the goal header still prints normally,
but is immediately followed by {{Loaded from the build cache, saving Xs}} with
no {{Running org....IT}} lines and no {{Tests run:}} summary -- the whole build
finishes in a couple of seconds instead of the real runtime.
h2. Reproduction
Concretely reproduced in {{components/camel-nats}}: after an initial successful
{{mvn verify}} run, a subsequent plain {{mvn verify}} reported BUILD SUCCESS in
~2s without re-running any of the 28 IT tests. Forcing a real run with {{mvn
verify -Ddevelocity.cache.local.enabled=false}} revealed 2 genuinely failing
tests ({{NatsJetstreamConsumerRedeliveryIT}},
{{NatsJetstreamConsumerMaxDeliverIT}}) that the cached result had been masking.
(Those test failures are being tracked/fixed separately from this ticket.)
h2. Impact
Developers can get false-positive local {{mvn verify}} results for integration
tests indefinitely, which undermines confidence in local IT runs and could let
real regressions slip through pre-PR checks.
h2. Suggested fix / discussion points
* Consider excluding the Failsafe {{integration-test}}/{{verify}} goal
executions from Develocity local build cache eligibility (these goals talk to
external/Testcontainers state that isn't a declared Maven input, so caching
them is unsound), or
* Document the {{-Ddevelocity.cache.local.enabled=false}} workaround
prominently (e.g. in CONTRIBUTING docs) if disabling caching for these goals
isn't feasible.
h2. Workaround
{{mvn verify -Ddevelocity.cache.local.enabled=false}} forces a real run. Note
{{mvn clean verify}} does *not* help -- the cache restores prior outputs
regardless of {{clean}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
