[
https://issues.apache.org/jira/browse/SOLR-18506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SOLR-18506:
----------------------------------
Labels: pull-request-available (was: )
> TestThinCache.testSimple is flaky: eviction assertion races Caffeine removal
> notification delivery
> --------------------------------------------------------------------------------------------------
>
> Key: SOLR-18506
> URL: https://issues.apache.org/jira/browse/SOLR-18506
> Project: Solr
> Issue Type: Bug
> Reporter: Nick Shanin
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> AI-generated, human-approved text below.
> org.apache.solr.search.TestThinCache.testSimple fails intermittently with
> "AssertionError: expected:<0> but was:<1>" at TestThinCache.java:170,
> together with a classMethod teardown error reporting 3 unreleased
> SolrMetricsContext objects. The test is the only coverage of ThinCache, the
> per-searcher scope over a shared node-level cache introduced in SOLR-16654.
> The failure has been recorded 14 times in the last 30 days across unrelated
> pull requests (list below), including 10 times on branch_9x. It is a test
> defect, not a product defect: no production code is involved in the race.
> Failure signature
> - TestThinCache.testSimple: expected:<0> but was:<1> at
> TestThinCache.java:170 (the second cache's evictions metric). The value is
> exactly 1 on every recorded occurrence.
> - classMethod teardown: 3 unreleased SolrMetricsContext objects (knock-on,
> see root cause).
> - Recorded occurrences (GitHub "Solr Tests via Crave" runs, 2026-09-09 to
> 2026-10-04): PR #4894 (seed C4077EAB80AE9E50), PR #4939 (60B12BB09380AC51),
> PR #4943 (E58B5E1FFDEBA340), PR #5013 (30A1B1A830109349), and PR #4976 on
> branch_9x ten times (E4DD10799981D14B, 2E53727468DE4DF1, 391A297131D7D9C1,
> D8FD97BF9A8A48B0, FCA68807697335BA, 861F48CD4DFF5F51, D5B22838079384ED,
> 6E0A7AD71C551639, 7A1C354A4C7E10DC, FA416BA281EBD0DF).
> Reproduction
> Locally, the failure does not reproduce: all 14 recorded seeds were run
> against current main with forced re-execution (cleanTest), plus default-seed
> runs, and every run passed. The #5013 seed also passed earlier on pre-merge
> main, current main, and the PR head. The race needs the timing of a loaded CI
> runner; the mechanism below is established from the code and from the
> invariant failure value.
> Root cause
> testSimple puts 101 entries through a first ThinCache into a shared backing
> cache of capacity 100, creating exactly one eviction (key 1; the test asserts
> its physical absence). It then warms a second ThinCache from the first.
> ThinCache.warm copies the first cache's counters into the second cache's
> priors, including evictions. The second cache's eviction metric is cumulative
> and prior-inclusive, like its hits, inserts, and lookups in the same
> assertion block (4 = 2 + 2, 102 = 101 + 1, 7 = 4 + 3), so the inherited value
> is 1. The assertion expects 0, which holds only if Caffeine has not yet
> delivered the first eviction's removal notification when warm copies the
> counter. Physical eviction and notification delivery are separate steps in
> Caffeine (the backing cache is an async Caffeine cache whose maintenance runs
> lazily), and the test has no synchronization point between them, so the
> sample races the delivery. The teardown leak is a knock-on: the failed
> assertion skips the test's closing of its SolrMetricsContext, and the
> framework reports the context and its two per-cache children as unreleased.
> Proposed fix
> Test-only, in testSimple: enlarge the shared backing cache (public
> CaffeineCache.setMaxSize, which also forces Caffeine maintenance via cleanUp)
> after the first cache's assertions and before warming, so the pending
> eviction is settled and counted before the priors copy and no further
> eviction can occur in the second phase; assert the first cache's eviction
> counter is exactly 1 at that point; and correct the second cache's expected
> evictions from 0 to the inherited 1, with a comment explaining the cumulative
> accounting. A pull request with this fix and a seed battery proof will follow.
> ### AI assistance
> AI agents assisted with research, implementation, review, and drafting. Nick
> Shanin directed the work and takes responsibility for this ticket description.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
