This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch branch-4.2 in repository https://gitbox.apache.org/repos/asf/doris.git
commit a701e06b1084275d01e7c2a2e1c331c43be1433d Author: Mingyu Chen (Rayner) <[email protected]> AuthorDate: Thu Sep 10 10:14:24 2026 +0800 branch-4.1: [fix](regression) Deflake the sql cache regression suites #67734 (#67735) ### What problem does this PR solve? Issue Number: close #xxx Related PR: #67734, #67529 Problem Summary: Two flaky sql cache suites on branch-4.1. Both fail the same way — a sql cache entry that was primed a moment earlier is gone by the time it is asserted — and both were red together in p0 build 124525 (`5657 passed / 2 failed`; no crash, no OOM, unrelated to the PR that build was testing). **Root cause.** The FE sql cache is a single Caffeine map shared by every session (`NereidsSqlCacheManager.sqlCaches`), bounded by `Config.sql_cache_manage_num`, which defaults to **100**. Caffeine admits a newcomer through a window sized at **1% of that bound**, so at the default the admission window holds a single entry and a just cached statement is dropped as soon as any other session caches anything. `SessionVariable.enableSqlCache` defaults to `true`, so the rest of the p0 suite running concurrently against the same FE is enough. Ordinary LRU pressure is ruled out: only 15 distinct selects ran cluster wide during the 299ms window in which the flight suite lost its entry. Verified against caffeine 3.2.4: ``` maximumSize=100 1 other insert after mine -> survived 7/20 maximumSize=10000 1 other insert after mine -> survived 20/20 ``` This is not an FE bug — the cache is best effort and gives no retention guarantee. ### Changes **1. `arrow_flight_sql_p0/test_sql_cache_over_arrow_flight`** — cherry-picked from #67734. Raise `sql_cache_manage_num` while the suite runs, as the other five sql cache suites (`mv_with_sql_cache`, `mtmv_with_sql_cache`, `parse_sql_from_sql_cache`, `union_all_compensate`, `union_rewrite_grace_big`) already do, and restore it afterwards so the rest of the run does not keep 10000 cached plans alive in the FE heap. This suite is identical on master and branch-4.1 and has failed on both. **2. `query_p0/cache/sql_cache_object_type`** — picks up the hunk that the #67556 backport missed. Master fixed this in #67529 on 2026-09-04 by replacing the closing `assertTrue(hasSqlCache(objectSql))` with a re-prime, but #67556 backported the original #67381 version of the file, so branch-4.1 kept the fragile assertion. It has failed **18 times on branch-4.1 since 2026-09-02** and is currently **muted** in TeamCity; the three most recent failures were all branch-4.1 PRs. After this change the file is byte identical to master again. **The TeamCity mute on `query_p0.cache.sql_cache_object_type.sql_cache_object_type` should be removed once this lands.** ### Release note None ### Check List (For Author) - Test - [x] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [x] No. - [ ] Yes. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01Njd8iDxdqc19QbLdNtZ7Pt Co-authored-by: Claude Opus 5 (1M context) <[email protected]> --- .../test_sql_cache_over_arrow_flight.groovy | 15 +++++++++++++++ .../suites/query_p0/cache/sql_cache_object_type.groovy | 7 ++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy b/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy index 577665b275b..a3fb39d38d9 100644 --- a/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy +++ b/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy @@ -84,6 +84,19 @@ suite("test_sql_cache_over_arrow_flight") { withGlobalLock("cache_last_version_interval_second") { runOnMysql "ADMIN SET ALL FRONTENDS CONFIG ('cache_last_version_interval_second' = '0')" + // The FE sql cache is a single Caffeine map shared by every session and bounded by + // Config.sql_cache_manage_num, which defaults to 100. Caffeine admits a newcomer through a + // window sized at 1% of that bound, so at the default a just cached statement is dropped + // again as soon as any other session caches anything -- and the whole p0 suite runs + // concurrently against this FE with enable_sql_cache on by default. The entries primed + // below would then be gone before the checks at the end of this suite, which is what made + // it flaky. Raise the bound while this suite runs, like the other sql cache suites do, and + // put it back afterwards so the rest of the run does not keep 10000 cached plans and their + // result rows alive in the FE heap. + def originalSqlCacheNum = + runOnMysql("ADMIN SHOW FRONTEND CONFIG LIKE 'sql_cache_manage_num'")[0][1].toString() + runOnMysql "ADMIN SET ALL FRONTENDS CONFIG ('sql_cache_manage_num' = '10000')" + def dbName = context.dbName runOnMysql "USE `${dbName}`" runOnFlight "USE `${dbName}`" @@ -164,5 +177,7 @@ suite("test_sql_cache_over_arrow_flight") { assertTrue(hasSqlCache(scalarSql)) assertTrue(hasSqlCache(rawStateSql)) assertTrue(hasSqlCache(convertedSql)) + + runOnMysql "ADMIN SET ALL FRONTENDS CONFIG ('sql_cache_manage_num' = '${originalSqlCacheNum}')" } } diff --git a/regression-test/suites/query_p0/cache/sql_cache_object_type.groovy b/regression-test/suites/query_p0/cache/sql_cache_object_type.groovy index 6300840d20f..4e2520e70c5 100644 --- a/regression-test/suites/query_p0/cache/sql_cache_object_type.groovy +++ b/regression-test/suites/query_p0/cache/sql_cache_object_type.groovy @@ -98,8 +98,13 @@ suite("sql_cache_object_type") { assertTrue(isNonEmpty(asBinaryCached[0][0])) assertTrue(isNonEmpty(asBinaryCached[0][1])) + // The sql cache is best-effort: the FE map holds soft values under a bounded size + // (Config.sql_cache_manage_num) and the rows themselves live in the BE result cache, so the + // entry created above may legitimately be gone by now. Re-prime it instead of asserting it + // survived; what must hold is that this setting is served its own NULLs and never the + // binary rows cached under the other one. run "set return_object_data_as_binary=false" - assertTrue(hasSqlCache(objectSql)) + primeSqlCache(objectSql) def asNullAgain = run(objectSql) assertNull(asNullAgain[0][0]) assertNull(asNullAgain[0][1]) --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
