This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new db2aceff2c8 branch-4.1: [fix](regression) Deflake the sql cache
regression suites #67734 (#67735)
db2aceff2c8 is described below
commit db2aceff2c8d30c939dde692170535e536ab56d9
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Thu Sep 10 10:14:24 2026 +0800
branch-4.1: [fix](regression) Deflake the sql cache regression suites
#67734 (#67735)
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #67734, #67529
Problem Summary:
Two flaky sql cache suites on branch-4.1. Both fail the same way — a sql
cache entry
that was primed a moment earlier is gone by the time it is asserted —
and both were red
together in p0 build 124525 (`5657 passed / 2 failed`; no crash, no OOM,
unrelated to
the PR that build was testing).
**Root cause.** The FE sql cache is a single Caffeine map shared by
every session
(`NereidsSqlCacheManager.sqlCaches`), bounded by
`Config.sql_cache_manage_num`, which
defaults to **100**. Caffeine admits a newcomer through a window sized
at **1% of that
bound**, so at the default the admission window holds a single entry and
a just cached
statement is dropped as soon as any other session caches anything.
`SessionVariable.enableSqlCache` defaults to `true`, so the rest of the
p0 suite running
concurrently against the same FE is enough. Ordinary LRU pressure is
ruled out: only 15
distinct selects ran cluster wide during the 299ms window in which the
flight suite lost
its entry. Verified against caffeine 3.2.4:
```
maximumSize=100 1 other insert after mine -> survived 7/20
maximumSize=10000 1 other insert after mine -> survived 20/20
```
This is not an FE bug — the cache is best effort and gives no retention
guarantee.
### Changes
**1. `arrow_flight_sql_p0/test_sql_cache_over_arrow_flight`** —
cherry-picked from
#67734. Raise `sql_cache_manage_num` while the suite runs, as the other
five sql cache
suites (`mv_with_sql_cache`, `mtmv_with_sql_cache`,
`parse_sql_from_sql_cache`,
`union_all_compensate`, `union_rewrite_grace_big`) already do, and
restore it afterwards
so the rest of the run does not keep 10000 cached plans alive in the FE
heap. This suite
is identical on master and branch-4.1 and has failed on both.
**2. `query_p0/cache/sql_cache_object_type`** — picks up the hunk that
the #67556
backport missed. Master fixed this in #67529 on 2026-09-04 by replacing
the closing
`assertTrue(hasSqlCache(objectSql))` with a re-prime, but #67556
backported the original
#67381 version of the file, so branch-4.1 kept the fragile assertion. It
has failed **18
times on branch-4.1 since 2026-09-02** and is currently **muted** in
TeamCity; the three
most recent failures were all branch-4.1 PRs. After this change the file
is byte
identical to master again.
**The TeamCity mute on
`query_p0.cache.sql_cache_object_type.sql_cache_object_type`
should be removed once this lands.**
### Release note
None
### Check List (For Author)
- Test
- [x] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01Njd8iDxdqc19QbLdNtZ7Pt
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
---
.../test_sql_cache_over_arrow_flight.groovy | 15 +++++++++++++++
.../suites/query_p0/cache/sql_cache_object_type.groovy | 7 ++++++-
2 files changed, 21 insertions(+), 1 deletion(-)
diff --git
a/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy
b/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy
index 577665b275b..a3fb39d38d9 100644
---
a/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy
+++
b/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy
@@ -84,6 +84,19 @@ suite("test_sql_cache_over_arrow_flight") {
withGlobalLock("cache_last_version_interval_second") {
runOnMysql "ADMIN SET ALL FRONTENDS CONFIG
('cache_last_version_interval_second' = '0')"
+ // The FE sql cache is a single Caffeine map shared by every session
and bounded by
+ // Config.sql_cache_manage_num, which defaults to 100. Caffeine admits
a newcomer through a
+ // window sized at 1% of that bound, so at the default a just cached
statement is dropped
+ // again as soon as any other session caches anything -- and the whole
p0 suite runs
+ // concurrently against this FE with enable_sql_cache on by default.
The entries primed
+ // below would then be gone before the checks at the end of this
suite, which is what made
+ // it flaky. Raise the bound while this suite runs, like the other sql
cache suites do, and
+ // put it back afterwards so the rest of the run does not keep 10000
cached plans and their
+ // result rows alive in the FE heap.
+ def originalSqlCacheNum =
+ runOnMysql("ADMIN SHOW FRONTEND CONFIG LIKE
'sql_cache_manage_num'")[0][1].toString()
+ runOnMysql "ADMIN SET ALL FRONTENDS CONFIG ('sql_cache_manage_num' =
'10000')"
+
def dbName = context.dbName
runOnMysql "USE `${dbName}`"
runOnFlight "USE `${dbName}`"
@@ -164,5 +177,7 @@ suite("test_sql_cache_over_arrow_flight") {
assertTrue(hasSqlCache(scalarSql))
assertTrue(hasSqlCache(rawStateSql))
assertTrue(hasSqlCache(convertedSql))
+
+ runOnMysql "ADMIN SET ALL FRONTENDS CONFIG ('sql_cache_manage_num' =
'${originalSqlCacheNum}')"
}
}
diff --git a/regression-test/suites/query_p0/cache/sql_cache_object_type.groovy
b/regression-test/suites/query_p0/cache/sql_cache_object_type.groovy
index 6300840d20f..4e2520e70c5 100644
--- a/regression-test/suites/query_p0/cache/sql_cache_object_type.groovy
+++ b/regression-test/suites/query_p0/cache/sql_cache_object_type.groovy
@@ -98,8 +98,13 @@ suite("sql_cache_object_type") {
assertTrue(isNonEmpty(asBinaryCached[0][0]))
assertTrue(isNonEmpty(asBinaryCached[0][1]))
+ // The sql cache is best-effort: the FE map holds soft values under a
bounded size
+ // (Config.sql_cache_manage_num) and the rows themselves live in the
BE result cache, so the
+ // entry created above may legitimately be gone by now. Re-prime it
instead of asserting it
+ // survived; what must hold is that this setting is served its own
NULLs and never the
+ // binary rows cached under the other one.
run "set return_object_data_as_binary=false"
- assertTrue(hasSqlCache(objectSql))
+ primeSqlCache(objectSql)
def asNullAgain = run(objectSql)
assertNull(asNullAgain[0][0])
assertNull(asNullAgain[0][1])
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
