This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 8966bdfc699 [fix](regression) Deflake test_sql_cache_over_arrow_flight
(#67734)
8966bdfc699 is described below
commit 8966bdfc699cd67ffe9b5eabda861edaeaabc1ee
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Thu Sep 10 09:59:22 2026 +0800
[fix](regression) Deflake test_sql_cache_over_arrow_flight (#67734)
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #67381
Problem Summary:
`arrow_flight_sql_p0/test_sql_cache_over_arrow_flight` is flaky. It
primes four sql
cache entries through the MySQL control session, runs the same
statements over Arrow
Flight, and then asserts the entries are still there, to prove that a
flight query
never consumes the cache. That closing block fails intermittently on
both master and
branch-4.1 — six times since the suite was added on 2026-09-02, most
recently in p0
build 124525, where the `select 1 as c, 'x' as s` entry was gone
**299ms** after it had
been primed:
```
Exception in
arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy(line 163):
assertTrue(hasSqlCache(constantSql))
org.opentest4j.AssertionFailedError: expected: <true> but was: <false>
```
**Root cause.** The FE sql cache is a single Caffeine map shared by
every session
(`NereidsSqlCacheManager.sqlCaches`), bounded by
`Config.sql_cache_manage_num`, which
defaults to **100**. Caffeine admits a newcomer through a window sized
at **1% of that
bound**, so at the default the admission window holds a single entry: a
just cached
statement has frequency ~1 and loses the admission contest to an
established victim as
soon as any other session caches anything.
`SessionVariable.enableSqlCache` defaults to
`true`, so the rest of the p0 suite running concurrently against the
same FE is already
enough to evict it.
The audit log rules out ordinary LRU pressure: only **15 distinct
selects** ran cluster
wide during that 299ms window, far fewer than the 100 an LRU would have
needed. A local
run against caffeine 3.2.4 reproduces the admission behaviour directly:
```
maximumSize=100 1 other insert after mine -> survived 7/20
maximumSize=100 15 other inserts after mine -> survived 8/20
maximumSize=10000 1 other insert after mine -> survived 20/20
maximumSize=10000 15 other inserts after mine -> survived 20/20
```
This is not an FE bug — the sql cache is best effort and gives no
retention guarantee.
It is the suite asserting a property the cache does not provide. The
other five sql
cache suites in the repo (`mv_with_sql_cache`, `mtmv_with_sql_cache`,
`parse_sql_from_sql_cache`, `union_all_compensate`,
`union_rewrite_grace_big`) already
raise the bound at their start for exactly this reason; this one was
missing it. In p0
build 124525 the failing suites ran at 21:54 and 22:01, before any of
those suites
raised the bound at 22:12.
**Fix.** Raise `sql_cache_manage_num` to 10000 while the suite runs, and
restore the
previous value afterwards. The restore is deliberate: this suite runs
about half an
hour earlier in the p0 run than the five existing ones, and the ones
that raise the
bound without restoring it left **97k live `SqlCacheContext` instances**
in the FE heap
in the same build (post-GC live heap peaked at 4G of 8G in that window).
### Release note
None
### Check List (For Author)
- Test
- [x] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01Njd8iDxdqc19QbLdNtZ7Pt
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
---
.../test_sql_cache_over_arrow_flight.groovy | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git
a/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy
b/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy
index 577665b275b..a3fb39d38d9 100644
---
a/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy
+++
b/regression-test/suites/arrow_flight_sql_p0/test_sql_cache_over_arrow_flight.groovy
@@ -84,6 +84,19 @@ suite("test_sql_cache_over_arrow_flight") {
withGlobalLock("cache_last_version_interval_second") {
runOnMysql "ADMIN SET ALL FRONTENDS CONFIG
('cache_last_version_interval_second' = '0')"
+ // The FE sql cache is a single Caffeine map shared by every session
and bounded by
+ // Config.sql_cache_manage_num, which defaults to 100. Caffeine admits
a newcomer through a
+ // window sized at 1% of that bound, so at the default a just cached
statement is dropped
+ // again as soon as any other session caches anything -- and the whole
p0 suite runs
+ // concurrently against this FE with enable_sql_cache on by default.
The entries primed
+ // below would then be gone before the checks at the end of this
suite, which is what made
+ // it flaky. Raise the bound while this suite runs, like the other sql
cache suites do, and
+ // put it back afterwards so the rest of the run does not keep 10000
cached plans and their
+ // result rows alive in the FE heap.
+ def originalSqlCacheNum =
+ runOnMysql("ADMIN SHOW FRONTEND CONFIG LIKE
'sql_cache_manage_num'")[0][1].toString()
+ runOnMysql "ADMIN SET ALL FRONTENDS CONFIG ('sql_cache_manage_num' =
'10000')"
+
def dbName = context.dbName
runOnMysql "USE `${dbName}`"
runOnFlight "USE `${dbName}`"
@@ -164,5 +177,7 @@ suite("test_sql_cache_over_arrow_flight") {
assertTrue(hasSqlCache(scalarSql))
assertTrue(hasSqlCache(rawStateSql))
assertTrue(hasSqlCache(convertedSql))
+
+ runOnMysql "ADMIN SET ALL FRONTENDS CONFIG ('sql_cache_manage_num' =
'${originalSqlCacheNum}')"
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
