yujun777 opened a new pull request, #69166:
URL: https://github.com/apache/doris/pull/69166
### What problem does this PR solve?
Issue Number: N/A
Problem Summary:
`OlapTable.getPartitionNames()` copies `nameToPartition.keySet()`
(`Sets.newHashSet(...)`) without holding any metadata lock, while
`nameToPartition` was a plain (fail-fast) `TreeMap` that is mutated in place by
partition add/drop/rename under the table write lock. Any lock-free reader
therefore races with the writer and can throw
`java.util.ConcurrentModificationException`.
Reproduced on P0 regression
`nereids_rules_p0/mv/create_part_and_up/range_datetime_part_up_rewrite`: the
suite runs `refresh MATERIALIZED VIEW <mv> auto` (whose refresh job adds
partitions to the MV base table) and immediately `explain materialized view
rewrite` on a query that is rewritten onto that MV. Rendering the memo plan
calls `PhysicalOlapScan.toString()` -> `OlapTable.getPartitionNames()`, and the
concurrent partition addition of the still-running refresh job makes the copy
throw:
```
2026-10-16 01:42:05,315 INFO (mtmv-task-execute-1-thread-2)
[EditLog.logAddPartition():1825] log add partition, ... "tableId":1792084617789
...
2026-10-16 01:42:05,315 INFO (mysql-nio-pool-112)
[StmtExecutor.executeByNereids():1003] Command(explain memo plan ...) process
failed.
java.util.ConcurrentModificationException
at java.util.TreeMap$KeyIterator.next(TreeMap.java:1540)
at com.google.common.collect.Sets.newHashSet(Sets.java:226)
at
org.apache.doris.catalog.OlapTable.getPartitionNames(OlapTable.java:1643)
at
org.apache.doris.nereids.trees.plans.physical.PhysicalOlapScan.toString(PhysicalOlapScan.java:326)
at org.apache.doris.nereids.memo.Memo.toString(Memo.java:922)
at
org.apache.doris.nereids.NereidsPlanner.getExplainString(NereidsPlanner.java:1092)
at
org.apache.doris.nereids.trees.plans.commands.ExplainCommand.run(ExplainCommand.java:138)
```
The rewrite itself succeeded; only the EXPLAIN output rendering crashed, so
this shows up as a flaky failure. But `getPartitionNames()` is reached from
~124 call sites (including MTMV partition sync checks that run without a table
lock), so this is a general latent race, not an EXPLAIN-only one.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [x] No.
- Does this need documentation?
- [x] No.
### How was this fixed
`OlapTable.nameToPartition` is now a `ConcurrentSkipListMap`:
- It keeps the same sorted-by-name semantics as the previous `TreeMap`, so
ordering-dependent readers are unaffected.
- Its iterators are weakly consistent instead of fail-fast, so a lock-free
reader either sees a consistent snapshot or a concurrently added/removed entry,
but never throws `ConcurrentModificationException`. Readers that need strict
consistency keep taking the table lock as before.
- The field has no `@SerializedName`, so `HiddenAnnotationExclusionStrategy`
excludes it from meta serialization; it is rebuilt from `idToPartition` in
`gsonPostProcess` on the constructor-initialized map. FE image and journal
formats are unchanged.
This follows the same direction as #36452, which already converted the
sibling `idToPartition` map to `ConcurrentHashMap`; `nameToPartition` was the
remaining plain map of the pair. It also supersedes the approach of #45260
(closed, not merged), which returned the live key set to one MTMV caller
instead of making the map itself safe to read — with a fail-fast `TreeMap` that
leaves every other reader exposed.
The added unit test
`OlapTableTest.testPartitionNameMapToleratesConcurrentAdd` inserts into the
partition name map while an iterator is active: it throws
`ConcurrentModificationException` on the old `TreeMap` (verified) and passes
with `ConcurrentSkipListMap`. `OlapTableTest` (18 tests) and FE checkstyle both
pass.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
