bosswnx opened a new pull request, #65659:
URL: https://github.com/apache/doris/pull/65659
### What problem does this PR solve?
Issue Number: nan
Related PR: #58877
Problem Summary:
Fix a TOCTOU (Time-of-Check Time-of-Use) race condition that causes
`NullPointerException` during partition pruning on external tables.
**Root cause:** In `PruneFileScanPartition.pruneExternalPartitions()`:
1. `nameToPartitionItem` — frozen at T1 inside
`LogicalFileScan.SelectedPartitions`
when the plan node is constructed (via `initSelectedPartitions()`)
2. `sortedPartitionRanges` — re-read from the `HivePartitionValues` cache
at T2
when the pruning rule executes (via
`externalTable.getSortedPartitionRanges()`)
If the cache is refreshed between T1 and T2 (e.g. concurrent `ALTER TABLE
ADD/DROP PARTITION`),
the two snapshots diverge. `binarySearchFiltering` uses the new snapshot
to decide
which partitions match the predicate, but the caller looks them up in the
old snapshot:
```java
for (String name : prunedPartitions) {
selectedPartitionItems.put(name, nameToPartitionItem.get(name));
// nameToPartitionItem.get(name) returns null for partitions that were
added after T1
}
// => ImmutableMap.copyOf() throws NPE: "null value in entry:
dt=2026-06-22=null"
Fix: Build sortedPartitionRanges from the same nameToPartitionItems map at
T1
and freeze it as a new field inside SelectedPartitions. At T2, the pruning
rule reads
sortedPartitionRanges from the frozen SelectedPartitions instead of
re-reading
the cache. This guarantees both data sources always come from one
consistent snapshot.
Additionally, a defensive null guard is added to the lookup loop as
defense-in-depth.
Release note
Fix NullPointerException in partition pruning when external table
partitions
are modified concurrently during query optimization (TOCTOU race in binary
search
partition filtering).
Check List (For Author)
- Test
- [x] Unit Test
- [x] Regression test (groovy stress test included)
- Behavior changed:
- [x] No.
- Does this need documentation?
- [x] No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
