This is an automated email from the ASF dual-hosted git repository.
morrySnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 8460676f3fc [Fix](nereids) Freeze sortedPartitionRanges in
SelectedPartitions to prevent TOCTOU NPE during partition pruning (#65659)
8460676f3fc is described below
commit 8460676f3fcb6304f27dba32379da7d0fe0ad9d9
Author: Nelson Boss <[email protected]>
AuthorDate: Mon Jul 20 18:58:55 2026 +0800
[Fix](nereids) Freeze sortedPartitionRanges in SelectedPartitions to
prevent TOCTOU NPE during partition pruning (#65659)
### What problem does this PR solve?
Issue Number: #64800
Related PR: #58877
Problem Summary:
Fix a TOCTOU (Time-of-Check Time-of-Use) race condition that causes
`NullPointerException` during partition pruning on external tables.
**Root cause:** In `PruneFileScanPartition.pruneExternalPartitions()`:
1. `nameToPartitionItem` — frozen at T1 inside
`LogicalFileScan.SelectedPartitions` when the plan node is constructed
(via `initSelectedPartitions()`)
2. `sortedPartitionRanges` — re-read from the `HivePartitionValues`
cache at T2 when the pruning rule executes (via
`externalTable.getSortedPartitionRanges()`)
If the cache is refreshed between T1 and T2 (e.g. concurrent `ALTER
TABLE ADD/DROP PARTITION`), the two snapshots diverge.
`binarySearchFiltering` uses the new snapshot to decide which partitions
match the predicate, but the caller looks them up in the old snapshot:
```java
for (String name : prunedPartitions) {
selectedPartitionItems.put(name, nameToPartitionItem.get(name));
// nameToPartitionItem.get(name) returns null for partitions that were
added after T1
}
// => ImmutableMap.copyOf() throws NPE: "null value in entry:
dt=2026-06-22=null"
```
**Concrete example:** A Hive table has 3 partitions
`dt=2026-06-20/21/23`. Session A runs `SELECT * FROM t WHERE
dt='2026-06-22'`:
```
T1 BindRelation: LogicalFileScan freezes nameToPartitionItem from cache
→ {2026-06-20, 2026-06-21, 2026-06-23} (no 2026-06-22)
[Session B runs ALTER TABLE ADD PARTITION (dt='2026-06-22')]
[cache is refreshed → now has 4 partitions including 2026-06-22]
T2 PruneFileScanPartition: re-reads sortedPartitionRanges from cache
→ {2026-06-20, 2026-06-21, 2026-06-22, 2026-06-23} (new snapshot)
binarySearchFiltering matches dt=2026-06-22 → returns "dt=2026-06-22"
nameToPartitionItem.get("dt=2026-06-22") → null (old snapshot has no
such key)
→ NPE: "null value in entry: dt=2026-06-22=null"
```
**Fix:** freeze both views from a single snapshot so T2 never re-reads
the cache.
- `SelectedPartitions` now carries an `Optional<SortedPartitionRanges>`
field.
- `HMSExternalTable.initSelectedPartitions` reads the cached
`HivePartitionValues` once and freezes both the partition map and the
cached sorted ranges together (reuses the cache, no just-in-time
rebuild).
- Hudi has no cached ranges, so `PruneFileScanPartition` builds them
lazily from the frozen map only when binary search filtering is enabled.
- A missing partition in the lookup loop is now an invariant failure
(`Preconditions.checkState`) instead of being silently skipped, which
previously produced a partial scan over fewer partitions.
### Release note
Fix `NullPointerException` in partition pruning when external table
partitions are modified concurrently during query optimization (TOCTOU
race in binary search partition filtering).
---
.../doris/datasource/hive/HMSExternalTable.java | 24 ++
.../rules/rewrite/PruneFileScanPartition.java | 14 +-
.../trees/plans/logical/LogicalFileScan.java | 29 ++-
.../BinarySearchPartitionInconsistencyTest.java | 255 +++++++++++++++++++++
4 files changed, 315 insertions(+), 7 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalTable.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalTable.java
index e1311237a60..9e2d1ad22e2 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalTable.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalTable.java
@@ -464,6 +464,28 @@ public class HMSExternalTable extends ExternalTable
implements MTMVRelatedTableI
return hivePartitionValues.getSortedPartitionRanges();
}
+ @Override
+ public SelectedPartitions initSelectedPartitions(Optional<MvccSnapshot>
snapshot) {
+ // For Hive, read the cached HivePartitionValues once and freeze both
the partition
+ // map and the cached sortedPartitionRanges from the same snapshot.
This reuses the
+ // cached sorted ranges (no just-in-time rebuild) and keeps the two
views consistent,
+ // avoiding the TOCTOU divergence where sortedPartitionRanges was
re-read from cache
+ // at pruning time while the partition map was frozen earlier.
+ if (getDlaType() != DLAType.HIVE) {
+ return super.initSelectedPartitions(snapshot);
+ }
+ if (CollectionUtils.isEmpty(this.getPartitionColumns())) {
+ return SelectedPartitions.NOT_PRUNED;
+ }
+ HiveExternalMetaCache.HivePartitionValues hivePartitionValues =
getHivePartitionValues(
+ MvccUtil.getSnapshotFromContext(this));
+ Map<String, PartitionItem> nameToPartitionItems =
hivePartitionValues.getNameToPartitionItem();
+ Optional<SortedPartitionRanges<String>> sortedPartitionRanges
+ = hivePartitionValues.getSortedPartitionRanges();
+ return new SelectedPartitions(nameToPartitionItems.size(),
nameToPartitionItems, false, false,
+ sortedPartitionRanges);
+ }
+
public SelectedPartitions
initHudiSelectedPartitions(Optional<TableSnapshot> tableSnapshot) {
if (getDlaType() != DLAType.HUDI) {
return SelectedPartitions.NOT_PRUNED;
@@ -482,6 +504,8 @@ public class HMSExternalTable extends ExternalTable
implements MTMVRelatedTableI
nameToPartitionItems.put(idToNameMap.get(entry.getKey()),
entry.getValue());
}
+ // Hudi has no cached sorted ranges; leave it empty here and let
PruneFileScanPartition
+ // build it lazily from this frozen map only when binary search
filtering is enabled.
return new SelectedPartitions(nameToPartitionItems.size(),
nameToPartitionItems, false);
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PruneFileScanPartition.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PruneFileScanPartition.java
index f3822215e8c..83db60b098a 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PruneFileScanPartition.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PruneFileScanPartition.java
@@ -31,6 +31,7 @@ import
org.apache.doris.nereids.trees.plans.logical.LogicalFileScan;
import
org.apache.doris.nereids.trees.plans.logical.LogicalFileScan.SelectedPartitions;
import org.apache.doris.nereids.trees.plans.logical.LogicalFilter;
+import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Maps;
import org.apache.commons.collections4.CollectionUtils;
@@ -94,8 +95,9 @@ public class PruneFileScanPartition extends
OneRewriteRuleFactory {
Optional<SortedPartitionRanges<String>> sortedPartitionRanges =
Optional.empty();
boolean enableBinarySearch = ctx.getConnectContext() == null
||
ctx.getConnectContext().getSessionVariable().enableBinarySearchFilteringPartitions;
- if (enableBinarySearch) {
- sortedPartitionRanges = (Optional)
externalTable.getSortedPartitionRanges(scan);
+ if (enableBinarySearch && !nameToPartitionItem.isEmpty()) {
+ sortedPartitionRanges =
scan.getSelectedPartitions().sortedPartitionRanges
+ .or(() ->
Optional.ofNullable(SortedPartitionRanges.build(nameToPartitionItem)));
}
PartitionPruneResult<String> result = PartitionPruner.pruneWithResult(
partitionSlots, filter.getPredicate(), nameToPartitionItem,
ctx,
@@ -103,7 +105,13 @@ public class PruneFileScanPartition extends
OneRewriteRuleFactory {
List<String> prunedPartitions = new ArrayList<>(result.partitions);
for (String name : prunedPartitions) {
- selectedPartitionItems.put(name, nameToPartitionItem.get(name));
+ PartitionItem item = nameToPartitionItem.get(name);
+ // Both nameToPartitionItem and sortedPartitionRanges now come
from the same frozen
+ // snapshot, so a missing item is an invariant violation rather
than a partition to
+ // skip. Failing here surfaces the bug instead of silently
returning a partial scan.
+ Preconditions.checkState(item != null,
+ "pruned partition %s is missing in the selected partitions
snapshot", name);
+ selectedPartitionItems.put(name, item);
}
return new SelectedPartitions(nameToPartitionItem.size(),
selectedPartitionItems, true,
result.hasPartitionPredicate);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalFileScan.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalFileScan.java
index e70933b5b64..29e2f84feac 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalFileScan.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalFileScan.java
@@ -28,6 +28,7 @@ import
org.apache.doris.datasource.iceberg.IcebergSysExternalTable;
import org.apache.doris.datasource.mvcc.MvccUtil;
import org.apache.doris.nereids.memo.GroupExpression;
import org.apache.doris.nereids.properties.LogicalProperties;
+import org.apache.doris.nereids.rules.expression.rules.SortedPartitionRanges;
import org.apache.doris.nereids.trees.TableSample;
import org.apache.doris.nereids.trees.expressions.ExprId;
import org.apache.doris.nereids.trees.expressions.NamedExpression;
@@ -273,7 +274,8 @@ public class LogicalFileScan extends LogicalCatalogRelation
implements SupportPr
// NOT_PRUNED means the Nereids planner does not handle the partition
pruning.
// This can be treated as the initial value of SelectedPartitions.
// Or used to indicate that the partition pruning is not processed.
- public static SelectedPartitions NOT_PRUNED = new
SelectedPartitions(0, ImmutableMap.of(), false, false);
+ public static SelectedPartitions NOT_PRUNED = new
SelectedPartitions(0, ImmutableMap.of(), false, false,
+ Optional.empty());
/**
* total partition number
*/
@@ -293,12 +295,19 @@ public class LogicalFileScan extends
LogicalCatalogRelation implements SupportPr
*/
public final boolean hasPartitionPredicate;
+ /**
+ * sorted partition ranges for binary search filtering.
+ * Frozen at construction time to ensure consistency with
selectedPartitions.
+ * Empty if binary search is not applicable (e.g., default partition
only).
+ */
+ public final Optional<SortedPartitionRanges<String>>
sortedPartitionRanges;
+
/**
* Constructor for SelectedPartitions.
*/
public SelectedPartitions(long totalPartitionNum, Map<String,
PartitionItem> selectedPartitions,
boolean isPruned) {
- this(totalPartitionNum, selectedPartitions, isPruned, false);
+ this(totalPartitionNum, selectedPartitions, isPruned, false,
Optional.empty());
}
/**
@@ -306,11 +315,21 @@ public class LogicalFileScan extends
LogicalCatalogRelation implements SupportPr
*/
public SelectedPartitions(long totalPartitionNum, Map<String,
PartitionItem> selectedPartitions,
boolean isPruned, boolean hasPartitionPredicate) {
+ this(totalPartitionNum, selectedPartitions, isPruned,
hasPartitionPredicate, Optional.empty());
+ }
+
+ /**
+ * Constructor for SelectedPartitions with sorted partition ranges.
+ */
+ public SelectedPartitions(long totalPartitionNum, Map<String,
PartitionItem> selectedPartitions,
+ boolean isPruned, boolean hasPartitionPredicate,
+ Optional<SortedPartitionRanges<String>> sortedPartitionRanges)
{
this.totalPartitionNum = totalPartitionNum;
this.selectedPartitions =
ImmutableMap.copyOf(Objects.requireNonNull(selectedPartitions,
"selectedPartitions is null"));
this.isPruned = isPruned;
this.hasPartitionPredicate = hasPartitionPredicate;
+ this.sortedPartitionRanges = sortedPartitionRanges;
}
@Override
@@ -325,12 +344,14 @@ public class LogicalFileScan extends
LogicalCatalogRelation implements SupportPr
return isPruned == that.isPruned
&& hasPartitionPredicate == that.hasPartitionPredicate
&& Objects.equals(
- selectedPartitions.keySet(),
that.selectedPartitions.keySet());
+ selectedPartitions.keySet(),
that.selectedPartitions.keySet())
+ && Objects.equals(
+ sortedPartitionRanges.isPresent(),
that.sortedPartitionRanges.isPresent());
}
@Override
public int hashCode() {
- return Objects.hash(selectedPartitions, isPruned,
hasPartitionPredicate);
+ return Objects.hash(selectedPartitions, isPruned,
hasPartitionPredicate, sortedPartitionRanges.isPresent());
}
}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/expression/rules/BinarySearchPartitionInconsistencyTest.java
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/expression/rules/BinarySearchPartitionInconsistencyTest.java
new file mode 100644
index 00000000000..db3d46489f3
--- /dev/null
+++
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/expression/rules/BinarySearchPartitionInconsistencyTest.java
@@ -0,0 +1,255 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rules;
+
+import org.apache.doris.analysis.PartitionValue;
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.ListPartitionItem;
+import org.apache.doris.catalog.PartitionItem;
+import org.apache.doris.catalog.PartitionKey;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.nereids.CascadesContext;
+import
org.apache.doris.nereids.rules.expression.rules.PartitionPruner.PartitionPruneResult;
+import
org.apache.doris.nereids.rules.expression.rules.PartitionPruner.PartitionTableType;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.SlotReference;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import
org.apache.doris.nereids.trees.plans.logical.LogicalFileScan.SelectedPartitions;
+import org.apache.doris.nereids.types.IntegerType;
+import org.apache.doris.utframe.TestWithFeService;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Maps;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.Map;
+import java.util.Optional;
+
+/**
+ * Verify fix for the TOCTOU bug in {@link SelectedPartitions}.
+ *
+ * <p>Before the fix: {@code nameToPartitionItem} was frozen in {@code
SelectedPartitions}
+ * at T1 but {@code sortedPartitionRanges} was re-read from cache at T2. If
the cache
+ * changed between T1 and T2 (concurrent ADD/DROP PARTITION), the two
snapshots diverged,
+ * binary search returned partitions not present in {@code
nameToPartitionItem}, and
+ * {@code ImmutableMap.copyOf()} threw NPE on the null value.
+ *
+ * <p>After the fix: {@code sortedPartitionRanges} is built from the same
+ * {@code nameToPartitionItems} and frozen inside {@code SelectedPartitions}
at T1.
+ * The pruning rule reads both from the frozen snapshot so they are always
consistent.
+ */
+public class BinarySearchPartitionInconsistencyTest extends TestWithFeService {
+ private final Column partitionColumn = new Column("a", PrimitiveType.INT);
+ private final SlotReference slotA = new SlotReference("a",
IntegerType.INSTANCE);
+ private CascadesContext cascadesContext;
+
+ @Override
+ protected void runBeforeAll() throws Exception {
+ cascadesContext = createCascadesContext("select * from t1");
+ }
+
+ private ListPartitionItem listItem(int value) throws AnalysisException {
+ PartitionValue partitionValue = new
PartitionValue(String.valueOf(value));
+ PartitionKey partitionKey = PartitionKey.createPartitionKey(
+ ImmutableList.of(partitionValue),
ImmutableList.of(partitionColumn));
+ return new ListPartitionItem(ImmutableList.of(partitionKey));
+ }
+
+ /**
+ * Test that SelectedPartitions correctly freezes sortedPartitionRanges
+ * from the same partition items map.
+ */
+ @Test
+ public void testSelectedPartitionsFreezesSortedPartitionRanges() throws
AnalysisException {
+ Map<String, PartitionItem> nameToPartitionItems =
Maps.newHashMapWithExpectedSize(3);
+ nameToPartitionItems.put("p1", listItem(1));
+ nameToPartitionItems.put("p2", listItem(2));
+ nameToPartitionItems.put("p3", listItem(3));
+
+ Optional<SortedPartitionRanges<String>> sortedRanges =
Optional.ofNullable(
+ SortedPartitionRanges.build(nameToPartitionItems));
+
+ SelectedPartitions sp = new SelectedPartitions(
+ nameToPartitionItems.size(), nameToPartitionItems, false,
false, sortedRanges);
+
+ // sortedPartitionRanges field is populated
+ Assertions.assertTrue(sp.sortedPartitionRanges.isPresent(),
+ "sortedPartitionRanges should be present when partitions
exist");
+ Assertions.assertNotNull(sp.sortedPartitionRanges.get(),
+ "sortedPartitionRanges should not be null");
+ }
+
+ /**
+ * Test that NOT_PRUNED has empty sortedPartitionRanges.
+ */
+ @Test
+ public void testNotPrunedHasEmptySortedPartitionRanges() {
+
Assertions.assertFalse(SelectedPartitions.NOT_PRUNED.sortedPartitionRanges.isPresent(),
+ "NOT_PRUNED should have empty sortedPartitionRanges");
+ Assertions.assertEquals(Optional.empty(),
SelectedPartitions.NOT_PRUNED.sortedPartitionRanges,
+ "NOT_PRUNED.sortedPartitionRanges should be Optional.empty()");
+ }
+
+ /**
+ * Test consistent snapshot: when both nameToPartitionItem and
sortedPartitionRanges
+ * come from the same data, no partition returned by binary search is
missing
+ * from the map. Simulates the PruneFileScanPartition flow after the fix.
+ */
+ @Test
+ public void testConsistentSnapshotNoMissingPartitions() throws
AnalysisException {
+ // One consistent snapshot built at T1
+ Map<String, PartitionItem> partitionItems =
Maps.newHashMapWithExpectedSize(4);
+ partitionItems.put("p1", listItem(1));
+ partitionItems.put("p2", listItem(2));
+ partitionItems.put("p3", listItem(3));
+ partitionItems.put("p4", listItem(4));
+ partitionItems = ImmutableMap.copyOf(partitionItems);
+
+ SortedPartitionRanges<String> sortedRanges =
SortedPartitionRanges.build(partitionItems);
+ Assertions.assertNotNull(sortedRanges);
+
+ // predicate hits p4, which DOES exist in both snapshots
+ Expression predicate = new EqualTo(slotA, Literal.of(4));
+
+ PartitionPruneResult<String> result = PartitionPruner.pruneWithResult(
+ ImmutableList.of(slotA), predicate, partitionItems,
cascadesContext,
+ PartitionTableType.EXTERNAL, Optional.of(sortedRanges));
+
+ // p4 should be in the result since it exists in the map
+ Assertions.assertTrue(result.partitions.contains("p4"),
+ "p4 should be returned when it exists in the consistent
snapshot");
+
+ // Simulate the PruneFileScanPartition lookup loop. Since both the
partition map
+ // and sortedPartitionRanges come from the same frozen snapshot, every
returned
+ // partition must be present in the map — assert the invariant rather
than skip.
+ Map<String, PartitionItem> selectedPartitionItems = Maps.newHashMap();
+ for (String name : result.partitions) {
+ PartitionItem item = partitionItems.get(name);
+ Assertions.assertNotNull(item,
+ "pruned partition " + name + " must be present in the
consistent snapshot");
+ selectedPartitionItems.put(name, item);
+ }
+
+ // All returned partitions are in the map, none skipped
+ Assertions.assertEquals(result.partitions.size(),
selectedPartitionItems.size(),
+ "all returned partitions should be present in the consistent
snapshot");
+ Assertions.assertNotNull(selectedPartitionItems.get("p4"),
+ "p4 PartitionItem should not be null");
+ }
+
+ /**
+ * Test: if inconsistent snapshots ever reach the lookup loop (which the
fix makes
+ * impossible by freezing both from the same snapshot), the invariant
check fails
+ * loudly instead of silently returning a partial scan over fewer
partitions.
+ *
+ * <p>This mirrors the {@code Preconditions.checkState} in
PruneFileScanPartition:
+ * a missing partition is treated as an invariant violation, not a
partition to skip.
+ */
+ @Test
+ public void testInvariantViolationFailsLoudly() throws AnalysisException {
+ // old snapshot (no p4)
+ Map<String, PartitionItem> nameToPartitionItem = ImmutableMap.of(
+ "p1", listItem(1),
+ "p2", listItem(2),
+ "p3", listItem(3));
+
+ // new snapshot (has p4) — simulating inconsistent data that the fix
prevents,
+ // but the invariant check must still surface it rather than silently
skip p4
+ Map<String, PartitionItem> newPartitions =
Maps.newHashMapWithExpectedSize(4);
+ newPartitions.put("p1", listItem(1));
+ newPartitions.put("p2", listItem(2));
+ newPartitions.put("p3", listItem(3));
+ newPartitions.put("p4", listItem(4));
+ SortedPartitionRanges<String> sortedPartitionRanges =
SortedPartitionRanges.build(newPartitions);
+ Assertions.assertNotNull(sortedPartitionRanges);
+
+ Expression predicate = new EqualTo(slotA, Literal.of(4));
+
+ PartitionPruneResult<String> result = PartitionPruner.pruneWithResult(
+ ImmutableList.of(slotA), predicate, nameToPartitionItem,
cascadesContext,
+ PartitionTableType.EXTERNAL,
Optional.of(sortedPartitionRanges));
+
+ // binary search returns p4 from the new snapshot
+ Assertions.assertTrue(result.partitions.contains("p4"),
+ "binary search returns p4 from the sortedPartitionRanges");
+
+ // The invariant check (mirroring PruneFileScanPartition) must fail
loudly:
+ // p4 is in the pruned result but missing from nameToPartitionItem.
+ Assertions.assertThrows(IllegalStateException.class, () -> {
+ for (String name : result.partitions) {
+ PartitionItem item = nameToPartitionItem.get(name);
+ if (item == null) {
+ throw new IllegalStateException(
+ "pruned partition " + name + " is missing in the
selected partitions snapshot");
+ }
+ }
+ }, "a missing partition must fail the invariant instead of being
silently skipped");
+ }
+
+ /**
+ * Test that with binary search disabled, sequentialFiltering is
unaffected.
+ */
+ @Test
+ public void testSequentialFilteringNotAffectedByInconsistentSnapshot()
throws AnalysisException {
+ Map<String, PartitionItem> nameToPartitionItem =
Maps.newHashMapWithExpectedSize(3);
+ nameToPartitionItem.put("p1", listItem(1));
+ nameToPartitionItem.put("p2", listItem(2));
+ nameToPartitionItem.put("p3", listItem(3));
+ nameToPartitionItem = ImmutableMap.copyOf(nameToPartitionItem);
+
+ Expression predicate = new EqualTo(slotA, Literal.of(4));
+
+ // binary search disabled -> sequentialFiltering path
+ PartitionPruneResult<String> result = PartitionPruner.pruneWithResult(
+ ImmutableList.of(slotA), predicate, nameToPartitionItem,
cascadesContext,
+ PartitionTableType.EXTERNAL, Optional.empty());
+
+ Assertions.assertFalse(result.partitions.contains("p4"),
+ "sequential filtering only uses nameToPartitionItem, p4 is
never returned");
+ Assertions.assertTrue(result.partitions.isEmpty(),
+ "no partition matches a=4 in this snapshot");
+ }
+
+ /**
+ * Test that SelectedPartitions factory method produces consistent state.
+ */
+ @Test
+ public void testNewSelectedPartitionsHasConsistentSortedRanges() throws
AnalysisException {
+ Map<String, PartitionItem> items = Maps.newHashMapWithExpectedSize(2);
+ items.put("p1", listItem(1));
+ items.put("p2", listItem(2));
+
+ Optional<SortedPartitionRanges<String>> ranges = Optional.ofNullable(
+ SortedPartitionRanges.build(items));
+
+ // The 5-arg constructor freezes both fields from the same snapshot
+ SelectedPartitions sp = new SelectedPartitions(items.size(), items,
false, false, ranges);
+
+ Assertions.assertTrue(sp.sortedPartitionRanges.isPresent());
+ SortedPartitionRanges<String> frozen = sp.sortedPartitionRanges.get();
+
+ // The frozen sortedPartitionRanges contains exactly the same
partitions as selectedPartitions
+ int partitionCountInSortedRanges = frozen.sortedPartitions.size() +
frozen.defaultPartitions.size();
+ Assertions.assertEquals(sp.selectedPartitions.size(),
partitionCountInSortedRanges,
+ "sortedPartitionRanges should cover all partitions in
selectedPartitions");
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
