szehon-ho commented on code in PR #17956:
URL: https://github.com/apache/iceberg/pull/17956#discussion_r4022256784
##########
spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkPartitioningAwareScan.java:
##########
@@ -264,4 +293,98 @@ protected String groupingKeyDesc() {
.map(NestedField::name)
.collect(Collectors.joining(", "));
}
+
+ protected List<PartitionPredicate> partitionPredicates() {
+ return partitionPredicates;
+ }
+
+ private boolean matchesPartitionPredicates(
+ T task, Map<Integer, PartitionPredicateEvaluator> evaluatorsBySpecId) {
+ PartitionPredicateEvaluator evaluator =
+ evaluatorsBySpecId.computeIfAbsent(
+ task.spec().specId(), ignored -> new
PartitionPredicateEvaluator(task.spec()));
+ return evaluator.eval(task.partition());
+ }
+
+ private static List<PartitionField> activePartitionFields(PartitionSpec
spec) {
+ return spec.fields().stream()
+ .filter(field -> !field.transform().isVoid())
+ .collect(Collectors.toList());
Review Comment:
Done in `fb37a8693a`. Added coverage that writes files under `(p0, p1, p2)`,
drops `p1`, writes files under the evolved spec, and pushes an opaque predicate
on `p2`. The test verifies matching rows from both specs and exactly two
planned input partitions. The focused test passes in both local and distributed
planning modes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
