JingsongLi commented on code in PR #8447:
URL: https://github.com/apache/paimon/pull/8447#discussion_r3520104485
##########
paimon-core/src/main/java/org/apache/paimon/table/source/AbstractDataTableScan.java:
##########
@@ -111,6 +117,37 @@ public final TableScan.Plan plan() {
protected abstract TableScan.Plan planWithoutAuth();
+ private void applyAuthFilter(@Nullable Predicate authPredicate) {
+ if (Objects.equals(authPredicate, appliedAuthPredicate)) {
+ return;
+ }
+ appliedAuthPredicate = authPredicate;
+
+ PartitionPredicate authPartitionFilter = null;
+ boolean hasNonPartitionPart = false;
+ if (authPredicate != null) {
+ // Remap field-id FieldRefs to positional indices by name (as
doAuth does on read), so
+ // pruning stays correct across schema evolution.
+ Predicate remappedAuth =
+ TableQueryAuthResult.remapPredicate(authPredicate,
schema.logicalRowType());
+ if (remappedAuth != null) {
+ Pair<Optional<PartitionPredicate>, List<Predicate>> split =
+
PartitionPredicate.splitPartitionPredicatesAndDataPredicates(
+ remappedAuth, schema.logicalRowType(),
schema.partitionKeys());
+ authPartitionFilter = split.getLeft().orElse(null);
+ hasNonPartitionPart = !split.getRight().isEmpty();
+ }
+ }
+
+ // Push only the partition part, to a dedicated slot
overwritten/cleared each plan() so a
+ // changed/removed auth leaves no stale pruning. The full filter is
enforced at read time.
+
snapshotReader.manifestsReader().withAuthPartitionFilter(authPartitionFilter);
+ if (hasNonPartitionPart) {
+ // Non-partition auth removes rows at read time, so limit push
down is unsafe.
+ snapshotReader.markHasNonPartitionFilter();
Review Comment:
If this marker is only used to make
`DataTableBatchScan.applyPushDownLimit()` return empty, can we keep this state
local to `DataTableBatchScan` instead? For example, when the auth predicate has
a non-partition part, set `pushDownLimit = null` before planning. `null`
already means no limit pushdown there, and this avoids adding
`markHasNonPartitionFilter()` to `SnapshotReader` for a read-time auth filter
that was not actually pushed through `withFilter(...)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
