singhpk234 commented on code in PR #16131:
URL: https://github.com/apache/iceberg/pull/16131#discussion_r3692687207
##########
data/src/main/java/org/apache/iceberg/data/IcebergGenerics.java:
##########
@@ -103,7 +108,16 @@ public ScanBuilder metricsReporter(MetricsReporter
reporter) {
}
public CloseableIterable<Record> build() {
- return new TableScanIterable(tableScan, reuseContainers);
+ Optional<ReadRestrictions> restrictions =
TableUtil.readRestrictions(table);
+ if (restrictions.isPresent() && restrictions.get().rowFilter() != null) {
+ this.tableScan = tableScan.filter(restrictions.get().rowFilter());
+ }
+
+ CloseableIterable<Record> records = new TableScanIterable(tableScan,
reuseContainers);
+ if (restrictions.isPresent()) {
+ records = ReadRestrictionsApplier.apply(records, restrictions.get(),
tableScan.schema());
Review Comment:
Fixed, and I think the spec already settles the mask half rather than
leaving it open — #13879 says "A reader must enforce projections on the columns
it is actually reading. Projections referencing columns that are not being read
do not apply." So throwing was a spec violation. Projections for unread columns
are now skipped.
Following your point about where enforcement binds: field ids are now
validated once in BaseRESTTable, where restrictions first meet a table, rather
than per scan, so every reader inherits the check instead of re-deriving it.
Validation spans every schema the table has had, which also fixes a case I hit
while doing this: a time-travel scan projects the historical snapshot schema,
so a restriction on a since-dropped column was being reported as an unknown
field id.
On the row filter half, I don't think that path is reachable. build() pushes
the filter into the scan before reading tableScan.schema(), and
BaseScan.lazyColumnProjection unions filter references into the projection, so
filter columns are always present on the .select() path. The only gap is
.project(Schema), which returns the schema verbatim, and there Evaluator binds
eagerly and throws. Fail-closed rather than passing rows unfiltered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
