anoopj opened a new pull request, #3117: URL: https://github.com/apache/iceberg-rust/pull/3117
## Which issue does this PR close? Refs https://github.com/apache/iceberg-rust/issues/2879 ## What changes are included in this PR? A projection of only metadata columns (e.g. SELECT _row_id) reads no data columns for their values, since they are synthesized or constant. Without an independent row-count source, the projection mask fell back to reading every data column just to recover the row count. Install the RowNumber virtual column (derived from row-group metadata, no column I/O) as that source for any metadata-only projection, so the read is pruned to zero data columns. COUNT(*) (an empty projection) is left unchanged. Applies to all metadata-only projections: _row_id, _last_updated_sequence_number, _file, _spec_id, and _partition. Measured on a 58 MB, 200k-row, 30-column Parquet file, projecting only _last_updated_sequence_number: this change (pruned): 524 KB read, 6 ms before (read all columns): 58.6 MB read, 1289 ms ## Are these changes tested? Unit tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
