mbutrovich commented on code in PR #2615:
URL: https://github.com/apache/iceberg-rust/pull/2615#discussion_r3395002730
##########
crates/iceberg/src/arrow/reader/row_filter.rs:
##########
@@ -160,8 +160,14 @@ impl ArrowReader {
/// Filters row groups by byte range to support Iceberg's file splitting.
///
- /// Iceberg splits large files at row group boundaries, so we only read
row groups
- /// whose byte ranges overlap with [start, start+length).
+ /// External engines (e.g. Spark via Comet) split a data file into
multiple scan tasks,
Review Comment:
That's good context, thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
