alamb commented on code in PR #24648:
URL: https://github.com/apache/datafusion/pull/24648#discussion_r3873271154
##########
datafusion/datasource/src/statistics.rs:
##########
@@ -44,6 +44,8 @@ pub(crate) struct MinMaxStatistics {
min_by_sort_order: Rows,
max_by_sort_order: Rows,
sort_order: LexOrdering,
+ /// Maps statistics row indices to input file indices when empty files
were skipped.
Review Comment:
would it be simpler to just skip empty files on construction? Like filter
them out entirely?
It seems like a lot of work/code to handle empty files that could be avoided
earlier on in a single location
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
