alamb commented on code in PR #10852:
URL: https://github.com/apache/arrow-rs/pull/10852#discussion_r4038140491
##########
parquet/src/arrow/arrow_reader/selection/mod.rs:
##########
@@ -668,6 +701,94 @@ impl RowSelection {
}
}
+/// Returns the minimum normalized run count for which Auto prefers a mask.
+///
+/// This matches `total_rows < run_count.saturating_mul(threshold)`. For totals
+/// below `usize::MAX`, the first matching run count is
+/// `floor(total_rows / threshold) + 1`. `None` means no attainable run count
Review Comment:
maybe this would be clearer if it said "Returns `None` when selectors are
always a better choice"
##########
parquet/src/arrow/arrow_reader/selection/mod.rs:
##########
@@ -668,6 +701,94 @@ impl RowSelection {
}
}
+/// Returns the minimum normalized run count for which Auto prefers a mask.
+///
+/// This matches `total_rows < run_count.saturating_mul(threshold)`. For totals
Review Comment:
What is the relevance of matching `total_rows <
run_count.saturating_mul(threshold)`? It seems like this comment may be trying
to explain how the code was changed in this PR but will not be relevant after
this PR merges.
##########
parquet/src/arrow/arrow_reader/selection/mod.rs:
##########
@@ -668,6 +701,94 @@ impl RowSelection {
}
}
+/// Returns the minimum normalized run count for which Auto prefers a mask.
Review Comment:
```suggestion
/// Returns the minimum normalized run count for which the Auto selection
policy
/// would prefer a mask to RLE.
```
##########
parquet/src/arrow/arrow_reader/selection/mod.rs:
##########
@@ -668,6 +701,94 @@ impl RowSelection {
}
}
+/// Returns the minimum normalized run count for which Auto prefers a mask.
+///
+/// This matches `total_rows < run_count.saturating_mul(threshold)`. For totals
Review Comment:
Maybe it is trying to say that this implements the definition of the Auto
policy threshold, namely that
"average selector length below which masks are preferred" and average run
length below threshold means `total_rows / run_count < threshold` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
