alamb commented on code in PR #9218:
URL: https://github.com/apache/arrow-rs/pull/9218#discussion_r2702882350
##########
arrow-select/src/coalesce.rs:
##########
@@ -804,9 +805,46 @@ mod tests {
.run();
}
+ /// Coalesce multiple batches, 8k rows, with mixed filers, including 100%
+ #[test]
+ fn test_coalesce_filtered_mixed() {
+ let mut filter_builder = RandomFilterBuilder {
+ num_rows: 800,
+ selectivity: 0.90,
+ seed: 0,
+ };
+
+ let mut test = Test::new("coalesce_filtered_mixed");
+ for _ in 0..3 {
+ // also add in a batch that selects almost all rows and when
+ // sliced will have some batches that are entirely used
+ let mut all_filter_builder = BooleanBufferBuilder::new(1000);
Review Comment:
this was my attempt to make sure that at least some all true slices are
appended to the filter
##########
arrow-select/src/coalesce.rs:
##########
@@ -1609,8 +1775,8 @@ mod tests {
let mut coalescer = BatchCoalescer::new(
Arc::new(Schema::new(vec![Field::new("c0", DataType::Int32,
false)])),
100,
- );
- coalescer.set_biggest_coalesce_batch_size(Some(500));
+ )
+ .with_biggest_coalesce_batch_size(Some(500));
Review Comment:
drive by cleanup: switch one test to use the alternate syntax to add coveage
##########
arrow-select/src/coalesce.rs:
##########
@@ -1254,17 +1307,43 @@ mod tests {
///
/// Returns the resulting output batches
fn run(self) -> Vec<RecordBatch> {
+ // Test several permutations of input batches:
+ // 1. Removing nulls from some batches (test non-null fast paths)
+ // 2. Empty batches
+ // 3. One column (from the batch)
+ let mut extra_tests = vec![];
Review Comment:
This is the core extra coverage -- it makes a bunch of extra permutations to
run after the first one is run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
