jayzhan211 commented on PR #26010:
URL: https://github.com/apache/datafusion/pull/26010#issuecomment-6071772766
Same bug in the semi/anti/mark stream: with a column-free filter, `try_new`
fails with `must either specify a row count or at least one column` for all six
types. SQL can't reach it because the planner pushes the predicate into an
input, but a `SortMergeJoinExec` built directly can. Fine to handle in a
follow-up; it's a one-line fix:
```diff
- let filter_batch = RecordBatch::try_new(Arc::clone(filter.schema()),
columns)?;
+ let filter_batch = RecordBatch::try_new_with_options(
+ Arc::clone(filter.schema()),
+ columns,
+ &RecordBatchOptions::new().with_row_count(Some(num_outer_rows)),
+ )?;
```
Test (fails on this branch, passes with the fix):
```rs
#[tokio::test]
async fn join_semi_anti_mark_column_free_filter() -> Result<()> {
for join_type in [
LeftSemi, LeftAnti, LeftMark, RightSemi, RightAnti, RightMark,
] {
let left = build_table(
("a1", &vec![1, 2, 3]),
("b1", &vec![4, 5, 7]),
("c1", &vec![7, 8, 9]),
);
let right = build_table(
("a2", &vec![10, 20, 30]),
("b1", &vec![4, 5, 6]),
("c2", &vec![70, 80, 90]),
);
let on = vec![(
Arc::new(Column::new_with_schema("b1", &left.schema())?) as _,
Arc::new(Column::new_with_schema("b1", &right.schema())?) as _,
)];
let filter = JoinFilter::new(
Arc::new(Literal::new(ScalarValue::Boolean(Some(false)))),
vec![],
Arc::new(Schema::empty()),
);
let (_, batches) =
join_collect_with_filter(left, right, on, filter,
join_type).await?;
let rows: usize = batches.iter().map(|b| b.num_rows()).sum();
let expected = match join_type {
LeftSemi | RightSemi => 0,
_ => 3,
};
assert_eq!(rows, expected, "{join_type:?}");
}
Ok(())
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
