jonathanc-n commented on code in PR #16660:
URL: https://github.com/apache/datafusion/pull/16660#discussion_r2190820204
##########
datafusion/physical-plan/src/joins/utils.rs:
##########
@@ -808,16 +809,22 @@ pub(crate) fn get_final_indices_from_shared_bitmap(
pub(crate) fn get_final_indices_from_bit_map(
left_bit_map: &BooleanBufferBuilder,
join_type: JoinType,
+ // We add a flag for whether this is being passed from the
`PiecewiseMergeJoin`
+ // because the bitmap can be for left + right `JoinType`s
+ piecewise: bool,
) -> (UInt64Array, UInt32Array) {
let left_size = left_bit_map.len();
- if join_type == JoinType::LeftMark {
+ if join_type == JoinType::LeftMark || (join_type == JoinType::RightMark &&
piecewise)
+ {
let left_indices = (0..left_size as u64).collect::<UInt64Array>();
let right_indices = (0..left_size)
.map(|idx| left_bit_map.get_bit(idx).then_some(0))
.collect::<UInt32Array>();
return (left_indices, right_indices);
}
- let left_indices = if join_type == JoinType::LeftSemi {
+ let left_indices = if join_type == JoinType::LeftSemi
+ || (join_type == JoinType::RightSemi && piecewise)
Review Comment:
No, it falls through with left semi as well. In left and right
semi/anti/mark join we use the bitmap to mark all matched sides on the buffered
side (this is done in `process_unmatched_buffered_batch`), we use the flag to
only allow right semi/anti/mark to follow through when calling from the
piecewise join. Usually the bitmap is only used to mark the unmatched rows on
left side, which is why it originally only holds support for Left
semi/anti/mark. I'll add a comment at the beginning of process unmatched
buffered batch to explain this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
