eejbyfeldt commented on code in PR #13134:
URL: https://github.com/apache/datafusion/pull/13134#discussion_r1822106767
##########
datafusion/physical-plan/src/joins/sort_merge_join.rs:
##########
@@ -784,6 +790,29 @@ fn get_corrected_filter_mask(
corrected_mask.extend(vec![Some(false); null_matched]);
Some(corrected_mask.finish())
}
+ JoinType::LeftMark => {
+ for i in 0..row_indices_length {
+ let last_index =
+ last_index_for_row(i, row_indices, batch_ids,
row_indices_length);
+ if filter_mask.value(i) && !seen_true {
Review Comment:
We can not use just a `LeftOuter` because it will produce multiple rows per
left row if there multiple matches in the right input. (Note that this is what
we do in master and this PR adds a test case that produce incorrect results on
master) But we could do a distinct + LeftOuter to handle the duplicates but
should be much slower than just doing a mark join.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
