viirya commented on code in PR #12090:
URL: https://github.com/apache/datafusion/pull/12090#discussion_r1724087441
##########
datafusion/physical-plan/src/joins/sort_merge_join.rs:
##########
@@ -596,7 +596,7 @@ struct BufferedBatch {
pub size_estimation: usize,
/// The indices of buffered batch that failed the join filter.
/// When dequeuing the buffered batch, we need to produce null joined rows
for these indices.
- pub join_filter_failed_idxs: HashSet<u64>,
+ pub join_filter_failed_map: HashMap<u64, bool>,
Review Comment:
Added comment.
##########
datafusion/physical-plan/src/joins/sort_merge_join.rs:
##########
@@ -1229,11 +1229,19 @@ impl SMJStream {
}
buffered_batch.null_joined.clear();
- // For buffered rows which are joined with streamed side but
doesn't satisfy the join filter
+ // For buffered row which is joined with streamed side rows but
all joined rows
+ // don't satisfy the join filter
if output_not_matched_filter {
+ let buffered_indices = buffered_batch
Review Comment:
Okay.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
