SubhamSinghal commented on code in PR #25467:
URL: https://github.com/apache/datafusion/pull/25467#discussion_r4052513334
##########
datafusion/physical-plan/src/joins/piecewise_merge_join/exec.rs:
##########
@@ -320,14 +320,21 @@ impl PiecewiseMergeJoinExec {
join_type: JoinType,
num_partitions: usize,
) -> Result<Self> {
- // Semi/Anti joins are handled by the existence streams; Mark joins
are not
- // supported yet.
- if is_existence_join(join_type) &&
!is_supported_existence_join(join_type) {
- return not_impl_err!(
- "Existence join {join_type} is currently not supported for
PiecewiseMergeJoin"
- );
- }
-
+ // There is no `null_aware` parameter here, unlike
`HashJoinExec::try_new`: this
+ // constructor cannot express a null-aware mark join (see
`JoinType::LeftMark`'s
+ // scalar-`NOT IN` variant, where `mark` is nullable), and
`mark_streamed_batch`/
+ // `emit_matched` always build a non-nullable `mark` column. That is
only sound
+ // because a null-aware mark join can never reach here:
`decorrelate_predicate_subquery`
+ // only sets `null_aware` when the whole predicate is pure
hash-equality with no
+ // residual (`mark_filter_is_hashable_only`), which means `join_on` is
always
+ // non-empty for one -- and the PWMJ branch in `physical_planner.rs`
only ever
+ // constructs this exec when `join_on` is empty. If either side of
that ever changes
+ // (the PWMJ gate growing to accept a residual equijoin condition
alongside a range
+ // predicate -- see the `TODO` on that branch -- or decorrelation
producing
+ // `null_aware` from something other than a pure-equality predicate),
this invariant
+ // breaks silently: a null-aware mark join would compute a plain
boolean `mark` where
+ // SQL requires `NULL` (`UNKNOWN`), and nothing here would notice.
+ //
Review Comment:
Addressed in 8885ca1914c081f0f5c97984d7248fcf5c18cd7d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
