jayzhan211 commented on code in PR #26771:
URL: https://github.com/apache/datafusion/pull/26771#discussion_r4551071000
##########
datafusion/physical-plan/src/joins/nested_loop_join.rs:
##########
@@ -2835,6 +2839,28 @@ impl NestedLoopJoinStream {
&mut self,
cx: &mut std::task::Context<'_>,
) -> ControlFlow<Poll<Option<Result<RecordBatch>>>> {
+ // Left existence joins need no more probe input once every build row
+ // has a match. Spill passes must still consume the right input so it
is
+ // available for replay against subsequent left chunks.
+ if !self.is_memory_limited()
+ && matches!(
Review Comment:
Optional cleanup: this join-type check is repeated in `process_probe_batch`.
A small helper next to `is_memory_limited` would keep the two in sync:
```diff
+ /// Left semi/anti/mark joins only need to know whether each left row
matched.
+ fn is_left_existence_join(&self) -> bool {
+ matches!(
+ self.join_type,
+ JoinType::LeftSemi | JoinType::LeftAnti | JoinType::LeftMark
+ )
+ }
@@ fn handle_fetching_right
if !self.is_memory_limited()
- && matches!(
- self.join_type,
- JoinType::LeftSemi | JoinType::LeftAnti | JoinType::LeftMark
- )
+ && self.is_left_existence_join()
&& let Some(left_data) = &self.buffered_left_data
@@ fn process_probe_batch
- if matches!(
- self.join_type,
- JoinType::LeftSemi | JoinType::LeftAnti | JoinType::LeftMark
- ) && self.left_probe_idx >= self.left_unmatched_end
+ if self.is_left_existence_join() && self.left_probe_idx >=
self.left_unmatched_end
{
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
