martin-g commented on code in PR #20228:
URL: https://github.com/apache/datafusion/pull/20228#discussion_r2782276949
##########
datafusion/physical-plan/src/joins/hash_join/exec.rs:
##########
@@ -1476,12 +1506,53 @@ impl ExecutionPlan for HashJoinExec {
filter: dynamic_filter,
build_accumulator: OnceLock::new(),
}),
+ fetch: self.fetch,
});
result = result.with_updated_node(new_node as Arc<dyn
ExecutionPlan>);
}
}
Ok(result)
}
+
+ fn supports_limit_pushdown(&self) -> bool {
+ // Hash join execution plan does not support pushing limit down
through to children
+ // because the children don't know about the join condition and can't
+ // determine how many rows to produce
+ false
+ }
+
+ fn fetch(&self) -> Option<usize> {
+ self.fetch
+ }
+
+ fn with_fetch(&self, limit: Option<usize>) -> Option<Arc<dyn
ExecutionPlan>> {
Review Comment:
Is this method really needed ?
If it is not then I'd suggest to remove it.
If the HashJoinExec is already executed then setting a new limit will be
confusing/inconsistent unless it is re-executed again.
Alternatively it could be implemented as:
```rust
HashJoinExecBuilder::from(self)
.with_fetch(limit)
.build()
.ok()
.map(|exec| Arc::new(exec) as _)
```
This way it won't keep the calculated state.
##########
datafusion/physical-plan/src/joins/hash_join/exec.rs:
##########
Review Comment:
The `fetch` field is not preserved here. Is this intentional ?
##########
datafusion/physical-plan/src/joins/hash_join/exec.rs:
##########
Review Comment:
The `fetch` field is not preserved here. Is this intentional ?
##########
datafusion/physical-plan/src/joins/hash_join/exec.rs:
##########
Review Comment:
Should the statistics take into account the `fetch`/limit when estimating ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
