andygrove merged PR #962:
URL: https://github.com/apache/datafusion-comet/pull/962
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@da
mbutrovich commented on code in PR #962:
URL: https://github.com/apache/datafusion-comet/pull/962#discussion_r1774297891
##
native/core/src/execution/datafusion/planner.rs:
##
@@ -917,15 +917,11 @@ impl PhysicalPlanner {
let fetch = sort.fetch.map(|num| num as
mbutrovich commented on code in PR #962:
URL: https://github.com/apache/datafusion-comet/pull/962#discussion_r1774297891
##
native/core/src/execution/datafusion/planner.rs:
##
@@ -917,15 +917,11 @@ impl PhysicalPlanner {
let fetch = sort.fetch.map(|num| num as
andygrove commented on code in PR #962:
URL: https://github.com/apache/datafusion-comet/pull/962#discussion_r1774114004
##
native/core/src/execution/datafusion/planner.rs:
##
@@ -1263,21 +1268,6 @@ impl PhysicalPlanner {
None
};
-// DataFusion Joi
andygrove commented on PR #962:
URL: https://github.com/apache/datafusion-comet/pull/962#issuecomment-2372397547
A single run of TPC-DS comparing main to this PR does not show a significant
difference (~1% difference). I did not expect this to make much difference, but
it does remove a redu
andygrove commented on code in PR #962:
URL: https://github.com/apache/datafusion-comet/pull/962#discussion_r1774084525
##
native/core/src/execution/datafusion/planner.rs:
##
@@ -1290,6 +1285,21 @@ impl PhysicalPlanner {
))
}
+fn wrap_in_copy_exec(
+i
andygrove commented on PR #962:
URL: https://github.com/apache/datafusion-comet/pull/962#issuecomment-2372364737
I ran TPC-DS benchmarks comparing the latest from the main branch with this
PR, and somehow, this PR introduces a considerable regression in performance
(~20%). I am investigatin
mbutrovich commented on code in PR #962:
URL: https://github.com/apache/datafusion-comet/pull/962#discussion_r1774012487
##
native/core/src/execution/datafusion/planner.rs:
##
@@ -1290,6 +1285,21 @@ impl PhysicalPlanner {
))
}
+fn wrap_in_copy_exec(
+
andygrove opened a new pull request, #962:
URL: https://github.com/apache/datafusion-comet/pull/962
## Which issue does this PR close?
N/A
## Rationale for this change
We were injecting too many CopyExecs for SMJ. Plans looked like this:
```
Sort
