andygrove opened a new issue, #901:
URL: https://github.com/apache/datafusion-comet/issues/901
### Describe the bug
When running TPC-DS benchmarks against 100 GB data set I see a large
regression in performance. For example, here are the timings for q72 before and
after adding support for SMJ with join condition.
Adding support for SMJ with join condition means that more of the plan is
likely running natively and the performance issue isn't necessarily directly
related to SMJ.
## before
```
"72": [
22.729433059692383,
18.11495876312256,
17.545786142349243
]
```
## after
```
"72": [
38.576566219329834,
35.433213233947754,
35.262585401535034
]
```
A secondary issue is that I do not see metrics for CometSort /
CometSortMergeJoin.
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
