Re: [PR] feat: Reenable tests for filtered SMJ anti join [datafusion-comet]

[via GitHub]

andygrove commented on PR #1211:
URL: 
https://github.com/apache/datafusion-comet/pull/1211#issuecomment-2568533415

   > I am running benchmarks with this PR with 
`spark.comet.exec.sortMergeJoinWithJoinFilter.enabled=true`, and there appears 
to be a serious performance issue. TPC-H q21 has been running for 13 minutes so 
far (with Spark, it usually takes ~70 seconds).
   
   It looks like the SMJ may be producing too many rows.
   
   ## Spark
   
   
![2025-01-02_16-54](https://github.com/user-attachments/assets/3d06618e-7eb1-4449-acfa-e07db0b1dbb6)
   
   ## Comet
   
   
![2025-01-02_16-51](https://github.com/user-attachments/assets/2c938c70-47a7-49ae-b5b4-fc17c76c3226)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org