kevinwilfong opened a new pull request, #13567:
URL: https://github.com/apache/gluten/pull/13567

   ## What changes are proposed in this pull request?
   I recently added an optimization in Velox's SortMergeJoin operator to only 
buffer the right side of the join and stream the left side, when the join type 
allows. This change allows us to take advantage of the optimization in Gluten 
by applying the same optimization used in HashJoin to swap the inputs if 
necessary and put the larger input on the left side of the join. 
   
   Preserves Spark children, keys, join types and output metadata. Preserves 
skew joins and joins without known input sizes, and retains forced hash join 
precedence. Other backends keep their existing conversion behavior.
   
   ## How was this patch tested?
   
   Changing the TPCH benchmark to use sort merge joins, Q3 and Q7 are relevant. 
Performance is on par, but memory usage is dramatically lower (this is 
expected):
   
   <html><head></head><body><p><strong>Memory:</strong> representative native 
join <code>peakMemoryBytes</code> snapshots:</p>
   
   Join | Optimization disabled | Optimization enabled | Reduction
   -- | -- | -- | --
   Q3: customer/orders | 576 KiB | 240 KiB | 58.3%
   Q7: supplier/lineitem | 864 KiB | 288 KiB | 66.7%
   
   
   </body></html>
   
   Added tests to cover the translation and execution.
   
   ## Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: OpenAI Codex GPT-6
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to