yew1eb opened a new issue, #2405:
URL: https://github.com/apache/auron/issues/2405
In TPC-H Q21 (SF=10, lineitem = 59,986,052 rows), the right side of the
lineitem `SortMergeJoin` is sorted on the single-column integer key l_orderkey
(Int64). The Auron native SortExec is ~6x slower than Spark's native sort on
this path, because it degrades a single-column integer sort into a generic
byte-comparison sort.
## Root cause
| Implementation | Sort approach | Single-column Int64 handling |
| :--- | :---: | ---: |
| Spark SortExec | prefix + radix sort (O(n)) | radix sort on 8-byte prefix |
| Auron SortExec | generic external sort | RowConverter encodes Int64 into 9
bytes (1B null + 8B big-endian, sign-flipped), then byte-compares |
▎ Therefore Spark holds an algorithmic advantage in this scenario, while
Auron's current implementation degrades single-column integer sorting into
generic byte-comparison sorting, amplifying the overhead.
**Auron NativeSort time : 13.8s**
<img width="946" height="884" alt="Image"
src="https://github.com/user-attachments/assets/8eb56012-68d4-4d94-a36f-b73dd89ecba0";
/>
**Spark Sort time: 1.6s**
<img width="913" height="817" alt="Image"
src="https://github.com/user-attachments/assets/c8194406-c485-4660-947f-aead670c78a6";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
