txwyy123 commented on PR #59841: URL: https://github.com/apache/spark/pull/59841#issuecomment-6554120652
## Benchmark Results: All Join Types **Environment**: local[16], 16 cores, 1M rows/side, 40+40 columns (80 output), Spark master @ `ea3e5df5` | Join Type | CG ON (ms) | CG OFF (ms) | Speedup | |---|---|---|---| | inner (equi-only) | 1,317 | 1,908 | **1.45x** | | inner (with condition) | 1,410 | 1,367 | 0.97x | | left outer | 1,405 | 1,820 | **1.30x** | | right outer | 1,056 | 1,214 | **1.15x** | | left semi | 751 | 686 | 0.91x | | left anti | 761 | 670 | 0.88x | | left outer (with condition) | 1,166 | 1,156 | 0.99x | | existence | 867 | 1,011 | **1.17x** | **Key findings**: - Full-output joins (inner, outer, existence): **1.15x-1.45x speedup** — both sides contribute columns to output, so field-based extraction + JIT compilation gives the biggest win - Semi/anti joins: codegen is ~10% slower — these output only the streamed side (40 cols), and the Volcano model efficiently copies the input UnsafeRow without per-column extraction - With-condition joins: near parity — condition evaluation overhead partially offsets the extraction speedup The fix targets the **8K JIT cliff** on wide full-output joins where `processNext()` exceeds 8K bytecode. For 80+ output columns, stock Spark either disables codegen entirely (`maxFields=100`) or produces interpreted codegen that's slower than Volcano. This fix keeps `processNext()` under 8K so C2 always compiles it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
