Awesome06 commented on PR #55171: URL: https://github.com/apache/arrow/pull/55171#issuecomment-6401981634
Replicated the benchmarks locally on Windows x86_64 (MSVC 2019), and there is a clear, substantial improvement across both execution throughput and memory consumption with the direct mixed approach! Times are microseconds per call: | Input | Same type (`utf8_view`) | CommonBinary promotion | Direct mixed (`268f038`) | | :--- | :--- | :--- | :--- | | 8 B, equal scalars | 2.88 | 3.25 | **2.92** | | 4 KiB, equal scalars | 3.02 | 3.44 | **3.07** | | 1 MiB, equal scalars | 44.26 | 88.80 | **45.49** | | 1 MiB, first byte differs | 2.81 | 44.06 | **2.90** *(15.2x faster)* | | 131,072 x 8 B, equal arrays | 899.77 | 2098.38 | **613.50** *(3.4x faster)* | | 131,072 x 64 B, equal arrays | 927.11 | 2236.50 | **847.05** *(2.6x faster)* | | 4,096 x 4 KiB, equal arrays | 687.33 | 1176.74 | **1125.24** | Beyond the **2.6x–3.4x speedup on arrays**, for **131,072 rows**, Arrow pool allocation drops from **~2.02 MiB** down to just **16 KiB** by avoiding temporary view descriptors entirely. On small scalars, fixed kernel dispatch overhead dominates the runtime, while on large payloads (e.g. **4,096 x 4 KiB = 16 MiB**), execution is bottlenecked by `memcmp` throughput scanning raw string data rather than descriptor allocation overhead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
