HappenLee opened a new pull request, #70754:
URL: https://github.com/apache/doris/pull/70754

   ### Background
   
   The single-argument MD5 function hashes a batch of independent strings. The 
existing AVX2 path processes multiple messages together, while ARM64 calls the 
single-message MD5 implementation for each row.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: None
   PR Type: Performance
   
   Problem Summary: Add four-lane NEON operations and reuse the existing 
two-group MD5 rounds, padding, and output conversion to process eight messages 
per batch on ARM64. Batches smaller than four messages use the existing serial 
path, and heavily imbalanced groups use serial hashing to avoid spending SIMD 
work on lanes that have already finished. Digest values and the AVX2 path are 
preserved.
   
   ### Details
   
   - **Why it was slow:** ARM64 did not use the existing multi-message 
execution framework.
   - **Optimization:** Native NEON add, rotate, Boolean operations, and a 4x4 
message-word transpose; architecture-specific selection of serial hashing for 
poorly utilized batches.
   - **Results:** Neoverse-N2, Clang 20.1.8, OpenSSL 3.0.13, `-O3 
-march=armv8-a+crc`, one pinned core; median of seven randomly interleaved 
repetitions (0.15 s minimum). At 4096 rows/batch, 16/64/256/1024-byte inputs 
achieve 2.51/2.94/3.40/3.70x throughput over the previous per-message ARM 
algorithm; mixed 0–1024-byte inputs achieve 2.05x. Highly skewed inputs are 
within 0.6% of serial in the measured cases after fallback. Single-row short 
inputs and four-row mixed inputs retain about 3% overhead. GCC 14 confirms 
2.79–3.50x on the equal-length cases.
   - **Applicability and cost:** Applies to batched single-argument MD5 on 
ARM64. A NEON batch uses a temporary 16-byte digest per row, as the existing 
AVX2 path does. Small batches and skewed message lengths are included in the 
benchmark. These are MD5 API microbenchmarks, not end-to-end SQL measurements.
   
   ARM Clang 20 results (4096 rows per batch):
   
   | Batch rows | Input bytes | Serial ns/row | Batch ns/row | Speedup |
   | ---: | --- | ---: | ---: | ---: |
   | 4096 | 0 | 141.72 | 54.66 | 2.593× |
   | 4096 | 16 | 138.45 | 55.11 | 2.512× |
   | 4096 | 64 | 250.58 | 85.21 | 2.941× |
   | 4096 | 256 | 591.60 | 173.99 | 3.400× |
   | 4096 | 1024 | 1954.94 | 528.34 | 3.700× |
   | 4096 | mixed 0–1024 | 1005.06 | 489.25 | 2.054× |
   | 4096 | 8192 + seven 16-byte strings | 1956.91 | 1962.15 | 0.997× |
   
   ### Testing
   
   - Actual `be/src/util/md5.cpp` and `be/test/util/md5_test.cpp` compiled in 
isolation with ASAN/UBSAN; seven tests cover known digests, padding boundaries, 
partial SIMD groups, binary data, unaligned input, zero-length null pointers, 
and guarded output.
   - Native ARM64 validation with GCC 14 and Clang 20; x86 AVX2 validation with 
Clang 20. [Reproducible workflow and 
artifacts](https://github.com/HappenLee/incubator-doris/actions/runs/44500133646).
   - Added `benchmark_md5.hpp` and registered it in the standard benchmark 
entry point; serial and batch implementations run on the same inputs with setup 
excluded from timing.
   - clang-format 16, build hygiene, changed-line clang-tidy, and `git diff 
--check` passed locally. clang-tidy used an isolated compilation database after 
the full UT build was blocked by missing third-party libraries.
   - The standard `run-be-ut.sh -j 48 --run --filter='Md5Test.*'` was attempted 
locally but could not finish because the installed third-party package lacks 
`libgoogle_cloud_cpp_oauth2.a`. A complete BE build and SQL regression run are 
not claimed.
   
   ### Release note
   
   Improve the performance of batched single-argument MD5 on ARM64 with NEON, 
retaining serial hashing for small or highly imbalanced groups.
   
   ### Check List (For Author)
   
   - Behavior changed: No
   - Does this need documentation: No
   - Pre-submission status: Partially verified: isolated native ARM64/x86 tests 
and benchmarks completed; full BE build and SQL regression are pending CI.
   
   ### AI Disclosure
   
   - Tool: OpenAI Codex
   - Generated: NEON implementation, tests, benchmark, and PR description.
   - Human review: Not yet performed.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to