houseme commented on PR #660: URL: https://github.com/apache/avro-rs/pull/660#issuecomment-5551450542
Yes. I added a dedicated benchmark for this in `avro/benches/crc32_snappy.rs`. It now measures three paths on both aarch64 and x86_64: - raw CRC-32/ISO-HDLC; - Snappy compression plus the Avro CRC32 trailer; - Snappy decompression plus CRC32 verification. The results are documented in `avro/benches/crc32_snappy_results.md`. The short version is that the isolated CRC gain does carry into the Snappy-shaped encode/decode paths, especially from 1 KiB upward. On the tested aarch64 machine, Snappy compress + CRC improved up to 1.58x and decompress + CRC up to 2.29x. On the tested x86_64 EPYC host, compress + CRC improved up to 4.32x and decompress + CRC up to 3.72x. The SIMD detail is architecture-specific: on aarch64 the improvement comes from `crc-fast` using PMULL/AES folding, with SHA3/EOR3-assisted variants when available. On x86_64, SSE4.2 CRC32 is CRC32C/ISCSI rather than CRC-32/ISO-HDLC, so the relevant acceleration is the PCLMULQDQ / VPCLMULQDQ carry-less multiply family. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
