iemejia opened a new pull request, #3567: URL: https://github.com/apache/parquet-java/pull/3567
Part of #3530 — Apache Parquet Java Performance Improvements ## Summary Optimize scalar hot paths for DELTA_BINARY_PACKED, DELTA_LENGTH_BYTE_ARRAY, and DELTA_BYTE_ARRAY. **DELTA_BINARY_PACKED reader**: Cache `BytePackerForLong` instances, add `unpack32Values` bulk method, replace `ByteBuffer` with reused `byte[]` for mini-block data. **DELTA_BINARY_PACKED writers**: Cache `BytePackerForLong` instances, add `pack32Values`. **DELTA_LENGTH_BYTE_ARRAY writer**: Remove `LittleEndianDataOutputStream` wrapper; write directly to `CapacityByteArrayOutputStream` via `BytesUtils`. **DELTA_BYTE_ARRAY reader**: `ByteArraySliceOutputStream` to eliminate temporary copies. **DELTA_BYTE_ARRAY writer**: `Arrays.mismatch` for SIMD prefix-length computation, direct `writeBytes(byte[],int,int)` to avoid `Binary` allocations. JMH benchmarks: `DeltaBinaryPackedEncodingBenchmark`, `DeltaBinaryPackedDecodingBenchmark`, `DeltaByteArrayEncodingBenchmark`, `DeltaByteArrayDecodingBenchmark`, `DeltaLengthByteArrayEncodingBenchmark`, `DeltaLengthByteArrayDecodingBenchmark`, `LongDeltaDecodingBenchmark`. ## Benchmark results **Environment**: JDK 25.0.3 (Temurin), OpenJDK 64-Bit Server VM, JMH 1.37, Linux x86_64. | Component | Avg Improvement | Range | |---|---:|---| | DELTA_BINARY_PACKED Decoding (INT32/INT64) | **+27.4%** | +16.9% to +43.4% | | Long Delta Decoding (INT64, 5 bit-width patterns) | **+22.9%** | +3.3% to +35.9% | | DELTA_BYTE_ARRAY Decoding (BINARY/FLBA) | **+31.4%** | +8.4% to +134.8% | | DELTA_BINARY_PACKED Encoding | **+6.6%** | +3.3% to +12.4% | | DELTA_BYTE_ARRAY Encoding | **+5.3%** | -0.6% to +16.0% | | DELTA_LENGTH_BYTE_ARRAY Encoding | **+2.6%** | -1.0% to +7.6% | | DELTA_LENGTH_BYTE_ARRAY Decoding | **+1.9%** | +0.0% to +3.8% | Key observations: - DELTA_BYTE_ARRAY SORTED decoding shows the largest gains (+27% to +135%) because `ByteArraySliceOutputStream` eliminates per-value suffix `byte[]` allocation. - TIMESTAMP_MILLIS (a common real-world INT64 delta pattern) improved +28%. - Encoding improvements are more modest since the write path was already relatively lean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
