Hi all,
Gorilla is the default encoding for FLOAT and DOUBLE values in TsFile. Floating-point columns are also common in TsFileDataFrame workloads, so the efficiency of Gorilla decoding can directly affect read performance in these scenarios. Compared with fixed-width encodings, Gorilla is less SIMD-friendly because decoding a single stream depends on the previous value and the stored leading/trailing-zero state. However, profiling showed that there was still considerable room for implementation-level optimization. In the initial profile, about 81.8% of the samples were spent on control-bit parsing and variable-width bit reads. The float and double batch paths also decoded values through temporary integer buffers, introducing an additional conversion and copy step. The main optimizations are: - Use a 64-bit bit reservoir and refill up to eight encoded bytes at a time. - Add a fast path for variable-width reads contained in the current reservoir. - Decode float and double batches directly into their output buffers. - Preserve prefetched state across batch, scalar, and skip operations. - Improve handling of truncated input, invalid metadata, and 64-bit bit operations. The on-disk Gorilla format remains unchanged, so existing encoded files remain compatible. I benchmarked the implementation on ARM64 macOS using a Release build with -O3 and LTO. The dataset contained 500,000 smoothly changing floating-point values, and the results below are the median of three runs: - float batch decoding: 3.04 → 1.73 ns/value, about 1.76x faster - double batch decoding: 5.43 → 2.12 ns/value, about 2.57x faster For validation, all 23 Gorilla-related tests and all 716 C++ tests passed. The implementation is available in the following PR: https://github.com/apache/tsfile/pull/873 Reviews and suggestions on the implementation, portability, and benchmark coverage are welcome. Thanks, Colin
