GitHub user HTHou added a comment to the discussion: Sharing benchmark observations on decimal floating-point compression in IoT time-series: comparing integer mapping with Gorilla/Chimp
Thanks for sharing this work. ALP itself is a credible and interesting direction, but I do not think the current benchmark supports the claimed comparison yet. The main issue is that the benchmark does not appear to compare equivalent workloads: 1. fastalp encodes only a &[f64] and decodes into a preallocated Vec<f64>. For Chimp128 and Gorilla, the benchmark first constructs Point(timestamp, value) objects and calls graupel::decode, which returns newly allocated Vec<Point> objects ([benchmark code](https://github.com/webc-site/wedb_embed/blob/d9f6d7835c6d9b0721e07b53dff075156fca1fb0/fastalp/examples/bench_all_codecs.rs#L269-L349)). 2. Graupel explicitly encodes and decodes both timestamps and values ([codec implementation](https://github.com/jocarrd/graupel/blob/fa8daedfa260504feb253f1963c2a38c0183eac2/src/codec/mod.rs#L124-L178)), while the benchmark calculates raw_bytes using only the size of the f64 values. Consequently, Chimp/Gorilla are charged for timestamp storage and timestamp processing, but those timestamps are excluded from the input-size denominator. This affects both bits/value and throughput. 3. The fastalp path uses preallocated buffers and a stateful Encoder, warmed up 100 times and then reused for 10,000 iterations ([fastalp benchmark](https://github.com/webc-site/wedb_embed/blob/d9f6d7835c6d9b0721e07b53dff075156fca1fb0/fastalp/examples/bench_all_codecs.rs#L75-L116)). This encoder caches parameters between calls. Competing implementations allocate their result vectors on every iteration and run far fewer iterations. Cached and fresh-sampling performance should be reported separately. 4. I could not find a reproducible benchmark target corresponding to the exact table in this discussion, particularly the Patas result. The discussion says batches contain 1,000 values, whereas the current benchmark uses 1,024. A commit hash, exact command, raw output, dataset list and compiler/CPU configuration would be needed to reproduce these numbers. 5. “Zero negative expansion” also seems too strong. The raw fallback still adds a frame header, and the published result data contains cases where 8,192 raw bytes become 8,193 bytes ([example](https://github.com/webc-site/wedb_embed/blob/d9f6d7835c6d9b0721e07b53dff075156fca1fb0/fastalp/benches/json/fastalp.json#L85-L92)). “Bounded expansion” would be more accurate. None of this proves that fastalp is slow—the underlying ALP approach is well established and SIMD-friendly. It means only that the reported advantage over Gorilla/Chimp cannot presently be inferred from this benchmark. A fair comparison should encode the same payload for every codec, use equivalent preallocated or allocating APIs, separate cold parameter discovery from cached encoding, include all framing bytes, and run every implementation through the same benchmark harness. Could you provide such a comparison together with the raw per-dataset results? 感谢分享。ALP 本身是一个可信且值得研究的方向，但我认为目前提供的 benchmark 还不足以支撑文中的横向对比结论。 最主要的问题是，各算法测量的工作负载似乎并不等价： 1. fastalp 只编码 &[f64]，并解码到预分配的 Vec<f64> 中。Chimp128 和 Gorilla 则先构造 Point(timestamp, value)，然后调用返回新分配 Vec<Point> 的 graupel::decode（[benchmark 源码](https://github.com/webc-site/wedb_embed/blob/d9f6d7835c6d9b0721e07b53dff075156fca1fb0/fastalp/examples/bench_all_codecs.rs#L269-L349)）。 2. Graupel 实际同时编解码时间戳和值（[codec 实现](https://github.com/jocarrd/graupel/blob/fa8daedfa260504feb253f1963c2a38c0183eac2/src/codec/mod.rs#L124-L178)），但 benchmark 计算 raw_bytes 时只统计了 f64 值的大小。也就是说，Chimp/Gorilla 承担了时间戳的存储与处理开销，计算压缩比和吞吐量时却没有把时间戳计入原始数据大小。这会同时影响 bits/value 和吞吐量结果。 3. fastalp 使用预分配缓冲区和状态化 Encoder，预热 100 次后复用同一个编码器执行 10,000 次（[fastalp benchmark](https://github.com/webc-site/wedb_embed/blob/d9f6d7835c6d9b0721e07b53dff075156fca1fb0/fastalp/examples/bench_all_codecs.rs#L75-L116)）。该编码器会在调用之间缓存参数，而其他实现每次都重新分配结果容器，迭代次数也少得多。因此，应分别报告从零开始进行参数采样的性能和命中缓存后的性能。 4. 在当前仓库中，我没有找到能够复现本讨论中这张表格的 benchmark，尤其没有找到 Patas 对应的测试实现。讨论中称每批数据包含 1,000 个值，而当前代码使用的是 1,024 个值。若要复现这些数字，至少需要提供对应的 commit、完整命令、原始输出、数据集列表，以及 CPU 和编译器配置。 5. “保证零负压缩”这个说法似乎也不够准确。RAW fallback 仍然需要额外的帧头；已发布的结果中也存在 8,192 字节原始数据变成 8,193 字节的情况（[示例](https://github.com/webc-site/wedb_embed/blob/d9f6d7835c6d9b0721e07b53dff075156fca1fb0/fastalp/benches/json/fastalp.json#L85-L92)）。称为“膨胀量有上限”可能更加准确。 上述问题并不能证明 fastalp 的性能不好——ALP 的基本思路已经得到研究验证，也确实适合向量化。它们只能说明：目前还不能根据这组 benchmark 得出 fastalp 比 Gorilla/Chimp 快这么多的结论。 建议使用完全相同的输入负载，对所有算法采用等价的预分配或动态分配 API，将首次参数探测与缓存命中分别测试，把所有帧头和元数据计入压缩大小，并让全部实现运行在同一个 benchmark 框架中。能否提供这样的对比，以及各数据集的原始测试结果？ GitHub link: https://github.com/apache/iotdb/discussions/18581#discussioncomment-18280135 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
