wgtmac commented on PR #38453: URL: https://github.com/apache/arrow/pull/38453#issuecomment-1779467281
> For the record, benchmark numbers here: > > ``` > ReferenceStreamingCompression<Compression::GZIP> 170571860 ns 169802391 ns 4 bytes_per_second=47.1136M/s ratio=6.95102 > ReferenceCompression<Compression::GZIP> 171569422 ns 171399551 ns 4 bytes_per_second=46.6746M/s ratio=6.95102 > ReferenceStreamingDecompression<Compression::GZIP> 12079114 ns 12075244 ns 58 bytes_per_second=662.513M/s ratio=6.95102 > ReferenceDecompression<Compression::GZIP> 11983466 ns 11926139 ns 58 bytes_per_second=670.795M/s ratio=6.95102 > > ReferenceStreamingCompression<Compression::BROTLI> 254898609 ns 253547102 ns 3 bytes_per_second=31.5523M/s ratio=8.31174 > ReferenceCompression<Compression::BROTLI> 258946902 ns 257261268 ns 3 bytes_per_second=31.0968M/s ratio=8.31175 > ReferenceStreamingDecompression<Compression::BROTLI> 8415786 ns 8343701 ns 83 bytes_per_second=958.807M/s ratio=8.31174 > ReferenceDecompression<Compression::BROTLI> 7481414 ns 7443618 ns 94 bytes_per_second=1074.75M/s ratio=8.31175 > > ReferenceStreamingCompression<Compression::ZSTD> 17918513 ns 17833898 ns 39 bytes_per_second=448.584M/s ratio=6.876 > ReferenceCompression<Compression::ZSTD> 16608187 ns 16545738 ns 42 bytes_per_second=483.508M/s ratio=6.8771 > ReferenceStreamingDecompression<Compression::ZSTD> 5673372 ns 5646730 ns 123 bytes_per_second=1.38354G/s ratio=6.876 > ReferenceDecompression<Compression::ZSTD> 5227017 ns 5202190 ns 132 bytes_per_second=1.50177G/s ratio=6.8771 > > ReferenceStreamingCompression<Compression::LZ4_FRAME> 14577950 ns 14570210 ns 47 bytes_per_second=549.066M/s ratio=3.52824 > ReferenceCompression<Compression::LZ4_FRAME> 12065500 ns 12007357 ns 58 bytes_per_second=666.258M/s ratio=3.52824 > ReferenceStreamingDecompression<Compression::LZ4_FRAME> 2008312 ns 1998786 ns 349 bytes_per_second=3.90862G/s ratio=3.52824 > ReferenceDecompression<Compression::LZ4_FRAME> 1983992 ns 1972508 ns 355 bytes_per_second=3.96069G/s ratio=3.52824 > > ReferenceCompression<Compression::LZ4> 11887179 ns 11835535 ns 59 bytes_per_second=675.931M/s ratio=3.53112 > ReferenceDecompression<Compression::LZ4> 1935571 ns 1924247 ns 363 bytes_per_second=4.06003G/s ratio=3.53112 > > ReferenceCompression<Compression::SNAPPY> 11624693 ns 11564830 ns 61 bytes_per_second=691.752M/s ratio=3.58312 > ReferenceDecompression<Compression::SNAPPY> 7049123 ns 7016010 ns 100 bytes_per_second=1.11352G/s ratio=3.58312 > ``` It seems that streaming (de)compression is generally slower than the non-streaming parity. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
