richardstartin commented on issue #7973:
URL: https://github.com/apache/pinot/issues/7973#issuecomment-1013526820
Here are the sizes of 8KB (1024 doubles) of different distributions/patterns
with Snappy and LZ4. There are encodings which can be introduced to reduce the
size of metric columns (e.g. xor or delta encoding) but making it possible to
compress metric column with general purpose compression algorithms isn't in the
user's interest.
|Compression|Distribution |Compressed Size (KB)|
|-----------|-----------------------------|--------------------|
|Uncompressed|integer increments |8.00 |
|LZ4 |integer increments |4.09 |
|Snappy |integer increments |4.02 |
|Uncompressed|noisy increments |8.00 |
|LZ4 |noisy increments |8.03 |
|Snappy |noisy increments |8.00 |
|Uncompressed|sinusoidal |8.00 |
|LZ4 |sinusoidal |8.03 |
|Snappy |sinusoidal |8.00 |
|Uncompressed|normal(0,1) |8.00
|
|LZ4 |normal(0,1) |8.03
|
|Snappy |normal(0,1) |8.00
|
|Uncompressed|exp(0.999) |8.00 |
|
|LZ4 |exp(0.999) |7.23 |
|
|Snappy |exp(0.999) |7.16 |
|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
