[
https://issues.apache.org/jira/browse/CASSANDRA-18259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17687471#comment-17687471
]
Stefan Miklosovic edited comment on CASSANDRA-18259 at 2/11/23 6:28 PM:
------------------------------------------------------------------------
I run org.apache.cassandra.io.compress.CompressorPerformance test and results
are (I attach old number for comparison)
current library - 1.5.0
{code}
Level FAST
OFF_HEAP->OFF_HEAP compress 0.432 ns/b 2209.955 mb/s uncompress 0.255 ns/b
3738.902 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.392 ns/b 2432.159 mb/s uncompress 0.244 ns/b
3904.887 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.400 ns/b 2384.139 mb/s uncompress 0.236 ns/b
4041.985 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.381 ns/b 2501.406 mb/s uncompress 0.239 ns/b
3990.974 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.384 ns/b 2481.372 mb/s uncompress 0.239 ns/b
3994.512 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.400 ns/b 2386.742 mb/s uncompress 0.232 ns/b
4115.977 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.395 ns/b 2414.109 mb/s uncompress 0.245 ns/b
3893.434 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.402 ns/b 2372.485 mb/s uncompress 0.244 ns/b
3904.035 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.423 ns/b 2255.288 mb/s uncompress 0.240 ns/b
3974.008 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.406 ns/b 2351.665 mb/s uncompress 0.234 ns/b
4080.688 mb/s ratio 1.00:1.
Level DEFAULT
OFF_HEAP->OFF_HEAP compress 12.662 ns/b 75.321 mb/s uncompress 2.217 ns/b
430.154 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.886 ns/b 74.011 mb/s uncompress 2.118 ns/b
450.203 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 13.055 ns/b 73.052 mb/s uncompress 2.150 ns/b
443.623 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.864 ns/b 74.136 mb/s uncompress 2.239 ns/b
426.003 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 13.814 ns/b 69.038 mb/s uncompress 2.269 ns/b
420.311 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 13.010 ns/b 73.305 mb/s uncompress 2.154 ns/b
442.806 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.751 ns/b 74.791 mb/s uncompress 2.170 ns/b
439.518 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.936 ns/b 73.724 mb/s uncompress 2.262 ns/b
421.523 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.683 ns/b 75.196 mb/s uncompress 2.177 ns/b
438.138 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.953 ns/b 73.623 mb/s uncompress 2.182 ns/b
437.060 mb/s ratio 3.16:1.
Level BEST
OFF_HEAP->OFF_HEAP compress 623.386 ns/b 1.530 mb/s uncompress 2.386 ns/b
399.748 mb/s ratio 3.75:1.
OFF_HEAP->OFF_HEAP compress 609.736 ns/b 1.564 mb/s uncompress 2.442 ns/b
390.497 mb/s ratio 3.75:1.
OFF_HEAP->OFF_HEAP compress 610.197 ns/b 1.563 mb/s uncompress 2.065 ns/b
461.740 mb/s ratio 3.75:1.
{code}
new version: 1.5.4-1
{code}
Level FAST
OFF_HEAP->OFF_HEAP compress 0.374 ns/b 2546.834 mb/s uncompress 0.194 ns/b
4915.309 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.311 ns/b 3061.594 mb/s uncompress 0.180 ns/b
5302.387 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.322 ns/b 2966.186 mb/s uncompress 0.176 ns/b
5415.077 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.322 ns/b 2961.015 mb/s uncompress 0.177 ns/b
5375.655 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.307 ns/b 3103.594 mb/s uncompress 0.174 ns/b
5469.394 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.316 ns/b 3020.599 mb/s uncompress 0.173 ns/b
5507.859 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.308 ns/b 3091.532 mb/s uncompress 0.173 ns/b
5502.885 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.306 ns/b 3114.520 mb/s uncompress 0.174 ns/b
5486.201 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.313 ns/b 3048.983 mb/s uncompress 0.175 ns/b
5449.825 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.307 ns/b 3110.122 mb/s uncompress 0.173 ns/b
5507.140 mb/s ratio 1.00:1.
Level DEFAULT
OFF_HEAP->OFF_HEAP compress 9.891 ns/b 96.416 mb/s uncompress 1.588 ns/b
600.599 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 10.699 ns/b 89.140 mb/s uncompress 1.591 ns/b
599.238 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 9.977 ns/b 95.586 mb/s uncompress 1.716 ns/b
555.657 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 9.716 ns/b 98.160 mb/s uncompress 1.566 ns/b
608.948 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 10.348 ns/b 92.164 mb/s uncompress 1.575 ns/b
605.663 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 9.365 ns/b 101.829 mb/s uncompress 1.546 ns/b
617.027 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 9.350 ns/b 102.003 mb/s uncompress 1.555 ns/b
613.406 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 9.335 ns/b 102.157 mb/s uncompress 1.561 ns/b
610.766 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 10.070 ns/b 94.705 mb/s uncompress 1.732 ns/b
550.684 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 10.082 ns/b 94.593 mb/s uncompress 1.582 ns/b
602.773 mb/s ratio 3.18:1.
Level BEST
OFF_HEAP->OFF_HEAP compress 448.884 ns/b 2.125 mb/s uncompress 1.496 ns/b
637.660 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 489.057 ns/b 1.950 mb/s uncompress 1.479 ns/b
645.008 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 445.442 ns/b 2.141 mb/s uncompress 1.492 ns/b
638.989 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 444.666 ns/b 2.145 mb/s uncompress 1.474 ns/b
646.958 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 499.016 ns/b 1.911 mb/s uncompress 2.321 ns/b
410.956 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 480.040 ns/b 1.987 mb/s uncompress 1.502 ns/b
634.960 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 475.358 ns/b 2.006 mb/s uncompress 1.744 ns/b
546.847 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 490.382 ns/b 1.945 mb/s uncompress 1.550 ns/b
615.145 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 487.069 ns/b 1.958 mb/s uncompress 1.618 ns/b
589.323 mb/s ratio 3.74:1.
{code}
There seem to be improvements on all levels.
For DEFAULT level, with old version, we were getting around 75 mb/s for
compression and around 430 mb/s for uncompression.
DEFUALT level, with new version, we are getting around 100 mb/s for compression
and around 600 mb/s for uncompression. So compression is cca 30% faster and
uncompression ~40%.
DEFAULT level has also slighlty better compression ratio.
was (Author: smiklosovic):
I run org.apache.cassandra.io.compress.CompressorPerformance test and results
are (I attach old number for comparison)
current library - 1.5.0
{code}
Level FAST
OFF_HEAP->OFF_HEAP compress 0.432 ns/b 2209.955 mb/s uncompress 0.255 ns/b
3738.902 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.392 ns/b 2432.159 mb/s uncompress 0.244 ns/b
3904.887 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.400 ns/b 2384.139 mb/s uncompress 0.236 ns/b
4041.985 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.381 ns/b 2501.406 mb/s uncompress 0.239 ns/b
3990.974 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.384 ns/b 2481.372 mb/s uncompress 0.239 ns/b
3994.512 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.400 ns/b 2386.742 mb/s uncompress 0.232 ns/b
4115.977 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.395 ns/b 2414.109 mb/s uncompress 0.245 ns/b
3893.434 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.402 ns/b 2372.485 mb/s uncompress 0.244 ns/b
3904.035 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.423 ns/b 2255.288 mb/s uncompress 0.240 ns/b
3974.008 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.406 ns/b 2351.665 mb/s uncompress 0.234 ns/b
4080.688 mb/s ratio 1.00:1.
Level DEFAULT
OFF_HEAP->OFF_HEAP compress 12.662 ns/b 75.321 mb/s uncompress 2.217 ns/b
430.154 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.886 ns/b 74.011 mb/s uncompress 2.118 ns/b
450.203 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 13.055 ns/b 73.052 mb/s uncompress 2.150 ns/b
443.623 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.864 ns/b 74.136 mb/s uncompress 2.239 ns/b
426.003 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 13.814 ns/b 69.038 mb/s uncompress 2.269 ns/b
420.311 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 13.010 ns/b 73.305 mb/s uncompress 2.154 ns/b
442.806 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.751 ns/b 74.791 mb/s uncompress 2.170 ns/b
439.518 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.936 ns/b 73.724 mb/s uncompress 2.262 ns/b
421.523 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.683 ns/b 75.196 mb/s uncompress 2.177 ns/b
438.138 mb/s ratio 3.16:1.
OFF_HEAP->OFF_HEAP compress 12.953 ns/b 73.623 mb/s uncompress 2.182 ns/b
437.060 mb/s ratio 3.16:1.
Level BEST
OFF_HEAP->OFF_HEAP compress 623.386 ns/b 1.530 mb/s uncompress 2.386 ns/b
399.748 mb/s ratio 3.75:1.
OFF_HEAP->OFF_HEAP compress 609.736 ns/b 1.564 mb/s uncompress 2.442 ns/b
390.497 mb/s ratio 3.75:1.
OFF_HEAP->OFF_HEAP compress 610.197 ns/b 1.563 mb/s uncompress 2.065 ns/b
461.740 mb/s ratio 3.75:1.
{code}
new version: 1.5.4-1
{code}
Level FAST
OFF_HEAP->OFF_HEAP compress 0.374 ns/b 2546.834 mb/s uncompress 0.194 ns/b
4915.309 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.311 ns/b 3061.594 mb/s uncompress 0.180 ns/b
5302.387 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.322 ns/b 2966.186 mb/s uncompress 0.176 ns/b
5415.077 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.322 ns/b 2961.015 mb/s uncompress 0.177 ns/b
5375.655 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.307 ns/b 3103.594 mb/s uncompress 0.174 ns/b
5469.394 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.316 ns/b 3020.599 mb/s uncompress 0.173 ns/b
5507.859 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.308 ns/b 3091.532 mb/s uncompress 0.173 ns/b
5502.885 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.306 ns/b 3114.520 mb/s uncompress 0.174 ns/b
5486.201 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.313 ns/b 3048.983 mb/s uncompress 0.175 ns/b
5449.825 mb/s ratio 1.00:1.
OFF_HEAP->OFF_HEAP compress 0.307 ns/b 3110.122 mb/s uncompress 0.173 ns/b
5507.140 mb/s ratio 1.00:1.
Level DEFAULT
OFF_HEAP->OFF_HEAP compress 9.891 ns/b 96.416 mb/s uncompress 1.588 ns/b
600.599 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 10.699 ns/b 89.140 mb/s uncompress 1.591 ns/b
599.238 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 9.977 ns/b 95.586 mb/s uncompress 1.716 ns/b
555.657 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 9.716 ns/b 98.160 mb/s uncompress 1.566 ns/b
608.948 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 10.348 ns/b 92.164 mb/s uncompress 1.575 ns/b
605.663 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 9.365 ns/b 101.829 mb/s uncompress 1.546 ns/b
617.027 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 9.350 ns/b 102.003 mb/s uncompress 1.555 ns/b
613.406 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 9.335 ns/b 102.157 mb/s uncompress 1.561 ns/b
610.766 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 10.070 ns/b 94.705 mb/s uncompress 1.732 ns/b
550.684 mb/s ratio 3.18:1.
OFF_HEAP->OFF_HEAP compress 10.082 ns/b 94.593 mb/s uncompress 1.582 ns/b
602.773 mb/s ratio 3.18:1.
Level BEST
OFF_HEAP->OFF_HEAP compress 448.884 ns/b 2.125 mb/s uncompress 1.496 ns/b
637.660 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 489.057 ns/b 1.950 mb/s uncompress 1.479 ns/b
645.008 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 445.442 ns/b 2.141 mb/s uncompress 1.492 ns/b
638.989 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 444.666 ns/b 2.145 mb/s uncompress 1.474 ns/b
646.958 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 499.016 ns/b 1.911 mb/s uncompress 2.321 ns/b
410.956 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 480.040 ns/b 1.987 mb/s uncompress 1.502 ns/b
634.960 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 475.358 ns/b 2.006 mb/s uncompress 1.744 ns/b
546.847 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 490.382 ns/b 1.945 mb/s uncompress 1.550 ns/b
615.145 mb/s ratio 3.74:1.
OFF_HEAP->OFF_HEAP compress 487.069 ns/b 1.958 mb/s uncompress 1.618 ns/b
589.323 mb/s ratio 3.74:1.
{code}
There seem to be improvements on all levels.
For DEFAULT level, with old version, we were getting around 75 mb/s for
compression and around 430 mb/s for uncompression.
DEFUALT level, with new version, we are getting around 100 mb/s for compression
and around 600 mb/s for uncompression. So compression is cca 30% faster and
uncompression ~40%.
> Upgrade Zstd to 1.5.4
> ---------------------
>
> Key: CASSANDRA-18259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18259
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Compression
> Reporter: Stefan Miklosovic
> Priority: Normal
>
> Zstd 1.5.4 was released yesterday. There is a lot of non-trivial performance
> improvements.
> Upgrade zstd-jni 1.5.0 to 1.5.4.
> (1) https://github.com/facebook/zstd/releases/tag/v1.5.4
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
