apurtell edited a comment on pull request #3244:
URL: https://github.com/apache/hbase/pull/3244#issuecomment-840922267
**Microbrenchmark Results**
Site configuration used:
<!-- retain all WALs -->
<property>
<name>hbase.master.logcleaner.ttl</name>
<value>604800000</value>
</property>
<!-- enable compression -->
<property>
<name>hbase.regionserver.wal.enablecompression</name>
<value>true</value>
</property>
<!-- enable value compression -->
<property>
<name>hbase.regionserver.wal.value.enablecompression</name>
<value>true</value>
</property>
<!-- set value compression algorithm —>
<property>
<name>hbase.regionserver.wal.value.compression.type</name>
<value>snappy</value>
</property>
Loader: IntegrationTestLoadCommonCrawl
Input:
s3n://commoncrawl/crawl-data/CC-MAIN-2021-10/segments/1614178347293.1/warc/CC-MAIN-20210224165708-20210224195708-00000.warc.gz
Mode | WALs aggregate size | WALs aggregate size difference | WAL writer
append time (ms avg)
-- | -- | -- | --
Default | 5,117,369,553 | - | 0.290 (stdev 0.328)
Compression enabled, value compression not enabled | 5,002,683,600 |
(2.241%) | 0.372 (stddev 0.336)
~~Compression enabled, value compression enabled, v1 patch, Deflate (best
speed)~~ | ~~1,209,947,515~~ | ~~(76.4%)~~ | ~~12.694 (stddev 8.48)~~
Compression enabled, value compression enabled, v2 patch, algorithm=SNAPPY |
1,614,336,180 | (68.5%) | 2.971 (stddev 2.029)
Compression enabled, value compression enabled, v2 patch, algorithm=ZSTD
(best speed) | 1,076,863,466 | (78.95%) | 4.630 (stddev 4.050)
Compression enabled, value compression enabled, v2 patch, algorithm=ZSTD
(default) | 1,075,140,058 | (78.99%) | 6.191 (stddev 4.974)
Compression enabled, value compression enabled, v2 patch, algorithm=ZSTD
(best compression) | 941,505,173 | (81.6%) | 26.322 (stddev 13.38)
Compression enabled, value compression enabled, v2 patch, algorithm=GZ |
1,082,470,144 | (78.8%) | 27.962 (stddev 15.276)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
