>>Compaction is no longer a serious problem?
I noticed there a few hundreds of compaction requests in queue. I might take
many disk I/O and lead to less throughput at that time.
>>Can you publish test/hardware configuration and HBase version/config
>>options: Max Heap 64GB, how much was allocated to memstores? What was JVM
>>version/options? What GC did you use?
Sure.
8 SSD (product name is INTEL SSDSC2BB800G4, each disk has around 600GB, some
of the disks had data besides the data for tests) in each node, we have 290G
memory in each node (the OS cache might help in the tests?, we assign 64GB
heap for RS, and use default memstore limit (0.4).
I use G1GC for the test, and use jdk1.8.0_60.
I use hbase 1.0.0 release to do the test.
<property>
<name>hbase.hstore.compaction.throughput.higher.bound</name>
<value>20971520</value>
</property>
<property>
<name>hbase.hstore.compaction.throughput.lower.bound</name>
<value>10485760</value>
</property>
<property>
<name>hbase.regionserver.thread.compaction.small</name>
<value>100</value>
</property>
<property>
<name>hbase.regionserver.thread.compaction.large</name>
<value>5</value>
</property>
<property>
<name>hbase.wal.regiongrouping.numgroups</name>
<value>4</value>
</property>
<property>
<name>hbase.wal.provider</name>
<value>multiwal</value>
</property>
<property>
<name>hbase.hstore.flusher.count</name>
<value>15</value>
</property>
<property>
<name>hbase.ipc.server.read.threadpool.size</name>
<value>50</value>
</property>
<property>
<name>hbase.hstore.blockingStoreFiles</name>
<value>120</value>
</property>
<property>
<name>hbase.regionserver.maxlogs</name>
<value>50</value>
</property>
<property>
<name>hbase.regionserver.metahandler.count</name>
<value>30</value>
</property>
<property>
<name>hbase.regionserver.handler.count</name>
<value>100</value>
</property>
Meanwhile, I use hdfs 2.6.0 and 200 for dfs.datanode.handler.count in hdfs.
<property>
<name>dfs.datanode.handler.count</name>
<value>200</value>
</property>
--
View this message in context:
http://apache-hbase.679495.n3.nabble.com/Multiwal-performance-with-HBase-1-x-tp4074403p4074729.html
Sent from the HBase User mailing list archive at Nabble.com.
