a temporary solution of multi-thread compaction for 0.20.6
----------------------------------------------------------
Key: HBASE-3106
URL: https://issues.apache.org/jira/browse/HBASE-3106
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.20.6
Environment: 1master, 7 region servers, 4 * 7 clients(all clients run
on region server host), sequential put
configuration of each host is:
memory: 16G (region server uses 10G)
disk: 12T
cpu: 8 core
Reporter: andychen
Each region's memtable size is 64M, global memtable limit is 4G on one region
server
In heavy write scenario, especially disable WAL, sequential put fills memtable
to full very quickly, and will generates too many storefiles in a short time.
Then there are more and more compaction tasks put into compactionQueue, single
thread excutes compaction task too slowly, finally forms a vicious cycle.
This will block some region's data importing for a long time
We simplely using ThreadPoolExecutor as multi-thread compaction sulution
(As we know, 0.90 or 0.92 will optimize memflush, compaction, open, close...etc
using multi-thread, but this magnificent version hasn't bee released yet)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
