[
https://issues.apache.org/jira/browse/HBASE-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
andychen updated HBASE-3106:
----------------------------
Attachment: multithread-compaction-0.20.6-v2.patch
two modifications:
1. interrupt compaction threads immediately when region server shutdown
2. disable ThreadPoolExecutor automatically adjust the pool size, that means
fix compaction thread number
> a temporary solution of multi-thread compaction for 0.20.6
> ----------------------------------------------------------
>
> Key: HBASE-3106
> URL: https://issues.apache.org/jira/browse/HBASE-3106
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.20.6
> Environment: 1master, 7 region servers, 4 * 7 clients(all clients run
> on region server host), sequential put
> configuration of each host is:
> memory: 16G (region server uses 10G)
> disk: 12T
> cpu: 8 core
> Reporter: andychen
> Attachments: multithread-compaction-0.20.6-v2.patch,
> multithread-compaction-0.20.6.patch
>
>
> Each region's memtable size is 64M, global memtable limit is 4G on one region
> server
> In heavy write scenario, especially disable WAL, sequential put fills
> memtable to full very quickly, and will generates too many storefiles in a
> short time.
> Then there are more and more compaction tasks put into compactionQueue,
> single thread excutes compaction task too slowly, finally forms a vicious
> cycle.
> This will block some region's data importing for a long time
> We simplely using ThreadPoolExecutor as multi-thread compaction sulution
> (As we know, 0.90 or 0.92 will optimize memflush, compaction, open,
> close...etc using multi-thread, but this magnificent version hasn't bee
> released yet)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
