[
https://issues.apache.org/jira/browse/HBASE-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14693166#comment-14693166
]
Enis Soztutar commented on HBASE-11368:
---------------------------------------
My concern with the patch is that it is acquiring yet another lock per get/scan
on top of the already existing ones. Agreed that the region close lock is
abused here for multi-CF bulkloads and have to be fixed.
I believe the actual long term solution to this is to do ref-counting to Store
files in the store, and have the store file list per scan immutable. Then we do
not need the costly mechanism for keeping the store files updated between
KVHea, scanner and store file list ({{notifyChangedReadersObservers}}). leveldb
is doing ref counting for files I believe. [~lhofhansl] you had a jira for
this?
> Multi-column family BulkLoad fails if compactions go on too long
> ----------------------------------------------------------------
>
> Key: HBASE-11368
> URL: https://issues.apache.org/jira/browse/HBASE-11368
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Qiang Tian
> Attachments: hbase-11368-0.98.5.patch, hbase11368-master.patch,
> key_stacktrace_hbase10882.TXT, performance_improvement_verification_98.5.patch
>
>
> Compactions take a read lock. If a multi-column family region, before bulk
> loading, we want to take a write lock on the region. If the compaction takes
> too long, the bulk load fails.
> Various recipes include:
> + Making smaller regions (lame)
> + [~victorunique] suggests major compacting just before bulk loading over in
> HBASE-10882 as a work around.
> Does the compaction need a read lock for that long? Does the bulk load need
> a full write lock when multiple column families? Can we fail more gracefully
> at least?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
