[
https://issues.apache.org/jira/browse/HBASE-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696724#comment-14696724
]
Enis Soztutar commented on HBASE-11368:
---------------------------------------
bq. How? Would the following case be true without the bulk load getting the
region write lock?
We do not do this now, but in theory it can be done similarly to regular
writes.
- obtain new seqId as a write transaction
- bulk load all files across CFs with the seqId.
- advance mvcc read point only when all bulk loads are complete.
This way the scanners are guaranteed to atomically observe the bulk loaded data
atomically without the region-write-lock.
bq. In the 0.98 code line, we don't have seqid, and the atomicity is still
guaranteed there.
Yes. Not worth changing 0.98 line.
bq. I think it is being propagated properly to the scanner. Think about the
same notifyChangedReadersObservers is being used at the end of compaction and
flushes as well. The reset of the readers should work.
I am not sure about that. Agreed that the cells at the store level will
actually get re-ordered, but the heap at the region level is never re-ordered.
So, after a bulk load, the ordering of store scanners at the region level might
change, but the scanner will miss it if I understand this correctly.
bq. Atomicity may be a false blanket considering HBASE-4652 is still unresolved.
Very good point. We need a transactional commit for the BL files.
> Multi-column family BulkLoad fails if compactions go on too long
> ----------------------------------------------------------------
>
> Key: HBASE-11368
> URL: https://issues.apache.org/jira/browse/HBASE-11368
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Qiang Tian
> Attachments: hbase-11368-0.98.5.patch, hbase11368-master.patch,
> key_stacktrace_hbase10882.TXT, performance_improvement_verification_98.5.patch
>
>
> Compactions take a read lock. If a multi-column family region, before bulk
> loading, we want to take a write lock on the region. If the compaction takes
> too long, the bulk load fails.
> Various recipes include:
> + Making smaller regions (lame)
> + [~victorunique] suggests major compacting just before bulk loading over in
> HBASE-10882 as a work around.
> Does the compaction need a read lock for that long? Does the bulk load need
> a full write lock when multiple column families? Can we fail more gracefully
> at least?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
