[
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-17292:
---------------------------
Description:
Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded
hfiles.
However, if bulk load fails after hfile is moved to region directory but before
postBulkLoadHFile() hook is called, there is no way for pluggable components
(replication - see HBASE-17290, backup / restore) to know which hfile(s) have
been moved to region directory.
Even if postBulkLoadHFile() is called in finally block, the write (to backup
table or zookeeper) issued from postBulkLoadHFile() may fail, ending up with
same situation.
This issue adds a preCommitStoreFile() hook which notifies path of to be
committed hfile before bulk loaded hfile is moved to region directory.
With preCommitStoreFile() hook, write (to backup table or zookeeper) can be
issued before the movement of hfile.
If write fails, IOException would make bulk load fail, not leaving hfile in
region directory.
was:
Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded
hfiles.
However, if bulk load fails after hfile is moved to region directory but before
postBulkLoadHFile() hook is called, there is no way for pluggable components
(replication - see HBASE-17290, backup / restore) to know which hfile(s) have
been moved to region directory.
This issue adds a preCommitStoreFile() hook which notifies path of to be
committed hfile before bulk loaded hfile is moved to region directory.
> Add observer notification before bulk loaded hfile is moved to region
> directory
> -------------------------------------------------------------------------------
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
> Issue Type: Improvement
> Reporter: Ted Yu
> Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 17292.v1.txt, 17292.v2.txt, 17292.v3.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but
> before postBulkLoadHFile() hook is called, there is no way for pluggable
> components (replication - see HBASE-17290, backup / restore) to know which
> hfile(s) have been moved to region directory.
> Even if postBulkLoadHFile() is called in finally block, the write (to backup
> table or zookeeper) issued from postBulkLoadHFile() may fail, ending up with
> same situation.
> This issue adds a preCommitStoreFile() hook which notifies path of to be
> committed hfile before bulk loaded hfile is moved to region directory.
> With preCommitStoreFile() hook, write (to backup table or zookeeper) can be
> issued before the movement of hfile.
> If write fails, IOException would make bulk load fail, not leaving hfile in
> region directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
