[
https://issues.apache.org/jira/browse/HBASE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13759632#comment-13759632
]
Richard Ding commented on HBASE-9426:
-------------------------------------
Hi John,
The background is that we are implementing incremental backups using WAL files.
We need 'distributed log roll' between the backups to make sure all WAL files
are closed and can be copied.
During the implementation of 'distributed log roll', we realized that we had to
add standard code to the master and region servers to enable the procedure
(mainly to region servers, the procedures can be invoked directly from client).
And anyone wants to add a new procedure has to add similar life-cycle code
(init/start/stop/exec) to HBase core in order to use it.
Currently the table flush from HBaseAdmin is not distributed (and potentially
can take long time). I think the globally barriered procedure is a good
mechanism to implement a distributed table flush. But again I need to add
init/start/stop code to the region server.
In general, the globally barriered procedure is a useful framework for
implementing distributed, coordinated functions on region servers. I think it
would be good if we can invoke these custom procedures without having to modify
the HBase core.
You are right that the idea is to have an an hbase extension mechnism for 3rd
party code a la coprocessors. Since only the life-cycle management code
(init/start/stop) involved, it would still work if the procedure is changed to
region-centric or table-centric.
> Make custom distributed barrier procedure pluggable
> ----------------------------------------------------
>
> Key: HBASE-9426
> URL: https://issues.apache.org/jira/browse/HBASE-9426
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.95.2, 0.94.11
> Reporter: Richard Ding
> Assignee: Richard Ding
>
> Currently if one wants to implement a custom distributed barrier procedure
> (e.g., distributed log roll or distributed table flush), the HBase core code
> needs to be modified in order for the procedure to work.
> Looking into the snapshot code (especially on region server side), most of
> the code to enable the procedure are generic life-cycle management (i.e.,
> init, start, stop). We can make this part pluggable.
> Here is the proposal. Following the coprocessor example, we define two
> properties:
> {code}
> hbase.procedure.regionserver.classes
> hbase.procedure.master.classes
> {code}
> The values for both are comma delimited list of classes. On region server
> side, the classes implements the following interface:
> {code}
> public interface RegionServerProcedureManager {
> public void initialize(RegionServerServices rss) throws KeeperException;
> public void start();
> public void stop(boolean force) throws IOException;
> public String getProcedureName();
> }
> {code}
> While on Master side, the classes implement the interface:
> {code}
> public interface MasterProcedureManager {
> public void initialize(MasterServices master) throws KeeperException,
> IOException, UnsupportedOperationException;
> public void stop(String why);
> public String getProcedureName();
> public void execProcedure(ProcedureDescription desc) throws IOException;
> IOException;
> }
> {code}
> Where the ProcedureDescription is defined as
> {code}
> message ProcedureDescription {
> required string name = 1;
> required string instance = 2;
> optional int64 creationTime = 3 [default = 0];
> message Property {
> required string tag = 1;
> optional string value = 2;
> }
> repeated Property props = 4;
> }
> {code}
> A generic API can be defined on HMaster to trigger a procedure:
> {code}
> public boolean execProcedure(ProcedureDescription desc) throws IOException;
> {code}
> _SnapshotManager_ and _RegionServerSnapshotManager_ are special examples of
> _MasterProcedureManager_ and _RegionServerProcedureManager_. They will be
> automatically included (users don't need to specify them in the conf file).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
