[
https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791957#comment-15791957
]
stack commented on HBASE-14614:
-------------------------------
Pushed first cut at core assignment made from patches pulled from
[~mbertozzi]'s repo (as per his guidance).
Here's the change log:
{code}
HBASE-14614 Procedure v2 - Core Assignment Manager (Matteo Bertozzi)
Below are commits from Matteo's repo adding in core AM.
Adds running of remote procedure. Adds batching of remote calls.
Adds support for assign/unassign in procedures. Adds version info
reporting in rpc. Adds start of an AMv2.
First is from
https://github.com/matteobertozzi/hbase/commit/3622cba4e331d2fc7bfc1932abb4c9cbf5802efa
Also adding in RS version info from
https://github.com/matteobertozzi/hbase/commit/ddb4df3964e8298c88c0210e83493aa91ac0942d.patch
And remote dispath
https://github.com/matteobertozzi/hbase/commit/186b9e7c4dae61a79509a6c3aad7f80ec61345e5
A
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
Dispatch remote procedures every 150ms or 32 items -- which ever
happens first (configurable). Runs a timeout thread.
Carries notion of a remote procedure and of a buffer full of these.
M hbase-protocol-shaded/src/main/protobuf/Admin.proto
b/hbase-protocol-shaded/src/main/protobuf/Admin.proto
Add execute procedures call.
M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Adds assign and unassign support.
M
hbase-server/src/main/java/org/apache/hadoop/hbase/client/VersionInfoUtil.java
Adds getting RS version out of RPC
Examples: (1.3.4 is 0x0103004, 2.1.0 is 0x0201000)
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Add start/stop of remote precedure engine. Add reference to AM2.
M
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
Extract version number of the server making rpc.
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
Add new assign procedure.
There can only be one RegionTransitionProcedure per region running at the
time,
since each procedure takes a lock on the region.
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
A procedure-based AM (AMv2).
TODO
- handle region migration
- handle meta assignment first
- handle sys table assignment first (e.g. acl, namespace)
- handle table priorities
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
Adds new move region procedure.
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
Store region state (in hbase:meta by default).
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
In-memory state of all regions.
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Base RIT procedure.
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
Unassign procedure.
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
Run region assignement in a manner that pays attention to target
server version.
{code}
> Procedure v2: Core Assignment Manager
> -------------------------------------
>
> Key: HBASE-14614
> URL: https://issues.apache.org/jira/browse/HBASE-14614
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
> Affects Versions: 2.0.0
> Reporter: Stephen Yuan Jiang
> Assignee: Matteo Bertozzi
> Fix For: 2.0.0
>
> Attachments: HBASE-14614.master.001.patch
>
>
> New AssignmentManager implemented using proc-v2.
> - AssignProcedure handle assignment operation
> - UnassignProcedure handle unassign operation
> - MoveRegionProcedure handle move/balance operation
> Concurrent Assign operations are batched together and sent to the balancer
> Concurrent Assign and Unassign operation ready to be sent to the RS are
> batched together
> This patch is an intermediate state where we add the new AM as
> AssignmentManager2() to the master, to be reached by tests. but the new AM
> will not be integrated with the rest of the system. Only new am unit-tests
> will exercise the new assigment manager. The integration with the master code
> is part of HBASE-14616
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
