[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2019-02-01 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19937:
---
Fix Version/s: (was: 1.5.0)

> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.0.0-beta-2, 1.4.2, 2.0.0
>
> Attachments: HBASE-19937.branch-2.001.patch, 
> HBASE-19937.branch-2.002.patch, HBASE-19937.branch-2.003.patch, 
> HBASE-19937.branch-2.004.patch, HBASE-19937.branch-2.005.patch, 
> HBASE-19937.branch-2.006.patch, import-order.png
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master 
> load system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounter Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-09 Thread Andrew Purtell (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-19937:
---
Fix Version/s: 1.4.2
   1.5.0

> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19937.branch-2.001.patch, 
> HBASE-19937.branch-2.002.patch, HBASE-19937.branch-2.003.patch, 
> HBASE-19937.branch-2.004.patch, HBASE-19937.branch-2.005.patch, 
> HBASE-19937.branch-2.006.patch, import-order.png
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master 
> load system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounter Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Ted Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-19937:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.0.0-beta-2
   Status: Resolved  (was: Patch Available)

Thanks for the patch, Xiaolin

> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19937.branch-2.001.patch, 
> HBASE-19937.branch-2.002.patch, HBASE-19937.branch-2.003.patch, 
> HBASE-19937.branch-2.004.patch, HBASE-19937.branch-2.005.patch, 
> HBASE-19937.branch-2.006.patch, import-order.png
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master 
> load system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounter Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Xiaolin Ha (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaolin Ha updated HBASE-19937:
---
Attachment: HBASE-19937.branch-2.006.patch

> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-19937.branch-2.001.patch, 
> HBASE-19937.branch-2.002.patch, HBASE-19937.branch-2.003.patch, 
> HBASE-19937.branch-2.004.patch, HBASE-19937.branch-2.005.patch, 
> HBASE-19937.branch-2.006.patch, import-order.png
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master 
> load system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounter Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> rootMetaFound=true
>  2018-02-02,16:23:17,730 INFO 
> 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Xiaolin Ha (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaolin Ha updated HBASE-19937:
---
Attachment: import-order.png

Can you tell me the right order of imports? My order now is as follows:

​




> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-19937.branch-2.001.patch, 
> HBASE-19937.branch-2.002.patch, HBASE-19937.branch-2.003.patch, 
> HBASE-19937.branch-2.004.patch, HBASE-19937.branch-2.005.patch, 
> import-order.png
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master 
> load system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounter Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> rootMetaFound=true
>  

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Xiaolin Ha (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaolin Ha updated HBASE-19937:
---
Attachment: HBASE-19937.branch-2.005.patch

> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-19937.branch-2.001.patch, 
> HBASE-19937.branch-2.002.patch, HBASE-19937.branch-2.003.patch, 
> HBASE-19937.branch-2.004.patch, HBASE-19937.branch-2.005.patch
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master 
> load system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounter Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> rootMetaFound=true
>  2018-02-02,16:23:17,730 INFO 
> 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Xiaolin Ha (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaolin Ha updated HBASE-19937:
---
Attachment: HBASE-19937.branch-2.004.patch

> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-19937.branch-2.001.patch, 
> HBASE-19937.branch-2.002.patch, HBASE-19937.branch-2.003.patch, 
> HBASE-19937.branch-2.004.patch
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master 
> load system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounter Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> rootMetaFound=true
>  2018-02-02,16:23:17,730 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Xiaolin Ha (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaolin Ha updated HBASE-19937:
---
Attachment: HBASE-19937.branch-2.003.patch

> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-19937.branch-2.001.patch, 
> HBASE-19937.branch-2.002.patch, HBASE-19937.branch-2.003.patch
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master 
> load system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounter Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> rootMetaFound=true
>  2018-02-02,16:23:17,730 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Ted Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-19937:
---
Status: Patch Available  (was: Open)

> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-19937.branch-2.001.patch, 
> HBASE-19937.branch-2.002.patch
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master 
> load system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounter Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> rootMetaFound=true
>  2018-02-02,16:23:17,730 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Xiaolin Ha (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaolin Ha updated HBASE-19937:
---
Description: 
hbase:rsgroup table will be created by calling createRSGroupTable when master 
load system coprocessors in 

 
{code:java}
844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
will encounter Exception as follows,

 
{noformat}
Exception in thread 
"org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
 java.lang.IllegalArgumentException
at 
org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
at 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
at 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
at 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
while ProcedureExecutor  initialized by calling
{code:java}
848  startServiceThreads();{code}
And LoadBalancer is initialized by calling 
{code:java}
868  this.balancer.initialize();{code}
When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
encounters Exception as follows,

2018-02-02,16:12:45,688 ERROR 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
CreateTableProcedure table=hbase:rsgroup
 java.lang.NullPointerException
 at 
org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
 at 
org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
 at 
org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
 at 
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
 at 
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
 at 
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
 at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
 at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
 at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
 at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
 at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)

 

As a result of CreateTableProcedure.rollbackState, it may then print logs 
warning TableExistsException as follows,

2018-02-02,16:12:55,503 WARN 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
Failed to perform check
 java.io.IOException: Failed to create group table. 
org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
 at 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)

 

After some auto-retries, it loops running the thread RSGroupStartupWorker, will 
print logs as follows, 

2018-02-02,16:23:17,626 INFO 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
rootMetaFound=true
 2018-02-02,16:23:17,730 INFO 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
rootMetaFound=true
 2018-02-02,16:23:17,834 INFO 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
rootMetaFound=true
 2018-02-02,16:23:17,937 INFO 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
rootMetaFound=true

 

And using shells of rsgroup, it will tips that currently is in "offline mode".

 

The reason of this problem is that the order of createRSGroupTable and 
initializing of ProcedureExecutor and LoadBalancer is out of control. If the 
former is excuted earlier, it will encounter Exception mentioned before.

 

  was:
rsgroup table will be created by calling 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Xiaolin Ha (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaolin Ha updated HBASE-19937:
---
Description: 
rsgroup table will be created by calling createRSGroupTable when master load 
system coprocessors in 

 
{code:java}
844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
will encounter Exception as follows,

 
{noformat}
Exception in thread 
"org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
 java.lang.IllegalArgumentException
at 
org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
at 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
at 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
at 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
while ProcedureExecutor  initialized by calling
{code:java}
848  startServiceThreads();{code}
And LoadBalancer is initialized by calling 
{code:java}
868  this.balancer.initialize();{code}
When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
encounters Exception as follows,

2018-02-02,16:12:45,688 ERROR 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
CreateTableProcedure table=hbase:rsgroup
 java.lang.NullPointerException
 at 
org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
 at 
org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
 at 
org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
 at 
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
 at 
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
 at 
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
 at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
 at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
 at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
 at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
 at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)

 

As a result of CreateTableProcedure.rollbackState, it may then print logs 
warning TableExistsException as follows,

2018-02-02,16:12:55,503 WARN 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
Failed to perform check
 java.io.IOException: Failed to create group table. 
org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
 at 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)

 

After some auto-retries, it loops running the thread RSGroupStartupWorker, will 
print logs as follows, 

2018-02-02,16:23:17,626 INFO 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
rootMetaFound=true
 2018-02-02,16:23:17,730 INFO 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
rootMetaFound=true
 2018-02-02,16:23:17,834 INFO 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
rootMetaFound=true
 2018-02-02,16:23:17,937 INFO 
org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
rootMetaFound=true

 

And using shells of rsgroup, it will tips that currently is in "offline mode".

 

The reason of this problem is that the order of createRSGroupTable and 
initializing of ProcedureExecutor and LoadBalancer is out of control. If the 
former is excuted earlier, it will encounter Exception mentioned before.

 

  was:
rsgroup table will be created by calling createRSGroupTable 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Xiaolin Ha (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaolin Ha updated HBASE-19937:
---
Attachment: HBASE-19937.branch-2.002.patch

> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-19937.branch-2.001.patch, 
> HBASE-19937.branch-2.002.patch
>
>
> rsgroup table will be created by calling createRSGroupTable when master load 
> system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounters Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> rootMetaFound=true
>  2018-02-02,16:23:17,730 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> 

[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized

2018-02-08 Thread Xiaolin Ha (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaolin Ha updated HBASE-19937:
---
Summary: Ensure createRSGroupTable be called after ProcedureExecutor and 
LoadBalancer are initialized   (was: createRSGroupTable may encounter 
Exceptions before ProcedureExecutor or LoadBalancer is initialized )

> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer 
> are initialized 
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-19937.branch-2.001.patch
>
>
> rsgroup table will be created by calling createRSGroupTable when master load 
> system coprocessors in 
>  
> {code:java}
> 844  this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it 
> will encounters Exception as follows,
>  
> {noformat}
> Exception in thread 
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
>  java.lang.IllegalArgumentException
> at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor  initialized by calling
> {code:java}
> 848  startServiceThreads();{code}
> And LoadBalancer is initialized by calling 
> {code:java}
> 868  this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will 
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught 
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; 
> CreateTableProcedure table=hbase:rsgroup
>  java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
>  at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
>  at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
>  at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
>  at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
>  at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>  
> As a result of CreateTableProcedure.rollbackState, it may then print logs 
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> Failed to perform check
>  java.io.IOException: Failed to create group table. 
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
>  at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>  
> After some auto-retries, it loops running the thread RSGroupStartupWorker, 
> will print logs as follows, 
> 2018-02-02,16:23:17,626 INFO 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: 
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, 
> rootMetaFound=true
>  2018-02-02,16:23:17,730 INFO 
>