Josh Elser commented on HBASE-20131:

Parking an initial patch here. I have a UT which fails without my change and 
passes with it.

I'm not positive that this is the right fix, but I'm not sure where a proper 
fix would go. The fix I've added will guard against the MoveProcedure being 
submitted when the Region is closed. However, I don't think there's any guard 
against a race-condition where the Region is being closed when the Move comes 

I don't know if this race condition is something we need to explicitly handle, 
or if this already fails gracefully. Would greatly appreciate your opinions, 
mighty [~stack] or some other(s) you might suggest I bother :)

> NPE in MoveRegionProcedure via IntegrationTestLoadAndVerify with CM
> -------------------------------------------------------------------
>                 Key: HBASE-20131
>                 URL: https://issues.apache.org/jira/browse/HBASE-20131
>             Project: HBase
>          Issue Type: Bug
>          Components: proc-v2
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Critical
>             Fix For: 2.0.0
>         Attachments: HBASE-20131.001.patch
> I believe the error is that a MoveRegionProcedure comes in via ChaosMonkey 
> for an unassigned region that was from a disabled table (also due to CM) 
> which causes an NPE as we try to set a null original location into the 
> protobuf which fails.
> {noformat}
> 2018-03-02 23:07:00,146 ERROR 
> [RpcServer.default.FPBQ.Fifo.handler=23,queue=2,port=20000] ipc.RpcServer: 
> Unexpected throwable object 
> java.lang.NullPointerException
>       at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos$MoveRegionStateData$Builder.setSourceServer(MasterProcedureProtos.java:26127)
>       at 
> org.apache.hadoop.hbase.master.assignment.MoveRegionProcedure.serializeStateData(MoveRegionProcedure.java:133)
>       at 
> org.apache.hadoop.hbase.procedure2.ProcedureUtil.convertToProtoProcedure(ProcedureUtil.java:198)
>       at 
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeEntry(ProcedureWALFormat.java:211)
>       at 
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeInsert(ProcedureWALFormat.java:222)
>       at 
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.insert(WALProcedureStore.java:490)
>       at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:863)
>       at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:832)
>       at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitProcedure(ProcedureSyncWait.java:111)
>       at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.moveAsync(AssignmentManager.java:561)
>       at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1707)
>       at 
> org.apache.hadoop.hbase.master.MasterRpcServices.moveRegion(MasterRpcServices.java:1324)
>       at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
>       at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
>       at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
>       at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>       at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){noformat}
> IntegrationTestLoadAndVerify also failed, but I'm not sure if it's related to 
> this, or just a problem with the test. The test failed because the table was 
> left offline after it was disabled, and appears to not have been re-enabled. 
> Still debugging that side..

This message was sent by Atlassian JIRA

Reply via email to