[
https://issues.apache.org/jira/browse/HBASE-23808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066981#comment-17066981
]
Bharath Vissapragada commented on HBASE-23808:
----------------------------------------------
I took an initial stab at it, here is what I see in the logs..
{noformat}
egrep "Sending shutdown RPC|Failure sending shutdown RPC"
TEST-org.apache.hadoop.hbase.master.TestMasterShutdown.xml
2020-03-24 18:38:06,799 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(180): Sending shutdown RPC.
2020-03-24 18:39:55,484 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(186): Failure sending shutdown RPC.
2020-03-24 18:39:56,034 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(180): Sending shutdown RPC.
2020-03-24 18:41:44,621 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(186): Failure sending shutdown RPC.
2020-03-24 18:41:45,155 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(180): Sending shutdown RPC.
2020-03-24 18:43:33,957 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(186): Failure sending shutdown RPC.
2020-03-24 18:43:34,492 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(180): Sending shutdown RPC.
2020-03-24 18:45:23,282 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(186): Failure sending shutdown RPC.
2020-03-24 18:45:23,825 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(180): Sending shutdown RPC.
2020-03-24 18:47:12,608 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(186): Failure sending shutdown RPC.
2020-03-24 18:47:13,151 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(180): Sending shutdown RPC.
2020-03-24 18:49:01,913 DEBUG [ForkJoinPool.commonPool-worker-19]
master.TestMasterShutdown(186): Failure sending shutdown RPC.
{noformat}
Each of the shutdown() RPCs to the master failed. While I see there are some
exceptions on the server side, especially InterruptedExceptions when deleting
the /hbase/master znodes as a part of teardown, all of them are caught and
logged (none of them should be propagated back to the client). Should we
include the exception that the client gets for the shutdown() to see what it is
failing at?
> [Flakey Test]
> TestMasterShutdown#testMasterShutdownBeforeStartingAnyRegionServer
> --------------------------------------------------------------------------------
>
> Key: HBASE-23808
> URL: https://issues.apache.org/jira/browse/HBASE-23808
> Project: HBase
> Issue Type: Test
> Components: test
> Affects Versions: 2.3.0
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.4
>
> Attachments:
> TEST-org.apache.hadoop.hbase.master.TestMasterShutdown.xml
>
>
> Reproduces locally from time to time. Not much to go on here. Looks like the
> test is trying to do some fancy HBase cluster initialization order on top of
> a mini-cluster. Failure seems related to trying to start the HBase master
> before HDFS is fully initialized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
