[
https://issues.apache.org/jira/browse/HBASE-30466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HBASE-30466:
-----------------------------------
Labels: pull-request-available (was: )
> TestSyncReplication* and replication/WAL replay tests do not shut down mini
> clusters after a failed start or a hidden @AfterAll
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-30466
> URL: https://issues.apache.org/jira/browse/HBASE-30466
> Project: HBase
> Issue Type: Test
> Reporter: Jose Luis López
> Priority: Major
> Labels: pull-request-available
>
> On apache/hbase#8743 (large-wave-3), TestSyncReplicationStandbyKillMaster
> failed three times in a row, but only the first failure was real:
> # Run 1 failed in the test body (the flake tracked in HBASE-30344).
> # Rerun 1: {{UTIL1.startMiniCluster}}
> ({{SyncReplicationTestBaseNoBeforeAll.startClusters:111}}) started DFS, then
> the master did not initialise in 200 s, so it threw "IOException: Shutting
> down". DFS stayed up and the util stayed marked running.
> # {{@AfterAll}} called {{shutdown(UTIL1)}}, which returns early when
> {{util.getHBaseCluster() == null}} and so never called
> {{shutdownMiniCluster()}}.
> # Rerun 2 failed at once with "IllegalStateException: A mini-cluster is
> already running"; its log still shows the leaked master writing to the leaked
> DFS.
> h3. Problems
> * {{SyncReplicationTestBaseNoBeforeAll.shutdown(util)}} skips
> {{shutdownMiniCluster()}} exactly in the state a failed start leaves behind.
> * {{tearDown()}} runs admin calls and then shuts down UTIL1, UTIL2 and ZK in
> sequence with no finally, so one failing call leaks the rest. This affects
> all 15 subclasses.
> * {{TestClaimReplicationQueue}} and {{TestRemovePeerProcedureWaitForSCP}}
> declare a static {{tearDownAfterClass()}} that hides
> {{TestReplicationBaseNoBeforeAll.tearDownAfterClass()}} without calling it,
> so their two clusters are never shut down.
> * {{TestAsyncWALReplay}} does the same with
> {{AbstractTestWALReplay.tearDownAfterClass()}}, which affects it and its 3
> subclasses.
> h3. Fix
> Always call {{shutdownMiniCluster()}} in finally, chain UTIL1 -> UTIL2 -> ZK
> through finally, and make the hiding methods call the base teardown in
> finally. Test-only change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
