[
https://issues.apache.org/jira/browse/HBASE-24250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099419#comment-17099419
]
Huaxiang Sun commented on HBASE-24250:
--------------------------------------
Hi [~niuyulin], I added a unitest case and applied your patch. It still failed.
The reason is that two procedures are being executed at the same time.
{code:java}
index 86971431dc..a50006a5f4 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
@@ -23,6 +23,7 @@ import static org.junit.Assert.assertTrue;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
+import java.util.Map;
import java.util.function.BooleanSupplier;
import org.apache.hadoop.hbase.HBaseClassTestRule;
import org.apache.hadoop.hbase.HBaseTestingUtility;
@@ -31,6 +32,10 @@ import org.apache.hadoop.hbase.MetaTableAccessor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.RegionInfo;
import org.apache.hadoop.hbase.client.RegionInfoBuilder;
+import org.apache.hadoop.hbase.client.Result;
+import
org.apache.hadoop.hbase.master.assignment.GCMultipleMergedRegionsProcedure;
+import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv;
+import org.apache.hadoop.hbase.procedure2.ProcedureExecutor;
import org.apache.hadoop.hbase.testclassification.LargeTests;
import org.apache.hadoop.hbase.testclassification.MasterTests;
import org.apache.hadoop.hbase.util.Threads;
@@ -164,15 +169,34 @@ public class TestMetaFixer {
assertEquals(1, MetaFixer.calculateMerges(10,
report.getOverlaps()).size());
MetaFixer fixer = new MetaFixer(services);
fixer.fixOverlaps(report);
+ CatalogJanitor cj = services.getCatalogJanitor();
await(10, () -> {
try {
- services.getCatalogJanitor().scan();
- final CatalogJanitor.Report postReport =
services.getCatalogJanitor().getLastReport();
- return postReport.isEmpty();
+ if (cj.scan() > 0) {
+ Map<RegionInfo, Result> mergedRegions =
cj.getLastReport().mergedRegions;
+ for (Map.Entry<RegionInfo, Result> e : mergedRegions.entrySet()) {
+ List<RegionInfo> parents =
MetaTableAccessor.getMergeRegions(e.getValue().rawCells());
+ if (parents != null) {
+ ProcedureExecutor<MasterProcedureEnv> pe =
services.getMasterProcedureExecutor();
+ pe.submitProcedure(new
GCMultipleMergedRegionsProcedure(pe.getEnvironment(),
+ e.getKey(), parents));
+ }
+ }
+ return true;
+ }
+ return false;
} catch (Exception e) {
throw new RuntimeException(e);
}
});
+
+ await(10, () -> {
+ return
services.getMasterProcedureExecutor().getActiveProcIds().isEmpty();
+ });
+
+ services.getCatalogJanitor().scan();
+ final CatalogJanitor.Report postReport =
services.getCatalogJanitor().getLastReport();
+ assertTrue(postReport.isEmpty());
}
/**
iMac-Pro:hbase hsun$ {code}
> CatalogJanitor resubmits GCMultipleMergedRegionsProcedure for the same region
> -----------------------------------------------------------------------------
>
> Key: HBASE-24250
> URL: https://issues.apache.org/jira/browse/HBASE-24250
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 2.2.4
> Environment: hdfs 3.1.3 with erasure coding
> hbase 2.2.4
> Reporter: Andrey Elenskiy
> Assignee: niuyulin
> Priority: Major
>
> If a lot of regions were merged (due to change of region sizes, for example),
> there can be a long backlog of procedures to clean up the merged regions. If
> going through this backlog is slower than the CatalogJanitor's scan interval,
> it will end resubmitting GCMultipleMergedRegionsProcedure for the same
> regions over and over again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
