[
https://issues.apache.org/jira/browse/HBASE-24250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17097774#comment-17097774
]
Huaxiang Sun commented on HBASE-24250:
--------------------------------------
[~niuyulin] and [~timoha], I spent some time on this issue today.
To prevent CJ from submitting multiple GCMultipleMergedRegionsProcedure, we can
maintain a in-memory hashset for the child region. Pseudo code as follows, it
does not handle cases like master switchover nicely as the in-memory hashset is
lost, but should be good enough to reduce most duplicate
GCMultipleMergedRegionsProcedures.
For split case, it can check the regionMap to prevent duplicate procedure
submissions.
What do you think?
{code:java}
HashSet<RegionInfo> newOngoingMergeProcedures = new HashSet<>();
for (Map.Entry<RegionInfo, Result> e : mergedRegions.entrySet()) {
if (this.services.isInMaintenanceMode()) {
// Stop cleaning if the master is in maintenance mode
break;
}
// New pseudo code
newOngoingMergeProcedures.add(e.getKey());
if (this.ongoingMergeProcedures.contains(e.getValue)) {
continue;
}
List<RegionInfo> parents =
MetaTableAccessor.getMergeRegions(e.getValue().rawCells());
if (parents != null && cleanMergeRegion(e.getKey(), parents)) {
gcs++;
}
}
// New pseudo code
this.ongoingMergeProcedures = newOngoingMergeProcedures;{code}
> CatalogJanitor resubmits GCMultipleMergedRegionsProcedure for the same region
> -----------------------------------------------------------------------------
>
> Key: HBASE-24250
> URL: https://issues.apache.org/jira/browse/HBASE-24250
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 2.2.4
> Environment: hdfs 3.1.3 with erasure coding
> hbase 2.2.4
> Reporter: Andrey Elenskiy
> Assignee: niuyulin
> Priority: Major
>
> If a lot of regions were merged (due to change of region sizes, for example),
> there can be a long backlog of procedures to clean up the merged regions. If
> going through this backlog is slower than the CatalogJanitor's scan interval,
> it will end resubmitting GCMultipleMergedRegionsProcedure for the same
> regions over and over again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
