[
https://issues.apache.org/jira/browse/HBASE-24250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093918#comment-17093918
]
Andrey Elenskiy commented on HBASE-24250:
-----------------------------------------
Yes, GCMultipleMergedRegionsProcedure and GCRegionProcedure are idempotent.
However, the piling up of those can lead to pretty annoying situation of having
hbasemaster to churn through all of them before proceeding to actually useful
procedures.
For example, in our cluster we ended up merging over 700 regions and then
something when wrong and caused a different procedure to stall (which
unfortunately happens more often then I would like to). As we didn't notice the
issue of stalled procedure right away, we ended up with over 20k
GCMultipleMergedRegionsProcedure in the backlog. It was quite tedious to figure
out why we have so many of those, figure out that we need to disable catalog
janitor, to bypass all of the CG procedures via HBCK2, and then get to actually
fixing the stalled procedure. This caused pretty long downtime for the entire
cluster.
> CatalogJanitor resubmits GCMultipleMergedRegionsProcedure for the same region
> -----------------------------------------------------------------------------
>
> Key: HBASE-24250
> URL: https://issues.apache.org/jira/browse/HBASE-24250
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 2.2.4
> Environment: hdfs 3.1.3 with erasure coding
> hbase 2.2.4
> Reporter: Andrey Elenskiy
> Assignee: niuyulin
> Priority: Major
>
> If a lot of regions were merged (due to change of region sizes, for example),
> there can be a long backlog of procedures to clean up the merged regions. If
> going through this backlog is slower than the CatalogJanitor's scan interval,
> it will end resubmitting GCMultipleMergedRegionsProcedure for the same
> regions over and over again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
