[
https://issues.apache.org/jira/browse/HBASE-22567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883529#comment-16883529
]
stack edited comment on HBASE-22567 at 7/12/19 5:10 AM:
--------------------------------------------------------
[~wchevreuil] I added an OfflineMetaRepair to hbck over in HBASE-22680. It is
based on hbck1 implementation trying to exploit already-written code being
selective about what is revealed from hbck1 in hbck2. I see the extant hbck1
tooling as useful implementing not only OfflineMetaRepair but also other items
listed up in HBASE-21745: e.g. 'enumerate store files to determine file level
corruption and sideline corrupt files' and `fix hfile link problems (dangling /
broken)`.
[~busbey] asks in review of HBASE-22680 if overlap with the work here. Let me
review your latest but I don't think so (correct me if I'm wrong). The patch
here works against an online meta selectively making fixes, posing as a "...a
lighter version of 'OfflineMetaRepair'". HBASE-22680 does a brute, wholesale
rewrite of meta w/ the cluster offline.
When I asked above, you said this patch could perhaps evolve to subsume
OfflineMetaRepair. When that is the case, we could remove HBASE-22680 -- or
keep it for when a radical cure is wanted. What you reckon?
On [~daisuke.kobayashi]'s note, that table state is migrated from zk on
startup, my comments above were backed by a misunderstanding on my part (only
table 'state' is repopulated; I thought it more than this). My comments above
probably confuse because of this. Pardon me (D and W). Having been in the code
recently, I was reminded that the mirroring of table state to zk was supposed
to be turned off; the zk table state mirroring is so hbase1 clients can work
against hbase2. So, yeah, was you say above, you probably want to assume
control over what state the table is in and making sure the state is populated.
was (Author: stack):
[~wchevreuil] I added an OfflineMetaRepair to hbck over in HBASE-22680. It is
based on hbck1 implementation trying to exploit already-written code being
selective about what is revealed from hbck1 in hbck2. I see the extant hbck1
tooling as useful implementing not only OfflineMetaRepair but also other items
listed up in HBASE-21745: e.g. 'enumerate store files to determine file level
corruption and sideline corrupt files' and `fix hfile link problems (dangling /
broken)`.
[~busbey] asks in review of HBASE-22680 if overlap with the work here. Let me
review your latest but I don't think so (correct me if I'm wrong). The patch
here works against an online meta selectively posing as a "a lighter version of
'OfflineMetaRepair'". HBASE-22680 does a wholesale rewrite of meta w/ the
cluster offline.
When I asked above, you said this patch could perhaps evolve to subsume
OfflineMetaRepair. When that is the case, we could remove HBASE-22680. What you
reckon?
On [~daisuke.kobayashi]'s note, that table state is migrated from zk on
startup, my comments above were backed by a misunderstanding on my part (only
table 'state' is repopulated; I thought it more than this). My comments above
probably confuse because of this. Having been in the code recently, I was
reminded that the mirroring of table state to zk was supposed to be turned off;
the zk table state mirroring is so hbase1 clients can work against hbase2.
> HBCK2 addMissingRegionsToMeta
> -----------------------------
>
> Key: HBASE-22567
> URL: https://issues.apache.org/jira/browse/HBASE-22567
> Project: HBase
> Issue Type: New Feature
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Major
>
> Following latest discussion on HBASE-21745, this proposes an hbck2 command
> that allows for inserting back regions missing in META that still have
> *regioninfo* available in HDFS. Although this is still an interactive and
> simpler version than the old _OfflineMetaRepair_, it still relies on hdfs
> state as the source of truth, and performs META updates mostly independently
> from Master (apart from requiring Meta table been online).
> For a more detailed explanation on this command behaviour, pasting _command
> usage_ text:
> {noformat}
> To be used for scenarios where some regions may be missing in META,
> but there's still a valid 'regioninfo' metadata file on HDFS.
> This is a lighter version of 'OfflineMetaRepair' tool commonly used for
> similar issues on 1.x release line.
> This command needs META to be online. For each table name passed as
> parameter, it performs a diff between regions available in META,
> against existing regions dirs on HDFS. Then, for region dirs with
> no matches in META, it reads regioninfo metadata file and
> re-creates given region in META. Regions are re-created in 'CLOSED'
> state at META table only, but not in Masters' cache, and are not
> assigned either. A rolling Masters restart, followed by a
> hbck2 'assigns' command with all re-inserted regions is required.
> This hbck2 'assigns' command is printed for user convenience.
> WARNING: To avoid potential region overlapping problems due to ongoing
> splits, this command disables given tables while re-inserting regions.
> An example adding missing regions for tables 'table_1' and 'table_2':
> $ HBCK2 addMissingRegionsInMeta table_1 table_2
> Returns hbck2 'assigns' command with all re-inserted regions.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
