wchevreuil commented on a change in pull request #45: HBASE-23371 [HBCK2]
Provide client side method for removing ghost regions in meta.
URL:
https://github.com/apache/hbase-operator-tools/pull/45#discussion_r356640053
##########
File path:
hbase-hbck2/src/main/java/org/apache/hbase/FsRegionsMetaRecoverer.java
##########
@@ -70,61 +77,207 @@ public FsRegionsMetaRecoverer(Configuration
configuration) throws IOException {
this.fs = fileSystem;
}
- private List<Path> getTableRegionsDirs(String table) throws Exception {
+ private List<Path> getTableRegionsDirs(String table) throws IOException {
String hbaseRoot = this.config.get(HConstants.HBASE_DIR);
Path tableDir = FSUtils.getTableDir(new Path(hbaseRoot),
TableName.valueOf(table));
return FSUtils.getRegionDirs(fs, tableDir);
}
public Map<TableName,List<Path>> reportTablesMissingRegions(final
List<String> namespacesOrTables)
throws IOException {
- final Map<TableName,List<Path>> result = new HashMap<>();
- List<TableName> tableNames =
MetaTableAccessor.getTableStates(this.conn).keySet().stream()
- .filter(tableName -> {
- if(namespacesOrTables==null || namespacesOrTables.isEmpty()){
- return true;
- } else {
- Optional<String> findings = namespacesOrTables.stream().filter(
- name -> (name.indexOf(":") > 0) ?
- tableName.equals(TableName.valueOf(name)) :
- tableName.getNamespaceAsString().equals(name)).findFirst();
- return findings.isPresent();
+ InternalMetaChecker<Path> missingChecker = new InternalMetaChecker<>();
+ return missingChecker.reportTablesRegions(namespacesOrTables,
this::findMissingRegionsInMETA);
+ }
+
+ public Map<TableName,List<RegionInfo>>
+ reportTablesExtraRegions(final List<String> namespacesOrTables) throws
IOException {
+ InternalMetaChecker<RegionInfo> extraChecker = new InternalMetaChecker<>();
+ return extraChecker.reportTablesRegions(namespacesOrTables,
this::findExtraRegionsInMETA);
+ }
+
+ List<Path> findMissingRegionsInMETA(String table) throws IOException {
+ InternalMetaChecker<Path> missingChecker = new InternalMetaChecker<>();
+ return missingChecker.checkRegionsInMETA(table, (regions, dirs) -> {
+ final List<Path> missingRegions = new ArrayList<>();
+ HashSet<String> regionsInMeta = regions.stream().map(info ->
+ info.getEncodedName()).collect(Collectors.toCollection(HashSet::new));
+ dirs.forEach(dir -> {
+ if(!regionsInMeta.contains(dir.getName())){
+ LOG.debug("{} is not in META.", dir);
+ missingRegions.add(dir);
}
- }).collect(Collectors.toList());
- tableNames.stream().forEach(tableName -> {
- try {
- result.put(tableName,
-
findMissingRegionsInMETA(tableName.getNameWithNamespaceInclAsString()));
- } catch (Exception e) {
- LOG.warn("Can't get missing regions from meta", e);
- }
+ });
+ return missingRegions;
+ });
+ }
+
+ List<RegionInfo> findExtraRegionsInMETA(String table) throws IOException {
+ InternalMetaChecker<RegionInfo> extraChecker = new InternalMetaChecker<>();
+ return extraChecker.checkRegionsInMETA(table, (regions,dirs) -> {
Review comment:
Yeah, I guess we could basically move the whole loop to InternalMetaChecker.
Let me work on that and try get it improved in the next commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
