NihalJain commented on PR #131:
URL:
https://github.com/apache/hbase-operator-tools/pull/131#issuecomment-1636839483
[Following was done to test the code changes:
- Build the repo with HBase 2.5.x
- Print help to see it works as expected
- Create a table with 7070 regions and do full hbase meta repair by dropping
meta table data from hdfs
- Tested the newly added functionality against a 2.5.x cluster and
verified assigns is able to pickup the output
- Verified expected number of files are created as per input and data
Truncated test logs for reference:
```
$ export HBCK_JAR=hbase-hbck2-1.3.0-SNAPSHOT.jar
$ hbase hbck -j $HBCK_JAR
usage: HBCK2 [OPTIONS] COMMAND <ARGS>
Options:
-d,--debug run with debug output
-h,--help output this help message
-p,--hbase.zookeeper.property.clientPort <arg> port of hbase ensemble
-q,--hbase.zookeeper.quorum <arg> hbase ensemble
-s,--skip skip hbase version check
(PleaseHoldException)
-v,--version this hbck2 version
-z,--zookeeper.znode.parent <arg> parent znode of hbase
ensemble
Command:
addFsRegionsMissingInMeta [OPTIONS]
[<NAMESPACE|NAMESPACE:TABLENAME>...|-i <INPUTFILES>...]
Options:
-i,--inputFiles take one or more files of namespace or table names
-o,--outputFile name/prefix of the file(s) to dump region names
-n,--numLines number of lines to be written to each output file
To be used when regions missing from hbase:meta but directories
are present still in HDFS. Can happen if user has run _hbck1_
'OfflineMetaRepair' against an hbase-2.x cluster. Needs hbase:meta
to be online. For each table name passed as parameter, performs diff
between regions available in hbase:meta and region dirs on HDFS.
Then for dirs with no hbase:meta matches, it reads the 'regioninfo'
metadata file and re-creates given region in hbase:meta. Regions are
re-created in 'CLOSED' state in the hbase:meta table, but not in the
Masters' cache, and they are not assigned either. To get these
regions online, run the HBCK2 'assigns'command printed when this
command-run completes.
NOTE: If using hbase releases older than 2.3.0, a rolling restart of
HMasters is needed prior to executing the set of 'assigns' output.
An example adding missing regions for tables 'tbl_1' in the default
namespace, 'tbl_2' in namespace 'n1' and for all tables from
namespace 'n2':
$ HBCK2 addFsRegionsMissingInMeta default:tbl_1 n1:tbl_2 n2
Returns HBCK2 an 'assigns' command with all re-inserted regions.
SEE ALSO: reportMissingRegionsInMeta
SEE ALSO: fixMeta
If -i or --inputFiles is specified, pass one or more input file names.
Each file contains <NAMESPACE|NAMESPACE:TABLENAME>, one per line.
For example:
$ HBCK2 addFsRegionsMissingInMeta -i fileName1 fileName2
If -o or --outputFile is specified, the output file(s) can be passed as
input to assigns command via -i or -inputFiles option.
If -n or --numLines is specified, and say it is set to 100, this will
create files with prefix as value passed by -o or --outputFile option.
Each file will have 100 region names (max.), one per line.
For example:
$ HBCK2 addFsRegionsMissingInMeta -i fileName1 fileName2
-o outputFilePrefix -n 100
But if -n is not specified, but -o is specified, it will dump all
region names in a single file, one per line.
For example:
$ HBCK2 addFsRegionsMissingInMeta -i fileName1 fileName2
-o outputFileName
assigns [OPTIONS] [<ENCODED_REGIONNAME>...|-i <INPUT_FILE>...]
.
.
$ cat namesapce.txt
hbase:namespace
$ cat acl.txt
hbase:acl
hbase hbck -j $HBCK_JAR addFsRegionsMissingInMeta -i namesapce.txt acl.txt
-o test/system_regions.txt
Dumping region names to file: /tmp/test/system_regions.txt
Regions re-added into Meta: 2
WARNING:
2 regions were added to META, but these are not yet on Masters cache.
You need to restart Masters, then run hbck2 'assigns' command below:
assigns 708ade6cca88d7a2b8d7e8b045b1c715
8343b51276a70f081f6574c1a6cb5296
$ cat test/system_regions.txt
708ade6cca88d7a2b8d7e8b045b1c715
8343b51276a70f081f6574c1a6cb5296
$ hbase hbck -j $HBCK_JAR assigns -i test/system_regions.txt
[4, 5]
$ hbase hbck -j $HBCK_JAR addFsRegionsMissingInMeta -o test/user_regions.txt
-n 1000 default
Dumping region names to file: /tmp/test/user_regions.txt.0
Dumping region names to file: /tmp/test/user_regions.txt.1
Dumping region names to file: /tmp/test/user_regions.txt.2
Dumping region names to file: /tmp/test/user_regions.txt.3
Dumping region names to file: /tmp/test/user_regions.txt.4
Dumping region names to file: /tmp/test/user_regions.txt.5
Dumping region names to file: /tmp/test/user_regions.txt.6
Dumping region names to file: /tmp/test/user_regions.txt.7
Regions re-added into Meta: 7070
WARNING:
7070 regions were added to META, but these are not yet on Masters cache.
You need to restart Masters, then run hbck2 'assigns' command below:
assigns 000c83ba37b585bf2870af11d9b8acce
00117ed0e16e098d0aa9df14461756b0 001295576c21b98ec63b99149f54497e . . .
$ ls test/user_regions.txt.*
test/user_regions.txt.0 test/user_regions.txt.2 test/user_regions.txt.4
test/user_regions.txt.6
test/user_regions.txt.1 test/user_regions.txt.3 test/user_regions.txt.5
test/user_regions.txt.7
$ hbase hbck -j $HBCK_JAR assigns -i test/user_regions.txt.0
[8, 9, 10, . . . ]
$ hbase hbck -j $HBCK_JAR assigns -i test/user_regions.txt.1
[2008, 2009, 2010, . . . ]
$ hbase hbck -j $HBCK_JAR assigns -i test/user_regions.txt.2
[4008, 4009, 4010, . . . ]
$ hbase hbck -j $HBCK_JAR assigns -i test/user_regions.txt.3
[6008, 6009, 6010, . . . ]
$ hbase hbck -j $HBCK_JAR assigns -i test/user_regions.txt.4
[8008, 8009, 8010, . . . ]
$ hbase hbck -j $HBCK_JAR assigns -i test/user_regions.txt.5
[10008, 10009, 10010, . . . ]
$ hbase hbck -j $HBCK_JAR assigns -i test/user_regions.txt.6
[12008, 12009, 12010, . . . ]
$ hbase hbck -j $HBCK_JAR assigns -i test/user_regions.txt.7
[14008, 14009, 14010, . . . ]
$
```
Full test logs for reference:
[test_HBASE-27724.txt](https://github.com/apache/hbase-operator-tools/files/12062387/test_HBASE-27724.txt)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
