bulk loading stuff works with hfiles. recovered.edits files are formatted the same as WAL files rather than as HFiles. for wal files you can use the wal replayer to ensure those edits are all present in the table.
IIRC there is an unknown sequence of events that can result in the recovered edits sticking around for a region after they've already been recovered. Presuming your use case will work for having the same edit played multiple times (basically if you do not mess about with cell level timestamps or keeping multiple versions around) then it should be fine to sideline those edits and then replay them using the wal player. If your use case isn't fine with that, then you can use the wal pretty printer to examine the edits that are there and check to ensure the cells are already in the table in a current region. sounds like we should update the troubleshooting tips to include some coverage of stray recovered.edits files. On Tue, Sep 22, 2020 at 8:58 AM Martin Braun <[email protected]> wrote: > > Hello all, > > I have an issue with hbase 2.2.5 (and hadoop-2.8.5) after a full disk event I > have 38 inconsistencies, when I do a > > hbase --internal-classpath hbck > > I get a bunch of these errors: > > ERROR: Orphan region in HDFS: Unable to load .regioninfo from table > tt_ix_bizStep_inserting in hdfs dir > hdfs://localhost:9000/hbase/data/default/tt_ix_bizStep_inserting/8a1acb499bf454b072daeee5960daa73! > It may be an invalid format or version file. Treating as an orphaned > regiondir. > ERROR: Orphan region in HDFS: Unable to load .regioninfo from table > tt_ix_bizStep_inserting in hdfs dir > hdfs://localhost:9000/hbase/data/default/tt_ix_bizStep_inserting/8f64025b68958ebddeb812297facdfc6! > It may be an invalid format or version file. Treating as an orphaned > regiondir. > > > When looking into these directories I see that there is indeed no .regioninfo > file: > > hdfs dfs -ls -R > hdfs://localhost:9000/hbase/data/default/tt_ix_parent_item/ae1553c4d6140110c51c535ba1dbc1a0 > > drwxr-xr-x - jenkins supergroup 0 2020-09-21 11:23 > hdfs://localhost:9000/hbase/data/default/tt_ix_parent_item/ae1553c4d6140110c51c535ba1dbc1a0/recovered.edits > -rw-r--r-- 3 jenkins supergroup 74133 2020-09-21 11:11 > hdfs://localhost:9000/hbase/data/default/tt_ix_parent_item/ae1553c4d6140110c51c535ba1dbc1a0/recovered.edits/0000000000000000285 > -rw-r--r-- 3 jenkins supergroup 74413 2020-09-16 19:03 > hdfs://localhost:9000/hbase/data/default/tt_ix_parent_item/ae1553c4d6140110c51c535ba1dbc1a0/recovered.edits/0000000000000000286 > -rw-r--r-- 3 jenkins supergroup 74693 2020-09-16 19:05 > hdfs://localhost:9000/hbase/data/default/tt_ix_parent_item/ae1553c4d6140110c51c535ba1dbc1a0/recovered.edits/0000000000000000287 > -rw-r--r-- 3 jenkins supergroup 79427 2020-09-16 18:27 > hdfs://localhost:9000/hbase/data/default/tt_ix_parent_item/ae1553c4d6140110c51c535ba1dbc1a0/recovered.edits/0000000000000000305 > > > The hbck2 manual from the hbase-operator tools tells me for Orphan Data to > read http://hbase.apache.org/book.html#arch.bulk.load.complete.strays, > chapter “72.4.1. 'Adopting' Stray Data" > > However it seems that this is another case a completebuldload on the named > directories seems to do nothing… > > A scan 'hbase:meta', {COLUMN=>'info:regioninfo’} does not show any errors. > > > How can I resolve these inconsistencies of the missing .regioninfo? > > TIA > > best, > Martin > -- Sean
