saintstack opened a new pull request #685: HBASE-23106 [HBCK2/hbase-operator-tools] Add a WAL verifier URL: https://github.com/apache/hbase/pull/685 Adds simple WALReader. Here is example of what it looks like when it runs: ``` kalashnikov:hbase.apache.git stack$ ./bin/hbase org.apache.hadoop.hbase.wal.WALReader /tmp/0000000000000932379 2019-10-03 12:28:46,375 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable /tmp/0000000000000932379 3448 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932373 3271 11309 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932374 7767 18565 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932375 7162 34217 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932376 15558 44758 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932377 10447 57357 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932378 12505 60722 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932379 3271 kalashnikov:hbase.apache.git stack$ ./bin/hbase org.apache.hadoop.hbase.wal.WALReader /tmp/0000000000000932380 2019-10-03 12:28:52,110 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable /tmp/0000000000000932380 2019-10-03 12:28:52,568 WARN [main] hbase.KeyValueUtil: 8\x8E\xF8o\xF4\x03"\xD1\x89\xB8\x86\x5C\xECo\x16\x96\x5C,\xC4EH\xACO\x1E\xA4VF\xA7\xFB\xF9\x7F\xAF](\xFC\xE2\x07QQ\xE6\xA8-\xD1l\x0C\xE7\x13YX_[\x89+n&JS \xE5\x81\xF4\xFAH/\xBF\xF4\xBD\x95\x1F\x8F*\x96\x09\xC8Q\xEAoJt2\xA3E'\xA2\x01\xAB\xF2y%B\x18\xBAI)\x02\x0B\x0F\xFBv^\xF4\x85$^>\x14\x84\xA7\xBEb\xFE{N\xEB7W~~\xFB\x5C\x0E=\xD6FG\xB1ms"\x04\x81\xC69\x97i\x16m\xD0\x1F\xBC0iR.\xBB6\xEE\xEF\xDE\x05OF>\x029\x01\xF6\xFBJ\x8D\xA7\xDA\xE8x\xB2\x5C\xED20\x85Z\x09y\x82x\x98\xBAQ~,\x80@\x87r!\xAFa\x8F$\xDB\xC7\x06\x84\xE6\xEAe\xD0\xE2xF\x8E\x9A\x17mtbpb\x0F\xF2\xA4\x13\xBAn\x06j\x86\xFA\x92p\x1F\xBB\x97\xD7WZ\x09\xED(\x15\x8B\xAA\xDE\x09\xB8\xFE\x91\x8C\x0F\x17\x19s\x09\xF6\x95\x8DT\xAC\x0A\x12\xDALq@\xAALB\x8C\xB3\x0A\xCE\xA4k\xA5\x9F\xA25\xDBh.\xA5\xED\xD1(\x17s\x01\xCA\x94Gm\xDB%\x19\x9CI\x9FZw\xB2\xB8\xE9:\x8B\x9B\xD2\xF6\xE0N\x07j\xEDE)\x9C\xBA5"\xA5n\x9D\x088qh/\x85\xE6\xC4\x85\x10\xD9\x0A)=1+\x98\x05\x09\xBE\xCA\x99C\xDD!\x9F\x08\x9D\x94\x0F\xD2\xFA\xD3\xFB\xA6\x91, offset=0, length=1080 2019-10-03 12:28:52,569 WARN [main] wal.ProtobufLogReader: Encountered a malformed edit, seeking back to last good position in file, from 1261 to 83 java.io.EOFException: EOF while reading 12 WAL KVs; started reading at 177 and read up to 1261 at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:397) at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:98) at org.apache.hadoop.hbase.wal.WALReader.read(WALReader.java:41) at org.apache.hadoop.hbase.wal.WALReader.main(WALReader.java:59) Caused by: java.lang.IllegalArgumentException: Invalid value length in KeyValue, valueLength=1986, KeyValueBytesHex=\x00\x00\x00T\x00\x00\x07\xC2\x00$efbb2495-8230-4262-a09d-e3261e54eb51\x04blobpm|com.apple.contact.people|pmap\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\x04\x1F\x8B\x08\x00\x00\x00\x00\x00\x00\x00\xEDW_l\x1CG\x19\x1F\x9F\xFF\xC4\xB1\x1D\xC7I\x13D+H\xF3`J\x03\xD9\xFB\xBB\xB7w\x974\x98\xB3\x0F\x07'\xD7\xC4\xF8\xE2@bJ5\xBB;\xEB\x1Bgwg33k\xDF%P\xB5O\xF0\x82T\x09\x1E\xA8hy\x00!\xD1\xD2\xAAOH \x90\xFA\xD8\x0A\x04\x0FH}\xA2\xA2\x0F \xF1\x80\xE0\x85J\x88"\x01\xDF\xEC\x9F\xBB\xB5\xE3\x18\xF3\xCEJ\xB7\xDA\xF9f\xBE\xDF\xF7\xFF\xFB\xE6^\xFF\x0B\x1A\x17\x1C\x9D\xDF\xC2\xDB8\x1FJ\xEA\xE6\x97\x98\xEB\x12KR\xE6\x8B\xF9u\xDFc6u(6]2\xA4?\xBA\x88\x9E\xFF\xD5W\xFE~&\x87F\xDAh\xC4\x92\xE8#m\xC5^P\xEC\x85\xE1\xB1\x8B\xBD\x00\x90KCd\x8B\xF9V\xC89\xF1%\x08I??\x8FE\xF7i\x1C\xCC\xDF\xC4nH\xC4MJv\x1E7\xDA/~\xE9[\xE7.\xE5\x10\xEAqT=$\xFFP\xAE\xC2\xB0_\xFA\xFD\xC9W\xBE\xAE0@\xC5Q\x0F\x07\x12=\x99Qr\x88Tx\x00)V\xFB\x89\xC3\x89M\xE5\x8C\xA2\xD1\x154-\xC8\xA6\x07{Ocqg\x05\xCD$\xABN\x97:r\x03M&K!Qi\xE3\x90\xAA\xCCwb\x1EP\x09!t\x1C~\x1F\x0By\xCA~\x08\x9F\xA4\xECk;\x0B\xCD\xF2w\x1BRyT!\xCD\x81\x81\xF9\xFF\x0Dc\x18\x94\x91e4\xE52l/cK2\x0E\x11zr_$\x97YwD~\x8D\xC07\xC7\xBEl\xC3\xD2Y\xFF\xD1\xF9\xF3\xEFl\xFD9\x0A\xCA\x98\xE8\xFB\x90:\x85\xFD]\x11q\x17vq\xCFw\x80!\x0E\x8E~X\x89\xF3\xD7\x98\xEF`\xCA\x15+\xF9F\xF9O\x9D\xFBoM\xC7i\xA5\x1D\x1AB\xF1\xFE\xF4\xCC\xF7\xBF\xFDZ\xEB\xF6W\x0FJ\xC9\x98\xB7i\x0A`\xB5\xE4\x17B\x12\x12[\xF1v9\xF3\xE9=\xC2\xC1\xFA\xA5pa\xED\x0F`\xFD\x0A\xD4\x9C\xC4\x92\x00\x96q\x08\xAC\xEB;\xBE*\xC0,X\xE5\xFD7\x7F\xF6\x86\xC7^M#\x8A\x16>\x8B\x90\xB8\x8B\x9E\x83\xB4S\xEF\xB9\xFF\xD3\x0E\xA6I\xD4\xB2\x98\x97\xC7A\xE0\x12\xE5y\x09~\xCE\x07\x84\xC1\xF2B\xB9Z\xB4\xEAV\xA3\xA1\xE9Ub\xC2\xAB^\xD3\xCC\x92ch\x95F\xADQ\xAD\x98\xF5Z\xB1b.\xB0K%\xCB\xB1 \x19?\xA5p\x04\xE5\x14`8\xC44\x8F-\x02H\x5CP!\xF3M\x8B\xAC\xC6\x9F*\x827N\xFD\xEB\xB1\x17\xFF\x96\xFBg\x0E\x8Dn\xA0#p\xAE\x85!\x0DPnc\xB1\x8D\x8E%\xCB\xEB\xE6\x16\xB41\x89\xCE\xB4\x01\xB7\xA0p\x0B\xB0S\x00\xA0\x0E\xE1\x14\xBB\xF4\x9EB\xBA\xD8\x8E\xF8o\xF4\x03"\xD1\x89\xB8\x86\x5C\xECo\x16\x96\x5C,\xC4EH\xACO\x1E\xA4VF\xA7\xFB\xF9\x7F\xAF](\xFC\xE2\x07QQ\xE6\xA8-\xD1l\x0C\xE7\x13YX_[\x89+n&JS \xE5\x81\xF4\xFAH/\xBF\xF4\xBD\x95\x1F\x8F*\x96\x09\xC8Q\xEAoJt2\xA3E'\xA2\x01\xAB\xF2y%B\x18\xBAI)\x02\x0B\x0F\xFBv^\xF4\x85$^>\x14\x84\xA7\xBEb\xFE{N\xEB7W~~\xFB\x5C\x0E=\xD6FG\xB1ms"\x04\x81\xC69\x97i\x16m\xD0\x1F\xBC0iR.\xBB6\xEE\xEF\xDE\x05OF>\x029\x01\xF6\xFBJ\x8D\xA7\xDA\xE8x\xB2\x5C\xED20\x85Z\x09y\x82x\x98\xBAQ~,\x80@\x87r!\xAFa\x8F$\xDB\xC7\x06\x84\xE6\xEAe\xD0\xE2xF\x8E\x9A\x17mtbpb\x0F\xF2\xA4\x13\xBAn\x06j\x86\xFA\x92p\x1F\xBB\x97\xD7WZ\x09\xED(\x15\x8B\xAA\xDE\x09\xB8\xFE\x91\x8C\x0F\x17\x19s\x09\xF6\x95\x8DT\xAC\x0A\x12\xDALq@\xAALB\x8C\xB3\x0A\xCE\xA4k\xA5\x9F\xA25\xDBh.\xA5\xED\xD1(\x17s\x01\xCA\x94Gm\xDB%\x19\x9CI\x9FZw\xB2\xB8\xE9:\x8B\x9B\xD2\xF6\xE0N\x07j\xEDE)\x9C\xBA5"\xA5n\x9D\x088qh/\x85\xE6\xC4\x85\x10\xD9\x0A)=1+\x98\x05\x09\xBE\xCA\x99C\xDD!\x9F\x08\x9D\x94\x0F\xD2\xFA\xD3\xFB\xA6\x91, offset=0, length=1080 at org.apache.hadoop.hbase.KeyValueUtil.checkKeyValueBytes(KeyValueUtil.java:556) at org.apache.hadoop.hbase.KeyValue.<init>(KeyValue.java:344) at org.apache.hadoop.hbase.KeyValueUtil.createKeyValueFromInputStream(KeyValueUtil.java:717) at org.apache.hadoop.hbase.codec.KeyValueCodecWithTags$KeyValueDecoder.parseCell(KeyValueCodecWithTags.java:81) at org.apache.hadoop.hbase.codec.BaseDecoder.advance(BaseDecoder.java:68) at org.apache.hadoop.hbase.wal.WALEdit.readFromCells(WALEdit.java:214) at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:382) ... 3 more ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
