[jira] [Commented] (HBASE-23106) [HBCK2/hbase-operator-tools] Add a WAL verifier

2019-10-04 Thread Michael Stack (Jira) 


[ 
https://issues.apache.org/jira/browse/HBASE-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944672#comment-16944672
 ] 

Michael Stack commented on HBASE-23106:
---

[~wchevreuil] Thank you. I had forgotten this tool. Nor could I find a WAL 
Reader/Verifier when poking around in doc when under duress. Let me change this 
issue so instead it is some doc cleanup so these things are easier to find. 
Thanks for the intercession.

> [HBCK2/hbase-operator-tools] Add a WAL verifier
> ---
>
> Key: HBASE-23106
> URL: https://issues.apache.org/jira/browse/HBASE-23106
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Michael Stack
>Priority: Major
>
> We had a WALPlayer that loads edits up into hbase cluster but what I wanted 
> yesterday was a WAL verifier so I could find the bad WAL messing me up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23106) [HBCK2/hbase-operator-tools] Add a WAL verifier

2019-10-04 Thread Wellington Chevreuil (Jira) 


[ 
https://issues.apache.org/jira/browse/HBASE-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944409#comment-16944409
 ] 

Wellington Chevreuil commented on HBASE-23106:
--

Hey [~stack], is this a replacement for WALPretyPrinter?

> [HBCK2/hbase-operator-tools] Add a WAL verifier
> ---
>
> Key: HBASE-23106
> URL: https://issues.apache.org/jira/browse/HBASE-23106
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Michael Stack
>Priority: Major
>
> We had a WALPlayer that loads edits up into hbase cluster but what I wanted 
> yesterday was a WAL verifier so I could find the bad WAL messing me up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23106) [HBCK2/hbase-operator-tools] Add a WAL verifier

2019-10-03 Thread Michael Stack (Jira) 


[ 
https://issues.apache.org/jira/browse/HBASE-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943845#comment-16943845
 ] 

Michael Stack commented on HBASE-23106:
---

Here is example of what output looks like when we run the WALReader... passing 
two files -- first a small recovered.edits file and then a corrupted 
recovered.edits file:
{code}
kalashnikov:hbase.apache.git stack$ ./bin/hbase 
org.apache.hadoop.hbase.wal.WALReader /tmp/0932379
2019-10-03 12:28:46,375 WARN  [main] util.NativeCodeLoader: Unable to load 
native-hadoop library for your platform... using builtin-java classes where 
applicable
/tmp/0932379
3448 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932373 3271
11309 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932374 7767
18565 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932375 7162
34217 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932376 15558
44758 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932377 10447
57357 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932378 12505
60722 GENIE2_modality_syncdata/3c2fc74c7d67cd20883799b3f7789b3b/932379 3271
kalashnikov:hbase.apache.git stack$  ./bin/hbase 
org.apache.hadoop.hbase.wal.WALReader /tmp/0932380
2019-10-03 12:28:52,110 WARN  [main] util.NativeCodeLoader: Unable to load 
native-hadoop library for your platform... using builtin-java classes where 
applicable
/tmp/0932380
2019-10-03 12:28:52,568 WARN  [main] hbase.KeyValueUtil: 
8\x8E\xF8o\xF4\x03"\xD1\x89\xB8\x86\x5C\xECo\x16\x96\x5C,\xC4EH\xACO\x1E\xA4VF\xA7\xFB\xF9\x7F\xAF](\xFC\xE2\x07QQ\xE6\xA8-\xD1l\x0C\xE7\x13YX_[\x89+n
 
\xE5\x81\xF4\xFAH/\xBF\xF4\xBD\x95\x1F\x8F*\x96\x09\xC8Q\xEAoJt2\xA3E'\xA2\x01\xAB\xF2y%B\x18\xBAI)\x02\x0B\x0F\xFBv^\xF4\x85$^>\x14\x84\xA7\xBEb\xFE{N\xEB7W~~\xFB\x5C\x0E=\xD6FG\xB1ms"\x04\x81\xC69\x97i\x16m\xD0\x1F\xBC0iR.\xBB6\xEE\xEF\xDE\x05OF>\x029\x01\xF6\xFBJ\x8D\xA7\xDA\xE8x\xB2\x5C\xED20\x85Z\x09y\x82x\x98\xBAQ~,\x80@\x87r!\xAFa\x8F$\xDB\xC7\x06\x84\xE6\xEAe\xD0\xE2xF\x8E\x9A\x17mtbpb\x0F\xF2\xA4\x13\xBAn\x06j\x86\xFA\x92p\x1F\xBB\x97\xD7WZ\x09\xED(\x15\x8B\xAA\xDE\x09\xB8\xFE\x91\x8C\x0F\x17\x19s\x09\xF6\x95\x8DT\xAC\x0A\x12\xDALq@\xAALB\x8C\xB3\x0A\xCE\xA4k\xA5\x9F\xA25\xDBh.\xA5\xED\xD1(\x17s\x01\xCA\x94Gm\xDB%\x19\x9CI\x9FZw\xB2\xB8\xE9:\x8B\x9B\xD2\xF6\xE0N\x07j\xEDE)\x9C\xBA5"\xA5n\x9D\x088qh/\x85\xE6\xC4\x85\x10\xD9\x0A)=1+\x98\x05\x09\xBE\xCA\x99C\xDD!\x9F\x08\x9D\x94\x0F\xD2\xFA\xD3\xFB\xA6\x91,
 offset=0, length=1080
2019-10-03 12:28:52,569 WARN  [main] wal.ProtobufLogReader: Encountered a 
malformed edit, seeking back to last good position in file, from 1261 to 83
java.io.EOFException: EOF  while reading 12 WAL KVs; started reading at 177 and 
read up to 1261
at 
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:397)
at 
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:98)
at org.apache.hadoop.hbase.wal.WALReader.read(WALReader.java:41)
at org.apache.hadoop.hbase.wal.WALReader.main(WALReader.java:59)
Caused by: java.lang.IllegalArgumentException: Invalid value length in 
KeyValue, valueLength=1986, 
KeyValueBytesHex=\x00\x00\x00T\x00\x00\x07\xC2\x00$efbb2495-8230-4262-a09d-e3261e54eb51\x04blobpm|com.apple.contact.people|pmap\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\x04\x1F\x8B\x08\x00\x00\x00\x00\x00\x00\x00\xEDW_l\x1CG\x19\x1F\x9F\xFF\xC4\xB1\x1D\xC7I\x13D+H\xF3`J\x03\xD9\xFB\xBB\xB7w\x974\x98\xB3\x0F\x07'\xD7\xC4\xF8\xE2@bJ5\xBB;\xEB\x1Bgwg33k\xDF%P\xB5O\xF0\x82T\x09\x1E\xA8hy\x00!\xD1\xD2\xAAOH
 \x90\xFA\xD8\x0A\x04\x0FH}\xA2\xA2\x0F

[jira] [Commented] (HBASE-23106) [HBCK2/hbase-operator-tools] Add a WAL verifier

2019-10-02 Thread Michael Stack (Jira) 


[ 
https://issues.apache.org/jira/browse/HBASE-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943179#comment-16943179
 ] 

Michael Stack commented on HBASE-23106:
---

The files under the recovered.edits folder could use this too. They have same 
format.  

> [HBCK2/hbase-operator-tools] Add a WAL verifier
> ---
>
> Key: HBASE-23106
> URL: https://issues.apache.org/jira/browse/HBASE-23106
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Michael Stack
>Priority: Major
>
> We had a WALPlayer that loads edits up into hbase cluster but what I wanted 
> yesterday was a WAL verifier so I could find the bad WAL messing me up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23106) [HBCK2/hbase-operator-tools] Add a WAL verifier

2019-10-01 Thread Michael Stack (Jira) 


[ 
https://issues.apache.org/jira/browse/HBASE-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942281#comment-16942281
 ] 

Michael Stack commented on HBASE-23106:
---

Perhaps there is one. I couldn't find it (could have looked harder I suppose -- 
if it exists...need to add to refguide, etc.)

> [HBCK2/hbase-operator-tools] Add a WAL verifier
> ---
>
> Key: HBASE-23106
> URL: https://issues.apache.org/jira/browse/HBASE-23106
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Michael Stack
>Priority: Major
>
> We had a WALPlayer that loads edits up into hbase cluster but what I wanted 
> yesterday was a WAL verifier so I could find the bad WAL messing me up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)