Re: Region-server crash: Added a key not lexically larger than previous

2019-08-17 Thread Alexander Batyrshin
Replayd at https://issues.apache.org/jira/browse/HBASE-22862 > On 16 Aug 2019, at 20:16, Stack wrote: > > Interesting. The Cells differ in sequence id. Would seem to imply race > around the getting of sequenceid writing the WAL. The crashes

Re: Region-server crash: Added a key not lexically larger than previous

2019-08-16 Thread OpenInx
We have an issue https://issues.apache.org/jira/browse/HBASE-22862, Let's discuss there. Thanks. On Sat, Aug 17, 2019 at 1:17 AM Stack wrote: > Interesting. The Cells differ in sequence id. Would seem to imply race > around the getting of sequenceid writing the WAL. The crashes are happening >

Re: Region-server crash: Added a key not lexically larger than previous

2019-08-16 Thread Stack
Interesting. The Cells differ in sequence id. Would seem to imply race around the getting of sequenceid writing the WAL. The crashes are happening on regionserver replay of WAL files recovering crashed server? Or when is it happening? If the former, you might have to find the offending WAL and

Re: Region-server crash: Added a key not lexically larger than previous

2019-08-15 Thread Alexander Batyrshin
Size ~ 4 000M, Store File Size ~ 1TB (FAST_DIFF) 23 columns in 1 column family > On 16 Aug 2019, at 04:14, Sean Kennedy wrote: > > Alex, > > How large is the table and how many columns? > > Thx > > Sean > > On Thursday, August 15, 2019, Alexander Batyrshin <0x62...@gmail.com> wrote: > >>

Re: Region-server crash: Added a key not lexically larger than previous

2019-08-15 Thread Sean Kennedy
Alex, How large is the table and how many columns? Thx Sean On Thursday, August 15, 2019, Alexander Batyrshin <0x62...@gmail.com> wrote: > > Hello all, > > We observer error "Added a key not lexically larger than previous” that > cause most of our region-servers to crash in our cluster. >

Re: Region-server crash: Added a key not lexically larger than previous

2019-08-15 Thread Rushabh Shah
Hi Alexander, This is not the right forum to report a bug. Please open a jira in HBASE project with relevant stack traces. Rushabh Shah - Software Engineering SMTS | Salesforce - - Mobile: 213 422 9052 On Thu, Aug 15, 2019 at 8:57

Region-server crash: Added a key not lexically larger than previous

2019-08-15 Thread Alexander Batyrshin
Hello all, We observer error "Added a key not lexically larger than previous” that cause most of our region-servers to crash in our cluster. HBase-1.4.10 2019-08-15 18:02:10,554 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 1/1 column families, memstore=56.08 MB 2019-08-15