Thank you. >From the proposed resolution I imagine that the RS would then die in case of a handler error. So the question remains what error originally occured in the handler in the first place. The log of the entire lifecycle of the RS (http://schabby.de/wp-content/uploads/2014/08/filtered.txt) does not reveal much to me unfortunately. Do you find anything in there that hints to something that may cause the handler to end up in the soon-to-be-fixed recursion?
@Ted, the line "at org.apache.hadoop.hbase.CellUtil$1.advance(CellUtil.java:210)" is all I can see unfortunately :( On Sat, Aug 23, 2014 at 9:43 PM, Andrew Purtell <[email protected]> wrote: > On Sat, Aug 23, 2014 at 12:11 PM, Johannes Schaback < > [email protected]> wrote: > > > Exception in thread "defaultRpcServer.handler=5,queue=2,port=60020" > > java.lang.StackOverflowError > > at org.apache.hadoop.hbase.CellUtil$1.advance(CellUtil.java:210) > > at org.apache.hadoop.hbase.CellUtil$1.advance(CellUtil.java:210) > > at org.apache.hadoop.hbase.CellUtil$1.advance(CellUtil.java:210) > > at org.apache.hadoop.hbase.CellUtil$1.advance(CellUtil.java:210) > > (and so on...) > > > > > > > That is the anonymous CellScanner instance we create from > CellUtil#createCellScanner. See > https://issues.apache.org/jira/browse/HBASE-11813 > > > > Filtering the .out file for "Exception" shows that several handlers > > crashed > > > > like that: > > > > Exception in thread "defaultRpcServer.handler=5,queue=2,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=18,queue=0,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=23,queue=2,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=24,queue=0,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=2,queue=2,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=11,queue=2,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=25,queue=1,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=20,queue=2,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=19,queue=1,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=15,queue=0,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=1,queue=1,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=7,queue=1,port=60020" > > java.lang.StackOverflowError > > Exception in thread "defaultRpcServer.handler=4,queue=1,port=60020" > > java.lang.StackOverflowError > > > > > We should fix this so the RegionServer aborts if it loses a handler to an > Error. > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > -- LadenZeile.de <http://www.ladenzeile.de/> powered by Visual Meta GmbH - www.visual-meta.com Tel.: +49 30 / 609 84 88 20 Fax: +49 30 / 609 84 88 21 E-Mail: [email protected] Visual Meta GmbH, Schützenstraße 25, 10117 Berlin Geschäftsführer: Robert M. Maier, Johannes Schaback Handelsregister HRB 115795 B, Amtsgericht Charlottenburg USt-IdNr.: DE263760203
