Ted: To your previous question about HA mode - on the test cluster where this is, we have a single node HA - there's only one namenode specified in dfs.ha.namenodes.HDFSNAME. Do you think this configuration is the root of this error?
----- Original Message ----- From: [email protected] To: Sudarshan Kadambi (BLOOMBERG/ 731 LEXIN) Cc: [email protected] At: Feb 25 2014 12:04:51 Looking at the stack trace for IllegalArgumentException, it reminds me of HBASE-10029: Proxy created by HFileSystem#createReorderingProxy() should properly close when connecting to HA namenode The above issue has been resolved in the recently released 0.98.0 FYI On Tue, Feb 25, 2014 at 7:59 AM, Ted Yu <[email protected]> wrote: > In the jstack pastebin, I don't see StackOverflowError. > Not in region server log, either. > > I saw the following: > > java.lang.IllegalArgumentException: object is not an instance of declaring > class > > Are you using hdfs HA ? > > > > On Tue, Feb 25, 2014 at 7:54 AM, Sudarshan Kadambi (BLOOMBERG/ 731 LEXIN) > <[email protected]> wrote: > >> Ted - thanks for taking a look. >> >> regionserver.log -> http://paste.ubuntu.com/6995011/ >> regionserver.out -> http://paste.ubuntu.com/6994999/ >> jstack -> http://paste.ubuntu.com/6995016/ >> >> ----- Original Message ----- >> From: [email protected] >> To: Sudarshan Kadambi (BLOOMBERG/ 731 LEXIN) <[email protected]>, >> [email protected] >> At: Feb 25 2014 10:41:48 >> >> Do you have more of the stack trace ? >> If so, please pastebin it. >> >> Thanks >> >> >> On Tue, Feb 25, 2014 at 7:21 AM, Sudarshan Kadambi (BLOOMBERG/ 731 LEXIN) >> <[email protected]> wrote: >> >>> I recently saw some exceptions that I haven't seen in HBase before. The >>> trace looks something like: >>> >>> Exception in thread "RpcServer.handler=96,port=60020" >>> java.lang.StackOverflowError >>> at org.apache.hadoop.hbase.CellUtil$1.advance(CellUtil.java:160) >>> >>> Is this a known issue? The RegionServer went into a hung state and all >>> gets to that RS began to hang indefinitely. This is on HBase 0.96. >> >> >> >
