Good idea - I'll try that. -JZ
On Jul 8, 2012, at 8:11 PM, Camille Fournier wrote: > What does a jstack on another long-running instance show? Anything out of > the norm? > > C > > On Sun, Jul 8, 2012 at 11:08 PM, Jordan Zimmerman < > [email protected]> wrote: > >> I'm not sure which of the instances went down, but it could be a few weeks. >> >> -JZ >> >> On Jul 8, 2012, at 8:06 PM, Camille Fournier wrote: >> >>> Yeah a tickle in the back of my mind says that this is a JVM bug. When >> was >>> the last time you restarted these instances? >>> >>> C >>> >>> On Sun, Jul 8, 2012 at 11:04 PM, Jordan Zimmerman < >>> [email protected]> wrote: >>> >>>> It's not really out of memory. It's "unable to create new native >> thread". >>>> It appears to have to do with JMX (notice the RMI). >>>> >>>> -JZ >>>> >>>> On Jul 8, 2012, at 7:43 PM, Camille Fournier wrote: >>>> >>>>> Not known specifically that I know of. Can you give us more details? >> Did >>>>> you actually run out of memory? File descriptors? What does a heap dump >>>>> tell you? >>>>> >>>>> C >>>>> >>>>> On Sun, Jul 8, 2012 at 10:29 PM, Jordan Zimmerman < >>>>> [email protected]> wrote: >>>>> >>>>>> Is this a known issue in 3.3.3? If so, is it fixed in 3.3.5? >>>>>> >>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: WARNING: RMI TCP >>>>>> Accept-0: accept loop for ServerSocket[addr= >>>>>> 0.0.0.0/0.0.0.0,port=0,localport=59462] throws >>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: Jul 8, 2012 >>>> 11:01:28 >>>>>> PM sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop >>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at >>>>>> java.lang.Thread.run(Thread.java:636) >>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at >>>>>> >> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:359) >>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at >>>>>> >>>> >> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:402) >>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at >>>>>> >>>> >> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1336) >>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at >>>>>> >>>> >> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:943) >>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at >>>>>> java.lang.Thread.start(Thread.java:614) >>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at >>>>>> java.lang.Thread.start0(Native Method) >>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: >>>>>> java.lang.OutOfMemoryError: unable to create new native thread >>>>>> >>>>>> >>>> >>>> >> >>
