Sorry for resurrecting this 'old' thread but I need to open a ticket for this.
How do I do that ? Thanks, Mike G. From: Keith Turner [mailto:[email protected]] Sent: Wednesday, April 24, 2013 12:23 PM To: [email protected] Subject: Re: PermGen leak in JBoss On Wed, Apr 24, 2013 at 11:21 AM, Heath Abelson <[email protected]<mailto:[email protected]>> wrote: Hi, I am a colleague of Michael's, I can provide more info The following are the 5 threads that are not being cleaned up on an undeploy. It's hard to get more stack info than this. Daemon Thread [FileWatchdog] (Running) Daemon Thread [MSC service thread 1-5-SendThread(cbdbtestbox:2181)] (Running) Daemon Thread [MSC service thread 1-5-EventThread] (Running) Daemon Thread [Thrift Connection Pool Checker] (Running) Daemon Thread [FileWatchdog] (Running) Using Plumbr, the report states that the SendThread and EventThread above both contain an instance of org.apache.zookeeper.ClientCnxn I tried this on 1.4 and then ran jstack using following program. public class Test { public static void main(String[] args) throws Exception { ZooKeeperInstance instance = new ZooKeeperInstance("test14", "localhost"); Connector connector = instance.getConnector("root", "secret"); UtilWaitThread.sleep(600000); } } I see the following threads after filtering out java threads and the thread calling main. I am running this code in a different way. I used the accumulo script to execute it, which uses Accumulo start, which introduces the AccumuloFilesystemAlterationMonitor thread. I am seeing three of the threads you see, I am not seeing the FileWatchDog threads. "Thrift Connection Pool Checker" daemon prio=10 tid=0x00007f861839f800 nid=0x61f0 waiting on condition [0x00007f8652a62000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.accumulo.core.client.impl.ThriftTransportPool$Closer.run(ThriftTransportPool.java:129) at java.lang.Thread.run(Thread.java:662) "Test-EventThread" daemon prio=10 tid=0x00007f8618119000 nid=0x61ee waiting on condition [0x00007f8652d02000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000bb402270> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) "Test-SendThread(localhost:2181)" daemon prio=10 tid=0x00007f8618103000 nid=0x61ed runnable [0x00007f8652e03000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x00000000bb3e0e88> (a sun.nio.ch.Util$2) - locked <0x00000000bb3e0e78> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000bb3e0aa8> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131) "org.apache.accumulo.start.classloader.AccumuloFilesystemAlterationMonitor" daemon prio=10 tid=0x00007f86a4387000 nid=0x61e9 waiting on condition [0x00007f865367c000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.accumulo.start.classloader.AccumuloFilesystemAlterationMonitor.run(AccumuloFilesystemAlterationMonitor.java:127) at java.lang.Thread.run(Thread.java:662) Should probably open a ticket about this issue. We could probably do something about the Thrift Connection Pool Checker threads. I am not sure if the zookeeper threads are an issue w/ zookeeper or an issue w/ how accumulo is using zookeeper. -Heath -----Original Message----- From: Michael Giordano Sent: Wednesday, April 24, 2013 9:48 AM To: [email protected]<mailto:[email protected]> Subject: PermGen leak in JBoss Under version 1.3.7 we are using the following code to initialize a cloudbase connection during initialization of our web app: ZooKeeperInstance instance = new ZooKeeperInstance(instanceName, zooKeepers); connector = instance.getConnector(userId, password.getBytes()); The problem is that under the hood, this call creates several (5) threads that are not cleaned up when the app is undeployed in JBoss. This is occurring without performing any scans or interacting with cloudbase in any other way. After relatively few redeploys of the app, the PermGen Space is OOM. I can't find any reference in the cloudbase API akin to a close() method for the Connector object. This is a classloader leak effecting any webapp that is accessing cloudbase directly. The result of this leak is not simply orphaned threads, but thousands of classes not gc'd because the classloader itself can't be gc'd. This is what is filling up PermGen. Has anyone discovered this particular issue? Has anyone discovered a solution? Thanks, Mike G. NetCentric Technology, Inc. 3349 Route 138, Building A Wall, NJ 07719 Phone: 732-544-0888<tel:732-544-0888>
