Barrie, It looks like those are all JMS/JCA flow threads. Could you try disabling those two flows in conf/servicemix.xml and see if that fixes the issue? It might only be a workaround if you're not using those flows, but it will also give us a clue where to look for solving this issue.
Regards, Gert Vanthienen ------------------------ Open Source SOA: http://fusesource.com Blog: http://gertvanthienen.blogspot.com/ 2009/10/19 Barrie Treloar <[email protected]>: > Ok here are the results of the suggestions. > > Stuart Roe: > Try: If you stop SMX can you delete the data folder in the SMX installation? > Result: Success > Selected "data" directory in ${SMX_HOME} and successfully deleted > directory via Windows Explorer > Delete > > Gert Vanthienen: > virus scanners - Not tried. > > Gert Vanthienen: > Try: Use hotdeploy instead of the jbi maven plugin. > Result: Success. > Fresh SMX start, copy the <installer>.zip created > INFO - AutoDeploymentService - Directory: hotdeploy: > Finished installation of archive: mcp-sa-1.0-SNAPSHOT.zip > > Even tried deleting the hotdeploy archive and that uninstalls correctly as > well. > Also rebuilt sa and re-hotdeployed and that works with no errors. > > Barrie: > Stop SMX > Delete data directory > Start SMX > Wait for re-installation of data/ directory. > Restart SMX to avoid any re-installation potential issues. > Searched for "mcp" in ${SMX_HOME} - found nothing. > Hotdeploy > Searched for "mcp" in ${SMX_HOME} > - found in data\smx\service-assemblies\mcp-sa\... > - found in data\amq\kr-store\data (hash index files) > - found in hotdeploy\ (obviously) > Hotundeploy > Searched for "mcp" in ${SMX_HOME} > - found in data\smx\service-assemblies\mcp-sa\... > - found in data\amq\kr-store\data (hash index files) > Delete directory > - Fails because java has the locks on the files - specifically the > libraries in one of my service-units. > > My code is pretty dumb (its only a proof of concept) and I can't see > anything that would be left in memory after an un-install. > I thought perhaps it was Log4J and the static Loggers holding > references to classes, but I've removed that code and it still > happens. > > I was just about to remove the guts of my code thinking something like > XStream might be holding onto classes, when I realized that I am not > actually running the service-unit. > I'm just installing/uninstalling which therefore should not create any locks. > I suspect that SMX is not unloading classes correctly then. > > So doing a full thread dump I can see that there are threads blocked > on waiting condition - which is probably why the classes are not being > released. > > So the problem appears to be uninstall does not stop running threads > created during the install. > Scanning JIRA on "thread waiting" shows > http://issues.apache.org/activemq/browse/SM-1807 "Classes, Objects and > Threads still alive after Undeploying a servicemix component" > http://issues.apache.org/activemq/browse/SM-1578 "When shutting down > endpoints, the component should wait for all exchanges to be > completely processed before unregistering the endpoint" > > Does this sound familiar to anyone? Have I identified the correct defects? > Cheers > > "QueueThread:queue://org.apache.servicemix.jca.{urn:mcp:testing}file:sender" > daemon prio=6 tid=0x00a39090 nid=0x172c waiting on condition > [0x3088f000..0x3088fa98] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:359) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > > "QueueThread:queue://org.apache.servicemix.jca.{urn:mcp}incoming:endpoint" > daemon prio=6 tid=0x2af22d58 nid=0x11d8 waiting on condition > [0x307cf000..0x307cfb18] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:359) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > > "QueueThread:queue://org.apache.servicemix.jms.{urn:mcp}incoming:endpoint" > daemon prio=6 tid=0x28fbfc18 nid=0xce4 waiting on condition > [0x3078f000..0x3078fd98] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:359) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > > "QueueThread:queue://org.apache.servicemix.jms.{urn:mcp:testing}file:sender" > daemon prio=6 tid=0x2b898db0 nid=0x1420 waiting on condition > [0x3068f000..0x3068fb18] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:359) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > > "QueueThread:queue://org.apache.servicemix.jms.{http://activemq.apache.org/camel/schema/jbi}provider:camel:mcp-eip-su-controlBus"; > daemon prio=6 tid=0x28b90638 nid=0xb50 waiting on condition > [0x304cf000..0x304cfb18] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:359) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > > "QueueThread:queue://org.apache.servicemix.jca.{http://activemq.apache.org/camel/schema/jbi}provider:camel:mcp-eip-su-controlBus"; > daemon prio=6 tid=0x28f6d408 nid=0x7e0 waiting on condition > [0x3040f000..0x3040fc18] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:359) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > > "QueueThread:queue://org.apache.servicemix.jca.{urn:mcp}outgoing:endpoint" > daemon prio=6 tid=0x2878b390 nid=0x828 waiting on condition > [0x3038f000..0x3038fb18] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:359) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) > > "QueueThread:queue://org.apache.servicemix.jms.{urn:mcp}outgoing:endpoint" > daemon prio=6 tid=0x2bc35408 nid=0x14e4 waiting on condition > [0x300df000..0x300dfa18] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:359) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674) > at java.lang.Thread.run(Thread.java:595) >
