[
https://issues.apache.org/jira/browse/AMQ-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900454#comment-14900454
]
Stefan commented on AMQ-5228:
-----------------------------
Any news here? Right now our LevelDB is growing as well and the only way to
circumvent is to wait until all queues are empty, shut-down ActiveMQ and delete
the files manually. Without the compact, its not usable so we switched back to
kahaDB
> java.lang.NoClassDefFoundError: org/fusesource/leveldbjni/internal/JniDB
> error during cleanup
> ---------------------------------------------------------------------------------------------
>
> Key: AMQ-5228
> URL: https://issues.apache.org/jira/browse/AMQ-5228
> Project: ActiveMQ
> Issue Type: Bug
> Components: activemq-leveldb-store
> Affects Versions: 5.9.1, 5.10.0
> Environment: Linux 2.6.32-279.5.2.el6.x86_64 #1 SMP Thu Aug 23
> 12:05:59 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
> JDK 1.7.0_51
> Apache Karaf 2.3.5 with activemq-osgi, activemq-blueprint, activemq-client,
> activemq-webconsole, activemq-camel, activemq features installed. Using
> version 5.9.1 (and tried 5.10.0)
> Reporter: Timothy Stewart
>
> In our production environment, our storage folder runs out of disk space
> every few days (75 Gb). Restarting the container addresses the issue, it
> cleans it up. I noticed a stack trace coming on system.err in the
> wrapper.log (Karaf starts with a wrapper service). It may or may not be
> related to our disk space issue:
> INFO | jvm 1 | 2014/06/13 03:22:36 | Exception in thread "Thread-117"
> java.lang.NoClassDefFoundError: org/fusesource/leveldbjni/internal/JniDB
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> org.apache.activemq.leveldb.LevelDBClient$RichDB.compact(LevelDBClient.scala:377)
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> org.apache.activemq.leveldb.LevelDBClient.gc(LevelDBClient.scala:1647)
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> org.apache.activemq.leveldb.DBManager$$anonfun$pollGc$1$$anonfun$apply$mcV$sp$2.apply$mcV$sp(DBManager.scala:648)
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> org.fusesource.hawtdispatch.package$$anon$4.run(hawtdispatch.scala:357)
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> java.lang.Thread.run(Thread.java:744)
> INFO | jvm 1 | 2014/06/13 03:22:36 | Caused by:
> java.lang.ClassNotFoundException: org.fusesource.leveldbjni.internal.JniDB
> not found by org.apache.activemq.activemq-osgi [105]
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1460)
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72)
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843)
> INFO | jvm 1 | 2014/06/13 03:22:36 | at
> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> INFO | jvm 1 | 2014/06/13 03:22:36 | ... 7 more
> I see the same problem in our dev environment but could not replicate it. I
> was finally able to replicate by using the hawtio console to execute the
> compact operation. Everytime I do this, the same stack trace outputs and the
> operation cycles endlessly (well as long as I've waited anyhow). The 5.10.0
> stack trace when I execute the operation is:
> INFO | jvm 2 | 2014/06/14 21:00:44 | Exception in thread "Thread-106"
> java.lang.NoClassDefFoundError: org/fusesource/leveldbjni/internal/JniDB
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> org.apache.activemq.leveldb.LevelDBClient$RichDB.compact(LevelDBClient.scala:378)
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> org.apache.activemq.leveldb.LevelDBClient.gc(LevelDBClient.scala:1654)
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> org.apache.activemq.leveldb.LevelDBStoreView$$anonfun$compact$1.apply$mcV$sp(LevelDBStore.scala:126)
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> org.fusesource.hawtdispatch.package$$anon$4.run(hawtdispatch.scala:330)
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> java.lang.Thread.run(Thread.java:744)
> INFO | jvm 2 | 2014/06/14 21:00:44 | Caused by:
> java.lang.ClassNotFoundException: org.fusesource.leveldbjni.internal.JniDB
> not found by org.apache.activemq.activemq-osgi [390]
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1460)
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72)
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843)
> INFO | jvm 2 | 2014/06/14 21:00:44 | at
> java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> INFO | jvm 2 | 2014/06/14 21:00:44 | ... 7 more
> I started looking at the activemq-osgi and activemq-leveldb project poms, and
> noticed that the leveldbjni projects are commented out with a note that we
> want to focus on hardening the pure java version. With that in mind, I
> switched the indexFactory on my leveldb config to
> indexFactory="org.iq80.leveldb.impl.Iq80DBFactory". I restarted, and tried
> the compact operation again, but got the same stacktrace.
> My leveldb config now looks like:
> <amq:persistenceAdapter>
> <amq:levelDB
> directory="[[karaf.storage]]/activemq/default/leveldb" logSize="107374182"
> logCompression="snappy" indexFactory="org.iq80.leveldb.impl.Iq80DBFactory" />
> </amq:persistenceAdapter>
> The logCompression and the indexFactory were both added as an attempt to
> mitigate the problem, but the issue exists either way. I also saw a note on
> the replicated levedb store about scheduled jobs, which are used a lot, but
> when I looked at those folders in our storage folder it seems to use a
> completely different kahadb store, so I threw that out as a reason.
> I did attempt to add leveldbjni-linux64 or leveldbjni-all to the osgi
> deployment, but it does not export the internal package which activemq-osgi
> wants to import. I could build a new package of these, or rebuild
> activemq-osgi with the commented dependencies removed.. perhaps that will
> work, but it seems that as it is with the dependencies commented out that
> there is something broken within the compact routine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
