Hi, I'm running the 5.3 release as a standalone broker. In one case, a producer is running without a consumer, producing small, persistent messages, with the FileCursor pendingQueuePolicy (per https://issues.apache.org/activemq/browse/AMQ-2512) option and flow control memoryLimit set to 100mb for the queue in question. (Through a policy entry)
As the queue grows above 300k messages, KahaDB indexing starts climbing above 1 second. At around 350k messages, the indexing is taking over 8 seconds. At this point, I start getting java out of heap space errors in essentially random parts of the code. After a while, the producers timeout with a channel inactive for too long error, and the entire broker basically wedges itself. At this point, consumers are generally unable to bind to the broker quitting with timeout errors. When they can connect, consuming a single message triggers an index re-build, which takes 2-8seconds. Turning on verbose garbage collection, the jvm is collecting like mad but reclaiming no space. If I restart the broker, it comes back up, I can consume the old messages, and can handle another 350k messages until it wedges. I can reproduce under both default gc and incremental gc. Two questions: - It seems like someone is holding onto a handle to the messages after they have been persisted to disk - is this a known issue? Should I open a JIRA for it? (Or is there another explanation?) - Is there any documentation about the internals of KahaDB - the kind of indices etc? I'd like to get a better understanding of the index performance and in general how KahaDB compares to something like BerkeleyDB. Thanks
