Messages keep hanging in JDBCStore without delivery to client
-------------------------------------------------------------
Key: AMQ-2184
URL: https://issues.apache.org/activemq/browse/AMQ-2184
Project: ActiveMQ
Issue Type: Bug
Components: Broker
Affects Versions: 5.2.0, 5.1.0
Environment: OS: Linux version 2.6.26-amd64-85 (r...@etch64) (gcc
version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21))
ActiveMQ: 5.1.0
1 Broker, 12 JBoss's with round about 20 consumers and 20 Producers on 7 JDBC
persistant queues
Clients use transacted consumers.
Reporter: Norbert Pfistner
Priority: Blocker
Fix For: 5.3.0
We currently suffer from hanging messages in the JDBC persistant storage in our
production environment.
Occasionally (once in a day or week) a few Messages (most time 1 or 2,
sometimes more) seems to stuck in the DB Table activemq_msg without being
delivered even once to a consumer. They are never deliverd to a consumer until
the broker is rebooted. Strange enough new messages produced into the same
queue with those stuck messages are delivered to the consumers as normal. It
simply looks like the borker is blind about the stuck messages.
Our system depends on processing every message, so missing even one of them is
realy a big problem in production.
Everything seems to be fine: No warnings, errors or even debug logging gives
any hint about a problem.
Unfortunately we can't debug the broker 'cuse it's our production environment.
Our Testing envirenment is not capable to reproduce the problem (actually it's
only one Machine with a Broker and a signle jaboss running a small number of
producers/clients).
Our messaging characteristics show moderate producers and slow consumers. The
count of simultanous messages is moderate (about 300-1000). Actually the
producers have hot phases generating bulks of Messages followed by phases of
long inactivity. The consumers are slow related to the producers but always
keep within an hour or two. We do not experience any memory problems.
Because this bug may be related to
https://issues.apache.org/activemq/browse/AMQ-1918, I had a look at the code
around the StoreCursors and stumbled apon a strange code fragment called when
removing entries.
The suspicous code is found in
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.remove() in
5.1.0, 5.2 .0 and 5.3.0:
public final synchronized void remove() {
size--;
if (size==0 && isStarted() && cacheEnabled) {
cacheEnabled=true;
}
}
As far as i can see, the if statement ist actually a NOP, 'cause it will not
modify cacheEnabled in any case.
Maybe this causes some strange effects when using the StoreCursor's derived
from this class..
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
