[
https://issues.apache.org/jira/browse/AMQ-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mat Sharpe updated AMQ-3473:
----------------------------
Attachment: activemq.xml
> Messages (possibly) stuck and pending messages count showing high number of
> pending message which do not get sent to a consumer.
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: AMQ-3473
> URL: https://issues.apache.org/jira/browse/AMQ-3473
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.5.0
> Environment: Ubuntu 11.04 (64-bit)
> Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
> Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode)
> Reporter: Mat Sharpe
> Attachments: activemq.xml
>
>
> Two brokers, each with a network connection to the other. We have two
> producers producing persistent messages to a single queue at a rate of
> 20-50/second. There is a single consumer. All clients prefer the primary
> broker.
> The consumer is 'bursty' - i.e. it grabs 5000 messages and then processes
> them. During processing, new messages build up on the broker.
> If the primary broker is restarted we will see it come back with, as you
> would expect, with a number of pending messages. This message count never
> fully returns to 0 even if the producers are stopped and browsing the queue
> through the GUI shows either no messages or only messages that were produced
> since the restart.
> I have turned on Kaha debugging and, after the initial restart, we see the
> following during every checkpoint:
> [eckpoint Worker] TRACE MessageDatabase - Last update:
> 3974:2450180, full gc candidates set: [3950, 3951, 3973, 3974]
> ...
> [eckpoint Worker] TRACE MessageDatabase - gc candidates after
> dest:1:MyQueue, [3951, 3973]
> ...
> [eckpoint Worker] TRACE MessageDatabase - gc candidates:
> [3951, 3973]
> [eckpoint Worker] TRACE MessageDatabase - not removing data
> file: 3951 as contained ack(s) refer to referenced file: [3950, 3951]
> [eckpoint Worker] DEBUG MessageDatabase - Cleanup removing
> the data files: [3973]
> (I assume that is supposed to say '[Checkpoint Worker]', incidentally)
> After the second restart we will see many:
> [0.8.0.200:47300] WARN MessageDatabase - Duplicate message
> add attempt rejected. Destination: MyQueue, Message id:
> ID:node001-58675-1314038640553-0:17:1:1:470776
> Followed by:
> [eckpoint Worker] TRACE MessageDatabase - Last update:
> 3974:13515407, full gc candidates set: [3950, 3951, 3974]
> ...
> [eckpoint Worker] TRACE MessageDatabase - gc candidates after
> dest:1:MyQueue, [3951]
> ...
> [eckpoint Worker] TRACE MessageDatabase - gc candidates:
> [3951]
> [eckpoint Worker] DEBUG MessageDatabase - Cleanup removing
> the data files: [3951]
> This is very similar, if not the same, to AMQ-2955. I have tried setting
> 'useCache=false' but this does not rectify the issue. This could also be a
> similar issue to AMQ-3281.
> I will attach a config. Please advise if you would like me to enable further
> debugging.
> I don't currently have a test harness that replicates this issue and due to
> the fact this is only happening in our production environment, I'm unable to
> verify reliably whether messages are being lost, delayed or if this is purely
> a stats issue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
