On Fri, Jul 21, 2017 at 8:30 AM, jbar <[email protected]> wrote: > >When you say the messages "are indeed purged," do you mean to say (in more > >words, to make sure we're clear) "When I reconnect my consumer that has > >been offline for longer than the TTL I set, all messages older than the > TTL > >are expired (as confirmed by an increase in the ExpiredCount JMX attribute > >on the queue in question or by some similar mechanism)"? > > In case of my project, the consumer is a big message collecting system with > gui, so I literally see that there are no new messages after reconnecting > it > (even though they were sent by the producer and enqueued before) in both > cases, with and without given TTL. As soon as the consumer goes online, the > pending messages counter starts decreasing, and the number of dequeued > messages starts going up. >
I would expect the ExpiredCount JMX counter to increase, not the DequeueCount one. That implies that the messages are in fact being sent to the consumer even though they're expired. If so, you'd see that in the JMX stats (the InFlightCount would be non-zero). Do you see evidence that the messages are being dispatched to the consumer in the case of a consumer that was offline and only reconnects after all messages are expired? > >What about your test procedure causes them to be clearly pending for > >more than half a second? Are you saying you're publishing messages far > >faster than your consumer can process them and therefore there's a large > >backlog? > > Yes. > > >How are you determining that message expiration isn't happening? > > I just see 10k delivered messages on a consumer side. > Does "delivered" mean "successfully consumed by your application," or something else? I want to make sure I'm not misinterpreting what you're saying. > >With this scenario, message expiration could be taking place on either the > >broker or the consumer > > With my scenario messages are pending for more than 500 ms (it takes a > dozen > or so seconds to dequeue them) therefore I guess it should take place on > broker. > I agree with that analysis, based on that information. While messages are still being produced, it could be taking place in both locations (though only if the consumer finished consuming the first message while messages are still being produced), but 500 ms after the last message is produced, expiration should happen in the broker for all messages except the ones in the consumer's prefetch buffer. Since your prefetch is not set for this topic, it will use the default value of 1000 per consumer, so I'd expect to see ~1000 messages expired on the consumer side and ~9000 expired on the broker. BTW, your process is only using a single consumer on this queue, right? You're not doing anything like running 10 consumers in parallel, or running a threadpool of 10,000 threads that each process a single message at a time, such that all of the messages could dispatched before they expire, are you? I assume not, but just need to ask. > >But I would expect that 1/2 > >second after your last message is produced, your consumer would receive no > >further messages to process. > > That's excactly what I'm trying to achieve. > > >are the clocks synchronized on whatever machines are > >running your producers, consumers, and brokers? > > In my tests the producer, consumer and broker are running on the same > machine, therefore that's not the case since they're using the same system > time. > OK, thanks for confirming. > -- > View this message in context: http://activemq.2283324.n4. > nabble.com/ActiveMQ-Messages-are-not-discarded-from-queue- > when-they-re-expired-tp4728731p4728770.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > Can you please use the queue browser page in the web console to view the headers of a message *before it expires* so we can confirm that the JMSExpiration header is in fact being set (and set to a correct value that's greater than the creation time by the TTL value)? For this to work, you might have to extend the TTL window, because with only half a second, your odds of accessing the message via the web console are pretty slim. I assume that you'll find that the value of that header is correct (meaning that the problem is in the broker and/or the consumer rather than the producer), but let's confirm that assumption so we can rule out part of the potential problem space. Tim
