Nicolas Grondin created AMQ-7425:
------------------------------------
Summary: Messages in journal persistence don't get deleted upon ack
Key: AMQ-7425
URL: https://issues.apache.org/jira/browse/AMQ-7425
Project: ActiveMQ
Issue Type: Bug
Components: JDBC
Affects Versions: 5.15.11
Reporter: Nicolas Grondin
When using a journal JDBC persistence, we noticed that some messages in the DB
table (activemq_msgs) don't always get deleted upon the messages being
consumed. The adverse effect is that if the broker restarts (hosted in a K8S
pod, so local journal files also lost) the broker will consider all non-deleted
message as still unconsumed and will offer them up for consumption (this was an
issue for us in production)
This seems to be cause by the fact that the persistence adapter does not use
the right ID in the SQL statement to delete the line.
This comes from the fact that sometimes, the messageId has a
"futureOrSequenceLong" of 0 as opposed to the correct ID (from the DB). This is
because the "futureOrSequenceLong" gets set on that message only at the time
the message is being persisted. But in a journal persistence, this happens on a
frequency (5 minutes by default).
If a browse action occurs before the message is persisted, then a copy of the
message is taken, which also copies of the wrong "futureOrSequenceLong". Upon
consumption (assuming no restarts of the broker in the meantime), it is this
copy of the messageId that is used when comes the time to remove the message
from the DB.
The removal code actually caters for the missing "futureOrSequenceLong" in
JDBCMessageStore [257] with this line:
{color:#FF0000} long seq = ack.getLastMessageId().getFutureOrSequenceLong()
!= null ? long seq = ack.getLastMessageId().getFutureOrSequenceLong() !=
null ? (Long) ack.getLastMessageId().getFutureOrSequenceLong() :
persistenceAdapter.getStoreSequenceIdForMessageId(context,
ack.getLastMessageId(), destination)[0];{color}
In which case it will get it from the DB directly.
But the issue is that in the described case, the "futureOrSequenceLong" is not
null, but has a value of 0L.
This is due to this line in JournalMessageStore.addMessage [142] (called when
the message is originally produced)
{color:#FF0000}message.getMessageId().setFutureOrSequenceLong(0l);{color}
I actually think that creating a message on a JournalMessageStore should leave
the "futureOrSequenceLong" null so that later condition checks can pick it up
as such and properly get the sequence from the database.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
