Pavel Moravec created QPID-5642:
-----------------------------------
Summary: Message sequence numbers (enabled with qpid.msg_sequence)
should be persistent across broker restarts
Key: QPID-5642
URL: https://issues.apache.org/jira/browse/QPID-5642
Project: Qpid
Issue Type: Bug
Components: C++ Broker
Affects Versions: 0.26
Reporter: Pavel Moravec
Assignee: Pavel Moravec
Priority: Minor
The C++ broker supports currently only at-least-once and at-most-once
reliability. Therefore duplicate messages may occur, especially in situations
involving failovers, reconnects etc. Currently, the broker doesn't offer any
specific solution for duplicate detection. For many messaging scenarios, it is
crucial to process every message only once to avoid errors.
The broker supports message sequencing on exchange level. The message
sequencing assigns a sequence number to every message which is routed via the
exchange. In theory, this can be used to detect duplicates. Unfortunately, the
sequence number isn't stored persistently and is restarted with every restart
of the broker (HA cluster) and starts again from 1. That makes the use of the
sequence number for duplicate detection quite complicated, especially since the
restart of the broker is also the situation when the duplicates may occur.
Therefore, it is required to update message sequence numbers (enabled with
qpid.msg_sequence) to store/journal after every (pre)route of a durable
exchange, in order to persist the sequence numbers accross broker restart.
Reproducer:
rm -rf /var/lib/qpidd/* output.txt
service qpidd restart
qpid-config add exchange fanout myFanout --sequence --durable
qpid-receive -a myFanout --connection-options="{reconnect:true}" -f
--print-content=no --print-header=yes -m 6 > output.txt 2>/dev/null &
sleep 1
qpid-send -a myFanout -m 3
service qpidd restart
sleep 1
qpid-send -a myFanout -m 3
cat output.txt
Current output:
Properties: {qpid.msg_sequence:1, sn:1, ts:1394611002529779545,
x-amqp-0-10.routing-key:}
Properties: {qpid.msg_sequence:2, sn:2, ts:1394611002529879589,
x-amqp-0-10.routing-key:}
Properties: {qpid.msg_sequence:3, sn:3, ts:1394611002529896423,
x-amqp-0-10.routing-key:}
Properties: {qpid.msg_sequence:1, sn:1, ts:1394611004142278196,
x-amqp-0-10.routing-key:}
Properties: {qpid.msg_sequence:2, sn:2, ts:1394611004142340093,
x-amqp-0-10.routing-key:}
Properties: {qpid.msg_sequence:3, sn:3, ts:1394611004142354743,
x-amqp-0-10.routing-key:}
Expected output:
Properties: {qpid.msg_sequence:1, sn:1, ts:1394611002529779545,
x-amqp-0-10.routing-key:}
Properties: {qpid.msg_sequence:2, sn:2, ts:1394611002529879589,
x-amqp-0-10.routing-key:}
Properties: {qpid.msg_sequence:3, sn:3, ts:1394611002529896423,
x-amqp-0-10.routing-key:}
Properties: {qpid.msg_sequence:4, sn:1, ts:1394611004142278196,
x-amqp-0-10.routing-key:}
Properties: {qpid.msg_sequence:5, sn:2, ts:1394611004142340093,
x-amqp-0-10.routing-key:}
Properties: {qpid.msg_sequence:6, sn:3, ts:1394611004142354743,
x-amqp-0-10.routing-key:}
(sn comes from qpid-send so it will be re-set to 1)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
