Pavel Moravec created QPID-5642: ----------------------------------- Summary: Message sequence numbers (enabled with qpid.msg_sequence) should be persistent across broker restarts Key: QPID-5642 URL: https://issues.apache.org/jira/browse/QPID-5642 Project: Qpid Issue Type: Bug Components: C++ Broker Affects Versions: 0.26 Reporter: Pavel Moravec Assignee: Pavel Moravec Priority: Minor
The C++ broker supports currently only at-least-once and at-most-once reliability. Therefore duplicate messages may occur, especially in situations involving failovers, reconnects etc. Currently, the broker doesn't offer any specific solution for duplicate detection. For many messaging scenarios, it is crucial to process every message only once to avoid errors. The broker supports message sequencing on exchange level. The message sequencing assigns a sequence number to every message which is routed via the exchange. In theory, this can be used to detect duplicates. Unfortunately, the sequence number isn't stored persistently and is restarted with every restart of the broker (HA cluster) and starts again from 1. That makes the use of the sequence number for duplicate detection quite complicated, especially since the restart of the broker is also the situation when the duplicates may occur. Therefore, it is required to update message sequence numbers (enabled with qpid.msg_sequence) to store/journal after every (pre)route of a durable exchange, in order to persist the sequence numbers accross broker restart. Reproducer: rm -rf /var/lib/qpidd/* output.txt service qpidd restart qpid-config add exchange fanout myFanout --sequence --durable qpid-receive -a myFanout --connection-options="{reconnect:true}" -f --print-content=no --print-header=yes -m 6 > output.txt 2>/dev/null & sleep 1 qpid-send -a myFanout -m 3 service qpidd restart sleep 1 qpid-send -a myFanout -m 3 cat output.txt Current output: Properties: {qpid.msg_sequence:1, sn:1, ts:1394611002529779545, x-amqp-0-10.routing-key:} Properties: {qpid.msg_sequence:2, sn:2, ts:1394611002529879589, x-amqp-0-10.routing-key:} Properties: {qpid.msg_sequence:3, sn:3, ts:1394611002529896423, x-amqp-0-10.routing-key:} Properties: {qpid.msg_sequence:1, sn:1, ts:1394611004142278196, x-amqp-0-10.routing-key:} Properties: {qpid.msg_sequence:2, sn:2, ts:1394611004142340093, x-amqp-0-10.routing-key:} Properties: {qpid.msg_sequence:3, sn:3, ts:1394611004142354743, x-amqp-0-10.routing-key:} Expected output: Properties: {qpid.msg_sequence:1, sn:1, ts:1394611002529779545, x-amqp-0-10.routing-key:} Properties: {qpid.msg_sequence:2, sn:2, ts:1394611002529879589, x-amqp-0-10.routing-key:} Properties: {qpid.msg_sequence:3, sn:3, ts:1394611002529896423, x-amqp-0-10.routing-key:} Properties: {qpid.msg_sequence:4, sn:1, ts:1394611004142278196, x-amqp-0-10.routing-key:} Properties: {qpid.msg_sequence:5, sn:2, ts:1394611004142340093, x-amqp-0-10.routing-key:} Properties: {qpid.msg_sequence:6, sn:3, ts:1394611004142354743, x-amqp-0-10.routing-key:} (sn comes from qpid-send so it will be re-set to 1) -- This message was sent by Atlassian JIRA (v6.2#6252) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org