Matteo: thanks for the fast reply! Few follow up questions inline:
On 3/13/23 14:52, Matteo Merli wrote:
Hi Adam,
On Mon, Mar 13, 2023 at 2:24 PM Adam Brenner via users
<users@pulsar.apache.org <mailto:users@pulsar.apache.org>> wrote:
Hello,
I have a few questions regarding Pulsar Async mode and what, if any,
guarantees are made with respect to packet ordering:
1) Does Pulsar offer any guarantee packet ordering in Async send mode?
Let us assume best case with single producer and exclusive access mode.
Yes, messages are guaranteed to be sent in order in both sync & async mode.
Is this because all messages created by the producer are put into a
message queue for that topic? The work flow being the entire queue is
flushed to the broker and the client will continue to re-send until
broker ack's. Is that correct?
Since everything is being operated as a message queue, order is thus
guaranteed?
2) Does the client library do any coordination with Pulsar Broker to
guarantee packet ordering in async send mode?
In case of failures, the client library ensures that all messages
pending in the producer queue are resent in the same order.
This could lead to a sequence of dups, (unless the exactly once mode is
enabled), though you wouldn't see a message out of order.
NOTE: to have a 100% guarantee you need to disable publish timeout
(sendTimeout=0). This is mostly to ease you application code.
In the presence of a timeout, Pulsar client guarantees anyway that any
message present in the queue will also fail, even if not all of them
have already timed out.
The application though could be already trying to publish the next
message. Unless there's proper synchronization. For that, the easier way
is to disable timeouts and let the Pulsar client retrying until a
successful publish.
This is the scenario I was thinking about. Messages 1, 2, 3 are sent and
ack'ed by the broker. Messages 4, 5, 6 are sent but somehow 4 and 5 did
not get ack'ed by the broker but 6 did.
With Pulsar and the async message queue, I'm assuming 4, 5 and 6 would
be sent as one "unit" to the broker so all succeed or do not?
3) Would using batch mode with async send mode help in any way to
guaranteed packet ordering?
Batching will improve the throughput and the overall efficiency, less
CPU usage, and better compression ratio, though it doesn't affect ordering.
--
Matteo Merli
<mme...@apache.org <mailto:mme...@apache.org>>
Thanks,
-Adam
