[
https://issues.apache.org/jira/browse/BOOKKEEPER-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114583#comment-13114583
]
Ivan Kelly commented on BOOKKEEPER-37:
--------------------------------------
I've put a first attempt and fixing this at:
https://github.com/ivankelly/bookkeeper/tree/HedwigConnectionManagement
The core of the problem is that when you used to start a connection, you give
it a single PubSubData object, to be transmitted once the connection is up.
However, if multiple publishes are made at the same target in close succession,
then the order in which these will be sent is arbitrary, as multiple
connections will be started. I've tackled this by making sure that once one
connection is started for a host, it will be used by all subsequent. However,
im still not at all sure how to deal with redirect cases.
For example, if you have 3 hubs, and the client connects and sends 3
publishes(P1,P2 & P3) to the VIP the following race can happen.
1. P1 sends to VIP
2. P2 sends to VIP
3. VIP responds to P1, saying that HUB2 is responsible for topic.
4. P1 sends to HUB2
5. P3 sends to HUB2
6. VIP responds to P2, redirect to HUB2
7. P2 sends to HUB2
> Ordering of published messages is not preserved when doing asynchronous
> publication
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-37
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-37
> Project: Bookkeeper
> Issue Type: Bug
> Components: hedwig-client
> Reporter: Matthieu Morel
> Assignee: Matthieu Morel
> Attachments: BOOKKEEPER-37.patch, ClientMessageOrderingTest.java
>
>
> Symptoms:
> - if a publisher sends messages asynchronously through the asyncPublish
> method, the ordering is not preserved in Hedwig
> Example:
> - a publisher sends M1, M2, M3 by invoking the asyncPublish method
> The Hedwig broker may see these messages in the following order: M2, M1, M3
> A subscriber will also see messages as M2, M1, M3
> How to reproduce:
> - see attached test case: synchronous publishing preserves ordering, but
> asynchronous publishing does not.
> The cause of the problem:
> - my understanding is that this is due to asynchronous creation of multiple
> netty channels on the publisher side. There is no ordering since messages are
> not sent through the same channel.
> Suggested solution:
> Some buffering on the publisher side would allow to reuse the same channel
> and maintain ordering.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
