[
https://issues.apache.org/jira/browse/BOOKKEEPER-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618672#comment-13618672
]
Jiannan Wang commented on BOOKKEEPER-37:
----------------------------------------
I have an idea for this issue: if client side supplys an order information for
pub then we can resolve it more graceful. In detail, the publisher gives a
relative and consecutive sequence id to specify the order of messages, and in
Hub messages are collected by this consecutive order before persistenting
messages.
Basically, this idea lets client determine the message order so we get around
the possible inconsistent ordering issue during whole publish pipeline. The
only concern I have is if there is one message pub failed, it will cause
subsequent messages been blocked until the message are re-pub. In this case,
the pub rate and the delivery process will be impacted. However, I think it's
not easy to avoid if user require completed ordering guarantee.
I haven't think of the details about this idea, but here are two items in my
mind now:
* Limit the pub message window size in server side.
* If persistence storage can support pre-allocation space for message, pub
rate would not been largely affected.
> Ordering of published messages is not preserved when doing asynchronous
> publication
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-37
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-37
> Project: Bookkeeper
> Issue Type: Bug
> Components: hedwig-client
> Affects Versions: 4.0.0
> Reporter: Matthieu Morel
> Assignee: Matthieu Morel
> Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: BOOKKEEPER-37.patch, ClientMessageOrderingTest.java
>
>
> Symptoms:
> - if a publisher sends messages asynchronously through the asyncPublish
> method, the ordering is not preserved in Hedwig
> Example:
> - a publisher sends M1, M2, M3 by invoking the asyncPublish method
> The Hedwig broker may see these messages in the following order: M2, M1, M3
> A subscriber will also see messages as M2, M1, M3
> How to reproduce:
> - see attached test case: synchronous publishing preserves ordering, but
> asynchronous publishing does not.
> The cause of the problem:
> - my understanding is that this is due to asynchronous creation of multiple
> netty channels on the publisher side. There is no ordering since messages are
> not sent through the same channel.
> Suggested solution:
> Some buffering on the publisher side would allow to reuse the same channel
> and maintain ordering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
