Hi Sowmya, Sorry for the late reply, do you have any update on this project?
In RocketMQ, one message queue is a FIFO queue natively, so I proposed a simple solution that performs merge sort on multiple queues to improve performance and scalability. While the order issue across producers is difficult, we could assume that the message first arrives the broker should be consumed firstly. But it would be wonderful if you have a real design about the order issue across producers based on the RocketMQ design and the storage structure. Looking forward your design ~ Regards, yukon On Fri, Mar 2, 2018 at 2:50 AM, sowmya s <[email protected]> wrote: > Hello all, > > Adding a few more thoughts on the problem of establishing an order of > messages across producers. > > Consider the Scenario > > Producer-1 produces messages a1, b1 and c1 into a queue Queue1 > Producer-2 produces messages a2, b2 and c2 into queue Queue2. > > If we assume that time(a1) < time(b1) < time(c1) and similarly time(a2) < > time(b2) < time(c2) > > Are the following orders acceptable to the consumer? > > > a1, a2, b1, b2, c1, c2 > > a1, b1, c1, a2, b2, c2 > > a1 b1, b2, a2, a3, b3 > > and an order displayed at one consumer is consistent across all consumers. > > This can be achieved using Total Causal Ordering at the Producer or Queue > level, using Leslie Lamport's clock and synchronization approach. > > For reference is the paper attached, http://lamport.azurewebsites. > net/pubs/time-clocks.pdf > > > -- > Regards, > > Sowmya > > On Sun, Feb 25, 2018 at 7:37 PM, sowmya s <[email protected]> wrote: > >> Hello, >> >> I'm trying to work on the issue ROCKETMQ-122 >> <https://issues.apache.org/jira/browse/ROCKETMQ-122?filter=12343065> as >> a part of Google Summer of Code 2018. I've been spending some time to >> understand the system, architecture and the existing Messaging Patterns. >> I still have a few questions and would like to clarify my assumptions. >> >> - Is the current FIFO order example limited to one message queue per >> producer? Can the producer send the same message to multiple queues? Will >> the consumers of the queues be able to read messages in Order? >> - Can I assume that each producer will send messages to one queue? >> - Global Order is to be identified across all GlobalOrderedProducer >> (a new producer that is to be used for global order) instances that are >> running. >> - I think using a global clock can help establish the order between 2 >> or more producers, however using some form of vector clock might also >> help identify the global order of messages between the producers. >> - A GlobalOrderedConsumer ( consumer that knows how to read globally >> ordered messages) can then compare messages across all message queues from >> the corresponding producers and extract the messages. [ this is the >> approach recommended by yukon on the issue page ] >> - We can also potentially have another layer in the Message Queue >> which accumulates all messages sent from producers and provides one >> ordered >> message queue for consumers to read from. >> >> Thank you for your patience and please let me know if my understanding of >> the problem and the assumptions are right. >> >> Best Regards, >> Sowmya >> >> > > > -- > Regards, > > Sowmya > >
