Ordering in distributed systems is usually a best-effort and only primitive that is required is “happen-before”. I have seen “happen-before” sequencing being implemented in distributed systems using “barriers”. DRBD is a good example of such a system. Another example of that model is the block system inside linux kernel. File systems (especially journal based) require block subsystems to support “write-barriers” so that they can get a “happen-before” primitive.
A simple such system can be implemented by adding a special message type called barrier. And once a barrier is received by the receiver, it will have to acknowledge that packet. Only after the acknowledgement is receiver at the sender side, will the next packet in the order be sent. This means that ordering is on-demand and not forced. I haven’t spent much time analyzing how this model fits into mesos but thought maybe something worth looking into. thanks Jojy > On Nov 15, 2015, at 4:20 AM, Neil Conway <[email protected]> wrote: > > Good point -- following what Erlang and Akka provide (ordering but not > reliability) is probably a reasonable starting-point. > > I suggested earlier that this would require doing our own > retransmission logic (which would be pretty unfortunate), but on > reflection that is obviously not true. We can instead: > > * assign sender-side sequence numbers > * at the receiver, remember the last sequence number we've delivered > from each sender > * drop inbound messages whose sequence number is < the last delivered > message from that sender > > That requires each receiver to keep a sequence number for every > sending PID it has ever seen a message from. For a very large cluster > with a lot of PID churn, that might be a pretty sizable amount of > state. We could improve that incrementally by keeping a single > sequence number for each sending network::Address. I suppose that's > acceptable... > > Neil > > On Sun, Nov 15, 2015 at 12:43 PM, haosdent <[email protected]> wrote: >> As I know, Akka could guaranteed message ordering for per sender-receiver >> pair, but not guaranteed message delivery. Erlang also similar to this. I >> think if to implement own sequencing, acking, and retransmission logic, >> this work nearly to reimplement a TCP stack. To implement a new TCP-like >> stack in TCP looks strange. How about just make sure only have one >> connection between per sender-receiver pair, so TCP could help us >> guaranteed message order.
