Hi all,
It turns out this design doesn't work. The reason is that we made the
following assumption: given a client like:
s1 = connect(); send(s1, "msg1"); close(s1);
s2 = connect(); send(s2, "msg2"); close(s2);
And a server like:
char buf[...];
while (true) {
s = accept();
recv(s, buf);
close(s);
}
Our design required that the server-side socket corresponding to "s1"
would always be accepted _before_ the server-side socket for "s2".
That is, that we can rely on the order in which server-side sockets
are accept()'d to infer the order in which client-side sockets are
connect()'d, as long as the client only makes at most one connect() to
a given host at a given time.
It turns out that this isn't the case on either OSX or Linux. I
haven't dug too deeply into the reasons why, but one plausible
explanation is that the kernel is using multiple accept() queues and
dispatching inbound connections to a queue in some unpredictable way.
It seems that implementing ordered delivery will require adding
metadata to the libprocess socket-establishment protocol to give the
recipient enough information to determine which of the inbound
connections from a given libprocess instance is the "newest".
Neil
On Fri, Mar 25, 2016 at 12:50 PM, Neil Conway <neil.con...@gmail.com> wrote:
> A few months ago, there was a dev list thread on whether libprocess
> should provide ordered delivery [1]. The consensus then was that
> libprocess doesn't provide ordered delivery in a few corner cases, but
> that we should fix that behavior to guarantee ordered (but unreliable)
> message delivery.
>
> Please see this design doc for a proposal of how to achieve this:
>
> https://docs.google.com/document/d/1oqv42-GYXfh0TsyXGCBhxakXw8ckXMCnEb0T-ydEyYM/edit#
>
> Comments welcome.
>
> Thanks,
> Neil
>
> [1]
> https://mail-archives.apache.org/mod_mbox/mesos-dev/201511.mbox/%3CCAOW5sYaANyaRD-Mbk7E_VsWwF=xrtcvhszuvmunj4sq8jdt...@mail.gmail.com%3E
