On 19/11/16 22:28, Mark Mason (Redacted sender mmlmason for DMARC) wrote:
Hello,
We have written a POSIX multi-threaded TCP-IP Rump application. It runs as a
Rump unikernel guest over Xen 4.4.2. To achieve a build, we followed
instructions provided by the tutorial here:
https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikernels.
The application is very simple. There is one source file, a header file, a
Makefile, and a run script.
There are 4 threads managed by the application. One of threads is started via
a call to pthread_create(). The remaining 3 threads are not started. The
thread function simply creates a socket, connects to a netcat listener, sends a
message to it, closes the socket, and then exits.
After pthread_create() returns, a call to a user-function named local_yield()
is called. It's purpose is to yield execution to the thread just spawned so
that it may execute the TCP/IP function and return.
There are two implementations for local_yield() controlled by a binary flag.
When the flag is 0, the yield is accomplished via a call to sched_yield().
When the flag is 1, the yield is accomplished via a call to nanosleep(). In
the former case, when the flag is 0 and sched_yield() is called, we see the
TCP/IP call to connect() does not return before the call to pthread_join(). In
the latter case, when the flag is 1 and nanosleep() is called, the thread
executes normally, and the call to connect() returns before the call to
pthread_join(), as it should.
We much prefer calling sched_yield() rather then nanosleep() because in our
larger application, it incurs a lot of overhead, slowing down the application
considerably. Is there a fix for the shed_yield() behavior. I have attached
the files so that a Rump developer may reproduce the bug.
Mark,
The bug is in your application: you cannot assume that yielding (or
sleeping) guarantees that a TCP handshake completes. Yielding once
actually guarantees that the handshake will *not* complete under
Rumprun. Just use the pthread/socket interfaces normally, Rumprun takes
care of the rest -- automatically and correctly -- for you.
- antti
