Andrew Schwartzmeyer created MESOS-8668:

             Summary: Transition libprocess on Windows to use the Thread Pool 
                 Key: MESOS-8668
             Project: Mesos
          Issue Type: Epic
          Components: libprocess
         Environment: Windows 10
            Reporter: Andrew Schwartzmeyer

This epic covers the work necessary to use the Windows [Thread Pool 
 instead of libev or libevent on Windows.

The design of libprocess closely mimics the built-in Thread Pool API, using I/O 
Completion Ports for asynchronous I/O. It makes sense to use this API because 
the semantics of libprocess map 1-1 to the Windows async asynchronous support.

Transitioning to IOCP is necessary to implement non-blocking pipes, as the 
existing eventing libraries (e.g. libevent) only implement asynchronous sockets 
on Windows; but there are many instances of Mesos requiring asynchronous pipes 
too. As such, the existing use of libevent is fundamentally broken.

Specifically, this was first discovered in test 
{{HungDockerTest.ROOT_DOCKER_InspectHungDuringPull}}, but is also evidenced 
through {{IOTest.Read}} and others which use non-blocking pipes.

The alternative eventing libraries libev and libuv were considered, but we are 
choosing not to use them as they do not implement non-blocking pipes 
sufficiently for Windows. libev is not easy to build on Windows, and does not 
support IOCP. While libevent uses IOCP for reading and writing sockets, it uses 
{{select()}} to detect and dispatch events (which results in poor performance), 
and does not use support IOCP for pipes. libuv claims to support IOCP on 
Windows, but they do not support [named 
pipes|] and only supportes 
[synchronous pipes|]. Moreover, it is 
simpler to use the Thread Pool API in libprocess directly, instead of using a 
library to abstract over that API, as libprocess already provides the 
abstraction we're seeking for Mesos.

Additionally, by using the built-in Windows IOCP support, we can eventually 
support SSL using the Windows crypto libraries, instead of requiring OpenSSL on 
Windows (the current case for libprocess).

This message was sent by Atlassian JIRA

Reply via email to