Joseph Wu created MESOS-7858:
--------------------------------
Summary: Launching a nested container with namespace/pid
isolation, with glibc < 2.25, may deadlock the LinuxLauncher and
MesosContainerizer
Key: MESOS-7858
URL: https://issues.apache.org/jira/browse/MESOS-7858
Project: Mesos
Issue Type: Bug
Components: containerization
Affects Versions: 1.3.0
Reporter: Joseph Wu
This bug in glibc (fixed in glibc 2.25) will sometimes cause a child process of
a {{fork}} to {{assert}} incorrectly, if the parent enters a new pid namespace
before forking:
https://sourceware.org/bugzilla/show_bug.cgi?id=15392
https://sourceware.org/bugzilla/show_bug.cgi?id=21386
The LinuxLauncher code happens to do this when launching nested containers:
* The MesosContainerizer process launches a subprocess, with a customized
{{ns::clone}} function as an argument. The thread then basically waits for the
launch to succeed and return a child PID:
https://github.com/apache/mesos/blob/1.3.x/src/slave/containerizer/mesos/linux_launcher.cpp#L495
* A separate thread in the Mesos agent forks and then waits for the grandchild
to report a PID:
https://github.com/apache/mesos/blob/1.3.x/src/linux/ns.hpp#L453
* The child of the fork first enters the namespaces (including a pid namespace)
and then forks a grandchild. The child then calls {{waitpid}} on the
grandchild: https://github.com/apache/mesos/blob/1.3.x/src/linux/ns.hpp#L555
* Due to the glibc bug, the grandchild sometimes never returns from the
{{fork}} here: https://github.com/apache/mesos/blob/1.3.x/src/linux/ns.hpp#L540
According to the glibc bug, we can work around this by:
{quote}
The obvious solution is just to use clone() after setns() and never use fork()
- and one can certainly patch both programs to do so. Nevertheless it would be
nice to see if fork() also worked after setns(), especially since there is no
inherent reason for it not to.
{quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
