Craig Condit created YARN-8638:
----------------------------------
Summary: Allow linux container runtimes to be pluggable
Key: YARN-8638
URL: https://issues.apache.org/jira/browse/YARN-8638
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 3.2.0
Reporter: Craig Condit
YARN currently supports three different Linux container runtimes (default,
docker, and javasandbox). However, it would be relatively straightforward to
support arbitrary runtime implementations. This would enable easier
experimentation with new and emerging runtime technologies (runc, containerd,
etc.) without requiring a rebuild and redeployment of Hadoop.
This could be accomplished via a simple configuration change:
{code:xml}
<property>
<name>yarn.nodemanager.runtime.linux.allowed-runtimes</name>
<value>default,docker,experimental</value>
</property>
<property>
<name>yarn.nodemanager.runtime.linux.experimental.class</name>
<value>com.somecompany.yarn.runtime.ExperimentalLinuxContainerRuntime</value>
</property>{code}
In this example, {{yarn.nodemanager.runtime.linux.allowed-runtimes}} would now
allow arbitrary values. Additionally,
{{yarn.nodemanager.runtime.linux.\{RUNTIME_KEY}.class}} would indicate the
{{LinuxContainerRuntime}} implementation to instantiate. A no-argument
constructor should be sufficient, as {{LinuxContainerRuntime}} already provides
an {{initialize()}} method.
{{DockerLinuxContainerRuntime.isDockerContainerRequested(Map<String, String>
env)}} and {{JavaSandboxLinuxContainerRuntime.isSandboxContainerRequested()}}
could be generalized to {{isRuntimeRequested(Map<String, String> env)}} and
added to the {{LinuxContainerRuntime}} interface. This would allow
{{DelegatingLinuxContainerRuntime}} to select an appropriate runtime based on
whether that runtime claimed ownership of the current container execution.
For backwards compatibility, the existing values (default,docker,javasandbox)
would continue to be supported as-is. Under the current logic, the evaluation
order is javasandbox, docker, default (with default being chosen if no other
candidates are available). Under the new evaluation logic, pluggable runtimes
would be evaluated after docker and before default, in the order in which they
are defined in the allowed-runtimes list. This will change no behavior on
current clusters (as there would be no pluggable runtimes defined), and
preserves behavior with respect to ordering of existing runtimes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
