[
https://issues.apache.org/jira/browse/YARN-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590376#comment-16590376
]
Eric Badger commented on YARN-8638:
-----------------------------------
bq. It will be very clean if we can make runc/containerd to be a separate
ContainerRuntime implementation. But not sure that if all the common logics
like ContainerLaunch/LinuxContainerExecutor works fine for containerd/runc. If
involved changes required, we may have to consider to move the abstraction to
ContainerExecutor level, etc.
I think that the only thing that we will need to change outside of the runtime
is the Docker lifecycle changes that were made in YARN-5366.
{noformat}
+ @Override
+ public boolean isRuntimeRequested(Map<String, String> env) {
+ return !DockerLinuxContainerRuntime.isDockerContainerRequested(env);
+ }
+
{noformat}
I understand why you made this change, since it's just replacing the code that
was already there. But that logic only worked because of the control flow where
we knew if the user didn't explicitly ask for docker that they would be asking
for default. But now we have to ask if they're asking for docker or a pluggable
runtime. So the function return value doesn't really make sense outside of the
logic that its being used in right now.
Other than that, the code lgtm
> Allow linux container runtimes to be pluggable
> ----------------------------------------------
>
> Key: YARN-8638
> URL: https://issues.apache.org/jira/browse/YARN-8638
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Affects Versions: 3.2.0
> Reporter: Craig Condit
> Assignee: Craig Condit
> Priority: Minor
> Attachments: YARN-8638.001.patch, YARN-8638.002.patch
>
>
> YARN currently supports three different Linux container runtimes (default,
> docker, and javasandbox). However, it would be relatively straightforward to
> support arbitrary runtime implementations. This would enable easier
> experimentation with new and emerging runtime technologies (runc, containerd,
> etc.) without requiring a rebuild and redeployment of Hadoop.
> This could be accomplished via a simple configuration change:
> {code:xml}
> <property>
> <name>yarn.nodemanager.runtime.linux.allowed-runtimes</name>
> <value>default,docker,experimental</value>
> </property>
>
> <property>
> <name>yarn.nodemanager.runtime.linux.experimental.class</name>
> <value>com.somecompany.yarn.runtime.ExperimentalLinuxContainerRuntime</value>
> </property>{code}
>
> In this example, {{yarn.nodemanager.runtime.linux.allowed-runtimes}} would
> now allow arbitrary values. Additionally,
> {{yarn.nodemanager.runtime.linux.\{RUNTIME_KEY}.class}} would indicate the
> {{LinuxContainerRuntime}} implementation to instantiate. A no-argument
> constructor should be sufficient, as {{LinuxContainerRuntime}} already
> provides an {{initialize()}} method.
> {{DockerLinuxContainerRuntime.isDockerContainerRequested(Map<String, String>
> env)}} and {{JavaSandboxLinuxContainerRuntime.isSandboxContainerRequested()}}
> could be generalized to {{isRuntimeRequested(Map<String, String> env)}} and
> added to the {{LinuxContainerRuntime}} interface. This would allow
> {{DelegatingLinuxContainerRuntime}} to select an appropriate runtime based on
> whether that runtime claimed ownership of the current container execution.
> For backwards compatibility, the existing values (default,docker,javasandbox)
> would continue to be supported as-is. Under the current logic, the evaluation
> order is javasandbox, docker, default (with default being chosen if no other
> candidates are available). Under the new evaluation logic, pluggable runtimes
> would be evaluated after docker and before default, in the order in which
> they are defined in the allowed-runtimes list. This will change no behavior
> on current clusters (as there would be no pluggable runtimes defined), and
> preserves behavior with respect to ordering of existing runtimes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
