[
https://issues.apache.org/jira/browse/YARN-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907493#comment-16907493
]
Eric Badger commented on YARN-9562:
-----------------------------------
{noformat:title=Required Configs}
<property>
<name>yarn.nodemanager.runtime.linux.allowed-runtimes</name>
<value>docker,default,runc</value>
</property>
<property>
<name>yarn.nodemanager.runtime.linux.runc.allowed-images</name>
<value>$Name of image tag</value>
</property>
<property>
<name>yarn.nodemanager.runtime.linux.runc.image-name</name>
<value>$Name of image tag</value>
</property>
<property>
<name>yarn.nodemanager.runtime.linux.runc.image-toplevel-dir</name>
<value>/runc-root</value>
</property>
{noformat}
{noformat:title=Doesn't need to be nscd, but you need some strategy to make
sure that your username can be resolved in the container via its uid.}
<property>
<name>yarn.nodemanager.runtime.linux.runc.default-rw-mounts</name>
<value>/var/run/nscd:/var/run/nscd</value>
</property>
{noformat}
{noformat:title=At least 1 of the following 2 configs needs to be set. If you
use the docker-to-squash tool from YARN 9564 then you should only need to set
the hdfs hash file}
<property>
<name>yarn.nodemanager.runtime.linux.runc.image-tag-to-manifest-plugin.local-hash-file</name>
<value>/home/ebadger/image-tag-to-hash</value>
</property>
<property>
<name>yarn.nodemanager.runtime.linux.runc.image-tag-to-manifest-plugin.hdfs-hash-file</name>
<value>/runc-root/image-tag-to-hash</value>
</property>
{noformat}
[~eyang], I went ahead and moved the toplevel config into YarnConfiguration
since it is relevant to multiple pieces of the runc runtime. So the config key
is now {{yarn.nodemanager.runtime.linux.runc.image-toplevel-dir}}. I have
copied and edited the original list of configs that I gave. Additionally, I
have tested YARN-9562.004.patch with YARN-9561.003.patch and
YARN-9564.003.patch together.
{noformat}
export
vars="YARN_CONTAINER_RUNTIME_TYPE=runc,YARN_CONTAINER_RUNTIME_RUNC_IMAGE=hadoop/rhel6:current";
$HADOOP_HOME/bin/hadoop jar
$HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.*-tests.jar
sleep -Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME=$HADOOP_HOME"
-Dmapreduce.admin.user.env="HADOOP_MAPRED_HOME=$HADOOP_HOME"
-Dyarn.app.mapreduce.am.env=$vars -Dmapreduce.map.env=$vars
-Dmapreduce.reduce.env=$vars -mt 1 -rt 1 -m 1 -r 1
{noformat}
I successfully ran the above sleep job (after running docker-to-squash.py on
hadoop/rhel6:current). Also note that the image that you use will need to
include java and any other Hadoop dependencies. We have created our own custom
image that is based off of rhel6 that add java, zstandard, lzo, and some other
packages.
> Add Java changes for the new RuncContainerRuntime
> -------------------------------------------------
>
> Key: YARN-9562
> URL: https://issues.apache.org/jira/browse/YARN-9562
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Badger
> Assignee: Eric Badger
> Priority: Major
> Attachments: YARN-9562.001.patch, YARN-9562.002.patch,
> YARN-9562.003.patch, YARN-9562.004.patch
>
>
> This JIRA will be used to add the Java changes for the new
> RuncContainerRuntime. This will work off of YARN-9560 to use much of the
> existing DockerLinuxContainerRuntime code once it is moved up into an
> abstract class that can be extended.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
