Wei Chen created YARN-5914:
------------------------------
Summary: NodeManager will report "Error: No such image, container
or task" when DockerContainerLauncher launches a container
Key: YARN-5914
URL: https://issues.apache.org/jira/browse/YARN-5914
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.6.0
Environment: Ubuntu 16.04, Docker1.12.1,
Spark-2.0.1,Spark-1.6.2,Hadoop2.6.0,Hadoop-2.7.1
Reporter: Wei Chen
Hi, all
I have tested MapReduce and Spark(1.6.2,2.0.1) with Docker execution enabled.
I found DockerContainerExecutor will report "Error: No such image, container or
task" each time it launch a task container. Then I checked the
docker_container_executor_session.sh
and find this:
echo `/usr/bin/docker inspect --format {{.State.Pid}}
container_1479428705108_0002_01_000001` >
/home/cwei/project/hadoop-2.7.3/yarn-temp/nm-local-dir/nmPrivate/application_1479428705108_0002/container_1479428705108_0002_01_000001/container_1479428705108_0002_01_000001.pid.tmp
/bin/mv -f
/home/cwei/project/hadoop-2.7.3/yarn-temp/nm-local-dir/nmPrivate/application_1479428705108_0002/container_1479428705108_0002_01_000001/container_1479428705108_0002_01_000001.pid.tmp
/home/cwei/project/hadoop-2.7.3/yarn-temp/nm-local-dir/nmPrivate/application_1479428705108_0002/container_1479428705108_0002_01_000001/container_1479428705108_0002_01_000001.pid
/usr/bin/docker run --memory=1024m --memory-swap -1 -it --net=host --name
container_1479428705108_0002_01_000001 -v
/home/cwei/project/hadoop-2.7.3/yarn-temp/nm-local-dir:/home/cwei/project/hadoop-2.7.3/yarn-temp/nm-local-dir
-v
/home/cwei/project/hadoop-2.7.3/logs/userlogs:/home/cwei/project/hadoop-2.7.3/logs/userlogs
-v
/home/cwei/project/hadoop-2.7.3/yarn-temp/nm-local-dir/usercache/cwei/appcache/application_1479428705108_0002/container_1479428705108_0002_01_000001:/home/cwei/project/hadoop-2.7.3/yarn-temp/nm-local-dir/usercache/cwei/appcache/application_1479428705108_0002/container_1479428705108_0002_01_000001
sequenceiq/hadoop-docker:2.7.1 bash
"/home/cwei/project/hadoop-2.7.3/yarn-temp/nm-local-dir/usercache/cwei/appcache/application_1479428705108_0002/container_1479428705108_0002_01_000001/launch_container.sh"
Since `/usr/bin/docker inspect --format {{.State.Pid}}
container_1479428705108_0002_01_000001` is called before the container is
launched by calling `docker run...` , so it always cause this error log message.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
