[
https://issues.apache.org/jira/browse/YARN-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665360#comment-16665360
]
Eric Yang commented on YARN-8927:
---------------------------------
[~tangzhankun] [~ebadger] I agreed on using "library" keyword to allow both
local and Docker hub is clean and simple. It will satisfy 78% of users. One
minor detail that we don't agree on is whether we should precheck if image
exist locally to prevent docker pull of image to give local image precedence
over public image on Docker hub. In more extreme case, don't ever use Docker
hub image, if local image is already available. The best way to solve the
minor detail is to have another switch explicitly trust local image. If this
option is off, public image will get pulled from docker hub. This gives the
most fresh latest image from docker hub. When this option is on, local image
take precedence to be used and never get overwritten. This seems like the best
way to solve ambiguity.
I am in favor of the 78% "library" keyword option because without deploying
trusted registry, there is a lot to gamble on local image consistency. There
is probably company out there that have developed mechanism to push local
images without using a repository. I can't help to think that trust local
image option is a snow flake that will fade away when more intelligence is
built into YARN's docker pull strategy.
> Better handling of "docker.trusted.registries" in container-executor's
> "trusted_image_check" function
> -----------------------------------------------------------------------------------------------------
>
> Key: YARN-8927
> URL: https://issues.apache.org/jira/browse/YARN-8927
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Zhankun Tang
> Assignee: Zhankun Tang
> Priority: Major
>
> There are some missing cases that we need to catch when handling
> "docker.trusted.registries".
> The container-executor.cfg configuration is as follows:
> {code:java}
> docker.trusted.registries=tangzhankun,ubuntu,centos{code}
> It works if run DistrubutedShell with "tangzhankun/tensorflow"
> {code:java}
> "yarn ... -shell_env YARN_CONTAINER_RUNTIME_TYPE=docker -shell_env
> YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=tangzhankun/tensorflow
> {code}
> But run a DistrubutedShell job with "centos", "centos[:tagName]", "ubuntu"
> and "ubuntu[:tagName]" fails:
> The error message is like:
> {code:java}
> "image: centos is not trusted"
> {code}
> We need better handling the above cases.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
