[
https://issues.apache.org/jira/browse/YARN-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456432#comment-15456432
]
Shane Kumpf commented on YARN-3854:
-----------------------------------
Today, when credential issues occur during the implicit docker pull, the "image
<image name> not found" error is hidden away in the NM logs. Initially my
thought was that failing fast and exposing the error to the user via the
application logs would be sufficient without credential validation, however,
after testing, docker pull has some differing behaviors regarding various
credential issue scenerios.
If the credentials file is missing or the image does not exist in the registry,
docker pull returns an "image <image name> not found" error (and rc=1). As a
result, it will be difficult to validate if the user supplied a non-existent
image or if it's a missing credentials problem. This deficiency could have been
easily addressed via a log message and some documentation though.
If the credentials exist, but are incorrect (incorrect auth token), it's more
challenging to address as docker falls back to the interactive docker login
prompt. Ideally, docker would provide a command line option to avoid this fall
back to make the behavior consistent, and it may be worth starting that
discussion within the docker community towards adding this option. Otherwise,
we would need to try to handle detecting the difference between a long running
docker pull and a docker pull that is waiting for the user to input credentials.
I'll note that, unfortunately, I'm not aware of any options in the docker
client CLI to validate credentials. It may be possible by resorting to checks
against the registry REST API, but ideally, we want to stick with the docker
client CLI as we've done so far. I'll look into a clean way to address
validation.
> Add localization support for docker images
> ------------------------------------------
>
> Key: YARN-3854
> URL: https://issues.apache.org/jira/browse/YARN-3854
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Sidharta Seethana
> Assignee: Zhankun Tang
> Attachments: YARN-3854-branch-2.8.001.patch,
> YARN-3854_Localization_support_for_Docker_image_v1.pdf,
> YARN-3854_Localization_support_for_Docker_image_v2.pdf,
> YARN-3854_Localization_support_for_Docker_image_v3.pdf
>
>
> We need the ability to localize docker images when those images aren't
> already available locally. There are various approaches that could be used
> here with different trade-offs/issues : image archives on HDFS + docker load
> , docker pull during the localization phase or (automatic) docker pull
> during the run/launch phase.
> We also need the ability to clean-up old/stale, unused images.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
