[jira] [Comment Edited] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally
[
https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763883#comment-16763883
]
Eric Yang edited comment on YARN-9184 at 2/8/19 9:14 PM:
-
[~ebadger] {quote}This isn't necessarily true. You can query the docker
registry to get the hash of any arbitrary image and tag.{quote}
Credential to access docker is stored in ~/.docker/config.json. We need to
aggregate docker configuration to find out protocol, server, port, username,
and secret to login to the remote docker registry. Do we want to let
container-executor venture outside of working directory to aggregation those
information? It seems risky.
[~billie.rinaldi] Pull digest does not guarantee the retrieved image has same
name as user specified image name because the latest tag on repository may have
moved to another target. I don't see a way for container-executor ACL to match
pulled digest images.
We should take job image consistency to a separate JIRA from this one.
YARN-9292 filed to track image consistency per application.
was (Author: eyang):
[~ebadger] {quote}This isn't necessarily true. You can query the docker
registry to get the hash of any arbitrary image and tag.{quote}
Credential to access docker is stored in ~/.docker/config.json. We need to
aggregate docker configuration to find out protocol, server, port, username,
and secret to login to the remote docker registry. Do we want to let
container-executor venture outside of working directory to aggregation those
information? It seems risky.
[~billie.rinaldi] Pull digest does not guarantee the retrieved image has same
name as user specified image name because the latest tag on repository may have
moved to another target. I don't see a way for container-executor ACL to match
pulled digest images.
We should take job image consistency to a separate JIRA from this one.
> Docker run doesn't pull down latest image if the image exists locally
> --
>
> Key: YARN-9184
> URL: https://issues.apache.org/jira/browse/YARN-9184
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Affects Versions: 3.1.0, 3.0.3
>Reporter: Zhaohui Xin
>Assignee: Zhaohui Xin
>Priority: Major
> Attachments: YARN-9184.001.patch, YARN-9184.002.patch,
> YARN-9184.003.patch, YARN-9184.004.patch
>
>
> See [docker run doesn't pull down latest image if the image exists
> locally|https://github.com/moby/moby/issues/13331].
> So, I think we should pull image before run to make image always latest.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally
[
https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758170#comment-16758170
]
Zhaohui Xin edited comment on YARN-9184 at 2/1/19 10:14 AM:
[~liushaohui], Thanks for your comment. If this config is in job side, it will
bring some problems. You can read the comments above for more details.:D
{quote}The images could be significantly different if an image was pushed
halfway through a job. Then some containers would run with the old image and
the rest would run with the newer image.
{quote}
was (Author: uranus):
[~liushaohui], Hi, Thanks for your comment. If this config is in job side, it
will bring some problems. You can read the comments above for more details.:D
{quote}The images could be significantly different if an image was pushed
halfway through a job. Then some containers would run with the old image and
the rest would run with the newer image.
{quote}
> Docker run doesn't pull down latest image if the image exists locally
> --
>
> Key: YARN-9184
> URL: https://issues.apache.org/jira/browse/YARN-9184
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Affects Versions: 3.1.0, 3.0.3
>Reporter: Zhaohui Xin
>Assignee: Zhaohui Xin
>Priority: Major
> Attachments: YARN-9184.001.patch, YARN-9184.002.patch,
> YARN-9184.003.patch, YARN-9184.004.patch
>
>
> See [docker run doesn't pull down latest image if the image exists
> locally|https://github.com/moby/moby/issues/13331].
> So, I think we should pull image before run to make image always latest.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally
[ https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758134#comment-16758134 ] Liu Shaohui edited comment on YARN-9184 at 2/1/19 9:24 AM: --- Nice feature. It will be better if this option can be set per job, Eg: setting DOCKER_PULL_POLICY=always in container env. was (Author: liushaohui): Nice feature. It will better if this option can be set per job. Eg: setting DOCKER_PULL_POLICY=always in container env > Docker run doesn't pull down latest image if the image exists locally > -- > > Key: YARN-9184 > URL: https://issues.apache.org/jira/browse/YARN-9184 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0, 3.0.3 >Reporter: Zhaohui Xin >Assignee: Zhaohui Xin >Priority: Major > Attachments: YARN-9184.001.patch, YARN-9184.002.patch, > YARN-9184.003.patch, YARN-9184.004.patch > > > See [docker run doesn't pull down latest image if the image exists > locally|https://github.com/moby/moby/issues/13331]. > So, I think we should pull image before run to make image always latest. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally
[
https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739968#comment-16739968
]
Zhaohui Xin edited comment on YARN-9184 at 1/11/19 2:49 AM:
Hi [~ebadger], thanks for the review. I added new patch.
Move containerIdStr to above just because _pullImageFromRemote_ function need
it for logging.
{code:java}
pullImageFromRemote(String containerIdStr, String imageName){code}
was (Author: uranus):
Hi [~ebadger], thanks for the review. I added new patch.
Move containerIdStr to above just because _pullImageFromRemote_ function __
need it for logging.
{code:java}
pullImageFromRemote(String containerIdStr, String imageName){code}
> Docker run doesn't pull down latest image if the image exists locally
> --
>
> Key: YARN-9184
> URL: https://issues.apache.org/jira/browse/YARN-9184
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Affects Versions: 3.1.0, 3.0.3
>Reporter: Zhaohui Xin
>Assignee: Zhaohui Xin
>Priority: Major
> Attachments: YARN-9184.001.patch, YARN-9184.002.patch
>
>
> See [docker run doesn't pull down latest image if the image exists
> locally|https://github.com/moby/moby/issues/13331].
> So, I think we should pull image before run to make image always latest.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally
[ https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738960#comment-16738960 ] Zhaohui Xin edited comment on YARN-9184 at 1/10/19 3:42 AM: Hi, [~ebadger], very goog suggestions. I attached patch, can you help me review this? :D was (Author: uranus): [~ebadger], very goog suggestions. I attached patch, can you help me review this? :D > Docker run doesn't pull down latest image if the image exists locally > -- > > Key: YARN-9184 > URL: https://issues.apache.org/jira/browse/YARN-9184 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0, 3.0.3 >Reporter: Zhaohui Xin >Assignee: Zhaohui Xin >Priority: Major > Attachments: YARN-9184.001.patch > > > See [docker run doesn't pull down latest image if the image exists > locally|https://github.com/moby/moby/issues/13331]. > So, I think we should pull image before run to make image always latest. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally
[ https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738960#comment-16738960 ] Zhaohui Xin edited comment on YARN-9184 at 1/10/19 3:42 AM: Hi, [~ebadger], very good suggestions. I attached patch, can you help me review this? :D was (Author: uranus): Hi, [~ebadger], very goog suggestions. I attached patch, can you help me review this? :D > Docker run doesn't pull down latest image if the image exists locally > -- > > Key: YARN-9184 > URL: https://issues.apache.org/jira/browse/YARN-9184 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0, 3.0.3 >Reporter: Zhaohui Xin >Assignee: Zhaohui Xin >Priority: Major > Attachments: YARN-9184.001.patch > > > See [docker run doesn't pull down latest image if the image exists > locally|https://github.com/moby/moby/issues/13331]. > So, I think we should pull image before run to make image always latest. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
