[
https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021545#comment-17021545
]
Eric Yang commented on YARN-9292:
---------------------------------
>From today's YARN Docker community meeting, we have decided to abandon this
>patch. There is possibilities that AM can fail over a node which has
>different latest tag than previous node. The frame of reference to latest tag
>is relative to the node where AM is running. If there are inconsistency in
>the cluster, this patch will not solve the consistency problem. Newly spawned
>AM will use a different sha id that maps to latest tag, which leads to
>inconsistent sha id used by the same application.
The ideal design is to have YARN client to discover the latest tag is
referencing, then populate that information to rest of the job. Unfortunately,
there is no connection between YARN and where docker registry might be running.
Hence, it is not possible to implement this proper for YARN and Docker
integration. The community settle on document this wrinkle and try to avoid
using latest tag as best practice.
For Runc container, it will be possible to use HDFS as source of truth to look
up the global hash designation for runc container. YARN client can query HDFS
for the latest tag and it will be consistent on all nodes. This will add some
extra protocol interactions between YARN client and RM to solve this problem by
the ideal design.
> Implement logic to keep docker image consistent in application that uses
> :latest tag
> ------------------------------------------------------------------------------------
>
> Key: YARN-9292
> URL: https://issues.apache.org/jira/browse/YARN-9292
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Major
> Attachments: YARN-9292.001.patch, YARN-9292.002.patch,
> YARN-9292.003.patch, YARN-9292.004.patch, YARN-9292.005.patch,
> YARN-9292.006.patch, YARN-9292.007.patch, YARN-9292.008.patch
>
>
> Docker image with latest tag can run in YARN cluster without any validation
> in node managers. If a image with latest tag is changed during containers
> launch. It might produce inconsistent results between nodes. This is surfaced
> toward end of development for YARN-9184 to keep docker image consistent
> within a job. One of the ideas to keep :latest tag consistent for a job, is
> to use docker image command to figure out the image id and use image id to
> propagate to rest of the container requests. There are some challenges to
> overcome:
> # The latest tag does not exist on the node where first container starts.
> The first container will need to download the latest image, and find image
> ID. This can introduce lag time for other containers to start.
> # If image id is used to start other container, container-executor may have
> problems to check if the image is coming from a trusted source. Both image
> name and ID must be supply through .cmd file to container-executor. However,
> hacker can supply incorrect image id and defeat container-executor security
> checks.
> If we can over come those challenges, it maybe possible to keep docker image
> consistent with one application.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
