[
https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012300#comment-17012300
]
Eric Yang commented on YARN-9292:
---------------------------------
[~ebadger] Thanks for the review. Here are my feedback:
{quote}Doesn't the container know what image it was started with in its
environment?{quote}
ServiceScheduler runs as part of application master for YARN service. YARN AM
is not containerized. Docker command to resolve image digest id happens prior
to any docker container is launched. The lookup for the docker image is from
the node which AM is running. We use the sha256 digest from AM node as
authoritative signature to give the application equal chance of acquiring
docker digest id on any node manager.
{quote}If we don't care about the container and just want to know what the sha
of the image:tag is, then I agree with Chandni Singh that we don't need to use
the containerId.{quote}
Container ID is used by container executor to properly permission the working
directory, generate .cmd file for container-executor binary, and all output and
exit code are stored to the container id directory. Without container ID, we
will need to craft a complete separate path to acquire privileges to launch
docker commands, which is extra code duplication and not follow the security
practice that was baked in place to prevent parameter hijacking. I choose to
follow the existing process to avoid code bloat.
{quote}But if there are many, couldn't that not be the correct one?{quote}
The output given from docker image [image-id] -f "{{.RepoDigests}}" may contain
similar names, like local/centos and centos at the same time due to fuzzy
matching. For loop matches the exact name instead of prefix matching. Hence,
it is always the correct one that is matched.
{quote}I think we should import this instead of including the full path{quote}
Sorry, can't do. There is another import that reference to
org.apache.hadoop.yarn.service.component.Component, which prevent use of the
same name.
{quote}Spacing issues on the operators.{quote}
Checkstyle did not find spacing issue with the existing patch, and the issue is
not clear to me. Care to elaborate?
{quote}The first part of both of these regexes is identical. I think we should
create a subregex and append to it to avoid having to make changes in multiple
places in the future. One if the image followed by a tag and the other is an
image followed by a sha. Should be easy to do.{quote}
Sure, I will compact this to rebase this patch to trunk.
{quote}The else clause syntax doesn't seem to work for me. Did I do something
wrong?{quote}
Yes, unlike C exec, when running docker command on cli, it needs to be quoted
to prevent shell expansion:
{code}docker images --format="{{json .}}" --filter="dangling=flase"{code}. For
clarity, we are using:
{code}docker image [image-id] -f "{{.RepoDigests}}"{code} to find the real
digest hash due to bugs with docker images output.
{quote}Another possible solution is to have the AM get the sha256 hash of the
image that it is running in and then passing that sha to all of the containers
that it starts. This would move the query into the Hadoop cluster itself.{quote}
I think the patch is implementing what you are suggesting that Hadoop query
into the cluster itself via a node manager rest endpoint.
> Implement logic to keep docker image consistent in application that uses
> :latest tag
> ------------------------------------------------------------------------------------
>
> Key: YARN-9292
> URL: https://issues.apache.org/jira/browse/YARN-9292
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Major
> Attachments: YARN-9292.001.patch, YARN-9292.002.patch,
> YARN-9292.003.patch, YARN-9292.004.patch, YARN-9292.005.patch,
> YARN-9292.006.patch
>
>
> Docker image with latest tag can run in YARN cluster without any validation
> in node managers. If a image with latest tag is changed during containers
> launch. It might produce inconsistent results between nodes. This is surfaced
> toward end of development for YARN-9184 to keep docker image consistent
> within a job. One of the ideas to keep :latest tag consistent for a job, is
> to use docker image command to figure out the image id and use image id to
> propagate to rest of the container requests. There are some challenges to
> overcome:
> # The latest tag does not exist on the node where first container starts.
> The first container will need to download the latest image, and find image
> ID. This can introduce lag time for other containers to start.
> # If image id is used to start other container, container-executor may have
> problems to check if the image is coming from a trusted source. Both image
> name and ID must be supply through .cmd file to container-executor. However,
> hacker can supply incorrect image id and defeat container-executor security
> checks.
> If we can over come those challenges, it maybe possible to keep docker image
> consistent with one application.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
