Eric Badger commented on YARN-10494:

Hey [~ccondit], thanks for the document. I'm really excited about this tool.

bq. This tool will connect to a Docker repository
Is this a requirement of the tool? I think we definitely need to at least have 
support for local image import from the docker daemon. Ideally we would also 
include support for any OCI-compliant registry, but that is probably outside of 
the scope of the initial design. We just want to make sure to leave the door 
open for that support in the future

Another question: Does this tool support reproducible builds as was added to 
squashfs-tools 4.4 

And as we discussed in the most recent YARN call, we'll need to figure out how 
to run this tool (e.g. a service in the RM, standalone, Hadoop job, etc.) and 
with what user it needs to be run as. There are certainly challenges around 
permissions and security where we won't want arbitrary users creating 
potentially malicious squashfs images that will be blindly loaded by the 
kernel. This is outside of the scope of this specific JIRA, but wanted to 
mention it here for posterity. 

> CLI tool for docker-to-squashfs conversion (pure Java)
> ------------------------------------------------------
>                 Key: YARN-10494
>                 URL: https://issues.apache.org/jira/browse/YARN-10494
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>    Affects Versions: 3.3.0
>            Reporter: Craig Condit
>            Assignee: Craig Condit
>            Priority: Major
>         Attachments: docker-to-squashfs-conversion-tool-design.pdf
> *YARN-9564* defines a docker-to-squashfs image conversion tool that relies on 
> python2, multiple libraries, squashfs-tools and root access in order to 
> convert Docker images to squashfs images for use with the runc container 
> runtime in YARN.
> *YARN-9943* was created to investigate alternatives, as the response to 
> merging YARN-9564 has not been very positive. This proposal outlines the 
> design for a CLI conversion tool in 100% pure Java that will work out of the 
> box.

This message was sent by Atlassian Jira

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to