[
https://issues.apache.org/jira/browse/YARN-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16869634#comment-16869634
]
Eric Badger commented on YARN-9014:
-----------------------------------
[~ccondit], uploaded v002 of the design document. It includes more details
regarding the image tag -> manifest and manifest -> resources plugins as well
as some refined details on how exactly the runtime works. Notably we don't
unmount the layers when the container is finished anymore. Instead, we use a
thread to clean them up periodically so that we don't have to remount so often.
> OCI/squashfs container runtime
> ------------------------------
>
> Key: YARN-9014
> URL: https://issues.apache.org/jira/browse/YARN-9014
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Jason Lowe
> Assignee: Eric Badger
> Priority: Major
> Labels: Docker
> Attachments: OciSquashfsRuntime.v001.pdf,
> RuncContainerRuntime.v002.pdf
>
>
> This JIRA tracks a YARN container runtime that supports running containers in
> images built by Docker but the runtime does not use Docker directly, and
> Docker does not have to be installed on the nodes. The runtime leverages the
> [OCI runtime standard|https://github.com/opencontainers/runtime-spec] to
> launch containers, so an OCI-compliant runtime like {{runc}} is required.
> {{runc}} has the benefit of not requiring a daemon like {{dockerd}} to be
> running in order to launch/control containers.
> The layers comprising the Docker image are uploaded to HDFS as
> [squashfs|http://tldp.org/HOWTO/SquashFS-HOWTO/whatis.html] images, enabling
> the runtime to efficiently download and execute directly on the compressed
> layers. This saves image unpack time and space on the local disk. The image
> layers, like other entries in the YARN distributed cache, can be spread
> across the YARN local disks, increasing the available space for storing
> container images on each node.
> A design document will be posted shortly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
