[
https://issues.apache.org/jira/browse/YARN-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang updated YARN-9669:
----------------------------
Issue Type: New Feature (was: Improvement)
> YARN Container phase 3
> ----------------------
>
> Key: YARN-9669
> URL: https://issues.apache.org/jira/browse/YARN-9669
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Eric Yang
> Priority: Major
>
> Docker container can run on Hadoop cluster fine today in Hadoop 3.1.1+
> release and trunk code base. We are getting feedback from potential
> customers for advance features that may take Docker support to the next
> level. The frequently requested features are:
> h2. Improve cgroups support
> h3. Support systemd based cgorup
> YARN-9660 provides instruction to setup cgroups fs to control Docker
> container with cgroup. The next step is to support Docker cgroups with
> systemd natively. This will reduce some configuration steps on Docker side
> to make the user experience more seamless.
> h3. Disk throttling
> Docker supports IO throttling with several docker parameters:
> {code}--device-read-bps=""
> --device-write-bps=""
> --device-read-iops=""
> --device-write-iops=""{code}
> This looks like a good feature for YARN to specify IO throughput for stress
> test simulation.
> h2. Docker volume support
> Docker can create data container of fixed size. This is useful to limit disk
> usage for containers. This feature provides resource management to data, and
> ability to create docker volume and anchor docker containers to colocate with
> data container.
> h2. Protection against overlay network spike
> Badly designed docker container can cause system to become non-responsive, if
> network spikes occurs or forking uncontrollably. Some protection maybe
> useful to guard against network flood and fork bomb. Active prevention can
> shutdown the crashing docker container before system hangs.
> h2. Docker-compose support
> A few people have expressed interests to see the possibility to consume
> docker compose yaml file, and start a dock swarm like cluster. YARN service
> REST API can have alignment in this area. In term, support docker swarm like
> cluster in Hadoop.
> h2. OCI Runtime support
> Deeper integration with Runc can provide more standard API integration with
> Docker core engine. This is continuation work to follow up on YARN-9560 and
> associated JIRAs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
