[
https://issues.apache.org/jira/browse/MESOS-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jie Yu updated MESOS-3596:
--------------------------
Description:
Package distribution in a large scale cluster is a non-trivial problem. This
problem will get worse once we start to support container images (MESOS-2840).
Imaging O(10000) agents simultaneously fetching an O(GB) container image from
HDFS which only has 3 replicas for the given image file.
It'll be great if Mesos can support fetching packages using a P2P protocol
(e.g., BitTorrent). We can also configure the P2P clients so that they are
locality aware (e.g., fetching from peers in the same rack).
was:
Package distribution in a large scale cluster is a non-trivial problem. This
problem will get worse once we start to support container images (MESOS-2840).
Imaging O(10000) agents simultaneously fetching an O(GB) container image from
HDFS which only has 3 replicas for the given image file.
It'll be great if Mesos can support fetching packages using a P2P protocol
(e.g., BitTorrent).
> Supporting fetching using P2P protocol.
> ---------------------------------------
>
> Key: MESOS-3596
> URL: https://issues.apache.org/jira/browse/MESOS-3596
> Project: Mesos
> Issue Type: Wish
> Reporter: Jie Yu
>
> Package distribution in a large scale cluster is a non-trivial problem. This
> problem will get worse once we start to support container images
> (MESOS-2840). Imaging O(10000) agents simultaneously fetching an O(GB)
> container image from HDFS which only has 3 replicas for the given image file.
> It'll be great if Mesos can support fetching packages using a P2P protocol
> (e.g., BitTorrent). We can also configure the P2P clients so that they are
> locality aware (e.g., fetching from peers in the same rack).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
