[ 
https://issues.apache.org/jira/browse/YARN-574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Shao updated YARN-574:
----------------------------
    Release Note: YARN-574. Allow parallel download of resources in 
PrivateLocalizer. Contributed by Zheng Shao.  (was: YARN-543. Allow parallel 
download of resources in PrivateLocalizer. Contributed by Zheng Shao.)

> PrivateLocalizer does not support parallel resource download via 
> ContainerLocalizer
> -----------------------------------------------------------------------------------
>
>                 Key: YARN-574
>                 URL: https://issues.apache.org/jira/browse/YARN-574
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 2.6.0, 2.8.0, 2.7.1
>            Reporter: Omkar Vinit Joshi
>            Assignee: Omkar Vinit Joshi
>         Attachments: YARN-574.1.patch
>
>
> At present private resources will be downloaded in parallel only if multiple 
> containers request the same resource. However otherwise it will be serial. 
> The protocol between PrivateLocalizer and ContainerLocalizer supports 
> multiple downloads however it is not used and only one resource is sent for 
> downloading at a time.
> I think we can increase / assure parallelism (even for single container 
> requesting resource) for private/application resources by making multiple 
> downloads per ContainerLocalizer.
> Total Parallelism before
> = number of threads allotted for PublicLocalizer [public resource] + number 
> of containers[private and application resource]
> Total Parallelism after
> = number of threads allotted for PublicLocalizer [public resource] + number 
> of containers * max downloads per container [private and application resource]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to