Omkar Vinit Joshi created YARN-574:
--------------------------------------
Summary: PrivateLocalizer does not support parallel resource
download via ContainerLocalizer
Key: YARN-574
URL: https://issues.apache.org/jira/browse/YARN-574
Project: Hadoop YARN
Issue Type: Bug
Reporter: Omkar Vinit Joshi
Assignee: Omkar Vinit Joshi
At present private resources will be downloaded in parallel only if multiple
containers request the same resource. However otherwise it will be serial. The
protocol between PrivateLocalizer and ContainerLocalizer supports multiple
downloads however it is not used and only one resource is sent for downloading
at a time.
I think we can increase / assure parallelism (even for single container
requesting resource) for private/application resources by making multiple
downloads per ContainerLocalizer.
Total Parallelism before
= number of threads allotted for PublicLocalizer [public resource] + number of
containers[private and application resource]
Total Parallelism after
= number of threads allotted for PublicLocalizer [public resource] + number of
containers * max downloads per container [private and application resource]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
