[
https://issues.apache.org/jira/browse/YARN-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376380#comment-15376380
]
Arun Suresh commented on YARN-4888:
-----------------------------------
Thanks for the patch [~subru]
Couple of initial comments:
# In {{AppSchedulingInfo::checkForDeactivation()}}, given that you are
introducing an inner for loop, the deactivate = false should be set only if ALL
requests in the {{mappedRequest.values()}} has {{request.getNumContainers() >
0}} right ?
# In {{AppSchedulingInfo::allocateNodeLocal()}}, That {{rackLocalRequest}} and
{{offRackRequest}} selected from the {{resoureRequestMap}} should correspond to
the same allocationRequestId of the {{nodeLocalRequest}}. Only when you have a
single allocationRequestId, can you guarantee that {{.firstEntry().getValue()}}
will correspond to the same nodeLocalRequest in both cases.
With regard to the getMergeResource() method. I have a feeling we should not be
merging all requests of a {{Priority}}. Consider this alternate approach, which
I feel might allow us to more easily verify if we are breaking any invariants
in the Scheduler:
My intuition is based on the fact that If we agree that, in the absence of an
*allocateRequestId*, we can simulate the same functionality by using a unique
Priority to tie all requests we need to be of the same allocateRequestId....
Then, why not replace the 'Priority' in the
{{AppSchedulingInfo::resourceRequestMap}} with a new type (called
*SchedulerPriority*) which is essentially a composite of *Priority +
allocateRequestId*. If not allocateRequestId is provided, it will default to 0.
If the *SchedulerPriority* is a subclass of *Priority*, then we wont even need
to change any of the APIs.
Thoughts ?
I can help provide a quick prototype patch to verify if this works..
> Changes in RM AppSchedulingInfo for identifying resource-requests explicitly
> ----------------------------------------------------------------------------
>
> Key: YARN-4888
> URL: https://issues.apache.org/jira/browse/YARN-4888
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Subru Krishnan
> Assignee: Subru Krishnan
> Attachments: YARN-4888-v0.patch
>
>
> YARN-4879 puts forward the notion of identifying allocate requests
> explicitly. This JIRA is to track the changes in RM app scheduling data
> structures to accomplish it. Please refer to the design doc in the parent
> JIRA for details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
