[
https://issues.apache.org/jira/browse/YARN-7903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357923#comment-16357923
]
Steven Rand commented on YARN-7903:
-----------------------------------
Agreed that having a concept of delay scheduling for preemption is a good idea
and would help with both JIRAs. We might be able to use
{{FSAppAttempt.getAllowedLocalityLevel}} or
{{FSAppAttempt.getAllowedLocalityLevelByTime}}, since those already have logic
for checking whether the app has waited longer than the threshold for requests
with some {{SchedulerKey}} (which seems to really just mean priority?). I'll
defer to others though on whether it makes sense for delay logic in preemption
to match delay logic in allocation -- possibly there are differences between
the two that call for separate logic.
I'm also quite confused as to how we should be thinking about different RRs
from the same app at the same priority. I spent some time digging through the
code today, but don't really understand it yet. There are a couple pieces of
code I found that deal with deduping/deconflicting RRs, but I wasn't sure how
to interpret them:
* {{VisitedResourceRequestTracker}} seems to consider RRs with the same
priority and capability to be logically the same
* {{AppSchedulingInfo#internalAddResourceRequests}} seems to consider RRs with
the same {{SchedulerRequestKey}} and resourceName to be logically the same
> Method getStarvedResourceRequests() only consider the first encountered
> resource
> --------------------------------------------------------------------------------
>
> Key: YARN-7903
> URL: https://issues.apache.org/jira/browse/YARN-7903
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 3.1.0
> Reporter: Yufei Gu
> Priority: Major
>
> We need to specify rack and ANY while submitting a node local resource
> request, as YARN-7561 discussed. For example:
> {code}
> ResourceRequest nodeRequest =
> createResourceRequest(GB, node1.getHostName(), 1, 1, false);
> ResourceRequest rackRequest =
> createResourceRequest(GB, node1.getRackName(), 1, 1, false);
> ResourceRequest anyRequest =
> createResourceRequest(GB, ResourceRequest.ANY, 1, 1, false);
> List<ResourceRequest> resourceRequests =
> Arrays.asList(nodeRequest, rackRequest, anyRequest);
> {code}
> However, method getStarvedResourceRequests() only consider the first
> encountered resource, which most likely is ResourceRequest.ANY. That's a
> mismatch for locality request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
