[
https://issues.apache.org/jira/browse/YARN-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263135#comment-17263135
]
VADAGA ANANYO RAO commented on YARN-10559:
------------------------------------------
Thank you [~sunilg] and [~wangda] for the comments.
I have tried addressing the comments in the new patch.
Following are the changes:
# Fixed the *checkstyle* warnings.
# Added *check if 0 active apps* are there in the leaf queue. _Note:_ we won't
have to check for 0 active apps in users because users are temp objects
initialised when apps are getting created. So each user will at least have 1
corresponding active app.
# Ensured *fair-share is non-negative* value. However, fair-share can be 0
value. So not adding checks for that. Fair-share can be 0 in following
situations _(its a non-exhaustive list of conditions)_:
## User-limit is somehow 0 for a user
## queueReassignableResources are 0
## No running apps in the queue
# If a container needs to be skipped from preemption in
*skipContainerBasedOnIntraQueuePolicy*, we will continue checks for other
containers of the app instead of breaking from the app.
# Added additional *logs* for when we reduce actually_to_be_preempted of an
app.
Thank you [~sunilg] for catching these points. Please let me know if some
points needs to be addressed.
> Fair sharing intra-queue preemption support in Capacity Scheduler
> -----------------------------------------------------------------
>
> Key: YARN-10559
> URL: https://issues.apache.org/jira/browse/YARN-10559
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
> Affects Versions: 3.1.4
> Reporter: VADAGA ANANYO RAO
> Assignee: VADAGA ANANYO RAO
> Priority: Major
> Attachments: FairOP_preemption-design_doc_v1.pdf,
> FairOP_preemption-design_doc_v2.pdf, YARN-10559.0001.patch,
> YARN-10559.0002.patch, YARN-10559.0003.patch, YARN-10559.0004.patch,
> YARN-10559.0005.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Usecase:
> Due to the way Capacity Scheduler preemption works, If a single user submits
> a large application to a queue (using 100% of resources), that job will not
> be preempted by future applications from the same user within the same queue.
> This implies that the later applications will be forced to wait for
> completion of the long running application. This prevents multiple long
> running, large, applications from running concurrently.
> Support fair sharing among apps while preempting applications from same queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
