[
https://issues.apache.org/jira/browse/YARN-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17011781#comment-17011781
]
Tao Yang commented on YARN-9538:
--------------------------------
Attached v3 patch in which most comments are addressed, updates need more
discussion are as follows:
CS
1. the table of content can be auto-generated by Doxia Macros via defining
"MACRO\{toc|fromDepth=0|toDepth=3}", so there's nothing we can do for this.
I have updated other modifications, please help to review them as well, thanks:
// Activities
Scheduling activities are activity messages used for debugging on some critical
scheduling path, they can be recorded and exposed via RESTful API with minor
impact on the scheduler performance.
// Scheduler Activities
Scheduler activities include useful scheduling info in a scheduling cycle,
which illustrate how the scheduler allocates a container.
Scheduler activities REST API
(`http://rm-http-address:port/ws/v1/cluster/scheduler/activities`) provides a
way to enable recording scheduler activities and fetch them from cache.To
eliminate the performance impact, scheduler automatically disables recording
activities at the end of a scheduling cycle, you can query the RESTful API
again to get the latest scheduler activities.
// Application Activities
Application activities include useful scheduling info for a specified
application, which illustrate how the requirements are satisfied or just
skipped. Application activities REST API
(`http://rm-http-address:port/ws/v1/cluster/scheduler/app-activities/\{appid}`)
provides a way to enable recording application activities for a specified
application within a few seconds or fetch historical application activities
from cache, available actions which include "refresh" and "get" can be
specified by the "actions" parameter:
RM
1. +The scheduler activities API currently supports Capacity Scheduler and
provides a way to get scheduler activities in a single scheduling process, it
will trigger recording scheduler activities in next scheduling process and then
take last required scheduler activities from cache as the response. The
response have hierarchical structure with multiple levels and important
scheduling details which are organized by the sequence of scheduling process:
->
The scheduler activities Restful API {color:#FF0000}is available if you are
using capacity scheduler and{color} can fetch scheduler activities info
recorded in a scheduling cycle. The API returns a message that includes
important scheduling activities info {color:#FF0000}which has a hierarchical
layout with following fields:{color}
7. + Application activities include useful scheduling info for a specified
application, the response have hierarchical structure with multiple levels:
->
Application activities Restful API {color:#FF0000}is available if you are using
capacity scheduler and can fetch useful scheduling info for a specified
application{color}, the response has a hierarchical layout with following
fields:
8. * *AppActivities* - AppActivities are root structure of application
activities within basic information.
->
is the root element?
Yes, updated: AppActivities are root {color:#FF0000}element{color} ...
9. +* *Applications* - Allocations are allocation attempts at app level queried
from the cache.
->
shouldn't here be applications?
Right, updated: +* {color:#FF0000}*Allocations*{color} - Allocations ...
> Document scheduler/app activities and REST APIs
> -----------------------------------------------
>
> Key: YARN-9538
> URL: https://issues.apache.org/jira/browse/YARN-9538
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: documentation
> Reporter: Tao Yang
> Assignee: Tao Yang
> Priority: Major
> Attachments: YARN-9538.001.patch, YARN-9538.002.patch,
> YARN-9538.003.patch
>
>
> Add documentation for scheduler/app activities in CapacityScheduler.md and
> ResourceManagerRest.md.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
