Wangda Tan commented on YARN-4091:

Thanks [~sunilg].

I can understand why you have this proposal, but I'm not sure if your approach 
works in following scenario. I feel getting a over-all state of an app and a 
last-container-assignment-state may not works well for them:

- App wants only a small proportion of a cluster (such as hard locality)
- Similar to above, app want to run on specific partition only
- App's leafqueue or parent queue beyond its limit
- App asks mappers in one partition (A), and reducers in another partition(B), 
when A has little available resource and B has more available resource. User 
wants to see why mappers allocation is slow.

And also, we cannot get order of allocation with your approach, which is an 
important thing to look at when we enable fairness/priority scheduling for apps.

> Improvement: Introduce more debug/diagnostics information to detail out 
> scheduler activity
> ------------------------------------------------------------------------------------------
>                 Key: YARN-4091
>                 URL: https://issues.apache.org/jira/browse/YARN-4091
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacity scheduler, resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: Improvement on debugdiagnostic information - YARN.pdf
> As schedulers are improved with various new capabilities, more configurations 
> which tunes the schedulers starts to take actions such as limit assigning 
> containers to an application, or introduce delay to allocate container etc. 
> There are no clear information passed down from scheduler to outerworld under 
> these various scenarios. This makes debugging very tougher.
> This ticket is an effort to introduce more defined states on various parts in 
> scheduler where it skips/rejects container assignment, activate application 
> etc. Such information will help user to know whats happening in scheduler.
> Attaching a short proposal for initial discussion. We would like to improve 
> on this as we discuss.

This message was sent by Atlassian JIRA

Reply via email to