[
https://issues.apache.org/jira/browse/YARN-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642496#comment-16642496
]
Arun Suresh commented on YARN-8827:
-----------------------------------
Thanks for the review [~elgoiri]
Attached patch addressing most of your comments. save the few below:
bq. .. Given that this class is only used for this, I would make the NMs part
of the class and trigger registrations and so on as part of the class. ...
Hmmm, not sure what you meant by this. The test case uses the {{MockNM}} and
{{MockRM}} classes which I think we use throughout most CapacityScheduler
tests. Not sure how we can move it inside the class.
bq. .. I would also add a couple more comments throughout explaining how the
apps are submitting requests, etc. Specially the numbers after #184. ...
Ive added a javadoc before the testcase and some demarcation within the
testcase - to mark begining and end of each step - hope that clears things ?
bq. mkmap() and e() are an OK pattern but I would probably call the second one
mkmapentry and add javadoc for the two explaining the purpose.
If you don't mind, id like to keep it as 'e' :) The point was to reduce the
typing and length of the line. Also I don't plan to re-use it ouside the this
testcase, so lets keep it as private. If I d reuse it, I will create a TestUtil
class and put everything there - and probably rename it.
I've also cleaned the findbugs. The test case error though seems unrelated.
> Plumb per app, per user and per queue resource utilization from the NM to RM
> ----------------------------------------------------------------------------
>
> Key: YARN-8827
> URL: https://issues.apache.org/jira/browse/YARN-8827
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Arun Suresh
> Assignee: Arun Suresh
> Priority: Major
> Attachments: YARN-8827-YARN-1011.01.patch,
> YARN-8827-YARN-1011.02.patch
>
>
> Opportunistic Containers for OverAllocation need to be allocated to pending
> applications in some fair manner. Rather than evaluating queue and user
> resource usage (allocated resource usage) and comparing against queue and
> user limits to decide the allocation, it might make more sense to use a
> snapshot of actual resource utilization of the queue and user.
> To facilitate this, this JIRA proposes to aggregate per user, per app (and
> maybe per queue) resource utilization in addition to aggregated Container and
> Node Utilization and send it along with the NM heartbeat. It should be fairly
> inexpensive to aggregate - since it can be performed in the same loop of the
> {{ContainersMonitorImpl}}'s Monitoring thread.
> A snapshot aggregate can be made every couple of seconds in the RM. This
> instantaneous resource utilization should be used to decide if Opportunistic
> containers can be allocated to an App, Queue or User.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
