[
https://issues.apache.org/jira/browse/YARN-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Po updated YARN-5385:
--------------------------
Attachment: YARN-5385.v005.patch
Thanks [~curino] for the review! I have addressed most of the comments you have
given me. My comments below:
1. The CapacitySchedulerConfiguration implements the get by queue method.
2. I have changed the words 'accommodate for' in code and comments to 'make
room for'.
3. In our clusters, we currently do not see many priority inversions. For our
use, it seems that this log won't be too verbose, and having the logs will be
more beneficial than harmful.
4. The makeRoomForReservation method is abstract for PriorityReservationAgent
because subclasses will want to determine 2 things. 1, the order for deletion,
and 2, the order for re-addition. I have documented this in the java doc to
make it more clear that makeRoomForReservation is meant to return an ordered
list of reservation allocations.
5. This problem will require a lot of work, and it may be more appropriate for
this to be completed in a different task. I have created YARN-6227 to track
this feature.
7. I have added locks around the implementation of the ReservationAgent API for
the PriorityReservationAgent. I understand this solution may not be the best in
the event that we get a lot of multi-threaded access in a short period of time,
but a more appropriate solution will be complicated, and it may be better if it
is done in a separate task. YARN-6226 was created to track this improvement.
10. In RMWebService, if the priority is not given, it will be set to an
undefined value. PriorityReservationAgent converts undefined priority to the
lowest possible priority.
11. I do have tests that check invariants.
> Add a PriorityAgent in ReservationSystem
> -----------------------------------------
>
> Key: YARN-5385
> URL: https://issues.apache.org/jira/browse/YARN-5385
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler, fairscheduler, resourcemanager
> Reporter: Sean Po
> Assignee: Sean Po
> Labels: oct16-hard
> Attachments: YARN-5385.v002.patch, YARN-5385.v003.patch,
> YARN-5385.v004.patch, YARN-5385.v005.patch, YARN-5385.v1.patch
>
>
> YARN-5211 proposes adding support for generalized priorities for reservations
> in the YARN ReservationSystem. This JIRA is a sub-task to track the addition
> of a priority agent to accomplish it. Please refer to the design doc in the
> parent JIRA for details.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
