[
https://issues.apache.org/jira/browse/YARN-8898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677011#comment-16677011
]
Bibin A Chundatt edited comment on YARN-8898 at 11/6/18 5:27 PM:
-----------------------------------------------------------------
[~botong]
{quote}
On the other hand, the existing priority value is in AllocateResponse and thus
we are relying on the RM version rather than AM version.
{quote}
Yes . you are correct.
{quote}
We can cherry-pick YARN-4170 to 2.7 if needed. For old RM versions where this
value is not fed in, I guess we can leave the UAM to default priority. What do
you think?
{quote}
Since RM will be always latest version in upgrade scenario. I don't think back
port will be required.
{quote}
Down the line we might need to deal with two source of truth issues (from
StateStore vs RM allocate response) as well.
{quote}
If we start using SubmissionContext from Federation Store i dont this we need
to depend on Register response. We expect the configuration accross clusters to
be same. rt??
Initially i was under the impression that its only application priority and
label, On further analysis found that we might require a few more for all
client API's to work.
*Available in Register Response*
# ApplicationACLs - Available (getContainer user access control.)
# Queue -- Available
*Not available in AM register response*
# ApplicationNodeLabel - Multiple subcluster have same label and containers has
to be distributed based on label
# ApplicationPriority - Container allocation across apps in subcluster
# ApplicationType - Required for query like getApps
# ApplicationTags - getApps query probably later for timelinerservice.
# LogAggregationContext - ContainerToken container containerIdentifier which
encapsulates logaggreationContext required for in subcluster how the container
aggregation should happen.
# keepContainers require -- Might require for internal handling in
FederationInterceptor.
Above mentioned fields also should be set while submitting UAM to secondary
subclusters.
*Solutions*
# *Solution 1* : Add fields AM register response. (Any new field in
applicationSubmissionContext might require handling in Register Side too.)
# *Solution 2* : Push applicationSubmissionContext also to federationStore at
router side.
Solution 2 require (Router, StateStore, AMRMProxy) changes. Advantage no API
change required in future.
Looping [~subru] too.
was (Author: bibinchundatt):
[~botong]
{quote}
On the other hand, the existing priority value is in AllocateResponse and thus
we are relying on the RM version rather than AM version.
{quote}
Yes . you are correct.
{quote}
We can cherry-pick YARN-4170 to 2.7 if needed. For old RM versions where this
value is not fed in, I guess we can leave the UAM to default priority. What do
you think?
{quote}
Since RM will be always latest version in upgrade scenario. I don't think back
port will be required.
Initially i was under the impression that its only application priority and
label, On further analysis found that we might require a few more for all
client API's to work.
*Available in Register Response*
# ApplicationACLs - Available (getContainer user access control.)
# Queue -- Available
*Not available in AM register response*
# ApplicationNodeLabel - Multiple subcluster have same label and containers has
to be distributed based on label
# ApplicationPriority - Container allocation across apps in subcluster
# ApplicationType - Required for query like getApps
# ApplicationTags - getApps query probably later for timelinerservice.
# LogAggregationContext - ContainerToken container containerIdentifier which
encapsulates logaggreationContext required for in subcluster how the container
aggregation should happen.
# keepContainers require -- Might require for internal handling in
FederationInterceptor.
Above mentioned fields also should be set while submitting UAM to secondary
subclusters.
*Solutions*
# *Solution 1* : Add fields AM register response. (Any new field in
applicationSubmissionContext might require handling in Register Side too.)
# *Solution 2* : Push applicationSubmissionContext also to federationStore at
router side.
Solution 2 require (Router, StateStore, AMRMProxy) changes. Advantage no API
change required in future.
Looping [~subru] too.
> Fix FederationInterceptor#allocate to set application priority in
> allocateResponse
> ----------------------------------------------------------------------------------
>
> Key: YARN-8898
> URL: https://issues.apache.org/jira/browse/YARN-8898
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Bibin A Chundatt
> Assignee: Bilwa S T
> Priority: Major
>
> In case of FederationInterceptor#mergeAllocateResponses skips
> application_priority in response returned
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
