[jira] [Commented] (YARN-9643) Federation: Add subClusterID in nodes page of Router web
[ https://issues.apache.org/jira/browse/YARN-9643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368620#comment-17368620 ] hunshenshi commented on YARN-9643: -- [~Song Jiacheng]. The PR is Ready, but not merge into the master. You can use the PR [GitHub Pull Request #1018|https://github.com/apache/hadoop/pull/1018] > Federation: Add subClusterID in nodes page of Router web > > > Key: YARN-9643 > URL: https://issues.apache.org/jira/browse/YARN-9643 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.2.0 >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > Attachments: nodes.png > > > In nodes page of router web, there only are node info, No cluster id > corresponding to the node. > [http://127.0.0.1:8089/cluster/nodes|http://192.168.169.72:8089/cluster/nodes] > !nodes.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7652) Handle AM register requests asynchronously in FederationInterceptor
[
https://issues.apache.org/jira/browse/YARN-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hunshenshi reassigned YARN-7652:
Assignee: hunshenshi (was: Botong Huang)
> Handle AM register requests asynchronously in FederationInterceptor
> ---
>
> Key: YARN-7652
> URL: https://issues.apache.org/jira/browse/YARN-7652
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: amrmproxy, federation
>Affects Versions: 2.9.0, 3.0.0
>Reporter: Subru Krishnan
>Assignee: hunshenshi
>Priority: Major
> Fix For: 2.10.0, 3.3.0
>
> Attachments: YARN-7652.v1.patch, YARN-7652.v2.patch
>
>
> We (cc [~goiri]/[~botong]) observed that the {{FederationInterceptor}} in
> {{AMRMProxy}} (and consequently the AM) is blocked if the _StateStore_ has
> outdated info about a _SubCluster_. This is because we handle AM register
> requests synchronously. This jira proposes to move to async similar to how we
> operate with allocate invocations.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9665) Change the value of 'Cluster ID' to value of `yarn.resourcemanager.cluster-id` in RM webUI
[ https://issues.apache.org/jira/browse/YARN-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881085#comment-16881085 ] hunshenshi commented on YARN-9665: -- update the pic of changing clusterId > Change the value of 'Cluster ID' to value of > `yarn.resourcemanager.cluster-id` in RM webUI > --- > > Key: YARN-9665 > URL: https://issues.apache.org/jira/browse/YARN-9665 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, yarn >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > Attachments: change-clusterid.png, clusterid.png > > > The value of 'Cluster ID' in RM webUI is the cluster timestamp. > I think the value of 'Cluster ID' should be the value of > `yarn.resourcemanager.cluster-id`, especially in federation mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9665) Change the value of 'Cluster ID' to value of `yarn.resourcemanager.cluster-id` in RM webUI
[ https://issues.apache.org/jira/browse/YARN-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi updated YARN-9665: - Attachment: change-clusterid.png > Change the value of 'Cluster ID' to value of > `yarn.resourcemanager.cluster-id` in RM webUI > --- > > Key: YARN-9665 > URL: https://issues.apache.org/jira/browse/YARN-9665 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, yarn >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > Attachments: change-clusterid.png, clusterid.png > > > The value of 'Cluster ID' in RM webUI is the cluster timestamp. > I think the value of 'Cluster ID' should be the value of > `yarn.resourcemanager.cluster-id`, especially in federation mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9665) Change the value of 'Cluster ID' to value of `yarn.resourcemanager.cluster-id` in RM webUI
[ https://issues.apache.org/jira/browse/YARN-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi updated YARN-9665: - Description: The value of 'Cluster ID' in RM webUI is the cluster timestamp. I think the value of 'Cluster ID' should be the value of `yarn.resourcemanager.cluster-id`, especially in federation mode. was: The value of 'Cluster ID' in RM webUI is the cluster timestamp. I think the value of 'Cluster ID' should be the value of `yarn.resourcemanager.cluster-id`, especially in federation mode. !clusterid.png! > Change the value of 'Cluster ID' to value of > `yarn.resourcemanager.cluster-id` in RM webUI > --- > > Key: YARN-9665 > URL: https://issues.apache.org/jira/browse/YARN-9665 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, yarn >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > Attachments: clusterid.png > > > The value of 'Cluster ID' in RM webUI is the cluster timestamp. > I think the value of 'Cluster ID' should be the value of > `yarn.resourcemanager.cluster-id`, especially in federation mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9665) Change the value of 'Cluster ID' to value of `yarn.resourcemanager.cluster-id` in RM webUI
[ https://issues.apache.org/jira/browse/YARN-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi updated YARN-9665: - Attachment: clusterid.png > Change the value of 'Cluster ID' to value of > `yarn.resourcemanager.cluster-id` in RM webUI > --- > > Key: YARN-9665 > URL: https://issues.apache.org/jira/browse/YARN-9665 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, yarn >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > Attachments: clusterid.png > > > The value of 'Cluster ID' in RM webUI is the cluster timestamp. > I think the value of 'Cluster ID' should be the value of > `yarn.resourcemanager.cluster-id`, especially in federation mode. > !clusterid.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9665) Change the value of 'Cluster ID' to value of `yarn.resourcemanager.cluster-id` in RM webUI
[ https://issues.apache.org/jira/browse/YARN-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi updated YARN-9665: - Description: The value of 'Cluster ID' in RM webUI is the cluster timestamp. I think the value of 'Cluster ID' should be the value of `yarn.resourcemanager.cluster-id`, especially in federation mode. !clusterid.png! was: The value of 'Cluster ID' in RM webUI is the cluster timestamp. I think the value of 'Cluster ID' should be the value of `yarn.resourcemanager.cluster-id`, especially in federation mode. > Change the value of 'Cluster ID' to value of > `yarn.resourcemanager.cluster-id` in RM webUI > --- > > Key: YARN-9665 > URL: https://issues.apache.org/jira/browse/YARN-9665 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, yarn >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > Attachments: clusterid.png > > > The value of 'Cluster ID' in RM webUI is the cluster timestamp. > I think the value of 'Cluster ID' should be the value of > `yarn.resourcemanager.cluster-id`, especially in federation mode. > !clusterid.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9665) Change the value of 'Cluster ID' to value of `yarn.resourcemanager.cluster-id` in RM webUI
[ https://issues.apache.org/jira/browse/YARN-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi updated YARN-9665: - Description: The value of 'Cluster ID' in RM webUI is the cluster timestamp. I think the value of 'Cluster ID' should be the value of `yarn.resourcemanager.cluster-id`, especially in federation mode. > Change the value of 'Cluster ID' to value of > `yarn.resourcemanager.cluster-id` in RM webUI > --- > > Key: YARN-9665 > URL: https://issues.apache.org/jira/browse/YARN-9665 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, yarn >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > > The value of 'Cluster ID' in RM webUI is the cluster timestamp. > I think the value of 'Cluster ID' should be the value of > `yarn.resourcemanager.cluster-id`, especially in federation mode. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9665) Change the value of 'Cluster ID' to value of `yarn.resourcemanager.cluster-id` in RM webUI
hunshenshi created YARN-9665: Summary: Change the value of 'Cluster ID' to value of `yarn.resourcemanager.cluster-id` in RM webUI Key: YARN-9665 URL: https://issues.apache.org/jira/browse/YARN-9665 Project: Hadoop YARN Issue Type: Improvement Components: federation, yarn Reporter: hunshenshi -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9665) Change the value of 'Cluster ID' to value of `yarn.resourcemanager.cluster-id` in RM webUI
[ https://issues.apache.org/jira/browse/YARN-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi reassigned YARN-9665: Assignee: hunshenshi > Change the value of 'Cluster ID' to value of > `yarn.resourcemanager.cluster-id` in RM webUI > --- > > Key: YARN-9665 > URL: https://issues.apache.org/jira/browse/YARN-9665 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, yarn >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9663) ApplicationID may be duplicated in YARN Federation
[
https://issues.apache.org/jira/browse/YARN-9663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878316#comment-16878316
]
hunshenshi commented on YARN-9663:
--
Just add the clusterId of the resourcemanager in applicationId, it happens in
rm, can't expose to client. Client do as before.
It will fix from the root.
Default applicationId is application_timestamp_xxx1, I will add `clusterId`
after `application`, then the applicationId will be
application_{color:#FF}clusterId{color}_timestamp_xxx1.
> ApplicationID may be duplicated in YARN Federation
> --
>
> Key: YARN-9663
> URL: https://issues.apache.org/jira/browse/YARN-9663
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation, yarn
>Reporter: hunshenshi
>Assignee: hunshenshi
>Priority: Major
>
> ApplicationId represents the globally unique identifier for an application.
> The globally unique nature of the identifier is achieved by using the cluster
> timestamp. i.e. start-time of the ResourceManager along with a monotonically
> increasing counter for the application.
> But in yarn federation, the applicationId will be duplicated if the timestamp
> of subClusters is same.
> Shall we add clusterId in applicationId, like
> application_cluseterId_timestamp_xxx1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9663) ApplicationID may be duplicated in YARN Federation
[ https://issues.apache.org/jira/browse/YARN-9663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878311#comment-16878311 ] hunshenshi edited comment on YARN-9663 at 7/4/19 5:02 AM: -- [~bibinchundatt] I don't think so. YARN-9528 just update sql of State store. I will change applicationId to fix this. I think YARN-9528 can't fix all scenes was (Author: hunhun): I don't think so. YARN-9528 just update sql of State store. I will change applicationId to fix this. I think YARN-9528 can't fix all scenes > ApplicationID may be duplicated in YARN Federation > -- > > Key: YARN-9663 > URL: https://issues.apache.org/jira/browse/YARN-9663 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, yarn >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > > ApplicationId represents the globally unique identifier for an application. > The globally unique nature of the identifier is achieved by using the cluster > timestamp. i.e. start-time of the ResourceManager along with a monotonically > increasing counter for the application. > But in yarn federation, the applicationId will be duplicated if the timestamp > of subClusters is same. > Shall we add clusterId in applicationId, like > application_cluseterId_timestamp_xxx1 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9663) ApplicationID may be duplicated in YARN Federation
[ https://issues.apache.org/jira/browse/YARN-9663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878311#comment-16878311 ] hunshenshi edited comment on YARN-9663 at 7/4/19 5:01 AM: -- I don't think so. YARN-9528 just update sql of State store. I will change applicationId to fix this. I think YARN-9528 can't fix all scenes was (Author: hunhun): I don't think so. [YARN-9528|https://issues.apache.org/jira/browse/YARN-9528] just update sql of State store. I will change applicationId to fix this. > ApplicationID may be duplicated in YARN Federation > -- > > Key: YARN-9663 > URL: https://issues.apache.org/jira/browse/YARN-9663 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, yarn >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > > ApplicationId represents the globally unique identifier for an application. > The globally unique nature of the identifier is achieved by using the cluster > timestamp. i.e. start-time of the ResourceManager along with a monotonically > increasing counter for the application. > But in yarn federation, the applicationId will be duplicated if the timestamp > of subClusters is same. > Shall we add clusterId in applicationId, like > application_cluseterId_timestamp_xxx1 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9663) ApplicationID may be duplicated in YARN Federation
[ https://issues.apache.org/jira/browse/YARN-9663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878311#comment-16878311 ] hunshenshi commented on YARN-9663: -- I don't think so. [YARN-9528|https://issues.apache.org/jira/browse/YARN-9528] just update sql of State store. I will change applicationId to fix this. > ApplicationID may be duplicated in YARN Federation > -- > > Key: YARN-9663 > URL: https://issues.apache.org/jira/browse/YARN-9663 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, yarn >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > > ApplicationId represents the globally unique identifier for an application. > The globally unique nature of the identifier is achieved by using the cluster > timestamp. i.e. start-time of the ResourceManager along with a monotonically > increasing counter for the application. > But in yarn federation, the applicationId will be duplicated if the timestamp > of subClusters is same. > Shall we add clusterId in applicationId, like > application_cluseterId_timestamp_xxx1 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9468) Fix inaccurate documentations in Placement Constraints
[
https://issues.apache.org/jira/browse/YARN-9468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hunshenshi reassigned YARN-9468:
Assignee: hunshenshi (was: Charan Hebri)
> Fix inaccurate documentations in Placement Constraints
> --
>
> Key: YARN-9468
> URL: https://issues.apache.org/jira/browse/YARN-9468
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Assignee: hunshenshi
>Priority: Major
>
> Document Placement Constraints
> *First*
> {code:java}
> zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
> * place 5 containers with tag “hbase” with affinity to a rack on which
> containers with tag “zk” are running (i.e., an “hbase” container
> should{color:#ff} not{color} be placed at a rack where an “zk” container
> is running, given that “zk” is the TargetTag of the second constraint);
> The _*not*_ word in brackets should be delete.
>
> *Second*
> {code:java}
> PlacementSpec => "" | KeyVal;PlacementSpec
> {code}
> The semicolon should be replaced by colon
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9655) AllocateResponse in FederationInterceptor lost applicationPriority
[
https://issues.apache.org/jira/browse/YARN-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877458#comment-16877458
]
hunshenshi commented on YARN-9655:
--
Thanks for review [~cheersyang]
> AllocateResponse in FederationInterceptor lost applicationPriority
> ---
>
> Key: YARN-9655
> URL: https://issues.apache.org/jira/browse/YARN-9655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Assignee: hunshenshi
>Priority: Major
> Fix For: 3.0.4, 3.3.0, 3.2.1, 2.9.3, 3.1.3
>
> Attachments: YARN-9655.branch-2.9.patch, YARN-9655.branch-3.0.patch
>
>
> In YARN Federation mode using FederationInterceptor, when submitting
> application, am will report an error.
> {code:java}
> 2019-06-25 11:44:00,977 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
> java.lang.NullPointerException at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.handleJobPriorityChange(RMContainerAllocator.java:1025)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:880)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:286)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:280)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The reason is that applicationPriority is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9651) Resource Manager throws NPE
[
https://issues.apache.org/jira/browse/YARN-9651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876838#comment-16876838
]
hunshenshi commented on YARN-9651:
--
[~zhangqw] all application will case rm shutdown ?
How can I reproduce this scene?
> Resource Manager throws NPE
> ---
>
> Key: YARN-9651
> URL: https://issues.apache.org/jira/browse/YARN-9651
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.1
> Environment: os: centos 7.1
> hadoop 3.1.1 release
>
>Reporter: zhangqw
>Priority: Major
>
> We use hadoop 3.1.1 release，running some regular job when RM Stopped with NPE.
> {code:java}
> 2019-06-13 17:06:06,664 FATAL event.EventDispatcher
> (EventDispatcher.java:run(75)) - Error in handling event type
> APP_ATTEMPT_ADDED to the Event Dispatcher
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt.transferStateFromPreviousAttempt(SchedulerApplicationAttempt.java:1158)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.transferStateFromPreviousAttempt(FiCaSchedulerApp.java:852)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.addApplicationAttempt(CapacityScheduler.java:982)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:1730)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:167)
> at
> org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> I checked [related issue:
> YARN-2340|https://issues.apache.org/jira/browse/YARN-2340] , but it's
> already fixed in my running version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9663) ApplicationID may be duplicated in YARN Federation
hunshenshi created YARN-9663: Summary: ApplicationID may be duplicated in YARN Federation Key: YARN-9663 URL: https://issues.apache.org/jira/browse/YARN-9663 Project: Hadoop YARN Issue Type: Bug Components: federation, yarn Reporter: hunshenshi Assignee: hunshenshi ApplicationId represents the globally unique identifier for an application. The globally unique nature of the identifier is achieved by using the cluster timestamp. i.e. start-time of the ResourceManager along with a monotonically increasing counter for the application. But in yarn federation, the applicationId will be duplicated if the timestamp of subClusters is same. Shall we add clusterId in applicationId, like application_cluseterId_timestamp_xxx1 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9655) AllocateResponse in FederationInterceptor lost applicationPriority
[
https://issues.apache.org/jira/browse/YARN-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876635#comment-16876635
]
hunshenshi commented on YARN-9655:
--
I upload patch for branch-2.9 and branch-3.0，please review.
Thanks [~cheersyang]
> AllocateResponse in FederationInterceptor lost applicationPriority
> ---
>
> Key: YARN-9655
> URL: https://issues.apache.org/jira/browse/YARN-9655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Assignee: hunshenshi
>Priority: Major
> Fix For: 3.3.0, 3.2.1, 3.1.3
>
> Attachments: YARN-9655.branch-2.9.patch, YARN-9655.branch-3.0.patch
>
>
> In YARN Federation mode using FederationInterceptor, when submitting
> application, am will report an error.
> {code:java}
> 2019-06-25 11:44:00,977 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
> java.lang.NullPointerException at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.handleJobPriorityChange(RMContainerAllocator.java:1025)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:880)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:286)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:280)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The reason is that applicationPriority is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9655) AllocateResponse in FederationInterceptor lost applicationPriority
[
https://issues.apache.org/jira/browse/YARN-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hunshenshi updated YARN-9655:
-
Attachment: YARN-9655.branch-3.0.patch
> AllocateResponse in FederationInterceptor lost applicationPriority
> ---
>
> Key: YARN-9655
> URL: https://issues.apache.org/jira/browse/YARN-9655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Assignee: hunshenshi
>Priority: Major
> Fix For: 3.3.0, 3.2.1, 3.1.3
>
> Attachments: YARN-9655.branch-2.9.patch, YARN-9655.branch-3.0.patch
>
>
> In YARN Federation mode using FederationInterceptor, when submitting
> application, am will report an error.
> {code:java}
> 2019-06-25 11:44:00,977 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
> java.lang.NullPointerException at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.handleJobPriorityChange(RMContainerAllocator.java:1025)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:880)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:286)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:280)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The reason is that applicationPriority is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9655) AllocateResponse in FederationInterceptor lost applicationPriority
[
https://issues.apache.org/jira/browse/YARN-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hunshenshi updated YARN-9655:
-
Attachment: YARN-9655.branch-2.9.patch
> AllocateResponse in FederationInterceptor lost applicationPriority
> ---
>
> Key: YARN-9655
> URL: https://issues.apache.org/jira/browse/YARN-9655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Assignee: hunshenshi
>Priority: Major
> Fix For: 3.3.0, 3.2.1, 3.1.3
>
> Attachments: YARN-9655.branch-2.9.patch
>
>
> In YARN Federation mode using FederationInterceptor, when submitting
> application, am will report an error.
> {code:java}
> 2019-06-25 11:44:00,977 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
> java.lang.NullPointerException at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.handleJobPriorityChange(RMContainerAllocator.java:1025)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:880)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:286)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:280)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The reason is that applicationPriority is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9655) AllocateResponse in FederationInterceptor lost applicationPriority
[
https://issues.apache.org/jira/browse/YARN-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876626#comment-16876626
]
hunshenshi commented on YARN-9655:
--
OK，I will check
> AllocateResponse in FederationInterceptor lost applicationPriority
> ---
>
> Key: YARN-9655
> URL: https://issues.apache.org/jira/browse/YARN-9655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Assignee: hunshenshi
>Priority: Major
> Fix For: 3.3.0, 3.2.1, 3.1.3
>
>
> In YARN Federation mode using FederationInterceptor, when submitting
> application, am will report an error.
> {code:java}
> 2019-06-25 11:44:00,977 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
> java.lang.NullPointerException at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.handleJobPriorityChange(RMContainerAllocator.java:1025)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:880)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:286)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:280)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The reason is that applicationPriority is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9601) Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations
[
https://issues.apache.org/jira/browse/YARN-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hunshenshi reassigned YARN-9601:
Assignee: hunshenshi
> Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations
>
>
> Key: YARN-9601
> URL: https://issues.apache.org/jira/browse/YARN-9601
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation, yarn
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Assignee: hunshenshi
>Priority: Major
>
> Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations
> The code of ZookeeperFederationStateStore#getPoliciesConfigurations
> {code:java}
> for (String child : zkManager.getChildren(policiesZNode)) {
> SubClusterPolicyConfiguration policy = getPolicy(child);
> result.add(policy);
> }
> {code}
> The result of `getPolicy` may be null, so policy should be checked
> The new code
> {code:java}
> for (String child : zkManager.getChildren(policiesZNode)) {
> SubClusterPolicyConfiguration policy = getPolicy(child);
> // policy maybe null, should check
> if (policy == null) {
> LOG.warn("Policy for queue: {} does not exist.", child);
> continue;
> }
> result.add(policy);
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9643) Federation: Add subClusterID in nodes page of Router web
[ https://issues.apache.org/jira/browse/YARN-9643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi reassigned YARN-9643: Assignee: hunshenshi > Federation: Add subClusterID in nodes page of Router web > > > Key: YARN-9643 > URL: https://issues.apache.org/jira/browse/YARN-9643 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.2.0 >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > Attachments: nodes.png > > > In nodes page of router web, there only are node info, No cluster id > corresponding to the node. > [http://127.0.0.1:8089/cluster/nodes|http://192.168.169.72:8089/cluster/nodes] > !nodes.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9250) hadoop-yarn-server-nodemanager build failed: make failed with error code 2
[ https://issues.apache.org/jira/browse/YARN-9250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876185#comment-16876185 ] hunshenshi commented on YARN-9250: -- [~linlong] you can use -X see more error info, maybe it will help you. > hadoop-yarn-server-nodemanager build failed: make failed with error code 2 > -- > > Key: YARN-9250 > URL: https://issues.apache.org/jira/browse/YARN-9250 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.0 >Reporter: charlie mao >Priority: Blocker > > when i compile hadoop-3.2.0 release,i encountered the following errors： > [ERROR] Failed to execute goal > org.apache.hadoop:hadoop-maven-plugins:3.2.0:cmake-compile (cmake-compile) on > project hadoop-yarn-server-nodemanager: make failed with error code 2 -> > [Help 1] > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal org.apache.hadoop:hadoop-maven-plugins:3.2.0:cmake-compile > (cmake-compile) on project hadoop-yarn-server-nodemanager: make failed with > error code 2 > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) > at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) > at > org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) > at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) > at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) > at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) > at > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) > at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) > Caused by: org.apache.maven.plugin.MojoExecutionException: make failed with > error code 2 > at > org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.runMake(CompileMojo.java:231) > at > org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.execute(CompileMojo.java:98) > at > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) > ... 20 more > [ERROR] > [ERROR] > [ERROR] For more information about the errors and possible solutions, please > read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > [ERROR] > [ERROR] After correcting the problems, you can resume the build with the > command > [ERROR] mvn -rf :hadoop-yarn-server-nodemanager > > my compiling environment： > jdk 1.8.0_181 > maven:3.3.9(/3.6.0) > cmake version 3.12.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9661) Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java
[ https://issues.apache.org/jira/browse/YARN-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi reassigned YARN-9661: Assignee: hunshenshi > Fix typo in LocalityMulticastAMRMProxyPolicy.java and > AbstractConfigurableFederationPolicy.java > --- > > Key: YARN-9661 > URL: https://issues.apache.org/jira/browse/YARN-9661 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, yarn >Affects Versions: 3.2.0 >Reporter: hunshenshi >Assignee: hunshenshi >Priority: Major > > There are some typo in LocalityMulticastAMRMProxyPolicy.java and > AbstractConfigurableFederationPolicy.java -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9661) Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java
[ https://issues.apache.org/jira/browse/YARN-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi updated YARN-9661: - Component/s: federation > Fix typo in LocalityMulticastAMRMProxyPolicy.java and > AbstractConfigurableFederationPolicy.java > --- > > Key: YARN-9661 > URL: https://issues.apache.org/jira/browse/YARN-9661 > Project: Hadoop YARN > Issue Type: Bug > Components: federation >Affects Versions: 3.2.0 >Reporter: hunshenshi >Priority: Major > > There are some typo in LocalityMulticastAMRMProxyPolicy.java and > AbstractConfigurableFederationPolicy.java -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9661) Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java
[ https://issues.apache.org/jira/browse/YARN-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi updated YARN-9661: - Component/s: yarn > Fix typo in LocalityMulticastAMRMProxyPolicy.java and > AbstractConfigurableFederationPolicy.java > --- > > Key: YARN-9661 > URL: https://issues.apache.org/jira/browse/YARN-9661 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, yarn >Affects Versions: 3.2.0 >Reporter: hunshenshi >Priority: Major > > There are some typo in LocalityMulticastAMRMProxyPolicy.java and > AbstractConfigurableFederationPolicy.java -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9661) Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java
[ https://issues.apache.org/jira/browse/YARN-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi updated YARN-9661: - Description: There are some typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java (was: There are some typo in ) > Fix typo in LocalityMulticastAMRMProxyPolicy.java and > AbstractConfigurableFederationPolicy.java > --- > > Key: YARN-9661 > URL: https://issues.apache.org/jira/browse/YARN-9661 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.2.0 >Reporter: hunshenshi >Priority: Major > > There are some typo in LocalityMulticastAMRMProxyPolicy.java and > AbstractConfigurableFederationPolicy.java -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9661) Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java
[ https://issues.apache.org/jira/browse/YARN-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi updated YARN-9661: - Description: There are some typo in > Fix typo in LocalityMulticastAMRMProxyPolicy.java and > AbstractConfigurableFederationPolicy.java > --- > > Key: YARN-9661 > URL: https://issues.apache.org/jira/browse/YARN-9661 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.2.0 >Reporter: hunshenshi >Priority: Major > > There are some typo in -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9661) Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java
hunshenshi created YARN-9661: Summary: Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java Key: YARN-9661 URL: https://issues.apache.org/jira/browse/YARN-9661 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.2.0 Reporter: hunshenshi -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6740) Federation Router (hiding multiple RMs for ApplicationClientProtocol) phase 2
[ https://issues.apache.org/jira/browse/YARN-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875351#comment-16875351 ] hunshenshi commented on YARN-6740: -- Thanks [~abmodi] [~giovanni.fumarola] > Federation Router (hiding multiple RMs for ApplicationClientProtocol) phase 2 > - > > Key: YARN-6740 > URL: https://issues.apache.org/jira/browse/YARN-6740 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: Abhishek Modi >Priority: Major > > This JIRA tracks the implementation of the layer for routing > ApplicaitonClientProtocol requests to the appropriate RM(s) in a federated > YARN cluster. > Under the YARN-3659 we only implemented getNewApplication, submitApplication, > forceKillApplication and getApplicationReport to execute applications E2E. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9655) AllocateResponse in FederationInterceptor lost applicationPriority
[
https://issues.apache.org/jira/browse/YARN-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875350#comment-16875350
]
hunshenshi commented on YARN-9655:
--
Sure，I add a UT in TestFederationInterceptor#testAllocateResponse。
Thanks for review [~cheersyang]
> AllocateResponse in FederationInterceptor lost applicationPriority
> ---
>
> Key: YARN-9655
> URL: https://issues.apache.org/jira/browse/YARN-9655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Assignee: hunshenshi
>Priority: Major
>
> In YARN Federation mode using FederationInterceptor, when submitting
> application, am will report an error.
> {code:java}
> 2019-06-25 11:44:00,977 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
> java.lang.NullPointerException at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.handleJobPriorityChange(RMContainerAllocator.java:1025)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:880)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:286)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:280)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The reason is that applicationPriority is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9655) AllocateResponse in FederationInterceptor lost applicationPriority
[
https://issues.apache.org/jira/browse/YARN-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874595#comment-16874595
]
hunshenshi edited comment on YARN-9655 at 6/28/19 1:54 AM:
---
OK，Thanks [~cheersyang], I will fix it
was (Author: hunhun):
OK，I will fix it
> AllocateResponse in FederationInterceptor lost applicationPriority
> ---
>
> Key: YARN-9655
> URL: https://issues.apache.org/jira/browse/YARN-9655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Priority: Major
>
> In YARN Federation mode using FederationInterceptor, when submitting
> application, am will report an error.
> {code:java}
> 2019-06-25 11:44:00,977 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
> java.lang.NullPointerException at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.handleJobPriorityChange(RMContainerAllocator.java:1025)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:880)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:286)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:280)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The reason is that applicationPriority is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9655) AllocateResponse in FederationInterceptor lost applicationPriority
[
https://issues.apache.org/jira/browse/YARN-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874595#comment-16874595
]
hunshenshi commented on YARN-9655:
--
OK，I will fix it
> AllocateResponse in FederationInterceptor lost applicationPriority
> ---
>
> Key: YARN-9655
> URL: https://issues.apache.org/jira/browse/YARN-9655
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Priority: Major
>
> In YARN Federation mode using FederationInterceptor, when submitting
> application, am will report an error.
> {code:java}
> 2019-06-25 11:44:00,977 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
> java.lang.NullPointerException at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.handleJobPriorityChange(RMContainerAllocator.java:1025)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:880)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:286)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:280)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The reason is that applicationPriority is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9643) Federation: Add subClusterID in nodes page of Router web
[ https://issues.apache.org/jira/browse/YARN-9643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hunshenshi updated YARN-9643: - Issue Type: Improvement (was: Bug) > Federation: Add subClusterID in nodes page of Router web > > > Key: YARN-9643 > URL: https://issues.apache.org/jira/browse/YARN-9643 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.2.0 >Reporter: hunshenshi >Priority: Major > Attachments: nodes.png > > > In nodes page of router web, there only are node info, No cluster id > corresponding to the node. > [http://127.0.0.1:8089/cluster/nodes|http://192.168.169.72:8089/cluster/nodes] > !nodes.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9655) AllocateResponse in FederationInterceptor lost applicationPriority
hunshenshi created YARN-9655:
Summary: AllocateResponse in FederationInterceptor lost
applicationPriority
Key: YARN-9655
URL: https://issues.apache.org/jira/browse/YARN-9655
Project: Hadoop YARN
Issue Type: Bug
Components: federation
Affects Versions: 3.2.0
Reporter: hunshenshi
In YARN Federation mode using FederationInterceptor, when submitting
application, am will report an error.
{code:java}
2019-06-25 11:44:00,977 ERROR [RMCommunicator Allocator]
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM.
java.lang.NullPointerException at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.handleJobPriorityChange(RMContainerAllocator.java:1025)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:880)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:286)
at
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:280)
at java.lang.Thread.run(Thread.java:748)
{code}
The reason is that applicationPriority is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9643) Federation: Add subClusterID in nodes page of Router web
[ https://issues.apache.org/jira/browse/YARN-9643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16870048#comment-16870048 ] hunshenshi commented on YARN-9643: -- I will add subClusterID before Node Labels, is it ok? > Federation: Add subClusterID in nodes page of Router web > > > Key: YARN-9643 > URL: https://issues.apache.org/jira/browse/YARN-9643 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.2.0 >Reporter: hunshenshi >Priority: Major > Attachments: nodes.png > > > In nodes page of router web, there only are node info, No cluster id > corresponding to the node. > [http://127.0.0.1:8089/cluster/nodes|http://192.168.169.72:8089/cluster/nodes] > !nodes.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9643) Federation: Add subClusterID in nodes page of Router web
hunshenshi created YARN-9643: Summary: Federation: Add subClusterID in nodes page of Router web Key: YARN-9643 URL: https://issues.apache.org/jira/browse/YARN-9643 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.2.0 Reporter: hunshenshi Attachments: nodes.png In nodes page of router web, there only are node info, No cluster id corresponding to the node. [http://127.0.0.1:8089/cluster/nodes|http://192.168.169.72:8089/cluster/nodes] !nodes.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6740) Federation Router (hiding multiple RMs for ApplicationClientProtocol) phase 2
[ https://issues.apache.org/jira/browse/YARN-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867608#comment-16867608 ] hunshenshi commented on YARN-6740: -- still on developing? > Federation Router (hiding multiple RMs for ApplicationClientProtocol) phase 2 > - > > Key: YARN-6740 > URL: https://issues.apache.org/jira/browse/YARN-6740 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: Abhishek Modi >Priority: Major > > This JIRA tracks the implementation of the layer for routing > ApplicaitonClientProtocol requests to the appropriate RM(s) in a federated > YARN cluster. > Under the YARN-3659 we only implemented getNewApplication, submitApplication, > forceKillApplication and getApplicationReport to execute applications E2E. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9601) Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations
hunshenshi created YARN-9601:
Summary: Potential NPE in
ZookeeperFederationStateStore#getPoliciesConfigurations
Key: YARN-9601
URL: https://issues.apache.org/jira/browse/YARN-9601
Project: Hadoop YARN
Issue Type: Bug
Components: federation, yarn
Affects Versions: 3.2.0
Reporter: hunshenshi
Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations
The code of ZookeeperFederationStateStore#getPoliciesConfigurations
{code:java}
for (String child : zkManager.getChildren(policiesZNode)) {
SubClusterPolicyConfiguration policy = getPolicy(child);
result.add(policy);
}
{code}
The result of `getPolicy` may be null, so policy should be checked
The new code
{code:java}
for (String child : zkManager.getChildren(policiesZNode)) {
SubClusterPolicyConfiguration policy = getPolicy(child);
// policy maybe null, should check
if (policy == null) {
LOG.warn("Policy for queue: {} does not exist.", child);
continue;
}
result.add(policy);
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9579) the property of sharedcache in mapred-default.xml
[
https://issues.apache.org/jira/browse/YARN-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hunshenshi updated YARN-9579:
-
Description:
add "enabled" category to `mapreduce.job.sharedcache.mode` in
mapred-default.xml.
Add the following content
{code:java}
If "enable" is specified then the job submission code will use the shared cache
with all resouce, include jobjar, libjars, files and archives.
{code}
was:
add "enabled" category in mapred-default.xml.
Add the following content
{code:java}
If "enable" is specified then the job submission code will use the shared cache
with all resouce, include jobjar, libjars, files and archives.
{code}
> the property of sharedcache in mapred-default.xml
> -
>
> Key: YARN-9579
> URL: https://issues.apache.org/jira/browse/YARN-9579
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Priority: Major
>
> add "enabled" category to `mapreduce.job.sharedcache.mode` in
> mapred-default.xml.
> Add the following content
> {code:java}
> If "enable" is specified then the job submission code will use the shared
> cache with all resouce, include jobjar, libjars, files and archives.
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9579) the property of sharedcache in mapred-default.xml
hunshenshi created YARN-9579:
Summary: the property of sharedcache in mapred-default.xml
Key: YARN-9579
URL: https://issues.apache.org/jira/browse/YARN-9579
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 3.2.0
Reporter: hunshenshi
add "enabled" category in mapred-default.xml.
Add the following content
{code:java}
If "enable" is specified then the job submission code will use the shared cache
with all resouce, include jobjar, libjars, files and archives.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9484) Fix SLF4j logging API in AbstractManagedParentQueue#removeChildQueue
[
https://issues.apache.org/jira/browse/YARN-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818697#comment-16818697
]
hunshenshi commented on YARN-9484:
--
OK，Thanks
> Fix SLF4j logging API in AbstractManagedParentQueue#removeChildQueue
>
>
> Key: YARN-9484
> URL: https://issues.apache.org/jira/browse/YARN-9484
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Priority: Major
>
> {code:java}
> if (LOG.isDebugEnabled()) {
> LOG.debug("Removed child queue: {}" + cs.getQueueName());
> }
> {code}
> The '+' should be ','
> {code:java}
> if (LOG.isDebugEnabled()) {
> LOG.debug("Removed child queue: {}", cs.getQueueName());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9484) Fix SLF4j logging API in AbstractManagedParentQueue#removeChildQueue
hunshenshi created YARN-9484:
Summary: Fix SLF4j logging API in
AbstractManagedParentQueue#removeChildQueue
Key: YARN-9484
URL: https://issues.apache.org/jira/browse/YARN-9484
Project: Hadoop YARN
Issue Type: Bug
Components: capacity scheduler
Affects Versions: 3.2.0
Reporter: hunshenshi
{code:java}
if (LOG.isDebugEnabled()) {
LOG.debug("Removed child queue: {}" + cs.getQueueName());
}
{code}
The '+' should be ','
{code:java}
if (LOG.isDebugEnabled()) {
LOG.debug("Removed child queue: {}", cs.getQueueName());
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9468) Fix inaccurate documentations in Placement Constraints
[
https://issues.apache.org/jira/browse/YARN-9468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814145#comment-16814145
]
hunshenshi commented on YARN-9468:
--
Thanks for fixing the title. [~cheersyang]
> Fix inaccurate documentations in Placement Constraints
> --
>
> Key: YARN-9468
> URL: https://issues.apache.org/jira/browse/YARN-9468
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Priority: Major
>
> Document Placement Constraints
> *First*
> {code:java}
> zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
> * place 5 containers with tag “hbase” with affinity to a rack on which
> containers with tag “zk” are running (i.e., an “hbase” container
> should{color:#ff} not{color} be placed at a rack where an “zk” container
> is running, given that “zk” is the TargetTag of the second constraint);
> The _*not*_ word in brackets should be delete.
>
> *Second*
> {code:java}
> PlacementSpec => "" | KeyVal;PlacementSpec
> {code}
> The semicolon should be replaced by colon
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9468) Document Placement Constraints
[
https://issues.apache.org/jira/browse/YARN-9468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hunshenshi updated YARN-9468:
-
Description:
Document Placement Constraints
*First*
{code:java}
zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
* place 5 containers with tag “hbase” with affinity to a rack on which
containers with tag “zk” are running (i.e., an “hbase” container
should{color:#ff} not{color} be placed at a rack where an “zk” container is
running, given that “zk” is the TargetTag of the second constraint);
The _*not*_ word in brackets should be delete.
*Second*
{code:java}
PlacementSpec => "" | KeyVal;PlacementSpec
{code}
The semicolon should be replaced by colon
was:
Document Placement Constraints
*First*
{code:java}
zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
* place 5 containers with tag “hbase” with affinity to a rack on which
containers with tag “zk” are running (i.e., an “hbase” container
should{color:#ff} not{color} be placed at a rack where an “zk” container is
running, given that “zk” is the TargetTag of the second constraint);
The _*not*_ word in brackets should be delete.
*Second*
{code:java}
PlacementSpec => "" | KeyVal;PlacementSpec
{code}
The semicolon should be replaced by colon
> Document Placement Constraints
> --
>
> Key: YARN-9468
> URL: https://issues.apache.org/jira/browse/YARN-9468
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Priority: Major
>
> Document Placement Constraints
> *First*
> {code:java}
> zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
> * place 5 containers with tag “hbase” with affinity to a rack on which
> containers with tag “zk” are running (i.e., an “hbase” container
> should{color:#ff} not{color} be placed at a rack where an “zk” container
> is running, given that “zk” is the TargetTag of the second constraint);
> The _*not*_ word in brackets should be delete.
>
> *Second*
> {code:java}
> PlacementSpec => "" | KeyVal;PlacementSpec
> {code}
> The semicolon should be replaced by colon
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9468) Document Placement Constraints
hunshenshi created YARN-9468:
Summary: Document Placement Constraints
Key: YARN-9468
URL: https://issues.apache.org/jira/browse/YARN-9468
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 3.2.0
Reporter: hunshenshi
Document Placement Constraints
*First*
{code:java}
zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
* place 5 containers with tag “hbase” with affinity to a rack on which
containers with tag “zk” are running (i.e., an “hbase” container
should{color:#FF} not{color} be placed at a rack where an “zk” container is
running, given that “zk” is the TargetTag of the second constraint);
The _*not*_ word in brackets should be delete.
*Second*
{code:java}
PlacementSpec => "" | KeyVal;PlacementSpec
{code}
The semicolon should be replaced by colon
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9468) Document Placement Constraints
[
https://issues.apache.org/jira/browse/YARN-9468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hunshenshi updated YARN-9468:
-
Description:
Document Placement Constraints
*First*
{code:java}
zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
* place 5 containers with tag “hbase” with affinity to a rack on which
containers with tag “zk” are running (i.e., an “hbase” container
should{color:#ff} not{color} be placed at a rack where an “zk” container is
running, given that “zk” is the TargetTag of the second constraint);
The _*not*_ word in brackets should be delete.
*Second*
{code:java}
PlacementSpec => "" | KeyVal;PlacementSpec
{code}
The semicolon should be replaced by colon
was:
Document Placement Constraints
*First*
{code:java}
zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
* place 5 containers with tag “hbase” with affinity to a rack on which
containers with tag “zk” are running (i.e., an “hbase” container
should{color:#FF} not{color} be placed at a rack where an “zk” container is
running, given that “zk” is the TargetTag of the second constraint);
The _*not*_ word in brackets should be delete.
*Second*
{code:java}
PlacementSpec => "" | KeyVal;PlacementSpec
{code}
The semicolon should be replaced by colon
> Document Placement Constraints
> --
>
> Key: YARN-9468
> URL: https://issues.apache.org/jira/browse/YARN-9468
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Priority: Major
>
> Document Placement Constraints
> *First*
> {code:java}
> zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
>
> * place 5 containers with tag “hbase” with affinity to a rack on which
> containers with tag “zk” are running (i.e., an “hbase” container
> should{color:#ff} not{color} be placed at a rack where an “zk” container
> is running, given that “zk” is the TargetTag of the second constraint);
> The _*not*_ word in brackets should be delete.
>
> *Second*
> {code:java}
> PlacementSpec => "" | KeyVal;PlacementSpec
> {code}
> The semicolon should be replaced by colon
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9065) App's diags is too long,written zk error
[
https://issues.apache.org/jira/browse/YARN-9065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700344#comment-16700344
]
hunshenshi commented on YARN-9065:
--
e,
It seems.
> App's diags is too long,written zk error
>
>
> Key: YARN-9065
> URL: https://issues.apache.org/jira/browse/YARN-9065
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications, resourcemanager
>Affects Versions: 2.8.3, 2.7.7
>Reporter: hunshenshi
>Priority: Major
> Attachments: YARN-9065.patch
>
>
> When use ZKRMStateStore to store app info, App's diags is too long, written
> zk error.
> The zk error log:
> {code:java}
> 2018-11-27 15:54:30,208 [myid:1] - WARN
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@373] - Exception
> causing close of session 0x36753e37803 due to java.io.IOException: Len
> error 8603591
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9065) App's diags is too long,written zk error
[
https://issues.apache.org/jira/browse/YARN-9065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hunshenshi updated YARN-9065:
-
Description:
When use ZKRMStateStore to store app info, App's diags is too long, written zk
error.
The zk error log:
{code:java}
2018-11-27 15:54:30,208 [myid:1] - WARN
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@373] - Exception
causing close of session 0x36753e37803 due to java.io.IOException: Len
error 8603591
{code}
was:
App's diags is too long,written zk error.
The zk error log:
{code:java}
2018-11-27 15:54:30,208 [myid:1] - WARN
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@373] - Exception
causing close of session 0x36753e37803 due to java.io.IOException: Len
error 8603591
{code}
> App's diags is too long,written zk error
>
>
> Key: YARN-9065
> URL: https://issues.apache.org/jira/browse/YARN-9065
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications, resourcemanager
>Affects Versions: 2.8.3, 2.7.7
>Reporter: hunshenshi
>Priority: Major
> Attachments: YARN-9065.patch
>
>
> When use ZKRMStateStore to store app info, App's diags is too long, written
> zk error.
> The zk error log:
> {code:java}
> 2018-11-27 15:54:30,208 [myid:1] - WARN
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@373] - Exception
> causing close of session 0x36753e37803 due to java.io.IOException: Len
> error 8603591
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9065) App's diags is too long,written zk error
[
https://issues.apache.org/jira/browse/YARN-9065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hunshenshi updated YARN-9065:
-
Attachment: YARN-9065.patch
> App's diags is too long,written zk error
>
>
> Key: YARN-9065
> URL: https://issues.apache.org/jira/browse/YARN-9065
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications, resourcemanager
>Affects Versions: 2.8.3, 2.7.7
>Reporter: hunshenshi
>Priority: Major
> Attachments: YARN-9065.patch
>
>
> App's diags is too long,written zk error.
> The zk error log:
> {code:java}
> 2018-11-27 15:54:30,208 [myid:1] - WARN
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@373] - Exception
> causing close of session 0x36753e37803 due to java.io.IOException: Len
> error 8603591
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9065) App's diags is too long,written zk error
hunshenshi created YARN-9065:
Summary: App's diags is too long,written zk error
Key: YARN-9065
URL: https://issues.apache.org/jira/browse/YARN-9065
Project: Hadoop YARN
Issue Type: Bug
Components: applications, resourcemanager
Affects Versions: 2.7.7, 2.8.3
Reporter: hunshenshi
App's diags is too long,written zk error.
The zk error log:
{code:java}
2018-11-27 15:54:30,208 [myid:1] - WARN
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@373] - Exception
causing close of session 0x36753e37803 due to java.io.IOException: Len
error 8603591
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
