[jira] [Created] (YARN-10322) FairScheduler limit resources for a single application
yehuanhuan created YARN-10322: - Summary: FairScheduler limit resources for a single application Key: YARN-10322 URL: https://issues.apache.org/jira/browse/YARN-10322 Project: Hadoop YARN Issue Type: Improvement Components: fairscheduler Affects Versions: 2.7.2 Reporter: yehuanhuan When the cluster is busy, too many resources saved by a single task will cause other tasks can't get resources and block. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10322) FairScheduler limit reserved resources for a single application
[ https://issues.apache.org/jira/browse/YARN-10322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yehuanhuan updated YARN-10322: -- Description: When the cluster is busy, too many resources reserved by a single task will cause other tasks can't get resources and block. (was: When the cluster is busy, too many resources saved by a single task will cause other tasks can't get resources and block.) Summary: FairScheduler limit reserved resources for a single application (was: FairScheduler limit resources for a single application) > FairScheduler limit reserved resources for a single application > --- > > Key: YARN-10322 > URL: https://issues.apache.org/jira/browse/YARN-10322 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.7.2 >Reporter: yehuanhuan >Priority: Major > > When the cluster is busy, too many resources reserved by a single task will > cause other tasks can't get resources and block. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10322) FairScheduler limit reserved resources for a single application
[ https://issues.apache.org/jira/browse/YARN-10322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yehuanhuan updated YARN-10322: -- Attachment: YARN-10322.001.patch > FairScheduler limit reserved resources for a single application > --- > > Key: YARN-10322 > URL: https://issues.apache.org/jira/browse/YARN-10322 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.7.2 >Reporter: yehuanhuan >Priority: Major > Attachments: YARN-10322.001.patch > > > When the cluster is busy, too many resources reserved by a single task will > cause other tasks can't get resources and block. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10322) FairScheduler limit reserved resources for a single application
[ https://issues.apache.org/jira/browse/YARN-10322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yehuanhuan updated YARN-10322: -- Attachment: (was: YARN-10322.001.patch) > FairScheduler limit reserved resources for a single application > --- > > Key: YARN-10322 > URL: https://issues.apache.org/jira/browse/YARN-10322 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.7.2 >Reporter: yehuanhuan >Priority: Major > Attachments: YARN-10322.001.patch > > > When the cluster is busy, too many resources reserved by a single task will > cause other tasks can't get resources and block. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10315) Avoid sending RMNodeResoureupdate event if resource is same
[ https://issues.apache.org/jira/browse/YARN-10315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17143537#comment-17143537 ] yehuanhuan commented on YARN-10315: --- Can it be solved by comparing node resource usage before and after processing the completed container at AbstractYarnScheduler#nodeUpdate? > Avoid sending RMNodeResoureupdate event if resource is same > --- > > Key: YARN-10315 > URL: https://issues.apache.org/jira/browse/YARN-10315 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin Chundatt >Assignee: Sushil Ks >Priority: Major > > When the node is in DECOMMISSIONING state the RMNodeResourceUpdateEvent is > send for every heartbeat . Which will result in scheduler resource update. > Avoid sending the same. > Scheduler node resource update iterates through all the queues for resource > update which is costly.. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10315) Avoid sending RMNodeResoureupdate event if resource is same
[ https://issues.apache.org/jira/browse/YARN-10315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17143537#comment-17143537 ] yehuanhuan edited comment on YARN-10315 at 6/24/20, 5:33 AM: - hi [~Sushil-K-S], Can it be solved by comparing node resource usage before and after processing the completed container at AbstractYarnScheduler#nodeUpdate? was (Author: yehuanhuan): Can it be solved by comparing node resource usage before and after processing the completed container at AbstractYarnScheduler#nodeUpdate? > Avoid sending RMNodeResoureupdate event if resource is same > --- > > Key: YARN-10315 > URL: https://issues.apache.org/jira/browse/YARN-10315 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin Chundatt >Assignee: Sushil Ks >Priority: Major > > When the node is in DECOMMISSIONING state the RMNodeResourceUpdateEvent is > send for every heartbeat . Which will result in scheduler resource update. > Avoid sending the same. > Scheduler node resource update iterates through all the queues for resource > update which is costly.. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10326) ApplicationHistory Web UI should show the queue where the task is actually running
yehuanhuan created YARN-10326: - Summary: ApplicationHistory Web UI should show the queue where the task is actually running Key: YARN-10326 URL: https://issues.apache.org/jira/browse/YARN-10326 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 3.2.1, 2.7.2 Reporter: yehuanhuan ApplicationHistory Web UI should show the queue where the task is actually running instead of the queue specified by the client. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10332) RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state
yehuanhuan created YARN-10332: - Summary: RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state Key: YARN-10332 URL: https://issues.apache.org/jira/browse/YARN-10332 Project: Hadoop YARN Issue Type: Improvement Components: resourcemanager Affects Versions: 3.2.1 Reporter: yehuanhuan RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10332) RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state
[ https://issues.apache.org/jira/browse/YARN-10332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149336#comment-17149336 ] yehuanhuan commented on YARN-10332: --- [~bibinchundatt] In RMNodeImpl, RESOURCE_UPDATE event was registered twice in DECOMMISSIONING state. > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state > > > Key: YARN-10332 > URL: https://issues.apache.org/jira/browse/YARN-10332 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.2.1 >Reporter: yehuanhuan >Priority: Minor > Attachments: YARN-10332.001.patch > > > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10332) RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state
[ https://issues.apache.org/jira/browse/YARN-10332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149338#comment-17149338 ] yehuanhuan edited comment on YARN-10332 at 7/1/20, 11:22 AM: - [~bibinchundatt] [~adam.antal] Thank you for your reply. This transition was registered twice in RMNodeImpl. was (Author: yehuanhuan): [~bibinchundatt][~adam.antal] Thank you for your reply. This transition was registered twice in RMNodeImpl. > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state > > > Key: YARN-10332 > URL: https://issues.apache.org/jira/browse/YARN-10332 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.2.1 >Reporter: yehuanhuan >Priority: Minor > Attachments: YARN-10332.001.patch > > > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-10332) RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state
[ https://issues.apache.org/jira/browse/YARN-10332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yehuanhuan updated YARN-10332: -- Comment: was deleted (was: [~bibinchundatt] In RMNodeImpl, RESOURCE_UPDATE event was registered twice in DECOMMISSIONING state. ) > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state > > > Key: YARN-10332 > URL: https://issues.apache.org/jira/browse/YARN-10332 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.2.1 >Reporter: yehuanhuan >Priority: Minor > Attachments: YARN-10332.001.patch > > > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10332) RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state
[ https://issues.apache.org/jira/browse/YARN-10332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149338#comment-17149338 ] yehuanhuan edited comment on YARN-10332 at 7/1/20, 11:22 AM: - [~bibinchundatt][~adam.antal] Thank you for your reply. This transition was registered twice in RMNodeImpl. was (Author: yehuanhuan): [~adam.antal] Thank you for your reply. This transition was registered twice in RMNodeImpl. > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state > > > Key: YARN-10332 > URL: https://issues.apache.org/jira/browse/YARN-10332 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.2.1 >Reporter: yehuanhuan >Priority: Minor > Attachments: YARN-10332.001.patch > > > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10332) RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state
[ https://issues.apache.org/jira/browse/YARN-10332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149338#comment-17149338 ] yehuanhuan commented on YARN-10332: --- [~adam.antal] Thank you for your reply. This transition was registered twice in RMNodeImpl. > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state > > > Key: YARN-10332 > URL: https://issues.apache.org/jira/browse/YARN-10332 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.2.1 >Reporter: yehuanhuan >Priority: Minor > Attachments: YARN-10332.001.patch > > > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10332) RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state
[ https://issues.apache.org/jira/browse/YARN-10332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149338#comment-17149338 ] yehuanhuan edited comment on YARN-10332 at 7/1/20, 11:23 AM: - [~bibinchundatt] and [~adam.antal] Thank you for your reply. This transition was registered twice in RMNodeImpl. was (Author: yehuanhuan): [~bibinchundatt] [~adam.antal] Thank you for your reply. This transition was registered twice in RMNodeImpl. > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state > > > Key: YARN-10332 > URL: https://issues.apache.org/jira/browse/YARN-10332 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.2.1 >Reporter: yehuanhuan >Priority: Minor > Attachments: YARN-10332.001.patch > > > RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10338) AM log redirect failed when app finished in RM web UI.
[ https://issues.apache.org/jira/browse/YARN-10338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yehuanhuan resolved YARN-10338. --- Resolution: Not A Bug > AM log redirect failed when app finished in RM web UI. > -- > > Key: YARN-10338 > URL: https://issues.apache.org/jira/browse/YARN-10338 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.2.1 >Reporter: yehuanhuan >Priority: Major > Attachments: am_log_request_cancel.png > > > AM log redirect failed when app finished in RM web UI. The redirected request > returned a canceled status and cannot be redirected. As shown below. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10338) AM log redirect failed when app finished in RM web UI.
[ https://issues.apache.org/jira/browse/YARN-10338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150922#comment-17150922 ] yehuanhuan commented on YARN-10338: --- yarn.log.server.url Configuration must start with "http://";. > AM log redirect failed when app finished in RM web UI. > -- > > Key: YARN-10338 > URL: https://issues.apache.org/jira/browse/YARN-10338 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.2.1 >Reporter: yehuanhuan >Priority: Major > Attachments: am_log_request_cancel.png > > > AM log redirect failed when app finished in RM web UI. The redirected request > returned a canceled status and cannot be redirected. As shown below. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10338) AM log redirect failed when app finished in RM web UI.
[ https://issues.apache.org/jira/browse/YARN-10338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yehuanhuan updated YARN-10338: -- Description: AM log redirect failed when app finished in RM web UI. The redirected request returned a canceled status and cannot be redirected. As shown below. (was: AM log redirect failed when app finished in RM web UI. The redirected request returned a canceled status. As shown below.) > AM log redirect failed when app finished in RM web UI. > -- > > Key: YARN-10338 > URL: https://issues.apache.org/jira/browse/YARN-10338 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.2.1 >Reporter: yehuanhuan >Priority: Major > Attachments: am_log_request_cancel.png > > > AM log redirect failed when app finished in RM web UI. The redirected request > returned a canceled status and cannot be redirected. As shown below. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10338) AM log redirect failed when app finished in RM web UI.
yehuanhuan created YARN-10338: - Summary: AM log redirect failed when app finished in RM web UI. Key: YARN-10338 URL: https://issues.apache.org/jira/browse/YARN-10338 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.2.1 Reporter: yehuanhuan Attachments: am_log_request_cancel.png AM log redirect failed when app finished in RM web UI. The redirected request returned a canceled status. As shown below. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10395) ReservedContainer Node is added to blackList of application due to this node can not allocate other container
[ https://issues.apache.org/jira/browse/YARN-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176853#comment-17176853 ] yehuanhuan commented on YARN-10395: --- The same problem exists in fairscheduler. And this problem can be solved in FairScheduler#attemptScheduling. > ReservedContainer Node is added to blackList of application due to this node > can not allocate other container > - > > Key: YARN-10395 > URL: https://issues.apache.org/jira/browse/YARN-10395 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 2.9.2 >Reporter: chan >Priority: Major > > Now,if a app reserved a node,but the node is added to app`s blacklist. > when this node send heartbeat to resourcemanager,the reserved container > allocate fails,it will make this node can not allocate other container even > thought this node have enough memory or vcores.so i think we can release this > reserved container when the reserved node is in the black list of this app. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10971) Continuous scheduling reduces invalid calculations
[ https://issues.apache.org/jira/browse/YARN-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426566#comment-17426566 ] yehuanhuan commented on YARN-10971: --- When node is null should not break the loop. > Continuous scheduling reduces invalid calculations > -- > > Key: YARN-10971 > URL: https://issues.apache.org/jira/browse/YARN-10971 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.7.2 >Reporter: Jiangminghai >Priority: Minor > Fix For: 2.7.2 > > Attachments: YARN-10971.0.patch > > > In FairScheduler#continuousSchedulingAttempt(), nodes has been sorted by > available resources，iterate all nodes in continuous scheduling. > Should break the loop, when > *Resources.fitsIn(minimumAllocation,node.getUnallocatedResource()) == false*. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9977) Support monitor threads number in ContainersMonitorImpl
[ https://issues.apache.org/jira/browse/YARN-9977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988416#comment-16988416 ] yehuanhuan commented on YARN-9977: -- hi zhoukang Currently, we are using the system file that is in the /proc/pid/status path to monitor the number of threads. Do you have a good idea？ > Support monitor threads number in ContainersMonitorImpl > --- > > Key: YARN-9977 > URL: https://issues.apache.org/jira/browse/YARN-9977 > Project: Hadoop YARN > Issue Type: New Feature > Components: nodemanager >Reporter: zhoukang >Assignee: zhoukang >Priority: Major > > In this jira, we want add a feature to monitor thread number for given > container. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10088) Too many threads created by container caused NM shutdown
yehuanhuan created YARN-10088: - Summary: Too many threads created by container caused NM shutdown Key: YARN-10088 URL: https://issues.apache.org/jira/browse/YARN-10088 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.7.2 Reporter: yehuanhuan Because containerMonitorImpl only monitors physical memory and virtual memory. When the number of threads created by the container exceeds the number of user threads in the system, NM will exit. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10095) Fix help message for yarn rmadmin
[
https://issues.apache.org/jira/browse/YARN-10095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019300#comment-17019300
]
yehuanhuan commented on YARN-10095:
---
Hi Xieming Li, I found that using this command “yarn rmadmin -help
-refreshNodes” can solve.
> Fix help message for yarn rmadmin
> -
>
> Key: YARN-10095
> URL: https://issues.apache.org/jira/browse/YARN-10095
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Xieming Li
>Assignee: Xieming Li
>Priority: Minor
>
> (This issue is identified by [~aajisaka] in
> https://issues.apache.org/jira/browse/HADOOP-16753)
> The help message of yarn rmadmin seems broken.
> Current:
> {code:java}
> $ yarn rmadmin -help refreshNodes 2>/dev/null
> $
> $ yarn rmadmin -help refreshNodes
> Usage: yarn rmadmin
>-refreshQueues
>-refreshNodes [-g|graceful [timeout in seconds] -client|server]
>-refreshNodesResources
>-refreshSuperUserGroupsConfiguration
>-refreshUserToGroupsMappings
>-refreshAdminAcls
>-refreshServiceAcl
>-getGroups [username]
>-addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">
>-removeFromClusterNodeLabels (label splitted by ",")
>-replaceLabelsOnNode <"node1[:port]=label1,label2
> node2[:port]=label1,label2"> [-failOnUnknownNodes]
>-directlyAccessNodeLabelStore
>-refreshClusterMaxPriority
>-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout])
> or
> [NodeID] [resourcetypes] ([OvercommitTimeout]).
>-help [cmd]Generic options supported are:
> -conf specify an application configuration file
> -Ddefine a value for a given property
> -fs specify default filesystem URL to use,
> overrides 'fs.defaultFS' property from configurations.
> -jt specify a ResourceManager
> -files specify a comma-separated list of files to
> be copied to the map reduce cluster
> -libjarsspecify a comma-separated list of jar files
> to be included in the classpath
> -archives specify a comma-separated list of archives
> to be unarchived on the compute machinesThe general command line syntax is:
> command [genericOptions] [commandOptions]
> {code}
>
>
> Expected:
> {code:java}
> $ yarn rmadmin -help refreshNodes 2>/dev/null
> -refreshNodes [-g|graceful [timeout in seconds] -client|server]
> $ yarn rmadmin -help refreshNodes
> -refreshNodes [-g|graceful [timeout in seconds] -client|server]
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10145) Error in creating hbase tables
yehuanhuan created YARN-10145:
-
Summary: Error in creating hbase tables
Key: YARN-10145
URL: https://issues.apache.org/jira/browse/YARN-10145
Project: Hadoop YARN
Issue Type: Bug
Components: ATSv2
Affects Versions: 3.2.1
Reporter: yehuanhuan
When using the command(bin/hadoop
org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
-create) to create the timeline service schema，I get the following error.
{panel:title=ERROR}
2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in creating
hbase tables:
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
tried to access method shaded.com.google.common.base.Stopwatch.()V from
class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
at
org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
at
org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
at
org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
at
org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
at
org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
at
org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
at
org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
at
org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
at
org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
at
org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
at
org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
Caused by: java.lang.IllegalAccessError: tried to access method
shaded.com.google.common.base.Stopwatch.()V from class
org.apache.hadoop.hbase.zookeeper.MetaTableLocator
at
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
at
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
at
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
at
org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
at
org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
{panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039625#comment-17039625
]
yehuanhuan commented on YARN-10145:
---
https://issues.apache.org/jira/secure/ViewProfile.jspa?name=prabhujoseph Thanks
for your comment, I think this problem may be caused by the guava package
conflict.
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yehuanhuan updated YARN-10145:
--
Comment: was deleted
(was: https://issues.apache.org/jira/secure/ViewProfile.jspa?name=prabhujoseph
Thanks for your comment, I think this problem may be caused by the guava
package conflict.)
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yehuanhuan updated YARN-10145:
--
Comment: was deleted
(was: name=prabhujoseph Thanks for your comment, I think this problem may be
caused by the guava package conflict.)
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039632#comment-17039632
]
yehuanhuan commented on YARN-10145:
---
name=prabhujoseph Thanks for your comment, I think this problem may be caused
by the guava package conflict.
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039642#comment-17039642
]
yehuanhuan commented on YARN-10145:
---
[~prabhujoseph] Thanks for your comment, I think this problem may be caused by
the guava package conflict.
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039811#comment-17039811
]
yehuanhuan commented on YARN-10145:
---
The error does not occur when building with hbase2.0. To use HBase 2.0.0-beta1,
build from source with option -Dhbase.profile=2.0
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10290) Resourcemanager recover failed when fair scheduler queue acl changed
yehuanhuan created YARN-10290:
-
Summary: Resourcemanager recover failed when fair scheduler queue
acl changed
Key: YARN-10290
URL: https://issues.apache.org/jira/browse/YARN-10290
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.7.2
Reporter: yehuanhuan
Resourcemanager recover failed when fair scheduler queue acl changed. Because
of queue acl changed, when recover the application (addApplication() in
fairscheduler) is rejected. Then recover the applicationAttempt
(addApplicationAttempt() in fairscheduler) get Application is null. Repeat as
follows.
# user run a long running application.
# change queue acl (aclSubmitApps) so that the user does not have permission.
# restart the RM.
{code:java}
2020-05-25 16:04:06,191 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Updating
application application_1590393162216_0005 with final state: FAILED
2020-05-25 16:04:06,192 ERROR
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Failed to
load/recover state
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java:663)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1246)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:116)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1072)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1036)
at
org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:789)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:105)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.recoverAppAttempts(RMAppImpl.java:845)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.access$1900(RMAppImpl.java:102)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl$RMAppRecoveredTransition.transition(RMAppImpl.java:897)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl$RMAppRecoveredTransition.transition(RMAppImpl.java:850)
at
org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:723)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recoverApplication(RMAppManager.java:322)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recover(RMAppManager.java:427)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.recover(ResourceManager.java:1173)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:584)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:980)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1021)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1017)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1659)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1017)
at
org.apache.hadoop.yarn.server.resourcemanager.AdminService.transitionToActive(AdminService.java:301)
at
org.apache.hadoop.yarn.server.r
[jira] [Updated] (YARN-10290) Resourcemanager recover failed when fair scheduler queue acl changed
[
https://issues.apache.org/jira/browse/YARN-10290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yehuanhuan updated YARN-10290:
--
Description:
Resourcemanager recover failed when fair scheduler queue acl changed. Because
of queue acl changed, when recover the application (addApplication() in
fairscheduler) is rejected. Then recover the applicationAttempt
(addApplicationAttempt() in fairscheduler) get Application is null. This will
lead to two RM is at standby. Repeat as follows.
# user run a long running application.
# change queue acl (aclSubmitApps) so that the user does not have permission.
# restart the RM.
{code:java}
2020-05-25 16:04:06,191 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Updating
application application_1590393162216_0005 with final state: FAILED
2020-05-25 16:04:06,192 ERROR
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Failed to
load/recover state
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java:663)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1246)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:116)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1072)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1036)
at
org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:789)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:105)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.recoverAppAttempts(RMAppImpl.java:845)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.access$1900(RMAppImpl.java:102)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl$RMAppRecoveredTransition.transition(RMAppImpl.java:897)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl$RMAppRecoveredTransition.transition(RMAppImpl.java:850)
at
org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:723)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recoverApplication(RMAppManager.java:322)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recover(RMAppManager.java:427)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.recover(ResourceManager.java:1173)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:584)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:980)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1021)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1017)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1659)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1017)
at
org.apache.hadoop.yarn.server.resourcemanager.AdminService.transitionToActive(AdminService.java:301)
at
org.apache.hadoop.yarn.server.resourcemanager.EmbeddedElectorService.becomeActive(EmbeddedElectorService.java:126)
at
org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.jav
