[jira] [Commented] (YARN-10110) In Federation Secure cluster Application submission fails when authorization is enabled
[ https://issues.apache.org/jira/browse/YARN-10110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039751#comment-17039751 ] Bilwa S T commented on YARN-10110: -- [~bibinchundatt] YARN-6539 supports security. > In Federation Secure cluster Application submission fails when authorization > is enabled > --- > > Key: YARN-10110 > URL: https://issues.apache.org/jira/browse/YARN-10110 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sushanta Sen >Assignee: Bilwa S T >Priority: Blocker > Attachments: YARN-10110.001.patch, YARN-10110.002.patch > > > 【Precondition】： > 1. Secure Federated cluster is available > 2. Add the below configuration in Router and client core-site.xml > hadoop.security.authorization= true > 3. Restart the router service > 【Test step】： > 1. Go to router client bin path and submit a MR PI job > 2. Observe the client console screen > 【Expect Output】： > No error should be thrown and Job should be successful > 【Actual Output】： > Job failed prompting "Protocol interface > org.apache.hadoop.yarn.api.ApplicationClientProtocolPB is not known.," > 【Additional Note】: > But on setting the parameter as false, job is submitted and success. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10142) Distributed shell: add support for localization visibility
[
https://issues.apache.org/jira/browse/YARN-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko resolved YARN-10142.
-
Resolution: Duplicate
> Distributed shell: add support for localization visibility
> --
>
> Key: YARN-10142
> URL: https://issues.apache.org/jira/browse/YARN-10142
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: distributed-shell
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> The localization is now hard coded in DistributedShell:
> {noformat}
> FileStatus scFileStatus = fs.getFileStatus(dst);
> LocalResource scRsrc =
> LocalResource.newInstance(
> URL.fromURI(dst.toUri()),
> LocalResourceType.FILE, LocalResourceVisibility.APPLICATION,
> scFileStatus.getLen(), scFileStatus.getModificationTime());
> localResources.put(fileDstPath, scRsrc);
> {noformat}
> However, sometimes it's useful if you have the possibility to change this to
> PRIVATE/PUBLIC for testing purposes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9208) Distributed shell allow LocalResourceVisibility to be specified
[
https://issues.apache.org/jira/browse/YARN-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039737#comment-17039737
]
Hadoop QA commented on YARN-9208:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red} YARN-9208 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-9208 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958513/YARN-9208-004.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25537/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Distributed shell allow LocalResourceVisibility to be specified
> ---
>
> Key: YARN-9208
> URL: https://issues.apache.org/jira/browse/YARN-9208
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Bibin Chundatt
>Assignee: Prabhu Joseph
>Priority: Minor
> Attachments: YARN-9208-001.patch, YARN-9208-002.patch,
> YARN-9208-003.patch, YARN-9208-004.patch
>
>
> YARN-9008 add feature to add list of files to be localized.
> Would be great to have Visibility type too. Allows testing of PRIVATE and
> PUBLIC type too
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10142) Distributed shell: add support for localization visibility
[
https://issues.apache.org/jira/browse/YARN-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039736#comment-17039736
]
Peter Bacsko commented on YARN-10142:
-
Yes, indeed. Closing this as duplicate.
> Distributed shell: add support for localization visibility
> --
>
> Key: YARN-10142
> URL: https://issues.apache.org/jira/browse/YARN-10142
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: distributed-shell
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> The localization is now hard coded in DistributedShell:
> {noformat}
> FileStatus scFileStatus = fs.getFileStatus(dst);
> LocalResource scRsrc =
> LocalResource.newInstance(
> URL.fromURI(dst.toUri()),
> LocalResourceType.FILE, LocalResourceVisibility.APPLICATION,
> scFileStatus.getLen(), scFileStatus.getModificationTime());
> localResources.put(fileDstPath, scRsrc);
> {noformat}
> However, sometimes it's useful if you have the possibility to change this to
> PRIVATE/PUBLIC for testing purposes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10142) Distributed shell: add support for localization visibility
[
https://issues.apache.org/jira/browse/YARN-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039733#comment-17039733
]
Prabhu Joseph commented on YARN-10142:
--
[~pbacsko] This looks like a duplicate of YARN-9208. Thanks.
> Distributed shell: add support for localization visibility
> --
>
> Key: YARN-10142
> URL: https://issues.apache.org/jira/browse/YARN-10142
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: distributed-shell
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> The localization is now hard coded in DistributedShell:
> {noformat}
> FileStatus scFileStatus = fs.getFileStatus(dst);
> LocalResource scRsrc =
> LocalResource.newInstance(
> URL.fromURI(dst.toUri()),
> LocalResourceType.FILE, LocalResourceVisibility.APPLICATION,
> scFileStatus.getLen(), scFileStatus.getModificationTime());
> localResources.put(fileDstPath, scRsrc);
> {noformat}
> However, sometimes it's useful if you have the possibility to change this to
> PRIVATE/PUBLIC for testing purposes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10139) ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB
[
https://issues.apache.org/jira/browse/YARN-10139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039722#comment-17039722
]
Hudson commented on YARN-10139:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17965 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17965/])
YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster
(sunilg: rev 6526f95bd281fc011f8776d21ff933087c5924de)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesConfigurationMutation.java
> ValidateAndGetSchedulerConfiguration API fails when cluster max allocation >
> default 8GB
>
>
> Key: YARN-10139
> URL: https://issues.apache.org/jira/browse/YARN-10139
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0, 3.2.2, 3.1.4
>
> Attachments: YARN-10139-001.patch, YARN-10139-002.patch
>
>
> ValidateAndGetSchedulerConfiguration fails when the cluster max allocation
> (yarn.scheduler.maximum-allocation-mb) is set to resource (eg: 16GB) >
> default 8GB in yarn-site.xml.
> As part of validation API, there are two configuration used -
> CapacitySchedulerConfiguration and Configuration (yarn-site.xml). When
> CapacityScheduler is initialized with CapacitySchedulerConfiguration, as part
> of queues initialization, it checks the queue maximum allocation which is not
> present and so checks cluster max allocation which is not present (it is
> present only in YarnConfiguration) and defaults to 8GB. This will fail as
> queue max allocation 8GB is decreased from previous 16GB.
> {code}
> 2020-02-14 07:38:46,087 WARN
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices:
> CapacityScheduler configuration validation failed:java.io.IOException: Failed
> to re-init queues : Trying to reinitialize root.default.c1.c3 the maximum
> allocation size can not be decreased! Current setting: vCores:88>, trying to set it to:
> {code}
> CapacityScheduler initialize code reads a yarn config from
> CapacitySchedulerConfiguration causing the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10139) ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB
[
https://issues.apache.org/jira/browse/YARN-10139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039716#comment-17039716
]
Prabhu Joseph commented on YARN-10139:
--
Thanks [~sunilg].
> ValidateAndGetSchedulerConfiguration API fails when cluster max allocation >
> default 8GB
>
>
> Key: YARN-10139
> URL: https://issues.apache.org/jira/browse/YARN-10139
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0, 3.2.2, 3.1.4
>
> Attachments: YARN-10139-001.patch, YARN-10139-002.patch
>
>
> ValidateAndGetSchedulerConfiguration fails when the cluster max allocation
> (yarn.scheduler.maximum-allocation-mb) is set to resource (eg: 16GB) >
> default 8GB in yarn-site.xml.
> As part of validation API, there are two configuration used -
> CapacitySchedulerConfiguration and Configuration (yarn-site.xml). When
> CapacityScheduler is initialized with CapacitySchedulerConfiguration, as part
> of queues initialization, it checks the queue maximum allocation which is not
> present and so checks cluster max allocation which is not present (it is
> present only in YarnConfiguration) and defaults to 8GB. This will fail as
> queue max allocation 8GB is decreased from previous 16GB.
> {code}
> 2020-02-14 07:38:46,087 WARN
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices:
> CapacityScheduler configuration validation failed:java.io.IOException: Failed
> to re-init queues : Trying to reinitialize root.default.c1.c3 the maximum
> allocation size can not be decreased! Current setting: vCores:88>, trying to set it to:
> {code}
> CapacityScheduler initialize code reads a yarn config from
> CapacitySchedulerConfiguration causing the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10133) For yarn Federation, there is no routeradmin command to manage
[ https://issues.apache.org/jira/browse/YARN-10133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilwa S T reassigned YARN-10133: Assignee: Bilwa S T > For yarn Federation, there is no routeradmin command to manage > -- > > Key: YARN-10133 > URL: https://issues.apache.org/jira/browse/YARN-10133 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, yarn >Reporter: Sushanta Sen >Assignee: Bilwa S T >Priority: Major > > In an yarn Federated cluster, there does not exist any routeradmin command to > manage, like HDFS has 'dfsrouteradmin' command. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10152) Fix findbugs warnings in hadoop-yarn-applications-mawo-core module
[
https://issues.apache.org/jira/browse/YARN-10152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned YARN-10152:
Assignee: Akira Ajisaka
> Fix findbugs warnings in hadoop-yarn-applications-mawo-core module
> --
>
> Key: YARN-10152
> URL: https://issues.apache.org/jira/browse/YARN-10152
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>
> {noformat}
> FindBugs :
>
> module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core
> Class org.apache.hadoop.applications.mawo.server.common.TaskStatus
> implements Cloneable but does not define or use clone method At
> TaskStatus.java:does not define or use clone method At TaskStatus.java:[lines
> 39-346]
> Equals method for
> org.apache.hadoop.applications.mawo.server.worker.WorkerId assumes the
> argument is of type WorkerId At WorkerId.java:the argument is of type
> WorkerId At WorkerId.java:[line 114]
>
> org.apache.hadoop.applications.mawo.server.worker.WorkerId.equals(Object)
> does not check for null argument At WorkerId.java:null argument At
> WorkerId.java:[lines 114-115] {noformat}
> Detail:
> [https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1414/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-mawo_hadoop-yarn-applications-mawo-core-warnings.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10110) In Federation Secure cluster Application submission fails when authorization is enabled
[ https://issues.apache.org/jira/browse/YARN-10110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039688#comment-17039688 ] Bibin Chundatt commented on YARN-10110: --- [~BilwaST] Could you point me to the jira which supports Federation security. Also its better to group all the security related federation under one subtask and link it to YARN-5597 > In Federation Secure cluster Application submission fails when authorization > is enabled > --- > > Key: YARN-10110 > URL: https://issues.apache.org/jira/browse/YARN-10110 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sushanta Sen >Assignee: Bilwa S T >Priority: Blocker > Attachments: YARN-10110.001.patch, YARN-10110.002.patch > > > 【Precondition】： > 1. Secure Federated cluster is available > 2. Add the below configuration in Router and client core-site.xml > hadoop.security.authorization= true > 3. Restart the router service > 【Test step】： > 1. Go to router client bin path and submit a MR PI job > 2. Observe the client console screen > 【Expect Output】： > No error should be thrown and Job should be successful > 【Actual Output】： > Job failed prompting "Protocol interface > org.apache.hadoop.yarn.api.ApplicationClientProtocolPB is not known.," > 【Additional Note】: > But on setting the parameter as false, job is submitted and success. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10151) Disable Capacity Scheduler's move app between queue functionality
[
https://issues.apache.org/jira/browse/YARN-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039665#comment-17039665
]
Tao Yang commented on YARN-10151:
-
Hi, [~leftnoteasy] FYI, a related issue which can make that happen has been
solved in YARN-9838.
> Disable Capacity Scheduler's move app between queue functionality
> -
>
> Key: YARN-10151
> URL: https://issues.apache.org/jira/browse/YARN-10151
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Wangda Tan
>Priority: Critical
>
> Saw this happened in many clusters: Capacity Scheduler cannot work correctly
> with the move app between queue features. It will cause weird JMX issue,
> resource accounting issue, etc. In a lot of causes it will cause RM
> completely hung and available resource became negative, nothing can be
> allocated after that. We should turn off CapacityScheduler's move app between
> queue feature. (see:
> {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler#moveApplication}}
> )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039642#comment-17039642
]
yehuanhuan commented on YARN-10145:
---
[~prabhujoseph] Thanks for your comment, I think this problem may be caused by
the guava package conflict.
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10152) Fix findbugs warnings in hadoop-yarn-applications-mawo-core module
Akira Ajisaka created YARN-10152:
Summary: Fix findbugs warnings in
hadoop-yarn-applications-mawo-core module
Key: YARN-10152
URL: https://issues.apache.org/jira/browse/YARN-10152
Project: Hadoop YARN
Issue Type: Bug
Reporter: Akira Ajisaka
{noformat}
FindBugs :
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core
Class org.apache.hadoop.applications.mawo.server.common.TaskStatus
implements Cloneable but does not define or use clone method At
TaskStatus.java:does not define or use clone method At TaskStatus.java:[lines
39-346]
Equals method for
org.apache.hadoop.applications.mawo.server.worker.WorkerId assumes the argument
is of type WorkerId At WorkerId.java:the argument is of type WorkerId At
WorkerId.java:[line 114]
org.apache.hadoop.applications.mawo.server.worker.WorkerId.equals(Object) does
not check for null argument At WorkerId.java:null argument At
WorkerId.java:[lines 114-115] {noformat}
Detail:
[https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1414/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-mawo_hadoop-yarn-applications-mawo-core-warnings.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yehuanhuan updated YARN-10145:
--
Comment: was deleted
(was: name=prabhujoseph Thanks for your comment, I think this problem may be
caused by the guava package conflict.)
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039632#comment-17039632
]
yehuanhuan commented on YARN-10145:
---
name=prabhujoseph Thanks for your comment, I think this problem may be caused
by the guava package conflict.
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yehuanhuan updated YARN-10145:
--
Comment: was deleted
(was: https://issues.apache.org/jira/secure/ViewProfile.jspa?name=prabhujoseph
Thanks for your comment, I think this problem may be caused by the guava
package conflict.)
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039625#comment-17039625
]
yehuanhuan commented on YARN-10145:
---
https://issues.apache.org/jira/secure/ViewProfile.jspa?name=prabhujoseph Thanks
for your comment, I think this problem may be caused by the guava package
conflict.
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-6506) Fix the code vulnerability of org.apache.hadoop.yarn.sls.SLSRunner.simulateInfoMap
[ https://issues.apache.org/jira/browse/YARN-6506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved YARN-6506. - Resolution: Cannot Reproduce Now there are no findbugs warnings in the module. Closing. > Fix the code vulnerability of > org.apache.hadoop.yarn.sls.SLSRunner.simulateInfoMap > -- > > Key: YARN-6506 > URL: https://issues.apache.org/jira/browse/YARN-6506 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Yufei Gu >Priority: Major > > It is reported by findbugs in YARN-6423. > MS_MUTABLE_COLLECTION: Field is a mutable collection > A mutable collection instance is assigned to a final static field, thus can > be changed by malicious code or by accident from another package. Consider > wrapping this field into Collections.unmodifiableSet/List/Map/etc. to avoid > this vulnerability. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-9981) Fix findbugs warning in timelineservice
[
https://issues.apache.org/jira/browse/YARN-9981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka resolved YARN-9981.
-
Target Version/s: (was: 3.3.0)
Resolution: Cannot Reproduce
Now there are no warnings in the module. Closing.
> Fix findbugs warning in timelineservice
> ---
>
> Key: YARN-9981
> URL: https://issues.apache.org/jira/browse/YARN-9981
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Priority: Minor
>
> Findbugs are complaining about this:
> {noformat}
> module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client
>Boxed value is unboxed and then immediately reboxed in
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result,
> byte[], byte[], KeyConverter, ValueConverter, boolean) At
> ColumnRWHelper.java:then immediately reboxed in
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result,
> byte[], byte[], KeyConverter, ValueConverter, boolean) At
> ColumnRWHelper.java:[line 335]
> {noformat}
> Let's fix it!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-7576) Findbug warning for Resource exposing internal representation
[
https://issues.apache.org/jira/browse/YARN-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka resolved YARN-7576.
-
Assignee: (was: Wangda Tan)
Resolution: Duplicate
Yes, we can close this.
> Findbug warning for Resource exposing internal representation
> -
>
> Key: YARN-7576
> URL: https://issues.apache.org/jira/browse/YARN-7576
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.0.0
>Reporter: Jason Darrell Lowe
>Priority: Major
> Attachments: YARN-7576.001.patch
>
>
> Precommit builds are complaining about a findbugs warning:
> {noformat}
> EIorg.apache.hadoop.yarn.api.records.Resource.getResources() may expose
> internal representation by returning Resource.resources
>
> Bug type EI_EXPOSE_REP (click for details)
> In class org.apache.hadoop.yarn.api.records.Resource
> In method org.apache.hadoop.yarn.api.records.Resource.getResources()
> Field org.apache.hadoop.yarn.api.records.Resource.resources
> At Resource.java:[line 213]
> Returning a reference to a mutable object value stored in one of the object's
> fields exposes the internal representation of the object. If instances are
> accessed by untrusted code, and unchecked changes to the mutable object would
> compromise security or other important properties, you will need to do
> something different. Returning a new copy of the object is better approach in
> many situations.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8374) Upgrade objenesis to 2.6
[ https://issues.apache.org/jira/browse/YARN-8374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039608#comment-17039608 ] Hudson commented on YARN-8374: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17963 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17963/]) YARN-8374. Upgrade objenesis to 2.6 (#1798) (github: rev d6d7f8d8c5b68d56f53515b2ad26b921a5eefabb) * (edit) LICENSE-binary * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/pom.xml * (edit) hadoop-project/pom.xml > Upgrade objenesis to 2.6 > > > Key: YARN-8374 > URL: https://issues.apache.org/jira/browse/YARN-8374 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineservice >Reporter: Jason Darrell Lowe >Assignee: Akira Ajisaka >Priority: Major > Fix For: 3.3.0 > > > After HADOOP-14918 is committed we should be able to remove the explicit > objenesis dependency and objenesis exclusion from the fst dependency to pick > up the version fst wants naturally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10151) Disable Capacity Scheduler's move app between queue functionality
[
https://issues.apache.org/jira/browse/YARN-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039495#comment-17039495
]
Wangda Tan commented on YARN-10151:
---
And this should apply to all branches.
> Disable Capacity Scheduler's move app between queue functionality
> -
>
> Key: YARN-10151
> URL: https://issues.apache.org/jira/browse/YARN-10151
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Wangda Tan
>Priority: Critical
>
> Saw this happened in many clusters: Capacity Scheduler cannot work correctly
> with the move app between queue features. It will cause weird JMX issue,
> resource accounting issue, etc. In a lot of causes it will cause RM
> completely hung and available resource became negative, nothing can be
> allocated after that. We should turn off CapacityScheduler's move app between
> queue feature. (see:
> {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler#moveApplication}}
> )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10151) Disable Capacity Scheduler's move app between queue functionality
Wangda Tan created YARN-10151:
-
Summary: Disable Capacity Scheduler's move app between queue
functionality
Key: YARN-10151
URL: https://issues.apache.org/jira/browse/YARN-10151
Project: Hadoop YARN
Issue Type: Bug
Reporter: Wangda Tan
Saw this happened in many clusters: Capacity Scheduler cannot work correctly
with the move app between queue features. It will cause weird JMX issue,
resource accounting issue, etc. In a lot of causes it will cause RM completely
hung and available resource became negative, nothing can be allocated after
that. We should turn off CapacityScheduler's move app between queue feature.
(see:
{{org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler#moveApplication}}
)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10150) Incorrect link at http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ReservationSystem.html
Srinivasa Meka created YARN-10150:
-
Summary: Incorrect link at
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ReservationSystem.html
Key: YARN-10150
URL: https://issues.apache.org/jira/browse/YARN-10150
Project: Hadoop YARN
Issue Type: Task
Components: documentation
Affects Versions: 3.2.1
Reporter: Srinivasa Meka
Go to
[http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ReservationSystem.html]
Click on the top of the page displayed navigation link
("[Apache|http://www.apache.org/] > [Hadoop|http://hadoop.apache.org/] >
[Apache Hadoop
YARN|http://hadoop.apache.org/docs/current/hadoop-yarn/index.html] > [Apache
Hadoop
3.2.1|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/index.html]
> Reservation System") and click on one level up ("Apache Hadoop 3.2.1). We
get this error:
h1. Not Found
The requested URL was not found on this server.
Also, there is no navigational link in the left frame (under "Yarn" section)
and user is not sure how this content fits into the overall architecture and
flow with the content.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[
https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039360#comment-17039360
]
Hadoop QA commented on YARN-10148:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 22s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 14 new + 13 unchanged - 0 fixed = 27 total (was 13)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 10s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 88m
23s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10148 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993791/YARN-10148.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f8fdea2dd208 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / a562942 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/25534/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25534/testReport/ |
| Max. process+thread count | 820 (vs. ulimit of 5500) |
| modules | C:
[jira] [Commented] (YARN-10130) Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039356#comment-17039356
]
Hadoop QA commented on YARN-10130:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 27s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 8s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m
18s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}150m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10130 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993790/YARN-10130.003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 3fbbdde657db 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / a562942 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_232 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25535/testReport/ |
| Max. process+thread count | 850 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25535/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Do not allow output dir to be the same as input
[jira] [Commented] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039270#comment-17039270
]
Hadoop QA commented on YARN-10143:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 11s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
40s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10143 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993794/YARN-10143.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 98d1a2aa2a91 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / a562942 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25536/testReport/ |
| Max. process+thread count | 310 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25536/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> YARN-10101
[jira] [Commented] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039223#comment-17039223
]
Adam Antal commented on YARN-10147:
---
Thanks for the patch [~pbacsko]!
Minor nits:
- I'd raise this to warn or error level, since the user configured the FPGA
plugin, but it misses the IP file:
{code:java}
LOG.info("Requested IP file not found");
{code}
- Two ; characters in test imports:
{code:java}
import static org.junit.Assert.assertNull;;
{code}
(wondering why checkstyle didn't catch it)
- Could you please add meaningful messages in the assert statements in
{{TestIntelFpgaOpenclPlugin}}? "Path" is not really describes the failure.
> FPGA plugin can't find the localized aocx file
> --
>
> Key: YARN-10147
> URL: https://issues.apache.org/jira/browse/YARN-10147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10147-001.patch, YARN-10147-002.patch
>
>
> There's a bug in the FPGA plugin which is intended to find the localized
> "aocx" file:
> {noformat}
> ...
> if (localizedResources != null) {
> Optional aocxPath = localizedResources
> .keySet()
> .stream()
> .filter(path -> matchesIpid(path, id))
> .findFirst();
> if (aocxPath.isPresent()) {
> ipFilePath = aocxPath.get().toUri().toString();
> LOG.debug("Found: " + ipFilePath);
> }
> } else {
> LOG.warn("Localized resource is null!");
> }
> return ipFilePath;
> }
> private boolean matchesIpid(Path p, String id) {
> return p.getName().toLowerCase().equals(id.toLowerCase())
> && p.getName().endsWith(".aocx");
> }
> {noformat}
> The method {{matchesIpid()}} works incorrecty: the {{id}} argument is the
> expected filename, but without the extension. Therefore the {{equals()}}
> comparison will always be false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Antal updated YARN-10143:
--
Attachment: YARN-10143.002.patch
> YARN-10101 broke Yarn logs CLI
> ---
>
> Key: YARN-10143
> URL: https://issues.apache.org/jira/browse/YARN-10143
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0, 3.2.2
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Blocker
> Attachments: YARN-10143.001.patch, YARN-10143.002.patch
>
>
> YARN-10101 broke the Yarn logs CLI.
> In {{LogsCLI#359}} a {{ContainerLogsRequest}} has been created with null set
> as appAttemptId, while the {{LogAggregationFileController}} instances are
> configured badly to handle this case. The new JHS API works as expected for
> defined application attempt, but we should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039216#comment-17039216
]
Adam Antal commented on YARN-10143:
---
Tested patch v2 against a cluster setting to use IFile log aggregation
controller. Both UIv2 and Logs CLI works now. Will test it against TFile as
well tomorrow.
> YARN-10101 broke Yarn logs CLI
> ---
>
> Key: YARN-10143
> URL: https://issues.apache.org/jira/browse/YARN-10143
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0, 3.2.2
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Blocker
> Attachments: YARN-10143.001.patch, YARN-10143.002.patch
>
>
> YARN-10101 broke the Yarn logs CLI.
> In {{LogsCLI#359}} a {{ContainerLogsRequest}} has been created with null set
> as appAttemptId, while the {{LogAggregationFileController}} instances are
> configured badly to handle this case. The new JHS API works as expected for
> defined application attempt, but we should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[ https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton updated YARN-10148: Attachment: YARN-10148.001.patch > Add Unit test for queue ACL for both FS and CS > -- > > Key: YARN-10148 > URL: https://issues.apache.org/jira/browse/YARN-10148 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Attachments: YARN-10148.001.patch > > > Add some unit tests covering the queue ACL evaluation for both FS and CS. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10130) Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039200#comment-17039200
]
Adam Antal commented on YARN-10130:
---
Fixed one checkstyle warning.
> Do not allow output dir to be the same as input dir
> ---
>
> Key: YARN-10130
> URL: https://issues.apache.org/jira/browse/YARN-10130
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Adam Antal
>Priority: Major
> Attachments: YARN-10130.001.patch, YARN-10130.002.patch,
> YARN-10130.003.patch
>
>
> If the input dir where fair-scheduler.xml / yarn-site.xml sits is the same as
> the output dir (defined by the -o switch), the fs2cs tool overwrites the
> source config files, i.e. yarn-site.xml.
> Reproduce this is easy, just run fs2cs tool with this command:
> {code:java}
> /bin/yarn fs2cs --cluster-resource memory-mb=18044928,vcores=16
> --no-terminal-rule-check -y yarn-site.xml -f fair-scheduler.xml -o .
> {code}
> The following (or similar) is emitted by the tool:
> {code:java}
> WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.20/02/10 12:51:42 INFO converter.FSConfigToCSConfigConverter:
> Output directory for yarn-site.xml and capacity-scheduler.xml is: .20/02/10
> 12:51:42 INFO converter.FSConfigToCSConfigConverter: Conversion rules file is
> not defined, using default conversion config!20/02/10 12:51:42 ERROR
> conf.Configuration: error parsing conf
> yarn-site.xmlcom.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog at
> [row,col,system-id]: [1,0,"yarn-site.xml"] at
> com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:687) at
> com.ctc.wstx.sr.BasicStreamReader.handleEOF(BasicStreamReader.java:2220) at
> com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2126)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181) at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3343)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3137) at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3030) at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2996)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2871) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1389) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1361) at
> org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1702) at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.createConfiguration(FSConfigToCSConfigConverter.java:166)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:98)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:137)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:40)20/02/10
> 12:51:42 ERROR converter.FSConfigToCSConfigConverterMain: Error while
> starting FS configuration conversion!
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10130) Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Antal updated YARN-10130:
--
Attachment: YARN-10130.003.patch
> Do not allow output dir to be the same as input dir
> ---
>
> Key: YARN-10130
> URL: https://issues.apache.org/jira/browse/YARN-10130
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Adam Antal
>Priority: Major
> Attachments: YARN-10130.001.patch, YARN-10130.002.patch,
> YARN-10130.003.patch
>
>
> If the input dir where fair-scheduler.xml / yarn-site.xml sits is the same as
> the output dir (defined by the -o switch), the fs2cs tool overwrites the
> source config files, i.e. yarn-site.xml.
> Reproduce this is easy, just run fs2cs tool with this command:
> {code:java}
> /bin/yarn fs2cs --cluster-resource memory-mb=18044928,vcores=16
> --no-terminal-rule-check -y yarn-site.xml -f fair-scheduler.xml -o .
> {code}
> The following (or similar) is emitted by the tool:
> {code:java}
> WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.20/02/10 12:51:42 INFO converter.FSConfigToCSConfigConverter:
> Output directory for yarn-site.xml and capacity-scheduler.xml is: .20/02/10
> 12:51:42 INFO converter.FSConfigToCSConfigConverter: Conversion rules file is
> not defined, using default conversion config!20/02/10 12:51:42 ERROR
> conf.Configuration: error parsing conf
> yarn-site.xmlcom.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog at
> [row,col,system-id]: [1,0,"yarn-site.xml"] at
> com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:687) at
> com.ctc.wstx.sr.BasicStreamReader.handleEOF(BasicStreamReader.java:2220) at
> com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2126)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181) at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3343)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3137) at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3030) at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2996)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2871) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1389) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1361) at
> org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1702) at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.createConfiguration(FSConfigToCSConfigConverter.java:166)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:98)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:137)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:40)20/02/10
> 12:51:42 ERROR converter.FSConfigToCSConfigConverterMain: Error while
> starting FS configuration conversion!
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039194#comment-17039194
]
Peter Bacsko commented on YARN-10147:
-
[~snemeth] please review & commit this JIRA.
> FPGA plugin can't find the localized aocx file
> --
>
> Key: YARN-10147
> URL: https://issues.apache.org/jira/browse/YARN-10147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10147-001.patch, YARN-10147-002.patch
>
>
> There's a bug in the FPGA plugin which is intended to find the localized
> "aocx" file:
> {noformat}
> ...
> if (localizedResources != null) {
> Optional aocxPath = localizedResources
> .keySet()
> .stream()
> .filter(path -> matchesIpid(path, id))
> .findFirst();
> if (aocxPath.isPresent()) {
> ipFilePath = aocxPath.get().toUri().toString();
> LOG.debug("Found: " + ipFilePath);
> }
> } else {
> LOG.warn("Localized resource is null!");
> }
> return ipFilePath;
> }
> private boolean matchesIpid(Path p, String id) {
> return p.getName().toLowerCase().equals(id.toLowerCase())
> && p.getName().endsWith(".aocx");
> }
> {noformat}
> The method {{matchesIpid()}} works incorrecty: the {{id}} argument is the
> expected filename, but without the extension. Therefore the {{equals()}}
> comparison will always be false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10149) container-executor exits with 139 when the permissions of yarn log directory is improper
[
https://issues.apache.org/jira/browse/YARN-10149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tarun Parimi updated YARN-10149:
Description:
container-executor fails with segmentation fault and exit code 139 when the
permission of the yarn log directory is not proper.
While running the container-executor manually, we get the below message.
{code:java}
Error checking file stats for /hadoop/yarn/log -1 Permission denied.
{code}
But the exit code is 139 which corresponds to a segmentation fault. This is
misleading especially since the "Permission denied" is not getting printed in
the applogs or the NM logs. Only the exit code 139 message is present.
was:
container-executor fails with segmentation fault and exit code 139 when the
permission of the yarn log directory is not proper.
While running the container-executor manually, we get the below message.
{code:java}
Error checking file stats for /hadoop/yarn/log Permission denied -1
{code}
But the exit code is 139 which corresponds to a segmentation fault. This is
misleading especially since the "Permission denied" is not getting printed in
the applogs or the NM logs. Only the exit code 139 message is present.
> container-executor exits with 139 when the permissions of yarn log directory
> is improper
>
>
> Key: YARN-10149
> URL: https://issues.apache.org/jira/browse/YARN-10149
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.1.0
>Reporter: Tarun Parimi
>Assignee: Tarun Parimi
>Priority: Major
>
> container-executor fails with segmentation fault and exit code 139 when the
> permission of the yarn log directory is not proper.
> While running the container-executor manually, we get the below message.
> {code:java}
> Error checking file stats for /hadoop/yarn/log -1 Permission denied.
> {code}
> But the exit code is 139 which corresponds to a segmentation fault. This is
> misleading especially since the "Permission denied" is not getting printed in
> the applogs or the NM logs. Only the exit code 139 message is present.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10149) container-executor exits with 139 when the permissions of yarn log directory is improper
[
https://issues.apache.org/jira/browse/YARN-10149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tarun Parimi updated YARN-10149:
Description:
container-executor fails with segmentation fault and exit code 139 when the
permission of the yarn log directory is not proper.
While running the container-executor manually, we get the below message.
{code:java}
Error checking file stats for /hadoop/yarn/log Permission denied -1
{code}
But the exit code is 139 which corresponds to a segmentation fault. This is
misleading especially since the "Permission denied" is not getting printed in
the applogs or the NM logs. Only the exit code 139 message is present.
was:
container-executor fails with segmentation fault and exit code 139 when the
permission of the yarn log directory was not proper.
While running the container-executor manually, we get the below message.
{code:java}
Error checking file stats for /hadoop/yarn/log Permission denied -1
{code}
But the exit code is 139 which corresponds to a segmentation fault. This is
misleading especially since the "Permission denied" is not getting printed in
the applogs or the NM logs.
> container-executor exits with 139 when the permissions of yarn log directory
> is improper
>
>
> Key: YARN-10149
> URL: https://issues.apache.org/jira/browse/YARN-10149
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.1.0
>Reporter: Tarun Parimi
>Assignee: Tarun Parimi
>Priority: Major
>
> container-executor fails with segmentation fault and exit code 139 when the
> permission of the yarn log directory is not proper.
> While running the container-executor manually, we get the below message.
> {code:java}
> Error checking file stats for /hadoop/yarn/log Permission denied -1
> {code}
> But the exit code is 139 which corresponds to a segmentation fault. This is
> misleading especially since the "Permission denied" is not getting printed in
> the applogs or the NM logs. Only the exit code 139 message is present.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10149) container-executor exits with 139 when the permissions of yarn log directory is improper
Tarun Parimi created YARN-10149:
---
Summary: container-executor exits with 139 when the permissions of
yarn log directory is improper
Key: YARN-10149
URL: https://issues.apache.org/jira/browse/YARN-10149
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 3.1.0
Reporter: Tarun Parimi
Assignee: Tarun Parimi
container-executor fails with segmentation fault and exit code 139 when the
permission of the yarn log directory was not proper.
While running the container-executor manually, we get the below message.
{code:java}
Error checking file stats for /hadoop/yarn/log Permission denied -1
{code}
But the exit code is 139 which corresponds to a segmentation fault. This is
misleading especially since the "Permission denied" is not getting printed in
the applogs or the NM logs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039159#comment-17039159
]
Hadoop QA commented on YARN-10147:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 26s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 37s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
20s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 82m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10147 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993765/YARN-10147-002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux e677b5692d2d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / a562942 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25533/testReport/ |
| Max. process+thread count | 307 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25533/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> FPGA plugin can't find the localized aocx file
>
[jira] [Updated] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YARN-10147:
Attachment: YARN-10147-002.patch
> FPGA plugin can't find the localized aocx file
> --
>
> Key: YARN-10147
> URL: https://issues.apache.org/jira/browse/YARN-10147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10147-001.patch, YARN-10147-002.patch
>
>
> There's a bug in the FPGA plugin which is intended to find the localized
> "aocx" file:
> {noformat}
> ...
> if (localizedResources != null) {
> Optional aocxPath = localizedResources
> .keySet()
> .stream()
> .filter(path -> matchesIpid(path, id))
> .findFirst();
> if (aocxPath.isPresent()) {
> ipFilePath = aocxPath.get().toUri().toString();
> LOG.debug("Found: " + ipFilePath);
> }
> } else {
> LOG.warn("Localized resource is null!");
> }
> return ipFilePath;
> }
> private boolean matchesIpid(Path p, String id) {
> return p.getName().toLowerCase().equals(id.toLowerCase())
> && p.getName().endsWith(".aocx");
> }
> {noformat}
> The method {{matchesIpid()}} works incorrecty: the {{id}} argument is the
> expected filename, but without the extension. Therefore the {{equals()}}
> comparison will always be false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039068#comment-17039068
]
Hadoop QA commented on YARN-10147:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
37s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 45s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 19s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 3 new + 2 unchanged - 0 fixed = 5 total (was 2) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 3s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
25s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10147 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993754/YARN-10147-001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux dd04dcc9d129 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / a562942 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/25532/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25532/testReport/ |
| Max. process+thread count | 307 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
[jira] [Created] (YARN-10148) Add Unit test for queue ACL for both FS and CS
Kinga Marton created YARN-10148: --- Summary: Add Unit test for queue ACL for both FS and CS Key: YARN-10148 URL: https://issues.apache.org/jira/browse/YARN-10148 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Reporter: Kinga Marton Assignee: Kinga Marton Add some unit tests covering the queue ACL evaluation for both FS and CS. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10117) FS-CS converter: adjust queue ACL to have the same output with CS as for FS has
[ https://issues.apache.org/jira/browse/YARN-10117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton resolved YARN-10117. - Resolution: Not A Problem > FS-CS converter: adjust queue ACL to have the same output with CS as for FS > has > --- > > Key: YARN-10117 > URL: https://issues.apache.org/jira/browse/YARN-10117 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Kinga Marton >Priority: Major > > Both FS and CS seems to check the ACL recursively: from the leaf via the > parent(s) to the root (inclusive). However there are some differences in > evaluating them, what cause to have different results with the two schedulers. > Some examples are the following ones: > ||Tested scenario||FS output||CS output|| > |Root - “ ” > C - * > C1 - *|Denied by root ACL|OK| > |Root - “ ” > C - “ ” > C1 - “ ”|Denied by root ACL|OK| > |Root - “ ” > C - * > C1 - “ ”|Denied by root ACL|OK| > |Root - “ ” > C - “ ” > C1 - *|Denied by root ACL|OK| > Note: I have set the same value for both submit application and administer > queue ACLs -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10117) FS-CS converter: adjust queue ACL to have the same output with CS as for FS has
[ https://issues.apache.org/jira/browse/YARN-10117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039028#comment-17039028 ] Kinga Marton commented on YARN-10117: - There is no need for adjustments, because I passed only the queue name instead of the full queue path, so the apps were placed in the wrong queue(root.c1 created dinamically instead of root.c.c1). Passing the full queue path will generate the same results. I will open a new Jira for creating some unit tests for the manually tested scenarios. > FS-CS converter: adjust queue ACL to have the same output with CS as for FS > has > --- > > Key: YARN-10117 > URL: https://issues.apache.org/jira/browse/YARN-10117 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Kinga Marton >Priority: Major > > Both FS and CS seems to check the ACL recursively: from the leaf via the > parent(s) to the root (inclusive). However there are some differences in > evaluating them, what cause to have different results with the two schedulers. > Some examples are the following ones: > ||Tested scenario||FS output||CS output|| > |Root - “ ” > C - * > C1 - *|Denied by root ACL|OK| > |Root - “ ” > C - “ ” > C1 - “ ”|Denied by root ACL|OK| > |Root - “ ” > C - * > C1 - “ ”|Denied by root ACL|OK| > |Root - “ ” > C - “ ” > C1 - *|Denied by root ACL|OK| > Note: I have set the same value for both submit application and administer > queue ACLs -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YARN-10147:
Attachment: YARN-10147-001.patch
> FPGA plugin can't find the localized aocx file
> --
>
> Key: YARN-10147
> URL: https://issues.apache.org/jira/browse/YARN-10147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10147-001.patch
>
>
> There's a bug in the FPGA plugin which is intended to find the localized
> "aocx" file:
> {noformat}
> ...
> if (localizedResources != null) {
> Optional aocxPath = localizedResources
> .keySet()
> .stream()
> .filter(path -> matchesIpid(path, id))
> .findFirst();
> if (aocxPath.isPresent()) {
> ipFilePath = aocxPath.get().toUri().toString();
> LOG.debug("Found: " + ipFilePath);
> }
> } else {
> LOG.warn("Localized resource is null!");
> }
> return ipFilePath;
> }
> private boolean matchesIpid(Path p, String id) {
> return p.getName().toLowerCase().equals(id.toLowerCase())
> && p.getName().endsWith(".aocx");
> }
> {noformat}
> The method {{matchesIpid()}} works incorrecty: the {{id}} argument is the
> expected filename, but without the extension. Therefore the {{equals()}}
> comparison will always be false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9593) Updating scheduler conf with comma in config value fails
[
https://issues.apache.org/jira/browse/YARN-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17038996#comment-17038996
]
Tanu Ajmera commented on YARN-9593:
---
[~sunil.gov...@gmail.com] can you please assign this to me? I want to work on
this.
cc : [~prabhujoseph]
> Updating scheduler conf with comma in config value fails
>
>
> Key: YARN-9593
> URL: https://issues.apache.org/jira/browse/YARN-9593
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 3.0.0, 3.2.0, 3.1.2
>Reporter: Anthony Hsu
>Priority: Major
>
> For example:
> {code:java}
> $ yarn schedulerconf -update "root.gridops:acl_administer_queue=user1,user2
> group1,group2"
> Specify configuration key value as confKey=confVal.{code}
> This fails because there is a comma in the config value and the SchedConfCLI
> splits on comma first, expecting each split to a k=v pair.
> {noformat}
> void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
> if (args == null) {
> return;
> }
> HashMap globalUpdates = new HashMap<>();
> for (String globalUpdate : args.split(",")) {
> putKeyValuePair(globalUpdates, globalUpdate);
> }
> updateInfo.setGlobalParams(globalUpdates);
> }{noformat}
> Cc: [~jhung]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17038992#comment-17038992
]
Hadoop QA commented on YARN-10143:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
40s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 11s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 6s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
40s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10143 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993744/YARN-10143.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux b85933832a43 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / a562942 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25531/testReport/ |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25531/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> YARN-10101
[jira] [Created] (YARN-10147) FPGA plugin can't find the localized aocx file
Peter Bacsko created YARN-10147:
---
Summary: FPGA plugin can't find the localized aocx file
Key: YARN-10147
URL: https://issues.apache.org/jira/browse/YARN-10147
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Reporter: Peter Bacsko
Assignee: Peter Bacsko
There's a bug in the FPGA plugin which is intended to find the localized "aocx"
file:
{noformat}
...
if (localizedResources != null) {
Optional aocxPath = localizedResources
.keySet()
.stream()
.filter(path -> matchesIpid(path, id))
.findFirst();
if (aocxPath.isPresent()) {
ipFilePath = aocxPath.get().toUri().toString();
LOG.debug("Found: " + ipFilePath);
}
} else {
LOG.warn("Localized resource is null!");
}
return ipFilePath;
}
private boolean matchesIpid(Path p, String id) {
return p.getName().toLowerCase().equals(id.toLowerCase())
&& p.getName().endsWith(".aocx");
}
{noformat}
The method {{matchesIpid()}} works incorrecty: the {{id}} argument is the
expected filename, but without the extension. Therefore the {{equals()}}
comparison will always be false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Moved] (YARN-10146) [Federation] Add missing REST APIs for Router
[ https://issues.apache.org/jira/browse/YARN-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena moved HDFS-15179 to YARN-10146: Key: YARN-10146 (was: HDFS-15179) Project: Hadoop YARN (was: Hadoop HDFS) > [Federation] Add missing REST APIs for Router > - > > Key: YARN-10146 > URL: https://issues.apache.org/jira/browse/YARN-10146 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bilwa S T >Priority: Major > > In FederationInterceptorREST following API are not implemented: > |getClusterInfo| > |getSchedulerInfo| > |dumpSchedulerLogs| > |getActivities| > |getAppActivities| > |getAppStatistics| > |getAppState| > |getNodeToLabels| > |getLabelsToNodes| > |replaceLabelsOnNodes| > |replaceLabelsOnNode| > |getClusterNodeLabels| > |addToClusterNodeLabels| > |removeFromCluserNodeLabels| > |getLabelsOnNode| > |getAppPriority| > |updateApplicationPriority| > |getAppQueue| > |updateAppQueue| > |postDelegationToken| > |postDelegationTokenExpiration| > |cancelDelegationToken| > |createNewReservation| > |submitReservation| > |updateReservation| > |deleteReservation| > |listReservation| > |getAppTimeout| > |getAppTimeouts| > |updateApplicationTimeout| > |getAppAttempts| > |checkUserAccessToQueue| > |getAppAttempt| > |getContainers| > |getContainer| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10146) [Federation] Add missing REST APIs for Router
[ https://issues.apache.org/jira/browse/YARN-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilwa S T reassigned YARN-10146: Assignee: Bilwa S T > [Federation] Add missing REST APIs for Router > - > > Key: YARN-10146 > URL: https://issues.apache.org/jira/browse/YARN-10146 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > > In FederationInterceptorREST following API are not implemented: > |getClusterInfo| > |getSchedulerInfo| > |dumpSchedulerLogs| > |getActivities| > |getAppActivities| > |getAppStatistics| > |getAppState| > |getNodeToLabels| > |getLabelsToNodes| > |replaceLabelsOnNodes| > |replaceLabelsOnNode| > |getClusterNodeLabels| > |addToClusterNodeLabels| > |removeFromCluserNodeLabels| > |getLabelsOnNode| > |getAppPriority| > |updateApplicationPriority| > |getAppQueue| > |updateAppQueue| > |postDelegationToken| > |postDelegationTokenExpiration| > |cancelDelegationToken| > |createNewReservation| > |submitReservation| > |updateReservation| > |deleteReservation| > |listReservation| > |getAppTimeout| > |getAppTimeouts| > |updateApplicationTimeout| > |getAppAttempts| > |checkUserAccessToQueue| > |getAppAttempt| > |getContainers| > |getContainer| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17038948#comment-17038948
]
Prabhu Joseph commented on YARN-10145:
--
[~yehuanhuan] Thanks for reporting the issue. This is a duplicate of YARN-9924.
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Antal updated YARN-10143:
--
Attachment: YARN-10143.001.patch
> YARN-10101 broke Yarn logs CLI
> ---
>
> Key: YARN-10143
> URL: https://issues.apache.org/jira/browse/YARN-10143
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0, 3.2.2
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Blocker
> Attachments: YARN-10143.001.patch
>
>
> YARN-10101 broke the Yarn logs CLI.
> In {{LogsCLI#359}} a {{ContainerLogsRequest}} has been created with null set
> as appAttemptId, while the {{LogAggregationFileController}} instances are
> configured badly to handle this case. The new JHS API works as expected for
> defined application attempt, but we should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10145) Error in creating hbase tables
yehuanhuan created YARN-10145:
-
Summary: Error in creating hbase tables
Key: YARN-10145
URL: https://issues.apache.org/jira/browse/YARN-10145
Project: Hadoop YARN
Issue Type: Bug
Components: ATSv2
Affects Versions: 3.2.1
Reporter: yehuanhuan
When using the command(bin/hadoop
org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
-create) to create the timeline service schema，I get the following error.
{panel:title=ERROR}
2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in creating
hbase tables:
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
tried to access method shaded.com.google.common.base.Stopwatch.()V from
class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
at
org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
at
org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
at
org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
at
org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
at
org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
at
org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
at
org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
at
org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
at
org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
at
org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
at
org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
Caused by: java.lang.IllegalAccessError: tried to access method
shaded.com.google.common.base.Stopwatch.()V from class
org.apache.hadoop.hbase.zookeeper.MetaTableLocator
at
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
at
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
at
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
at
org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
at
org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
{panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
