[jira] [Updated] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[ https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10141: Attachment: YARN-10141.003.patch > Interceptor in FederationInterceptorREST doesnt update on RM switchover > --- > > Key: YARN-10141 > URL: https://issues.apache.org/jira/browse/YARN-10141 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, restapi >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10141.001.patch, YARN-10141.002.patch, > YARN-10141.003.patch > > > In Federation Setup, In the event of a RM switchover in a subcluster the > interceptor for that subcluster in FederationInterceptorREST doesnt get > updated. Due to this, Cluster Nodes REST API doesnt return the nodes from the > subcluster in which the RM switchover has occured. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10153) Problems of offline analysis with showSimulationTrace.html
[
https://issues.apache.org/jira/browse/YARN-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041443#comment-17041443
]
YingmingWang edited comment on YARN-10153 at 2/21/20 7:34 AM:
--
OK, I describe my question again. The web service you mentioned is real time
service, http:host:port/simulate. The offline analysis is that open the
showSimulationTrace.html, then, choose the realtimetrack.json to generate.
There are two ways. I know your meaning that I can see the many charts and the
data changing when my SLS is running. If I just see the real time website,
http:host:port/simulate, it is ok. After the running of SLS example, it
generate realtimetrack.json and jobruntime.csv to record the information of
this simulation. Then, I choose the realtimetrack.json in
showSimulationTrace.html but it cannot generate offinle analysis outcome. It
has no response and change. The problem is that why I just can see the real
time website but I cannot open the offline analysis. I am afraid that my SLS
configuration has problem, it leads to problems in offline analysis and even
simulation results. Another question is whether my INFO is ok even if it has '
resource-types.xml not found' ?
If I directy open the realtimetrack.json by website, it displays: SyntaxERROR:
JSON.parse: end of data when ',' or ']' was expected at line 134 column 1 of
the JSON data.
I place more files as rar in attachment.
Thank for your help!
was (Author: lmy777):
OK, I describe my question again. I know your meaning that I can see the many
charts and the data changing when my SLS is running. If I just see the real
time website, http:host:port/simulate, it is ok. After the running of SLS
example, it generate realtimetrack.json and jobruntime.csv to record the
information of this simulation. Then, I choose the realtimetrack.json in
showSimulationTrace.html but it cannot generate offinle analysis outcome. It
has no response and change. The problem is that why I just can see the real
time website but I cannot open the offline analysis. I am afraid that my SLS
configuration has problem, it leads to problems in offline analysis and even
simulation results. Another question is whether my INFO is ok even if it has '
resource-types.xml not found' ?
If I directy open the realtimetrack.json by website, it displays: SyntaxERROR:
JSON.parse: end of data when ',' or ']' was expected at line 134 column 1 of
the JSON data.
I place more files as rar in attachment.
Thank for your help!
> Problems of offline analysis with showSimulationTrace.html
> --
>
> Key: YARN-10153
> URL: https://issues.apache.org/jira/browse/YARN-10153
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.2.1
> Environment: Ubuntu18.04
> JAVA 1.8.0_241
> Hadoop-3.2.1
>Reporter: YingmingWang
>Priority: Major
> Fix For: 3.2.1
>
> Attachments: INFO.txt, sls_file.rar
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> I am new to learn YARN SLS. After the configuration of HADOOP and YARN SLS, I
> run the example in sample-data. It doesn't have WARNING or ERROR. I can see
> the chart and data changing in real time by host:port/simulate but I can not
> analyse the realtimetrack.json file by showSimulationTrace.html after the
> run. I modified my file with the improvement of YARN-9836 but it still can't
> analyze trace. The whole INFO file is placed in .txt.
> {code:java}
> // code placeholder
> # Begining
> INFO conf.Configuration: resource-types.xml not found
> INFO resource.ResourceUtils:Unable to find 'resource-types.xml'.
> # Finally
> 2020-02-20 14:59:40,838 INFO sls.SLSRunner: SLSRunner tears down.
> 2020-02-20 14:59:40,839 INFO resourcemanager.RMAppManager$ApplicationSummary:
> appId=application_1582181515746_0002,name=N/A,user=jenkins,queue=sls_queue_1,state=FINISHED,trackingUrl=http://host2-virtual-machine:18088/proxy/application_1582181515746_0002/,appMasterHost=localhost,submitTime=1582181622389,startTime=1582181622389,launchTime=-1,finishTime=1582181980390,finalStatus=SUCCEEDED,memorySeconds=2433739,vcoreSeconds=2313,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources= vCores:0>,applicationType=YARN,resourceSeconds=2433739 MB-seconds\, 2313
> vcore-seconds,preemptedResourceSeconds=0 MB-seconds\, 0
> vcore-seconds,applicationTags=
> {code}
> I want to know how to solve it or a more correct configuration process. Thank
> you very much.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional
[jira] [Comment Edited] (YARN-10153) Problems of offline analysis with showSimulationTrace.html
[
https://issues.apache.org/jira/browse/YARN-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041443#comment-17041443
]
YingmingWang edited comment on YARN-10153 at 2/21/20 7:29 AM:
--
OK, I describe my question again. I know your meaning that I can see the many
charts and the data changing when my SLS is running. If I just see the real
time website, http:host:port/simulate, it is ok. After the running of SLS
example, it generate realtimetrack.json and jobruntime.csv to record the
information of this simulation. Then, I choose the realtimetrack.json in
showSimulationTrace.html but it cannot generate offinle analysis outcome. It
has no response and change. The problem is that why I just can see the real
time website but I cannot open the offline analysis. I am afraid that my SLS
configuration has problem, it leads to problems in offline analysis and even
simulation results. Another question is whether my INFO is ok even if it has '
resource-types.xml not found' ?
If I directy open the realtimetrack.json by website, it displays: SyntaxERROR:
JSON.parse: end of data when ',' or ']' was expected at line 134 column 1 of
the JSON data.
I place more files as rar in attachment.
Thank for your help!
was (Author: lmy777):
OK, I describe my question again. I know your meaning that I can see the many
charts and the data changing when my SLS is running. If I just see the real
time websitte, http:host:port/simulate, it is ok. After the running of SLS
example, it generate realtimetrack.json and jobruntime.csv to record the
information of this simulation.Then, I choose the realtimetrack.json in
showSimulationTrace.html but it cannot generate offinle analysis outcome. It
has no response and change. The problem is that why I just see the real time
website but I cannot see offline analysis. I am afraid that my SLS
configuration has problem, it leads to problems in offline analysis and even
simulation results. Another question is whether my INFO is ok even if it has '
resource-types.xml not found' ?
If I directy open the realtimetrack.json by website, it displays: SyntaxERROR:
JSON.parse: end of data when ',' or ']' was expected at line 134 column 1 of
the JSON data.
I place more files as rar in attachment.
Thank for your help!
> Problems of offline analysis with showSimulationTrace.html
> --
>
> Key: YARN-10153
> URL: https://issues.apache.org/jira/browse/YARN-10153
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.2.1
> Environment: Ubuntu18.04
> JAVA 1.8.0_241
> Hadoop-3.2.1
>Reporter: YingmingWang
>Priority: Major
> Fix For: 3.2.1
>
> Attachments: INFO.txt, sls_file.rar
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> I am new to learn YARN SLS. After the configuration of HADOOP and YARN SLS, I
> run the example in sample-data. It doesn't have WARNING or ERROR. I can see
> the chart and data changing in real time by host:port/simulate but I can not
> analyse the realtimetrack.json file by showSimulationTrace.html after the
> run. I modified my file with the improvement of YARN-9836 but it still can't
> analyze trace. The whole INFO file is placed in .txt.
> {code:java}
> // code placeholder
> # Begining
> INFO conf.Configuration: resource-types.xml not found
> INFO resource.ResourceUtils:Unable to find 'resource-types.xml'.
> # Finally
> 2020-02-20 14:59:40,838 INFO sls.SLSRunner: SLSRunner tears down.
> 2020-02-20 14:59:40,839 INFO resourcemanager.RMAppManager$ApplicationSummary:
> appId=application_1582181515746_0002,name=N/A,user=jenkins,queue=sls_queue_1,state=FINISHED,trackingUrl=http://host2-virtual-machine:18088/proxy/application_1582181515746_0002/,appMasterHost=localhost,submitTime=1582181622389,startTime=1582181622389,launchTime=-1,finishTime=1582181980390,finalStatus=SUCCEEDED,memorySeconds=2433739,vcoreSeconds=2313,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources= vCores:0>,applicationType=YARN,resourceSeconds=2433739 MB-seconds\, 2313
> vcore-seconds,preemptedResourceSeconds=0 MB-seconds\, 0
> vcore-seconds,applicationTags=
> {code}
> I want to know how to solve it or a more correct configuration process. Thank
> you very much.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041625#comment-17041625
]
Szilard Nemeth commented on YARN-10147:
---
As discussed with [~pbacsko], I'm resolving this jira as the backport to 3.2 is
not required.
> FPGA plugin can't find the localized aocx file
> --
>
> Key: YARN-10147
> URL: https://issues.apache.org/jira/browse/YARN-10147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-10147-001.patch, YARN-10147-002.patch,
> YARN-10147-003.patch
>
>
> There's a bug in the FPGA plugin which is intended to find the localized
> "aocx" file:
> {noformat}
> ...
> if (localizedResources != null) {
> Optional aocxPath = localizedResources
> .keySet()
> .stream()
> .filter(path -> matchesIpid(path, id))
> .findFirst();
> if (aocxPath.isPresent()) {
> ipFilePath = aocxPath.get().toUri().toString();
> LOG.debug("Found: " + ipFilePath);
> }
> } else {
> LOG.warn("Localized resource is null!");
> }
> return ipFilePath;
> }
> private boolean matchesIpid(Path p, String id) {
> return p.getName().toLowerCase().equals(id.toLowerCase())
> && p.getName().endsWith(".aocx");
> }
> {noformat}
> The method {{matchesIpid()}} works incorrecty: the {{id}} argument is the
> expected filename, but without the extension. Therefore the {{equals()}}
> comparison will always be false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10152) Fix findbugs warnings in hadoop-yarn-applications-mawo-core module
[
https://issues.apache.org/jira/browse/YARN-10152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041563#comment-17041563
]
Eric Yang commented on YARN-10152:
--
+1
> Fix findbugs warnings in hadoop-yarn-applications-mawo-core module
> --
>
> Key: YARN-10152
> URL: https://issues.apache.org/jira/browse/YARN-10152
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>
> {noformat}
> FindBugs :
>
> module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core
> Class org.apache.hadoop.applications.mawo.server.common.TaskStatus
> implements Cloneable but does not define or use clone method At
> TaskStatus.java:does not define or use clone method At TaskStatus.java:[lines
> 39-346]
> Equals method for
> org.apache.hadoop.applications.mawo.server.worker.WorkerId assumes the
> argument is of type WorkerId At WorkerId.java:the argument is of type
> WorkerId At WorkerId.java:[line 114]
>
> org.apache.hadoop.applications.mawo.server.worker.WorkerId.equals(Object)
> does not check for null argument At WorkerId.java:null argument At
> WorkerId.java:[lines 114-115] {noformat}
> Detail:
> [https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1414/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-mawo_hadoop-yarn-applications-mawo-core-warnings.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10152) Fix findbugs warnings in hadoop-yarn-applications-mawo-core module
[
https://issues.apache.org/jira/browse/YARN-10152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041504#comment-17041504
]
Akira Ajisaka commented on YARN-10152:
--
Hi [~eyang], would you review this pull request?
> Fix findbugs warnings in hadoop-yarn-applications-mawo-core module
> --
>
> Key: YARN-10152
> URL: https://issues.apache.org/jira/browse/YARN-10152
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>
> {noformat}
> FindBugs :
>
> module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core
> Class org.apache.hadoop.applications.mawo.server.common.TaskStatus
> implements Cloneable but does not define or use clone method At
> TaskStatus.java:does not define or use clone method At TaskStatus.java:[lines
> 39-346]
> Equals method for
> org.apache.hadoop.applications.mawo.server.worker.WorkerId assumes the
> argument is of type WorkerId At WorkerId.java:the argument is of type
> WorkerId At WorkerId.java:[line 114]
>
> org.apache.hadoop.applications.mawo.server.worker.WorkerId.equals(Object)
> does not check for null argument At WorkerId.java:null argument At
> WorkerId.java:[lines 114-115] {noformat}
> Detail:
> [https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1414/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-mawo_hadoop-yarn-applications-mawo-core-warnings.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10153) Problems of offline analysis with showSimulationTrace.html
[
https://issues.apache.org/jira/browse/YARN-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041443#comment-17041443
]
YingmingWang edited comment on YARN-10153 at 2/21/20 2:03 AM:
--
OK, I describe my question again. I know your meaning that I can see the many
charts and the data changing when my SLS is running. If I just see the real
time websitte, http:host:port/simulate, it is ok. After the running of SLS
example, it generate realtimetrack.json and jobruntime.csv to record the
information of this simulation.Then, I choose the realtimetrack.json in
showSimulationTrace.html but it cannot generate offinle analysis outcome. It
has no response and change. The problem is that why I just see the real time
website but I cannot see offline analysis. I am afraid that my SLS
configuration has problem, it leads to problems in offline analysis and even
simulation results. Another question is whether my INFO is ok even if it has '
resource-types.xml not found' ?
If I directy open the realtimetrack.json by website, it displays: SyntaxERROR:
JSON.parse: end of data when ',' or ']' was expected at line 134 column 1 of
the JSON data.
I place more files as rar in attachment.
Thank for your help!
was (Author: lmy777):
OK, I describe my question again. I know your meaning that I can see the many
charts and the data changing when my SLS is running. If I just see the real
time websitte, http:host:port/simulate, it is ok. After the running of SLS
example, it generate realtimetrack.json and jobruntime.csv to record the
information of this simulation.Then, I choose the realtimetrack.json in
showSimulationTrace.html but it cannot generate offinle analysis outcome. It
has no response and change. The problem is that why I just see the real time
website but I cannot see offline analysis. I am afraid that my SLS
configuration has problem, it leads to problems in offline analysis and even
simulation results. Another question is whether my INFO is ok even if it has '
resource-types.xml not found' ?
If I directy open the realtimetrack.json by website, it displays: SyntaxERROR:
JSON.parse: end of data when ',' or ']' was expected at line 134 column 1 of
the JSON data.
I place more files as zip in attachment.
Thank for your help.
> Problems of offline analysis with showSimulationTrace.html
> --
>
> Key: YARN-10153
> URL: https://issues.apache.org/jira/browse/YARN-10153
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.2.1
> Environment: Ubuntu18.04
> JAVA 1.8.0_241
> Hadoop-3.2.1
>Reporter: YingmingWang
>Priority: Major
> Fix For: 3.2.1
>
> Attachments: INFO.txt, sls_file.rar
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> I am new to learn YARN SLS. After the configuration of HADOOP and YARN SLS, I
> run the example in sample-data. It doesn't have WARNING or ERROR. I can see
> the chart and data changing in real time by host:port/simulate but I can not
> analyse the realtimetrack.json file by showSimulationTrace.html after the
> run. I modified my file with the improvement of YARN-9836 but it still can't
> analyze trace. The whole INFO file is placed in .txt.
> {code:java}
> // code placeholder
> # Begining
> INFO conf.Configuration: resource-types.xml not found
> INFO resource.ResourceUtils:Unable to find 'resource-types.xml'.
> # Finally
> 2020-02-20 14:59:40,838 INFO sls.SLSRunner: SLSRunner tears down.
> 2020-02-20 14:59:40,839 INFO resourcemanager.RMAppManager$ApplicationSummary:
> appId=application_1582181515746_0002,name=N/A,user=jenkins,queue=sls_queue_1,state=FINISHED,trackingUrl=http://host2-virtual-machine:18088/proxy/application_1582181515746_0002/,appMasterHost=localhost,submitTime=1582181622389,startTime=1582181622389,launchTime=-1,finishTime=1582181980390,finalStatus=SUCCEEDED,memorySeconds=2433739,vcoreSeconds=2313,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources= vCores:0>,applicationType=YARN,resourceSeconds=2433739 MB-seconds\, 2313
> vcore-seconds,preemptedResourceSeconds=0 MB-seconds\, 0
> vcore-seconds,applicationTags=
> {code}
> I want to know how to solve it or a more correct configuration process. Thank
> you very much.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10153) Problems of offline analysis with showSimulationTrace.html
[
https://issues.apache.org/jira/browse/YARN-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YingmingWang updated YARN-10153:
Attachment: sls_file.rar
> Problems of offline analysis with showSimulationTrace.html
> --
>
> Key: YARN-10153
> URL: https://issues.apache.org/jira/browse/YARN-10153
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.2.1
> Environment: Ubuntu18.04
> JAVA 1.8.0_241
> Hadoop-3.2.1
>Reporter: YingmingWang
>Priority: Major
> Fix For: 3.2.1
>
> Attachments: INFO.txt, sls_file.rar
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> I am new to learn YARN SLS. After the configuration of HADOOP and YARN SLS, I
> run the example in sample-data. It doesn't have WARNING or ERROR. I can see
> the chart and data changing in real time by host:port/simulate but I can not
> analyse the realtimetrack.json file by showSimulationTrace.html after the
> run. I modified my file with the improvement of YARN-9836 but it still can't
> analyze trace. The whole INFO file is placed in .txt.
> {code:java}
> // code placeholder
> # Begining
> INFO conf.Configuration: resource-types.xml not found
> INFO resource.ResourceUtils:Unable to find 'resource-types.xml'.
> # Finally
> 2020-02-20 14:59:40,838 INFO sls.SLSRunner: SLSRunner tears down.
> 2020-02-20 14:59:40,839 INFO resourcemanager.RMAppManager$ApplicationSummary:
> appId=application_1582181515746_0002,name=N/A,user=jenkins,queue=sls_queue_1,state=FINISHED,trackingUrl=http://host2-virtual-machine:18088/proxy/application_1582181515746_0002/,appMasterHost=localhost,submitTime=1582181622389,startTime=1582181622389,launchTime=-1,finishTime=1582181980390,finalStatus=SUCCEEDED,memorySeconds=2433739,vcoreSeconds=2313,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources= vCores:0>,applicationType=YARN,resourceSeconds=2433739 MB-seconds\, 2313
> vcore-seconds,preemptedResourceSeconds=0 MB-seconds\, 0
> vcore-seconds,applicationTags=
> {code}
> I want to know how to solve it or a more correct configuration process. Thank
> you very much.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10153) Problems of offline analysis with showSimulationTrace.html
[
https://issues.apache.org/jira/browse/YARN-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041443#comment-17041443
]
YingmingWang commented on YARN-10153:
-
OK, I describe my question again. I know your meaning that I can see the many
charts and the data changing when my SLS is running. If I just see the real
time websitte, http:host:port/simulate, it is ok. After the running of SLS
example, it generate realtimetrack.json and jobruntime.csv to record the
information of this simulation.Then, I choose the realtimetrack.json in
showSimulationTrace.html but it cannot generate offinle analysis outcome. It
has no response and change. The problem is that why I just see the real time
website but I cannot see offline analysis. I am afraid that my SLS
configuration has problem, it leads to problems in offline analysis and even
simulation results. Another question is whether my INFO is ok even if it has '
resource-types.xml not found' ?
If I directy open the realtimetrack.json by website, it displays: SyntaxERROR:
JSON.parse: end of data when ',' or ']' was expected at line 134 column 1 of
the JSON data.
I place more files as zip in attachment.
Thank for your help.
> Problems of offline analysis with showSimulationTrace.html
> --
>
> Key: YARN-10153
> URL: https://issues.apache.org/jira/browse/YARN-10153
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.2.1
> Environment: Ubuntu18.04
> JAVA 1.8.0_241
> Hadoop-3.2.1
>Reporter: YingmingWang
>Priority: Major
> Fix For: 3.2.1
>
> Attachments: INFO.txt
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> I am new to learn YARN SLS. After the configuration of HADOOP and YARN SLS, I
> run the example in sample-data. It doesn't have WARNING or ERROR. I can see
> the chart and data changing in real time by host:port/simulate but I can not
> analyse the realtimetrack.json file by showSimulationTrace.html after the
> run. I modified my file with the improvement of YARN-9836 but it still can't
> analyze trace. The whole INFO file is placed in .txt.
> {code:java}
> // code placeholder
> # Begining
> INFO conf.Configuration: resource-types.xml not found
> INFO resource.ResourceUtils:Unable to find 'resource-types.xml'.
> # Finally
> 2020-02-20 14:59:40,838 INFO sls.SLSRunner: SLSRunner tears down.
> 2020-02-20 14:59:40,839 INFO resourcemanager.RMAppManager$ApplicationSummary:
> appId=application_1582181515746_0002,name=N/A,user=jenkins,queue=sls_queue_1,state=FINISHED,trackingUrl=http://host2-virtual-machine:18088/proxy/application_1582181515746_0002/,appMasterHost=localhost,submitTime=1582181622389,startTime=1582181622389,launchTime=-1,finishTime=1582181980390,finalStatus=SUCCEEDED,memorySeconds=2433739,vcoreSeconds=2313,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources= vCores:0>,applicationType=YARN,resourceSeconds=2433739 MB-seconds\, 2313
> vcore-seconds,preemptedResourceSeconds=0 MB-seconds\, 0
> vcore-seconds,applicationTags=
> {code}
> I want to know how to solve it or a more correct configuration process. Thank
> you very much.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10119) Cannot reset the AM failure count for YARN Service
[ https://issues.apache.org/jira/browse/YARN-10119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041424#comment-17041424 ] kyungwan nam commented on YARN-10119: - Thanks [~prabhujoseph] for your review and commit. > Cannot reset the AM failure count for YARN Service > -- > > Key: YARN-10119 > URL: https://issues.apache.org/jira/browse/YARN-10119 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: kyungwan nam >Assignee: kyungwan nam >Priority: Major > Labels: Reviewed > Fix For: 3.3.0 > > Attachments: YARN-10119.001.patch > > > Currently, YARN Service does not support to reset AM failure count, which > introduced in YARN-611 > Since the AM failure count is never reset, eventually that will reach > yarn.service.am-restart.max-attempts and the app will be stopped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS
[
https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041265#comment-17041265
]
Hadoop QA commented on YARN-9879:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 16 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 53s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
47s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 8s{color} | {color:orange} root: The patch generated 78 new + 2135
unchanged - 5 fixed = 2213 total (was 2140) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 46s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 88m 9s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 19s{color}
| {color:red} hadoop-sls in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
41s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}206m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue |
| |
hadoop.yarn.server.resourcemanager.reservation.TestCapacitySchedulerPlanFollower
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
|
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities |
| |
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicyIntraQueueUserLimit
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoQueueCreation
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueManagementDynamicEditPolicy
|
| |
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicyMockFramework
|
[jira] [Commented] (YARN-10130) FS-CS converter: Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041207#comment-17041207
]
Hadoop QA commented on YARN-10130:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 42s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 41s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 84m 47s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}148m 4s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10130 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994011/YARN-10130.005.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 8df6204093bc 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 181e6d0 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/25552/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
[jira] [Commented] (YARN-10135) FS-CS converter tool: issue warning on dynamic auto-create mapping rules
[
https://issues.apache.org/jira/browse/YARN-10135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041199#comment-17041199
]
Hadoop QA commented on YARN-10135:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
37s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 28s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 6s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 83m
49s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}141m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10135 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994010/YARN-10135-001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 4c62192c030b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 181e6d0 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/25551/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25551/testReport/ |
| Max. process+thread count | 904 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
[jira] [Commented] (YARN-10130) FS-CS converter: Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041180#comment-17041180
]
Hadoop QA commented on YARN-10130:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 22s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 43s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 84m 25s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}138m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.fair.converter.TestFSConfigToCSConfigArgumentHandler
|
| | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10130 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994009/YARN-10130.004.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 341ebca84d5a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 181e6d0 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/25550/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25550/testReport/ |
| Max. process+thread count | 885 (vs. ulimit of 5500) |
| modules | C:
[jira] [Commented] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[
https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041167#comment-17041167
]
Hadoop QA commented on YARN-10148:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 30s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 21s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 45s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}156m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10148 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994006/YARN-10148.004.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 7079866aeb68 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 4af2556 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/25549/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25549/testReport/ |
| Max. process+thread count | 819 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
[jira] [Created] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources
Sunil G created YARN-10154: -- Summary: CS Dynamic Queues cannot be configured with absolute resources Key: YARN-10154 URL: https://issues.apache.org/jira/browse/YARN-10154 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.1.3 Reporter: Sunil G Assignee: Sunil G In CS, ManagedParent Queue and its template cannot take absolute resource value like [memory=8192,vcores=8] Thsi Jira is to track and improve the configuration reading module of DynamicQueue to support absolute resource values. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9879) Allow multiple leaf queues with the same name in CS
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gergely Pollak updated YARN-9879: - Attachment: YARN-9879.POC005.patch > Allow multiple leaf queues with the same name in CS > --- > > Key: YARN-9879 > URL: https://issues.apache.org/jira/browse/YARN-9879 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Gergely Pollak >Priority: Major > Labels: fs2cs > Attachments: CSQueue.getQueueUsage.txt, DesignDoc_v1.pdf, > YARN-9879.POC001.patch, YARN-9879.POC002.patch, YARN-9879.POC003.patch, > YARN-9879.POC004.patch, YARN-9879.POC005.patch > > > Currently the leaf queue's name must be unique regardless of its position in > the queue hierarchy. > Design doc and first proposal is being made, I'll attach it as soon as it's > done. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10153) Problems of offline analysis with showSimulationTrace.html
[
https://issues.apache.org/jira/browse/YARN-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041110#comment-17041110
]
Adam Antal commented on YARN-10153:
---
Hi [~lmy777],
Could you please provide more information on the issue? I could not see clearly
what is the error here.
The SLS creates the local web server which serves the simulationTrace.html
website. So you can use that web page as long as the SLS is running.
> Problems of offline analysis with showSimulationTrace.html
> --
>
> Key: YARN-10153
> URL: https://issues.apache.org/jira/browse/YARN-10153
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.2.1
> Environment: Ubuntu18.04
> JAVA 1.8.0_241
> Hadoop-3.2.1
>Reporter: YingmingWang
>Priority: Major
> Fix For: 3.2.1
>
> Attachments: INFO.txt
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> I am new to learn YARN SLS. After the configuration of HADOOP and YARN SLS, I
> run the example in sample-data. It doesn't have WARNING or ERROR. I can see
> the chart and data changing in real time by host:port/simulate but I can not
> analyse the realtimetrack.json file by showSimulationTrace.html after the
> run. I modified my file with the improvement of YARN-9836 but it still can't
> analyze trace. The whole INFO file is placed in .txt.
> {code:java}
> // code placeholder
> # Begining
> INFO conf.Configuration: resource-types.xml not found
> INFO resource.ResourceUtils:Unable to find 'resource-types.xml'.
> # Finally
> 2020-02-20 14:59:40,838 INFO sls.SLSRunner: SLSRunner tears down.
> 2020-02-20 14:59:40,839 INFO resourcemanager.RMAppManager$ApplicationSummary:
> appId=application_1582181515746_0002,name=N/A,user=jenkins,queue=sls_queue_1,state=FINISHED,trackingUrl=http://host2-virtual-machine:18088/proxy/application_1582181515746_0002/,appMasterHost=localhost,submitTime=1582181622389,startTime=1582181622389,launchTime=-1,finishTime=1582181980390,finalStatus=SUCCEEDED,memorySeconds=2433739,vcoreSeconds=2313,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources= vCores:0>,applicationType=YARN,resourceSeconds=2433739 MB-seconds\, 2313
> vcore-seconds,preemptedResourceSeconds=0 MB-seconds\, 0
> vcore-seconds,applicationTags=
> {code}
> I want to know how to solve it or a more correct configuration process. Thank
> you very much.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041107#comment-17041107
]
Hadoop QA commented on YARN-10143:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 9m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-3.2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
42s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
43s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
30s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} branch-3.2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 21s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
30s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 74m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:0f25cbbb251 |
| JIRA Issue | YARN-10143 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994005/YARN-10143.branch-3.2.001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 97fd8d503385 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-3.2 / d411a84 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25548/testReport/ |
| Max. process+thread count | 345 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25548/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This
[jira] [Commented] (YARN-10130) FS-CS converter: Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041082#comment-17041082
]
Adam Antal commented on YARN-10130:
---
Oops, forgot the fix a TODO in the test class, so I uploaded patchset v5.
> FS-CS converter: Do not allow output dir to be the same as input dir
>
>
> Key: YARN-10130
> URL: https://issues.apache.org/jira/browse/YARN-10130
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Adam Antal
>Priority: Major
> Attachments: YARN-10130.001.patch, YARN-10130.002.patch,
> YARN-10130.003.patch, YARN-10130.004.patch, YARN-10130.005.patch
>
>
> If the input dir where fair-scheduler.xml / yarn-site.xml sits is the same as
> the output dir (defined by the -o switch), the fs2cs tool overwrites the
> source config files, i.e. yarn-site.xml.
> Reproduce this is easy, just run fs2cs tool with this command:
> {code:java}
> /bin/yarn fs2cs --cluster-resource memory-mb=18044928,vcores=16
> --no-terminal-rule-check -y yarn-site.xml -f fair-scheduler.xml -o .
> {code}
> The following (or similar) is emitted by the tool:
> {code:java}
> WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.20/02/10 12:51:42 INFO converter.FSConfigToCSConfigConverter:
> Output directory for yarn-site.xml and capacity-scheduler.xml is: .20/02/10
> 12:51:42 INFO converter.FSConfigToCSConfigConverter: Conversion rules file is
> not defined, using default conversion config!20/02/10 12:51:42 ERROR
> conf.Configuration: error parsing conf
> yarn-site.xmlcom.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog at
> [row,col,system-id]: [1,0,"yarn-site.xml"] at
> com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:687) at
> com.ctc.wstx.sr.BasicStreamReader.handleEOF(BasicStreamReader.java:2220) at
> com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2126)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181) at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3343)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3137) at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3030) at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2996)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2871) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1389) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1361) at
> org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1702) at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.createConfiguration(FSConfigToCSConfigConverter.java:166)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:98)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:137)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:40)20/02/10
> 12:51:42 ERROR converter.FSConfigToCSConfigConverterMain: Error while
> starting FS configuration conversion!
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10130) FS-CS converter: Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Antal updated YARN-10130:
--
Attachment: YARN-10130.005.patch
> FS-CS converter: Do not allow output dir to be the same as input dir
>
>
> Key: YARN-10130
> URL: https://issues.apache.org/jira/browse/YARN-10130
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Adam Antal
>Priority: Major
> Attachments: YARN-10130.001.patch, YARN-10130.002.patch,
> YARN-10130.003.patch, YARN-10130.004.patch, YARN-10130.005.patch
>
>
> If the input dir where fair-scheduler.xml / yarn-site.xml sits is the same as
> the output dir (defined by the -o switch), the fs2cs tool overwrites the
> source config files, i.e. yarn-site.xml.
> Reproduce this is easy, just run fs2cs tool with this command:
> {code:java}
> /bin/yarn fs2cs --cluster-resource memory-mb=18044928,vcores=16
> --no-terminal-rule-check -y yarn-site.xml -f fair-scheduler.xml -o .
> {code}
> The following (or similar) is emitted by the tool:
> {code:java}
> WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.20/02/10 12:51:42 INFO converter.FSConfigToCSConfigConverter:
> Output directory for yarn-site.xml and capacity-scheduler.xml is: .20/02/10
> 12:51:42 INFO converter.FSConfigToCSConfigConverter: Conversion rules file is
> not defined, using default conversion config!20/02/10 12:51:42 ERROR
> conf.Configuration: error parsing conf
> yarn-site.xmlcom.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog at
> [row,col,system-id]: [1,0,"yarn-site.xml"] at
> com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:687) at
> com.ctc.wstx.sr.BasicStreamReader.handleEOF(BasicStreamReader.java:2220) at
> com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2126)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181) at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3343)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3137) at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3030) at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2996)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2871) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1389) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1361) at
> org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1702) at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.createConfiguration(FSConfigToCSConfigConverter.java:166)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:98)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:137)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:40)20/02/10
> 12:51:42 ERROR converter.FSConfigToCSConfigConverterMain: Error while
> starting FS configuration conversion!
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10135) FS-CS converter tool: issue warning on dynamic auto-create mapping rules
[
https://issues.apache.org/jira/browse/YARN-10135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YARN-10135:
Attachment: YARN-10135-001.patch
> FS-CS converter tool: issue warning on dynamic auto-create mapping rules
>
>
> Key: YARN-10135
> URL: https://issues.apache.org/jira/browse/YARN-10135
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Labels: fs2cs
> Attachments: YARN-10135-001.patch
>
>
> The converter tool should issue a warning whenever the conversion results in
> mapping rules similar to these:
> {{u:%user:[managedParentQueueName].[queueName]}}
> {{u:%user:[managedParentQueueName].%user}}
> {{u:%user:[managedParentQueueName].%primary_group}}
> {{u:%user:[managedParentQueueName].%secondary_group}}
> {{u:%user:%primary_group.%user}}
> {{u:%user:%secondary_group.%user}}
> {{u:%user:[managedParentQueuePath].%user}}
>
> The reason is that right now it's fully clear how we'll handle a case like
> "u:%user:%primary_group.%user", where "%primary_group.%user" might result in
> something like "users.john".
> In case of "u:%user:[managedParentQueuePath].%user" , the
> [managedParentQueuePath] is a result of a full path from Fair Scheduler.
> Therefore it's not going to be a leaf queue.
> The user might be required to do some fine tuning and adjust the property
> "auto-create-child-queues". We should display a warning about these
> additional steps.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10130) FS-CS converter: Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Antal updated YARN-10130:
--
Attachment: YARN-10130.004.patch
> FS-CS converter: Do not allow output dir to be the same as input dir
>
>
> Key: YARN-10130
> URL: https://issues.apache.org/jira/browse/YARN-10130
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Adam Antal
>Priority: Major
> Attachments: YARN-10130.001.patch, YARN-10130.002.patch,
> YARN-10130.003.patch, YARN-10130.004.patch
>
>
> If the input dir where fair-scheduler.xml / yarn-site.xml sits is the same as
> the output dir (defined by the -o switch), the fs2cs tool overwrites the
> source config files, i.e. yarn-site.xml.
> Reproduce this is easy, just run fs2cs tool with this command:
> {code:java}
> /bin/yarn fs2cs --cluster-resource memory-mb=18044928,vcores=16
> --no-terminal-rule-check -y yarn-site.xml -f fair-scheduler.xml -o .
> {code}
> The following (or similar) is emitted by the tool:
> {code:java}
> WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.20/02/10 12:51:42 INFO converter.FSConfigToCSConfigConverter:
> Output directory for yarn-site.xml and capacity-scheduler.xml is: .20/02/10
> 12:51:42 INFO converter.FSConfigToCSConfigConverter: Conversion rules file is
> not defined, using default conversion config!20/02/10 12:51:42 ERROR
> conf.Configuration: error parsing conf
> yarn-site.xmlcom.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog at
> [row,col,system-id]: [1,0,"yarn-site.xml"] at
> com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:687) at
> com.ctc.wstx.sr.BasicStreamReader.handleEOF(BasicStreamReader.java:2220) at
> com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2126)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181) at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3343)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3137) at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3030) at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2996)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2871) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1389) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1361) at
> org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1702) at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.createConfiguration(FSConfigToCSConfigConverter.java:166)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:98)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:137)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:40)20/02/10
> 12:51:42 ERROR converter.FSConfigToCSConfigConverterMain: Error while
> starting FS configuration conversion!
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10130) FS-CS converter: Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041072#comment-17041072
]
Adam Antal commented on YARN-10130:
---
Thanks for the review [~snemeth], let me react to your comments.
1. The first, extra "s" was indeed a typo, but this function multiple xml
files, so I'd like to keep the second "s" character, since it describes better
what this function actually does.
2. The relative path does work, if it starts with "./". I don't know whether
the tool supports relative path at all, but since other code places also
constructs File object out of the String input parameter, it will work the same
way.
3. Done
4. Done
5. Actually I realized that I totally messed up that function, so I modified a
few things around there - I hope now it makes more sense.
6. At first I deleted the parts you mentioned, but then I realized that the
input yarn-site.xml is not necessarily named as "yarn-site.xml" (see tests for
examples, where the input parameter is named as test-yarn-site.xml). So I think
it is important to check these cases separately, as in a general case the two
names do not necessarily match.
7. Indeed, switched them.
> FS-CS converter: Do not allow output dir to be the same as input dir
>
>
> Key: YARN-10130
> URL: https://issues.apache.org/jira/browse/YARN-10130
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Adam Antal
>Priority: Major
> Attachments: YARN-10130.001.patch, YARN-10130.002.patch,
> YARN-10130.003.patch
>
>
> If the input dir where fair-scheduler.xml / yarn-site.xml sits is the same as
> the output dir (defined by the -o switch), the fs2cs tool overwrites the
> source config files, i.e. yarn-site.xml.
> Reproduce this is easy, just run fs2cs tool with this command:
> {code:java}
> /bin/yarn fs2cs --cluster-resource memory-mb=18044928,vcores=16
> --no-terminal-rule-check -y yarn-site.xml -f fair-scheduler.xml -o .
> {code}
> The following (or similar) is emitted by the tool:
> {code:java}
> WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.20/02/10 12:51:42 INFO converter.FSConfigToCSConfigConverter:
> Output directory for yarn-site.xml and capacity-scheduler.xml is: .20/02/10
> 12:51:42 INFO converter.FSConfigToCSConfigConverter: Conversion rules file is
> not defined, using default conversion config!20/02/10 12:51:42 ERROR
> conf.Configuration: error parsing conf
> yarn-site.xmlcom.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog at
> [row,col,system-id]: [1,0,"yarn-site.xml"] at
> com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:687) at
> com.ctc.wstx.sr.BasicStreamReader.handleEOF(BasicStreamReader.java:2220) at
> com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2126)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181) at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3343)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3137) at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3030) at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2996)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2871) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1389) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1361) at
> org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1702) at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.createConfiguration(FSConfigToCSConfigConverter.java:166)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:98)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:137)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:40)20/02/10
> 12:51:42 ERROR converter.FSConfigToCSConfigConverterMain: Error while
> starting FS configuration conversion!
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10119) Cannot reset the AM failure count for YARN Service
[ https://issues.apache.org/jira/browse/YARN-10119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041066#comment-17041066 ] Hudson commented on YARN-10119: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17970 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17970/]) YARN-10119. Option to reset AM failure count for YARN Service (pjoseph: rev 181e6d033ea764012b5d52757bafcf19713fd20a) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/client/ServiceClient.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/conf/YarnServiceConf.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/test/java/org/apache/hadoop/yarn/service/TestYarnNativeServices.java > Cannot reset the AM failure count for YARN Service > -- > > Key: YARN-10119 > URL: https://issues.apache.org/jira/browse/YARN-10119 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: kyungwan nam >Assignee: kyungwan nam >Priority: Major > Labels: Reviewed > Fix For: 3.3.0 > > Attachments: YARN-10119.001.patch > > > Currently, YARN Service does not support to reset AM failure count, which > introduced in YARN-611 > Since the AM failure count is never reset, eventually that will reach > yarn.service.am-restart.max-attempts and the app will be stopped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10119) Cannot reset the AM failure count for YARN Service
[ https://issues.apache.org/jira/browse/YARN-10119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041058#comment-17041058 ] Prabhu Joseph commented on YARN-10119: -- Thanks [~kyungwan nam] for the patch. The patch looks good. Have committed to trunk. > Cannot reset the AM failure count for YARN Service > -- > > Key: YARN-10119 > URL: https://issues.apache.org/jira/browse/YARN-10119 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: kyungwan nam >Assignee: kyungwan nam >Priority: Major > Attachments: YARN-10119.001.patch > > > Currently, YARN Service does not support to reset AM failure count, which > introduced in YARN-611 > Since the AM failure count is never reset, eventually that will reach > yarn.service.am-restart.max-attempts and the app will be stopped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10119) Cannot reset the AM failure count for YARN Service
[ https://issues.apache.org/jira/browse/YARN-10119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-10119: - Fix Version/s: 3.3.0 > Cannot reset the AM failure count for YARN Service > -- > > Key: YARN-10119 > URL: https://issues.apache.org/jira/browse/YARN-10119 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: kyungwan nam >Assignee: kyungwan nam >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-10119.001.patch > > > Currently, YARN Service does not support to reset AM failure count, which > introduced in YARN-611 > Since the AM failure count is never reset, eventually that will reach > yarn.service.am-restart.max-attempts and the app will be stopped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10119) Cannot reset the AM failure count for YARN Service
[ https://issues.apache.org/jira/browse/YARN-10119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-10119: - Labels: Reviewed (was: ) > Cannot reset the AM failure count for YARN Service > -- > > Key: YARN-10119 > URL: https://issues.apache.org/jira/browse/YARN-10119 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: kyungwan nam >Assignee: kyungwan nam >Priority: Major > Labels: Reviewed > Fix For: 3.3.0 > > Attachments: YARN-10119.001.patch > > > Currently, YARN Service does not support to reset AM failure count, which > introduced in YARN-611 > Since the AM failure count is never reset, eventually that will reach > yarn.service.am-restart.max-attempts and the app will be stopped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[
https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041053#comment-17041053
]
Hadoop QA commented on YARN-10141:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 48s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 8s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
36s{color} | {color:green} hadoop-yarn-server-router in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10141 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993921/YARN-10141.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 4e5592154ce1 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ec75071 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25547/testReport/ |
| Max. process+thread count | 721 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25547/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Interceptor in FederationInterceptorREST doesnt update on RM switchover
>
[jira] [Updated] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[ https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton updated YARN-10148: Attachment: YARN-10148.004.patch > Add Unit test for queue ACL for both FS and CS > -- > > Key: YARN-10148 > URL: https://issues.apache.org/jira/browse/YARN-10148 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Attachments: YARN-10148.001.patch, YARN-10148.002.patch, > YARN-10148.003.patch, YARN-10148.004.patch > > > Add some unit tests covering the queue ACL evaluation for both FS and CS. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10140) TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041048#comment-17041048
]
Jim Brennan commented on YARN-10140:
Thanks [~ahussein]! I am +1 (non-binding) on patch 003.
> TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
> --
>
> Key: YARN-10140
> URL: https://issues.apache.org/jira/browse/YARN-10140
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
>Affects Versions: 2.10.0
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: YARN-10140-branch-2.10.001.patch,
> YARN-10140-branch-2.10.002.patch, YARN-10140-branch-2.10.003.patch
>
>
> For branch-2.10, it seems that {{TestTimelineAuthFilterForV2}} was broken for
> some time.
> Using {{git bisect}} tool, I found that the first bad commit is
> "{{a3470c65d8b4e205c8a16d0c0b8dad10d0134bb8}}" in HADOOP-15959 .
> For trunk, the JUnit passes without failures.
> The stack trace is:
> {code:bash}
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 20.231 s <<< FAILURE! - in
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
> [ERROR]
> testPutTimelineEntities[0](org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2)
> Time elapsed: 0.877 s <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> HTTP/localh...@example.com from keytab
> /home/ahussein/workspace/repos/amahadoop-yhadoop-3090/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/bf6554f1-1df4-4ce0-8516-e6c337067870
> javax.security.auth.login.LoginException:
> java.lang.IllegalArgumentException: Illegal principal name
> HTTP/localh...@example.com:
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
> No rules applied to HTTP/localh...@example.com
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1104)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:312)
> at
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.initialize(TestTimelineAuthFilterForV2.java:209)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
>
[jira] [Commented] (YARN-10101) Support listing of aggregated logs for containers belonging to an application attempt
[ https://issues.apache.org/jira/browse/YARN-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041044#comment-17041044 ] Hudson commented on YARN-10101: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17969 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17969/]) YARN-10143. YARN-10101 broke Yarn logs CLI. Contributed by Adam Antal (snemeth: rev 4af2556b48e01150851c7f273a254a16324ba843) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/LogAggregationIndexedFileController.java > Support listing of aggregated logs for containers belonging to an application > attempt > - > > Key: YARN-10101 > URL: https://issues.apache.org/jira/browse/YARN-10101 > Project: Hadoop YARN > Issue Type: Sub-task > Components: log-aggregation, yarn >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Fix For: 3.3.0, 3.2.2 > > Attachments: YARN-10101.001.patch, YARN-10101.002.patch, > YARN-10101.003.patch, YARN-10101.004.patch, YARN-10101.005.patch, > YARN-10101.006.patch, YARN-10101.007.patch, YARN-10101.008.patch, > YARN-10101.009.patch, YARN-10101.branch-3.2.001.patch, > YARN-10101.branch-3.2.001.patch, YARN-10101.branch-3.2.002.patch > > > To display logs without access to the timeline server, we need an interface > where we can query the list of containers with aggregated logs belonging to > an application attempt. > We should add support for this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[ https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041042#comment-17041042 ] Adam Antal commented on YARN-10148: --- Cool, thanks [~kmarton] for fixing the items! You got some checkstyles warning for patch v3, but from my point of view I give a +1 (non-binding) to that patchset. > Add Unit test for queue ACL for both FS and CS > -- > > Key: YARN-10148 > URL: https://issues.apache.org/jira/browse/YARN-10148 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Attachments: YARN-10148.001.patch, YARN-10148.002.patch, > YARN-10148.003.patch > > > Add some unit tests covering the queue ACL evaluation for both FS and CS. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041043#comment-17041043
]
Hudson commented on YARN-10143:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17969 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17969/])
YARN-10143. YARN-10101 broke Yarn logs CLI. Contributed by Adam Antal (snemeth:
rev 4af2556b48e01150851c7f273a254a16324ba843)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/LogAggregationIndexedFileController.java
> YARN-10101 broke Yarn logs CLI
> ---
>
> Key: YARN-10143
> URL: https://issues.apache.org/jira/browse/YARN-10143
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0, 3.2.2
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Blocker
> Attachments: YARN-10143.001.patch, YARN-10143.002.patch,
> YARN-10143.branch-3.2.001.patch
>
>
> YARN-10101 broke the Yarn logs CLI.
> In {{LogsCLI#359}} a {{ContainerLogsRequest}} has been created with null set
> as appAttemptId, while the {{LogAggregationFileController}} instances are
> configured badly to handle this case. The new JHS API works as expected for
> defined application attempt, but we should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041033#comment-17041033
]
Adam Antal commented on YARN-10143:
---
Cherry-pick was clean, but uploaded the patch to have a green jenkins result.
> YARN-10101 broke Yarn logs CLI
> ---
>
> Key: YARN-10143
> URL: https://issues.apache.org/jira/browse/YARN-10143
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0, 3.2.2
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Blocker
> Attachments: YARN-10143.001.patch, YARN-10143.002.patch,
> YARN-10143.branch-3.2.001.patch
>
>
> YARN-10101 broke the Yarn logs CLI.
> In {{LogsCLI#359}} a {{ContainerLogsRequest}} has been created with null set
> as appAttemptId, while the {{LogAggregationFileController}} instances are
> configured badly to handle this case. The new JHS API works as expected for
> defined application attempt, but we should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Antal updated YARN-10143:
--
Attachment: YARN-10143.branch-3.2.001.patch
> YARN-10101 broke Yarn logs CLI
> ---
>
> Key: YARN-10143
> URL: https://issues.apache.org/jira/browse/YARN-10143
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0, 3.2.2
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Blocker
> Attachments: YARN-10143.001.patch, YARN-10143.002.patch,
> YARN-10143.branch-3.2.001.patch
>
>
> YARN-10101 broke the Yarn logs CLI.
> In {{LogsCLI#359}} a {{ContainerLogsRequest}} has been created with null set
> as appAttemptId, while the {{LogAggregationFileController}} instances are
> configured badly to handle this case. The new JHS API works as expected for
> defined application attempt, but we should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041021#comment-17041021
]
Adam Antal commented on YARN-10143:
---
Yes, I'd probably need it, didn't check whether it applies cleanly though.
> YARN-10101 broke Yarn logs CLI
> ---
>
> Key: YARN-10143
> URL: https://issues.apache.org/jira/browse/YARN-10143
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0, 3.2.2
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Blocker
> Attachments: YARN-10143.001.patch, YARN-10143.002.patch
>
>
> YARN-10101 broke the Yarn logs CLI.
> In {{LogsCLI#359}} a {{ContainerLogsRequest}} has been created with null set
> as appAttemptId, while the {{LogAggregationFileController}} instances are
> configured badly to handle this case. The new JHS API works as expected for
> defined application attempt, but we should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041019#comment-17041019
]
Szilard Nemeth commented on YARN-10143:
---
Hi [~adam.antal],
Thanks for this fix, committed to trunk!
[~kmarton]: Thanks for the review.
[~adam.antal]: Do we need the backport to 3.2? YARN-10101 is backported.
> YARN-10101 broke Yarn logs CLI
> ---
>
> Key: YARN-10143
> URL: https://issues.apache.org/jira/browse/YARN-10143
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0, 3.2.2
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Blocker
> Attachments: YARN-10143.001.patch, YARN-10143.002.patch
>
>
> YARN-10101 broke the Yarn logs CLI.
> In {{LogsCLI#359}} a {{ContainerLogsRequest}} has been created with null set
> as appAttemptId, while the {{LogAggregationFileController}} instances are
> configured badly to handle this case. The new JHS API works as expected for
> defined application attempt, but we should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9593) Updating scheduler conf with comma in config value fails
[
https://issues.apache.org/jira/browse/YARN-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041015#comment-17041015
]
Prabhu Joseph commented on YARN-9593:
-
[~tanu.ajmera] The regex logic looks good. The patch supports below ways to
keep comma in the value of a property during globalupdate.
{code:java}
yarn schedulerconf -global "key1=user1\,user2 group1\,group2"
or
yarn schedulerconf -global key1=user1\\,user2
or
yarn schedulerconf -global "key1=user1\\,user2 group1\\,group2"
{code}
Can you also include this logic for splitting the key value pairs while add and
update operations. And also update the {{SchedConfCLI#printUsage}} with
examples. Thanks.
> Updating scheduler conf with comma in config value fails
>
>
> Key: YARN-9593
> URL: https://issues.apache.org/jira/browse/YARN-9593
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 3.0.0, 3.2.0, 3.1.2
>Reporter: Anthony Hsu
>Assignee: Tanu Ajmera
>Priority: Major
> Attachments: YARN-9593-001.patch
>
>
> For example:
> {code:java}
> $ yarn schedulerconf -update "root.gridops:acl_administer_queue=user1,user2
> group1,group2"
> Specify configuration key value as confKey=confVal.{code}
> This fails because there is a comma in the config value and the SchedConfCLI
> splits on comma first, expecting each split to a k=v pair.
> {noformat}
> void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
> if (args == null) {
> return;
> }
> HashMap globalUpdates = new HashMap<>();
> for (String globalUpdate : args.split(",")) {
> putKeyValuePair(globalUpdates, globalUpdate);
> }
> updateInfo.setGlobalParams(globalUpdates);
> }{noformat}
> Cc: [~jhung]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9593) Updating scheduler conf with comma in config value fails
[
https://issues.apache.org/jira/browse/YARN-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9593:
Description:
For example:
{code:java}
$ yarn schedulerconf -global "root.gridops:acl_administer_queue=user1,user2
group1,group2"
Specify configuration key value as confKey=confVal.{code}
This fails because there is a comma in the config value and the SchedConfCLI
splits on comma first, expecting each split to a k=v pair.
{noformat}
void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
if (args == null) {
return;
}
HashMap globalUpdates = new HashMap<>();
for (String globalUpdate : args.split(",")) {
putKeyValuePair(globalUpdates, globalUpdate);
}
updateInfo.setGlobalParams(globalUpdates);
}{noformat}
Cc: [~jhung]
was:
For example:
{code:java}
$ yarn schedulerconf -update "root.gridops:acl_administer_queue=user1,user2
group1,group2"
Specify configuration key value as confKey=confVal.{code}
This fails because there is a comma in the config value and the SchedConfCLI
splits on comma first, expecting each split to a k=v pair.
{noformat}
void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
if (args == null) {
return;
}
HashMap globalUpdates = new HashMap<>();
for (String globalUpdate : args.split(",")) {
putKeyValuePair(globalUpdates, globalUpdate);
}
updateInfo.setGlobalParams(globalUpdates);
}{noformat}
Cc: [~jhung]
> Updating scheduler conf with comma in config value fails
>
>
> Key: YARN-9593
> URL: https://issues.apache.org/jira/browse/YARN-9593
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 3.0.0, 3.2.0, 3.1.2
>Reporter: Anthony Hsu
>Assignee: Tanu Ajmera
>Priority: Major
> Attachments: YARN-9593-001.patch
>
>
> For example:
> {code:java}
> $ yarn schedulerconf -global "root.gridops:acl_administer_queue=user1,user2
> group1,group2"
> Specify configuration key value as confKey=confVal.{code}
> This fails because there is a comma in the config value and the SchedConfCLI
> splits on comma first, expecting each split to a k=v pair.
> {noformat}
> void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
> if (args == null) {
> return;
> }
> HashMap globalUpdates = new HashMap<>();
> for (String globalUpdate : args.split(",")) {
> putKeyValuePair(globalUpdates, globalUpdate);
> }
> updateInfo.setGlobalParams(globalUpdates);
> }{noformat}
> Cc: [~jhung]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9593) Updating scheduler conf with comma in config value fails
[
https://issues.apache.org/jira/browse/YARN-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9593:
Description:
For example:
{code:java}
$ yarn schedulerconf -update "root.gridops:acl_administer_queue=user1,user2
group1,group2"
Specify configuration key value as confKey=confVal.{code}
This fails because there is a comma in the config value and the SchedConfCLI
splits on comma first, expecting each split to a k=v pair.
{noformat}
void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
if (args == null) {
return;
}
HashMap globalUpdates = new HashMap<>();
for (String globalUpdate : args.split(",")) {
putKeyValuePair(globalUpdates, globalUpdate);
}
updateInfo.setGlobalParams(globalUpdates);
}{noformat}
Cc: [~jhung]
was:
For example:
{code:java}
$ yarn schedulerconf -global "root.gridops:acl_administer_queue=user1,user2
group1,group2"
Specify configuration key value as confKey=confVal.{code}
This fails because there is a comma in the config value and the SchedConfCLI
splits on comma first, expecting each split to a k=v pair.
{noformat}
void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
if (args == null) {
return;
}
HashMap globalUpdates = new HashMap<>();
for (String globalUpdate : args.split(",")) {
putKeyValuePair(globalUpdates, globalUpdate);
}
updateInfo.setGlobalParams(globalUpdates);
}{noformat}
Cc: [~jhung]
> Updating scheduler conf with comma in config value fails
>
>
> Key: YARN-9593
> URL: https://issues.apache.org/jira/browse/YARN-9593
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 3.0.0, 3.2.0, 3.1.2
>Reporter: Anthony Hsu
>Assignee: Tanu Ajmera
>Priority: Major
> Attachments: YARN-9593-001.patch
>
>
> For example:
> {code:java}
> $ yarn schedulerconf -update "root.gridops:acl_administer_queue=user1,user2
> group1,group2"
> Specify configuration key value as confKey=confVal.{code}
> This fails because there is a comma in the config value and the SchedConfCLI
> splits on comma first, expecting each split to a k=v pair.
> {noformat}
> void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
> if (args == null) {
> return;
> }
> HashMap globalUpdates = new HashMap<>();
> for (String globalUpdate : args.split(",")) {
> putKeyValuePair(globalUpdates, globalUpdate);
> }
> updateInfo.setGlobalParams(globalUpdates);
> }{noformat}
> Cc: [~jhung]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040990#comment-17040990
]
Adam Antal commented on YARN-10143:
---
I checked that using TFile aggregation the Yarn logs CLI is working as intended
(checked logs for running and finished application) - the problem was only with
IndexedFile.
[~snemeth], could you look at it?
> YARN-10101 broke Yarn logs CLI
> ---
>
> Key: YARN-10143
> URL: https://issues.apache.org/jira/browse/YARN-10143
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0, 3.2.2
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Blocker
> Attachments: YARN-10143.001.patch, YARN-10143.002.patch
>
>
> YARN-10101 broke the Yarn logs CLI.
> In {{LogsCLI#359}} a {{ContainerLogsRequest}} has been created with null set
> as appAttemptId, while the {{LogAggregationFileController}} instances are
> configured badly to handle this case. The new JHS API works as expected for
> defined application attempt, but we should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[
https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040972#comment-17040972
]
Hadoop QA commented on YARN-10148:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 19s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 29s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 6 new + 9 unchanged - 0 fixed = 15 total (was 9) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 1s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 84m
33s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}139m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10148 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993989/YARN-10148.003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux a650fdd53e60 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ec75071 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/25546/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25546/testReport/ |
| Max. process+thread count | 868 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS
[
https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040904#comment-17040904
]
Hadoop QA commented on YARN-9879:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
42s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 16 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 41s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
23s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 6m
35s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 6m 35s{color}
| {color:red} root in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 55s{color} | {color:orange} root: The patch generated 78 new + 2135
unchanged - 5 fixed = 2213 total (was 2140) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
24s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 4m
5s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
24s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 26s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
53s{color} | {color:green} hadoop-sls in the patch passed. {color} |
| {color:blue}0{color} | {color:blue} asflicense {color} | {color:blue} 0m
44s{color} | {color:blue} ASF License check generated no output? {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-9879 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993985/YARN-9879.POC004.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 4435371a8212 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ec75071 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| mvninstall |
[jira] [Commented] (YARN-9593) Updating scheduler conf with comma in config value fails
[
https://issues.apache.org/jira/browse/YARN-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040873#comment-17040873
]
Hadoop QA commented on YARN-9593:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 23s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 17s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 2 new +
0 unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 26s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 26m
28s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 83m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-9593 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993981/YARN-9593-001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 89ce52d7c392 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ec75071 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/25544/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25544/testReport/ |
| Max. process+thread count | 531 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client |
| Console output |
[jira] [Commented] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[
https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040871#comment-17040871
]
Kinga Marton commented on YARN-10148:
-
Thank you [~adam.antal] for the review. I have addressed your comments in
YARN-10148.003.patch:
- changed the newly introduced queue names to D and D1, where D1 is a child
queue of D. Also added a small comment for {{updateConfigWithDAndD1Queues}}
method
- {{in updateConfigWithDAndD1Queues}} there are mixed the queues from parent
class and the actual one because I just want to update the configuration with
the new queues instead of creating a new one. In this case if I would keep only
the new queues I would have to do a lot of configuration change related the old
ones (stop them, modify the capacities, etc.)
- fixed the indentation (mi idea was set to use 8 spaces for continuation
indent)
- Added assertion message
- reverted TestAbsoluteResourceConfiguration changes: yes, it was by mistake.
This was a good example of why I should not use QUEUE_C again :)
- replaced "test.build.data", in {{TestFairSchedulerQueueACLs with
GenericTestUtils.SYSPROP_TEST_DATA_DIR}}
> Add Unit test for queue ACL for both FS and CS
> --
>
> Key: YARN-10148
> URL: https://issues.apache.org/jira/browse/YARN-10148
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler
>Reporter: Kinga Marton
>Assignee: Kinga Marton
>Priority: Major
> Attachments: YARN-10148.001.patch, YARN-10148.002.patch,
> YARN-10148.003.patch
>
>
> Add some unit tests covering the queue ACL evaluation for both FS and CS.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[ https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton updated YARN-10148: Attachment: YARN-10148.003.patch > Add Unit test for queue ACL for both FS and CS > -- > > Key: YARN-10148 > URL: https://issues.apache.org/jira/browse/YARN-10148 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Attachments: YARN-10148.001.patch, YARN-10148.002.patch, > YARN-10148.003.patch > > > Add some unit tests covering the queue ACL evaluation for both FS and CS. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040829#comment-17040829 ] Gergely Pollak commented on YARN-9879: -- Did some very basic smoke testing, and in this version it is possible to submit applications with leaf queue names and full path as well. However it need some excessive testing, for ACLs, placement rules, preemtion just to name a few. But since the initial manual tests worked I've updated the testcases to use getQueuePath instead of getQueueName, probably a lot will fail this way too, but then I'm moving on to fixing those. After the tests are fixed, I start manually testing the more complex features of the CS. > Allow multiple leaf queues with the same name in CS > --- > > Key: YARN-9879 > URL: https://issues.apache.org/jira/browse/YARN-9879 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Gergely Pollak >Priority: Major > Labels: fs2cs > Attachments: CSQueue.getQueueUsage.txt, DesignDoc_v1.pdf, > YARN-9879.POC001.patch, YARN-9879.POC002.patch, YARN-9879.POC003.patch, > YARN-9879.POC004.patch > > > Currently the leaf queue's name must be unique regardless of its position in > the queue hierarchy. > Design doc and first proposal is being made, I'll attach it as soon as it's > done. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9879) Allow multiple leaf queues with the same name in CS
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gergely Pollak updated YARN-9879: - Attachment: YARN-9879.POC004.patch > Allow multiple leaf queues with the same name in CS > --- > > Key: YARN-9879 > URL: https://issues.apache.org/jira/browse/YARN-9879 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Gergely Pollak >Priority: Major > Labels: fs2cs > Attachments: CSQueue.getQueueUsage.txt, DesignDoc_v1.pdf, > YARN-9879.POC001.patch, YARN-9879.POC002.patch, YARN-9879.POC003.patch, > YARN-9879.POC004.patch > > > Currently the leaf queue's name must be unique regardless of its position in > the queue hierarchy. > Design doc and first proposal is being made, I'll attach it as soon as it's > done. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9593) Updating scheduler conf with comma in config value fails
[
https://issues.apache.org/jira/browse/YARN-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanu Ajmera updated YARN-9593:
--
Attachment: YARN-9593-001.patch
> Updating scheduler conf with comma in config value fails
>
>
> Key: YARN-9593
> URL: https://issues.apache.org/jira/browse/YARN-9593
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 3.0.0, 3.2.0, 3.1.2
>Reporter: Anthony Hsu
>Assignee: Tanu Ajmera
>Priority: Major
> Attachments: YARN-9593-001.patch
>
>
> For example:
> {code:java}
> $ yarn schedulerconf -update "root.gridops:acl_administer_queue=user1,user2
> group1,group2"
> Specify configuration key value as confKey=confVal.{code}
> This fails because there is a comma in the config value and the SchedConfCLI
> splits on comma first, expecting each split to a k=v pair.
> {noformat}
> void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
> if (args == null) {
> return;
> }
> HashMap globalUpdates = new HashMap<>();
> for (String globalUpdate : args.split(",")) {
> putKeyValuePair(globalUpdates, globalUpdate);
> }
> updateInfo.setGlobalParams(globalUpdates);
> }{noformat}
> Cc: [~jhung]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10153) Problems of offline analysis with showSimulationTrace.html
YingmingWang created YARN-10153:
---
Summary: Problems of offline analysis with showSimulationTrace.html
Key: YARN-10153
URL: https://issues.apache.org/jira/browse/YARN-10153
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler-load-simulator
Affects Versions: 3.2.1
Environment: Ubuntu18.04
JAVA 1.8.0_241
Hadoop-3.2.1
Reporter: YingmingWang
Fix For: 3.2.1
Attachments: INFO.txt
I am new to learn YARN SLS. After the configuration of HADOOP and YARN SLS, I
run the example in sample-data. It doesn't have WARNING or ERROR. I can see the
chart and data changing in real time by host:port/simulate but I can not
analyse the realtimetrack.json file by showSimulationTrace.html after the run.
I modified my file with the improvement of YARN-9836 but it still can't analyze
trace. The whole INFO file is placed in .txt.
{code:java}
// code placeholder
# Begining
INFO conf.Configuration: resource-types.xml not found
INFO resource.ResourceUtils:Unable to find 'resource-types.xml'.
# Finally
2020-02-20 14:59:40,838 INFO sls.SLSRunner: SLSRunner tears down.
2020-02-20 14:59:40,839 INFO resourcemanager.RMAppManager$ApplicationSummary:
appId=application_1582181515746_0002,name=N/A,user=jenkins,queue=sls_queue_1,state=FINISHED,trackingUrl=http://host2-virtual-machine:18088/proxy/application_1582181515746_0002/,appMasterHost=localhost,submitTime=1582181622389,startTime=1582181622389,launchTime=-1,finishTime=1582181980390,finalStatus=SUCCEEDED,memorySeconds=2433739,vcoreSeconds=2313,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=,applicationType=YARN,resourceSeconds=2433739 MB-seconds\, 2313
vcore-seconds,preemptedResourceSeconds=0 MB-seconds\, 0
vcore-seconds,applicationTags=
{code}
I want to know how to solve it or a more correct configuration process. Thank
you very much.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
