[jira] [Commented] (YARN-8253) HTTPS Ats v2 api call fails with "bad HTTP parsed"
[
https://issues.apache.org/jira/browse/YARN-8253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466903#comment-16466903
]
Charan Hebri commented on YARN-8253:
[~rohithsharma] I can provide a fix if you haven't started.
> HTTPS Ats v2 api call fails with "bad HTTP parsed"
> --
>
> Key: YARN-8253
> URL: https://issues.apache.org/jira/browse/YARN-8253
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.1.0
>Reporter: Yesha Vora
>Assignee: Rohith Sharma K S
>Priority: Major
>
> When Yarn http policy is set to Https_only, ATS v2 should use HTTPS address.
> Here, ATS v2 call is failing with below error.
> {code:java}
> [hrt_qa@xxx root]$ curl -i -k -s -1 -H 'Content-Type: application/json' -H
> 'Accept: application/json' --negotiate -u:
> 'https://xxx:8199/ws/v2/timeline/apps/application_1525238789838_0003/entities/COMPONENT_INSTANCE?fields=ALL'
> [hrt_qa@xxx root]$ echo $?
> 35{code}
> {code:java|title=Ats v2}
> 2018-05-02 05:45:40,427 WARN http.HttpParser (HttpParser.java:(1832))
> - Illegal character 0x16 in state=START for buffer
> HeapByteBuffer@dba438[p=1,l=222,c=8192,r=221]={\x16<<<\x03\x01\x00\xD9\x01\x00\x00\xD5\x03\x03;X\xEd\xD1orq...\x01\x05\x01\x06\x01\x02\x01\x04\x02\x05\x02\x06\x02\x02\x02>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}
> 2018-05-02 05:45:40,428 WARN http.HttpParser
> (HttpParser.java:parseNext(1435)) - bad HTTP parsed: 400 Illegal character
> 0x16 for HttpChannelOverHttp@2efbda6c{r=0,c=false,a=IDLE,uri=null}{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8253) HTTPS Ats v2 api call fails with "bad HTTP parsed"
[
https://issues.apache.org/jira/browse/YARN-8253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8253:
---
Attachment: YARN-8253.01.patch
> HTTPS Ats v2 api call fails with "bad HTTP parsed"
> --
>
> Key: YARN-8253
> URL: https://issues.apache.org/jira/browse/YARN-8253
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.1.0
>Reporter: Yesha Vora
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-8253.01.patch
>
>
> When Yarn http policy is set to Https_only, ATS v2 should use HTTPS address.
> Here, ATS v2 call is failing with below error.
> {code:java}
> [hrt_qa@xxx root]$ curl -i -k -s -1 -H 'Content-Type: application/json' -H
> 'Accept: application/json' --negotiate -u:
> 'https://xxx:8199/ws/v2/timeline/apps/application_1525238789838_0003/entities/COMPONENT_INSTANCE?fields=ALL'
> [hrt_qa@xxx root]$ echo $?
> 35{code}
> {code:java|title=Ats v2}
> 2018-05-02 05:45:40,427 WARN http.HttpParser (HttpParser.java:(1832))
> - Illegal character 0x16 in state=START for buffer
> HeapByteBuffer@dba438[p=1,l=222,c=8192,r=221]={\x16<<<\x03\x01\x00\xD9\x01\x00\x00\xD5\x03\x03;X\xEd\xD1orq...\x01\x05\x01\x06\x01\x02\x01\x04\x02\x05\x02\x06\x02\x02\x02>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}
> 2018-05-02 05:45:40,428 WARN http.HttpParser
> (HttpParser.java:parseNext(1435)) - bad HTTP parsed: 400 Illegal character
> 0x16 for HttpChannelOverHttp@2efbda6c{r=0,c=false,a=IDLE,uri=null}{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8226) Improve anti-affinity section description in YARN Service API doc
Charan Hebri created YARN-8226:
--
Summary: Improve anti-affinity section description in YARN Service
API doc
Key: YARN-8226
URL: https://issues.apache.org/jira/browse/YARN-8226
Project: Hadoop YARN
Issue Type: Improvement
Components: docs, documentation
Reporter: Charan Hebri
The anti-affinity section in the YARN services doc says,
{noformat}
Note, that the 3 containers will come up on 3 different nodes. If there are
less than 3 NMs running in the cluster, then all 3 container requests will not
be fulfilled and the service will be in non-STABLE state.{noformat}
Based on the description the expected behavior for the case of number of NMs
less than the number of containers requested isn't very obvious. Opening issue
to improve the comment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8223) ClassNotFoundException when auxiliary service is loaded from HDFS
Charan Hebri created YARN-8223:
--
Summary: ClassNotFoundException when auxiliary service is loaded
from HDFS
Key: YARN-8223
URL: https://issues.apache.org/jira/browse/YARN-8223
Project: Hadoop YARN
Issue Type: Bug
Reporter: Charan Hebri
Loading an auxiliary jar from a local location on a node manager works as
expected,
{noformat}
2018-04-26 15:09:26,179 INFO util.ApplicationClassLoader
(ApplicationClassLoader.java:(98)) - classpath:
[file:/grid/0/hadoop/yarn/local/aux-service-local.jar]
2018-04-26 15:09:26,179 INFO util.ApplicationClassLoader
(ApplicationClassLoader.java:(99)) - system classes: [java.,
javax.accessibility., javax.activation., javax.activity., javax.annotation.,
javax.annotation.processing., javax.crypto., javax.imageio., javax.jws.,
javax.lang.model., -javax.management.j2ee., javax.management., javax.naming.,
javax.net., javax.print., javax.rmi., javax.script.,
-javax.security.auth.message., javax.security.auth., javax.security.cert.,
javax.security.sasl., javax.sound., javax.sql., javax.swing., javax.tools.,
javax.transaction., -javax.xml.registry., -javax.xml.rpc., javax.xml.,
org.w3c.dom., org.xml.sax., org.apache.commons.logging., org.apache.log4j.,
-org.apache.hadoop.hbase., org.apache.hadoop., core-default.xml,
hdfs-default.xml, mapred-default.xml, yarn-default.xml]
2018-04-26 15:09:26,181 INFO containermanager.AuxServices
(AuxServices.java:serviceInit(252)) - The aux service:test_aux_local are using
the custom classloader
2018-04-26 15:09:26,182 WARN containermanager.AuxServices
(AuxServices.java:serviceInit(268)) - The Auxiliary Service named
'test_aux_local' in the configuration is for class
org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader
which has a name of 'org.apache.auxtest.AuxServiceFromLocal with custom class
loader'. Because these are not the same tools trying to send ServiceData and
read Service Meta Data may have issues unless the refer to the name in the
config.
2018-04-26 15:09:26,182 INFO containermanager.AuxServices
(AuxServices.java:addService(103)) - Adding auxiliary service
org.apache.auxtest.AuxServiceFromLocal with custom class loader,
"test_aux_local"{noformat}
But loading the same jar from a location on HDFS fails with a
ClassNotFoundException.
{noformat}
018-04-26 15:14:39,683 INFO util.ApplicationClassLoader
(ApplicationClassLoader.java:(98)) - classpath: []
2018-04-26 15:14:39,683 INFO util.ApplicationClassLoader
(ApplicationClassLoader.java:(99)) - system classes: [java.,
javax.accessibility., javax.activation., javax.activity., javax.annotation.,
javax.annotation.processing., javax.crypto., javax.imageio., javax.jws.,
javax.lang.model., -javax.management.j2ee., javax.management., javax.naming.,
javax.net., javax.print., javax.rmi., javax.script.,
-javax.security.auth.message., javax.security.auth., javax.security.cert.,
javax.security.sasl., javax.sound., javax.sql., javax.swing., javax.tools.,
javax.transaction., -javax.xml.registry., -javax.xml.rpc., javax.xml.,
org.w3c.dom., org.xml.sax., org.apache.commons.logging., org.apache.log4j.,
-org.apache.hadoop.hbase., org.apache.hadoop., core-default.xml,
hdfs-default.xml, mapred-default.xml, yarn-default.xml]
2018-04-26 15:14:39,687 INFO service.AbstractService
(AbstractService.java:noteFailure(267)) - Service
org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices failed
in state INITED
java.lang.ClassNotFoundException: org.apache.auxtest.AuxServiceFromLocal
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at
org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:189)
at
org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:157)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader.getInstance(AuxiliaryServiceWithCustomClassLoader.java:169)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:249)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:316)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
[jira] [Assigned] (YARN-8223) ClassNotFoundException when auxiliary service is loaded from HDFS
[
https://issues.apache.org/jira/browse/YARN-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri reassigned YARN-8223:
--
Assignee: Zian Chen
> ClassNotFoundException when auxiliary service is loaded from HDFS
> -
>
> Key: YARN-8223
> URL: https://issues.apache.org/jira/browse/YARN-8223
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Charan Hebri
>Assignee: Zian Chen
>Priority: Blocker
>
> Loading an auxiliary jar from a local location on a node manager works as
> expected,
> {noformat}
> 2018-04-26 15:09:26,179 INFO util.ApplicationClassLoader
> (ApplicationClassLoader.java:(98)) - classpath:
> [file:/grid/0/hadoop/yarn/local/aux-service-local.jar]
> 2018-04-26 15:09:26,179 INFO util.ApplicationClassLoader
> (ApplicationClassLoader.java:(99)) - system classes: [java.,
> javax.accessibility., javax.activation., javax.activity., javax.annotation.,
> javax.annotation.processing., javax.crypto., javax.imageio., javax.jws.,
> javax.lang.model., -javax.management.j2ee., javax.management., javax.naming.,
> javax.net., javax.print., javax.rmi., javax.script.,
> -javax.security.auth.message., javax.security.auth., javax.security.cert.,
> javax.security.sasl., javax.sound., javax.sql., javax.swing., javax.tools.,
> javax.transaction., -javax.xml.registry., -javax.xml.rpc., javax.xml.,
> org.w3c.dom., org.xml.sax., org.apache.commons.logging., org.apache.log4j.,
> -org.apache.hadoop.hbase., org.apache.hadoop., core-default.xml,
> hdfs-default.xml, mapred-default.xml, yarn-default.xml]
> 2018-04-26 15:09:26,181 INFO containermanager.AuxServices
> (AuxServices.java:serviceInit(252)) - The aux service:test_aux_local are
> using the custom classloader
> 2018-04-26 15:09:26,182 WARN containermanager.AuxServices
> (AuxServices.java:serviceInit(268)) - The Auxiliary Service named
> 'test_aux_local' in the configuration is for class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader
> which has a name of 'org.apache.auxtest.AuxServiceFromLocal with custom
> class loader'. Because these are not the same tools trying to send
> ServiceData and read Service Meta Data may have issues unless the refer to
> the name in the config.
> 2018-04-26 15:09:26,182 INFO containermanager.AuxServices
> (AuxServices.java:addService(103)) - Adding auxiliary service
> org.apache.auxtest.AuxServiceFromLocal with custom class loader,
> "test_aux_local"{noformat}
> But loading the same jar from a location on HDFS fails with a
> ClassNotFoundException.
> {noformat}
> 018-04-26 15:14:39,683 INFO util.ApplicationClassLoader
> (ApplicationClassLoader.java:(98)) - classpath: []
> 2018-04-26 15:14:39,683 INFO util.ApplicationClassLoader
> (ApplicationClassLoader.java:(99)) - system classes: [java.,
> javax.accessibility., javax.activation., javax.activity., javax.annotation.,
> javax.annotation.processing., javax.crypto., javax.imageio., javax.jws.,
> javax.lang.model., -javax.management.j2ee., javax.management., javax.naming.,
> javax.net., javax.print., javax.rmi., javax.script.,
> -javax.security.auth.message., javax.security.auth., javax.security.cert.,
> javax.security.sasl., javax.sound., javax.sql., javax.swing., javax.tools.,
> javax.transaction., -javax.xml.registry., -javax.xml.rpc., javax.xml.,
> org.w3c.dom., org.xml.sax., org.apache.commons.logging., org.apache.log4j.,
> -org.apache.hadoop.hbase., org.apache.hadoop., core-default.xml,
> hdfs-default.xml, mapred-default.xml, yarn-default.xml]
> 2018-04-26 15:14:39,687 INFO service.AbstractService
> (AbstractService.java:noteFailure(267)) - Service
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices failed
> in state INITED
> java.lang.ClassNotFoundException: org.apache.auxtest.AuxServiceFromLocal
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:189)
> at
> org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:157)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader.getInstance(AuxiliaryServiceWithCustomClassLoader.java:169)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:249)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
>
[jira] [Updated] (YARN-8247) Incorrect HTTP status code returned by ATSv2 for non-whitelisted users
[
https://issues.apache.org/jira/browse/YARN-8247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8247:
---
Description:
When using the below configuration in ATSv2 reader,
{noformat}
yarn.timeline-service.read.authentication.enabled=true
yarn.timeline-service.read.allowed.users=user1,user2{noformat}
A query with user3 throws a Forbidden Exception with a status code of 500
(Internal Server Error) instead of the expected 403 for Forbidden. Stack trace
of the response,
{noformat}
HTTP ERROR 500
Problem accessing /ws/v2/timeline/apps/application_1525427743175_0009. Reason:
Server Error
Caused by:
org.apache.hadoop.yarn.webapp.ForbiddenException: java.lang.Exception: user
user3 is not allowed to read TimelineService V2 data
at
org.apache.hadoop.yarn.server.timelineservice.reader.security.TimelineReaderWhitelistAuthorizationFilter.doFilter(TimelineReaderWhitelistAuthorizationFilter.java:80)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1601)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: user user3 is not allowed to read
TimelineService V2 data
at
org.apache.hadoop.yarn.webapp.ForbiddenException.(ForbiddenException.java:41)
... 34 more{noformat}
cc [~vrushalic] [~rohithsharma]
was:
When using the below configuration in ATSv2 reader,
{noformat}
yarn.timeline-service.read.authentication.enabled=true
yarn.timeline-service.read.allowed.users=user1,user2{noformat}
A query with user3 throws a Forbidden Exception with a status code of 500
(Internal Server Error) instead of the expected 403 for Forbidden. Stack trace
of the response,
{noformat}
HTTP ERROR 500
Problem accessing /ws/v2/timeline/apps/application_1525427743175_0009. Reason:
Server Error
Caused by:
org.apache.hadoop.yarn.webapp.ForbiddenException: java.lang.Exception: user
user3 is not allowed to read TimelineService V2 data
[jira] [Updated] (YARN-8247) Incorrect HTTP status code returned by ATSv2 for non-whitelisted users
[
https://issues.apache.org/jira/browse/YARN-8247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8247:
---
Description:
When using the below configuration in ATSv2 reader,
{noformat}
yarn.timeline-service.read.authentication.enabled=true
yarn.timeline-service.read.allowed.users=user1,user2{noformat}
A query with user3 throws a Forbidden Exception with a status code of 500
(Internal Server Error) instead of the expected 403 for Forbidden. Stack trace
of the response,
{noformat}
HTTP ERROR 500
Problem accessing /ws/v2/timeline/apps/application_1525427743175_0009. Reason:
Server Error
Caused by:
org.apache.hadoop.yarn.webapp.ForbiddenException: java.lang.Exception: user
user3 is not allowed to read TimelineService V2 data
at
org.apache.hadoop.yarn.server.timelineservice.reader.security.TimelineReaderWhitelistAuthorizationFilter.doFilter(TimelineReaderWhitelistAuthorizationFilter.java:80)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1601)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: user user3 is not allowed to read
TimelineService V2 data
at
org.apache.hadoop.yarn.webapp.ForbiddenException.(ForbiddenException.java:41)
... 34 more{noformat}
cc [~vrushalic]
was:
When using the below configuration in ATSv2 reader,
{noformat}
yarn.timeline-service.read.authentication.enabled=true
yarn.timeline-service.read.allowed.users=user1,user2{noformat}
A query with user3 throws a Forbidden Exception with a status code of 500
(internal Server Error) instead of the expected 403 for Forbidden. Stack trace
of the response,
{noformat}
HTTP ERROR 500
Problem accessing /ws/v2/timeline/apps/application_1525427743175_0009. Reason:
Server Error
Caused by:
org.apache.hadoop.yarn.webapp.ForbiddenException: java.lang.Exception: user
user3 is not allowed to read TimelineService V2 data
at
[jira] [Created] (YARN-8247) Incorrect HTTP status code returned by ATSv2 for non-whitelisted users
Charan Hebri created YARN-8247:
--
Summary: Incorrect HTTP status code returned by ATSv2 for
non-whitelisted users
Key: YARN-8247
URL: https://issues.apache.org/jira/browse/YARN-8247
Project: Hadoop YARN
Issue Type: Bug
Components: ATSv2
Reporter: Charan Hebri
When using the below configuration in ATSv2 reader,
{noformat}
yarn.timeline-service.read.authentication.enabled=true
yarn.timeline-service.read.allowed.users=user1,user2{noformat}
A query with user3 throws a Forbidden Exception with a status code of 500
(internal Server Error) instead of the expected 403 for Forbidden. Stack trace
of the response,
{noformat}
HTTP ERROR 500
Problem accessing /ws/v2/timeline/apps/application_1525427743175_0009. Reason:
Server Error
Caused by:
org.apache.hadoop.yarn.webapp.ForbiddenException: java.lang.Exception: user
user3 is not allowed to read TimelineService V2 data
at
org.apache.hadoop.yarn.server.timelineservice.reader.security.TimelineReaderWhitelistAuthorizationFilter.doFilter(TimelineReaderWhitelistAuthorizationFilter.java:80)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1601)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: user user3 is not allowed to read
TimelineService V2 data
at
org.apache.hadoop.yarn.webapp.ForbiddenException.(ForbiddenException.java:41)
... 34 more{noformat}
cc [~vrushalic]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8396) Click on an individual container continuously spins and doesn't load the page
Charan Hebri created YARN-8396:
--
Summary: Click on an individual container continuously spins and
doesn't load the page
Key: YARN-8396
URL: https://issues.apache.org/jira/browse/YARN-8396
Project: Hadoop YARN
Issue Type: Bug
Reporter: Charan Hebri
For a running application, a click on an individual container leads to an
infinite spinner which doesn't load the corresponding page. To reproduce, with
a running application click:
Nodes -> \{Node_HTTP_Address} -> List of Containers on this Node ->
\{Container_id}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8396) Click on an individual container continuously spins and doesn't load the page
[
https://issues.apache.org/jira/browse/YARN-8396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8396:
---
Attachment: Screen Shot 2018-05-31 at 3.24.09 PM.png
> Click on an individual container continuously spins and doesn't load the page
> -
>
> Key: YARN-8396
> URL: https://issues.apache.org/jira/browse/YARN-8396
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Charan Hebri
>Priority: Blocker
> Attachments: Screen Shot 2018-05-31 at 3.24.09 PM.png
>
>
> For a running application, a click on an individual container leads to an
> infinite spinner which doesn't load the corresponding page. To reproduce,
> with a running application click:
> Nodes -> \{Node_HTTP_Address} -> List of Containers on this Node ->
> \{Container_id}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8372) ApplicationAttemptNotFoundException should be handled correctly by Distributed Shell App Master
Charan Hebri created YARN-8372:
--
Summary: ApplicationAttemptNotFoundException should be handled
correctly by Distributed Shell App Master
Key: YARN-8372
URL: https://issues.apache.org/jira/browse/YARN-8372
Project: Hadoop YARN
Issue Type: Bug
Components: distributed-shell
Reporter: Charan Hebri
{noformat}
try {
response = client.allocate(progress);
} catch (ApplicationAttemptNotFoundException e) {
handler.onShutdownRequest();
LOG.info("Shutdown requested. Stopping callback.");
return;{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8372) ApplicationAttemptNotFoundException should be handled correctly by Distributed Shell App Master
[
https://issues.apache.org/jira/browse/YARN-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8372:
---
Description:
{noformat}
try {
response = client.allocate(progress);
} catch (ApplicationAttemptNotFoundException e) {
handler.onShutdownRequest();
LOG.info("Shutdown requested. Stopping callback.");
return;{noformat}
is a code snippet from AMRMClientAsyncImpl. The corresponding onShutdownRequest
call for the Distributed Shell App master is,
{noformat}
@Override
public void onShutdownRequest() {
done = true;
}{noformat}
Due to this, the current behavior is that whenever an application attempt fails
due to a NM restart (where the DS AM is running), an
was:
{noformat}
try {
response = client.allocate(progress);
} catch (ApplicationAttemptNotFoundException e) {
handler.onShutdownRequest();
LOG.info("Shutdown requested. Stopping callback.");
return;{noformat}
> ApplicationAttemptNotFoundException should be handled correctly by
> Distributed Shell App Master
> ---
>
> Key: YARN-8372
> URL: https://issues.apache.org/jira/browse/YARN-8372
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Priority: Major
>
> {noformat}
> try {
> response = client.allocate(progress);
> } catch (ApplicationAttemptNotFoundException e) {
> handler.onShutdownRequest();
> LOG.info("Shutdown requested. Stopping callback.");
> return;{noformat}
> is a code snippet from AMRMClientAsyncImpl. The corresponding
> onShutdownRequest call for the Distributed Shell App master is,
> {noformat}
> @Override
> public void onShutdownRequest() {
> done = true;
> }{noformat}
> Due to this, the current behavior is that whenever an application attempt
> fails due to a NM restart (where the DS AM is running), an
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8372) ApplicationAttemptNotFoundException should be handled correctly by Distributed Shell App Master
[
https://issues.apache.org/jira/browse/YARN-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8372:
---
Description:
{noformat}
try {
response = client.allocate(progress);
} catch (ApplicationAttemptNotFoundException e) {
handler.onShutdownRequest();
LOG.info("Shutdown requested. Stopping callback.");
return;{noformat}
is a code snippet from AMRMClientAsyncImpl. The corresponding onShutdownRequest
call for the Distributed Shell App master,
{noformat}
@Override
public void onShutdownRequest() {
done = true;
}{noformat}
Due to the above change, the current behavior is that whenever an application
attempt fails due to a NM restart (NM where the DS AM is running), an
ApplicationAttemptNotFoundException is thrown and all containers for that
attempt including the ones that are running on other NMs are killed by the AM
and marked as COMPLETE. The subsequent attempt spawns new containers just like
a new attempt. This behavior is different to a Map Reduce application where the
containers are not killed.
cc [~rohithsharma]
was:
{noformat}
try {
response = client.allocate(progress);
} catch (ApplicationAttemptNotFoundException e) {
handler.onShutdownRequest();
LOG.info("Shutdown requested. Stopping callback.");
return;{noformat}
is a code snippet from AMRMClientAsyncImpl. The corresponding onShutdownRequest
call for the Distributed Shell App master is,
{noformat}
@Override
public void onShutdownRequest() {
done = true;
}{noformat}
Due to this, the current behavior is that whenever an application attempt fails
due to a NM restart (where the DS AM is running), an
> ApplicationAttemptNotFoundException should be handled correctly by
> Distributed Shell App Master
> ---
>
> Key: YARN-8372
> URL: https://issues.apache.org/jira/browse/YARN-8372
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Priority: Major
>
> {noformat}
> try {
> response = client.allocate(progress);
> } catch (ApplicationAttemptNotFoundException e) {
> handler.onShutdownRequest();
> LOG.info("Shutdown requested. Stopping callback.");
> return;{noformat}
> is a code snippet from AMRMClientAsyncImpl. The corresponding
> onShutdownRequest call for the Distributed Shell App master,
> {noformat}
> @Override
> public void onShutdownRequest() {
> done = true;
> }{noformat}
> Due to the above change, the current behavior is that whenever an application
> attempt fails due to a NM restart (NM where the DS AM is running), an
> ApplicationAttemptNotFoundException is thrown and all containers for that
> attempt including the ones that are running on other NMs are killed by the AM
> and marked as COMPLETE. The subsequent attempt spawns new containers just
> like a new attempt. This behavior is different to a Map Reduce application
> where the containers are not killed.
> cc [~rohithsharma]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7894) Improve ATS response for DS_CONTAINER when container launch fails
Charan Hebri created YARN-7894:
--
Summary: Improve ATS response for DS_CONTAINER when container
launch fails
Key: YARN-7894
URL: https://issues.apache.org/jira/browse/YARN-7894
Project: Hadoop YARN
Issue Type: Bug
Components: ATSv2
Reporter: Charan Hebri
When a distributed shell application starts running and a container launch
fails the web service call to the API,
{noformat}
http:///ws/v1/timeline/DS_CONTAINER/{noformat}
return a "Not Found". The message returned in this case should be improved to
signify that a container launch failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7894) Improve ATS response for DS_CONTAINER when container launch fails
[
https://issues.apache.org/jira/browse/YARN-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-7894:
---
Component/s: (was: ATSv2)
timelineserver
> Improve ATS response for DS_CONTAINER when container launch fails
> -
>
> Key: YARN-7894
> URL: https://issues.apache.org/jira/browse/YARN-7894
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Reporter: Charan Hebri
>Priority: Major
>
> When a distributed shell application starts running and a container launch
> fails the web service call to the API,
> {noformat}
> http:// address>/ws/v1/timeline/DS_CONTAINER/{noformat}
> return a "Not Found". The message returned in this case should be improved to
> signify that a container launch failed.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7770) Support for setting application priority for Distributed Shell jobs
[
https://issues.apache.org/jira/browse/YARN-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344902#comment-16344902
]
Charan Hebri commented on YARN-7770:
[~sunilg] yes using *-priority* for setting the priority while submitting a job
works. I am not sure if it has been documented though.
> Support for setting application priority for Distributed Shell jobs
> ---
>
> Key: YARN-7770
> URL: https://issues.apache.org/jira/browse/YARN-7770
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Reporter: Charan Hebri
>Assignee: Sunil G
>Priority: Major
>
> Currently there isn't a way to submit a Distributed Shell job with an
> application priority like how it is done via the property
> {noformat}
> mapred.job.priority{noformat}
> for MapReduce jobs. Creating this issue to track support for the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7937) Fix http method name in Cluster Application Timeout Update API example request
[ https://issues.apache.org/jira/browse/YARN-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-7937: --- Description: In section, Cluster Application Timeout Update API, https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Application_Timeout_Update_API the example requests for both XML and JSON formats show "GET" as the http method. This should actually be a PUT. was:In section, Cluster Application Timeout Update API, the example requests for both XML and JSON formats show "GET" as the http method. This should actually be a PUT. > Fix http method name in Cluster Application Timeout Update API example request > -- > > Key: YARN-7937 > URL: https://issues.apache.org/jira/browse/YARN-7937 > Project: Hadoop YARN > Issue Type: Bug > Components: docs, documentation >Affects Versions: 2.9.0, 3.0.0 >Reporter: Charan Hebri >Priority: Minor > > In section, Cluster Application Timeout Update API, > https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Application_Timeout_Update_API > the example requests for both XML and JSON formats show "GET" as the http > method. This should actually be a PUT. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7937) Fix http method name in Cluster Application Timeout Update API example request
Charan Hebri created YARN-7937: -- Summary: Fix http method name in Cluster Application Timeout Update API example request Key: YARN-7937 URL: https://issues.apache.org/jira/browse/YARN-7937 Project: Hadoop YARN Issue Type: Bug Components: docs, documentation Affects Versions: 3.0.0, 2.9.0 Reporter: Charan Hebri In section, Cluster Application Timeout Update API, the example requests for both XML and JSON formats show "GET" as the http method. This should actually be a PUT. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7937) Fix http method name in Cluster Application Timeout Update API example request
[ https://issues.apache.org/jira/browse/YARN-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-7937: --- Attachment: YARN-7937.001.patch > Fix http method name in Cluster Application Timeout Update API example request > -- > > Key: YARN-7937 > URL: https://issues.apache.org/jira/browse/YARN-7937 > Project: Hadoop YARN > Issue Type: Bug > Components: docs, documentation >Affects Versions: 2.9.0, 3.0.0 >Reporter: Charan Hebri >Priority: Minor > Attachments: YARN-7937.001.patch > > > In section, Cluster Application Timeout Update API, > https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Application_Timeout_Update_API > the example requests for both XML and JSON formats show "GET" as the http > method. This should actually be a PUT. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7937) Fix http method name in Cluster Application Timeout Update API example request
[ https://issues.apache.org/jira/browse/YARN-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri reassigned YARN-7937: -- Assignee: Charan Hebri > Fix http method name in Cluster Application Timeout Update API example request > -- > > Key: YARN-7937 > URL: https://issues.apache.org/jira/browse/YARN-7937 > Project: Hadoop YARN > Issue Type: Bug > Components: docs, documentation >Affects Versions: 2.9.0, 3.0.0 >Reporter: Charan Hebri >Assignee: Charan Hebri >Priority: Minor > Attachments: YARN-7937.001.patch > > > In section, Cluster Application Timeout Update API, > https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Application_Timeout_Update_API > the example requests for both XML and JSON formats show "GET" as the http > method. This should actually be a PUT. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7692) Resource Manager goes down when a user not included in a priority acl submits a job
Charan Hebri created YARN-7692:
--
Summary: Resource Manager goes down when a user not included in a
priority acl submits a job
Key: YARN-7692
URL: https://issues.apache.org/jira/browse/YARN-7692
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 3.0.0
Reporter: Charan Hebri
Test scenario
--
1. A cluster is created, no ACLs are included
2. Submit jobs with an existing user say 'user_a'
3. Enable ACLs and create a priority ACL entry via the property
yarn.scheduler.capacity.priority-acls. Do not include the user, 'user_a' in
this ACL.
4. Submit a job with the 'user_a'
The observed behavior in this case is that the job is rejected as 'user_a' does
not have the permission to run the job which is expected behavior. But Resource
Manager also goes down when it tries to recover previous applications and fails
to recover them.
Below is the exception seen,
{noformat}
2017-12-27 10:52:30,064 INFO conf.Configuration
(Configuration.java:getConfResourceAsInputStream(2659)) - found resource
yarn-site.xml at file:/etc/hadoop/3.0.0.0-636/0/yarn-site.xml
2017-12-27 10:52:30,065 INFO scheduler.AbstractYarnScheduler
(AbstractYarnScheduler.java:setClusterMaxPriority(911)) - Updated the cluste
max priority to maxClusterLevelAppPriority = 10
2017-12-27 10:52:30,066 INFO resourcemanager.ResourceManager
(ResourceManager.java:transitionToActive(1177)) - Transitioning to active state
2017-12-27 10:52:30,097 INFO resourcemanager.ResourceManager
(ResourceManager.java:serviceStart(765)) - Recovery started
2017-12-27 10:52:30,102 INFO recovery.RMStateStore
(RMStateStore.java:checkVersion(747)) - Loaded RM state version info 1.5
2017-12-27 10:52:30,375 INFO security.RMDelegationTokenSecretManager
(RMDelegationTokenSecretManager.java:recover(196)) - recovering
RMDelegationTokenSecretManager.
2017-12-27 10:52:30,380 INFO resourcemanager.RMAppManager
(RMAppManager.java:recover(561)) - Recovering 51 applications
2017-12-27 10:52:30,432 INFO resourcemanager.RMAppManager
(RMAppManager.java:recover(571)) - Successfully recovered 0 out of 51
applications
2017-12-27 10:52:30,432 ERROR resourcemanager.ResourceManager
(ResourceManager.java:serviceStart(776)) - Failed to load/recover state
org.apache.hadoop.yarn.exceptions.YarnException:
org.apache.hadoop.security.AccessControlException: User hrt_qa (auth:SIMPLE)
does not have permission to submit/update application_1514268754125_0001 for 0
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.checkAndGetApplicationPriority(CapacityScheduler.java:2348)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:396)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recoverApplication(RMAppManager.java:358)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recover(RMAppManager.java:567)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.recover(ResourceManager.java:1390)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:771)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:1143)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1183)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1179)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1179)
at
org.apache.hadoop.yarn.server.resourcemanager.AdminService.transitionToActive(AdminService.java:320)
at
org.apache.hadoop.yarn.server.resourcemanager.ActiveStandbyElectorBasedElectorService.becomeActive(ActiveStandbyElectorBasedElectorService.java:144)
at
org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:894)
at
org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:473)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:611)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)
Caused by: org.apache.hadoop.security.AccessControlException: User hrt_qa
(auth:SIMPLE) does not have permission to submit/update
application_1514268754125_0001 for 0
... 20 more
2017-12-27 10:52:30,434 INFO service.AbstractService
[jira] [Commented] (YARN-8036) Memory Available shows a negative value after running updateNodeResource
[ https://issues.apache.org/jira/browse/YARN-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552493#comment-16552493 ] Charan Hebri commented on YARN-8036: [~Zian Chen] This may have been fixed by YARN-8464. I haven't seen this issue recently. > Memory Available shows a negative value after running updateNodeResource > > > Key: YARN-8036 > URL: https://issues.apache.org/jira/browse/YARN-8036 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Assignee: Zian Chen >Priority: Major > > Running updateNodeResource for a node that already has applications running > on it doesn't update Memory Available with the right values. It may end up > showing negative values based on the requirements of the application. > Attached a screenshot for reference. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7986) ATSv2 REST API queries do not return results for uppercase application tags
[ https://issues.apache.org/jira/browse/YARN-7986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri reassigned YARN-7986: -- Assignee: Charan Hebri > ATSv2 REST API queries do not return results for uppercase application tags > --- > > Key: YARN-7986 > URL: https://issues.apache.org/jira/browse/YARN-7986 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Assignee: Charan Hebri >Priority: Critical > > When applications are submitted to YARN with application tags, the tags are > converted to lowercase. This can be seen on the old/new UI. But using the > original tags for ATSv2 REST API queries do not return results as they expect > the query url to have the tags in lowercase. > This is additional work for the client because each tag needs to be > lowercased before running a query. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7986) ATSv2 REST API queries do not return results for uppercase application tags
[ https://issues.apache.org/jira/browse/YARN-7986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389056#comment-16389056 ] Charan Hebri commented on YARN-7986: [~rohithsharma] yes will assign to myself. > ATSv2 REST API queries do not return results for uppercase application tags > --- > > Key: YARN-7986 > URL: https://issues.apache.org/jira/browse/YARN-7986 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Priority: Critical > > When applications are submitted to YARN with application tags, the tags are > converted to lowercase. This can be seen on the old/new UI. But using the > original tags for ATSv2 REST API queries do not return results as they expect > the query url to have the tags in lowercase. > This is additional work for the client because each tag needs to be > lowercased before running a query. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8107) NPE when incorrect format is used in ATSv2 filter attributes
[
https://issues.apache.org/jira/browse/YARN-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16425583#comment-16425583
]
Charan Hebri commented on YARN-8107:
[~rohithsharma] the query request is in the log attached above,
{noformat}
/ws/v2/timeline/users/hrt_qa/flows/flow4/runs/1/apps?infofilters=UIDeq{noformat}
Here the attribute value 'UIDeq' doesn't follow the convention of
used for filter attributes. The NPE isn't specifically for the
apps for a flow run API though. It can be seen in other APIs where filters are
supported.
> NPE when incorrect format is used in ATSv2 filter attributes
>
>
> Key: YARN-8107
> URL: https://issues.apache.org/jira/browse/YARN-8107
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Reporter: Charan Hebri
>Assignee: Rohith Sharma K S
>Priority: Major
>
> Using an incorrect format for infofilters, conffilters and metricfilters
> throws a NPE with no clear message to the caller. This should be tagged as a
> 400 Bad Request with an informative message. Below is the timeline reader log.
> {noformat}
> 2018-04-02 06:44:10,451 INFO reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(173)) - Processed URL
> /ws/v2/timeline/users/hrt_qa/flows/flow4/runs/1/apps?infofilters=UIDeq but
> encountered exception (Took 0 ms.)
> 2018-04-02 06:44:10,451 ERROR reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(188)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterUtils.createHBaseFilterList(TimelineFilterUtils.java:276)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.ApplicationEntityReader.constructFilterListBasedOnFilters(ApplicationEntityReader.java:126)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.createFilterList(TimelineEntityReader.java:157)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:277)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntities(HBaseTimelineReaderImpl.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntities(TimelineReaderManager.java:143)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getEntities(TimelineReaderWebServices.java:605)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRunApps(TimelineReaderWebServices.java:1962)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> at
>
[jira] [Assigned] (YARN-7627) [ATSv2] When passing a non-number as metricslimit, the error message is wrong
[
https://issues.apache.org/jira/browse/YARN-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri reassigned YARN-7627:
--
Assignee: Charan Hebri
> [ATSv2] When passing a non-number as metricslimit, the error message is wrong
> -
>
> Key: YARN-7627
> URL: https://issues.apache.org/jira/browse/YARN-7627
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.0.0-beta1
>Reporter: Grant Sohn
>Assignee: Charan Hebri
>Priority: Trivial
>
> curl
> "$ATS_URL/ws/v2/timeline/apps/application_1512430070811_0022/entities/MAPREDUCE_JOB?metricslimit=w"
> returns:
> {"exception":"BadRequestException","message":"java.lang.Exception:
> createdTime start/end or limit or flowrunid is not a numeric
> value.","javaClassName":"org.apache.hadoop.yarn.webapp.BadRequestException"}
> and:
> curl
> "$ATS_URL/ws/v2/timeline/apps/application_1512430070811_0022?metricslimit=ALL"
> {"exception":"BadRequestException","message":"java.lang.Exception: flowrunid
> is not a numeric
> value.","javaClassName":"org.apache.hadoop.yarn.webapp.BadRequestException"}
> This could be part of YARN-6389 which indicates this functionality was not
> completed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-8156: --- Attachment: YARN-8156.001.patch > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > Attachments: YARN-8156.001.patch > > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri reassigned YARN-8156: -- Assignee: Charan Hebri > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8138) No containers pre-empted from another queue when using node labels
Charan Hebri created YARN-8138:
--
Summary: No containers pre-empted from another queue when using
node labels
Key: YARN-8138
URL: https://issues.apache.org/jira/browse/YARN-8138
Project: Hadoop YARN
Issue Type: Bug
Reporter: Charan Hebri
There seems to be an issue with pre-emption when using node labels with queue
priority.
Test configuration:
queue A (capacity=50, priority=1)
queue B (capacity=50, priority=2)
both have accessible-node-labels set to x
A.accessible-node-labels.x.capacity = 50
B.accessible-node-labels.x.capacity = 50
Along with this pre-emption related properties have been set.
Test steps:
- Set NM memory = 6000MB and containerMemory = 750MB
- Submit an application A1 to B, with am-container = container =
(6000-750-1500), no. of containers = 2
- Submit an application A2 to A, with am-container = 750, container = 1500, no
of containers = (NUM_NM-1)
- Kill application A1
- Submit an application A3 to B with am-container=container=5000, no. of
containers=3
- Expectation is that containers are pre-empted from application A2 to A3 but
there is no container pre-emption happening
Container pre-emption is stuck with the message in the RM log,
{noformat}
2018-02-02 11:41:36,974 INFO capacity.CapacityScheduler
(CapacityScheduler.java:tryCommit(2673)) - Allocation proposal accepted
2018-02-02 11:41:36,984 INFO capacity.CapacityScheduler
(CapacityScheduler.java:allocateContainerOnSingleNode(1391)) - Trying to
fulfill reservation for application application_1517571510094_0003 on node:
XX:25454
2018-02-02 11:41:36,984 INFO allocator.AbstractContainerAllocator
(AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(97)) -
Reserved container application=application_1517571510094_0003
resource=
queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@3f04848e
cluster=
2018-02-02 11:41:36,984 INFO capacity.CapacityScheduler
(CapacityScheduler.java:tryCommit(2673)) - Allocation proposal accepted
2018-02-02 11:41:36,984 INFO capacity.CapacityScheduler
(CapacityScheduler.java:allocateContainerOnSingleNode(1391)) - Trying to
fulfill reservation for application application_1517571510094_0003 on node:
XX:25454
2018-02-02 11:41:36,984 INFO allocator.AbstractContainerAllocator
(AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(97)) -
Reserved container application=application_1517571510094_0003
resource=
queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@3f04848e
cluster=
2018-02-02 11:41:36,984 INFO capacity.CapacityScheduler
(CapacityScheduler.java:tryCommit(2673)) - Allocation proposal accepted
2018-02-02 11:41:36,994 INFO capacity.CapacityScheduler
(CapacityScheduler.java:allocateContainerOnSingleNode(1391)) - Trying to
fulfill reservation for application application_1517571510094_0003 on node:
XX:25454
2018-02-02 11:41:36,995 INFO allocator.AbstractContainerAllocator
(AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(97)) -
Reserved container application=application_1517571510094_0003
resource=
queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@3f04848e
cluster={noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8138) No containers pre-empted from another queue when using node labels
[
https://issues.apache.org/jira/browse/YARN-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri reassigned YARN-8138:
--
Assignee: Zian Chen
> No containers pre-empted from another queue when using node labels
> --
>
> Key: YARN-8138
> URL: https://issues.apache.org/jira/browse/YARN-8138
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Charan Hebri
>Assignee: Zian Chen
>Priority: Blocker
>
> There seems to be an issue with pre-emption when using node labels with queue
> priority.
> Test configuration:
> queue A (capacity=50, priority=1)
> queue B (capacity=50, priority=2)
> both have accessible-node-labels set to x
> A.accessible-node-labels.x.capacity = 50
> B.accessible-node-labels.x.capacity = 50
> Along with this pre-emption related properties have been set.
> Test steps:
> - Set NM memory = 6000MB and containerMemory = 750MB
> - Submit an application A1 to B, with am-container = container =
> (6000-750-1500), no. of containers = 2
> - Submit an application A2 to A, with am-container = 750, container = 1500,
> no of containers = (NUM_NM-1)
> - Kill application A1
> - Submit an application A3 to B with am-container=container=5000, no. of
> containers=3
> - Expectation is that containers are pre-empted from application A2 to A3
> but there is no container pre-emption happening
> Container pre-emption is stuck with the message in the RM log,
> {noformat}
> 2018-02-02 11:41:36,974 INFO capacity.CapacityScheduler
> (CapacityScheduler.java:tryCommit(2673)) - Allocation proposal accepted
> 2018-02-02 11:41:36,984 INFO capacity.CapacityScheduler
> (CapacityScheduler.java:allocateContainerOnSingleNode(1391)) - Trying to
> fulfill reservation for application application_1517571510094_0003 on node:
> XX:25454
> 2018-02-02 11:41:36,984 INFO allocator.AbstractContainerAllocator
> (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(97)) -
> Reserved container application=application_1517571510094_0003
> resource=
> queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@3f04848e
> cluster=
> 2018-02-02 11:41:36,984 INFO capacity.CapacityScheduler
> (CapacityScheduler.java:tryCommit(2673)) - Allocation proposal accepted
> 2018-02-02 11:41:36,984 INFO capacity.CapacityScheduler
> (CapacityScheduler.java:allocateContainerOnSingleNode(1391)) - Trying to
> fulfill reservation for application application_1517571510094_0003 on node:
> XX:25454
> 2018-02-02 11:41:36,984 INFO allocator.AbstractContainerAllocator
> (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(97)) -
> Reserved container application=application_1517571510094_0003
> resource=
> queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@3f04848e
> cluster=
> 2018-02-02 11:41:36,984 INFO capacity.CapacityScheduler
> (CapacityScheduler.java:tryCommit(2673)) - Allocation proposal accepted
> 2018-02-02 11:41:36,994 INFO capacity.CapacityScheduler
> (CapacityScheduler.java:allocateContainerOnSingleNode(1391)) - Trying to
> fulfill reservation for application application_1517571510094_0003 on node:
> XX:25454
> 2018-02-02 11:41:36,995 INFO allocator.AbstractContainerAllocator
> (AbstractContainerAllocator.java:getCSAssignmentFromAllocateResult(97)) -
> Reserved container application=application_1517571510094_0003
> resource=
> queue=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator@3f04848e
> cluster={noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8129) Improve error message for invalid value in fields attributes
Charan Hebri created YARN-8129:
--
Summary: Improve error message for invalid value in fields
attributes
Key: YARN-8129
URL: https://issues.apache.org/jira/browse/YARN-8129
Project: Hadoop YARN
Issue Type: Improvement
Components: ATSv2
Reporter: Charan Hebri
Query with invalid values for the 'fields' attributes throws a message that
isn't very informative.
Reader log,
{noformat}
2018-04-09 08:59:46,069 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:getEntities(595)) - Received URL
/ws/v2/timeline/users/hrt_qa/flows/test_flow/apps?limit=3=INFOS from
user hrt_qa
2018-04-09 08:59:46,070 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:handleException(173)) - Processed URL
/ws/v2/timeline/users/hrt_qa/flows/test_flow/apps?limit=3=INFOS but
encountered exception (Took 1 ms.){noformat}
Here INFOS is the invalid value for the fields attribute.
Response,
{noformat}
{
"exception": "BadRequestException",
"message": "java.lang.Exception: No enum constant
org.apache.hadoop.yarn.server.timelineservice.storage.TimelineReader.Field.INFOS",
"javaClassName": "org.apache.hadoop.yarn.webapp.BadRequestException"
}{noformat}
The message shouldn't ideally contain the enum information.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8129) Improve error message for invalid value in fields attribute
[
https://issues.apache.org/jira/browse/YARN-8129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8129:
---
Summary: Improve error message for invalid value in fields attribute (was:
Improve error message for invalid value in fields attributes)
> Improve error message for invalid value in fields attribute
> ---
>
> Key: YARN-8129
> URL: https://issues.apache.org/jira/browse/YARN-8129
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: ATSv2
>Reporter: Charan Hebri
>Priority: Minor
>
> Query with invalid values for the 'fields' attributes throws a message that
> isn't very informative.
> Reader log,
> {noformat}
> 2018-04-09 08:59:46,069 INFO reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:getEntities(595)) - Received URL
> /ws/v2/timeline/users/hrt_qa/flows/test_flow/apps?limit=3=INFOS from
> user hrt_qa
> 2018-04-09 08:59:46,070 INFO reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(173)) - Processed URL
> /ws/v2/timeline/users/hrt_qa/flows/test_flow/apps?limit=3=INFOS but
> encountered exception (Took 1 ms.){noformat}
> Here INFOS is the invalid value for the fields attribute.
> Response,
> {noformat}
> {
> "exception": "BadRequestException",
> "message": "java.lang.Exception: No enum constant
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineReader.Field.INFOS",
> "javaClassName": "org.apache.hadoop.yarn.webapp.BadRequestException"
> }{noformat}
> The message shouldn't ideally contain the enum information.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8130) Race condition when container events are published for KILLED applications
[
https://issues.apache.org/jira/browse/YARN-8130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8130:
---
Description:
There seems to be a race condition happening when an application is KILLED and
the corresponding container event information is being published. For completed
containers, a YARN_CONTAINER_FINISHED event is generated but for some
containers in a KILLED application this information is missing. Below is a node
manager log snippet,
{code:java}
2018-04-09 08:44:54,474 INFO shuffle.ExternalShuffleBlockResolver
(ExternalShuffleBlockResolver.java:applicationRemoved(186)) - Application
application_1523259757659_0003 removed, cleanupLocalDirs = false
2018-04-09 08:44:54,478 INFO application.ApplicationImpl
(ApplicationImpl.java:handle(632)) - Application application_1523259757659_0003
transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED
2018-04-09 08:44:54,478 ERROR timelineservice.NMTimelinePublisher
(NMTimelinePublisher.java:putEntity(298)) - Seems like client has been removed
before the entity could be published for TimelineEntity[type='YARN_CONTAINER',
id='container_1523259757659_0003_01_02']
2018-04-09 08:44:54,478 INFO logaggregation.AppLogAggregatorImpl
(AppLogAggregatorImpl.java:finishLogAggregation(520)) - Application just
finished : application_1523259757659_0003
2018-04-09 08:44:54,488 INFO logaggregation.AppLogAggregatorImpl
(AppLogAggregatorImpl.java:doContainerLogAggregation(576)) - Uploading logs for
container container_1523259757659_0003_01_01. Current good log dirs are
/grid/0/hadoop/yarn/log
2018-04-09 08:44:54,492 INFO logaggregation.AppLogAggregatorImpl
(AppLogAggregatorImpl.java:doContainerLogAggregation(576)) - Uploading logs for
container container_1523259757659_0003_01_02. Current good log dirs are
/grid/0/hadoop/yarn/log
2018-04-09 08:44:55,470 INFO collector.TimelineCollectorManager
(TimelineCollectorManager.java:remove(192)) - The collector service for
application_1523259757659_0003 was removed
2018-04-09 08:44:55,472 INFO containermanager.ContainerManagerImpl
(ContainerManagerImpl.java:handle(1572)) - couldn't find application
application_1523259757659_0003 while processing FINISH_APPS event. The
ResourceManager allocated resources for this application to the NodeManager but
no active containers were found to process{code}
The container id specified in the log, *container_1523259757659_0003_01_02*
is the one that has the finished event missing.
was:
There seems to be a race condition happening when an application is KILLED and
the corresponding container event information is being published. For completed
containers, a YARN_CONTAINER_FINISHED event is generated but for some
containers in a KILLED application this information is missing. Below is a node
manager log snippet,
{code:java}
2018-04-09 08:44:54,474 INFO shuffle.ExternalShuffleBlockResolver
(ExternalShuffleBlockResolver.java:applicationRemoved(186)) - Application
application_1523259757659_0003 removed, cleanupLocalDirs = false
2018-04-09 08:44:54,478 INFO application.ApplicationImpl
(ApplicationImpl.java:handle(632)) - Application application_1523259757659_0003
transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED
2018-04-09 08:44:54,478 ERROR timelineservice.NMTimelinePublisher
(NMTimelinePublisher.java:putEntity(298)) - Seems like client has been removed
before the entity could be published for TimelineEntity[type='YARN_CONTAINER',
id='container_1523259757659_0003_01_02']
2018-04-09 08:44:54,478 INFO logaggregation.AppLogAggregatorImpl
(AppLogAggregatorImpl.java:finishLogAggregation(520)) - Application just
finished : application_1523259757659_0003
2018-04-09 08:44:54,488 INFO logaggregation.AppLogAggregatorImpl
(AppLogAggregatorImpl.java:doContainerLogAggregation(576)) - Uploading logs for
container container_1523259757659_0003_01_01. Current good log dirs are
/grid/0/hadoop/yarn/log
2018-04-09 08:44:54,492 INFO logaggregation.AppLogAggregatorImpl
(AppLogAggregatorImpl.java:doContainerLogAggregation(576)) - Uploading logs for
container container_1523259757659_0003_01_02. Current good log dirs are
/grid/0/hadoop/yarn/log
2018-04-09 08:44:55,470 INFO collector.TimelineCollectorManager
(TimelineCollectorManager.java:remove(192)) - The collector service for
application_1523259757659_0003 was removed
2018-04-09 08:44:55,472 INFO containermanager.ContainerManagerImpl
(ContainerManagerImpl.java:handle(1572)) - couldn't find application
application_1523259757659_0003 while processing FINISH_APPS event. The
ResourceManager allocated resources for this application to the NodeManager but
no active containers were found to process{code}
The container id specified in the log, *container_1523259757659_0003_01_02*
is the one that has the finished event missing.
> Race condition when container events are
[jira] [Commented] (YARN-8130) Race condition when container events are published for KILLED applications
[
https://issues.apache.org/jira/browse/YARN-8130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430341#comment-16430341
]
Charan Hebri commented on YARN-8130:
cc [~rohithsharma]
> Race condition when container events are published for KILLED applications
> --
>
> Key: YARN-8130
> URL: https://issues.apache.org/jira/browse/YARN-8130
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Reporter: Charan Hebri
>Priority: Major
>
> There seems to be a race condition happening when an application is KILLED
> and the corresponding container event information is being published. For
> completed containers, a YARN_CONTAINER_FINISHED event is generated but for
> some containers in a KILLED application this information is missing. Below is
> a node manager log snippet,
> {code:java}
> 2018-04-09 08:44:54,474 INFO shuffle.ExternalShuffleBlockResolver
> (ExternalShuffleBlockResolver.java:applicationRemoved(186)) - Application
> application_1523259757659_0003 removed, cleanupLocalDirs = false
> 2018-04-09 08:44:54,478 INFO application.ApplicationImpl
> (ApplicationImpl.java:handle(632)) - Application
> application_1523259757659_0003 transitioned from
> APPLICATION_RESOURCES_CLEANINGUP to FINISHED
> 2018-04-09 08:44:54,478 ERROR timelineservice.NMTimelinePublisher
> (NMTimelinePublisher.java:putEntity(298)) - Seems like client has been
> removed before the entity could be published for
> TimelineEntity[type='YARN_CONTAINER',
> id='container_1523259757659_0003_01_02']
> 2018-04-09 08:44:54,478 INFO logaggregation.AppLogAggregatorImpl
> (AppLogAggregatorImpl.java:finishLogAggregation(520)) - Application just
> finished : application_1523259757659_0003
> 2018-04-09 08:44:54,488 INFO logaggregation.AppLogAggregatorImpl
> (AppLogAggregatorImpl.java:doContainerLogAggregation(576)) - Uploading logs
> for container container_1523259757659_0003_01_01. Current good log dirs
> are /grid/0/hadoop/yarn/log
> 2018-04-09 08:44:54,492 INFO logaggregation.AppLogAggregatorImpl
> (AppLogAggregatorImpl.java:doContainerLogAggregation(576)) - Uploading logs
> for container container_1523259757659_0003_01_02. Current good log dirs
> are /grid/0/hadoop/yarn/log
> 2018-04-09 08:44:55,470 INFO collector.TimelineCollectorManager
> (TimelineCollectorManager.java:remove(192)) - The collector service for
> application_1523259757659_0003 was removed
> 2018-04-09 08:44:55,472 INFO containermanager.ContainerManagerImpl
> (ContainerManagerImpl.java:handle(1572)) - couldn't find application
> application_1523259757659_0003 while processing FINISH_APPS event. The
> ResourceManager allocated resources for this application to the NodeManager
> but no active containers were found to process{code}
> The container id specified in the log,
> *container_1523259757659_0003_01_02* is the one that has the finished
> event missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8130) Race condition when container events are published for KILLED applications
Charan Hebri created YARN-8130:
--
Summary: Race condition when container events are published for
KILLED applications
Key: YARN-8130
URL: https://issues.apache.org/jira/browse/YARN-8130
Project: Hadoop YARN
Issue Type: Bug
Components: ATSv2
Reporter: Charan Hebri
There seems to be a race condition happening when an application is KILLED and
the corresponding container event information is being published. For completed
containers, a YARN_CONTAINER_FINISHED event is generated but for some
containers in a KILLED application this information is missing. Below is a node
manager log snippet,
{code:java}
2018-04-09 08:44:54,474 INFO shuffle.ExternalShuffleBlockResolver
(ExternalShuffleBlockResolver.java:applicationRemoved(186)) - Application
application_1523259757659_0003 removed, cleanupLocalDirs = false
2018-04-09 08:44:54,478 INFO application.ApplicationImpl
(ApplicationImpl.java:handle(632)) - Application application_1523259757659_0003
transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED
2018-04-09 08:44:54,478 ERROR timelineservice.NMTimelinePublisher
(NMTimelinePublisher.java:putEntity(298)) - Seems like client has been removed
before the entity could be published for TimelineEntity[type='YARN_CONTAINER',
id='container_1523259757659_0003_01_02']
2018-04-09 08:44:54,478 INFO logaggregation.AppLogAggregatorImpl
(AppLogAggregatorImpl.java:finishLogAggregation(520)) - Application just
finished : application_1523259757659_0003
2018-04-09 08:44:54,488 INFO logaggregation.AppLogAggregatorImpl
(AppLogAggregatorImpl.java:doContainerLogAggregation(576)) - Uploading logs for
container container_1523259757659_0003_01_01. Current good log dirs are
/grid/0/hadoop/yarn/log
2018-04-09 08:44:54,492 INFO logaggregation.AppLogAggregatorImpl
(AppLogAggregatorImpl.java:doContainerLogAggregation(576)) - Uploading logs for
container container_1523259757659_0003_01_02. Current good log dirs are
/grid/0/hadoop/yarn/log
2018-04-09 08:44:55,470 INFO collector.TimelineCollectorManager
(TimelineCollectorManager.java:remove(192)) - The collector service for
application_1523259757659_0003 was removed
2018-04-09 08:44:55,472 INFO containermanager.ContainerManagerImpl
(ContainerManagerImpl.java:handle(1572)) - couldn't find application
application_1523259757659_0003 while processing FINISH_APPS event. The
ResourceManager allocated resources for this application to the NodeManager but
no active containers were found to process{code}
The container id specified in the log, *container_1523259757659_0003_01_02*
is the one that has the finished event missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8132) Final Status of applications shown as UNDEFINED in ATS app queries
[
https://issues.apache.org/jira/browse/YARN-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8132:
---
Description:
Final Status is shown as UNDEFINED for applications that are KILLED/FAILED. A
sample request/response with INFO field for an application,
{noformat}
2018-04-09 13:10:02,126 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:getApp(1693)) - Received URL
/ws/v2/timeline/apps/application_1523259757659_0003?fields=INFO from user hrt_qa
2018-04-09 13:10:02,156 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:getApp(1716)) - Processed URL
/ws/v2/timeline/apps/application_1523259757659_0003?fields=INFO (Took 30
ms.){noformat}
{noformat}
{
"metrics": [],
"events": [],
"createdtime": 1523263360719,
"idprefix": 0,
"id": "application_1523259757659_0003",
"type": "YARN_APPLICATION",
"info": {
"YARN_APPLICATION_CALLER_CONTEXT": "CLI",
"YARN_APPLICATION_DIAGNOSTICS_INFO": "Application
application_1523259757659_0003 was killed by user xxx_xx at XXX.XXX.XXX.XXX",
"YARN_APPLICATION_FINAL_STATUS": "UNDEFINED",
"YARN_APPLICATION_NAME": "Sleep job",
"YARN_APPLICATION_USER": "hrt_qa",
"YARN_APPLICATION_UNMANAGED_APPLICATION": false,
"FROM_ID":
"yarn-cluster!hrt_qa!test_flow!1523263360719!application_1523259757659_0003",
"UID": "yarn-cluster!application_1523259757659_0003",
"YARN_APPLICATION_VIEW_ACLS": " ",
"YARN_APPLICATION_SUBMITTED_TIME": 1523263360718,
"YARN_AM_CONTAINER_LAUNCH_COMMAND": [
"$JAVA_HOME/bin/java -Djava.io.tmpdir=$PWD/tmp
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0
-Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog
-Dhdp.version=3.0.0.0-1163 -Xmx819m -Dhdp.version=3.0.0.0-1163
org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout
2>/stderr "
],
"YARN_APPLICATION_QUEUE": "default",
"YARN_APPLICATION_TYPE": "MAPREDUCE",
"YARN_APPLICATION_PRIORITY": 0,
"YARN_APPLICATION_LATEST_APP_ATTEMPT":
"appattempt_1523259757659_0003_01",
"YARN_APPLICATION_TAGS": [
"timeline_flow_name_tag:test_flow"
],
"YARN_APPLICATION_STATE": "KILLED"
},
"configs": {},
"isrelatedto": {},
"relatesto": {}
}{noformat}
This is different to what the Resource Manager reports. For KILLED applications
the final status is KILLED and for FAILED applications it is FAILED. This
behavior is seen in ATSv2 as well as older versions of ATS.
was:
Final Status is shown as UNDEFINED for applications that are KILLED/FAILED. A
sample request/response with INFO field for an application,
{noformat}
2018-04-09 13:10:02,126 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:getApp(1693)) - Received URL
/ws/v2/timeline/apps/application_1523259757659_0003?fields=INFO from user hrt_qa
2018-04-09 13:10:02,156 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:getApp(1716)) - Processed URL
/ws/v2/timeline/apps/application_1523259757659_0003?fields=INFO (Took 30
ms.){noformat}
{noformat}
{
"metrics": [],
"events": [],
"createdtime": 1523263360719,
"idprefix": 0,
"id": "application_1523259757659_0003",
"type": "YARN_APPLICATION",
"info": {
"YARN_APPLICATION_CALLER_CONTEXT": "CLI",
"YARN_APPLICATION_DIAGNOSTICS_INFO": "Application
application_1523259757659_0003 was killed by user xxx_xx at XXX.XXX.XXX.XXX",
"YARN_APPLICATION_FINAL_STATUS": "UNDEFINED",
"YARN_APPLICATION_NAME": "Sleep job",
"YARN_APPLICATION_USER": "hrt_qa",
"YARN_APPLICATION_UNMANAGED_APPLICATION": false,
"FROM_ID":
"yarn-cluster!hrt_qa!test_flow!1523263360719!application_1523259757659_0003",
"UID": "yarn-cluster!application_1523259757659_0003",
"YARN_APPLICATION_VIEW_ACLS": " ",
"YARN_APPLICATION_SUBMITTED_TIME": 1523263360718,
"YARN_AM_CONTAINER_LAUNCH_COMMAND": [
"$JAVA_HOME/bin/java -Djava.io.tmpdir=$PWD/tmp
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0
-Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog
-Dhdp.version=3.0.0.0-1163 -Xmx819m -Dhdp.version=3.0.0.0-1163
org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout
2>/stderr "
],
"YARN_APPLICATION_QUEUE": "default",
"YARN_APPLICATION_TYPE": "MAPREDUCE",
"YARN_APPLICATION_PRIORITY": 0,
"YARN_APPLICATION_LATEST_APP_ATTEMPT":
"appattempt_1523259757659_0003_01",
"YARN_APPLICATION_TAGS": [
"timeline_flow_name_tag:test_flow"
],
"YARN_APPLICATION_STATE": "KILLED"
},
"configs": {},
"isrelatedto": {},
"relatesto": {}
}{noformat}
This is different to what the Resource Manager reports. For KILLED applications
the final status is KILLED and for FAILED applications it is FAILED. This
behavior is seen in ATSv2 as well as older versions of ATS.
> Final
[jira] [Created] (YARN-8132) Final Status of applications shown as UNDEFINED in ATS app queries
Charan Hebri created YARN-8132:
--
Summary: Final Status of applications shown as UNDEFINED in ATS
app queries
Key: YARN-8132
URL: https://issues.apache.org/jira/browse/YARN-8132
Project: Hadoop YARN
Issue Type: Bug
Components: ATSv2, timelineservice
Reporter: Charan Hebri
Final Status is shown as UNDEFINED for applications that are KILLED/FAILED. A
sample request/response with INFO field for an application,
{noformat}
2018-04-09 13:10:02,126 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:getApp(1693)) - Received URL
/ws/v2/timeline/apps/application_1523259757659_0003?fields=INFO from user hrt_qa
2018-04-09 13:10:02,156 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:getApp(1716)) - Processed URL
/ws/v2/timeline/apps/application_1523259757659_0003?fields=INFO (Took 30
ms.){noformat}
{noformat}
{
"metrics": [],
"events": [],
"createdtime": 1523263360719,
"idprefix": 0,
"id": "application_1523259757659_0003",
"type": "YARN_APPLICATION",
"info": {
"YARN_APPLICATION_CALLER_CONTEXT": "CLI",
"YARN_APPLICATION_DIAGNOSTICS_INFO": "Application
application_1523259757659_0003 was killed by user xxx_xx at XXX.XXX.XXX.XXX",
"YARN_APPLICATION_FINAL_STATUS": "UNDEFINED",
"YARN_APPLICATION_NAME": "Sleep job",
"YARN_APPLICATION_USER": "hrt_qa",
"YARN_APPLICATION_UNMANAGED_APPLICATION": false,
"FROM_ID":
"yarn-cluster!hrt_qa!test_flow!1523263360719!application_1523259757659_0003",
"UID": "yarn-cluster!application_1523259757659_0003",
"YARN_APPLICATION_VIEW_ACLS": " ",
"YARN_APPLICATION_SUBMITTED_TIME": 1523263360718,
"YARN_AM_CONTAINER_LAUNCH_COMMAND": [
"$JAVA_HOME/bin/java -Djava.io.tmpdir=$PWD/tmp
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0
-Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog
-Dhdp.version=3.0.0.0-1163 -Xmx819m -Dhdp.version=3.0.0.0-1163
org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout
2>/stderr "
],
"YARN_APPLICATION_QUEUE": "default",
"YARN_APPLICATION_TYPE": "MAPREDUCE",
"YARN_APPLICATION_PRIORITY": 0,
"YARN_APPLICATION_LATEST_APP_ATTEMPT":
"appattempt_1523259757659_0003_01",
"YARN_APPLICATION_TAGS": [
"timeline_flow_name_tag:test_flow"
],
"YARN_APPLICATION_STATE": "KILLED"
},
"configs": {},
"isrelatedto": {},
"relatesto": {}
}{noformat}
This is different to what the Resource Manager reports. For KILLED applications
the final status is KILLED and for FAILED applications it is FAILED. This
behavior is seen in ATSv2 as well as older versions of ATS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436777#comment-16436777 ] Charan Hebri commented on YARN-8156: [~rohithsharma] attached patch with 60 secs as the default. > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > Attachments: YARN-8156.001.patch, YARN-8156.002.patch > > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-8156: --- Attachment: YARN-8156.002.patch > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > Attachments: YARN-8156.001.patch, YARN-8156.002.patch > > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7627) [ATSv2] When passing a non-number as metricslimit, the error message is wrong
[
https://issues.apache.org/jira/browse/YARN-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-7627:
---
Attachment: YARN-7627.001.patch
> [ATSv2] When passing a non-number as metricslimit, the error message is wrong
> -
>
> Key: YARN-7627
> URL: https://issues.apache.org/jira/browse/YARN-7627
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.0.0-beta1
>Reporter: Grant Sohn
>Assignee: Charan Hebri
>Priority: Trivial
> Attachments: YARN-7627.001.patch
>
>
> curl
> "$ATS_URL/ws/v2/timeline/apps/application_1512430070811_0022/entities/MAPREDUCE_JOB?metricslimit=w"
> returns:
> {"exception":"BadRequestException","message":"java.lang.Exception:
> createdTime start/end or limit or flowrunid is not a numeric
> value.","javaClassName":"org.apache.hadoop.yarn.webapp.BadRequestException"}
> and:
> curl
> "$ATS_URL/ws/v2/timeline/apps/application_1512430070811_0022?metricslimit=ALL"
> {"exception":"BadRequestException","message":"java.lang.Exception: flowrunid
> is not a numeric
> value.","javaClassName":"org.apache.hadoop.yarn.webapp.BadRequestException"}
> This could be part of YARN-6389 which indicates this functionality was not
> completed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8107) NPE when incorrect format is used in ATSv2 filter attributes
[
https://issues.apache.org/jira/browse/YARN-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8107:
---
Description:
Using an incorrect format for infofilters, conffilters and metricfilters throws
a NPE with no clear message to the caller. This is should be tagged as a 400
Bad Request with an informative message. Below is the timeline reader log.
{noformat}
2018-04-02 06:44:10,451 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:handleException(173)) - Processed URL
/ws/v2/timeline/users/hrt_qa/flows/flow4/runs/1/apps?infofilters=UIDeq but
encountered exception (Took 0 ms.)
2018-04-02 06:44:10,451 ERROR reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:handleException(188)) - Error while processing
REST request
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterUtils.createHBaseFilterList(TimelineFilterUtils.java:276)
at
org.apache.hadoop.yarn.server.timelineservice.storage.reader.ApplicationEntityReader.constructFilterListBasedOnFilters(ApplicationEntityReader.java:126)
at
org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.createFilterList(TimelineEntityReader.java:157)
at
org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:277)
at
org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntities(HBaseTimelineReaderImpl.java:87)
at
org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntities(TimelineReaderManager.java:143)
at
org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getEntities(TimelineReaderWebServices.java:605)
at
org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRunApps(TimelineReaderWebServices.java:1962)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
at
org.apache.hadoop.yarn.server.timelineservice.reader.security.TimelineReaderWhitelistAuthorizationFilter.doFilter(TimelineReaderWhitelistAuthorizationFilter.java:85)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1601)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
[jira] [Updated] (YARN-8107) NPE when incorrect format is used in ATSv2 filter attributes
[
https://issues.apache.org/jira/browse/YARN-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8107:
---
Description:
Using an incorrect format for infofilters, conffilters and metricfilters throws
a NPE with no clear message to the caller. This should be tagged as a 400 Bad
Request with an informative message. Below is the timeline reader log.
{noformat}
2018-04-02 06:44:10,451 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:handleException(173)) - Processed URL
/ws/v2/timeline/users/hrt_qa/flows/flow4/runs/1/apps?infofilters=UIDeq but
encountered exception (Took 0 ms.)
2018-04-02 06:44:10,451 ERROR reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:handleException(188)) - Error while processing
REST request
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterUtils.createHBaseFilterList(TimelineFilterUtils.java:276)
at
org.apache.hadoop.yarn.server.timelineservice.storage.reader.ApplicationEntityReader.constructFilterListBasedOnFilters(ApplicationEntityReader.java:126)
at
org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.createFilterList(TimelineEntityReader.java:157)
at
org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:277)
at
org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntities(HBaseTimelineReaderImpl.java:87)
at
org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntities(TimelineReaderManager.java:143)
at
org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getEntities(TimelineReaderWebServices.java:605)
at
org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRunApps(TimelineReaderWebServices.java:1962)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
at
org.apache.hadoop.yarn.server.timelineservice.reader.security.TimelineReaderWhitelistAuthorizationFilter.doFilter(TimelineReaderWhitelistAuthorizationFilter.java:85)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1601)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
[jira] [Created] (YARN-8107) NPE when incorrect format is used in ATSv2 filter attributes
Charan Hebri created YARN-8107:
--
Summary: NPE when incorrect format is used in ATSv2 filter
attributes
Key: YARN-8107
URL: https://issues.apache.org/jira/browse/YARN-8107
Project: Hadoop YARN
Issue Type: Bug
Components: ATSv2
Reporter: Charan Hebri
Using an incorrect format for infofilters, conffilters and metricfilters throws
a NPE with no clear message to the caller. This is should be tagged as a 400
Bad Request with an informative message. Below is the timeline reader log.
{noformat}
2018-04-02 06:44:10,451 INFO reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:handleException(173)) - Processed URL
/ws/v2/timeline/users/hrt_qa/flows/flow4/runs/1/apps?infofilters=UIDeq but
encountered exception (Took 0 ms.)
2018-04-02 06:44:10,451 ERROR reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:handleException(188)) - Error while processing
REST request
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterUtils.createHBaseFilterList(TimelineFilterUtils.java:276)
at
org.apache.hadoop.yarn.server.timelineservice.storage.reader.ApplicationEntityReader.constructFilterListBasedOnFilters(ApplicationEntityReader.java:126)
at
org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.createFilterList(TimelineEntityReader.java:157)
at
org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:277)
at
org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntities(HBaseTimelineReaderImpl.java:87)
at
org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntities(TimelineReaderManager.java:143)
at
org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getEntities(TimelineReaderWebServices.java:605)
at
org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRunApps(TimelineReaderWebServices.java:1962)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
at
org.apache.hadoop.yarn.server.timelineservice.reader.security.TimelineReaderWhitelistAuthorizationFilter.doFilter(TimelineReaderWhitelistAuthorizationFilter.java:85)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98)
at
[jira] [Updated] (YARN-8107) NPE when incorrect format is used in ATSv2 filter attributes
[
https://issues.apache.org/jira/browse/YARN-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-8107:
---
Priority: Major (was: Critical)
> NPE when incorrect format is used in ATSv2 filter attributes
>
>
> Key: YARN-8107
> URL: https://issues.apache.org/jira/browse/YARN-8107
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Reporter: Charan Hebri
>Priority: Major
>
> Using an incorrect format for infofilters, conffilters and metricfilters
> throws a NPE with no clear message to the caller. This should be tagged as a
> 400 Bad Request with an informative message. Below is the timeline reader log.
> {noformat}
> 2018-04-02 06:44:10,451 INFO reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(173)) - Processed URL
> /ws/v2/timeline/users/hrt_qa/flows/flow4/runs/1/apps?infofilters=UIDeq but
> encountered exception (Took 0 ms.)
> 2018-04-02 06:44:10,451 ERROR reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(188)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterUtils.createHBaseFilterList(TimelineFilterUtils.java:276)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.ApplicationEntityReader.constructFilterListBasedOnFilters(ApplicationEntityReader.java:126)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.createFilterList(TimelineEntityReader.java:157)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:277)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntities(HBaseTimelineReaderImpl.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntities(TimelineReaderManager.java:143)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getEntities(TimelineReaderWebServices.java:605)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRunApps(TimelineReaderWebServices.java:1962)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.security.TimelineReaderWhitelistAuthorizationFilter.doFilter(TimelineReaderWhitelistAuthorizationFilter.java:85)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
> at
>
[jira] [Created] (YARN-8036) Memory Available shows a negative value after running updateNodeResource
Charan Hebri created YARN-8036: -- Summary: Memory Available shows a negative value after running updateNodeResource Key: YARN-8036 URL: https://issues.apache.org/jira/browse/YARN-8036 Project: Hadoop YARN Issue Type: Bug Reporter: Charan Hebri Attachments: Memory_Available.jpg Running updateNodeResource for a node that already has applications running on it doesn't update Memory Available with the right values. It may end up showing negative values based on the requirements of the application. Attached a screenshot for reference. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8036) Memory Available shows a negative value after running updateNodeResource
[ https://issues.apache.org/jira/browse/YARN-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-8036: --- Attachment: (was: Memory_Available.jpg) > Memory Available shows a negative value after running updateNodeResource > > > Key: YARN-8036 > URL: https://issues.apache.org/jira/browse/YARN-8036 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Priority: Major > > Running updateNodeResource for a node that already has applications running > on it doesn't update Memory Available with the right values. It may end up > showing negative values based on the requirements of the application. > Attached a screenshot for reference. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7986) ATSv2 REST API queries do not return results for uppercase application tags
[ https://issues.apache.org/jira/browse/YARN-7986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-7986: --- Attachment: YARN-7986.001.patch > ATSv2 REST API queries do not return results for uppercase application tags > --- > > Key: YARN-7986 > URL: https://issues.apache.org/jira/browse/YARN-7986 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Assignee: Charan Hebri >Priority: Critical > Attachments: YARN-7986.001.patch > > > When applications are submitted to YARN with application tags, the tags are > converted to lowercase. This can be seen on the old/new UI. But using the > original tags for ATSv2 REST API queries do not return results as they expect > the query url to have the tags in lowercase. > This is additional work for the client because each tag needs to be > lowercased before running a query. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7986) ATSv2 REST API queries do not return results for uppercase application tags
Charan Hebri created YARN-7986: -- Summary: ATSv2 REST API queries do not return results for uppercase application tags Key: YARN-7986 URL: https://issues.apache.org/jira/browse/YARN-7986 Project: Hadoop YARN Issue Type: Bug Reporter: Charan Hebri When applications are submitted to YARN with application tags, the tags are converted to lowercase. This can be seen on the old/new UI. But using the original tags for ATSv2 REST API queries do not return results as they expect the query url to have the tags in lowercase. This is additional work for the client because each tag needs to be lowercased before running a query. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8853) Application Attempts tab not shown correctly when there are no attempts
Charan Hebri created YARN-8853: -- Summary: Application Attempts tab not shown correctly when there are no attempts Key: YARN-8853 URL: https://issues.apache.org/jira/browse/YARN-8853 Project: Hadoop YARN Issue Type: Bug Reporter: Charan Hebri When there are no attempts registered for an application the 'Application Attempts' tab overlaps the Attempts List tab. Attached screenshot. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8853) Application Attempts tab not shown correctly when there are no attempts
[ https://issues.apache.org/jira/browse/YARN-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-8853: --- Attachment: Application_Attempts.png > Application Attempts tab not shown correctly when there are no attempts > > > Key: YARN-8853 > URL: https://issues.apache.org/jira/browse/YARN-8853 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Priority: Major > Attachments: Application_Attempts.png > > > When there are no attempts registered for an application the 'Application > Attempts' tab overlaps the Attempts List tab. Attached screenshot. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8906) NM hostnames not displayed correctly in Node Heatmap Chart
Charan Hebri created YARN-8906: -- Summary: NM hostnames not displayed correctly in Node Heatmap Chart Key: YARN-8906 URL: https://issues.apache.org/jira/browse/YARN-8906 Project: Hadoop YARN Issue Type: Bug Reporter: Charan Hebri Attachments: Node_Heatmap_Chart.png Hostnames displayed on the Node Heatmap Chart look garbled and are not clearly visible. Attached screenshot. cc [~akhilpb] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8961) [UI2] Flow Run End Time shows 'Invalid date'
[
https://issues.apache.org/jira/browse/YARN-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669690#comment-16669690
]
Charan Hebri commented on YARN-8961:
[~akhilpb] below is the response for flow runs,
{noformat}
[
{
"metrics": [],
"events": [],
"createdtime": 1540969639076,
"idprefix": 0,
"id": "hrt_qa@test_flow/1540969639076",
"type": "YARN_FLOW_RUN",
"info": {
"UID": "yarn-cluster!hrt_qa!test_flow!1540969639076",
"SYSTEM_INFO_FLOW_NAME": "test_flow",
"SYSTEM_INFO_FLOW_RUN_ID": 1540969639076,
"SYSTEM_INFO_USER": "hrt_qa",
"FROM_ID": "yarn-cluster!hrt_qa!test_flow!1540969639076"
},
"isrelatedto": {},
"relatesto": {}
},
{
"metrics": [],
"events": [],
"createdtime": 1540969221139,
"idprefix": 0,
"id": "hrt_qa@test_flow/1540969221139",
"type": "YARN_FLOW_RUN",
"info": {
"UID": "yarn-cluster!hrt_qa!test_flow!1540969221139",
"SYSTEM_INFO_FLOW_RUN_END_TIME": 1540969587649,
"SYSTEM_INFO_FLOW_NAME": "test_flow",
"SYSTEM_INFO_FLOW_RUN_ID": 1540969221139,
"SYSTEM_INFO_USER": "hrt_qa",
"FROM_ID": "yarn-cluster!hrt_qa!test_flow!1540969221139"
},
"isrelatedto": {},
"relatesto": {}
}
]{noformat}
You can see that for a flow that hasn't completed yet,
SYSTEM_INFO_FLOW_RUN_END_TIME is not available in the response.
> [UI2] Flow Run End Time shows 'Invalid date'
>
>
> Key: YARN-8961
> URL: https://issues.apache.org/jira/browse/YARN-8961
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Charan Hebri
>Assignee: Akhil PB
>Priority: Major
> Attachments: Invalid_Date.png
>
>
> End Time for Flow Runs is shown as *Invalid date* for runs that are in
> progress. This should be shown as *N/A* just like how it is shown for 'CPU
> VCores' and 'Memory Used'. Attached relevant screenshot.
> cc [~akhilpb]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8961) [UI2] Flow Run End Time shows 'Invalid date'
Charan Hebri created YARN-8961: -- Summary: [UI2] Flow Run End Time shows 'Invalid date' Key: YARN-8961 URL: https://issues.apache.org/jira/browse/YARN-8961 Project: Hadoop YARN Issue Type: Bug Reporter: Charan Hebri Attachments: Invalid_Date.png End Time for Flow Runs is shown as *Invalid date* for runs that are in progress. This should be shown as *N/A* just like how it is shown for 'CPU VCores' and 'Memory Used'. Attached relevant screenshot. cc [~akhilpb] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9282) Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'
[ https://issues.apache.org/jira/browse/YARN-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-9282: --- Attachment: YARN-9282.001.patch > Typo in javadoc of class LinuxContainerExecutor: > hadoop.security.authetication should be 'authentication' > - > > Key: YARN-9282 > URL: https://issues.apache.org/jira/browse/YARN-9282 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Charan Hebri >Priority: Trivial > Labels: newbie, newbie++ > Attachments: YARN-9282.001.patch > > > There's a type in the javadoc of class LinuxContainerExecutor: > "hadoop.security.authetication" should be "hadoop.security.authetication" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9282) Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'
[ https://issues.apache.org/jira/browse/YARN-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri reassigned YARN-9282: -- Assignee: Charan Hebri > Typo in javadoc of class LinuxContainerExecutor: > hadoop.security.authetication should be 'authentication' > - > > Key: YARN-9282 > URL: https://issues.apache.org/jira/browse/YARN-9282 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Charan Hebri >Priority: Trivial > Labels: newbie, newbie++ > > There's a type in the javadoc of class LinuxContainerExecutor: > "hadoop.security.authetication" should be "hadoop.security.authetication" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8295) [UI2] The "Resource Usage" tab is pointless for finished applications
[ https://issues.apache.org/jira/browse/YARN-8295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-8295: --- Attachment: YARN-8295.001.patch > [UI2] The "Resource Usage" tab is pointless for finished applications > - > > Key: YARN-8295 > URL: https://issues.apache.org/jira/browse/YARN-8295 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn-ui-v2 >Reporter: Gergely Novák >Assignee: Charan Hebri >Priority: Minor > Attachments: YARN-8295.001.patch > > > If the user goes to Applications -> app -> Resource Usage for a finished > application, they get this message: "No resource usage data is available for > this application!". > I think it would be better to hide this tab for finished applications, or at > least add something like "this application is not using any resources because > it is finished" to the message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8295) [UI2] The "Resource Usage" tab is pointless for finished applications
[ https://issues.apache.org/jira/browse/YARN-8295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri reassigned YARN-8295: -- Assignee: Charan Hebri > [UI2] The "Resource Usage" tab is pointless for finished applications > - > > Key: YARN-8295 > URL: https://issues.apache.org/jira/browse/YARN-8295 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn-ui-v2 >Reporter: Gergely Novák >Assignee: Charan Hebri >Priority: Minor > > If the user goes to Applications -> app -> Resource Usage for a finished > application, they get this message: "No resource usage data is available for > this application!". > I think it would be better to hide this tab for finished applications, or at > least add something like "this application is not using any resources because > it is finished" to the message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9295) Fix 'Decomissioned' label typo in Cluster Overview page
Charan Hebri created YARN-9295: -- Summary: Fix 'Decomissioned' label typo in Cluster Overview page Key: YARN-9295 URL: https://issues.apache.org/jira/browse/YARN-9295 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Reporter: Charan Hebri Assignee: Charan Hebri Attachments: Decommissioned-typo.png Change label text from 'Decomissioned' to 'Decommissioned' in Node Managers section of the Cluster Overview page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9295) Fix 'Decomissioned' label typo in Cluster Overview page
[ https://issues.apache.org/jira/browse/YARN-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-9295: --- Attachment: YARN-9295.001.patch > Fix 'Decomissioned' label typo in Cluster Overview page > --- > > Key: YARN-9295 > URL: https://issues.apache.org/jira/browse/YARN-9295 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Reporter: Charan Hebri >Assignee: Charan Hebri >Priority: Trivial > Attachments: Decommissioned-typo.png, YARN-9295.001.patch > > > Change label text from 'Decomissioned' to 'Decommissioned' in Node Managers > section of the Cluster Overview page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9295) Fix 'Decomissioned' label typo in Cluster Overview page
[ https://issues.apache.org/jira/browse/YARN-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16764914#comment-16764914 ] Charan Hebri commented on YARN-9295: Hi [~sunilg] could you please help commit this trivial patch? > Fix 'Decomissioned' label typo in Cluster Overview page > --- > > Key: YARN-9295 > URL: https://issues.apache.org/jira/browse/YARN-9295 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Reporter: Charan Hebri >Assignee: Charan Hebri >Priority: Trivial > Attachments: Decommissioned-typo.png, YARN-9295.001.patch > > > Change label text from 'Decomissioned' to 'Decommissioned' in Node Managers > section of the Cluster Overview page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9257) Distributed Shell throws a NPE for a non-existent queue
Charan Hebri created YARN-9257:
--
Summary: Distributed Shell throws a NPE for a non-existent queue
Key: YARN-9257
URL: https://issues.apache.org/jira/browse/YARN-9257
Project: Hadoop YARN
Issue Type: Bug
Components: distributed-shell
Reporter: Charan Hebri
Assignee: Charan Hebri
When a non-existent queue is specified as a param for launching a distributed
shell, a NPE is thrown.
{noformat}
19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
java.lang.NullPointerException
at
org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
at
org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9257) Distributed Shell client throws a NPE for a non-existent queue
[
https://issues.apache.org/jira/browse/YARN-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-9257:
---
Summary: Distributed Shell client throws a NPE for a non-existent queue
(was: Distributed Shell throws a NPE for a non-existent queue)
> Distributed Shell client throws a NPE for a non-existent queue
> --
>
> Key: YARN-9257
> URL: https://issues.apache.org/jira/browse/YARN-9257
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-9257.01.patch
>
>
> When a non-existent queue is specified as a param for launching a distributed
> shell, a NPE is thrown.
> {noformat}
> 19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9257) Distributed Shell client throws a NPE for a non-existent queue
[
https://issues.apache.org/jira/browse/YARN-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756932#comment-16756932
]
Charan Hebri commented on YARN-9257:
cc [~sunilg]
> Distributed Shell client throws a NPE for a non-existent queue
> --
>
> Key: YARN-9257
> URL: https://issues.apache.org/jira/browse/YARN-9257
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-9257.01.patch
>
>
> When a non-existent queue is specified as a param for launching a distributed
> shell, a NPE is thrown.
> {noformat}
> 19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9257) Distributed Shell throws a NPE for a non-existent queue
[
https://issues.apache.org/jira/browse/YARN-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-9257:
---
Description:
When a non-existent queue is specified as a param for launching a distributed
shell, a NPE is thrown.
{noformat}
19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
java.lang.NullPointerException
at
org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
at
org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
was:
When a non-existent queue is specified as a param for launching a distributed
shell, a NPE is thrown.
{noformat}
19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
java.lang.NullPointerException
at
org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
at
org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
> Distributed Shell throws a NPE for a non-existent queue
> ---
>
> Key: YARN-9257
> URL: https://issues.apache.org/jira/browse/YARN-9257
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Major
>
> When a non-existent queue is specified as a param for launching a distributed
> shell, a NPE is thrown.
> {noformat}
> 19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9257) Distributed Shell throws a NPE for a non-existent queue
[
https://issues.apache.org/jira/browse/YARN-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-9257:
---
Attachment: YARN-9257.01.patch
> Distributed Shell throws a NPE for a non-existent queue
> ---
>
> Key: YARN-9257
> URL: https://issues.apache.org/jira/browse/YARN-9257
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-9257.01.patch
>
>
> When a non-existent queue is specified as a param for launching a distributed
> shell, a NPE is thrown.
> {noformat}
> 19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7627) [ATSv2] When passing a non-number as metricslimit, the error message is wrong
[
https://issues.apache.org/jira/browse/YARN-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-7627:
---
Attachment: YARN-7627.002.patch
> [ATSv2] When passing a non-number as metricslimit, the error message is wrong
> -
>
> Key: YARN-7627
> URL: https://issues.apache.org/jira/browse/YARN-7627
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.0.0-beta1
>Reporter: Grant Sohn
>Assignee: Charan Hebri
>Priority: Trivial
> Attachments: YARN-7627.001.patch, YARN-7627.002.patch
>
>
> curl
> "$ATS_URL/ws/v2/timeline/apps/application_1512430070811_0022/entities/MAPREDUCE_JOB?metricslimit=w"
> returns:
> {"exception":"BadRequestException","message":"java.lang.Exception:
> createdTime start/end or limit or flowrunid is not a numeric
> value.","javaClassName":"org.apache.hadoop.yarn.webapp.BadRequestException"}
> and:
> curl
> "$ATS_URL/ws/v2/timeline/apps/application_1512430070811_0022?metricslimit=ALL"
> {"exception":"BadRequestException","message":"java.lang.Exception: flowrunid
> is not a numeric
> value.","javaClassName":"org.apache.hadoop.yarn.webapp.BadRequestException"}
> This could be part of YARN-6389 which indicates this functionality was not
> completed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9257) Distributed Shell client throws a NPE for a non-existent queue
[
https://issues.apache.org/jira/browse/YARN-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-9257:
---
Attachment: (was: YARN-9257.02.patch)
> Distributed Shell client throws a NPE for a non-existent queue
> --
>
> Key: YARN-9257
> URL: https://issues.apache.org/jira/browse/YARN-9257
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-9257.01.patch
>
>
> When a non-existent queue is specified as a param for launching a distributed
> shell, a NPE is thrown.
> {noformat}
> 19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9257) Distributed Shell client throws a NPE for a non-existent queue
[
https://issues.apache.org/jira/browse/YARN-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-9257:
---
Attachment: YARN-9257-branch-3.2.01.patch
> Distributed Shell client throws a NPE for a non-existent queue
> --
>
> Key: YARN-9257
> URL: https://issues.apache.org/jira/browse/YARN-9257
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-9257-branch-3.2.01.patch, YARN-9257.01.patch
>
>
> When a non-existent queue is specified as a param for launching a distributed
> shell, a NPE is thrown.
> {noformat}
> 19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9257) Distributed Shell client throws a NPE for a non-existent queue
[
https://issues.apache.org/jira/browse/YARN-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-9257:
---
Attachment: (was: YARN-9257-branch-3.2.01.patch)
> Distributed Shell client throws a NPE for a non-existent queue
> --
>
> Key: YARN-9257
> URL: https://issues.apache.org/jira/browse/YARN-9257
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-9257.01.patch
>
>
> When a non-existent queue is specified as a param for launching a distributed
> shell, a NPE is thrown.
> {noformat}
> 19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9257) Distributed Shell client throws a NPE for a non-existent queue
[
https://issues.apache.org/jira/browse/YARN-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-9257:
---
Attachment: YARN-9257.02.patch
> Distributed Shell client throws a NPE for a non-existent queue
> --
>
> Key: YARN-9257
> URL: https://issues.apache.org/jira/browse/YARN-9257
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-9257.01.patch, YARN-9257.02.patch
>
>
> When a non-existent queue is specified as a param for launching a distributed
> shell, a NPE is thrown.
> {noformat}
> 19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9257) Distributed Shell client throws a NPE for a non-existent queue
[
https://issues.apache.org/jira/browse/YARN-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-9257:
---
Attachment: YARN-9257-branch-3.2.01.patch
> Distributed Shell client throws a NPE for a non-existent queue
> --
>
> Key: YARN-9257
> URL: https://issues.apache.org/jira/browse/YARN-9257
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-9257-branch-3.2.01.patch, YARN-9257.01.patch
>
>
> When a non-existent queue is specified as a param for launching a distributed
> shell, a NPE is thrown.
> {noformat}
> 19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9257) Distributed Shell client throws a NPE for a non-existent queue
[
https://issues.apache.org/jira/browse/YARN-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760786#comment-16760786
]
Charan Hebri commented on YARN-9257:
[~sunilg] attached patch for 3.2 branch.
> Distributed Shell client throws a NPE for a non-existent queue
> --
>
> Key: YARN-9257
> URL: https://issues.apache.org/jira/browse/YARN-9257
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-9257-branch-3.2.01.patch, YARN-9257.01.patch
>
>
> When a non-existent queue is specified as a param for launching a distributed
> shell, a NPE is thrown.
> {noformat}
> 19/01/30 09:41:26 ERROR distributedshell.Client: Error running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:652)
> at
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:265){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9625) UI2 - No link to a queue on the Queues page for Fair Scheduler
Charan Hebri created YARN-9625: -- Summary: UI2 - No link to a queue on the Queues page for Fair Scheduler Key: YARN-9625 URL: https://issues.apache.org/jira/browse/YARN-9625 Project: Hadoop YARN Issue Type: Bug Reporter: Charan Hebri When the scheduler is set as 'Capacity Scheduler' the Queues page has a tab on the right with a link to a certain queue which provides running app information for the queue. But for 'Fair Scheduler' there is no such link. Attached screenshots for both schedulers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9625) UI2 - No link to a queue on the Queues page for Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-9625: --- Attachment: Capacity_scheduler_page.png > UI2 - No link to a queue on the Queues page for Fair Scheduler > -- > > Key: YARN-9625 > URL: https://issues.apache.org/jira/browse/YARN-9625 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Priority: Major > Attachments: Capacity_scheduler_page.png, Fair_scheduler_page.png > > > When the scheduler is set as 'Capacity Scheduler' the Queues page has a tab > on the right with a link to a certain queue which provides running app > information for the queue. But for 'Fair Scheduler' there is no such link. > Attached screenshots for both schedulers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9625) UI2 - No link to a queue on the Queues page for Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-9625: --- Attachment: Fair_scheduler_page.png > UI2 - No link to a queue on the Queues page for Fair Scheduler > -- > > Key: YARN-9625 > URL: https://issues.apache.org/jira/browse/YARN-9625 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Priority: Major > Attachments: Capacity_scheduler_page.png, Fair_scheduler_page.png > > > When the scheduler is set as 'Capacity Scheduler' the Queues page has a tab > on the right with a link to a certain queue which provides running app > information for the queue. But for 'Fair Scheduler' there is no such link. > Attached screenshots for both schedulers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9626) UI2 - Fair scheduler queue apps page issues
Charan Hebri created YARN-9626: -- Summary: UI2 - Fair scheduler queue apps page issues Key: YARN-9626 URL: https://issues.apache.org/jira/browse/YARN-9626 Project: Hadoop YARN Issue Type: Bug Reporter: Charan Hebri There are a few issues with the apps page for a queue when Fair Scheduler is used. * Labels like configured capacity, configured max capacity etc. (marked in the attached image) are not needed as they are specific to Capacity Scheduler. * Steady fair memory, used memory and maximum memory are actual values but are shown as percentages. * Formatting of Pending, Allocated, Reserved Containers values is not correct (shown in the attached screenshot) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9535) Typos in Docker documentation
[ https://issues.apache.org/jira/browse/YARN-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri reassigned YARN-9535: -- Assignee: Charan Hebri > Typos in Docker documentation > - > > Key: YARN-9535 > URL: https://issues.apache.org/jira/browse/YARN-9535 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Szilard Nemeth >Assignee: Charan Hebri >Priority: Major > Labels: newbie, newbie++ > > container-executer should be replaced with container-executor in Docker > documentation page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9468) Fix inaccurate documentations in Placement Constraints
[
https://issues.apache.org/jira/browse/YARN-9468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri reassigned YARN-9468:
--
Assignee: Charan Hebri
> Fix inaccurate documentations in Placement Constraints
> --
>
> Key: YARN-9468
> URL: https://issues.apache.org/jira/browse/YARN-9468
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Assignee: Charan Hebri
>Priority: Major
>
> Document Placement Constraints
> *First*
> {code:java}
> zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
> * place 5 containers with tag “hbase” with affinity to a rack on which
> containers with tag “zk” are running (i.e., an “hbase” container
> should{color:#ff} not{color} be placed at a rack where an “zk” container
> is running, given that “zk” is the TargetTag of the second constraint);
> The _*not*_ word in brackets should be delete.
>
> *Second*
> {code:java}
> PlacementSpec => "" | KeyVal;PlacementSpec
> {code}
> The semicolon should be replaced by colon
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9535) Typos in Docker documentation
[ https://issues.apache.org/jira/browse/YARN-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-9535: --- Attachment: YARN-9535.001.patch > Typos in Docker documentation > - > > Key: YARN-9535 > URL: https://issues.apache.org/jira/browse/YARN-9535 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Szilard Nemeth >Assignee: Charan Hebri >Priority: Major > Labels: newbie, newbie++ > Attachments: YARN-9535.001.patch > > > container-executer should be replaced with container-executor in Docker > documentation page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9626) UI2 - Fair scheduler queue apps page issues
[ https://issues.apache.org/jira/browse/YARN-9626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863852#comment-16863852 ] Charan Hebri commented on YARN-9626: cc [~sunilg] > UI2 - Fair scheduler queue apps page issues > --- > > Key: YARN-9626 > URL: https://issues.apache.org/jira/browse/YARN-9626 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Priority: Major > Attachments: Fair_scheduler_apps_page.png > > > There are a few issues with the apps page for a queue when Fair Scheduler is > used. > * Labels like configured capacity, configured max capacity etc. (marked in > the attached image) are not needed as they are specific to Capacity Scheduler. > * Steady fair memory, used memory and maximum memory are actual values but > are shown as percentages. > * Formatting of Pending, Allocated, Reserved Containers values is not > correct (shown in the attached screenshot) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9626) UI2 - Fair scheduler queue apps page issues
[ https://issues.apache.org/jira/browse/YARN-9626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri updated YARN-9626: --- Attachment: Fair_scheduler_apps_page.png > UI2 - Fair scheduler queue apps page issues > --- > > Key: YARN-9626 > URL: https://issues.apache.org/jira/browse/YARN-9626 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Priority: Major > Attachments: Fair_scheduler_apps_page.png > > > There are a few issues with the apps page for a queue when Fair Scheduler is > used. > * Labels like configured capacity, configured max capacity etc. (marked in > the attached image) are not needed as they are specific to Capacity Scheduler. > * Steady fair memory, used memory and maximum memory are actual values but > are shown as percentages. > * Formatting of Pending, Allocated, Reserved Containers values is not > correct (shown in the attached screenshot) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9625) UI2 - No link to a queue on the Queues page for Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863848#comment-16863848 ] Charan Hebri commented on YARN-9625: cc [~sunilg] > UI2 - No link to a queue on the Queues page for Fair Scheduler > -- > > Key: YARN-9625 > URL: https://issues.apache.org/jira/browse/YARN-9625 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Priority: Major > Attachments: Capacity_scheduler_page.png, Fair_scheduler_page.png > > > When the scheduler is set as 'Capacity Scheduler' the Queues page has a tab > on the right with a link to a certain queue which provides running app > information for the queue. But for 'Fair Scheduler' there is no such link. > Attached screenshots for both schedulers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9982) Fix Container API example link in NodeManager REST API doc
[
https://issues.apache.org/jira/browse/YARN-9982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charan Hebri updated YARN-9982:
---
Attachment: YARN-9982.001.patch
> Fix Container API example link in NodeManager REST API doc
> --
>
> Key: YARN-9982
> URL: https://issues.apache.org/jira/browse/YARN-9982
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation
>Reporter: Charan Hebri
>Assignee: Charan Hebri
>Priority: Trivial
> Attachments: YARN-9982.001.patch
>
>
> In
> [https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerRest.html#Container_API]
> {noformat}
> GET
> http://nm-http-address:port/ws/v1/nodes/containers/container_1326121700862_0007_01_01{noformat}
> should be changed to,
> {noformat}
> GET
> http://nm-http-address:port/ws/v1/node/containers/container_1326121700862_0007_01_01{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9982) Fix Container API example link in NodeManager REST API doc
Charan Hebri created YARN-9982:
--
Summary: Fix Container API example link in NodeManager REST API doc
Key: YARN-9982
URL: https://issues.apache.org/jira/browse/YARN-9982
Project: Hadoop YARN
Issue Type: Bug
Components: documentation
Reporter: Charan Hebri
Assignee: Charan Hebri
In
[https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerRest.html#Container_API]
{noformat}
GET
http://nm-http-address:port/ws/v1/nodes/containers/container_1326121700862_0007_01_01{noformat}
should be changed to,
{noformat}
GET
http://nm-http-address:port/ws/v1/node/containers/container_1326121700862_0007_01_01{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
