[jira] [Comment Edited] (YARN-7931) [atsv2 read acls] Include domain table creation as part of schema creator
[ https://issues.apache.org/jira/browse/YARN-7931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426483#comment-16426483 ] Rohith Sharma K S edited comment on YARN-7931 at 4/5/18 4:55 AM: - overall patch looks good to me. One doubt in domain table row key. Domain table row key has only domainId. It should be appended with clusterId right? Otherwise it wont be able to track it for domainId belongs to which cluster right? was (Author: rohithsharma): overall patch looks good to me. One doubt in domain table row key. Domain table row key as only domainId. It should be appended with clusterId right? Otherwise it wont be able to track it for domainId belongs to which cluster right? > [atsv2 read acls] Include domain table creation as part of schema creator > - > > Key: YARN-7931 > URL: https://issues.apache.org/jira/browse/YARN-7931 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vrushali C >Assignee: Vrushali C >Priority: Major > Attachments: YARN-7391.0001.patch, YARN-7391.0002.patch > > > > Update the schema creator to create a domain table to store timeline entity > domain info. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7931) [atsv2 read acls] Include domain table creation as part of schema creator
[ https://issues.apache.org/jira/browse/YARN-7931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426483#comment-16426483 ] Rohith Sharma K S commented on YARN-7931: - overall patch looks good to me. One doubt in domain table row key. Domain table row key as only domainId. It should be appended with clusterId right? Otherwise it wont be able to track it for domainId belongs to which cluster right? > [atsv2 read acls] Include domain table creation as part of schema creator > - > > Key: YARN-7931 > URL: https://issues.apache.org/jira/browse/YARN-7931 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vrushali C >Assignee: Vrushali C >Priority: Major > Attachments: YARN-7391.0001.patch, YARN-7391.0002.patch > > > > Update the schema creator to create a domain table to store timeline entity > domain info. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426482#comment-16426482 ] Rohith Sharma K S commented on YARN-6936: - updated the patch fixing review comments. > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch, > YARN-6936.002.patch, YARN-6936.003.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-6936: Attachment: YARN-6936.003.patch > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch, > YARN-6936.002.patch, YARN-6936.003.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8117) Fix TestRMWebServicesNodes test failure
[ https://issues.apache.org/jira/browse/YARN-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426474#comment-16426474 ] Rohith Sharma K S edited comment on YARN-8117 at 4/5/18 4:38 AM: - [~bibinchundatt] As per Apache bylaw, you need to wait for other committers give +1. Patch provider can't commit the patch without reviewing by peer members. Please wait for peer members to review it. was (Author: rohithsharma): [~bibinchundatt] As per Apache bylaw, you need to wait for other committers give +1. The reporter itself can't commit the patch without reviewing by peer members. Please wait for peer members to review it. > Fix TestRMWebServicesNodes test failure > --- > > Key: YARN-8117 > URL: https://issues.apache.org/jira/browse/YARN-8117 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Major > Attachments: YARN-8117-YARN-3409.001.patch > > > Need to fix conflict in TestRMWebServices due to YARN-7792 and YARN-8092 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8117) Fix TestRMWebServicesNodes test failure
[ https://issues.apache.org/jira/browse/YARN-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426474#comment-16426474 ] Rohith Sharma K S commented on YARN-8117: - [~bibinchundatt] As per Apache bylaw, you need to wait for other committers give +1. The reporter itself can't commit the patch without reviewing by peer members. Please wait for peer members to review it. > Fix TestRMWebServicesNodes test failure > --- > > Key: YARN-8117 > URL: https://issues.apache.org/jira/browse/YARN-8117 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Major > Attachments: YARN-8117-YARN-3409.001.patch > > > Need to fix conflict in TestRMWebServices due to YARN-7792 and YARN-8092 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
[ https://issues.apache.org/jira/browse/YARN-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426457#comment-16426457 ] Rohith Sharma K S commented on YARN-8073: - Btw, this patch can be cherry picked to branch-3.1 as well. > TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration > - > > Key: YARN-8073 > URL: https://issues.apache.org/jira/browse/YARN-8073 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8073-branch-2.03.patch, YARN-8073.01.patch, > YARN-8073.02.patch, YARN-8073.03.patch > > > Post YARN-6736, RM support writing into ats v1 and v2 by new configuration > setting _yarn.timeline-service.versions_. > Couple of issues observed in deployment are > # TimelineClientImpl doesn't honor newly added configuration rather it still > get version number from _yarn.timeline-service.version_. This causes not > writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 > value._ > # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled > doesn't honor timeline-service.versions. > # JobHistoryEventHandler#serviceInit(), line no 271 check for version number > rather than calling YarnConfiguration#timelineServiceV2Enabled > cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
[ https://issues.apache.org/jira/browse/YARN-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426454#comment-16426454 ] Rohith Sharma K S commented on YARN-8073: - Anyway I attached branch-2 patch for triggering jenkins. > TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration > - > > Key: YARN-8073 > URL: https://issues.apache.org/jira/browse/YARN-8073 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8073-branch-2.03.patch, YARN-8073.01.patch, > YARN-8073.02.patch, YARN-8073.03.patch > > > Post YARN-6736, RM support writing into ats v1 and v2 by new configuration > setting _yarn.timeline-service.versions_. > Couple of issues observed in deployment are > # TimelineClientImpl doesn't honor newly added configuration rather it still > get version number from _yarn.timeline-service.version_. This causes not > writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 > value._ > # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled > doesn't honor timeline-service.versions. > # JobHistoryEventHandler#serviceInit(), line no 271 check for version number > rather than calling YarnConfiguration#timelineServiceV2Enabled > cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
[ https://issues.apache.org/jira/browse/YARN-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8073: Attachment: YARN-8073-branch-2.03.patch > TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration > - > > Key: YARN-8073 > URL: https://issues.apache.org/jira/browse/YARN-8073 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8073-branch-2.03.patch, YARN-8073.01.patch, > YARN-8073.02.patch, YARN-8073.03.patch > > > Post YARN-6736, RM support writing into ats v1 and v2 by new configuration > setting _yarn.timeline-service.versions_. > Couple of issues observed in deployment are > # TimelineClientImpl doesn't honor newly added configuration rather it still > get version number from _yarn.timeline-service.version_. This causes not > writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 > value._ > # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled > doesn't honor timeline-service.versions. > # JobHistoryEventHandler#serviceInit(), line no 271 check for version number > rather than calling YarnConfiguration#timelineServiceV2Enabled > cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
[ https://issues.apache.org/jira/browse/YARN-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426453#comment-16426453 ] Rohith Sharma K S commented on YARN-8073: - bq. the patch does not apply as is to branch-2 I just tried to cherry-pick this commit and got succeeded. Could you try it again? > TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration > - > > Key: YARN-8073 > URL: https://issues.apache.org/jira/browse/YARN-8073 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8073.01.patch, YARN-8073.02.patch, > YARN-8073.03.patch > > > Post YARN-6736, RM support writing into ats v1 and v2 by new configuration > setting _yarn.timeline-service.versions_. > Couple of issues observed in deployment are > # TimelineClientImpl doesn't honor newly added configuration rather it still > get version number from _yarn.timeline-service.version_. This causes not > writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 > value._ > # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled > doesn't honor timeline-service.versions. > # JobHistoryEventHandler#serviceInit(), line no 271 check for version number > rather than calling YarnConfiguration#timelineServiceV2Enabled > cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8107) NPE when incorrect format is used in ATSv2 filter attributes
[
https://issues.apache.org/jira/browse/YARN-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16425380#comment-16425380
]
Rohith Sharma K S commented on YARN-8107:
-
[~charanh] can you provide REST API used to test above?
> NPE when incorrect format is used in ATSv2 filter attributes
>
>
> Key: YARN-8107
> URL: https://issues.apache.org/jira/browse/YARN-8107
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Reporter: Charan Hebri
>Assignee: Rohith Sharma K S
>Priority: Major
>
> Using an incorrect format for infofilters, conffilters and metricfilters
> throws a NPE with no clear message to the caller. This should be tagged as a
> 400 Bad Request with an informative message. Below is the timeline reader log.
> {noformat}
> 2018-04-02 06:44:10,451 INFO reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(173)) - Processed URL
> /ws/v2/timeline/users/hrt_qa/flows/flow4/runs/1/apps?infofilters=UIDeq but
> encountered exception (Took 0 ms.)
> 2018-04-02 06:44:10,451 ERROR reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(188)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterUtils.createHBaseFilterList(TimelineFilterUtils.java:276)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.ApplicationEntityReader.constructFilterListBasedOnFilters(ApplicationEntityReader.java:126)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.createFilterList(TimelineEntityReader.java:157)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:277)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntities(HBaseTimelineReaderImpl.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntities(TimelineReaderManager.java:143)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getEntities(TimelineReaderWebServices.java:605)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRunApps(TimelineReaderWebServices.java:1962)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.security.TimelineReaderWhitelistAuthorizationFilter.doFilter(TimelineReaderWhitelistAuthorizationFilter.java:85)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at
>
[jira] [Assigned] (YARN-8107) NPE when incorrect format is used in ATSv2 filter attributes
[
https://issues.apache.org/jira/browse/YARN-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S reassigned YARN-8107:
---
Assignee: Rohith Sharma K S
> NPE when incorrect format is used in ATSv2 filter attributes
>
>
> Key: YARN-8107
> URL: https://issues.apache.org/jira/browse/YARN-8107
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Reporter: Charan Hebri
>Assignee: Rohith Sharma K S
>Priority: Major
>
> Using an incorrect format for infofilters, conffilters and metricfilters
> throws a NPE with no clear message to the caller. This should be tagged as a
> 400 Bad Request with an informative message. Below is the timeline reader log.
> {noformat}
> 2018-04-02 06:44:10,451 INFO reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(173)) - Processed URL
> /ws/v2/timeline/users/hrt_qa/flows/flow4/runs/1/apps?infofilters=UIDeq but
> encountered exception (Took 0 ms.)
> 2018-04-02 06:44:10,451 ERROR reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(188)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterUtils.createHBaseFilterList(TimelineFilterUtils.java:276)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.ApplicationEntityReader.constructFilterListBasedOnFilters(ApplicationEntityReader.java:126)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.createFilterList(TimelineEntityReader.java:157)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:277)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntities(HBaseTimelineReaderImpl.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntities(TimelineReaderManager.java:143)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getEntities(TimelineReaderWebServices.java:605)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRunApps(TimelineReaderWebServices.java:1962)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.security.TimelineReaderWhitelistAuthorizationFilter.doFilter(TimelineReaderWhitelistAuthorizationFilter.java:85)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at
>
[jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16425077#comment-16425077 ] Rohith Sharma K S commented on YARN-6936: - As offline discussion with Vrushali and Haibo, primary concern is malicious AM can overwrite the system entities data i.e YARN-3401. We track this issue separately as it was known issue discussed long back. This is very important JIRA as AM can corrupt system entities data. This need to get before ATSv2 beta release. Scope of current JIRA is to allow entities written into sub app table using new API. System entities which are defined in TimelineEntityType are not allowed to store under sub app tables. > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch, > YARN-6936.002.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
[ https://issues.apache.org/jira/browse/YARN-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8073: Attachment: YARN-8073.03.patch > TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration > - > > Key: YARN-8073 > URL: https://issues.apache.org/jira/browse/YARN-8073 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8073.01.patch, YARN-8073.02.patch, > YARN-8073.03.patch > > > Post YARN-6736, RM support writing into ats v1 and v2 by new configuration > setting _yarn.timeline-service.versions_. > Couple of issues observed in deployment are > # TimelineClientImpl doesn't honor newly added configuration rather it still > get version number from _yarn.timeline-service.version_. This causes not > writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 > value._ > # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled > doesn't honor timeline-service.versions. > # JobHistoryEventHandler#serviceInit(), line no 271 check for version number > rather than calling YarnConfiguration#timelineServiceV2Enabled > cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
[ https://issues.apache.org/jira/browse/YARN-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16425069#comment-16425069 ] Rohith Sharma K S commented on YARN-8073: - Updated the patch fixing checkstyle issues. > TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration > - > > Key: YARN-8073 > URL: https://issues.apache.org/jira/browse/YARN-8073 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8073.01.patch, YARN-8073.02.patch, > YARN-8073.03.patch > > > Post YARN-6736, RM support writing into ats v1 and v2 by new configuration > setting _yarn.timeline-service.versions_. > Couple of issues observed in deployment are > # TimelineClientImpl doesn't honor newly added configuration rather it still > get version number from _yarn.timeline-service.version_. This causes not > writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 > value._ > # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled > doesn't honor timeline-service.versions. > # JobHistoryEventHandler#serviceInit(), line no 271 check for version number > rather than calling YarnConfiguration#timelineServiceV2Enabled > cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-6936: Attachment: YARN-6936.002.patch > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch, > YARN-6936.002.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16420239#comment-16420239 ] Rohith Sharma K S commented on YARN-6936: - bq. Hence, I think we don't need to create another publisher just for SubApplication Entities. We still need separate publisher because SubApplicationEntity will not contain any predefined entity type rather it is user defined entity type unlike any other entities such as YARN_APPLICATION and others. When entities are posted to TimelineCollectorWebService, TimelineEntities are wrapped to corresponding Hierarchical entities which are predefined with entity type. But for sub application entities are user defined entity types. So separate API's and publisher in Timelinev2clientImpl is required to distinguish between normal entity and sub application entities. > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16420155#comment-16420155 ] Rohith Sharma K S commented on YARN-8048: - [~leftnoteasy] would you take a look at last patch please? > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch, > YARN-8048.003.patch, YARN-8048.004.patch, YARN-8048.005.patch, > YARN-8048.006.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8048: Attachment: YARN-8048.006.patch > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch, > YARN-8048.003.patch, YARN-8048.004.patch, YARN-8048.005.patch, > YARN-8048.006.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8085) RMContext#resourceProfilesManager is lost after RM went standby then back to active
[
https://issues.apache.org/jira/browse/YARN-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418667#comment-16418667
]
Rohith Sharma K S commented on YARN-8085:
-
+1 lgtm
> RMContext#resourceProfilesManager is lost after RM went standby then back to
> active
> ---
>
> Key: YARN-8085
> URL: https://issues.apache.org/jira/browse/YARN-8085
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
>Reporter: Tao Yang
>Assignee: Tao Yang
>Priority: Blocker
> Attachments: YARN-8085.001.patch, YARN-8085.002.patch,
> YARN-8085.003.patch, YARN-8085.004.patch
>
>
> We submited a distributed shell application after RM failover and back to
> active, then got NPE error in RM log:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getResourceProfiles(ClientRMService.java:1814)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getResourceProfiles(ApplicationClientProtocolPBServiceImpl.java:657)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:617)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
> {noformat}
> The cause is that currently resourceProfilesManager is not transferred to new
> RMContext instance in RMContext#resetRMContext. We should do this transfer to
> fix this error.
> {code:java}
> @@ -1488,6 +1488,10 @@ private void resetRMContext() {
> // transfer service context to new RM service Context
> rmContextImpl.setServiceContext(rmContext.getServiceContext());
> +// transfer resource profiles manager
> +rmContextImpl
> +.setResourceProfilesManager(rmContext.getResourceProfilesManager());
> +
> // reset dispatcher
> Dispatcher dispatcher = setupDispatcher();
> ((Service) dispatcher).init(this.conf);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8085) RMContext#resourceProfilesManager is lost after RM went standby then back to active
[
https://issues.apache.org/jira/browse/YARN-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418536#comment-16418536
]
Rohith Sharma K S commented on YARN-8085:
-
For 003.patch, profile adding at ResourceManager@serviceInit need to be
removed. Otherwise ResourceProfilesManager created and added into composite
service 2 times unnecessarily i.e one at RM level and active service init.
> RMContext#resourceProfilesManager is lost after RM went standby then back to
> active
> ---
>
> Key: YARN-8085
> URL: https://issues.apache.org/jira/browse/YARN-8085
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
>Reporter: Tao Yang
>Assignee: Tao Yang
>Priority: Blocker
> Attachments: YARN-8085.001.patch, YARN-8085.002.patch,
> YARN-8085.003.patch
>
>
> We submited a distributed shell application after RM failover and back to
> active, then got NPE error in RM log:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getResourceProfiles(ClientRMService.java:1814)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getResourceProfiles(ApplicationClientProtocolPBServiceImpl.java:657)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:617)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
> {noformat}
> The cause is that currently resourceProfilesManager is not transferred to new
> RMContext instance in RMContext#resetRMContext. We should do this transfer to
> fix this error.
> {code:java}
> @@ -1488,6 +1488,10 @@ private void resetRMContext() {
> // transfer service context to new RM service Context
> rmContextImpl.setServiceContext(rmContext.getServiceContext());
> +// transfer resource profiles manager
> +rmContextImpl
> +.setResourceProfilesManager(rmContext.getResourceProfilesManager());
> +
> // reset dispatcher
> Dispatcher dispatcher = setupDispatcher();
> ((Service) dispatcher).init(this.conf);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8085) RMContext#resourceProfilesManager is lost after RM went standby then back to active
[
https://issues.apache.org/jira/browse/YARN-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8085:
Priority: Blocker (was: Major)
> RMContext#resourceProfilesManager is lost after RM went standby then back to
> active
> ---
>
> Key: YARN-8085
> URL: https://issues.apache.org/jira/browse/YARN-8085
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
>Affects Versions: 3.1.0
>Reporter: Tao Yang
>Assignee: Tao Yang
>Priority: Blocker
> Attachments: YARN-8085.001.patch, YARN-8085.002.patch
>
>
> We submited a distributed shell application after RM failover and back to
> active, then got NPE error in RM log:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getResourceProfiles(ClientRMService.java:1814)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getResourceProfiles(ApplicationClientProtocolPBServiceImpl.java:657)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:617)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
> {noformat}
> The cause is that currently resourceProfilesManager is not transferred to new
> RMContext instance in RMContext#resetRMContext. We should do this transfer to
> fix this error.
> {code:java}
> @@ -1488,6 +1488,10 @@ private void resetRMContext() {
> // transfer service context to new RM service Context
> rmContextImpl.setServiceContext(rmContext.getServiceContext());
> +// transfer resource profiles manager
> +rmContextImpl
> +.setResourceProfilesManager(rmContext.getResourceProfilesManager());
> +
> // reset dispatcher
> Dispatcher dispatcher = setupDispatcher();
> ((Service) dispatcher).init(this.conf);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
[ https://issues.apache.org/jira/browse/YARN-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8073: Attachment: YARN-8073.02.patch > TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration > - > > Key: YARN-8073 > URL: https://issues.apache.org/jira/browse/YARN-8073 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8073.01.patch, YARN-8073.02.patch > > > Post YARN-6736, RM support writing into ats v1 and v2 by new configuration > setting _yarn.timeline-service.versions_. > Couple of issues observed in deployment are > # TimelineClientImpl doesn't honor newly added configuration rather it still > get version number from _yarn.timeline-service.version_. This causes not > writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 > value._ > # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled > doesn't honor timeline-service.versions. > # JobHistoryEventHandler#serviceInit(), line no 271 check for version number > rather than calling YarnConfiguration#timelineServiceV2Enabled > cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8085) RMContext#resourceProfilesManager is lost after RM went standby then back to active
[
https://issues.apache.org/jira/browse/YARN-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418494#comment-16418494
]
Rohith Sharma K S commented on YARN-8085:
-
thanks [~Tao Yang] for the patch! It is a blocker for 3.1.0.
Regarding the patch, we have clear separation of alwaysOnService vs
activeServiceContext. ResourceProfilesManager should be part of active service
context because activeServiceContext context is refreshed for every
transition. I see that profiles are loaded during service initialization. If it
get added into serviceContext, then profiles are NOT refreshed which get stale
profile information. cc :/ [~sunilg]
> RMContext#resourceProfilesManager is lost after RM went standby then back to
> active
> ---
>
> Key: YARN-8085
> URL: https://issues.apache.org/jira/browse/YARN-8085
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
>Affects Versions: 3.1.0
>Reporter: Tao Yang
>Assignee: Tao Yang
>Priority: Major
> Attachments: YARN-8085.001.patch, YARN-8085.002.patch
>
>
> We submited a distributed shell application after RM failover and back to
> active, then got NPE error in RM log:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getResourceProfiles(ClientRMService.java:1814)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getResourceProfiles(ApplicationClientProtocolPBServiceImpl.java:657)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:617)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
> {noformat}
> The cause is that currently resourceProfilesManager is not transferred to new
> RMContext instance in RMContext#resetRMContext. We should do this transfer to
> fix this error.
> {code:java}
> @@ -1488,6 +1488,10 @@ private void resetRMContext() {
> // transfer service context to new RM service Context
> rmContextImpl.setServiceContext(rmContext.getServiceContext());
> +// transfer resource profiles manager
> +rmContextImpl
> +.setResourceProfilesManager(rmContext.getResourceProfilesManager());
> +
> // reset dispatcher
> Dispatcher dispatcher = setupDispatcher();
> ((Service) dispatcher).init(this.conf);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7946) Update TimelineServerV2 doc as per YARN-7919
[ https://issues.apache.org/jira/browse/YARN-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417960#comment-16417960 ] Rohith Sharma K S commented on YARN-7946: - +1 lgtm.. [~vrushalic] would you take a look at the patch? > Update TimelineServerV2 doc as per YARN-7919 > > > Key: YARN-7946 > URL: https://issues.apache.org/jira/browse/YARN-7946 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7946.00.patch, YARN-7946.01.patch, > YARN-7946.02.patch > > > Post YARN-7919, document need to be updated for co processor jar name and > other related details if any. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417956#comment-16417956 ] Rohith Sharma K S commented on YARN-8048: - bq. For the 2nd level, it's better to only read files ended with .yarnfile I am in dilemma! Given native service framework is enforcing all spec file to be ended with .yarnfile i.e making standardized spec file extension, it make sense to check file extension. Otherwise it is normal json file which NOT required to check for file extension. However in my last patch I incorporated this file extension check, but I am _not sure_ does native service going to make it standard. I see that from CLI command which submits a service reads normal json file i.e ApiServiceClient#loadAppJsonFromLocalFS. It doesn't check for yarnfile extension. bq. I'm not sure about correctness of this behavior. Though it is called as async mode, it doesn't executed after some delay. It is started in separate thread which in turn launches services in no time. This makes admin service to release lock faster during transitioning to active phase. However, going with sync and async folder hierarchy also make sense to me. I will update the patch with this change. > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch, > YARN-8048.003.patch, YARN-8048.004.patch, YARN-8048.005.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7946) Update TimelineServerV2 doc as per YARN-7919
[ https://issues.apache.org/jira/browse/YARN-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417773#comment-16417773 ] Rohith Sharma K S commented on YARN-7946: - The similar changes required in Building.txt file also i.e 2nd sentence in 1st paragraph. > Update TimelineServerV2 doc as per YARN-7919 > > > Key: YARN-7946 > URL: https://issues.apache.org/jira/browse/YARN-7946 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7946.00.patch, YARN-7946.01.patch > > > Post YARN-7919, document need to be updated for co processor jar name and > other related details if any. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[
https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417766#comment-16417766
]
Rohith Sharma K S commented on YARN-6936:
-
bq. Let's add the scope of the entities to each of the four methods
OK, Does this modified sentence fine? {{Send the information of a number of
conceptual entities in the scope of a YARN application to the timeline service
v.2 collector.}}. Does all 4 API need to be modified with same way? For newer
API, it should be out side scope of application also right?
bq. Is it intended to extend updateAggregateStatus() so that sub application
metrics are rolled up?
I vaguely remember this we discussed in weekly call and decided to aggregate
for both APIs. Because newer APIs write into both tables i.e entity and subapp
table which. So aggregated metrics can also available in application scope as
well.
bq. The TimelineCollectorContext is bound to an application attempt. Adding a
subApplicationWrite flag to TimelineCollectorContext may not be the most
intuitive approach. How about we leave subApplicationWrite as a separate flag
instead?
I would inclined to send required information in record rather sending in
parameter. This avoids compatibility in future. May be let's define newer
record that contains context, ugi and subappwrite. cc :/ [~vrushalic]
> [Atsv2] Retrospect storing entities into sub application table from client
> perspective
> --
>
> Key: YARN-6936
> URL: https://issues.apache.org/jira/browse/YARN-6936
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-6936.000.patch, YARN-6936.001.patch
>
>
> Currently YARN-6734 stores entities into sub application table only if doAs
> user and submitted users are different. This holds good for Tez kind of use
> cases. But AM runs as same as submitted user like MR also need to store
> entities in sub application table so that it could read entities without
> application id.
> This would be a point of concern later stages when ATSv2 is deployed into
> production. This JIRA is to retrospect decision of storing entities into sub
> application table based on client side configuration driven rather than user
> driven.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7946) Update TimelineServerV2 doc as per YARN-7919
[
https://issues.apache.org/jira/browse/YARN-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417208#comment-16417208
]
Rohith Sharma K S commented on YARN-7946:
-
Overall looks good. Does below change make sense? However two bullet points
explains about each versions.
{code:java}
The version of Apache HBase that is supported with Timeline Service v.2 is
1.2.6 (default) and 2.0.0-beta1.
{code}
to
{code:java}
The supported version of Apache HBase are 1.2.6 (default) and 2.0.0-beta1.
{code}
> Update TimelineServerV2 doc as per YARN-7919
>
>
> Key: YARN-7946
> URL: https://issues.apache.org/jira/browse/YARN-7946
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7946.00.patch
>
>
> Post YARN-7919, document need to be updated for co processor jar name and
> other related details if any.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417178#comment-16417178 ] Rohith Sharma K S commented on YARN-6936: - [~haibochen] [~vrushalic] Could you review the patch? I will update new patch fixing test failures along with review comments > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8048: Attachment: YARN-8048.005.patch > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch, > YARN-8048.003.patch, YARN-8048.004.patch, YARN-8048.005.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416839#comment-16416839 ] Rohith Sharma K S commented on YARN-8048: - bq. Should we think of a better default location than under /tmp/? Wangda suggested to remove default location path. Yes, this is hdfs path. > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch, > YARN-8048.003.patch, YARN-8048.004.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416804#comment-16416804 ] Rohith Sharma K S commented on YARN-8048: - bq. I think we should hardcode this one unless we have any use case requires pluggability. Agree bq. I think we should skip system services if the folder doesn't exist. Agree. Admin need to create explicitly and give as yarn daemon user permission. bq. Why 777 permission? Ahh.. It should be 700 permission. Only yarn user can access it. Given yarn service expect a system service folder to be present, we need not to apply permission. This can be removed. bq. it's better to only read files ended with .yarnfile. Agree bq. It's better to make user name(s) configurable in yarn-site.xml . Services should be able to launch for any users. Same service can be launched by different user. Configuration will bring up additional work to admin after adding a yarn-file under system directory. User->Services mapping would grow more. Going with user folder makes easier for adding service for admin create folder and keep yarn files under user folder. bq. Otherwise some user could put some random folders which is recognized as user name and cause failure/security issues. As part of security, only admin specifically yarn user have permission to access this folder. This folder should be precreate with right permissions. No arbitrary users can create a folder under system-service directory. From code perspective, any empty folder under system-service are ignored. bq. I'm not sure if we should make system service launch asynchronized or synchronized. To me it makes more sense to launch synchronized. making synchronization will delay RM transition period if more services to be started. bq. Renames: Agree > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch, > YARN-8048.003.patch, YARN-8048.004.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S reassigned YARN-7213: --- Assignee: (was: Rohith Sharma K S) I am unassigning the umbrella JIRA as it is collaborative work from community. > [Umbrella] Test and validate HBase-2.0.x with Atsv2 > --- > > Key: YARN-7213 > URL: https://issues.apache.org/jira/browse/YARN-7213 > Project: Hadoop YARN > Issue Type: Task >Reporter: Rohith Sharma K S >Priority: Major > Attachments: YARN-7213.prelim.patch, YARN-7213.prelim.patch, > YARN-7213.wip.patch > > > Hbase-2.0.x officially support hadoop-alpha compilations. And also they are > getting ready for Hadoop-beta release so that HBase can release their > versions compatible with Hadoop-beta. So, this JIRA is to keep track of > HBase-2.0 integration issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7946) Update TimelineServerV2 doc as per YARN-7919
[ https://issues.apache.org/jira/browse/YARN-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S reassigned YARN-7946: --- Assignee: Haibo Chen (was: Rohith Sharma K S) > Update TimelineServerV2 doc as per YARN-7919 > > > Key: YARN-7946 > URL: https://issues.apache.org/jira/browse/YARN-7946 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Haibo Chen >Priority: Major > > Post YARN-7919, document need to be updated for co processor jar name and > other related details if any. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7946) Update TimelineServerV2 doc as per YARN-7919
[ https://issues.apache.org/jira/browse/YARN-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415691#comment-16415691 ] Rohith Sharma K S commented on YARN-7946: - I haven't yet. Sure you can take it. > Update TimelineServerV2 doc as per YARN-7919 > > > Key: YARN-7946 > URL: https://issues.apache.org/jira/browse/YARN-7946 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > > Post YARN-7919, document need to be updated for co processor jar name and > other related details if any. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
[ https://issues.apache.org/jira/browse/YARN-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415557#comment-16415557 ] Rohith Sharma K S commented on YARN-8073: - [~haibo.chen] can you review the patch please? > TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration > - > > Key: YARN-8073 > URL: https://issues.apache.org/jira/browse/YARN-8073 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8073.01.patch > > > Post YARN-6736, RM support writing into ats v1 and v2 by new configuration > setting _yarn.timeline-service.versions_. > Couple of issues observed in deployment are > # TimelineClientImpl doesn't honor newly added configuration rather it still > get version number from _yarn.timeline-service.version_. This causes not > writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 > value._ > # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled > doesn't honor timeline-service.versions. > # JobHistoryEventHandler#serviceInit(), line no 271 check for version number > rather than calling YarnConfiguration#timelineServiceV2Enabled > cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415543#comment-16415543 ] Rohith Sharma K S commented on YARN-6936: - As we discussed on weekly call, I have updated the patch that introduces two newer API in TimelineV2ClientImpl i.e putSubAppEntities and putSubAppEntitiesAsync. > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-6936: Attachment: YARN-6936.001.patch > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8048: Attachment: YARN-8048.004.patch > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch, > YARN-8048.003.patch, YARN-8048.004.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415394#comment-16415394 ] Rohith Sharma K S commented on YARN-8048: - updated patch fixing review comments. > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch, > YARN-8048.003.patch, YARN-8048.004.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415325#comment-16415325 ] Rohith Sharma K S commented on YARN-6936: - I caught up with some other issues. I will work on this and update the patch in a day. > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8048: Attachment: YARN-8048.003.patch > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch, > YARN-8048.003.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[
https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414910#comment-16414910
]
Rohith Sharma K S commented on YARN-8048:
-
Thanks [~leftnoteasy] for review..
bq. Why ServiceManager is being initialized in RM and SystemServiceManager is
being initialized in ApiServerWebApp? I guess adding to ApiServerWebApp is for
non-RM deployment of API server, correct? I think we don't have to do this in
the first phase.
Right, for non RM deployment scenarios to handle I added this. Though non RM
deployment not used, I think its better to keep it.
bq. And both of them are not added to CompositeService so stopService won't be
invoked during daemon shutdown. And is it enough to handle RM HA?
SystemServiceManager extends AbstractService. So it should be called. Here is
the log message from RM which I added
{noformat}
2018-03-26 13:30:55,815 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to
standby state
2018-03-26 13:30:55,816 INFO
org.apache.hadoop.yarn.service.client.SystemServiceManager: Stopping
org.apache.hadoop.yarn.service.client.SystemServiceManager
2018-03-26 13:30:55,818 WARN
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher:
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread
interrupted. Returning.
2018-03-26 13:30:55,819 INFO org.apache.hadoop.ipc.Server: Stopping server on
25001
{noformat}
bq. Why add a ServiceWebClient? Is it sufficient to use ServiceClient?
I took approach to go ahead with web service model rather than admin client
model because I wasn't sure how does ApiAdminClient behaves in secured
environment. ApiAdminClient does some local fs operations to read services
which is not required for us. And also ApiAdminClient doesn't have direct API
to launch services given Service in parameter. This API we need to introduce
it. I will check the feasibility and let me update the patch based on this.
> Support auto-spawning of admin configured services during bootstrap of
> rm/apiserver
> ---
>
> Key: YARN-8048
> URL: https://issues.apache.org/jira/browse/YARN-8048
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8048.001.patch, YARN-8048.002.patch
>
>
> Goal is to support auto-spawning of admin configured services during
> bootstrap of resourcemanager/apiserver.
> *Requirement:* Some of the services might required to be consumed by yarn
> itself ex: Hbase for atsv2. Instead of depending on user installed HBase or
> sometimes user may not required to install HBase at all, in such conditions
> running HBase app on YARN will help for ATSv2.
> Before YARN cluster is started, admin configure these services spec and place
> it in common location in HDFS. At the time of RM/apiServer bootstrap, these
> services will be submitted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
[ https://issues.apache.org/jira/browse/YARN-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8073: Attachment: YARN-8073.01.patch > TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration > - > > Key: YARN-8073 > URL: https://issues.apache.org/jira/browse/YARN-8073 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8073.01.patch > > > Post YARN-6736, RM support writing into ats v1 and v2 by new configuration > setting _yarn.timeline-service.versions_. > Couple of issues observed in deployment are > # TimelineClientImpl doesn't honor newly added configuration rather it still > get version number from _yarn.timeline-service.version_. This causes not > writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 > value._ > # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled > doesn't honor timeline-service.versions. > # JobHistoryEventHandler#serviceInit(), line no 271 check for version number > rather than calling YarnConfiguration#timelineServiceV2Enabled > cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
[ https://issues.apache.org/jira/browse/YARN-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S reassigned YARN-8073: --- Assignee: Rohith Sharma K S > TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration > - > > Key: YARN-8073 > URL: https://issues.apache.org/jira/browse/YARN-8073 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8073.01.patch > > > Post YARN-6736, RM support writing into ats v1 and v2 by new configuration > setting _yarn.timeline-service.versions_. > Couple of issues observed in deployment are > # TimelineClientImpl doesn't honor newly added configuration rather it still > get version number from _yarn.timeline-service.version_. This causes not > writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 > value._ > # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled > doesn't honor timeline-service.versions. > # JobHistoryEventHandler#serviceInit(), line no 271 check for version number > rather than calling YarnConfiguration#timelineServiceV2Enabled > cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8073) TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration
Rohith Sharma K S created YARN-8073: --- Summary: TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration Key: YARN-8073 URL: https://issues.apache.org/jira/browse/YARN-8073 Project: Hadoop YARN Issue Type: Bug Reporter: Rohith Sharma K S Post YARN-6736, RM support writing into ats v1 and v2 by new configuration setting _yarn.timeline-service.versions_. Couple of issues observed in deployment are # TimelineClientImpl doesn't honor newly added configuration rather it still get version number from _yarn.timeline-service.version_. This causes not writing into v1.5 API's even though _yarn.timeline-service.versions has 1.5 value._ # Similar line from 1st point, TimelineUtils#timelineServiceV1_5Enabled doesn't honor timeline-service.versions. # JobHistoryEventHandler#serviceInit(), line no 271 check for version number rather than calling YarnConfiguration#timelineServiceV2Enabled cc :/ [~agresch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8048: Attachment: YARN-8048.002.patch > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413696#comment-16413696 ] Rohith Sharma K S commented on YARN-8048: - Updated patch fixing findbug and test failures. > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch, YARN-8048.002.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8068) Upgrading apps from Hadoop 2.7 based clients to 2.8+ cause NPE in app timeline publish
[
https://issues.apache.org/jira/browse/YARN-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412846#comment-16412846
]
Rohith Sharma K S commented on YARN-8068:
-
+1lgtm. thanks Sunil for the patch.
> Upgrading apps from Hadoop 2.7 based clients to 2.8+ cause NPE in app
> timeline publish
> --
>
> Key: YARN-8068
> URL: https://issues.apache.org/jira/browse/YARN-8068
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 2.8.3
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Blocker
> Attachments: YARN-8068.001.patch
>
>
> [TimelineServiceV1Publisher|eclipse-javadoc:%E2%98%82=hadoop-yarn-server-resourcemanager/src%5C/main%5C/java%3Corg.apache.hadoop.yarn.server.resourcemanager.metrics%7BTimelineServiceV1Publisher.java%E2%98%83TimelineServiceV1Publisher].appCreated
> will cause NPE as we use like below
> {code:java}
> entityInfo.put(ApplicationMetricsConstants.APPLICATION_PRIORITY_INFO,
> app.getApplicationPriority().getPriority());{code}
> We have to handle this case while recovery.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410869#comment-16410869 ] Rohith Sharma K S commented on YARN-7581: - committing branch-2 patch shortly > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Fix For: 3.1.0, yarn-7055, 3.2.0 > > Attachments: YARN-7581-YARN-7055.04.patch, > YARN-7581-branch-2.05.patch, YARN-7581.00.patch, YARN-7581.01.patch, > YARN-7581.02.patch, YARN-7581.03.patch, YARN-7581.04.patch, YARN-7581.05.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8062) yarn rmadmin -getGroups returns group from which the user has been removed
[
https://issues.apache.org/jira/browse/YARN-8062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410853#comment-16410853
]
Rohith Sharma K S commented on YARN-8062:
-
+1 lgtm
> yarn rmadmin -getGroups returns group from which the user has been removed
> --
>
> Key: YARN-8062
> URL: https://issues.apache.org/jira/browse/YARN-8062
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Sunil G
>Priority: Critical
> Attachments: YARN-8062.001.patch, YARN-8062.002.patch
>
>
> {code:title= adding group hrt_yarn_rmadmin_test}
> sudo su - -c "groupadd hrt_yarn_rmadmin_test" root
> {code}
> {Code:title=adding user hrt_yarn_rmadmin_test to group hrt_yarn_rmadmin_test}
> sudo su - -c "useradd hrt_yarn_rmadmin_test -g hrt_yarn_rmadmin_test" root
> {Code}
> {Code:title= adding group hrt_yarn_rmadmin_test_group2 }
> sudo su - -c "groupadd hrt_yarn_rmadmin_test_group2" root
> {Code}
> {Code:title=adding user hrt_yarn_rmadmin_test to group
> hrt_yarn_rmadmin_test_group2}
> sudo su - -c "usermod -a -G hrt_yarn_rmadmin_test_group2
> hrt_yarn_rmadmin_test" root
> {Code}
> Refresh and getGroups
> {code}
> yarn rmadmin -refreshUserToGroupsMappings
> /usr/hdp/current/hadoop-yarn-client/bin/yarn rmadmin -getGroups
> hrt_yarn_rmadmin_test
> hrt_yarn_rmadmin_test : hrt_yarn_rmadmin_test hrt_yarn_rmadmin_test_group2
> {code}
> Delete group hrt_yarn_rmadmin_test_group2 from user hrt_yarn_rmadmin_test
> and refresh and do getGroups.
> We can still see group hrt_yarn_rmadmin_test_group2
> {code}
> sudo su - -c "gpasswd -d hrt_yarn_rmadmin_test hrt_yarn_rmadmin_test_group2"
> root
> {code}
> Removing user hrt_yarn_rmadmin_test from group hrt_yarn_rmadmin_test_group2
> {code}
> bash-4.2$ /usr/hdp/current/hadoop-yarn-client/bin/yarn rmadmin
> -refreshUserToGroupsMappings
> /usr/hdp/current/hadoop-yarn-client/bin/yarn rmadmin -getGroups
> hrt_yarn_rmadmin_test
> hrt_yarn_rmadmin_test : hrt_yarn_rmadmin_test hrt_yarn_rmadmin_test_group2
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7794) SLSRunner is not loading timeline service jars causing failure
[
https://issues.apache.org/jira/browse/YARN-7794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S reassigned YARN-7794:
---
Assignee: Yufei Gu (was: Rohith Sharma K S)
> SLSRunner is not loading timeline service jars causing failure
> --
>
> Key: YARN-7794
> URL: https://issues.apache.org/jira/browse/YARN-7794
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.1.0
>Reporter: Sunil G
>Assignee: Yufei Gu
>Priority: Blocker
> Attachments: YARN-7794.001.patch
>
>
> {code:java}
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollector
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 13 more
> Exception in thread "pool-2-thread-390" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yarn/server/timelineservice/collector/TimelineCollector
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:443)
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:321)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:641){code}
> We are getting this error while running SLS. new patch of timelineservice
> under share/hadoop/yarn is not loaded in SLS jvm (verified from slsrunner
> classpath)
> cc/ [~rohithsharma]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7794) SLSRunner is not loading timeline service jars causing failure
[
https://issues.apache.org/jira/browse/YARN-7794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410789#comment-16410789
]
Rohith Sharma K S commented on YARN-7794:
-
{quote}can I take this?
{quote}
Sure, assigned to you.
> SLSRunner is not loading timeline service jars causing failure
> --
>
> Key: YARN-7794
> URL: https://issues.apache.org/jira/browse/YARN-7794
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.1.0
>Reporter: Sunil G
>Assignee: Yufei Gu
>Priority: Blocker
> Attachments: YARN-7794.001.patch
>
>
> {code:java}
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollector
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 13 more
> Exception in thread "pool-2-thread-390" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yarn/server/timelineservice/collector/TimelineCollector
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:443)
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:321)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:641){code}
> We are getting this error while running SLS. new patch of timelineservice
> under share/hadoop/yarn is not loaded in SLS jvm (verified from slsrunner
> classpath)
> cc/ [~rohithsharma]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7032) [ATSv2] NPE while starting hbase co-processor when HBase authorization is enabled.
[
https://issues.apache.org/jira/browse/YARN-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129976#comment-16129976
]
Rohith Sharma K S edited comment on YARN-7032 at 3/22/18 6:17 PM:
--
Full stack trace is
{noformat}
2017-08-17 05:53:13,535 ERROR
[RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=16020]
coprocessor.CoprocessorHost: The coprocessor
org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
threw java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.hadoop.hbase.Tag.fromList(Tag.java:187)
at
org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor.prePut(FlowRunCoprocessor.java:102)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$30.call(RegionCoprocessorHost.java:885)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1660)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1734)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1692)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:881)
at
org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.java:3036)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3011)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2957)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:750)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:710)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2137)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32393)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
localhost,16020,1502949162490: The coprocessor
org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
threw java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.hadoop.hbase.Tag.fromList(Tag.java:187)
at
org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor.prePut(FlowRunCoprocessor.java:102)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$30.call(RegionCoprocessorHost.java:885)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1660)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1734)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1692)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:881)
at
org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.java:3036)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3011)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2957)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:750)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:710)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2137)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32393)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
2017-08-17 05:53:13,536 FATAL
[RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=16020]
regionserver.HRegionServer: RegionServer abort: loaded coprocessors are:
[org.apache.hadoop.hbase.security.access.AccessController,
org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor,
org.apache.hadoop.hbase.security.token.TokenProvider,
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,
org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
{noformat}
was (Author: rohithsharma):
Full stack trace is
{noformat}
2017-08-17 05:53:13,535 ERROR
[jira] [Updated] (YARN-7032) [ATSv2] NPE while starting hbase co-processor when HBase authorization is enabled.
[
https://issues.apache.org/jira/browse/YARN-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-7032:
Attachment: (was:
hbase-yarn-regionserver-ctr-e136-1513029738776-1405-01-02.hwx.site.log)
> [ATSv2] NPE while starting hbase co-processor when HBase authorization is
> enabled.
> --
>
> Key: YARN-7032
> URL: https://issues.apache.org/jira/browse/YARN-7032
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Critical
> Fix For: 3.1.0, 2.10.0, 3.0.1
>
> Attachments: YARN-7032.01.patch
>
>
> It is seen randomly that hbase co-processor fails to start with NPE. But
> again starting RegionServer, able to succeed in starting RS.
> {noformat}
> 2017-08-17 05:53:13,535 ERROR
> [RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=16020]
> coprocessor.CoprocessorHost: The coprocessor
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> threw java.lang.NullPointerException
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.Tag.fromList(Tag.java:187)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor.prePut(FlowRunCoprocessor.java:102)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$30.call(RegionCoprocessorHost.java:885)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409772#comment-16409772 ] Rohith Sharma K S commented on YARN-7581: - bq. The compilation fails even with JDK8 I agree, definitely I have done wrong branch compilation :-( I just did compile in branch-2 and failed. Thanks for pointing out, its high time for me to be more careful for branch-2 commits. > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Fix For: 3.1.0, yarn-7055, 3.2.0 > > Attachments: YARN-7581-YARN-7055.04.patch, > YARN-7581-branch-2.05.patch, YARN-7581.00.patch, YARN-7581.01.patch, > YARN-7581.02.patch, YARN-7581.03.patch, YARN-7581.04.patch, YARN-7581.05.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8063) DistributedShellTimelinePlugin wrongly check for entityId instead of entityType
[
https://issues.apache.org/jira/browse/YARN-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409531#comment-16409531
]
Rohith Sharma K S commented on YARN-8063:
-
Failed test case _TestDistributedShell#testDSShellWithoutDomainV2DefaultFlow_
is flaky and will be tracked as part of YARN-7771.
> DistributedShellTimelinePlugin wrongly check for entityId instead of
> entityType
> ---
>
> Key: YARN-8063
> URL: https://issues.apache.org/jira/browse/YARN-8063
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8063.01.patch
>
>
> DistributedShellTimelinePlugin#getTimelineEntityGroupId compare with entityId
> rather than entityType. This causes to fail to getTimelineEntityGroupId.
> {code}
> public Set getTimelineEntityGroupId(String entityId,
> String entityType) {
> if (ApplicationMaster.DSEntity.DS_CONTAINER.toString().equals(entityId)) {
> ContainerId containerId = ContainerId.fromString(entityId);
> ApplicationId appId = containerId.getApplicationAttemptId()
> .getApplicationId();
> return toEntityGroupId(appId.toString());
> }
> return null;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8063) DistributedShellTimelinePlugin wrongly check for entityId instead of entityType
[
https://issues.apache.org/jira/browse/YARN-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8063:
Attachment: YARN-8063.01.patch
> DistributedShellTimelinePlugin wrongly check for entityId instead of
> entityType
> ---
>
> Key: YARN-8063
> URL: https://issues.apache.org/jira/browse/YARN-8063
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8063.01.patch
>
>
> DistributedShellTimelinePlugin#getTimelineEntityGroupId compare with entityId
> rather than entityType. This causes to fail to getTimelineEntityGroupId.
> {code}
> public Set getTimelineEntityGroupId(String entityId,
> String entityType) {
> if (ApplicationMaster.DSEntity.DS_CONTAINER.toString().equals(entityId)) {
> ContainerId containerId = ContainerId.fromString(entityId);
> ApplicationId appId = containerId.getApplicationAttemptId()
> .getApplicationId();
> return toEntityGroupId(appId.toString());
> }
> return null;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8063) DistributedShellTimelinePlugin wrongly check for entityId instead of entityType
Rohith Sharma K S created YARN-8063:
---
Summary: DistributedShellTimelinePlugin wrongly check for entityId
instead of entityType
Key: YARN-8063
URL: https://issues.apache.org/jira/browse/YARN-8063
Project: Hadoop YARN
Issue Type: Bug
Reporter: Rohith Sharma K S
Assignee: Rohith Sharma K S
DistributedShellTimelinePlugin#getTimelineEntityGroupId compare with entityId
rather than entityType. This causes to fail to getTimelineEntityGroupId.
{code}
public Set getTimelineEntityGroupId(String entityId,
String entityType) {
if (ApplicationMaster.DSEntity.DS_CONTAINER.toString().equals(entityId)) {
ContainerId containerId = ContainerId.fromString(entityId);
ApplicationId appId = containerId.getApplicationAttemptId()
.getApplicationId();
return toEntityGroupId(appId.toString());
}
return null;
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Reopened] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S reopened YARN-7581: - Reopening to trigger branch-2 patch > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Fix For: 3.1.0, yarn-7055, 3.2.0 > > Attachments: YARN-7581-YARN-7055.04.patch, > YARN-7581-branch-2.05.patch, YARN-7581.00.patch, YARN-7581.01.patch, > YARN-7581.02.patch, YARN-7581.03.patch, YARN-7581.04.patch, YARN-7581.05.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408969#comment-16408969 ] Rohith Sharma K S commented on YARN-7581: - Thanks [~jlowe] for reporting and I apologies for missing this while committing the patch. I did compilation for branch-2 and committed. Looks my workspace was loaded with JDK1.8 which I missed while compiling branch-2. bq. Perhaps we should let jenkins run against the same patch uploaded for branch-2 henceforth before we commit it to branch-2. make sense, let's run all atsv2 patches for branch-2 as well similar to branch YARN-7055. > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Fix For: 3.1.0, yarn-7055, 3.2.0 > > Attachments: YARN-7581-YARN-7055.04.patch, YARN-7581.00.patch, > YARN-7581.01.patch, YARN-7581.02.patch, YARN-7581.03.patch, > YARN-7581.04.patch, YARN-7581.05.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408335#comment-16408335 ] Rohith Sharma K S commented on YARN-8048: - Attached the patch with following modifications # SystemServiceManager is a newly added class that is started at the bootstrap of RM/apiserver. ## Scan for configured common directory where files are kept under user hierarchy. Lets say, /tmp/system-service is common location then services to be launched should be kept under user directory i.e /tmp/system-service/user1/ ## Each user can have multiple spec files which will be submitted for that particular user. ## It directly submit configured spec file without any modifications. It is admin responsibility to validate the spec file and place it in common location. Any exception, log a error and continue for other service to launch. ## Scanning of FS directories will be done in background thread so that RM switch time doesn't impacted. ## RM integrated this service only if _yarn.webapp.api-service.enable_ is set to true. I will update on testing status once I verify in secure cluster. [~gsaha] [~billie.rina...@gmail.com] [~leftnoteasy] [~sunilg] please help to review the patch. > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8048: Attachment: YARN-8048.001.patch > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-8048.001.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-7581: Fix Version/s: 2.10.0 > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Fix For: 3.1.0, 2.10.0, yarn-7055, 3.2.0 > > Attachments: YARN-7581-YARN-7055.04.patch, YARN-7581.00.patch, > YARN-7581.01.patch, YARN-7581.02.patch, YARN-7581.03.patch, > YARN-7581.04.patch, YARN-7581.05.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407388#comment-16407388 ] Rohith Sharma K S commented on YARN-7581: - committed to branch-2 as well. thanks to [~haibochen] for the patch and thanks to [~vrushalic] for the review. > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Fix For: 3.1.0, 2.10.0, yarn-7055, 3.2.0 > > Attachments: YARN-7581-YARN-7055.04.patch, YARN-7581.00.patch, > YARN-7581.01.patch, YARN-7581.02.patch, YARN-7581.03.patch, > YARN-7581.04.patch, YARN-7581.05.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8053) Add hadoop-distcp in exclusion in hbase-server dependencies for timelineservice-hbase packages.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8053:
Fix Version/s: 3.1.0
> Add hadoop-distcp in exclusion in hbase-server dependencies for
> timelineservice-hbase packages.
> ---
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Fix For: 3.1.0, yarn-7055, 3.2.0
>
> Attachments: YARN-8053-YARN-7055.01.patch, YARN-8053.01.patch
>
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts for HBase-2 compilation.
> We see below error which tells that hbase-server has dependency on
> hadoop-distcp. We also need to exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8053) Add hadoop-distcp in exclusion in hbase-server dependencies for timelineservice-hbase packages.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8053:
Attachment: (was: YARN-8053.01.patch)
> Add hadoop-distcp in exclusion in hbase-server dependencies for
> timelineservice-hbase packages.
> ---
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8053-YARN-7055.01.patch, YARN-8053.01.patch
>
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts for HBase-2 compilation.
> We see below error which tells that hbase-server has dependency on
> hadoop-distcp. We also need to exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8053) Add hadoop-distcp in exclusion in hbase-server dependencies for timelineservice-hbase packages.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8053:
Attachment: YARN-8053.01.patch
> Add hadoop-distcp in exclusion in hbase-server dependencies for
> timelineservice-hbase packages.
> ---
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8053-YARN-7055.01.patch, YARN-8053.01.patch
>
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts for HBase-2 compilation.
> We see below error which tells that hbase-server has dependency on
> hadoop-distcp. We also need to exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8053) Add hadoop-distcp in exclusion in hbase-server dependencies for timelineservice-hbase packages.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8053:
Summary: Add hadoop-distcp in exclusion in hbase-server dependencies for
timelineservice-hbase packages. (was: Exclude hadoop-distcp dependencies in
hbase-server for timelineservice-hbase-client package.)
> Add hadoop-distcp in exclusion in hbase-server dependencies for
> timelineservice-hbase packages.
> ---
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8053-YARN-7055.01.patch, YARN-8053.01.patch
>
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts for HBase-2 compilation.
> We see below error which tells that hbase-server has dependency on
> hadoop-distcp. We also need to exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8053) Exclude hadoop-distcp dependencies in hbase-server for timelineservice-hbase-client package.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8053:
Description:
It is observed that when we change the version number of hadoop leading build
failure because of dependency resolution conflicts for HBase-2 compilation. We
see below error which tells that hbase-server has dependency on hadoop-distcp.
We also need to exclude hadoop-distcp from exclusion list.
{code}
07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
dependencies for project
org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT in
public
{code}
was:
It is observed that when we change the version number of hadoop leading build
failure because of dependency resolution conflicts. We see below error which
tells that hbase-server has dependency on hadoop-distcp. We also need to
exclude hadoop-distcp from exclusion list.
{code}
07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
dependencies for project
org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT in
public
{code}
> Exclude hadoop-distcp dependencies in hbase-server for
> timelineservice-hbase-client package.
>
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8053-YARN-7055.01.patch, YARN-8053.01.patch
>
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts for HBase-2 compilation.
> We see below error which tells that hbase-server has dependency on
> hadoop-distcp. We also need to exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8053) Exclude hadoop-distcp dependencies in hbase-server for timelineservice-hbase-client package.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8053:
Issue Type: Sub-task (was: Bug)
Parent: YARN-7213
> Exclude hadoop-distcp dependencies in hbase-server for
> timelineservice-hbase-client package.
>
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8053-YARN-7055.01.patch, YARN-8053.01.patch
>
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts. We see below error which
> tells that hbase-server has dependency on hadoop-distcp. We also need to
> exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8053) Exclude hadoop-distcp dependencies in hbase-server for timelineservice-hbase-client package.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405825#comment-16405825
]
Rohith Sharma K S commented on YARN-8053:
-
[~haibo.chen] would you review the patch please?
> Exclude hadoop-distcp dependencies in hbase-server for
> timelineservice-hbase-client package.
>
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8053-YARN-7055.01.patch, YARN-8053.01.patch
>
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts. We see below error which
> tells that hbase-server has dependency on hadoop-distcp. We also need to
> exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8053) Exclude hadoop-distcp dependencies in hbase-server for timelineservice-hbase-client package.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8053:
Attachment: YARN-8053-YARN-7055.01.patch
> Exclude hadoop-distcp dependencies in hbase-server for
> timelineservice-hbase-client package.
>
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8053-YARN-7055.01.patch, YARN-8053.01.patch
>
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts. We see below error which
> tells that hbase-server has dependency on hadoop-distcp. We also need to
> exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8053) Exclude hadoop-distcp dependencies in hbase-server for timelineservice-hbase-client package.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8053:
Attachment: YARN-8053.01.patch
> Exclude hadoop-distcp dependencies in hbase-server for
> timelineservice-hbase-client package.
>
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-8053.01.patch
>
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts. We see below error which
> tells that hbase-server has dependency on hadoop-distcp. We also need to
> exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8053) Exclude hadoop-distcp dependencies in hbase-server for timelineservice-hbase-client package.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405818#comment-16405818
]
Rohith Sharma K S commented on YARN-8053:
-
To reproduce the issue, follow the below steps
# Update trunk version to 3.3-snapshot using command *mvn versions:set
-DnewVersion=3.3.0-SNAPSHOT*
# Build hadoop with -Dhbase.profile=2.0, we see the error.
But in community it is likely to happen this error since default compilation is
hbase-1.2.6 which keeps hadoop-dist jar in repository. Later on this will be
cached and used.
> Exclude hadoop-distcp dependencies in hbase-server for
> timelineservice-hbase-client package.
>
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts. We see below error which
> tells that hbase-server has dependency on hadoop-distcp. We also need to
> exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8053) Exclude hadoop-distcp dependencies in hbase-server for timelineservice-hbase-client package.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-8053:
Description:
It is observed that when we change the version number of hadoop leading build
failure because of dependency resolution conflicts. We see below error which
tells that hbase-server has dependency on hadoop-distcp. We also need to
exclude hadoop-distcp from exclusion list.
{code}
07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
dependencies for project
org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT in
public
{code}
was:
It is observed that when we change the version number of hadoop leading build
failure because of dependency resolution conflicts. We see below error which
tells that hbase-server has dependency on hadoop-distcp. We also need to
exclude hadoop-distcp from exclusion list.
{code}
07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
dependencies for project
org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.0.0.3.0.0.0-1059:
Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.0.0.3.0.0.0-1059
in public
{code}
> Exclude hadoop-distcp dependencies in hbase-server for
> timelineservice-hbase-client package.
>
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts. We see below error which
> tells that hbase-server has dependency on hadoop-distcp. We also need to
> exclude hadoop-distcp from exclusion list.
> {code}
> 07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
> project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
> dependencies for project
> org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.3.0-SNAPSHOT:
> Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.3.0-SNAPSHOT
> in public
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8053) Exclude hadoop-distcp dependencies in hbase-server for timelineservice-hbase-client package.
[
https://issues.apache.org/jira/browse/YARN-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405804#comment-16405804
]
Rohith Sharma K S commented on YARN-8053:
-
The below dependency tree comparison shows that hbase-2 added dependency for
hadoop-dist.
* hbase-server:2.0.0-beta-1 introduced hadoop-distcp dependency which is
causing build failure.
{code}
[INFO] +- org.apache.hbase:hbase-server:jar:2.0.0-beta-1:provided
[INFO] | +- org.apache.hbase:hbase-http:jar:2.0.0-beta-1:provided
[INFO] | | +- org.eclipse.jetty:jetty-util-ajax:jar:9.3.19.v20170502:provided
[INFO] | | +- org.glassfish.jersey.core:jersey-server:jar:2.25.1:provided
[INFO] | | | +- org.glassfish.jersey.core:jersey-common:jar:2.25.1:provided
[INFO] | | | | +-
org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.25.1:provided
[INFO] | | | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:provided
[INFO] | | | +- org.glassfish.jersey.core:jersey-client:jar:2.25.1:provided
[INFO] | | | +-
org.glassfish.jersey.media:jersey-media-jaxb:jar:2.25.1:provided
[INFO] | | | +- javax.annotation:javax.annotation-api:jar:1.2:provided
[INFO] | | | +- org.glassfish.hk2:hk2-api:jar:2.5.0-b32:provided
[INFO] | | | | +- org.glassfish.hk2:hk2-utils:jar:2.5.0-b32:provided
[INFO] | | | | \-
org.glassfish.hk2.external:aopalliance-repackaged:jar:2.5.0-b32:provided
[INFO] | | | +-
org.glassfish.hk2.external:javax.inject:jar:2.5.0-b32:provided
[INFO] | | | +- org.glassfish.hk2:hk2-locator:jar:2.5.0-b32:provided
[INFO] | | | | \- org.javassist:javassist:jar:3.20.0-GA:provided
[INFO] | | | \- javax.validation:validation-api:jar:1.1.0.Final:provided
[INFO] | | \-
org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.25.1:provided
[INFO] | +- org.apache.hbase:hbase-procedure:jar:2.0.0-beta-1:provided
[INFO] | | \- org.apache.hbase:hbase-common:jar:tests:2.0.0-beta-1:test
[INFO] | +- org.apache.hbase:hbase-zookeeper:jar:2.0.0-beta-1:provided
[INFO] | +- org.apache.hbase:hbase-replication:jar:2.0.0-beta-1:provided
[INFO] | +- org.apache.hbase:hbase-metrics-api:jar:2.0.0-beta-1:compile
[INFO] | +- org.apache.hbase:hbase-metrics:jar:2.0.0-beta-1:compile
[INFO] | +- org.glassfish.web:javax.servlet.jsp:jar:2.3.2:provided
[INFO] | | +- org.glassfish:javax.el:jar:3.0.1-b10:provided (version selected
from constraint [3.0.0,))
[INFO] | | \- javax.servlet.jsp:javax.servlet.jsp-api:jar:2.3.1:provided
[INFO] | +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] | +- org.jamon:jamon-runtime:jar:2.4.1:provided
[INFO] | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:provided
[INFO] | +- org.apache.htrace:htrace-core:jar:3.2.0-incubating:provided
[INFO] | +- com.lmax:disruptor:jar:3.3.6:provided
[INFO] | \- org.apache.hadoop:hadoop-distcp:jar:3.2.0-SNAPSHOT:provided
[INFO] \- junit:junit:jar:4.11:test
[INFO] \- org.hamcrest:hamcrest-core:jar:1.3:test\
{code}
* hbase-server:1.2.6 doesn't have hadoop-distcp dependencies.
{code}
[INFO] +- org.apache.hbase:hbase-server:jar:1.2.6:provided
[INFO] | +- org.apache.hbase:hbase-procedure:jar:1.2.6:provided
[INFO] | | \- org.apache.hbase:hbase-common:jar:tests:1.2.6:test
[INFO] | +- org.apache.hbase:hbase-prefix-tree:jar:1.2.6:provided
[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:provided
[INFO] | +- org.apache.hbase:hbase-hadoop-compat:jar:1.2.6:provided
[INFO] | +- org.apache.hbase:hbase-hadoop2-compat:jar:1.2.6:provided
[INFO] | +- org.apache.commons:commons-math:jar:2.2:provided
[INFO] | +- org.mortbay.jetty:jsp-2.1:jar:6.1.14:provided
[INFO] | +- org.mortbay.jetty:jsp-api-2.1:jar:6.1.14:provided
[INFO] | +- org.mortbay.jetty:servlet-api-2.5:jar:6.1.14:provided
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile
[INFO] | +- tomcat:jasper-compiler:jar:5.5.23:provided
[INFO] | +- tomcat:jasper-runtime:jar:5.5.23:provided
[INFO] | | \- commons-el:commons-el:jar:1.0:provided
[INFO] | +- org.jamon:jamon-runtime:jar:2.4.1:provided
[INFO] | \- com.lmax:disruptor:jar:3.3.0:provided
[INFO] \- junit:junit:jar:4.11:test
[INFO]\- org.hamcrest:hamcrest-core:jar:1.3:test
{code}
> Exclude hadoop-distcp dependencies in hbase-server for
> timelineservice-hbase-client package.
>
>
> Key: YARN-8053
> URL: https://issues.apache.org/jira/browse/YARN-8053
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
>
> It is observed that when we change the version number of hadoop leading build
> failure because of dependency resolution conflicts. We see below error which
> tells that hbase-server has dependency on hadoop-distcp.
[jira] [Created] (YARN-8053) Exclude hadoop-distcp dependencies in hbase-server for timelineservice-hbase-client package.
Rohith Sharma K S created YARN-8053:
---
Summary: Exclude hadoop-distcp dependencies in hbase-server for
timelineservice-hbase-client package.
Key: YARN-8053
URL: https://issues.apache.org/jira/browse/YARN-8053
Project: Hadoop YARN
Issue Type: Bug
Reporter: Rohith Sharma K S
Assignee: Rohith Sharma K S
It is observed that when we change the version number of hadoop leading build
failure because of dependency resolution conflicts. We see below error which
tells that hbase-server has dependency on hadoop-distcp. We also need to
exclude hadoop-distcp from exclusion list.
{code}
07:42:36 2018/03/19 14:42:36 INFO: [ERROR] Failed to execute goal on
project hadoop-yarn-server-timelineservice-hbase-client: Could not resolve
dependencies for project
org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:jar:3.0.0.3.0.0.0-1059:
Could not find artifact org.apache.hadoop:hadoop-distcp:jar:3.0.0.3.0.0.0-1059
in public
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405793#comment-16405793 ] Rohith Sharma K S commented on YARN-7581: - [~vrushalic] Go ahead and commit it. thanks. > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7581-YARN-7055.04.patch, YARN-7581.00.patch, > YARN-7581.01.patch, YARN-7581.02.patch, YARN-7581.03.patch, > YARN-7581.04.patch, YARN-7581.05.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404458#comment-16404458 ] Rohith Sharma K S commented on YARN-7213: - The issue is specific to windows platform. Git has option to bypass this. See [git-pull-aborted-with-error-filename-too-long|https://stackoverflow.com/questions/21123415/git-pull-aborted-with-error-filename-too-long] > [Umbrella] Test and validate HBase-2.0.x with Atsv2 > --- > > Key: YARN-7213 > URL: https://issues.apache.org/jira/browse/YARN-7213 > Project: Hadoop YARN > Issue Type: Task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-7213.prelim.patch, YARN-7213.prelim.patch, > YARN-7213.wip.patch > > > Hbase-2.0.x officially support hadoop-alpha compilations. And also they are > getting ready for Hadoop-beta release so that HBase can release their > versions compatible with Hadoop-beta. So, this JIRA is to keep track of > HBase-2.0 integration issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
Rohith Sharma K S created YARN-8048: --- Summary: Support auto-spawning of admin configured services during bootstrap of rm/apiserver Key: YARN-8048 URL: https://issues.apache.org/jira/browse/YARN-8048 Project: Hadoop YARN Issue Type: Sub-task Reporter: Rohith Sharma K S Assignee: Rohith Sharma K S Goal is to support auto-spawning of admin configured services during bootstrap of resourcemanager/apiserver. *Requirement:* Some of the services might required to be consumed by yarn itself ex: Hbase for atsv2. Instead of depending on user installed HBase or sometimes user may not required to install HBase at all, in such conditions running HBase app on YARN will help for ATSv2. Before YARN cluster is started, admin configure these services spec and place it in common location in HDFS. At the time of RM/apiServer bootstrap, these services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404385#comment-16404385 ] Rohith Sharma K S commented on YARN-6936: - [~vrushalic] Would you take a look at this patch so that I can fix test failures along with your comments in next patch? > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401761#comment-16401761 ] Rohith Sharma K S commented on YARN-7581: - +1 for latest patch. > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7581.00.patch, YARN-7581.01.patch, > YARN-7581.02.patch, YARN-7581.03.patch, YARN-7581.04.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7933) [atsv2 read acls] Add TimelineWriter#writeDomain
[ https://issues.apache.org/jira/browse/YARN-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401510#comment-16401510 ] Rohith Sharma K S commented on YARN-7933: - Apache doc has good explanation for this. [TimelineServer|https://hadoop.apache.org/docs/r3.0.0/hadoop-yarn/hadoop-yarn-site/TimelineServer.html] > [atsv2 read acls] Add TimelineWriter#writeDomain > - > > Key: YARN-7933 > URL: https://issues.apache.org/jira/browse/YARN-7933 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vrushali C >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-7933.01.patch, YARN-7933.02.patch > > > > Add an API TimelineWriter#writeDomain for writing the domain info -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8006) Make Hbase-2 profile as default for YARN-7055 branch
[ https://issues.apache.org/jira/browse/YARN-8006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401507#comment-16401507 ] Rohith Sharma K S commented on YARN-8006: - thanks [~haibo.chen].. How about giving a YARN-7581 patch for YARN-7055 branch that test it.? > Make Hbase-2 profile as default for YARN-7055 branch > > > Key: YARN-8006 > URL: https://issues.apache.org/jira/browse/YARN-8006 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Fix For: yarn-7055 > > Attachments: YARN-8006-YARN-7055.001.patch, yetus-run.tar.gz > > > In last weekly call folks discussed that we should have separate branch with > hbase-2 as profile by default. Trunk default profile is hbase-1 which runs > all the tests under hbase-1 profile. But for hbase-2 profile tests are not > running. > As per the discussion, lets keep YARN-7055 branch for hbase-2 profile as > default. Any server side patches can be given to this branch as well which > runs tests for hbase-2 profile. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S reassigned YARN-6936: --- Assignee: Rohith Sharma K S > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8022) ResourceManager UI cluster/app/ page fails to render
[ https://issues.apache.org/jira/browse/YARN-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8022: Fix Version/s: 3.0.1 > ResourceManager UI cluster/app/ page fails to render > > > Key: YARN-8022 > URL: https://issues.apache.org/jira/browse/YARN-8022 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Reporter: Tarun Parimi >Assignee: Tarun Parimi >Priority: Blocker > Fix For: 3.1.0, 3.0.1, 3.2.0 > > Attachments: Screen Shot 2018-03-12 at 1.45.05 PM.png, > YARN-8022.001.patch, YARN-8022.002.patch > > > The page displays the message "Failed to read the attempts of the application" > > The following stack trace is observed in RM log. > org.apache.hadoop.yarn.server.webapp.AppBlock: Failed to read the attempts of > the application application_1520597233415_0002. > java.lang.NullPointerException > at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:283) > at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:280) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682) > at org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:279) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppBlock.render(RMAppBlock.java:71) > at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69) > at > org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79) > at org.apache.hadoop.yarn.webapp.View.render(View.java:235) > at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49) > at > org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117) > at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848) > at > org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71) > at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82) > at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.app(RmController.java:54) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
[ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-6936: Attachment: YARN-6936.000.patch > [Atsv2] Retrospect storing entities into sub application table from client > perspective > -- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Priority: Major > Attachments: YARN-6936.000.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs > user and submitted users are different. This holds good for Tez kind of use > cases. But AM runs as same as submitted user like MR also need to store > entities in sub application table so that it could read entities without > application id. > This would be a point of concern later stages when ATSv2 is deployed into > production. This JIRA is to retrospect decision of storing entities into sub > application table based on client side configuration driven rather than user > driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[
https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398132#comment-16398132
]
Rohith Sharma K S commented on YARN-7581:
-
Overall patch approach looks good. Some comments
TimelineEntityReader.java
l.no 211,
- method extractColumnFamiliesFromFiltersBasedOnFilters has duplicated code for
else if conditions.
- Can this _new String(cf, Charset.forName("UTF-8")_ be extracted to utils so
that we don't miss syntax anytime.
In all the *EntityReader implementations, by default cfsInFields added for INFO
family. Does it because of assumption that info field will be retrieved by
default all the time?
> HBase filters are not constructed correctly in ATSv2
>
>
> Key: YARN-7581
> URL: https://issues.apache.org/jira/browse/YARN-7581
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: ATSv2
>Affects Versions: 3.0.0-beta1
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7581.00.patch, YARN-7581.01.patch,
> YARN-7581.02.patch
>
>
> Post YARN-7346,
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters()
> start to fail when hbase.profile is set to 2.0)
> *Error Message*
> [ERROR] Failures:
> [ERROR]
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266
> expected:<2> but was:<0>
> [ERROR]
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523
> expected:<1> but was:<0>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[
https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398109#comment-16398109
]
Rohith Sharma K S commented on YARN-7581:
-
Looking into patch, I get one doubt that how does filter looks for combination
of metricstoretrieve and cofigstoretrieve?
Does it look like below ?
{code}
FilterList AND (2/2): [
FilterList AND (1/1): [
FilterList OR (2/2): [
SingleColumnValueFilter (c, config_param1, EQUAL, "value1"),
SingleColumnValueFilter (c, config_param1, EQUAL, "value3")
],
FilterList OR (2/2): [
SingleColumnValueFilter (m, metric_param1, EQUAL, "metric_value1"),
SingleColumnValueFilter (m, metric_param1, EQUAL, "metric_value3")
],
],
FilterList OR (2/2): [
FilterList AND (5/5): [
FamilyFilter (EQUAL, i),
QualifierFilter (NOT_EQUAL, e!),
QualifierFilter (NOT_EQUAL, i!),
QualifierFilter (NOT_EQUAL, s!),
QualifierFilter (NOT_EQUAL, r!)
],
FilterList AND (1/1): [
FamilyFilter(Equal, c)
],
FilterList AND (1/1): [
FamilyFilter(Equal, m)
],
]
]
{code}
> HBase filters are not constructed correctly in ATSv2
>
>
> Key: YARN-7581
> URL: https://issues.apache.org/jira/browse/YARN-7581
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: ATSv2
>Affects Versions: 3.0.0-beta1
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7581.00.patch, YARN-7581.01.patch,
> YARN-7581.02.patch
>
>
> Post YARN-7346,
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters()
> start to fail when hbase.profile is set to 2.0)
> *Error Message*
> [ERROR] Failures:
> [ERROR]
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266
> expected:<2> but was:<0>
> [ERROR]
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523
> expected:<1> but was:<0>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8022) ResourceManager UI cluster/app/ page fails to render
[ https://issues.apache.org/jira/browse/YARN-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398065#comment-16398065 ] Rohith Sharma K S commented on YARN-8022: - committing shortly > ResourceManager UI cluster/app/ page fails to render > > > Key: YARN-8022 > URL: https://issues.apache.org/jira/browse/YARN-8022 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Reporter: Tarun Parimi >Assignee: Tarun Parimi >Priority: Blocker > Attachments: Screen Shot 2018-03-12 at 1.45.05 PM.png, > YARN-8022.001.patch, YARN-8022.002.patch > > > The page displays the message "Failed to read the attempts of the application" > > The following stack trace is observed in RM log. > org.apache.hadoop.yarn.server.webapp.AppBlock: Failed to read the attempts of > the application application_1520597233415_0002. > java.lang.NullPointerException > at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:283) > at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:280) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682) > at org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:279) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppBlock.render(RMAppBlock.java:71) > at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69) > at > org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79) > at org.apache.hadoop.yarn.webapp.View.render(View.java:235) > at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49) > at > org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117) > at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848) > at > org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71) > at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82) > at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.app(RmController.java:54) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397342#comment-16397342 ] Rohith Sharma K S commented on YARN-7581: - I ran failing tests against hbase-2 profile with this patch and all tests passed. I will look into more details tomorrow. > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7581.00.patch, YARN-7581.01.patch, > YARN-7581.02.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8022) ResourceManager UI cluster/app/ page fails to render
[ https://issues.apache.org/jira/browse/YARN-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396813#comment-16396813 ] Rohith Sharma K S commented on YARN-8022: - thanks [~tarunparimi] for verification. [~sunilg] [~leftnoteasy] do you have any comments? Shall I commit this patch? > ResourceManager UI cluster/app/ page fails to render > > > Key: YARN-8022 > URL: https://issues.apache.org/jira/browse/YARN-8022 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Reporter: Tarun Parimi >Assignee: Tarun Parimi >Priority: Blocker > Attachments: Screen Shot 2018-03-12 at 1.45.05 PM.png, > YARN-8022.001.patch, YARN-8022.002.patch > > > The page displays the message "Failed to read the attempts of the application" > > The following stack trace is observed in RM log. > org.apache.hadoop.yarn.server.webapp.AppBlock: Failed to read the attempts of > the application application_1520597233415_0002. > java.lang.NullPointerException > at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:283) > at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:280) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682) > at org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:279) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppBlock.render(RMAppBlock.java:71) > at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69) > at > org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79) > at org.apache.hadoop.yarn.webapp.View.render(View.java:235) > at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49) > at > org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117) > at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848) > at > org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71) > at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82) > at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.app(RmController.java:54) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8022) ResourceManager UI cluster/app/ page fails to render
[
https://issues.apache.org/jira/browse/YARN-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396564#comment-16396564
]
Rohith Sharma K S commented on YARN-8022:
-
bq. Is there {color:#ff}any{color} behavior change comparing this to trunk
few days ago (before I revert these patches)?
yes, it is there. Before revert of HADOOP-14077 if callerUgi is null then
AuthenticationException is thrown. With this patch, we continue to retrieve
app report without ugi#doAs.
bq. Is there any behavior change comparing this to logics originally before
HADOOP-14077 get committed?
*NO*. This patch and before HADOOP-14077 get committed state should be same.
It looks confusion is because I said we need to revert the Appblock code in
earlier comments. Actually *NO*. We should *NOT* revert AppBlock modifications
after Owen commit. Sorry for the confusion.
YARN-7163 did some modifications to AppBlock class which got missed after Owen
commit that caused NPE. This patch bringing back those modifications without
any behavioral changes.
> ResourceManager UI cluster/app/ page fails to render
>
>
> Key: YARN-8022
> URL: https://issues.apache.org/jira/browse/YARN-8022
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Reporter: Tarun Parimi
>Assignee: Tarun Parimi
>Priority: Blocker
> Attachments: Screen Shot 2018-03-12 at 1.45.05 PM.png,
> YARN-8022.001.patch, YARN-8022.002.patch
>
>
> The page displays the message "Failed to read the attempts of the application"
>
> The following stack trace is observed in RM log.
> org.apache.hadoop.yarn.server.webapp.AppBlock: Failed to read the attempts of
> the application application_1520597233415_0002.
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:283)
> at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:280)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:279)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppBlock.render(RMAppBlock.java:71)
> at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
> at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
> at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
> at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
> at
> org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
> at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
> at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.app(RmController.java:54)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5627) [Atsv2] Support streaming reader API to fetch entities
[ https://issues.apache.org/jira/browse/YARN-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396547#comment-16396547 ] Rohith Sharma K S commented on YARN-5627: - Alternative to this JIRA, we did support for FROM_ID in all the REST queries which helps for pagination. See YARN-6027 YARN-5585 YARN-6064 YARN-6047. > [Atsv2] Support streaming reader API to fetch entities > -- > > Key: YARN-5627 > URL: https://issues.apache.org/jira/browse/YARN-5627 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelinereader >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > > There is no limit for size of TimelineEntitie object. It can be varied from > KB's to MB. While reading entities list, it would be an potential issue that > TimeLineReder would go into OOM situation based on the entity size and limit. > Proposal is to support an streaming API to read entities list. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8022) ResourceManager UI cluster/app/ page fails to render
[ https://issues.apache.org/jira/browse/YARN-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395029#comment-16395029 ] Rohith Sharma K S commented on YARN-8022: - However in secure cluster, unauthorized user will be going into else part since callerUGI will NOT be null at all. If user is unauthorized then this user will be blocked in AuthenticationFilter itself. CallerUGI can be null only in non-secure cluster. So we can proceed ahead with this modification. cc : [~leftnoteasy] [~sunilg] > ResourceManager UI cluster/app/ page fails to render > > > Key: YARN-8022 > URL: https://issues.apache.org/jira/browse/YARN-8022 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Affects Versions: 3.1.0, 3.2.0 >Reporter: Tarun Parimi >Assignee: Tarun Parimi >Priority: Blocker > Attachments: Screen Shot 2018-03-12 at 1.45.05 PM.png, > YARN-8022.001.patch > > > The page displays the message "Failed to read the attempts of the application" > > The following stack trace is observed in RM log. > org.apache.hadoop.yarn.server.webapp.AppBlock: Failed to read the attempts of > the application application_1520597233415_0002. > java.lang.NullPointerException > at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:283) > at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:280) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682) > at org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:279) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppBlock.render(RMAppBlock.java:71) > at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69) > at > org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79) > at org.apache.hadoop.yarn.webapp.View.render(View.java:235) > at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49) > at > org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117) > at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848) > at > org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71) > at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82) > at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.app(RmController.java:54) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8022) ResourceManager UI cluster/app/ page fails to render
[
https://issues.apache.org/jira/browse/YARN-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395006#comment-16395006
]
Rohith Sharma K S edited comment on YARN-8022 at 3/12/18 9:56 AM:
--
I would suggest following changes.
# Revert AppBlock as this patch is doing.
# Additionally, in line no. 145, can you add below code. Get app attempt report
directly if caller ugi is null.
{code}
+ if (callerUGI == null) {
+attempts = getApplicationAttemptsReport(request);
+ } else {
{code}
was (Author: rohithsharma):
I would suggest following changes.
# Revert AppBlock as this patch is doing.
# Additionally, in line no. 155, can you add below code. Get app attempt report
directly if caller ugi is null.
{code}
+ if (callerUGI == null) {
+attempts = getApplicationAttemptsReport(request);
+ } else {
{code}
> ResourceManager UI cluster/app/ page fails to render
>
>
> Key: YARN-8022
> URL: https://issues.apache.org/jira/browse/YARN-8022
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Affects Versions: 3.1.0, 3.2.0
>Reporter: Tarun Parimi
>Assignee: Tarun Parimi
>Priority: Blocker
> Attachments: Screen Shot 2018-03-12 at 1.45.05 PM.png,
> YARN-8022.001.patch
>
>
> The page displays the message "Failed to read the attempts of the application"
>
> The following stack trace is observed in RM log.
> org.apache.hadoop.yarn.server.webapp.AppBlock: Failed to read the attempts of
> the application application_1520597233415_0002.
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:283)
> at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:280)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:279)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppBlock.render(RMAppBlock.java:71)
> at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
> at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
> at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
> at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
> at
> org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
> at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
> at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.app(RmController.java:54)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8022) ResourceManager UI cluster/app/ page fails to render
[
https://issues.apache.org/jira/browse/YARN-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395006#comment-16395006
]
Rohith Sharma K S commented on YARN-8022:
-
I would suggest following changes.
# Revert AppBlock as this patch is doing.
# Additionally, in line no. 155, can you add below code. Get app attempt report
directly if caller ugi is null.
{code}
+ if (callerUGI == null) {
+attempts = getApplicationAttemptsReport(request);
+ } else {
{code}
> ResourceManager UI cluster/app/ page fails to render
>
>
> Key: YARN-8022
> URL: https://issues.apache.org/jira/browse/YARN-8022
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Affects Versions: 3.1.0, 3.2.0
>Reporter: Tarun Parimi
>Assignee: Tarun Parimi
>Priority: Blocker
> Attachments: Screen Shot 2018-03-12 at 1.45.05 PM.png,
> YARN-8022.001.patch
>
>
> The page displays the message "Failed to read the attempts of the application"
>
> The following stack trace is observed in RM log.
> org.apache.hadoop.yarn.server.webapp.AppBlock: Failed to read the attempts of
> the application application_1520597233415_0002.
> java.lang.NullPointerException
> at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:283)
> at org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:280)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:279)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppBlock.render(RMAppBlock.java:71)
> at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
> at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
> at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
> at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
> at
> org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
> at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
> at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.app(RmController.java:54)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394967#comment-16394967 ] Rohith Sharma K S commented on YARN-7581: - [~haibochen] would you update the patch fixing findbugs? > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7581.00.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
