[jira] [Assigned] (YARN-3560) Not able to navigate to the cluster from tracking url (proxy) generated after submission of job
[ https://issues.apache.org/jira/browse/YARN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan reassigned YARN-3560: -- Assignee: (was: Mohammad Shahid Khan) > Not able to navigate to the cluster from tracking url (proxy) generated after > submission of job > --- > > Key: YARN-3560 > URL: https://issues.apache.org/jira/browse/YARN-3560 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.7.0 >Reporter: Anushri >Priority: Minor > Attachments: YARN-3560.patch, YARN-3560.patch > > > a standalone web proxy server is enabled in the cluster > when a job is submitted the url generated contains proxy > track this url > in the web page , if we try to navigate to the cluster links [about. > applications, or scheduler] it gets redirected to some default port instead > of actual RM web port configured > as such it throws "webpage not available" -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-4441) Kill application request from the webservice(ui) is showing success even for the finished applications
[ https://issues.apache.org/jira/browse/YARN-4441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan reassigned YARN-4441: -- Assignee: (was: Mohammad Shahid Khan) > Kill application request from the webservice(ui) is showing success even for > the finished applications > -- > > Key: YARN-4441 > URL: https://issues.apache.org/jira/browse/YARN-4441 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha1 >Reporter: Mohammad Shahid Khan > > If the application is already finished ie either failled, killed, or succeded > the kill operation should not be logged as success. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-4442) create new application operation from the Rest API should be logged in the audit log
[
https://issues.apache.org/jira/browse/YARN-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan reassigned YARN-4442:
--
Assignee: (was: Mohammad Shahid Khan)
> create new application operation from the Rest API should be logged in the
> audit log
>
>
> Key: YARN-4442
> URL: https://issues.apache.org/jira/browse/YARN-4442
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.0.0-alpha1
>Reporter: Mohammad Shahid Khan
>
> create new application operation from the Rest API
> ("/ws/v1/cluster/apps/new-application") should be logged in
> the audit log
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-4490) RM restart the finished app shows wrong Diagnostics status
[ https://issues.apache.org/jira/browse/YARN-4490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan reassigned YARN-4490: -- Assignee: (was: Mohammad Shahid Khan) > RM restart the finished app shows wrong Diagnostics status > -- > > Key: YARN-4490 > URL: https://issues.apache.org/jira/browse/YARN-4490 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Reporter: Mohammad Shahid Khan > > RM restart the finished app shows wrong Diagnostics status. > Preconditions: > RM recovery enable true. > Steps: > 1. run an application, wait application is finished. > 2. Restart the RM > 3. Check the application status is RM web UI > Issue: > Check the Diagnostic message: Attempt recovered after RM restart. > Expected: > The Diagnostic message should be available only for the application waiting > for allocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-4521) Test Failure: TestJobHistoryEventHandler.testTimelineEventHandling
[ https://issues.apache.org/jira/browse/YARN-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan resolved YARN-4521. Resolution: Duplicate Duplicate of https://issues.apache.org/jira/browse/MAPREDUCE-6593 > Test Failure: TestJobHistoryEventHandler.testTimelineEventHandling > -- > > Key: YARN-4521 > URL: https://issues.apache.org/jira/browse/YARN-4521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Affects Versions: 3.0.0 > Environment: jdk1.8.0_66 >Reporter: Mohammad Shahid Khan > > in org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler > testTimelineEventHandling(org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler) > Time elapsed: 5.292 sec <<< ERROR! > java.lang.NullPointerException: null > at > org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:331) > at > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processEventForTimelineServer(JobHistoryEventHandler.java:1015) > at > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:586) > at > org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler.handleEvent(TestJobHistoryEventHandler.java:722) > at > org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler.testTimelineEventHandling(TestJobHistoryEventHandler.java:510) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-4521) Test Failure: TestJobHistoryEventHandler.testTimelineEventHandling
Mohammad Shahid Khan created YARN-4521: -- Summary: Test Failure: TestJobHistoryEventHandler.testTimelineEventHandling Key: YARN-4521 URL: https://issues.apache.org/jira/browse/YARN-4521 Project: Hadoop YARN Issue Type: Sub-task Components: yarn Affects Versions: 3.0.0 Environment: jdk1.8.0_66 Reporter: Mohammad Shahid Khan in org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler testTimelineEventHandling(org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler) Time elapsed: 5.292 sec <<< ERROR! java.lang.NullPointerException: null at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:331) at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processEventForTimelineServer(JobHistoryEventHandler.java:1015) at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:586) at org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler.handleEvent(TestJobHistoryEventHandler.java:722) at org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler.testTimelineEventHandling(TestJobHistoryEventHandler.java:510) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4482) Default values of several config parameters are missing
[
https://issues.apache.org/jira/browse/YARN-4482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-4482:
---
Assignee: (was: Mohammad Shahid Khan)
> Default values of several config parameters are missing
>
>
> Key: YARN-4482
> URL: https://issues.apache.org/jira/browse/YARN-4482
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: client
>Affects Versions: 2.6.2, 2.6.3
>Reporter: Tianyin Xu
>Priority: Minor
>
> In {{yarn-default.xml}}, the default values of the following parameters are
> commented out,
> {{yarn.client.failover-max-attempts}}
> {{yarn.client.failover-sleep-base-ms}}
> {{yarn.client.failover-sleep-max-ms}}
> Are these default values changed (I suppose so)? If so, we should update the
> new ones in {{yarn-default.xml}}. Right now, I don't know the real "default"
> values...
> (yarn-default.xml)
> https://hadoop.apache.org/docs/r2.6.2/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
> https://hadoop.apache.org/docs/r2.6.3/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4482) Default values of several config parameters are missing
[
https://issues.apache.org/jira/browse/YARN-4482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072484#comment-15072484
]
Mohammad Shahid Khan commented on YARN-4482:
Hi [#Tianyin Xu] agree with you.
we can mark this jira won't fix.
> Default values of several config parameters are missing
>
>
> Key: YARN-4482
> URL: https://issues.apache.org/jira/browse/YARN-4482
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: client
>Affects Versions: 2.6.2, 2.6.3
>Reporter: Tianyin Xu
>Priority: Minor
>
> In {{yarn-default.xml}}, the default values of the following parameters are
> commented out,
> {{yarn.client.failover-max-attempts}}
> {{yarn.client.failover-sleep-base-ms}}
> {{yarn.client.failover-sleep-max-ms}}
> Are these default values changed (I suppose so)? If so, we should update the
> new ones in {{yarn-default.xml}}. Right now, I don't know the real "default"
> values...
> (yarn-default.xml)
> https://hadoop.apache.org/docs/r2.6.2/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
> https://hadoop.apache.org/docs/r2.6.3/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4109) Exception on RM scheduler page loading with labels
[
https://issues.apache.org/jira/browse/YARN-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068001#comment-15068001
]
Mohammad Shahid Khan commented on YARN-4109:
UT Failure are unrelated to current patch.
UT addition/modification not required only small UI change.
> Exception on RM scheduler page loading with labels
> --
>
> Key: YARN-4109
> URL: https://issues.apache.org/jira/browse/YARN-4109
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Mohammad Shahid Khan
>Priority: Minor
> Attachments: YARN-4109_1.patch
>
>
> Configure node label and load scheduler Page
> On each reload of the page the below exception gets thrown in logs
> {code}
> 2015-09-03 11:27:08,544 ERROR org.apache.hadoop.yarn.webapp.Dispatcher: error
> handling URI: /cluster/scheduler
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:153)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
> at
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
> at
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:139)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
> at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:663)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:615)
> at
> org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter.doFilter(RMAuthenticationFilter.java:82)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1211)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
>
[jira] [Assigned] (YARN-4482) Default values of several config parameters are missing
[
https://issues.apache.org/jira/browse/YARN-4482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan reassigned YARN-4482:
--
Assignee: Mohammad Shahid Khan
> Default values of several config parameters are missing
>
>
> Key: YARN-4482
> URL: https://issues.apache.org/jira/browse/YARN-4482
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: client
>Affects Versions: 2.6.2, 2.6.3
>Reporter: Tianyin Xu
>Assignee: Mohammad Shahid Khan
>Priority: Minor
>
> In {{yarn-default.xml}}, the default values of the following parameters are
> commented out,
> {{yarn.client.failover-max-attempts}}
> {{yarn.client.failover-sleep-base-ms}}
> {{yarn.client.failover-sleep-max-ms}}
> Are these default values changed (I suppose so)? If so, we should update the
> new ones in {{yarn-default.xml}}. Right now, I don't know the real "default"
> values...
> (yarn-default.xml)
> https://hadoop.apache.org/docs/r2.6.2/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
> https://hadoop.apache.org/docs/r2.6.3/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (YARN-4490) RM restart the finished app shows wrong Diagnostics status
Mohammad Shahid Khan created YARN-4490: -- Summary: RM restart the finished app shows wrong Diagnostics status Key: YARN-4490 URL: https://issues.apache.org/jira/browse/YARN-4490 Project: Hadoop YARN Issue Type: Sub-task Reporter: Mohammad Shahid Khan Assignee: Mohammad Shahid Khan RM restart the finished app shows wrong Diagnostics status. Preconditions: RM recovery enable true. Steps: 1. run an application, wait application is finished. 2. Restart the RM 3. Check the application status is RM web UI Issue: Check the Diagnostic message: Attempt recovered after RM restart. Expected: The Diagnostic message should be available only for the application waiting for allocation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4490) RM restart the finished app shows wrong Diagnostics status
[ https://issues.apache.org/jira/browse/YARN-4490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065675#comment-15065675 ] Mohammad Shahid Khan commented on YARN-4490: Hi [#Naganarasimha G R] any thoughts? > RM restart the finished app shows wrong Diagnostics status > -- > > Key: YARN-4490 > URL: https://issues.apache.org/jira/browse/YARN-4490 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Reporter: Mohammad Shahid Khan >Assignee: Mohammad Shahid Khan > > RM restart the finished app shows wrong Diagnostics status. > Preconditions: > RM recovery enable true. > Steps: > 1. run an application, wait application is finished. > 2. Restart the RM > 3. Check the application status is RM web UI > Issue: > Check the Diagnostic message: Attempt recovered after RM restart. > Expected: > The Diagnostic message should be available only for the application waiting > for allocation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4490) RM restart the finished app shows wrong Diagnostics status
[ https://issues.apache.org/jira/browse/YARN-4490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065706#comment-15065706 ] Mohammad Shahid Khan commented on YARN-4490: Yes, [#Varun Saxena], if the application is recovered then diagnostic message does not add any value. And the message "Attempt recovered after RM restart", is also confusing for the app which is already finished. > RM restart the finished app shows wrong Diagnostics status > -- > > Key: YARN-4490 > URL: https://issues.apache.org/jira/browse/YARN-4490 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Reporter: Mohammad Shahid Khan >Assignee: Mohammad Shahid Khan > > RM restart the finished app shows wrong Diagnostics status. > Preconditions: > RM recovery enable true. > Steps: > 1. run an application, wait application is finished. > 2. Restart the RM > 3. Check the application status is RM web UI > Issue: > Check the Diagnostic message: Attempt recovered after RM restart. > Expected: > The Diagnostic message should be available only for the application waiting > for allocation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4490) RM restart the finished app shows wrong Diagnostics status
[ https://issues.apache.org/jira/browse/YARN-4490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065676#comment-15065676 ] Mohammad Shahid Khan commented on YARN-4490: in my view for finished applications the diagnostic message could be empty. > RM restart the finished app shows wrong Diagnostics status > -- > > Key: YARN-4490 > URL: https://issues.apache.org/jira/browse/YARN-4490 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Reporter: Mohammad Shahid Khan >Assignee: Mohammad Shahid Khan > > RM restart the finished app shows wrong Diagnostics status. > Preconditions: > RM recovery enable true. > Steps: > 1. run an application, wait application is finished. > 2. Restart the RM > 3. Check the application status is RM web UI > Issue: > Check the Diagnostic message: Attempt recovered after RM restart. > Expected: > The Diagnostic message should be available only for the application waiting > for allocation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4490) RM restart the finished app shows wrong Diagnostics status
[ https://issues.apache.org/jira/browse/YARN-4490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065715#comment-15065715 ] Mohammad Shahid Khan commented on YARN-4490: Hi [#Nagabarasimhar G R], this is not related to YARN-3946. My intension is if the application is finished then at the time of recovery the message "Attempt recovered after RM restart" is not needed. > RM restart the finished app shows wrong Diagnostics status > -- > > Key: YARN-4490 > URL: https://issues.apache.org/jira/browse/YARN-4490 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Reporter: Mohammad Shahid Khan >Assignee: Mohammad Shahid Khan > > RM restart the finished app shows wrong Diagnostics status. > Preconditions: > RM recovery enable true. > Steps: > 1. run an application, wait application is finished. > 2. Restart the RM > 3. Check the application status is RM web UI > Issue: > Check the Diagnostic message: Attempt recovered after RM restart. > Expected: > The Diagnostic message should be available only for the application waiting > for allocation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (YARN-3560) Not able to navigate to the cluster from tracking url (proxy) generated after submission of job
[ https://issues.apache.org/jira/browse/YARN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan reassigned YARN-3560: -- Assignee: Mohammad Shahid Khan > Not able to navigate to the cluster from tracking url (proxy) generated after > submission of job > --- > > Key: YARN-3560 > URL: https://issues.apache.org/jira/browse/YARN-3560 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.7.0 >Reporter: Anushri >Assignee: Mohammad Shahid Khan >Priority: Minor > Attachments: YARN-3560.patch, YARN-3560.patch > > > a standalone web proxy server is enabled in the cluster > when a job is submitted the url generated contains proxy > track this url > in the web page , if we try to navigate to the cluster links [about. > applications, or scheduler] it gets redirected to some default port instead > of actual RM web port configured > as such it throws "webpage not available" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4441) Kill application request from the webservice(ui) is showing success even for the finished applications
[
https://issues.apache.org/jira/browse/YARN-4441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055734#comment-15055734
]
Mohammad Shahid Khan commented on YARN-4441:
Hi [#Varun Vasudev] why to send the kill request if the application is already
finished?
Please check the ApplicationCLI killApplication API , we have similar check
before invoking the RPC call to kill the app.
{Code}
if (appReport.getYarnApplicationState() == YarnApplicationState.FINISHED
|| appReport.getYarnApplicationState() == YarnApplicationState.KILLED
|| appReport.getYarnApplicationState() == YarnApplicationState.FAILED) {
sysout.println("Application " + applicationId + " has already finished ");
} else {
sysout.println("Killing application " + applicationId);
client.killApplication(appId);
}
{Code}
> Kill application request from the webservice(ui) is showing success even for
> the finished applications
> --
>
> Key: YARN-4441
> URL: https://issues.apache.org/jira/browse/YARN-4441
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Mohammad Shahid Khan
>Assignee: Mohammad Shahid Khan
>
> If the application is already finished ie either failled, killed, or succeded
> the kill operation should not be logged as success.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4441) Kill application request from the webservice(ui) is showing success even for the finished applications
[
https://issues.apache.org/jira/browse/YARN-4441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055504#comment-15055504
]
Mohammad Shahid Khan commented on YARN-4441:
Thanks [#Sunil G] and [#Rohith Sharma K S] for the analysis.
Have discussed the same with [#Rohith Sharma K S] and want to clarify that
we do not want to throw the new exception just want the avoid the killApp call
in case the application's state is already stored in the app store.
In RMWebServices updateAppState API
//currentlty we are ignoring the killAPP call for the application which are
already killed.
{code}
if (!app.getState().toString().equals(targetState.getState())) {
// user is attempting to change state. right we only
// allow users to kill the app
if
(targetState.getState().equals(YarnApplicationState.KILLED.toString())) {
return killApp(app, callerUGI, hsr);
}
throw new BadRequestException("Only '"
+ YarnApplicationState.KILLED.toString()
+ "' is allowed as a target state.");
}
{code}
#My proposal is to ignore the call also for the applications whose state's are
stored to the application store.
{code}
if (!app.getState().toString().equals(targetState.getState()) &&
!(app.isAppFinalStateStored())) {
// user is attempting to change state. right we only
// allow users to kill the app
if
(targetState.getState().equals(YarnApplicationState.KILLED.toString())) {
return killApp(app, callerUGI, hsr);
}
throw new BadRequestException("Only '"
+ YarnApplicationState.KILLED.toString()
+ "' is allowed as a target state.");
}
{code}
> Kill application request from the webservice(ui) is showing success even for
> the finished applications
> --
>
> Key: YARN-4441
> URL: https://issues.apache.org/jira/browse/YARN-4441
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Mohammad Shahid Khan
>Assignee: Mohammad Shahid Khan
>
> If the application is already finished ie either failled, killed, or succeded
> the kill operation should not be logged as success.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4441) Kill application request from the webservice(ui) is showing success even for the finished applications
[
https://issues.apache.org/jira/browse/YARN-4441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052256#comment-15052256
]
Mohammad Shahid Khan commented on YARN-4441:
If an application is already finished, then the killing the application does
does not make any sense.
>From UI we can call the kill operation and that event is logged as below.
And in the audit log we are getting the success message for the operation Kill
Application Request is successful.
in the below code the problem is there at the first if check.
Suppose application_xyz is already finished and when the kill request will come
the target state will be KILLED and actual will be finished, the first if check
will be true.
Second if check also will be true as the target is KILLED and killApp method
will be called and same will log the
USER=dr.whoOPERATION=Kill Application Request TARGET=ClientRMService
RESULT=SUCCESS APPID=application_xyz | RMAuditLogger.java:91
USER=dr.whoOPERATION=Kill Application Request TARGET=RMWebService
RESULT=SUCCESS APPID=application_xyz | MAuditLogger.java:91
{Code}
if (!app.getState().toString().equals(targetState.getState())) {
// user is attempting to change state. right we only
// allow users to kill the app
if
(targetState.getState().equals(YarnApplicationState.KILLED.toString())) {
return killApp(app, callerUGI, hsr);
}
throw new BadRequestException("Only '"
+ YarnApplicationState.KILLED.toString()
+ "' is allowed as a target state.");
}
{Code}
I think we should not allow to call the killApp if the application is already
finished.
> Kill application request from the webservice(ui) is showing success even for
> the finished applications
> --
>
> Key: YARN-4441
> URL: https://issues.apache.org/jira/browse/YARN-4441
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Mohammad Shahid Khan
>Assignee: Mohammad Shahid Khan
>
> If the application is already finished ie either failled, killed, or succeded
> the kill operation should not be logged as success.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (YARN-4442) create new application operation from the Rest API should be logged in the audit log
Mohammad Shahid Khan created YARN-4442:
--
Summary: create new application operation from the Rest API should
be logged in the audit log
Key: YARN-4442
URL: https://issues.apache.org/jira/browse/YARN-4442
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 3.0.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
create new application operation from the Rest API
("/ws/v1/cluster/apps/new-application") should be logged in
the audit log
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (YARN-4441) Kill application request from the webservice(ui) is showing success even for the finished applications
Mohammad Shahid Khan created YARN-4441: -- Summary: Kill application request from the webservice(ui) is showing success even for the finished applications Key: YARN-4441 URL: https://issues.apache.org/jira/browse/YARN-4441 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.0.0 Reporter: Mohammad Shahid Khan Assignee: Mohammad Shahid Khan If the application is already finished ie either failled, killed, or succeded the kill operation should not be logged as success. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4109) Exception on RM scheduler page loading with labels
[
https://issues.apache.org/jira/browse/YARN-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-4109:
---
Attachment: YARN-4109_1.patch
> Exception on RM scheduler page loading with labels
> --
>
> Key: YARN-4109
> URL: https://issues.apache.org/jira/browse/YARN-4109
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Mohammad Shahid Khan
> Attachments: YARN-4109_1.patch
>
>
> Configure node label and load scheduler Page
> On each reload of the page the below exception gets thrown in logs
> {code}
> 2015-09-03 11:27:08,544 ERROR org.apache.hadoop.yarn.webapp.Dispatcher: error
> handling URI: /cluster/scheduler
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:153)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
> at
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
> at
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:139)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
> at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:663)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:615)
> at
> org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter.doFilter(RMAuthenticationFilter.java:82)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1211)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>
[jira] [Commented] (YARN-4109) Exception on RM scheduler page loading with labels
[
https://issues.apache.org/jira/browse/YARN-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042247#comment-15042247
]
Mohammad Shahid Khan commented on YARN-4109:
This issue is happening when the node-labels is enabled, one of the tag is not
closed properly.
Submitting the patch for the same.
> Exception on RM scheduler page loading with labels
> --
>
> Key: YARN-4109
> URL: https://issues.apache.org/jira/browse/YARN-4109
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Mohammad Shahid Khan
>
> Configure node label and load scheduler Page
> On each reload of the page the below exception gets thrown in logs
> {code}
> 2015-09-03 11:27:08,544 ERROR org.apache.hadoop.yarn.webapp.Dispatcher: error
> handling URI: /cluster/scheduler
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:153)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
> at
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
> at
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:139)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
> at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:663)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:615)
> at
> org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter.doFilter(RMAuthenticationFilter.java:82)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1211)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>
[jira] [Commented] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id > 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037736#comment-15037736 ] Mohammad Shahid Khan commented on YARN-3840: +1 Latest path looks good to me, thanks Varun Saxena, for handling the performance issue > Resource Manager web ui issue when sorting application by id (with > application having id > ) > > > Key: YARN-3840 > URL: https://issues.apache.org/jira/browse/YARN-3840 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.7.0 >Reporter: LINTE >Assignee: Varun Saxena > Fix For: 2.8.0, 2.7.3 > > Attachments: RMApps.png, RMApps_Sorted.png, YARN-3840-1.patch, > YARN-3840-2.patch, YARN-3840-3.patch, YARN-3840-4.patch, YARN-3840-5.patch, > YARN-3840-6.patch, YARN-3840.reopened.001.patch, yarn-3840-7.patch > > > On the WEBUI, the global main view page : > http://resourcemanager:8088/cluster/apps doesn't display applications over > . > With command line it works (# yarn application -list). > Regards, > Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id > 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14993412#comment-14993412 ] Mohammad Shahid Khan commented on YARN-3840: UT failiure -- not related to current patch findbugs -- not related to current patch > Resource Manager web ui issue when sorting application by id (with > application having id > ) > > > Key: YARN-3840 > URL: https://issues.apache.org/jira/browse/YARN-3840 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.7.0 >Reporter: LINTE >Assignee: Mohammad Shahid Khan > Attachments: RMApps.png, YARN-3840-1.patch, YARN-3840-2.patch, > YARN-3840-3.patch, YARN-3840-4.patch, YARN-3840-5.patch, YARN-3840-6.patch, > yarn-3840-7.patch > > > On the WEBUI, the global main view page : > http://resourcemanager:8088/cluster/apps doesn't display applications over > . > With command line it works (# yarn application -list). > Regards, > Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-4322) local framwork + io.ReadaheadPool is throwing Failed readahead on ifile EBADF: Bad file descriptor
Mohammad Shahid Khan created YARN-4322:
--
Summary: local framwork + io.ReadaheadPool is throwing Failed
readahead on ifile EBADF: Bad file descriptor
Key: YARN-4322
URL: https://issues.apache.org/jira/browse/YARN-4322
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.7.1
Environment: Linux 2.6.32.12-0.7-default x86_64
Reporter: Mohammad Shahid Khan
run the pi job with 100 map and 2 reduce
io.ReadaheadPool is throwing Failed readahead on ifile EBADF: Bad file
descriptor
stacktrace
{code}
15/10/30 16:47:23 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output
of size: 24, inMemoryMapOutputs.size() -> 89, commitMemory -> 2112, usedMemory
->2136
15/10/30 16:47:23 WARN io.ReadaheadPool: Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native
Method)
at
org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
at
org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
at
org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
15/10/30 16:47:23 WARN io.ReadaheadPool: Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native
Method)
at
org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
at
org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
at
org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
15/10/30 16:47:23 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle
output of map attempt_local108678459_0001_m_86_0 decomp: 24 len: 28 to
MEMORY
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (YARN-4109) Exception on RM scheduler page loading with labels
[
https://issues.apache.org/jira/browse/YARN-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan reassigned YARN-4109:
--
Assignee: Mohammad Shahid Khan
> Exception on RM scheduler page loading with labels
> --
>
> Key: YARN-4109
> URL: https://issues.apache.org/jira/browse/YARN-4109
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Mohammad Shahid Khan
>
> Configure node label and load scheduler Page
> {code}
> 2015-09-03 11:27:08,544 ERROR org.apache.hadoop.yarn.webapp.Dispatcher: error
> handling URI: /cluster/scheduler
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:153)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
> at
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
> at
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:139)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
> at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:663)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:615)
> at
> org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter.doFilter(RMAuthenticationFilter.java:82)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1211)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at
[jira] [Commented] (YARN-4037) Hadoop - failed redirect for container
[ https://issues.apache.org/jira/browse/YARN-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14693293#comment-14693293 ] Mohammad Shahid Khan commented on YARN-4037: It should go +to the link http://garima-pc:49712/jobhistory/logs/Garima-PC:50415/container_1439303739376_0001_02_01/container_1439303739376_0001_02_01/Garima Seems like you have not restarted the Cluster after the configuration change. Hadoop - failed redirect for container -- Key: YARN-4037 URL: https://issues.apache.org/jira/browse/YARN-4037 Project: Hadoop YARN Issue Type: Bug Components: yarn Affects Versions: 2.7.1 Environment: Windows 7, Apache Hadoop 2.7.1 Reporter: Gagan Attachments: mapred-site.xml, yarn-site.xml I believe this issue has been addressed earlier in https://issues.apache.org/jira/browse/YARN-1473 though I am not sure because the description of the JIRA does not talk about the following message Failed while trying to construct the redirect url to the log server. Log Server url may not be configured java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all. Could some one look at the same and provide detail on the root cause and resolution ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4037) Hadoop - failed redirect for container
[
https://issues.apache.org/jira/browse/YARN-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681316#comment-14681316
]
Mohammad Shahid Khan commented on YARN-4037:
Hi Gagan,
The below convention in configuring the log server url.
{CODE}
property
nameyarn.log.server.url/name
valuehttp://$jobhistoryserver.full.hostname:port/jobhistory/logs/value
descriptionURL for job history server /description
/property
{CODE}
Note:
The port should be the port being used for the
mapreduce.jobhistory.webapp.address.
for example if the mapreduce.jobhistory.webapp.address has the value ip1:10988,
then the host and port for the log server shuld ip1:10988
Please verify once your configuration.
Hadoop - failed redirect for container
--
Key: YARN-4037
URL: https://issues.apache.org/jira/browse/YARN-4037
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 2.7.1
Environment: Windows 7, Apache Hadoop 2.7.1
Reporter: Gagan
I believe this issue has been addressed earlier in
https://issues.apache.org/jira/browse/YARN-1473 though I am not sure because
the description of the JIRA does not talk about the following message
Failed while trying to construct the redirect url to the log server. Log
Server url may not be configured
java.lang.Exception: Unknown container. Container either has not started or
has already completed or doesn't belong to this node at all.
Could some one look at the same and provide detail on the root cause and
resolution ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3845) [YARN] YARN status in web ui does not show correctly in IE 11
[ https://issues.apache.org/jira/browse/YARN-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3845: --- Attachment: YARN-3845__1.patch Patch having the checkstyle fix [YARN] YARN status in web ui does not show correctly in IE 11 - Key: YARN-3845 URL: https://issues.apache.org/jira/browse/YARN-3845 Project: Hadoop YARN Issue Type: Bug Reporter: Jagadesh Kiran N Assignee: Mohammad Shahid Khan Priority: Trivial Attachments: IE11_yarn.gif, YARN-3845.patch, YARN-3845__1.patch In IE 11 , the color display is not proper for the scheduler . In other browser it is showing correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3845) [YARN] YARN status in web ui does not show correctly in IE 11
[ https://issues.apache.org/jira/browse/YARN-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3845: --- Attachment: YARN-3845_2.patch [YARN] YARN status in web ui does not show correctly in IE 11 - Key: YARN-3845 URL: https://issues.apache.org/jira/browse/YARN-3845 Project: Hadoop YARN Issue Type: Bug Reporter: Jagadesh Kiran N Assignee: Mohammad Shahid Khan Priority: Trivial Attachments: IE11_yarn.gif, YARN-3845.patch, YARN-3845_2.patch, YARN-3845__1.patch In IE 11 , the color display is not proper for the scheduler . In other browser it is showing correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3845) [YARN] YARN status in web ui does not show correctly in IE 11
[ https://issues.apache.org/jira/browse/YARN-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636814#comment-14636814 ] Mohammad Shahid Khan commented on YARN-3845: Thanks Rohith Sharma K S for review. Have incorporated the recommended change in the latest patch. [YARN] YARN status in web ui does not show correctly in IE 11 - Key: YARN-3845 URL: https://issues.apache.org/jira/browse/YARN-3845 Project: Hadoop YARN Issue Type: Bug Reporter: Jagadesh Kiran N Assignee: Mohammad Shahid Khan Priority: Trivial Attachments: IE11_yarn.gif, YARN-3845.patch, YARN-3845_2.patch, YARN-3845__1.patch In IE 11 , the color display is not proper for the scheduler . In other browser it is showing correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3845) [YARN] YARN status in web ui does not show correctly in IE 11
[ https://issues.apache.org/jira/browse/YARN-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3845: --- Attachment: YARN-3845.patch [YARN] YARN status in web ui does not show correctly in IE 11 - Key: YARN-3845 URL: https://issues.apache.org/jira/browse/YARN-3845 Project: Hadoop YARN Issue Type: Bug Reporter: Jagadesh Kiran N Assignee: Mohammad Shahid Khan Priority: Trivial Attachments: IE11_yarn.gif, YARN-3845.patch In IE 11 , the color display is not proper for the scheduler . In other browser it is showing correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625949#comment-14625949 ] Mohammad Shahid Khan commented on YARN-3840: I want somebody to review the solution. Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Reporter: LINTE Assignee: Mohammad Shahid Khan Attachments: RMApps.png, YARN-3840-1.patch, YARN-3840-2.patch, YARN-3840-3.patch, YARN-3840-4.patch, YARN-3840-5.patch, YARN-3840-6.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3845) [YARN] YARN status in web ui does not show correctly in IE 11
[
https://issues.apache.org/jira/browse/YARN-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14626322#comment-14626322
]
Mohammad Shahid Khan commented on YARN-3845:
IE 11 refused to work with rgba color format.
There are two solution for the same.
1. To correct the same we can add the below meta in the html.
{CODE} meta http-equiv=x-ua-compatible content=IE=edge / {CODE}
But this fix will induce the backword compatibitly issue, means in lower
version of IE8, IE9 etc will not work.
2. Along with with the rgba value we can use the hex value as well. OR we can
use only hex
For example;
{CODE} background:rgba(255, 140, 0, 0.8);background:#ff8c00 {CODE}
[YARN] YARN status in web ui does not show correctly in IE 11
-
Key: YARN-3845
URL: https://issues.apache.org/jira/browse/YARN-3845
Project: Hadoop YARN
Issue Type: Bug
Reporter: Jagadesh Kiran N
Priority: Trivial
Attachments: IE11_yarn.gif
In IE 11 , the color display is not proper for the scheduler . In other
browser it is showing correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (YARN-3845) [YARN] YARN status in web ui does not show correctly in IE 11
[ https://issues.apache.org/jira/browse/YARN-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan reassigned YARN-3845: -- Assignee: Mohammad Shahid Khan [YARN] YARN status in web ui does not show correctly in IE 11 - Key: YARN-3845 URL: https://issues.apache.org/jira/browse/YARN-3845 Project: Hadoop YARN Issue Type: Bug Reporter: Jagadesh Kiran N Assignee: Mohammad Shahid Khan Priority: Trivial Attachments: IE11_yarn.gif In IE 11 , the color display is not proper for the scheduler . In other browser it is showing correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Attachment: YARN-3840-6.patch Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Reporter: LINTE Assignee: Mohammad Shahid Khan Attachments: RMApps.png, YARN-3840-1.patch, YARN-3840-2.patch, YARN-3840-3.patch, YARN-3840-4.patch, YARN-3840-5.patch, YARN-3840-6.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Attachment: YARN-3840-5.patch Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Reporter: LINTE Assignee: Mohammad Shahid Khan Attachments: RMApps.png, YARN-3840-1.patch, YARN-3840-2.patch, YARN-3840-3.patch, YARN-3840-4.patch, YARN-3840-5.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3846) RM Web UI queue filter is not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-3846:
---
Attachment: YARN-3846.patch
Please review attached patch
RM Web UI queue filter is not working
-
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 3.0.0, 2.8.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Attachments: YARN-3846.patch, scheduler queue issue.png, scheduler
queue positive behavior.png
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Attachment: YARN-3840-4.patch Attached patch having test cases Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Reporter: LINTE Assignee: Mohammad Shahid Khan Attachments: RMApps.png, YARN-3840-1.patch, YARN-3840-2.patch, YARN-3840-3.patch, YARN-3840-4.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3846) RM Web UI queue filter is not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-3846:
---
Labels: PatchAvailable (was: )
Not adding any test case
Change is only js code.
RM Web UI queue filter is not working
-
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 3.0.0, 2.8.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Labels: PatchAvailable
Attachments: YARN-3846.patch, scheduler queue issue.png, scheduler
queue positive behavior.png
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609868#comment-14609868 ] Mohammad Shahid Khan commented on YARN-3840: Hi Devaraj K , Please ignore the first patch. in the current patch have taken the nodemanager web ui issue as well. The current patch is using the natural sort algorithm of the natural,js a plugin used for datatable to sort the data. The natural sort plugin https://github.com/DataTables/Plugins/blob/1.10.7/sorting/natural.js is having the MIT license. As per MIT license we can redistribute the code but we have to keep the license header. The haddop patch verification tool does not allow the auther info As of now i have not removed the @author tag from the path file. Please help me to address this issue. Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Reporter: LINTE Assignee: Mohammad Shahid Khan Attachments: RMApps.png, YARN-3840-1.patch, YARN-3840-2.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3846) RM Web UI queue filter is not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609937#comment-14609937
]
Mohammad Shahid Khan commented on YARN-3846:
you are right 2.7 is not having the issue.
My mistake i have raised in the wrong branch.
Affected version should be changed.
The issue is there in trunk.
The trunk is having the Queue: change. This change only has induced the
filter issue and even you only have handled
https://issues.apache.org/jira/browse/YARN-3707.
But if we have to keep the lbel Queue: the current issue should be fixed in
the trunk.
RM Web UI queue filter is not working
-
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 2.7.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Attachments: scheduler queue issue.png, scheduler queue positive
behavior.png
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Attachment: YARN-3840-2.patch Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Reporter: LINTE Assignee: Mohammad Shahid Khan Attachments: RMApps.png, YARN-3840-1.patch, YARN-3840-2.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3846) RM Web UI queue filter is not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-3846:
---
Affects Version/s: (was: 2.7.0)
2.8.0
3.0.0
RM Web UI queue filter is not working
-
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 3.0.0, 2.8.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Attachments: scheduler queue issue.png, scheduler queue positive
behavior.png
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3846) RM Web UI queue filter is not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609939#comment-14609939
]
Mohammad Shahid Khan commented on YARN-3846:
also in branch-2
RM Web UI queue filter is not working
-
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 2.7.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Attachments: scheduler queue issue.png, scheduler queue positive
behavior.png
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3846) RM Web UI queue filter is not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-3846:
---
Target Version/s: 3.0.0, 2.8.0 (was: 2.7.2)
RM Web UI queue filter is not working
-
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 3.0.0, 2.8.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Attachments: scheduler queue issue.png, scheduler queue positive
behavior.png
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3846) RM Web UI queue filter is not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609946#comment-14609946
]
Mohammad Shahid Khan commented on YARN-3846:
have changed the affected and target version.
RM Web UI queue filter is not working
-
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 3.0.0, 2.8.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Attachments: scheduler queue issue.png, scheduler queue positive
behavior.png
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Attachment: YARN-3840-3.patch Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Reporter: LINTE Assignee: Mohammad Shahid Khan Attachments: RMApps.png, YARN-3840-1.patch, YARN-3840-2.patch, YARN-3840-3.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Attachment: (was: YARN-3840.patch) Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Reporter: LINTE Assignee: Mohammad Shahid Khan Attachments: RMApps.png On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3798) ZKRMStateStore shouldn't create new session without occurrance of SESSIONEXPIED
[
https://issues.apache.org/jira/browse/YARN-3798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14608273#comment-14608273
]
Mohammad Shahid Khan commented on YARN-3798:
Hi Devaraj,
I was not the watcher of this issue.
ZKRMStateStore shouldn't create new session without occurrance of
SESSIONEXPIED
---
Key: YARN-3798
URL: https://issues.apache.org/jira/browse/YARN-3798
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.7.0
Environment: Suse 11 Sp3
Reporter: Bibin A Chundatt
Assignee: Varun Saxena
Priority: Blocker
Attachments: RM.log, YARN-3798-2.7.002.patch,
YARN-3798-branch-2.7.002.patch, YARN-3798-branch-2.7.003.patch,
YARN-3798-branch-2.7.patch
RM going down with NoNode exception during create of znode for appattempt
*Please find the exception logs*
{code}
2015-06-09 10:09:44,732 INFO
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore:
ZKRMStateStore Session connected
2015-06-09 10:09:44,732 INFO
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore:
ZKRMStateStore Session restored
2015-06-09 10:09:44,886 INFO
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore:
Exception while executing a ZK operation.
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
at org.apache.zookeeper.KeeperException.create(KeeperException.java:115)
at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1405)
at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:1310)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:926)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:923)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1101)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1122)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doStoreMultiWithRetries(ZKRMStateStore.java:923)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doStoreMultiWithRetries(ZKRMStateStore.java:937)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.createWithRetries(ZKRMStateStore.java:970)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.updateApplicationAttemptStateInternal(ZKRMStateStore.java:671)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppAttemptTransition.transition(RMStateStore.java:275)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppAttemptTransition.transition(RMStateStore.java:260)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:837)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:900)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:895)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:175)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:108)
at java.lang.Thread.run(Thread.java:745)
2015-06-09 10:09:44,887 INFO
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore: Maxed
out ZK retries. Giving up!
2015-06-09 10:09:44,887 ERROR
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Error
updating appAttempt: appattempt_1433764310492_7152_01
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
at org.apache.zookeeper.KeeperException.create(KeeperException.java:115)
at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1405)
at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:1310)
at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:926)
at
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Attachment: YARN-3840-1.patch Submitting the updated patch Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Reporter: LINTE Assignee: Mohammad Shahid Khan Attachments: RMApps.png, YARN-3840-1.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Fix Version/s: 2.7.2 Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Environment: Centos 6.6 Java 1.7 Reporter: LINTE Labels: PatchAvailable Fix For: 2.7.2 Attachments: RMApps.png, YARN-3840.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Fix Version/s: (was: 2.7.2) Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Environment: Centos 6.6 Java 1.7 Reporter: LINTE Labels: PatchAvailable Attachments: RMApps.png, YARN-3840.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3846) RM Web UI queue fileter not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-3846:
---
Attachment: scheduler queue issue.png
scheduler queue positive behavior.png
RM Web UI queue fileter not working
---
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 2.7.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Attachments: scheduler queue issue.png, scheduler queue positive
behavior.png
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Attachment: YARN-3840.patch Please review the attached patch Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Environment: Centos 6.6 Java 1.7 Reporter: LINTE Attachments: RMApps.png, YARN-3840.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3840: --- Labels: PatchAvailable (was: ) Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Environment: Centos 6.6 Java 1.7 Reporter: LINTE Labels: PatchAvailable Attachments: RMApps.png, YARN-3840.patch On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[
https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604020#comment-14604020
]
Mohammad Shahid Khan commented on YARN-3840:
The datatable string sort algorithm has limitation.
It can not properly sort the string having having the combination of the string
and numeric value. The application id application_numericValue
ie is why the sort is not working properly.
To fix the same we can use the datatable plugins natural sort alogorithm.
{CODE}
sb.append([\n)
.append({'sType':'natural', 'aTargets': [0])
.append(, 'mRender': parseHadoopID })
{CODE}
plugin - ref:
https://github.com/DataTables/Plugins/blob/1.10.7/sorting/natural.js
Resource Manager web ui issue when sorting application by id (with
application having id )
Key: YARN-3840
URL: https://issues.apache.org/jira/browse/YARN-3840
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.7.0
Environment: Centos 6.6
Java 1.7
Reporter: LINTE
Attachments: RMApps.png
On the WEBUI, the global main view page :
http://resourcemanager:8088/cluster/apps doesn't display applications over
.
With command line it works (# yarn application -list).
Regards,
Alexandre
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3846) RM Web UI queue fileter not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604131#comment-14604131
]
Mohammad Shahid Khan commented on YARN-3846:
https://issues.apache.org/jira/browse/YARN-2238 is for different fix.
The label Queue : was added before the queue name in
https://issues.apache.org/jira/browse/YARN-3362. After the search was not
working and the same has been handled in the
https://issues.apache.org/jira/browse/YARN-3707.
The fix is OK for the for the first child of the queue
but will not work for the second child
for example Queue: b.x
Queue: root
Queue: a _*For queue a it will work fine*_
Queue: b
Queue: b.x _*But for queue x and y this will not work*_
Queue: b.y
*My question: *
What is the significance of adding Label *Queue:* before the queue name?
RM Web UI queue fileter not working
---
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 2.7.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3846) RM Web UI queue fileter not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604132#comment-14604132
]
Mohammad Shahid Khan commented on YARN-3846:
Please confirm whether we have to keep the Label Queue: or not.
Then will submit the patch accordingly.
RM Web UI queue fileter not working
---
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 2.7.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (YARN-3846) RM Web UI queue fileter not working
Mohammad Shahid Khan created YARN-3846:
--
Summary: RM Web UI queue fileter not working
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 2.7.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the the
clicked queue.
The regular expression seems to be wrong
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3846) RM Web UI queue fileter not working
[
https://issues.apache.org/jira/browse/YARN-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-3846:
---
Description:
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the the
clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
was:
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the the
clicked queue.
The regular expression seems to be wrong
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
RM Web UI queue fileter not working
---
Key: YARN-3846
URL: https://issues.apache.org/jira/browse/YARN-3846
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 2.7.0
Reporter: Mohammad Shahid Khan
Assignee: Mohammad Shahid Khan
Click on root queue will show the complete applications
But click on the leaf queue is not filtering the application related to the
the clicked queue.
The regular expression seems to be wrong
{code}
q = '^' + q.substr(q.lastIndexOf(':') + 2) + '$';,
{code}
For example
1. Suppose queue name is b
them the above expression will try to substr at index 1
q.lastIndexOf(':') = -1
-1+2= 1
which is wrong. its should look at the 0 index.
2. if queue name is ab.x
then it will parse it to .x
but it should be x
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3017) ContainerID in ResourceManager Log Has Slightly Different Format From AppAttemptID
[
https://issues.apache.org/jira/browse/YARN-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-3017:
---
Attachment: YARN-3017_3.patch
Patch having the updated test cases
ContainerID in ResourceManager Log Has Slightly Different Format From
AppAttemptID
--
Key: YARN-3017
URL: https://issues.apache.org/jira/browse/YARN-3017
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.8.0
Reporter: MUFEED USMAN
Priority: Minor
Labels: PatchAvailable
Attachments: YARN-3017.patch, YARN-3017_1.patch, YARN-3017_2.patch,
YARN-3017_3.patch
Not sure if this should be filed as a bug or not.
In the ResourceManager log in the events surrounding the creation of a new
application attempt,
...
...
2014-11-14 17:45:37,258 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Launching
masterappattempt_1412150883650_0001_02
...
...
The application attempt has the ID format _1412150883650_0001_02.
Whereas the associated ContainerID goes by _1412150883650_0001_02_.
...
...
2014-11-14 17:45:37,260 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Setting
up
container Container: [ContainerId: container_1412150883650_0001_02_01,
NodeId: n67:55933, NodeHttpAddress: n67:8042, Resource: memory:2048,
vCores:1,
disks:0.0, Priority: 0, Token: Token { kind: ContainerToken, service:
10.10.70.67:55933 }, ] for AM appattempt_1412150883650_0001_02
...
...
Curious to know if this is kept like that for a reason. If not while using
filtering tools to, say, grep events surrounding a specific attempt by the
numeric ID part information may slip out during troubleshooting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3017) ContainerID in ResourceManager Log Has Slightly Different Format From AppAttemptID
[
https://issues.apache.org/jira/browse/YARN-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-3017:
---
Attachment: YARN-3017_2.patch
fixed the review comment
ContainerID in ResourceManager Log Has Slightly Different Format From
AppAttemptID
--
Key: YARN-3017
URL: https://issues.apache.org/jira/browse/YARN-3017
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.8.0
Reporter: MUFEED USMAN
Priority: Minor
Labels: PatchAvailable
Attachments: YARN-3017.patch, YARN-3017_1.patch, YARN-3017_2.patch
Not sure if this should be filed as a bug or not.
In the ResourceManager log in the events surrounding the creation of a new
application attempt,
...
...
2014-11-14 17:45:37,258 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Launching
masterappattempt_1412150883650_0001_02
...
...
The application attempt has the ID format _1412150883650_0001_02.
Whereas the associated ContainerID goes by _1412150883650_0001_02_.
...
...
2014-11-14 17:45:37,260 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Setting
up
container Container: [ContainerId: container_1412150883650_0001_02_01,
NodeId: n67:55933, NodeHttpAddress: n67:8042, Resource: memory:2048,
vCores:1,
disks:0.0, Priority: 0, Token: Token { kind: ContainerToken, service:
10.10.70.67:55933 }, ] for AM appattempt_1412150883650_0001_02
...
...
Curious to know if this is kept like that for a reason. If not while using
filtering tools to, say, grep events surrounding a specific attempt by the
numeric ID part information may slip out during troubleshooting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3017) ContainerID in ResourceManager Log Has Slightly Different Format From AppAttemptID
[
https://issues.apache.org/jira/browse/YARN-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-3017:
---
Attachment: YARN-3017_1.patch
latest patch
ContainerID in ResourceManager Log Has Slightly Different Format From
AppAttemptID
--
Key: YARN-3017
URL: https://issues.apache.org/jira/browse/YARN-3017
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.8.0
Reporter: MUFEED USMAN
Priority: Minor
Labels: PatchAvailable
Attachments: YARN-3017.patch, YARN-3017_1.patch
Not sure if this should be filed as a bug or not.
In the ResourceManager log in the events surrounding the creation of a new
application attempt,
...
...
2014-11-14 17:45:37,258 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Launching
masterappattempt_1412150883650_0001_02
...
...
The application attempt has the ID format _1412150883650_0001_02.
Whereas the associated ContainerID goes by _1412150883650_0001_02_.
...
...
2014-11-14 17:45:37,260 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Setting
up
container Container: [ContainerId: container_1412150883650_0001_02_01,
NodeId: n67:55933, NodeHttpAddress: n67:8042, Resource: memory:2048,
vCores:1,
disks:0.0, Priority: 0, Token: Token { kind: ContainerToken, service:
10.10.70.67:55933 }, ] for AM appattempt_1412150883650_0001_02
...
...
Curious to know if this is kept like that for a reason. If not while using
filtering tools to, say, grep events surrounding a specific attempt by the
numeric ID part information may slip out during troubleshooting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3017) ContainerID in ResourceManager Log Has Slightly Different Format From AppAttemptID
[
https://issues.apache.org/jira/browse/YARN-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Shahid Khan updated YARN-3017:
---
Attachment: YARN-3017.patch
Please find the fix in the attached patch
ContainerID in ResourceManager Log Has Slightly Different Format From
AppAttemptID
--
Key: YARN-3017
URL: https://issues.apache.org/jira/browse/YARN-3017
Project: Hadoop YARN
Issue Type: Improvement
Reporter: MUFEED USMAN
Priority: Minor
Attachments: YARN-3017.patch
Not sure if this should be filed as a bug or not.
In the ResourceManager log in the events surrounding the creation of a new
application attempt,
...
...
2014-11-14 17:45:37,258 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Launching
masterappattempt_1412150883650_0001_02
...
...
The application attempt has the ID format _1412150883650_0001_02.
Whereas the associated ContainerID goes by _1412150883650_0001_02_.
...
...
2014-11-14 17:45:37,260 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Setting
up
container Container: [ContainerId: container_1412150883650_0001_02_01,
NodeId: n67:55933, NodeHttpAddress: n67:8042, Resource: memory:2048,
vCores:1,
disks:0.0, Priority: 0, Token: Token { kind: ContainerToken, service:
10.10.70.67:55933 }, ] for AM appattempt_1412150883650_0001_02
...
...
Curious to know if this is kept like that for a reason. If not while using
filtering tools to, say, grep events surrounding a specific attempt by the
numeric ID part information may slip out during troubleshooting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3560) Not able to navigate to the cluster from tracking url (proxy) generated after submission of job
[ https://issues.apache.org/jira/browse/YARN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3560: --- Attachment: YARN-3560.patch The same implementation is already done. Attaching the patch file having the test case of the same implementation. Not able to navigate to the cluster from tracking url (proxy) generated after submission of job --- Key: YARN-3560 URL: https://issues.apache.org/jira/browse/YARN-3560 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.7.0 Reporter: Anushri Priority: Minor Attachments: YARN-3560.patch, YARN-3560.patch a standalone web proxy server is enabled in the cluster when a job is submitted the url generated contains proxy track this url in the web page , if we try to navigate to the cluster links [about. applications, or scheduler] it gets redirected to some default port instead of actual RM web port configured as such it throws webpage not available -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3645) ResourceManager can't start success if attribute value of aclSubmitApps is null in fair-scheduler.xml
[
https://issues.apache.org/jira/browse/YARN-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547601#comment-14547601
]
Mohammad Shahid Khan commented on YARN-3645:
Loading with invalid node configuration is not feasible.
But instead of throwing the NullPointerException, we can
*AllocationConfigurationException* with proper message so that the reason of
failure could be identified easily.
{code}
if (aclAdministerApps.equals(field.getTagName())) {
Text aclText = (Text)field.getFirstChild();
if (aclText == null) {
throw new AllocationConfigurationException(
Invalid admin ACL configuration in allocation file);
}
String text = ((Text)field.getFirstChild()).getData();
acls.put(QueueACL.ADMINISTER_QUEUE, new AccessControlList(text));
}
{code}
ResourceManager can't start success if attribute value of aclSubmitApps is
null in fair-scheduler.xml
Key: YARN-3645
URL: https://issues.apache.org/jira/browse/YARN-3645
Project: Hadoop YARN
Issue Type: Bug
Components: fairscheduler
Affects Versions: 2.5.2
Reporter: zhoulinlin
The aclSubmitApps is configured in fair-scheduler.xml like below:
queue name=mr
aclSubmitApps/aclSubmitApps
/queue
The resourcemanager log:
2015-05-14 12:59:48,623 INFO org.apache.hadoop.service.AbstractService:
Service ResourceManager failed in state INITED; cause:
org.apache.hadoop.service.ServiceStateException: java.io.IOException: Failed
to initialize FairScheduler
org.apache.hadoop.service.ServiceStateException: java.io.IOException: Failed
to initialize FairScheduler
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:493)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:920)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:240)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1159)
Caused by: java.io.IOException: Failed to initialize FairScheduler
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1301)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1318)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
... 7 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.loadQueue(AllocationFileLoaderService.java:458)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadAllocations(AllocationFileLoaderService.java:337)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1299)
... 9 more
2015-05-14 12:59:48,623 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning
to standby state
2015-05-14 12:59:48,623 INFO
com.zte.zdh.platformplugin.factory.YarnPlatformPluginProxyFactory: plugin
transitionToStandbyIn
2015-05-14 12:59:48,623 WARN org.apache.hadoop.service.AbstractService: When
stopping the service ResourceManager : java.lang.NullPointerException
java.lang.NullPointerException
at
com.zte.zdh.platformplugin.factory.YarnPlatformPluginProxyFactory.transitionToStandbyIn(YarnPlatformPluginProxyFactory.java:71)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToStandby(ResourceManager.java:997)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStop(ResourceManager.java:1058)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:171)
at
[jira] [Commented] (YARN-3575) Job using 2.5 jars fails on a 2.6 cluster whose RM has been restarted
[ https://issues.apache.org/jira/browse/YARN-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547808#comment-14547808 ] Mohammad Shahid Khan commented on YARN-3575: Can we support the binary compatibility? The API of 2.5 does not support the parsing of epoch character, may be its because that time the container id was not containing the epoch character. As per my understanding the binaries should be same across the cluster and client. Job using 2.5 jars fails on a 2.6 cluster whose RM has been restarted - Key: YARN-3575 URL: https://issues.apache.org/jira/browse/YARN-3575 Project: Hadoop YARN Issue Type: Bug Components: client Affects Versions: 2.6.0 Reporter: Jason Lowe Trying to launch a job that uses the 2.5 jars fails on a 2.6 cluster whose RM has been restarted (i.e.: epoch != 0) becaue the epoch number starts appearing in the container IDs and the 2.5 jars no longer know how to parse the container IDs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3560) Not able to navigate to the cluster from tracking url (proxy) generated after submission of job
[ https://issues.apache.org/jira/browse/YARN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3560: --- Attachment: YARN-3560.patch Please review the attached patch Not able to navigate to the cluster from tracking url (proxy) generated after submission of job --- Key: YARN-3560 URL: https://issues.apache.org/jira/browse/YARN-3560 Project: Hadoop YARN Issue Type: Bug Reporter: Anushri Priority: Minor Attachments: YARN-3560.patch a standalone web proxy server is enabled in the cluster when a job is submitted the url generated contains proxy track this url in the web page , if we try to navigate to the cluster links [about. applications, or scheduler] it gets redirected to some default port instead of actual RM web port configured as such it throws webpage not available -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3560) Not able to navigate to the cluster from tracking url (proxy) generated after submission of job
[ https://issues.apache.org/jira/browse/YARN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3560: --- Attachment: (was: YARN-3560.patch) Not able to navigate to the cluster from tracking url (proxy) generated after submission of job --- Key: YARN-3560 URL: https://issues.apache.org/jira/browse/YARN-3560 Project: Hadoop YARN Issue Type: Bug Reporter: Anushri Priority: Minor Attachments: YARN-3560.patch a standalone web proxy server is enabled in the cluster when a job is submitted the url generated contains proxy track this url in the web page , if we try to navigate to the cluster links [about. applications, or scheduler] it gets redirected to some default port instead of actual RM web port configured as such it throws webpage not available -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3560) Not able to navigate to the cluster from tracking url (proxy) generated after submission of job
[ https://issues.apache.org/jira/browse/YARN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3560: --- Attachment: YARN-3560.patch Please review the attached patch. Not able to navigate to the cluster from tracking url (proxy) generated after submission of job --- Key: YARN-3560 URL: https://issues.apache.org/jira/browse/YARN-3560 Project: Hadoop YARN Issue Type: Bug Reporter: Anushri Priority: Minor Attachments: YARN-3560.patch a standalone web proxy server is enabled in the cluster when a job is submitted the url generated contains proxy track this url in the web page , if we try to navigate to the cluster links [about. applications, or scheduler] it gets redirected to some default port instead of actual RM web port configured as such it throws webpage not available -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3560) Not able to navigate to the cluster from tracking url (proxy) generated after submission of job
[ https://issues.apache.org/jira/browse/YARN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated YARN-3560: --- Target Version/s: 2.8.0 Affects Version/s: 2.7.0 Not able to navigate to the cluster from tracking url (proxy) generated after submission of job --- Key: YARN-3560 URL: https://issues.apache.org/jira/browse/YARN-3560 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.7.0 Reporter: Anushri Priority: Minor Attachments: YARN-3560.patch a standalone web proxy server is enabled in the cluster when a job is submitted the url generated contains proxy track this url in the web page , if we try to navigate to the cluster links [about. applications, or scheduler] it gets redirected to some default port instead of actual RM web port configured as such it throws webpage not available -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3560) Not able to navigate to the cluster from tracking url (proxy) generated after submission of job
[ https://issues.apache.org/jira/browse/YARN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14527917#comment-14527917 ] Mohammad Shahid Khan commented on YARN-3560: The issue is happening due to the wrong hyperlink url formation. The system is always setting forming the url with the default port name even when the yarn.resourcemanager.webapp.address is being configured with different port numnber. Not able to navigate to the cluster from tracking url (proxy) generated after submission of job --- Key: YARN-3560 URL: https://issues.apache.org/jira/browse/YARN-3560 Project: Hadoop YARN Issue Type: Bug Reporter: Anushri Priority: Minor a standalone web proxy server is enabled in the cluster when a job is submitted the url generated contains proxy track this url in the web page , if we try to navigate to the cluster links [about. applications, or scheduler] it gets redirected to some default port instead of actual RM web port configured as such it throws webpage not available -- This message was sent by Atlassian JIRA (v6.3.4#6332)
