Sangjin Lee created MAPREDUCE-5751:
--------------------------------------
Summary: MR app master fails to start in some cases if
mapreduce.job.classloader is true
Key: MAPREDUCE-5751
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5751
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
If mapreduce.job.classloader is set to true, and the MR client includes a jetty
jar in its libjars or job jar, the MR app master fails to start. A typical
stack trace we get is as follows:
{noformat}
java.lang.ClassCastException: org.mortbay.jetty.webapp.WebInfConfiguration
cannot be cast to org.mortbay.jetty.webapp.Configuration
at
org.mortbay.jetty.webapp.WebAppContext.loadConfigurations(WebAppContext.java:890)
at
org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:462)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
at
org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
at org.mortbay.jetty.Server.doStart(Server.java:224)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:676)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:208)
at
org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:151)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1040)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1307)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1303)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1259)
{noformat}
This happens because as part of the MR app master start the jetty classes are
loaded normally through the app classloader, but WebAppContext tries to load
the specific Configuration class via the thread context classloader (which had
been set to the user job classloader).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
