[
https://issues.apache.org/jira/browse/MAPREDUCE-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923645#comment-13923645
]
Sangjin Lee commented on MAPREDUCE-5751:
----------------------------------------
I think it may be prudent to keep the jetty classes in the system classes. The
problem pattern was jetty trying to load a configuration class using the thread
context classloader as part of initialization.
If some jetty classes are loaded through the normal JVM app classloader, but
the configuration class is loaded through the hadoop's job classloader, it
would cause the ClassCastException, just like the one that's mentioned in the
description. It doesn't even need multiple jetty versions to have this issue.
Making jetty part of the system classes solves this issue if it should happen
later on as part of user code.
> MR app master fails to start in some cases if mapreduce.job.classloader is
> true
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5751
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5751
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.2.0
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
> Attachments: mapreduce-5751.patch
>
>
> If mapreduce.job.classloader is set to true, and the MR client includes a
> jetty jar in its libjars or job jar, the MR app master fails to start. A
> typical stack trace we get is as follows:
> {noformat}
> java.lang.ClassCastException: org.mortbay.jetty.webapp.WebInfConfiguration
> cannot be cast to org.mortbay.jetty.webapp.Configuration
> at
> org.mortbay.jetty.webapp.WebAppContext.loadConfigurations(WebAppContext.java:890)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:462)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:676)
> at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:208)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:151)
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1040)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1307)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1303)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1259)
> {noformat}
> This happens because as part of the MR app master start the jetty classes are
> loaded normally through the app classloader, but WebAppContext tries to load
> the specific Configuration class via the thread context classloader (which
> had been set to the user job classloader).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
