[
https://issues.apache.org/jira/browse/MAPREDUCE-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923971#comment-13923971
]
Jason Lowe commented on MAPREDUCE-5751:
---------------------------------------
My apologies, I still don't see why jetty needs to be part of the system
classes. If that's true then one could argue guava, jackson, jersey, avro,
etc. could also have the same issue. Couldn't those classes do some
initialization in the hadoop classloader context then later some user code
triggers some other internal classes of those dependencies to be loaded and we
end up mixing and matching as we did with jetty? Or am I misunderstanding the
issue and jetty is doing something none of our other dependencies will do?
I want to make sure I understand the criteria for what should be put in the
system classes. If I'm understanding the issue properly, it seems like a
slippery slope from adding jetty to adding a large chunk of our dependencies
which would effectively undermine the whole point of the app classloader.
> MR app master fails to start in some cases if mapreduce.job.classloader is
> true
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5751
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5751
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.2.0
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
> Attachments: mapreduce-5751.patch
>
>
> If mapreduce.job.classloader is set to true, and the MR client includes a
> jetty jar in its libjars or job jar, the MR app master fails to start. A
> typical stack trace we get is as follows:
> {noformat}
> java.lang.ClassCastException: org.mortbay.jetty.webapp.WebInfConfiguration
> cannot be cast to org.mortbay.jetty.webapp.Configuration
> at
> org.mortbay.jetty.webapp.WebAppContext.loadConfigurations(WebAppContext.java:890)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:462)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:676)
> at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:208)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:151)
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1040)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1307)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1303)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1259)
> {noformat}
> This happens because as part of the MR app master start the jetty classes are
> loaded normally through the app classloader, but WebAppContext tries to load
> the specific Configuration class via the thread context classloader (which
> had been set to the user job classloader).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
