[
https://issues.apache.org/jira/browse/MAPREDUCE-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923460#comment-13923460
]
Jason Lowe commented on MAPREDUCE-5751:
---------------------------------------
I think moving the app classloader setup to just before we start the job makes
a lot of sense. There should be no user code invoked up to that point, and
it's close to where user code will be called as part of the job setup.
As for the org.mortbay. addition to the system classes list, I'm not enough of
a jetty expert to know if this is really necessary. Does jetty do things that
prevent multiple jetty versions from co-existing if the app classloader is
doing its job? It sounds like it worked when you tried leaving it out but
still moving the app classloader init. If multiple jetty versions can really
coexist in this setup then I think we shouldn't preclude it by default. But
again I might be missing something jetty does that would break if we allow
multiple versions even with the app classloader in place.
> MR app master fails to start in some cases if mapreduce.job.classloader is
> true
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5751
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5751
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.2.0
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
> Attachments: mapreduce-5751.patch
>
>
> If mapreduce.job.classloader is set to true, and the MR client includes a
> jetty jar in its libjars or job jar, the MR app master fails to start. A
> typical stack trace we get is as follows:
> {noformat}
> java.lang.ClassCastException: org.mortbay.jetty.webapp.WebInfConfiguration
> cannot be cast to org.mortbay.jetty.webapp.Configuration
> at
> org.mortbay.jetty.webapp.WebAppContext.loadConfigurations(WebAppContext.java:890)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:462)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:676)
> at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:208)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:151)
> at
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1040)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1307)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1303)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1259)
> {noformat}
> This happens because as part of the MR app master start the jetty classes are
> loaded normally through the app classloader, but WebAppContext tries to load
> the specific Configuration class via the thread context classloader (which
> had been set to the user job classloader).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
