Lee Yang created SLIDER-783:
-------------------------------
Summary: Unable to use custom classpath in slider package
Key: SLIDER-783
URL: https://issues.apache.org/jira/browse/SLIDER-783
Project: Slider
Issue Type: Bug
Affects Versions: Slider 0.60
Environment: Hadoop 2.6
Reporter: Lee Yang
Priority: Minor
We have a custom hadoop JAR which is injected via our hadoop configuration.
However, slider does not seem to pick up the local hadoop classpath, so I get
the following error:
{noformat}
2015-02-11 22:18:57,502 [main] ERROR main.ServiceLauncher - Exception:
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.net.YahooDNSToSwitchMapping not found
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class
org.apache.hadoop.net.YahooDNSToSwitchMapping not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2106)
at org.apache.hadoop.yarn.util.RackResolver.init(RackResolver.java:52)
at
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.serviceInit(AMRMClientImpl.java:178)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.serviceInit(AMRMClientAsyncImpl.java:89)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.slider.server.services.utility.LaunchedWorkflowCompositeService.deployChildService(LaunchedWorkflowCompositeService.java:108)
at
org.apache.slider.server.appmaster.SliderAppMaster.createAndRunCluster(SliderAppMaster.java:677)
at
org.apache.slider.server.appmaster.SliderAppMaster.runService(SliderAppMaster.java:543)
at
org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:188)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:475)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
at
org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:630)
at
org.apache.slider.server.appmaster.SliderAppMaster.main(SliderAppMaster.java:2277)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.net.YahooDNSToSwitchMapping not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2074)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2098)
... 13 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.net.YahooDNSToSwitchMapping not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1980)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2072)
... 14 more
{noformat}
I've tried various attempts to inject this jar into the classpath, from the
"additional_cp" param, to modifying the process launch cmd line directly. None
of these seem to be respected, so I eventually worked around this by symlinking
the jar into my slider installation's lib directory, which currently includes
it's own copies of the hadoop 2.6 jars.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
