Joshua Hyde created FLUME-2377:
----------------------------------
Summary: Don't automatically include Hadoop and HBase classpaths
into Flume classpath
Key: FLUME-2377
URL: https://issues.apache.org/jira/browse/FLUME-2377
Project: Flume
Issue Type: Improvement
Components: Shell
Reporter: Joshua Hyde
HBase and Hadoop, for their own reasons, have Jersey 1.x libraries in their
classpath. A custom source we wrote uses Jersey 2.x. For a number of reasons,
the two versions of Jersey do not co-habitate on the same classpath nicely. As
a result, we get stacktraces like the following if, for example, a developer
has Hadoop or HBase installed on their system:
{noformat}
java.lang.NullPointerException
at
com.sun.jersey.core.provider.jaxb.AbstractJAXBProvider.setConfiguration(AbstractJAXBProvider.java:107)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.glassfish.hk2.utilities.reflection.ReflectionHelper.invoke(ReflectionHelper.java:1017)
at org.jvnet.hk2.internal.ClazzCreator.methodMe(ClazzCreator.java:374)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:427)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
at
org.jvnet.hk2.internal.PerLookupContext.findOrCreate(PerLookupContext.java:69)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2488)
at
org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:98)
at
org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:87)
at
org.glassfish.jersey.internal.inject.ProviderToService.apply(ProviderToService.java:58)
at
org.glassfish.jersey.internal.inject.ProviderToService.apply(ProviderToService.java:54)
at
jersey.repackaged.com.google.common.collect.Iterators$8.transform(Iterators.java:860)
at
jersey.repackaged.com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at java.util.AbstractCollection.addAll(AbstractCollection.java:333)
at java.util.LinkedHashSet.<init>(LinkedHashSet.java:169)
at
jersey.repackaged.com.google.common.collect.Sets.newLinkedHashSet(Sets.java:292)
at
org.glassfish.jersey.internal.inject.Providers.getClasses(Providers.java:364)
at
org.glassfish.jersey.internal.inject.Providers.getProviders(Providers.java:186)
at
org.glassfish.jersey.message.internal.MessageBodyFactory.<init>(MessageBodyFactory.java:304)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.glassfish.hk2.utilities.reflection.ReflectionHelper.makeMe(ReflectionHelper.java:1105)
at org.jvnet.hk2.internal.ClazzCreator.createMe(ClazzCreator.java:291)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:421)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
at
org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:114)
at
org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:102)
at
org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture$1.call(Cache.java:97)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture.run(Cache.java:154)
at org.glassfish.hk2.utilities.cache.Cache.compute(Cache.java:199)
at
org.jvnet.hk2.internal.SingletonContext.findOrCreate(SingletonContext.java:153)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2488)
at
org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:629)
at
org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:614)
at
org.glassfish.jersey.server.model.ComponentModelValidator.<init>(ComponentModelValidator.java:97)
at
org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:457)
at
org.glassfish.jersey.server.ApplicationHandler.access$500(ApplicationHandler.java:162)
at
org.glassfish.jersey.server.ApplicationHandler$3.run(ApplicationHandler.java:322)
at org.glassfish.jersey.internal.Errors$2.call(Errors.java:289)
at org.glassfish.jersey.internal.Errors$2.call(Errors.java:286)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at
org.glassfish.jersey.internal.Errors.processWithException(Errors.java:286)
at
org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:319)
at
org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:272)
at
org.glassfish.jersey.server.ContainerFactory.createContainer(ContainerFactory.java:79)
at
org.glassfish.jersey.jetty.JettyHttpContainerFactory.createServer(JettyHttpContainerFactory.java:139)
at com.me.CustomSource.start(CustomSource.java:186)
at
org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
{noformat}
The fact that {{flume-ng}} includes the Hadoop and HBase classpath
automatically, even if we have no need for it, is a bit surprising and, if you
don't know that the shell script is doing that, it can take a little bit of
digging to figure it out.
At the very least, I'd like the ability to tell {{flume-ng}}, "No, don't
include the Hadoop classpath," and, "No, don't include the HBase classpath,",
though if someone's willing to throw passivity to the wind, I'd be glad to just
not have those external classpaths added to Flume's classpath to begin with.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
