Sriharsha Chintalapani created STORM-967:
--------------------------------------------
Summary: Unable to package storm-kafka & storm-hdfs connectors in
a topology
Key: STORM-967
URL: https://issues.apache.org/jira/browse/STORM-967
Project: Apache Storm
Issue Type: Bug
Reporter: Sriharsha Chintalapani
Priority: Critical
Based on my testing storm-kafka & storm-hdfs connector along with it
dependencies are unable to be packaged together .
If we tried to package it using maven assembly plugin storm-hdfs errors out
saying
"java.lang.RuntimeException: Error preparing HdfsBolt: No FileSystem for
scheme: hdfs"
To avoid we need to use maven shade plugin as documented in the README here
https://github.com/apache/storm/tree/master/external/storm-hdfs
Now when we use maven shade plugin storm-kafka throws an initial error about
{code}
java.lang.NoClassDefFoundError: Could not initialize class
com.yammer.metrics.Metrics
at kafka.metrics.KafkaMetricsGroup$class.newTimer(KafkaMetricsGroup.scala:52)
at
kafka.consumer.FetchRequestAndResponseMetrics.newTimer(FetchRequestAndResponseStats.scala:25)
at
kafka.consumer.FetchRequestAndResponseMetrics.<init>(FetchRequestAndResponseStats.scala:26)
at
kafka.consumer.FetchRequestAndResponseStats.<init>(FetchRequestAndResponseStats.scala:37)
{code}
After we explicitly added metrics dependency to the pom file
KafkaSpout now errors out with
IllegalStateException KafkaMetricsGroup.newTimer
KafkaSpout works fine if we use assembly plugin. Due to the behavior of these
two connectors one works assembly and another works with shade it looks like we
won't be able to use them together.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
