My simple Spark streaming app is failing:
ConsumerFetcherManager$LeaderFinderThread:
[test-group_appdev-1431189419702-9d5d7cbc-leader-finder-thread], Failed to
find leader for Set([test-topic,0])
java.lang.NoClassDefFoundError: org/apache/kafka/common/utils/Utils
I created the topic 'test-topic'
There's something wrong with your classpath - are you using Maven to handle
dependencies? Make sure you are bringing in the correct Kafka client library.
See here:
http://stackoverflow.com/questions/28353316/kafka-utils-wrong-classpath-org-apache-kafka-common-utils-utils
On May 9, 2015, at 1:15
Hi James,
What are you trying to do exactly? If all you are trying to do is monitor
how far behind a consumer is getting you could use the ConsumerOffsetChecker.
As described in the link below.
http://community.spiceworks.com/how_to/77610-how-far-behind-is-your-kafka-consumer
Each message being pr
I've created a patch to expose the high end watermark, having this exact
requirement.
Still waiting for it to be accepted, but are using this in production at
the moment and it works quite nicely:
https://issues.apache.org/jira/browse/KAFKA-1977
On Sat, 9 May 2015 at 18:43 Charlie Knudsen
wrot
I'm running kafka_2.10-0.8.2.1, and loading kafka_2.10-0.8.2.1.jar via
--jars argument to spark-submit
sbt:
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.2.2"
libraryDependencies += "org.apache.spark" %% "spark-streaming" % "1.2.2"
libraryDependencies += "org.apache.spark" %% "spa
Hi Jonathan,
I agree we can have topic-per-table, but some transactions may span
multiple tables and therefore will get applied partially out-of-order. I
suspect this can be a consistency issue and create a state that is
different than the state in the original database, but I don't have good
proo
