Hi everyone,
i am trying to use the direct approach in streaming-kafka-integration
<http://spark.apache.org/docs/latest/streaming-kafka-integration.html>
pulling data from kafka as follow
JavaPairInputDStream<String, String> messages =
KafkaUatils.createDirectStream(jssc,
String.class,
String.class,
StringDecoder.class,
StringDecoder.class,
kafkaParams,
topicsSet);
messages.foreachRDD(
new Function<JavaPairRDD<String,String>, Void>() {
@Override
public Void call(JavaPairRDD<String,
String> rdd) throws IOException {
OffsetRange[] offsetRanges =
((HasOffsetRanges) rdd).offsetRanges();
//.....
return null;
}
}
);
then i got an error when running it
*java.lang.ClassCastException: org.apache.spark.api.java.JavaPairRDD cannot
be cast to org.apache.spark.streaming.kafka.HasOffsetRanges* at
"OffsetRange[] offsetRanges = ((HasOffsetRanges) rdd).offsetRanges();"
i am using the version 1.3.1 if is it a bug in this version ?
Thank you for spending time with me.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/STREAMING-KAFKA-Direct-Approach-JavaPairRDD-cannot-be-cast-to-HasOffsetRanges-tp22568.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]