Hi all, I need to transform KafkaRDD into a new stream of deserialized case classes. I want to use the new stream to save it to file and to perform additional transformations on it.
To save it I want to use offsets in filenames, hence I need OffsetRanges in transformed RDD. But KafkaRDD is private, hence I don't know how to do it. Alternatively I could deserialize directly in messageHandler before KafkaRDD but it seems it is 1:1 transformation while I need to drop bad messages (KafkaRDD => RDD it would be flatMap). Is there a way how to do it using messageHandler, is there another approach? Many thanks for any help. Petr
