Hi Mina I believe this is different for Structured Streaming from Kafka, specifically. I'm assuming you are using structured streaming based on the name of the dependency ("spark-streaming-kafka"). There is a note in the docs here: https://spark.apache.org/docs/2.2.0/structured-streaming-kafka-integration.html#kafka-specific-configurations
So, instead of setting Kafka's client property, instead set the Spark config "startingOffsets" to "earliest". There are examples here: https://spark.apache.org/docs/2.2.0/structured-streaming-kafka-integration.html#creating-a-kafka-source-for-batch-queries Setting "startingOffsets" has worked for me. I have not tried setting the kafka property directly. Cheers, Dave -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org