Zhenhao Li created SPARK-25315:
----------------------------------
Summary: setting "auto.offset.reset" to "earliest" has no effect
in Structured Streaming with Spark 2.3.1 and Kafka 1.0
Key: SPARK-25315
URL: https://issues.apache.org/jira/browse/SPARK-25315
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 2.3.1
Environment: Standalone; running in IDEA
Reporter: Zhenhao Li
The following code won't read from the beginning of the topic
```
{code:java}
val kafkaOptions = Map[String, String](
"kafka.bootstrap.servers" -> KAFKA_BOOTSTRAP_SERVERS,
"subscribe" -> TOPIC,
"group.id" -> GROUP_ID,
"auto.offset.reset" -> "earliest"
)
val myStream = sparkSession
.readStream
.format("kafka")
.options(kafkaOptions)
.load()
.selectExpr("CAST(key AS STRING)", "CAST(value AS STRING)")
myStream
.writeStream
.format("console")
.start()
.awaitTermination()
{code}
```
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
