bharath kumar avusherla created SPARK-24189:
-----------------------------------------------
Summary: Spark Strcutured Streaming not working with the Kafka
Transactions
Key: SPARK-24189
URL: https://issues.apache.org/jira/browse/SPARK-24189
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.3.0
Reporter: bharath kumar avusherla
Was trying to read kafka transactional topic using Spark Structured Streaming
2.3.0 with the kafka option isolation-level = "read_committed", but spark
reading the data immediately without waiting for the data in topic to be
committed. In spark documentation it was mentioned as Structured Streaming
supports Kafka version 0.10 or higher. I am using below command to test the
scenario.
val df = spark
.readStream
.format("kafka")
.option("kafka.bootstrap.servers", "localhost:9092")
.option("subscribe", "test-topic")
.option("isolation-level","read_committed")
.load()
Can you please let me know if the transactional read is supported in SPark
2.3.0 strcutured Streaming or am i missing anything.
Thank you.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
