Re: DStreams stop consuming from Kafka

2020-11-16 Thread liyuanjian
Maybe you can try the `foreachBatch` API in structured streaming, which allows reusing existing datasources. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apa

DStreams stop consuming from Kafka

2020-11-10 Thread Razvan-Daniel Mihai
Hello, I have a usecase where I have to stream events from Kafka to a JDBC sink. Kafka producers write events in bursts of hourly batches. I started with a structured streaming approach, but it turns out that structured streaming has no JDBC sink. I found an implementation in Apache Bahir, but it