Re: Process each kafka record for structured streaming

Hi, Can you use console sink and make sure that the pipeline shows some progress? Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski "The Internals Of" Online Books Follow me on https://twitter.com/jaceklaskowski On

Process each kafka record for structured streaming

Hi, I want to apply custom logic for each row of data I am getting through kafka and want to do it with microbatch. When I am running it , it is not progressing. kafka_stream_df \ .writeStream \ .foreach(process_records) \ .outputMode("append") \ .option("checkpoi