Process each kafka record for structured streaming

[rajat kumar]

Hi,

I want to apply custom logic for each row of data I am getting through
kafka and want to do it with microbatch.
When I am running it , it is not progressing.


kafka_stream_df \
        .writeStream \
        .foreach(process_records) \
        .outputMode("append") \
        .option("checkpointLocation", "checkpt") \
        .trigger(continuous="5 seconds").start()

Regards

Rajat