Hello All, I'm using StructuredStreaming to read data from Kafka, and need to do transformation on each individual row.
I'm trying to use 'foreach' (or foreachBatch), and running into issues. Basic question - how is the row passed to the function when foreach is used ? Also, when I use foreachBatch, seems the BatchId is available in the function called ? How do I access individual rows ? Details are in stackoverflow : https://stackoverflow.com/questions/71025312/structuredstreaming-foreach-foreachbatch-not-working What is the best approach for this use-case ? tia!
