Re: Question on how to get appended data from structured streaming

I am trying to implements some online algorithms based on structured streaming currently. My requirement is fetching only delta data at each trigger time from memory and calculating and updating global variables at the same time. Here are 2 points I found it's difficult: 1. with foreach writer, it

Re: Question on how to get appended data from structured streaming

What is your end goal? Right now the foreach writer is the way to do arbitrary processing on the data produced by various output modes. On Sun, Aug 20, 2017 at 12:23 PM, Yanpeng Lin wrote: > Hello, > > I am new to Spark. > It would be appreciated if anyone could help me