Re: Structured Streaming, Reading and Updating a variable

2018-05-16 Thread Martin Engen
n Engen; user@spark.apache.org Subject: Re: Structured Streaming, Reading and Updating a variable Do you have a code sample, and detailed error message/exception to show? From: Martin Engen Date: Tuesday, May 15, 2018 at 9:24 AM To: "user@spark.apache.org" Subject: Structured

Re: Structured Streaming, Reading and Updating a variable

2018-05-15 Thread Lalwani, Jayesh
Do you have a code sample, and detailed error message/exception to show? From: Martin Engen Date: Tuesday, May 15, 2018 at 9:24 AM To: "user@spark.apache.org" Subject: Structured Streaming, Reading and Updating a variable Hello, I'm working with Structured Streaming, and I n

Re: Structured Streaming, Reading and Updating a variable

2018-05-15 Thread Koert Kuipers
You use a windowed aggregation for this On Tue, May 15, 2018, 09:23 Martin Engen wrote: > Hello, > > > > I'm working with Structured Streaming, and I need a method of keeping a > running average based on last 24hours of data. > > To help with this, I can use Exponential Smoothing, which means I

Structured Streaming, Reading and Updating a variable

2018-05-15 Thread Martin Engen
Hello, I'm working with Structured Streaming, and I need a method of keeping a running average based on last 24hours of data. To help with this, I can use Exponential Smoothing, which means I really only need to store 1 value from a previous calculation into the new, and update this variable as