Re: Can mapWithState state func be called every batchInterval?

Actually each element of mapwithstate has a time out component. You can write a function to "treat" your time out. You can match it with your batch size and do fun stuff when the batch ends. People do session management with the same approach. When activity is registered the session is

Re: Can mapWithState state func be called every batchInterval?

What are you expecting? If you want to update every key on every batch, it's going to be linear on the number of keys... there's no real way around that. On Tue, Oct 11, 2016 at 9:49 AM, Daan Debie wrote: > That's nice and all, but I'd rather have a solution involving

Re: Can mapWithState state func be called every batchInterval?

That's nice and all, but I'd rather have a solution involving mapWithState of course :) I'm just wondering why it doesn't support this use case yet. On Tue, Oct 11, 2016 at 3:41 PM, Cody Koeninger wrote: > They're telling you not to use the old function because it's linear

Re: Can mapWithState state func be called every batchInterval?

They're telling you not to use the old function because it's linear on the total number of keys, not keys in the batch, so it's slow. But if that's what you really want, go ahead and do it, and see if it performs well enough. On Oct 11, 2016 6:28 AM, "DandyDev" wrote: Hi