Re: StateStoreSaveExec / StateStoreRestoreExec

2017-01-03 Thread Michael Armbrust
You might also be interested in this: https://issues.apache.org/jira/browse/SPARK-19031 On Tue, Jan 3, 2017 at 3:36 PM, Michael Armbrust wrote: > I think we should add something similar to mapWithState in 2.2. It would > be great if you could add the description of your problem to this ticket:

Re: StateStoreSaveExec / StateStoreRestoreExec

2017-01-03 Thread Michael Armbrust
I think we should add something similar to mapWithState in 2.2. It would be great if you could add the description of your problem to this ticket: https://issues.apache.org/jira/browse/SPARK-19067 On Mon, Jan 2, 2017 at 2:05 PM, Jeremy Smith wrote: > I have a question about state tracking in St

StateStoreSaveExec / StateStoreRestoreExec

2017-01-02 Thread Jeremy Smith
I have a question about state tracking in Structured Streaming. First let me briefly explain my use case: Given a mutable data source (i.e. an RDBMS) in which we assume we can retrieve a set of newly created row versions (being a row that was created or updated between two given `Offset`s, whateve