bump for better reach On Thu, Oct 12, 2023 at 4:26 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> wrote:
> Sorry, please use this link instead for SPIP doc: > https://docs.google.com/document/d/1_iVf_CIu2RZd3yWWF6KoRNlBiz5NbSIK0yThqG0EvPY/edit?usp=sharing > > > On Thu, Oct 12, 2023 at 3:58 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> > wrote: > >> Hi dev, >> >> I'd like to start a discussion on "State Data Source - Reader". >> >> This proposal aims to introduce a new data source "statestore" which >> enables reading the state rows from existing checkpoint via offline (batch) >> query. This will enable users to 1) create unit tests against stateful >> query verifying the state value (especially flatMapGroupsWithState), 2) >> gather more context on the status when an incident occurs, especially for >> incorrect output. >> >> *SPIP*: >> https://docs.google.com/document/d/1HjEupRv8TRFeULtJuxRq_tEG1Wq-9UNu-ctGgCYRke0/edit?usp=sharing >> *JIRA*: https://issues.apache.org/jira/browse/SPARK-45511 >> >> Looking forward to your feedback! >> >> Thanks, >> Jungtaek Lim (HeartSaVioR) >> >> ps. The scope of the project is narrowed to the reader in this SPIP, >> since the writer requires us to consider more cases. We are planning on it. >> >
