[
https://issues.apache.org/jira/browse/SPARK-48772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot reassigned SPARK-48772:
--------------------------------------
Assignee: Apache Spark
> State Data Source Read Change Feed
> ----------------------------------
>
> Key: SPARK-48772
> URL: https://issues.apache.org/jira/browse/SPARK-48772
> Project: Spark
> Issue Type: New Feature
> Components: Structured Streaming
> Affects Versions: 4.0.0
> Reporter: Yuchen Liu
> Assignee: Apache Spark
> Priority: Major
> Labels: pull-request-available
>
> The current state reader can only return the entire state at a specific
> version. If an error occurs related to state, knowing the change of state
> across versions to find out at which version state starts to go wrong is
> important for debugging purposes. This PR adds ability of showing the
> evolution of state as Change Data Capture (CDC) format to state data source.
> An example usage:
> {code:java}
> .format("statestore")
> .option("readChangeFeed", true)
> .option("changeStartBatchId", 5) #required
> .option("changeEndBatchId", 10) #not required, default: latest batch Id
> available
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
