HeartSaVioR commented on code in PR #47274: URL: https://github.com/apache/spark/pull/47274#discussion_r1673492943
########## docs/structured-streaming-state-data-source.md: ########## @@ -144,16 +143,126 @@ The following configurations are optional: <td>(none)</td> <td>Represents the target side to read from. This option is used when users want to read the state from stream-stream join.</td> </tr> +<tr> + <td>snapshotStartBatchId</td> + <td>numeric value</td> + <td></td> + <td>If specified, force to read the snapshot at this batch ID, then changelogs will be replayed until 'batchId' or its default. Note that snapshot batch ID starts with 0 and equals to snapshot version ID minus 1. This option must be used together with 'snapshotPartitionId'.</td> +</tr> +<tr> + <td>snapshotPartitionId</td> + <td>numeric value</td> + <td></td> + <td>If specified, only this specific partition will be read. Note that partition ID starts with 0. This option must be used together with 'snapshotStartBatchId'.</td> +</tr> +<tr> + <td>readChangeFeed</td> + <td>boolean</td> + <td>false</td> + <td>If set to true, will read the change of state over microbatches. The output table schema will also change. Two columns 'batch_id'(long) and 'change_type'(string) will be appended to the front. Option 'changeStartBatchId' must be specified with this option. Option 'batchId', 'joinSide', 'snapshotStartBatchId', 'snapshotPartitionId' is conflict with this option. An example usage of this option can be found below.</td> Review Comment: > The output table schema will also change. Two columns 'batch_id'(long) and 'change_type'(string) will be appended to the front. We could simply defer to the next section, to make the explanation to be concise. > is conflict with this option cannot be used with this option (probably more clearer) > An example usage of this option can be found below. probably better to explicitly mention the section name? link would be even better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
