LadyForest commented on code in PR #248:
URL: https://github.com/apache/flink-table-store/pull/248#discussion_r931754507
##########
docs/content/docs/development/streaming-query.md:
##########
@@ -75,17 +75,39 @@ streaming mode. This mode has a lower cost compared to
Kafka but has a higher la
depending on the checkpoint interval of the writing job.
By default, the downstream streaming consumption is disordered (ordered within
the key)
-stream of upsert data. If you expect an ordered CDC data stream, you can
configure it
-as follows (recommended):
+stream of upsert data. If you expect an ordered CDC data stream, and remove
downstream
+changelog normalized operator (which is costly), you can configure it as
follows
+(Recommended, but this requires that your input is inclusive of all
changelogs):
```sql
CREATE TABLE T (...)
WITH (
- 'changelog-producer' = 'input',
- 'log.changelog-mode' = 'all'
+ 'changelog-producer' = 'input'
)
```
+You can understand the difference between changelog-producer(none) and
changelog-producer(input) by the following pictures:
+
+{{< img src="/img/changelog-producer-none.png">}}
+
+When the changelog-producer is none, because the storage only retains the
upsert data and
+does not have the full changelog data containing update_before, so the
downstream consumption job needs
+to use the normalized node to generate the complete changelog.
+
+{{< hint info >}}
+__Note:__ The normalized node needs to persist all the data into the state,
which is very costly.
+{{< /hint >}}
+
+{{< img src="/img/changelog-producer-input.png">}}
+
+When the changelog-producer is input, the storage trust input data is a
complete changelog and
Review Comment:
```suggestion
When the changelog-producer is input, the storage trusts input data is saved
as a complete changelog so that ...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
