[
https://issues.apache.org/jira/browse/FLINK-29737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17624376#comment-17624376
]
yuzelin commented on FLINK-29737:
---------------------------------
Thanks for your idea. I have some points about this feature.
# As far as I'm concerned, the `datagen` connector is usually used in test
cases, so how the data are generated is usually ignored. I wonder whether you
can share more details of `wave form generation rate` usage scenarios? I think
maybe I need to discuss with [~fsk119] and other community members to estimate
whether to support this feature.
# To add support for sin-based waveform generation rate is not scalable
enough. In my opinion, a better way is to introduce an option like
`RateShape`[1], and mark current mode as `square`. It backs to point 1 that
whether it's generic enough to introduce this new feature.
Anyway, thanks again. [~fsk119] Please take a look.
[1][`RateShape`
example|https://github.com/nexmark/nexmark/blob/54974ef36a0d01ef8ebc0b4ba39cfc50136af0f6/nexmark-flink/src/main/java/com/github/nexmark/flink/utils/NexmarkUtils.java#L53]
> Support DataGen on waveform function
> ------------------------------------
>
> Key: FLINK-29737
> URL: https://issues.apache.org/jira/browse/FLINK-29737
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
> Reporter: chenzihao
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-10-24-16-09-47-386.png,
> image-2022-10-24-16-09-52-410.png
>
>
> In some scenarios, we need to simulate flow changes in the production
> environment. The current DATAGEN feature only supports data generation at a
> constant rate. We try to simulate increments of flow using batch jobs, but
> the production rate is not smooth, so I suggest that we can support sin-based
> data generation in order to get smooth changes.
> 1. add another batch job to simulate increments of flow.
> !image-2022-10-24-16-09-52-410.png!
> 2. sin-based.
> !image-2022-10-24-16-09-47-386.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
