[
https://issues.apache.org/jira/browse/FLINK-28728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martijn Visser closed FLINK-28728.
----------------------------------
Resolution: Incomplete
[~tanjialiang] The Flink Kafka Datastream connector can already be used for
specifying the offsets and therefore running in batch mode. If you do a
savepoint, then your job will restart next time with the previously defined
offsets. I do think that it makes sense that some connectors can be run in
batch mode, but there are already Jira tickets for that. For your use case, I
see more value in minimising cluster resources in case you have low volume.
There are already solutions for that in Flink and I'm sure that more
improvements are planned.
I'm closing the Jira ticket because it lacks clarity and more detailed/relevant
tickets already exist.
> Support to set the end offset for streaming connector
> -----------------------------------------------------
>
> Key: FLINK-28728
> URL: https://issues.apache.org/jira/browse/FLINK-28728
> Project: Flink
> Issue Type: Improvement
> Reporter: tanjialiang
> Priority: Major
>
> Like MQ or CDC connector, it support set the startup mode for reading, but
> not support set the end offset/position. When the MQ's or CDC's data is not
> frequently, consume their data in period can cut lots of kubernetes/YARN
> resource, such as startup from earliest, and end to the current offset, save
> current offset in savepoint, and using the savepoint to startup in the next
> period time, in a loop.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
