[
https://issues.apache.org/jira/browse/FLINK-23132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martijn Visser closed FLINK-23132.
----------------------------------
Resolution: Abandoned
> flink upgrade issue(1.11.3->1.13.0)
> -----------------------------------
>
> Key: FLINK-23132
> URL: https://issues.apache.org/jira/browse/FLINK-23132
> Project: Flink
> Issue Type: Bug
> Reporter: Jeff Hu
> Priority: Major
>
>
> In order to improve the performance of data process, we store events to a map
> and do not process them untill event count reaches 100. in the meantime,
> start a timer in open method, so data is processed every 60 seconds
> this works when flink version is *1.11.3*,
> after upgrading flink version to *1.13.0*
> I found sometimes events were consumed from Kafka continuously, but were not
> processed in RichFlatMapFunction, it means data was missing. after restarting
> service, it works well, but several hours later the same thing happened again.
> any known issue for this flink version? any suggestions are appreciated.
> {{public class MyJob \{
> public static void main(String[] args) throws Exception {
> ...
> DataStream<String> rawEventSource = env.addSource(flinkKafkaConsumer);
> ...
> }}} {{public class MyMapFunction extends RichFlatMapFunction<String,
> String> implements Serializable \{
> @Override
> public void open(Configuration parameters) {
> ...
> long periodTimeout = 60;
> pool.scheduleAtFixedRate(() -> {
> // processing data
> }, periodTimeout, periodTimeout, TimeUnit.SECONDS);
> }
>
> @Override
> public void flatMap(String message, Collector<String> out) \{
> // store event to map
> // count event,
> // when count = 100, start data processing
> }
> }}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
