[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2024-02-20 Thread Kevin Tseng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819079#comment-17819079 ] Kevin Tseng commented on FLINK-33545: - Hi [~mason6345]  I have replicated the behavior you referred

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2024-02-15 Thread Kevin Tseng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817758#comment-17817758 ] Kevin Tseng commented on FLINK-33545: - Hi [~mason6345] , [~yang]  I have simplified the approach to

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2024-02-13 Thread Mason Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817136#comment-17817136 ] Mason Chen commented on FLINK-33545: Hi [~aeolus811tw], have you had time to review the followup

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2024-01-29 Thread Mason Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812113#comment-17812113 ] Mason Chen commented on FLINK-33545: [~yang] yup that's what I'm proposing. We could track an

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2024-01-29 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811763#comment-17811763 ] Martijn Visser commented on FLINK-33545: Since the reported exception is related to the Kafka

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2024-01-28 Thread Yang LI (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811646#comment-17811646 ] Yang LI commented on FLINK-33545: - Hello, If I understand correctly, [~mason6345] 's proposal doesn't

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2024-01-28 Thread Kevin Tseng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811642#comment-17811642 ] Kevin Tseng commented on FLINK-33545: - Hi [~mason6345]  I'm not sure that's possible without going

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2024-01-26 Thread Mason Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811156#comment-17811156 ] Mason Chen commented on FLINK-33545: I do think this issue here is not with the Flink runtime but it

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2024-01-25 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811037#comment-17811037 ] Martijn Visser commented on FLINK-33545: [~mason6345] WDYT about [~aeolus811tw] his last

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2024-01-16 Thread Yang LI (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807401#comment-17807401 ] Yang LI commented on FLINK-33545: - Hello [~aeolus811tw] [~mason6345] , Just find this ticket, do you

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-12-06 Thread Kevin Tseng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793866#comment-17793866 ] Kevin Tseng commented on FLINK-33545: - Hi Mason, Kafka Producer only throw exception in following

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-12-05 Thread Mason Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793457#comment-17793457 ] Mason Chen commented on FLINK-33545: The reason why you don't find any synchronization is because

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-11-30 Thread Kevin Tseng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791903#comment-17791903 ] Kevin Tseng commented on FLINK-33545: - Hi Mason, thank you for the comment. In the issue reported

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-11-30 Thread Mason Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791866#comment-17791866 ] Mason Chen commented on FLINK-33545: Hi all, I just saw this ticket. I've read through the thread

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-11-16 Thread Kevin Tseng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786933#comment-17786933 ] Kevin Tseng commented on FLINK-33545: - I'm also aware of the issue FLINK-31305 i don't believe that

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-11-16 Thread Kevin Tseng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786907#comment-17786907 ] Kevin Tseng commented on FLINK-33545: - Hi Tzu-Li, Thanks for the response. Below is my analysis of

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-11-16 Thread Tzu-Li (Gordon) Tai (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786873#comment-17786873 ] Tzu-Li (Gordon) Tai commented on FLINK-33545: -

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-11-16 Thread Tzu-Li (Gordon) Tai (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786863#comment-17786863 ] Tzu-Li (Gordon) Tai commented on FLINK-33545: - {code} In a scenario that: * KafkaSource

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-11-16 Thread Tzu-Li (Gordon) Tai (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786860#comment-17786860 ] Tzu-Li (Gordon) Tai commented on FLINK-33545: - hey [~aeolus811tw], thanks for the detailed

[jira] [Commented] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-11-14 Thread Kevin Tseng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786084#comment-17786084 ] Kevin Tseng commented on FLINK-33545: - we have a fix to address this issue, can anyone able assign