[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-11-15 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634383#comment-17634383 ] Yordan Pavlov commented on FLINK-16419: --- Thanks [~pnowojski] for your extensive explanation. I

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-11-15 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634229#comment-17634229 ] Piotr Nowojski commented on FLINK-16419: Hi [~YordanPavlov], as explained in this ticket, Flink

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-11-14 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633871#comment-17633871 ] Martijn Visser commented on FLINK-16419: [~pnowojski] Could you help out here? > Avoid to

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-10-26 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17624315#comment-17624315 ] Yordan Pavlov commented on FLINK-16419: --- Let me try to provide some more info on how the error

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-10-21 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17622315#comment-17622315 ] Yordan Pavlov commented on FLINK-16419: --- Again, thanks for you input [~martijnvisser]  > 1. When

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-10-21 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1760#comment-1760 ] Martijn Visser commented on FLINK-16419: I've synced offline with [~pnowojski] on this topic.

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-10-20 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621314#comment-17621314 ] Yordan Pavlov commented on FLINK-16419: --- Thanks for your feedback [~martijnvisser]  > If you're

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-10-20 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621298#comment-17621298 ] Martijn Visser commented on FLINK-16419: [~YordanPavlov] Let's see if we can resolve this. A

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-10-20 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621293#comment-17621293 ] Martijn Visser commented on FLINK-16419: [~YordanPavlov] Just to verify, do you have unlimited

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-10-20 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621220#comment-17621220 ] Yordan Pavlov commented on FLINK-16419: --- [~martijnvisser] This issue has plagued our Flink setup

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-08-17 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580807#comment-17580807 ] Yordan Pavlov commented on FLINK-16419: --- We are having considerable difficulties with this setup

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-07-11 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564811#comment-17564811 ] Martijn Visser commented on FLINK-16419: > Savepoint usability shouldn't be bound to Kafka

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-07-07 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563927#comment-17563927 ] Yordan Pavlov commented on FLINK-16419: --- Savepoint usability shouldn't be bound to Kafka

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-07-07 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563910#comment-17563910 ] Martijn Visser commented on FLINK-16419: [~YordanPavlov] Why do you think that's not right? You

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-07-07 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563894#comment-17563894 ] Yordan Pavlov commented on FLINK-16419: --- [~martijnvisser] From what you are saying it sounds like

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-07-07 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563810#comment-17563810 ] Martijn Visser commented on FLINK-16419: The default value for

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-07-07 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563715#comment-17563715 ] Yordan Pavlov commented on FLINK-16419: --- [~martijnvisser] Yes, quite a lot. I am trying to recover

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-07-07 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563675#comment-17563675 ] Martijn Visser commented on FLINK-16419: Have you had checkpoints happen since that one of 3

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-07-07 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563616#comment-17563616 ] Yordan Pavlov commented on FLINK-16419: --- [~martijnvisser]  I am still seeing the issue under some

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-06-30 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17560854#comment-17560854 ] Martijn Visser commented on FLINK-16419: [~YordanPavlov] Did that also resolve the problem for

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-06-29 Thread Yordan Pavlov (Jira)
Title: Message Title Yordan Pavlov

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-06-29 Thread Martijn Visser (Jira)
Title: Message Title Martijn Visser

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-06-28 Thread Yordan Pavlov (Jira)
Title: Message Title Yordan Pavlov

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-05-04 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531601#comment-17531601 ] Martijn Visser commented on FLINK-16419: [~YordanPavlov] I highly recommend switching to the

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-05-04 Thread Yordan Pavlov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531594#comment-17531594 ] Yordan Pavlov commented on FLINK-16419: --- I am experiencing a similar issue, recovering from

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-02-17 Thread Jun Qin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493887#comment-17493887 ] Jun Qin commented on FLINK-16419: - I think we are going a bit off-topic here. The original request was

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-02-17 Thread Fabian Paul (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493758#comment-17493758 ] Fabian Paul commented on FLINK-16419: - The KafkaSink is a bit more relaxed when it comes to errors

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-02-17 Thread Jun Qin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493749#comment-17493749 ] Jun Qin commented on FLINK-16419: - My understanding is that [~fpaul] meant the problem mentioned by

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-02-16 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493724#comment-17493724 ] Martijn Visser commented on FLINK-16419: [~qinjunjerry]It does, according to this comment

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-02-16 Thread Jun Qin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493482#comment-17493482 ] Jun Qin commented on FLINK-16419: - [~MartijnVisser] I am not quite sure whether the new KafkaSink works

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2022-01-24 Thread Martijn Visser (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481049#comment-17481049 ] Martijn Visser commented on FLINK-16419: [~qinjunjerry] Should we close this ticket since we're

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-12-03 Thread Jun Qin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453218#comment-17453218 ] Jun Qin commented on FLINK-16419: - [~qzhzm173227] not sure you have opened a new Jira...But what you

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-11-24 Thread Jiahui Jiang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448700#comment-17448700 ] Jiahui Jiang commented on FLINK-16419: -- [~fpaul] thank you for pointing that out. I think upgrading

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-11-24 Thread Fabian Paul (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448680#comment-17448680 ] Fabian Paul commented on FLINK-16419: - Thanks for the additional information. I think we should

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-11-24 Thread Jiahui Jiang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448674#comment-17448674 ] Jiahui Jiang commented on FLINK-16419: -- [~fpaul] Correct me if I'm looking at the wrong code path,

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-11-23 Thread Fabian Paul (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448416#comment-17448416 ] Fabian Paul commented on FLINK-16419: - The error still seems very strange to me because every time

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-11-22 Thread Jiahui Jiang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447628#comment-17447628 ] Jiahui Jiang commented on FLINK-16419: -- [~fpaul] This happened in our production again today :(

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-11-18 Thread Jiahui Jiang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445945#comment-17445945 ] Jiahui Jiang commented on FLINK-16419: -- [~fpaul] we actually set it to be super high (2147483647ms

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-11-17 Thread Fabian Paul (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445711#comment-17445711 ] Fabian Paul commented on FLINK-16419: - [~qzhzm173227] every transaction is closed when the

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-11-16 Thread Jiahui Jiang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444812#comment-17444812 ] Jiahui Jiang commented on FLINK-16419: -- [~becket_qin] [~qinjunjerry]  We are seeing issues in

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-04-29 Thread Flink Jira Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17336286#comment-17336286 ] Flink Jira Bot commented on FLINK-16419: This issue was labeled "stale-major" 7 ago and has not

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2021-04-22 Thread Flink Jira Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17328011#comment-17328011 ] Flink Jira Bot commented on FLINK-16419: This major issue is unassigned and itself and all of

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2020-03-05 Thread Jun Qin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053086#comment-17053086 ] Jun Qin commented on FLINK-16419: - That should be sufficient. > Avoid to recommit transactions which

[jira] [Commented] (FLINK-16419) Avoid to recommit transactions which are known committed successfully to Kafka upon recovery

2020-03-05 Thread Jiangjie Qin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17052605#comment-17052605 ] Jiangjie Qin commented on FLINK-16419: -- I am wondering if a simple "Clean Shutdown" marker would be