Re: SQL return type change from 1.17 to 1.18

2023-12-07 Thread Jing Ge
Hi all, +1 for reverting it. I'd like to volunteer as the release manager of 1.18.1 and will send the discussion email for the 1.18 release asap. Best regards, Jing On Fri, Dec 8, 2023 at 1:26 AM Rui Fan <1996fan...@gmail.com> wrote: > Hi all, > > +1 for reverting it. > > Also, I’m not sure

[jira] [Created] (FLINK-33778) Cleanup usage of deprecated TableConfig#setIdleStateRetentionTime(Time, Time)

2023-12-07 Thread Jacky Lau (Jira)
Jacky Lau created FLINK-33778: - Summary: Cleanup usage of deprecated TableConfig#setIdleStateRetentionTime(Time, Time) Key: FLINK-33778 URL: https://issues.apache.org/jira/browse/FLINK-33778 Project:

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-07 Thread Yu Li
+1 Thanks Leonard for the proposal, and all Flink CDC contributors for the existing great works. I believe these two projects are a perfect fit and the donation will benifit both. Best Regards, Yu On Fri, Dec 8, 2023 at 2:49 PM bai_wentao1 wrote: > +1 for this exciting work > > > Best

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-07 Thread bai_wentao1
+1 for this exciting work Best regards, WT | | bai_wentao1 | | bai_went...@163.com | Replied Message | From | ConradJam | | Date | 12/8/2023 14:26 | | To | | | Subject | Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink | +1 Best Idea . Thanks to all the

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-07 Thread ConradJam
+1 Best Idea . Thanks to all the contributors of Flink CDC for the great work. yuxia 于2023年12月7日周四 20:46写道: > +1 for this. Thanks all the contributors of Flink CDC for the great work. > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Martijn Visser" > 收件人: "dev" > 发送时间: 星期四, 2023年 12 月

Re: SQL return type change from 1.17 to 1.18

2023-12-07 Thread Rui Fan
Hi all, +1 for reverting it. Also, I’m not sure should we release 1.18.1 as soon as possible after it’s merged. If I understand correctly, users cannot upgrade flink job from 1.18.0 to 1.18.1 directly if we reverting it in 1.18.1, right? If so, it’s better to release 1.18.1 asap, and try to

[jira] [Created] (FLINK-33777) ParquetTimestampITCase>FsStreamingSinkITCaseBase failing in CI

2023-12-07 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33777: -- Summary: ParquetTimestampITCase>FsStreamingSinkITCaseBase failing in CI Key: FLINK-33777 URL: https://issues.apache.org/jira/browse/FLINK-33777 Project: Flink

[jira] [Created] (FLINK-33776) Allow to specify optional profile for connectors

2023-12-07 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-33776: --- Summary: Allow to specify optional profile for connectors Key: FLINK-33776 URL: https://issues.apache.org/jira/browse/FLINK-33776 Project: Flink Issue

[jira] [Created] (FLINK-33775) Report JobInitialization traces

2023-12-07 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-33775: -- Summary: Report JobInitialization traces Key: FLINK-33775 URL: https://issues.apache.org/jira/browse/FLINK-33775 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

2023-12-07 Thread Matthias Pohl
Thanks for the feedback, everyone. I realized that there was a bit of a misunderstanding around my suggestion for the migration plan. My intention was what Yangze Guo and Xintong had in mind: Azure CI should be handled as the ground truth. GitHub Actions CI should be only used in the PRs to allow

Re: SQL return type change from 1.17 to 1.18

2023-12-07 Thread Xingcan Cui
Hi, Thanks for bringing this up, Peter. I'm +1 for reverting the change. Best, Xingcan On Thu, Dec 7, 2023 at 10:40 AM Martijn Visser wrote: > Hi all, > > Agree with what has been said already. I've marked > https://issues.apache.org/jira/browse/FLINK-33523 as a blocker ticket > for 1.18.1 >

Re: SQL return type change from 1.17 to 1.18

2023-12-07 Thread Martijn Visser
Hi all, Agree with what has been said already. I've marked https://issues.apache.org/jira/browse/FLINK-33523 as a blocker ticket for 1.18.1 Best regards, Martijn On Thu, Dec 7, 2023 at 4:25 PM Márton Balassi wrote: > > Thanks, for raising this Peter. +1 for reverting the change. > > Given the

Re: SQL return type change from 1.17 to 1.18

2023-12-07 Thread Márton Balassi
Thanks, for raising this Peter. +1 for reverting the change. Given the response from Timo and Aitozi, I believe it would be best if we could ship reverting the change in 1.18.1. On Thu, Dec 7, 2023 at 2:47 PM Aitozi wrote: > Hi Peter, Timo > Sorry for this breaking change, I didn't notice

[jira] [Created] (FLINK-33774) StreamTaskSelectiveReadingTest.testReadFinishedInput(StreamTaskSelectiveReadingTest timed out

2023-12-07 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33774: - Summary: StreamTaskSelectiveReadingTest.testReadFinishedInput(StreamTaskSelectiveReadingTest timed out Key: FLINK-33774 URL: https://issues.apache.org/jira/browse/FLINK-33774

Re: [DISCUSS] Change the default restart-strategy to exponential-delay

2023-12-07 Thread Maximilian Michels
Hey Rui, +1 for changing the default restart strategy to exponential-delay. This is something all users eventually run into. They end up changing the restart strategy to exponential-delay. I think the current defaults are quite balanced. Restarts happen quickly enough unless there are consecutive

[jira] [Created] (FLINK-33773) Add fairness to scaling decisions

2023-12-07 Thread Maximilian Michels (Jira)
Maximilian Michels created FLINK-33773: -- Summary: Add fairness to scaling decisions Key: FLINK-33773 URL: https://issues.apache.org/jira/browse/FLINK-33773 Project: Flink Issue Type:

[jira] [Created] (FLINK-33772) Python e2e test fails with exit code 1

2023-12-07 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33772: - Summary: Python e2e test fails with exit code 1 Key: FLINK-33772 URL: https://issues.apache.org/jira/browse/FLINK-33772 Project: Flink Issue Type:

[jira] [Created] (FLINK-33771) Add cluster capacity awareness to Autoscaler

2023-12-07 Thread Maximilian Michels (Jira)
Maximilian Michels created FLINK-33771: -- Summary: Add cluster capacity awareness to Autoscaler Key: FLINK-33771 URL: https://issues.apache.org/jira/browse/FLINK-33771 Project: Flink

[jira] [Created] (FLINK-33770) Autoscaler logs are full of deprecated key warnings

2023-12-07 Thread Maximilian Michels (Jira)
Maximilian Michels created FLINK-33770: -- Summary: Autoscaler logs are full of deprecated key warnings Key: FLINK-33770 URL: https://issues.apache.org/jira/browse/FLINK-33770 Project: Flink

Re: SQL return type change from 1.17 to 1.18

2023-12-07 Thread Aitozi
Hi Peter, Timo Sorry for this breaking change, I didn't notice that was a breaking change. I'm +1 to revert the FLINK-33523 Regards, aitozi Timo Walther 于2023年12月7日周四 20:41写道: > Hi Peter, > > thanks for reaching out to the Flink community. This is indeed a serious > issue. As the author of

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-07 Thread yuxia
+1 for this. Thanks all the contributors of Flink CDC for the great work. Best regards, Yuxia - 原始邮件 - 发件人: "Martijn Visser" 收件人: "dev" 发送时间: 星期四, 2023年 12 月 07日 下午 8:14:36 主题: Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink Hi Leonard, +1 for this. I think the

Re: SQL return type change from 1.17 to 1.18

2023-12-07 Thread Timo Walther
Hi Peter, thanks for reaching out to the Flink community. This is indeed a serious issue. As the author of the Flink type system, DataType and many related utilities I strongly vote for reverting FLINK-33523: - It changes the Flink type system without a FLIP. - It breaks backwards

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-07 Thread Martijn Visser
Hi Leonard, +1 for this. I think the CDC connectors are a great result and example of the Flink CDC community. Kudos to you all. Best regards, Martijn On Thu, Dec 7, 2023 at 10:40 AM Márton Balassi wrote: > > Hi Leonard, > > Thank you for the excellent work you and the team working on the CDC

[jira] [Created] (FLINK-33769) ExternalSorter hits "java.lang.RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: java.io.EOFExceptio

2023-12-07 Thread Vishal Palla (Jira)
Vishal Palla created FLINK-33769: Summary: ExternalSorter hits "java.lang.RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: java.io.EOFException: Can't collect further:

[DISCUSS] FLIP-401: REST API JSON response deserialization unknown field tolerance

2023-12-07 Thread Gabor Somogyi
Hi All, I'd like to start a discussion of FLIP-401: REST API JSON response deserialization unknown field tolerance[1]. *Problem statement:* At the moment Flink is not ignoring unknown fields when parsing REST responses. An example for such a class is *JobDetailsInfo* but this applies to all

Re: [DISCUSS] REST API response parsing throws exception on new fields

2023-12-07 Thread Gabor Somogyi
Switching to the new thread... G On Thu, Dec 7, 2023 at 10:25 AM Gyula Fóra wrote: > You should start a new discussion thread according to the FLIP guidelines, > so that it's identifiable that this is a FLIP discussion. > > Gyula > > On Thu, Dec 7, 2023 at 10:22 AM Gabor Somogyi > wrote: > >

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-07 Thread Márton Balassi
Hi Leonard, Thank you for the excellent work you and the team working on the CDC connectors project have been doing so far. I am +1 of having them under Flink's umbrella. On Thu, Dec 7, 2023 at 10:26 AM Etienne Chauchot wrote: > Big +1, thanks this will be a very useful addition to Flink. > >

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-07 Thread Etienne Chauchot
Big +1, thanks this will be a very useful addition to Flink. Best Etienne Le 07/12/2023 à 09:26, Hang Ruan a écrit : +1 for contributing CDC Connectors to Apache Flink. Best, Hang Yuxin Tan 于2023年12月7日周四 16:05写道: Cool, +1 for contributing CDC Connectors to Apache Flink. Best, Yuxin

Re: [DISCUSS] REST API response parsing throws exception on new fields

2023-12-07 Thread Gyula Fóra
You should start a new discussion thread according to the FLIP guidelines, so that it's identifiable that this is a FLIP discussion. Gyula On Thu, Dec 7, 2023 at 10:22 AM Gabor Somogyi wrote: > Gyula, thanks for sharing your opinion. > I've created a small FLIP as asked: > >

Re: [DISCUSS] REST API response parsing throws exception on new fields

2023-12-07 Thread Gabor Somogyi
Gyula, thanks for sharing your opinion. I've created a small FLIP as asked: https://cwiki.apache.org/confluence/display/FLINK/FLIP-401%3A+REST+API+JSON+response+deserialization+unknown+field+tolerance Let's wait on voices until beginning of next week. G On Wed, Dec 6, 2023 at 11:05 AM Gyula

Re: [DISCUSSION] Consider Flink operator having a way to monitor the status of bounded streaming jobs after they finish or error?

2023-12-07 Thread richard.su
Thanks, Gyula, this is one of our technical debt in our platform's develop progress, which is helpless for me. If the operator's monitor will sometimesmissing bounded job, we may change our strategy to modified the docker-entrypoint.sh of our flink-custom-image, which capture the exit

Re: Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-07 Thread Martijn Visser
Hi Xuyang, Thanks a lot for starting this discussion. At first, I was a bit confused because the FLIP talks about deprecating the Legacy Group Window Aggregations, but they have already been marked as deprecated in the documentation [1]. My understanding was that the big challenge was that we

Re: [DISCUSSION] Consider Flink operator having a way to monitor the status of bounded streaming jobs after they finish or error?

2023-12-07 Thread Gyula Fóra
This config has nothing to do with the operator (it's a core flink feature) and is not an issue after Flink 1.15. Newer operator versions (1.7+) drop support for Flink 1.13 and 1.14 as it's not feasible to maintain too many legacy codepaths. The only solution for you is to update your Flink

Re: [DISCUSSION] Consider Flink operator having a way to monitor the status of bounded streaming jobs after they finish or error?

2023-12-07 Thread richard.su
Hi Gyula, Flink version is 1.14 Our flink version is hard to upgrade since we have some user in our platform. sorry I have not noticed this configuration, it's confusing because flink operator announced support from 1.13 to 1.17/1.18 Has other solution will work in our situation? Thanks

Re: [DISCUSSION] Consider Flink operator having a way to monitor the status of bounded streaming jobs after they finish or error?

2023-12-07 Thread richard.su
Hi Gyula, Flink version is 1.14 Our flink version is hard to upgrade since we have some user in our platform. sorry I have not noticed this configuration, it's confusing because flink operator announced support from 1.13 to 1.17/1.18 Has other solution will work in our situation? Thanks

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-07 Thread Hang Ruan
+1 for contributing CDC Connectors to Apache Flink. Best, Hang Yuxin Tan 于2023年12月7日周四 16:05写道: > Cool, +1 for contributing CDC Connectors to Apache Flink. > > Best, > Yuxin > > > Jing Ge 于2023年12月7日周四 15:43写道: > > > Awesome! +1 > > > > Best regards, > > Jing > > > > On Thu, Dec 7, 2023 at

Re: [DISCUSSION] Consider Flink operator having a way to monitor the status of bounded streaming jobs after they finish or error?

2023-12-07 Thread Gyula Fóra
Hi! What Flink version are you using? The operator always sets: execution.shutdown-on-application-finish to false so that finished / failed application clusters should not exit immediately and we can observe them. This is however only available in Flink 1.15 and above. Cheers, Gyula On Thu,

[DISCUSSION] Consider Flink operator having a way to monitor the status of bounded streaming jobs after they finish or error?

2023-12-07 Thread richard.su
Hi, Community, I had found out this issue, but I'm not sure this issue have any solution. I have tried flink operator 1.6, which this issue is still exist. If not, I think this could create a jira issue to following. When we create a bounded streaming jobs which will finally to become Finished

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-07 Thread Yuxin Tan
Cool, +1 for contributing CDC Connectors to Apache Flink. Best, Yuxin Jing Ge 于2023年12月7日周四 15:43写道: > Awesome! +1 > > Best regards, > Jing > > On Thu, Dec 7, 2023 at 8:34 AM Sergey Nuyanzin > wrote: > > > thanks for working on this and driving it > > > > +1 > > > > On Thu, Dec 7, 2023 at