Re: [DISCUSS] Externalized connector release details

2022-09-16 Thread Danny Cranmer
c) I am torn here. I do not like the idea that the connector code could diverge for the same connector version, ie 2.1.0-1.15 and 2.1.0-1.16. If the Flink version change requires a change to the connector code, then this should result in a new connector version in my opinion. Going back to your

[jira] [Created] (FLINK-29327) Operator configs are showing up among standard Flink configs

2022-09-16 Thread Matyas Orhidi (Jira)
Matyas Orhidi created FLINK-29327: - Summary: Operator configs are showing up among standard Flink configs Key: FLINK-29327 URL: https://issues.apache.org/jira/browse/FLINK-29327 Project: Flink

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-16 Thread Lincoln Lee
Hi Dawid, Piotr, Agree with you that add finish() method to `TableFunction` only. Other `UserDefinedFunction`s (`ScalarFunction`, `AggregateFunction`, `AggregateTableFunction`) are not necessarily to have the finish method(they can not emit records in legacy close() method). A `TableFunction`

[jira] [Created] (FLINK-29326) EndOfDataDecoderException observed when uploading a jar

2022-09-16 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-29326: - Summary: EndOfDataDecoderException observed when uploading a jar Key: FLINK-29326 URL: https://issues.apache.org/jira/browse/FLINK-29326 Project: Flink

Re: [DISCUSS] Externalized connector release details

2022-09-16 Thread Ryan Skraba
I had to write down a diagram to fully understand the discussion :D If I recall correctly, during the externalization discussion, the "price to pay" for the (many) advantages of taking connectors out of the main repo was to maintain and manually consult a compatibility matrix per connector. I'm

[jira] [Created] (FLINK-29325) Fix documentation bug on how to enable batch mode

2022-09-16 Thread Jun He (Jira)
Jun He created FLINK-29325: -- Summary: Fix documentation bug on how to enable batch mode Key: FLINK-29325 URL: https://issues.apache.org/jira/browse/FLINK-29325 Project: Flink Issue Type:

[jira] [Created] (FLINK-29324) Calling Kinesis connector close method before subtask starts running results in NPE

2022-09-16 Thread Anthony Pounds-Cornish (Jira)
Anthony Pounds-Cornish created FLINK-29324: -- Summary: Calling Kinesis connector close method before subtask starts running results in NPE Key: FLINK-29324 URL:

Re: [DISCUSS] Restructure FLIP pages with page properties

2022-09-16 Thread Ryan Skraba
This sounds like an obvious improvement to save tedious labour -- at the same time, would this be something interesting to organize lists of FLIP pages around topics? If I'm interested in the connectors, for example. +1 and thanks, Ryan On Fri, Sep 16, 2022 at 12:53 PM Yun Tang wrote: > +1,

Re: [DISCUSS] Restructure FLIP pages with page properties

2022-09-16 Thread Yun Tang
+1, thanks for driving this, Chesnay! Best Yun Tang From: Matthias Pohl Sent: Friday, September 16, 2022 18:21 To: dev@flink.apache.org Subject: Re: [DISCUSS] Restructure FLIP pages with page properties +1 Thanks for proposing that, Chesnay. I guess, it's a

Re: [DISCUSS] Restructure FLIP pages with page properties

2022-09-16 Thread Matthias Pohl
+1 Thanks for proposing that, Chesnay. I guess, it's a good idea to reduce efforts around documenting by removing duplicated content. On Fri, Sep 16, 2022 at 11:56 AM Chesnay Schepler wrote: > Hello, > > The current FLIP overview page [1] is kind of a mess and inconvenient to > modify. It

[jira] [Created] (FLINK-29323) Refine Transformer for VectorAssembler

2022-09-16 Thread weibo zhao (Jira)
weibo zhao created FLINK-29323: -- Summary: Refine Transformer for VectorAssembler Key: FLINK-29323 URL: https://issues.apache.org/jira/browse/FLINK-29323 Project: Flink Issue Type: New Feature

[DISCUSS] Restructure FLIP pages with page properties

2022-09-16 Thread Chesnay Schepler
Hello, The current FLIP overview page [1] is kind of a mess and inconvenient to modify. It requires a fair amount of manual work to get right, requiring updates in multiple places. I'd like to improve that, and experiment with the Page properties (Reports) and labels to auto-generate the

Fwd: [Discuss] Let's Session Cluster JobManager take a breather (FLIP-257: Flink JobManager Process Split)

2022-09-16 Thread Shammon FY
Thanks @XintongSong and sorry for replying late. Also thanks Zhengyu for bringing this discussion up. We use flink session cluster to run olap queries in ByteDance, and have supported several users in production. The maximum single cluster has 2000 cores, and we focus on the job scheduling

[jira] [Created] (FLINK-29322) Expose savepoint format on Web UI

2022-09-16 Thread Matyas Orhidi (Jira)
Matyas Orhidi created FLINK-29322: - Summary: Expose savepoint format on Web UI Key: FLINK-29322 URL: https://issues.apache.org/jira/browse/FLINK-29322 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-29321) Release connector

2022-09-16 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-29321: Summary: Release connector Key: FLINK-29321 URL: https://issues.apache.org/jira/browse/FLINK-29321 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-29320) Setup release scripts

2022-09-16 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-29320: Summary: Setup release scripts Key: FLINK-29320 URL: https://issues.apache.org/jira/browse/FLINK-29320 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Externalized connector release details

2022-09-16 Thread Chesnay Schepler
a) 2 Flink versions would be the obvious answer. I don't think anything else makes much sense. I don't want us to increment versions just because the Flink versions change, so in your example I'd go with 2.0.0-1.16. c) Generally speaking I would love to avoid the Flink versions in branch

[jira] [Created] (FLINK-29319) Upgrade Calcite version to 1.32

2022-09-16 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-29319: -- Summary: Upgrade Calcite version to 1.32 Key: FLINK-29319 URL: https://issues.apache.org/jira/browse/FLINK-29319 Project: Flink Issue Type: Improvement