[jira] [Created] (FLINK-33614) Docker pull failed

2023-11-21 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33614: - Summary: Docker pull failed Key: FLINK-33614 URL: https://issues.apache.org/jira/browse/FLINK-33614 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

2023-11-21 Thread Hongshun Wang
Hi Becket, Thanks a lot, I have no problem any more. And I have made further modifications to FLIP-389[1]. In summary, this flip has 2 goals: - Annotate SingleThreadFetcherManager as PublicEvolving. - Shield FutureCompletingBlockingQueue from users and limit all operations on

[jira] [Created] (FLINK-33613) Python UDF Runner process leak in Process Mode

2023-11-21 Thread Yu Chen (Jira)
Yu Chen created FLINK-33613: --- Summary: Python UDF Runner process leak in Process Mode Key: FLINK-33613 URL: https://issues.apache.org/jira/browse/FLINK-33613 Project: Flink Issue Type: Bug

Re: [NOTICE] flink-table-planner module has been migrated to JUnit5

2023-11-21 Thread Jane Chan
Thanks Leonard and Jiabao for your tremendous efforts in this regard! Best, Jane On Wed, Nov 22, 2023 at 2:42 PM Leonard Xu wrote: > Hi, all > > There are a lot of JUnit4 tests in flink-table-planner module, but now > Jiabao and I have just migrated all tests to JUnit5 through FLINK-29451[1].

Re: [VOTE] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

2023-11-21 Thread Hangxiang Yu
+1(binding) On Wed, Nov 22, 2023 at 10:29 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > On Wed, Nov 22, 2023 at 1:20 AM Piotr Nowojski > wrote: > > > Hi All, > > > > I'd like to start a vote on the FLIP-385: Add OpenTelemetryTraceReporter > > and

Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-21 Thread Zakelly Lan
+1(non-binding) Best, Zakelly On Wed, Nov 22, 2023 at 3:04 PM Hangxiang Yu wrote: > +1 (binding) > Thanks for driving this again! > > On Wed, Nov 22, 2023 at 10:30 AM Rui Fan <1996fan...@gmail.com> wrote: > > > +1(binding) > > > > Best, > > Rui > > > > On Wed, Nov 22, 2023 at 6:43 AM Jing Ge

Re: [VOTE] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-21 Thread Hangxiang Yu
+1 (binding) Thanks for your efforts! On Mon, Nov 20, 2023 at 11:53 AM Rui Fan <1996fan...@gmail.com> wrote: > Hi everyone, > > Thank you to everyone for the feedback on FLIP-390: Support > System out and err to be redirected to LOG or discarded[1] > which has been discussed in this thread [2].

Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-21 Thread Hangxiang Yu
+1 (binding) Thanks for driving this again! On Wed, Nov 22, 2023 at 10:30 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > On Wed, Nov 22, 2023 at 6:43 AM Jing Ge > wrote: > > > +1(binding) Thanks! > > > > Best regards, > > Jing > > > > On Tue, Nov 21, 2023 at 6:17 PM

[NOTICE] flink-table-planner module has been migrated to JUnit5

2023-11-21 Thread Leonard Xu
Hi, all There are a lot of JUnit4 tests in flink-table-planner module, but now Jiabao and I have just migrated all tests to JUnit5 through FLINK-29451[1]. Please use JUnit5 tests when submitting new PRs for flink-table-planner module. The opened PRs for flink-table-planner module need to

Re:Re: [DISCUSS][FLINK-32993] Datagen connector handles length-constrained fields according to the schema definition by default

2023-11-21 Thread Xuyang
Hi, Yubin and Jane. Big +1 for this fix. I also agree with Lincoln's view about throwing an error instead of leave all the complexity to the framework if it is obvious that the length in schema and in with options are in conflict. About the four examples Jane provides, I think only the

[jira] [Created] (FLINK-33612) The table plan of hybrid shuffle may introduce additional blocking edges occasionally

2023-11-21 Thread Yuxin Tan (Jira)
Yuxin Tan created FLINK-33612: - Summary: The table plan of hybrid shuffle may introduce additional blocking edges occasionally Key: FLINK-33612 URL: https://issues.apache.org/jira/browse/FLINK-33612

Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-21 Thread Rui Fan
+1(binding) Best, Rui On Wed, Nov 22, 2023 at 6:43 AM Jing Ge wrote: > +1(binding) Thanks! > > Best regards, > Jing > > On Tue, Nov 21, 2023 at 6:17 PM Piotr Nowojski > wrote: > > > Hi All, > > > > I'd like to start a vote on the FLIP-384: Introduce TraceReporter and use > > it to create

Re: [VOTE] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

2023-11-21 Thread Rui Fan
+1(binding) Best, Rui On Wed, Nov 22, 2023 at 1:20 AM Piotr Nowojski wrote: > Hi All, > > I'd like to start a vote on the FLIP-385: Add OpenTelemetryTraceReporter > and OpenTelemetryMetricReporter [1]. The discussion thread is here [2]. > > The vote will be open for at least 72 hours unless

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

2023-11-21 Thread Becket Qin
Hi Hongshun, The constructor of the SplitFetcher is already package private. So it can only be accessed from the classes in the package org.apache.flink.connector.base.source.reader.fetcher. And apparently, user classes should not be in this package. Therefore, even if we mark the SplitFetcher

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

2023-11-21 Thread Hongshun Wang
Hi Becket, If SplitFetcherManager becomes PublicEvolving, that also means SplitFetcher > needs to be PublicEvolving, because it is returned by the protected method > SplitFetcherManager.createSplitFetcher(). > it looks like there is no need to expose the constructor of SplitFetcher > to the

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Lincoln Lee
+1 (binding) Best, Lincoln Lee Sergey Nuyanzin 于2023年11月22日周三 07:55写道: > +1 (binding) > > On Tue, Nov 21, 2023 at 3:17 PM Jim Hughes > wrote: > > > +1 (non-binding) > > > > Thanks, Dawid! > > > > Jim > > > > On Tue, Nov 21, 2023 at 7:20 AM Gyula Fóra wrote: > > > > > +1 (binding) > > > > >

Re: [DISCUSS] FLIP-386: Support adding custom metrics in Recovery Spans

2023-11-21 Thread Roman Khachatryan
Thanks for clarifying, I see your points (although reporting metrics as spans still seems counter intuitive to me). As for the aggregation, I'm concerned that it might be unnecessarily ambiguous: where the aggregation is performed (JM/TM); across what (tasks/time); and which aggregation should

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Sergey Nuyanzin
+1 (binding) On Tue, Nov 21, 2023 at 3:17 PM Jim Hughes wrote: > +1 (non-binding) > > Thanks, Dawid! > > Jim > > On Tue, Nov 21, 2023 at 7:20 AM Gyula Fóra wrote: > > > +1 (binding) > > > > Gyula > > > > On Tue, 21 Nov 2023 at 13:11, xiangyu feng wrote: > > > > > +1 (non-binding) > > > > > >

Re: [DISCUSSION] flink-connector-shared-utils release process

2023-11-21 Thread Sergey Nuyanzin
Hi Etienne thanks for starting this discussion +1 from my side for snapshots since I also faced same issue On Thu, Nov 9, 2023 at 11:21 AM Etienne Chauchot wrote: > Hi all, > > flink-connector-shared-utils contains utilities for connectors (parent > pom, ci scripts, template test connector

Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-21 Thread Jing Ge
+1(binding) Thanks! Best regards, Jing On Tue, Nov 21, 2023 at 6:17 PM Piotr Nowojski wrote: > Hi All, > > I'd like to start a vote on the FLIP-384: Introduce TraceReporter and use > it to create checkpointing and recovery traces [1]. The discussion thread > is here [2]. > > The vote will be

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-21 Thread Jing Ge
Hi Piotr, Thanks for the clarification. Looking forward to the great feature! Best regards, Jing On Mon, Nov 20, 2023 at 10:03 AM Piotr Nowojski wrote: > Hi Jing! > > > the upcoming OpenTelemetry based TraceReporter will use the same Span > > implementation and will not support trace_id and

Request to make ContinuousFileReaderOperator public

2023-11-21 Thread Darin Amos
Hi All! I posted on the community slack channel and was referred to this mailing list. I think it would be helpful if the ContinuousFileReaderOperator was made a public class and not removed in Flink 2.0 (or to have an equivalent created). I have a use case for it where FileSource isn't

[jira] [Created] (FLINK-33611) Add the ability to reuse variable names across different split method scopes

2023-11-21 Thread Sai Sharath Dandi (Jira)
Sai Sharath Dandi created FLINK-33611: - Summary: Add the ability to reuse variable names across different split method scopes Key: FLINK-33611 URL: https://issues.apache.org/jira/browse/FLINK-33611

[VOTE] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

2023-11-21 Thread Piotr Nowojski
Hi All, I'd like to start a vote on the FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter [1]. The discussion thread is here [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1] https://cwiki.apache.org/confluence/x/UAuZE

Re: [DISCUSS] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

2023-11-21 Thread Piotr Nowojski
Thanks Roman. It kind of does depend, as the API is defined in the FLIP-384. But the voting thread for that FLIP has started, so I think we can also start the voting process here. Best, Piotrek czw., 16 lis 2023 o 15:24 Roman Khachatryan napisał(a): > Thanks Piotr, the proposal totally makes

[VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-21 Thread Piotr Nowojski
Hi All, I'd like to start a vote on the FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces [1]. The discussion thread is here [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1]

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-21 Thread Piotr Nowojski
Hi, I will start the voting thread shortly. Jing, I'm making an assumption here/hoping that my answers have satisfied you. If not, please let me know. Best, Piotrek pon., 20 lis 2023 o 09:59 Piotr Nowojski napisał(a): > Hi Jing! > > > the upcoming OpenTelemetry based TraceReporter will use

Re: [DISCUSS][FLINK-32993] Datagen connector handles length-constrained fields according to the schema definition by default

2023-11-21 Thread Lincoln Lee
Thanks Yubin and Jane for the discussion! +1 to fix this bug, although it's usually used as a test source, it's important to provide the correct behavior for users. for the invalid field length configured by users, I think it's better to raise an error instead of using default value silently.

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

2023-11-21 Thread Matthias Pohl
That's a valid point. I updated the FLIP accordingly: > Currently, the secrets (e.g. for S3 access tokens) are maintained by > certain PMC members with access to the corresponding configuration in the > Azure CI project. This responsibility will be moved to Apache Infra. They > are in charge of

[RESULT] [VOTE] Apache Flink Kubernetes Operator Release 1.7.0, release candidate #1

2023-11-21 Thread Gyula Fóra
I'm happy to announce that we have unanimously approved this release. There are 5 approving votes, 3 of which are binding: * Marton Balassi (binding) * Gyula Fora (binding) * Maximilian Mixhels (binding) * Rui Fan (non-binding) * Mate Czagany (non-binding) There are no disapproving votes.

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.7.0, release candidate #1

2023-11-21 Thread Gyula Fóra
Closing this vote thread, results will be announced in a separate email. Gyula On Mon, Nov 20, 2023 at 7:22 PM Maximilian Michels wrote: > +1 (binding) > > 1. Downloaded the archives, checksums, and signatures > 2. Verified the signatures and checksums > 3. Extract and inspect the source code

[jira] [Created] (FLINK-33610) CoGroupTaskTest.testCancelCoGroupTaskWhileCoGrouping failed with an AssertionError

2023-11-21 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33610: - Summary: CoGroupTaskTest.testCancelCoGroupTaskWhileCoGrouping failed with an AssertionError Key: FLINK-33610 URL: https://issues.apache.org/jira/browse/FLINK-33610

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

2023-11-21 Thread Martijn Visser
Hi Matthias, Thanks for the write-up and for the efforts on this. I really hope that we can move away from Azure towards GHA for a better integration as well (directly seeing if a PR can be merged due to CI passing for example). The one thing I'm missing in the FLIP is how we would setup the

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Jim Hughes
+1 (non-binding) Thanks, Dawid! Jim On Tue, Nov 21, 2023 at 7:20 AM Gyula Fóra wrote: > +1 (binding) > > Gyula > > On Tue, 21 Nov 2023 at 13:11, xiangyu feng wrote: > > > +1 (non-binding) > > > > Thanks for driving this. > > > > Best, > > Xiangyu Feng > > > > > > Ferenc Csaky 于2023年11月21日周二

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Timo Walther
+1 (binding) Thanks for working on this FLIP. It will be a nice continuation of the previous work. Cheers, Timo On 21.11.23 13:19, Gyula Fóra wrote: +1 (binding) Gyula On Tue, 21 Nov 2023 at 13:11, xiangyu feng wrote: +1 (non-binding) Thanks for driving this. Best, Xiangyu Feng

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

2023-11-21 Thread Matthias Pohl
I realized that I mixed up FLIP IDs. FLIP-395 is already reserved [1]. I switched to FLIP-396 [2] for the sake of consistency. 8) [1] https://lists.apache.org/thread/wjd3nbvg6nt93lb0sd52f0lzls6559tv [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-396%3A+Migration+to+GitHub+Actions On

[DISCUSS] FLIP-395: Migration to GitHub Actions

2023-11-21 Thread Matthias Pohl
Hi everyone, The Flink community discussed migrating from Azure CI to GitHub Actions quite some time ago [1]. The efforts around that stalled due to limitations around self-hosted runner support from Apache Infra’s side. There were some recent developments on that topic. Apache Infra is

[jira] [Created] (FLINK-33609) Take into account the resource limit specified in the pod template.

2023-11-21 Thread Surendra Singh Lilhore (Jira)
Surendra Singh Lilhore created FLINK-33609: -- Summary: Take into account the resource limit specified in the pod template. Key: FLINK-33609 URL: https://issues.apache.org/jira/browse/FLINK-33609

Re: [DISCUSS][FLINK-32993] Datagen connector handles length-constrained fields according to the schema definition by default

2023-11-21 Thread Jane Chan
Hi Yubin, Thanks for driving this discussion. Perhaps a specific example can better illustrate the current issue. Considering the following DDL, f0 will always be generated with a default char length of 100, regardless of char(5), bcause the connector option 'fields.f0.length' is not specified

Re: [VOTE] Release 1.16.3, release candidate #1

2023-11-21 Thread Matthias Pohl
+1 (binding) * Downloaded artifacts * Built Flink from sources * Verified SHA512 checksums & GPG signatures * Compared checkout with provided sources * Verified pom file versions * Went over NOTICE/pom file changes without finding anything suspicious * Deployed standalone session cluster and ran

Re: [VOTE] Release 1.17.2, release candidate #1

2023-11-21 Thread Matthias Pohl
+1 (binding) * Downloaded artifacts * Built Flink from sources * Verified SHA512 checksums & GPG signatures * Compared checkout with provided sources * Verified pom file versions * Went over NOTICE/pom file changes without finding anything suspicious * Deployed standalone session cluster and ran

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Gyula Fóra
+1 (binding) Gyula On Tue, 21 Nov 2023 at 13:11, xiangyu feng wrote: > +1 (non-binding) > > Thanks for driving this. > > Best, > Xiangyu Feng > > > Ferenc Csaky 于2023年11月21日周二 20:07写道: > > > +1 (non-binding) > > > > Lookgin forward to this! > > > > Best, > > Ferenc > > > > > > > > > > On

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread xiangyu feng
+1 (non-binding) Thanks for driving this. Best, Xiangyu Feng Ferenc Csaky 于2023年11月21日周二 20:07写道: > +1 (non-binding) > > Lookgin forward to this! > > Best, > Ferenc > > > > > On Tuesday, November 21st, 2023 at 12:21, Martijn Visser < > martijnvis...@apache.org> wrote: > > > > > > > > +1

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Ferenc Csaky
+1 (non-binding) Lookgin forward to this! Best, Ferenc On Tuesday, November 21st, 2023 at 12:21, Martijn Visser wrote: > > > +1 (binding) > > Thanks for driving this. > > Best regards, > > Martijn > > On Tue, Nov 21, 2023 at 12:18 PM Benchao Li libenc...@apache.org wrote: > > > +1

[jira] [Created] (FLINK-33608) UpsertTestDynamicTableSink's keySerializationSchama should extract and serialize the primary key fields from RowData

2023-11-21 Thread Jiabao Sun (Jira)
Jiabao Sun created FLINK-33608: -- Summary: UpsertTestDynamicTableSink's keySerializationSchama should extract and serialize the primary key fields from RowData Key: FLINK-33608 URL:

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

2023-11-21 Thread Becket Qin
Hi Hongshun, Do we need to expose the constructor of SplitFetcher to the users? Ideally, users should always get a new fetcher instance by calling SplitFetcherManager.createSplitFetcher(). Or, they can get an existing SplitFetcher by looking up in the SplitFetcherManager.fetchers map. I think

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Martijn Visser
+1 (binding) Thanks for driving this. Best regards, Martijn On Tue, Nov 21, 2023 at 12:18 PM Benchao Li wrote: > > +1 (binding) > > Dawid Wysakowicz 于2023年11月21日周二 18:56写道: > > > > Hi everyone, > > > > Thank you to everyone for the feedback on FLIP-393: Make QueryOperations > > SQL

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Benchao Li
+1 (binding) Dawid Wysakowicz 于2023年11月21日周二 18:56写道: > > Hi everyone, > > Thank you to everyone for the feedback on FLIP-393: Make QueryOperations > SQL serializable[1] > which has been discussed in this thread [2]. > > I would like to start a vote for it. The vote will be open for at least 72

[jira] [Created] (FLINK-33607) Add checksum verification for Maven wrapper as well

2023-11-21 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33607: - Summary: Add checksum verification for Maven wrapper as well Key: FLINK-33607 URL: https://issues.apache.org/jira/browse/FLINK-33607 Project: Flink Issue

[jira] [Created] (FLINK-33606) Support streaming mode in Flink JDBC driver

2023-11-21 Thread Sangram Reddy (Jira)
Sangram Reddy created FLINK-33606: - Summary: Support streaming mode in Flink JDBC driver Key: FLINK-33606 URL: https://issues.apache.org/jira/browse/FLINK-33606 Project: Flink Issue Type:

[VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Dawid Wysakowicz
Hi everyone, Thank you to everyone for the feedback on FLIP-393: Make QueryOperations SQL serializable[1] which has been discussed in this thread [2]. I would like to start a vote for it. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1]

[jira] [Created] (FLINK-33605) flink1.17 fd leak

2023-11-21 Thread feng wang (Jira)
feng wang created FLINK-33605: - Summary: flink1.17 fd leak Key: FLINK-33605 URL: https://issues.apache.org/jira/browse/FLINK-33605 Project: Flink Issue Type: Bug Reporter: feng wang

[jira] [Created] (FLINK-33604) flink1.17 fd leak

2023-11-21 Thread feng wang (Jira)
feng wang created FLINK-33604: - Summary: flink1.17 fd leak Key: FLINK-33604 URL: https://issues.apache.org/jira/browse/FLINK-33604 Project: Flink Issue Type: Bug Reporter: feng wang