[jira] [Created] (FLINK-35077) Add package license check for Flink CDC modules.

2024-04-10 Thread Xiqian YU (Jira)
Xiqian YU created FLINK-35077: - Summary: Add package license check for Flink CDC modules. Key: FLINK-35077 URL: https://issues.apache.org/jira/browse/FLINK-35077 Project: Flink Issue Type:

Re: [DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-10 Thread Xuannan Su
Hi Muhammet, Thanks for the suggestion. I updated the FLIP to include the options class. Best regards, Xuannan On Wed, Apr 10, 2024 at 1:56 PM Muhammet Orazov wrote: > > Hey Xuannan, > > Thanks for the FLIP and your efforts! > > Minor clarification from my side: > > > We will relocate these

Re: [DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-10 Thread Xuannan Su
Hi Zakelly, Thanks for the comments. I updated the FLIP accordingly. Best regards, Xuannan On Wed, Apr 10, 2024 at 11:12 AM Zakelly Lan wrote: > > Thanks Xuannan for driving this! +1 for cleaning these up. > > And minor comments: It seems the StateBackendOptions is already annotated > with

Re: [VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-04-10 Thread Muhammet Orazov
Hey Rui, +1 (non-binding). Thanks for driving it! Best, Muhammet On 2024-04-10 04:36, Rui Fan wrote: Hi devs, Thank you to everyone for the feedback on FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI[1] which has been discussed in this thread [2]. I would like to start

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-10 Thread Jamie Grier
Sorry for coming very late to this thread. I have not contributed much to Flink publicly for quite some time but I have been involved with Flink, daily, for years now and I'm keenly interested in where we take Flink SQL going forward. Thanks for the proposal!! I think it's definitely a step in

Re: [VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-04-10 Thread Ahmed Hamdy
+1 (non-binding) Best Regards Ahmed Hamdy On Wed, 10 Apr 2024 at 06:39, Zhu Zhu wrote: > +1 > > Thanks, > Zhu > > gongzhongqiang 于2024年4月10日周三 13:11写道: > > > +1 (non binding) > > > > > > Bests, > > > > Zhongqiang Gong > > > > Rui Fan <1996fan...@gmail.com> 于2024年4月10日周三 12:36写道: > > > > > Hi

Re: Support minibatch for TopNFunction

2024-04-10 Thread Roman Boyko
Finally I published the POC of the minibatch for TopN function [1]. It covers all the implementations of TopN functions because it buffers the records before putting them to the collector inside AbstractTopNFunction. For proving the performance optimization I used the nexmark q19 which was

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-10 Thread Dawid Wysakowicz
Hi all, I thought I'd cast my vote as well to give extra data: 1. Materialized Table 2. Materialized View (generally speaking I am not too concerned with using a View here, but since there are concerns around updating a view I put it second) I think what is suggested in this FLIP is

[jira] [Created] (FLINK-35076) Watermark alignment will cause data flow to experience serious shake

2024-04-10 Thread elon_X (Jira)
elon_X created FLINK-35076: -- Summary: Watermark alignment will cause data flow to experience serious shake Key: FLINK-35076 URL: https://issues.apache.org/jira/browse/FLINK-35076 Project: Flink

[jira] [Created] (FLINK-35075) Migrate TwoStageOptimizedAggregateRule

2024-04-10 Thread Jacky Lau (Jira)
Jacky Lau created FLINK-35075: - Summary: Migrate TwoStageOptimizedAggregateRule Key: FLINK-35075 URL: https://issues.apache.org/jira/browse/FLINK-35075 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35074) SavepointITCase.testStopWithSavepointWithDrainGlobalFailoverIfSavepointAborted

2024-04-10 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-35074: --- Summary: SavepointITCase.testStopWithSavepointWithDrainGlobalFailoverIfSavepointAborted Key: FLINK-35074 URL: https://issues.apache.org/jira/browse/FLINK-35074

RE: [DISCUSS] FLINK-34440 Support Debezium Protobuf Confluent Format

2024-04-10 Thread David Radley
Hi, I notice in the draft pr that there is a schema id in the format config. I was wondering why? In the confluent avro and existing debezium formats, there is no schema id in the config, but there is the ability to specify a complete schema. In the protobuf format there is no schema id. I

[jira] [Created] (FLINK-35073) Deadlock in LocalBufferPool when NetworkBufferPool.internalRecycleMemorySegments is called concurrently

2024-04-10 Thread Julien Tournay (Jira)
Julien Tournay created FLINK-35073: -- Summary: Deadlock in LocalBufferPool when NetworkBufferPool.internalRecycleMemorySegments is called concurrently Key: FLINK-35073 URL:

[jira] [Created] (FLINK-35072) Doris pipeline sink does not support applying AlterColumnTypeEvent

2024-04-10 Thread Xiqian YU (Jira)
Xiqian YU created FLINK-35072: - Summary: Doris pipeline sink does not support applying AlterColumnTypeEvent Key: FLINK-35072 URL: https://issues.apache.org/jira/browse/FLINK-35072 Project: Flink

[jira] [Created] (FLINK-35071) Remove dependency on flink-shaded from cdc source connector

2024-04-10 Thread Hongshun Wang (Jira)
Hongshun Wang created FLINK-35071: - Summary: Remove dependency on flink-shaded from cdc source connector Key: FLINK-35071 URL: https://issues.apache.org/jira/browse/FLINK-35071 Project: Flink

Re: [DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-10 Thread Rui Fan
Thanks Xuannan for driving this proposal! > Ensure all the ConfigOptions are properly annotated as PublicEvolving Could we add a specification directly? All XxxOptions classes are PublicEvolving by default. I'm afraid some new classes still miss PublicEvolving in the future. If we have a