Re: [DISCUSS] FLIP-309: Enable operators to trigger checkpoints dynamically

Hi Piotr, Thanks again for proposing the isProcessingBacklog concept. After discussing with Becket Qin and thinking about this more, I agree it is a better idea to add a top-level concept to all source operators to address the target use-case. The main reason that changed my mind is that

[jira] [Created] (FLINK-32411) SourceCoordinator thread leaks when job recovers from checkpoint

Rui Fan created FLINK-32411: --- Summary: SourceCoordinator thread leaks when job recovers from checkpoint Key: FLINK-32411 URL: https://issues.apache.org/jira/browse/FLINK-32411 Project: Flink

Re: [DISCUSS] Deprecate SourceFunction APIs

I'd like to revive the efforts to deprecate the SourceFunction API. It would be great to get a review for this PR: https://github.com/apache/flink/pull/21774 It immediately unblocks marking the actual SourceFunction as deprecated. https://github.com/apache/flink/pull/20049 There is also this

Re: [DISCUSS] FLIP-313 Add support of User Defined AsyncTableFunction

Hi Aitozi, Thanks for your updates! By the design of hints, the hints after select clause belong to the query hints category, and this new hint is also a kind of join hints[1]. Join table function is one of the join type defined by flink sql joins[2], all existing join hints[1] omit the 'join'

[jira] [Created] (FLINK-32410) Allocate hash-based collections with sufficient capacity for expected size

Stefan Richter created FLINK-32410: -- Summary: Allocate hash-based collections with sufficient capacity for expected size Key: FLINK-32410 URL: https://issues.apache.org/jira/browse/FLINK-32410

[jira] [Created] (FLINK-32409) Remove MultipleComponentLeaderElectionDriverAdapter

Matthias Pohl created FLINK-32409: - Summary: Remove MultipleComponentLeaderElectionDriverAdapter Key: FLINK-32409 URL: https://issues.apache.org/jira/browse/FLINK-32409 Project: Flink Issue

[jira] [Created] (FLINK-32408) JobManager HA configuration update needed in Flink k8s Operator

dongwoo.kim created FLINK-32408: --- Summary: JobManager HA configuration update needed in Flink k8s Operator Key: FLINK-32408 URL: https://issues.apache.org/jira/browse/FLINK-32408 Project: Flink

[RESULT][VOTE] FLIP-287: Extend Sink#InitContext to expose TypeSerializer, ObjectReuse and JobID

Hi all, I am happy to announce that FLIP-287: Extend Sink#InitContext to expose TypeSerializer, ObjectReuse and JobID[1] has been accepted. There are 8 approving votes, 6 of which are binding: - Lijie Wang (binding) - Jing Ge (binding) - Tzu-Li (Gordon) Tai (binding) - Zhu Zhu (binding) - Yuepeng

[DISCUSS] Releasing Flink ML 2.3.0

Hi devs, Dong and I would like to start a discussion regarding the release of Flink ML 2.3.0. The release mainly provides the ability to run on multiple Flink versions. With 2.3.0, users can build and run Flink ML libraries with Flink 1.15, 1.16, and 1.17. Besides, some algorithm performence

Re: [DISCUSS] FLIP-307: Flink connector Redshift

Thanks Samrat for driving this FLIP. Since the community has already built a set of basic components for the connector, I only have three comments. 1 Reusing the capabilities of JDBC and Filesystem in the Redshift connector generally makes sense to me. However, since all of them are managed in

Re: [VOTE] FLIP-246: Dynamic Kafka Source (originally Multi Cluster Kafka Source)

+1 (binding) On Mon, Jun 19, 2023 at 8:09 AM Ryan van Huuksloot wrote: > +1 (non-binding) > > +1 for DynamicKafkaSource > > Ryan van Huuksloot > Sr. Production Engineer | Streaming Platform > [image: Shopify] > > > > On Mon,

[jira] [Created] (FLINK-32407) Notify catalog listener for table events

Fang Yong created FLINK-32407: - Summary: Notify catalog listener for table events Key: FLINK-32407 URL: https://issues.apache.org/jira/browse/FLINK-32407 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32406) Notify catalog listener for database events

Fang Yong created FLINK-32406: - Summary: Notify catalog listener for database events Key: FLINK-32406 URL: https://issues.apache.org/jira/browse/FLINK-32406 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32405) Initialize catalog listener for CatalogManager

Fang Yong created FLINK-32405: - Summary: Initialize catalog listener for CatalogManager Key: FLINK-32405 URL: https://issues.apache.org/jira/browse/FLINK-32405 Project: Flink Issue Type:

[jira] [Created] (FLINK-32404) Introduce catalog modification listener and factory interfaces

Fang Yong created FLINK-32404: - Summary: Introduce catalog modification listener and factory interfaces Key: FLINK-32404 URL: https://issues.apache.org/jira/browse/FLINK-32404 Project: Flink

[jira] [Created] (FLINK-32403) Add database related operations in catalog manager

Fang Yong created FLINK-32403: - Summary: Add database related operations in catalog manager Key: FLINK-32403 URL: https://issues.apache.org/jira/browse/FLINK-32403 Project: Flink Issue Type:

[jira] [Created] (FLINK-32402) FLIP-294: Support Customized Catalog Modification Listener

Fang Yong created FLINK-32402: - Summary: FLIP-294: Support Customized Catalog Modification Listener Key: FLINK-32402 URL: https://issues.apache.org/jira/browse/FLINK-32402 Project: Flink Issue

Re: [DISCUSS] FLIP-324: Introduce Runtime Filter for Flink Batch Jobs

Hi Ron, Thanks for sharing your thoughts! It makes sense. It would be helpful if these references of Hive, Polardb, etc. could be added into the FLIP. Best regards, Jing On Tue, Jun 20, 2023 at 5:41 PM liu ron wrote: > Hi, Jing > > The default value for this ratio is a reference to other

Re: [DISCUSS] FLIP-307: Flink connector Redshift

Hi Martijn, Thank you for sharing your thoughts on the matter. I understand that you don't have a strong opinion on whether to support exactly-once processing from the beginning or at a later stage. For initial implementation I will go ahead with at-least-once semantics. > The only

[jira] [Created] (FLINK-32401) KafkaSourceBuilder reset the 'auto.offset.reset'

KianChen created FLINK-32401: Summary: KafkaSourceBuilder reset the 'auto.offset.reset' Key: FLINK-32401 URL: https://issues.apache.org/jira/browse/FLINK-32401 Project: Flink Issue Type:

Re: [DISCUSS] Flexible Custom Commit Policy with Parameter Passing

Hi, Yuxia. Thank you for your suggestion! I agree with your point and have made the necessary modification. If you have any additional feedback or further suggestions, please feel free to let me know. : ) Best regards, Bangui Dunn yuxia 于2023年6月21日周三 10:56写道： > Correct what I said in the

[DISCUSS] Release 2.0 Work Items

Hi devs, As previously discussed in [1], we had been collecting work item proposals for the 2.0 release until June 15th, on the wiki page [2]. - As we have passed the due date, I'd like to kindly remind everyone *not to add / remove items directly on the wiki page*. If needed, please post