Re: Sink V2 interface replacement for GlobalCommitter

Martjin, thanks a lot for chiming in! Here are my concerns with adding GlobalCommitter in the PostCommitTopology 1. when we use TwoPhaseCommittingSink. We would need to create a noop/dummy committer. Actual Iceberg/DeltaLake commits happen in the PostCommit stage. The PostCommit stage should be

Re: Sink V2 interface replacement for GlobalCommitter

Thanks Martijn, I'm actually trying to run our V1 Delta connector on Flink 1.15 using SinkV1Adapter with GlobalCommitterOperator. Having said that, I might have found a potential issue with GlobalCommitterOperator, checkpoitining and failover recovery [1]. For "normal" scenarios it does look good

Re: Migrating Sink v1 to v2

It seems that there might be an issue with state recovery or some kind concurrency issue for GlobalCommitterOperator created by SinkV1Adapter that uses StandardSinkTopologies.addGlobalCommitter method in scenarios with cluster failover. I've tried to locate the place where the issue is and from

[jira] [Created] (FLINK-29252) Support create table-store table with 'connector'='table-store'

MOBIN created FLINK-29252: - Summary: Support create table-store table with 'connector'='table-store' Key: FLINK-29252 URL: https://issues.apache.org/jira/browse/FLINK-29252 Project: Flink Issue

Re: Sink V2 interface replacement for GlobalCommitter

Hi all, A couple of bits from when work was being done on the new sink: V1 is completely simulated as V2 [1]. V2 is strictly more expressive. If there's desire to stick to the `GlobalCommitter` interface, have a look at the StandardSinkTopologies. Or you can just add your own more fitting

Re: Recommended way to Enable SSL Flink Kubernetes Operator

Hi Biao thanks for the quick reply. The helm chart uses a standard Deployment to mount the keystore onto the webhook container using volumes/volumeMounts for the operator but it’s not clear to me how to mount the keystore using the FlinkDeployment CRD[2] for a Flink application. [2]

Re: [ANNOUNCE] New Apache Flink Committer - Caizhi Weng

Congratulations, Caizhi! On Fri, Sep 9, 2022 at 3:50 PM Yun Tang wrote: > Congratulations Caizhi, well deserved! > > Best > Yun Tang > > From: Martijn Visser > Sent: Friday, September 9, 2022 17:27 > To: dev@flink.apache.org > Subject: Re: [ANNOUNCE] New

Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser

Congrats, Martijn! Indeed, well deserved :-) On Fri, Sep 9, 2022 at 5:10 PM tison wrote: > Congrats! Well deserved. > > Best, > tison. > > > Timo Walther 于2022年9月9日周五 23:08写道： > > > Hi everyone, > > > > I'm very happy to announce that Martijn Visser has joined the Flink PMC! > > > > Martijn

Re: Recommended way to Enable SSL Flink Kubernetes Operator

Hi Hao, You may take a look at codes for webhook in the helm charts of the Flink k8s project as it also utilizes cert-manager to create and mount the keystore for ssl. Best, Biao Geng 获取 Outlook for iOS 发件人: Hao t Chang 发送时间: Saturday,

Recommended way to Enable SSL Flink Kubernetes Operator

Hi Is there a recommended way (similar to this [1] ) to enable the SSL REST/Internal connectivity for FlinkDeployment created by the Flink Kubernetes Operator? First I added the required SSL config inside the flink-operator-config configmap. The required SSL configs look like:

Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser

Congrats! Well deserved. Best, tison. Timo Walther 于2022年9月9日周五 23:08写道： > Hi everyone, > > I'm very happy to announce that Martijn Visser has joined the Flink PMC! > > Martijn has helped the community in many different ways over the past > months. Externalizing the connectors from the Flink

[ANNOUNCE] New Apache Flink PMC Member - Martijn Visser

Hi everyone, I'm very happy to announce that Martijn Visser has joined the Flink PMC! Martijn has helped the community in many different ways over the past months. Externalizing the connectors from the Flink repo to their own repository, continously updating dependencies, and performing other

[jira] [Created] (FLINK-29251) Send CREATED status and Cancel event via FlinkResourceListener

Matyas Orhidi created FLINK-29251: - Summary: Send CREATED status and Cancel event via FlinkResourceListener Key: FLINK-29251 URL: https://issues.apache.org/jira/browse/FLINK-29251 Project: Flink

Re: [ANNOUNCE] New Apache Flink Committer - Caizhi Weng

Congratulations Caizhi, well deserved! Best Yun Tang From: Martijn Visser Sent: Friday, September 9, 2022 17:27 To: dev@flink.apache.org Subject: Re: [ANNOUNCE] New Apache Flink Committer - Caizhi Weng Congratulations Caizhi, much deserved! Best regards,

Re: PMC support for setting up DynamoDB connector in Jira

Tch, component didnt make it through. Should be there now, and I assigned it to FLINK-29246. On 09/09/2022 15:29, Chesnay Schepler wrote: Done. On 09/09/2022 15:08, Danny Cranmer wrote: Hello, We recently approved FLIP-252 [1] and created the flink-connector-dynamodb repository [2]. In

[jira] [Created] (FLINK-29250) Remove RpcService#getTerminationFuture

Chesnay Schepler created FLINK-29250: Summary: Remove RpcService#getTerminationFuture Key: FLINK-29250 URL: https://issues.apache.org/jira/browse/FLINK-29250 Project: Flink Issue Type:

[jira] [Created] (FLINK-29249) Remove RpcService#execute/scheduleRunnable

Chesnay Schepler created FLINK-29249: Summary: Remove RpcService#execute/scheduleRunnable Key: FLINK-29249 URL: https://issues.apache.org/jira/browse/FLINK-29249 Project: Flink Issue

[jira] [Created] (FLINK-29248) Remove RpcService#fenceRpcServer

Chesnay Schepler created FLINK-29248: Summary: Remove RpcService#fenceRpcServer Key: FLINK-29248 URL: https://issues.apache.org/jira/browse/FLINK-29248 Project: Flink Issue Type:

Re: PMC support for setting up DynamoDB connector in Jira

Done. On 09/09/2022 15:08, Danny Cranmer wrote: Hello, We recently approved FLIP-252 [1] and created the flink-connector-dynamodb repository [2]. In order for us to manage the connector issues in Jira can you please create for me: - Component: "Connectors / DynamoDB" - Fix Version:

[jira] [Created] (FLINK-29247) Streamline RpcService interface

Chesnay Schepler created FLINK-29247: Summary: Streamline RpcService interface Key: FLINK-29247 URL: https://issues.apache.org/jira/browse/FLINK-29247 Project: Flink Issue Type:

PMC support for setting up DynamoDB connector in Jira

Hello, We recently approved FLIP-252 [1] and created the flink-connector-dynamodb repository [2]. In order for us to manage the connector issues in Jira can you please create for me: - Component: "Connectors / DynamoDB" - Fix Version: "dynamodb-1.0.0" As per mailing list discussion [3] the

[jira] [Created] (FLINK-29246) Setup DynamoDB Connector Project Structure

Danny Cranmer created FLINK-29246: - Summary: Setup DynamoDB Connector Project Structure Key: FLINK-29246 URL: https://issues.apache.org/jira/browse/FLINK-29246 Project: Flink Issue Type:

[jira] [Created] (FLINK-29245) Remove RetryRule

Matthias Pohl created FLINK-29245: - Summary: Remove RetryRule Key: FLINK-29245 URL: https://issues.apache.org/jira/browse/FLINK-29245 Project: Flink Issue Type: Bug Components:

Re: [SUMMARY] Flink 1.16 release sync of 2022-09-06

Hi tison, Thanks for the information. I think the pr is a bug fix, so it is fine to merge into the release branch. We are now waiting for three blocker issues to be resolved. After that we will prepare rc0. Best, Xingbo tison 于2022年9月9日周五 17:07写道： > Hi Xingbo, > > I just notice that 1.16.0

Re: [DISCUSS] FLIP-254 Redis Streams connector

Hi Konstantin, I don't know, perhaps Mohammad knows. Best regards, Martijn Op wo 31 aug. 2022 om 14:31 schreef Konstantin Knauf : > Hi Martijn, Hi Mohammad, > > thanks for creating this FLIP. > > I am wondering, do you know how much Redis Streams are used compared to > other Redis data

Re: [ANNOUNCE] New Apache Flink Committer - Caizhi Weng

Congratulations Caizhi, much deserved! Best regards, Martijn Op vr 9 sep. 2022 om 11:00 schreef Lincoln Lee : > Congratulations Caizhi! > > Best, > Lincoln Lee > > > Kurt Yang 于2022年9月9日周五 16:57写道： > > > Congrats Chaizhi! > > > > Best, > > Kurt > > > > > > On Fri, Sep 9, 2022 at 2:44 PM yuxia

[jira] [Created] (FLINK-29244) Add metric lastMaterializationDuration to ChangelogMaterializationMetricGroup

Feifan Wang created FLINK-29244: --- Summary: Add metric lastMaterializationDuration to ChangelogMaterializationMetricGroup Key: FLINK-29244 URL: https://issues.apache.org/jira/browse/FLINK-29244

Re: [SUMMARY] Flink 1.16 release sync of 2022-09-06

Hi Xingbo, I just notice that 1.16.0 didn't release yet after I merge https://github.com/apache/flink/pull/20798 into the release branch. Sorry for making this change. This is a trivial change but if you'd like to revert it, please do it and comment in the original PR. Best, tison. Xingbo

Re: [ANNOUNCE] New Apache Flink Committer - Caizhi Weng

Congratulations Caizhi! Best, Lincoln Lee Kurt Yang 于2022年9月9日周五 16:57写道： > Congrats Chaizhi! > > Best, > Kurt > > > On Fri, Sep 9, 2022 at 2:44 PM yuxia wrote: > > > Congrats Chaizhi! > > > > Best regards, > > Yuxia > > > > - 原始邮件 - > > 发件人: "weijie guo" > > 收件人: "dev" > > 发送时间:

Re: [ANNOUNCE] New Apache Flink Committer - Caizhi Weng

Congrats Chaizhi! Best, Kurt On Fri, Sep 9, 2022 at 2:44 PM yuxia wrote: > Congrats Chaizhi! > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "weijie guo" > 收件人: "dev" > 发送时间: 星期五, 2022年 9 月 09日 下午 2:00:50 > 主题: Re: [ANNOUNCE] New Apache Flink Committer - Caizhi Weng > > Congrats

[jira] [Created] (FLINK-29243) Introduce a COW mode for table store

Jingsong Lee created FLINK-29243: Summary: Introduce a COW mode for table store Key: FLINK-29243 URL: https://issues.apache.org/jira/browse/FLINK-29243 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-29242) [flink-filesystems] flink-gs-fs-hadoop Read time out when close write channel

Jian Zheng created FLINK-29242: -- Summary: [flink-filesystems] flink-gs-fs-hadoop Read time out when close write channel Key: FLINK-29242 URL: https://issues.apache.org/jira/browse/FLINK-29242 Project:

[jira] [Created] (FLINK-29241) Can not overwrite from empty input

Jingsong Lee created FLINK-29241: Summary: Can not overwrite from empty input Key: FLINK-29241 URL: https://issues.apache.org/jira/browse/FLINK-29241 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-29240) Unify the ClassLoader in StreamExecutionEnvironment and TableEnvironment

dalongliu created FLINK-29240: - Summary: Unify the ClassLoader in StreamExecutionEnvironment and TableEnvironment Key: FLINK-29240 URL: https://issues.apache.org/jira/browse/FLINK-29240 Project: Flink

Re: [ANNOUNCE] New Apache Flink Committer - Caizhi Weng

Congrats Chaizhi! Best regards, Yuxia - 原始邮件 - 发件人: "weijie guo" 收件人: "dev" 发送时间: 星期五, 2022年 9 月 09日 下午 2:00:50 主题: Re: [ANNOUNCE] New Apache Flink Committer - Caizhi Weng Congrats Chaizhi! Best regards, Weijie Jing Ge 于2022年9月9日周五 13:38写道： > Congrats Chaizhi! > > Best regards,

[jira] [Created] (FLINK-29239) KafkaSerializerWrapper.config should be Map for full configurability.

Bill G created FLINK-29239: -- Summary: KafkaSerializerWrapper.config should be Map for full configurability. Key: FLINK-29239 URL: https://issues.apache.org/jira/browse/FLINK-29239 Project: Flink

Re: [ANNOUNCE] New Apache Flink Committer - Caizhi Weng

Congrats Chaizhi! Best regards, Weijie Jing Ge 于2022年9月9日周五 13:38写道： > Congrats Chaizhi! > > Best regards, > Jing > > On Fri, Sep 9, 2022 at 7:23 AM Benchao Li wrote: > > > Congratulations Caizhi! > > > > Jark Wu 于2022年9月9日周五 11:46写道： > > > > > Congrats ChaiZhi! > > > > > > Cheers, > > >