Re: [Discussion] externalize Hive connector

2023-01-10 Thread yuxia
Hi, Chen. Cool! Of course not. Much appreciated that you can help continue this work. If you have any question about it, please let me know. Best regards, Yuxia - 原始邮件 - 发件人: "Chen Qin" 收件人: "dev" 发送时间: 星期三, 2023年 1 月 11日 上午 11:24:48 主题: Re: [Discussion] externalize Hive connector

[jira] [Created] (FLINK-30628) Kerberos in HiveCatalog is not work

2023-01-10 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-30628: Summary: Kerberos in HiveCatalog is not work Key: FLINK-30628 URL: https://issues.apache.org/jira/browse/FLINK-30628 Project: Flink Issue Type: Bug

Re:Re: Re: [DISCUSS] Allow source readers extending SourceReaderBase to override numRecordsIn report logic

2023-01-10 Thread Wencong Liu
Hi Dong and Martijin, I'll create a FLIP for this public API change and put related links in this discussion. Best regards, Wencong At 2023-01-10 09:34:05, "Dong Lin" wrote: >Hi Martijn, > >No worries. Thanks for the discussion! > >Suppose we agree to deprecate the existing

Re: [Discussion] externalize Hive connector

2023-01-10 Thread Chen Qin
Yuxia, thanks for details context, I see with your point on maintainability. If your don’t mind, I would like to help continue the work you mentioned. Chen On Tue, Jan 10, 2023 at 03:47 yuxia wrote: > Hi, Chen. > Appreciated your efforts. > I would like to shared my thonghts about it. > > I

[SUMMARY] Flink 1.17 Release Sync 1/10/2023

2023-01-10 Thread Qingsheng Ren
Hi devs and users, I'd like to share some highlights from the release sync on 1/10/2023. - CI stabilities: owners of blocker issues should have been pinged offline. - Priorities of the test instabilities: test instabilities are prioritized as Critical and become blocker as soon as we notice

[jira] [Created] (FLINK-30627) Refactor FileSystemTableSink to use FileSink instead of StreamingFileSink

2023-01-10 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-30627: -- Summary: Refactor FileSystemTableSink to use FileSink instead of StreamingFileSink Key: FLINK-30627 URL: https://issues.apache.org/jira/browse/FLINK-30627

[jira] [Created] (FLINK-30626) RabbitMQ connector has dependency convergence error for org.junit.jupiter:junit-jupiter

2023-01-10 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-30626: -- Summary: RabbitMQ connector has dependency convergence error for org.junit.jupiter:junit-jupiter Key: FLINK-30626 URL: https://issues.apache.org/jira/browse/FLINK-30626

Re: [ANNOUNCE] Performance Daily Monitoring Moved from Ververica to Apache Flink Slack Channel

2023-01-10 Thread Martijn Visser
Hi all, Related to Matthias' email, I've checked the notifications in the Slack channel and noticed three major benchmark regressions. In the end, I've decided to create Jira tickets for it [1] [2] [3] but I do agree that this work needs to be formalized as soon as possible to avoid regressions.

[jira] [Created] (FLINK-30625) Performance regression in tumblingWindow on 06.01.2023

2023-01-10 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-30625: -- Summary: Performance regression in tumblingWindow on 06.01.2023 Key: FLINK-30625 URL: https://issues.apache.org/jira/browse/FLINK-30625 Project: Flink

[jira] [Created] (FLINK-30624) Performance regression in stateBackends.FS / FS_ASYNC / MEMORY on 05.01.2023

2023-01-10 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-30624: -- Summary: Performance regression in stateBackends.FS / FS_ASYNC / MEMORY on 05.01.2023 Key: FLINK-30624 URL: https://issues.apache.org/jira/browse/FLINK-30624

[jira] [Created] (FLINK-30623) Performance regression in checkpointSingleInput.UNALIGNED on 04.01.2023

2023-01-10 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-30623: -- Summary: Performance regression in checkpointSingleInput.UNALIGNED on 04.01.2023 Key: FLINK-30623 URL: https://issues.apache.org/jira/browse/FLINK-30623 Project:

Re: [VOTE] FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs

2023-01-10 Thread Xintong Song
+1 (binding) weijie guo 于 2023年1月10日周二 18:38写道: > +1 (non-binding) > > Best regards, > > Weijie > > > yuxia 于2023年1月10日周二 18:06写道: > > > +1 (non-binding) > > > > Best regards, > > Yuxia > > > > - 原始邮件 - > > 发件人: "Zhu Zhu" > > 收件人: "dev" > > 发送时间: 星期二, 2023年 1 月 10日 下午 5:49:58 > > 主题:

[ANNOUNCE] Apache Flink Kubernetes Operator 1.3.1 released

2023-01-10 Thread Gyula Fóra
The Apache Flink community is very happy to announce the release of Apache Flink Kubernetes Operator 1.3.1. The Flink Kubernetes Operator allows users to manage their Apache Flink applications and their lifecycle through native k8s tooling like kubectl. Please check out the release blog post for

Re: [DISCUSS] Release Flink 1.16.1

2023-01-10 Thread Jing Ge
Hi Martijn, +1 and thanks for bringing this up. Since all blocking issues have been resolved afaik, when will the release process be kicked off? Do you have the estimated release date? Best regards, Jing On Thu, Jan 5, 2023 at 3:36 AM Dong Lin wrote: > Hi Martijn, > > Thank you for bringing

Re: [ANNOUNCE] Performance Daily Monitoring Moved from Ververica to Apache Flink Slack Channel

2023-01-10 Thread Matthias Pohl
Hi Yanfei, any updates on the performance tests? ...or more specifically, any updates on the script for alerting on performance regressions? Does it make sense to formalize/document the process? Currently, the release management doesn't do anything in terms of performance test monitoring.

[jira] [Created] (FLINK-30622) Support auto consume schema in Pulsar source

2023-01-10 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-30622: --- Summary: Support auto consume schema in Pulsar source Key: FLINK-30622 URL: https://issues.apache.org/jira/browse/FLINK-30622 Project: Flink Issue Type: New

Re: [VOTE] FLIP-281: Sink Supports Speculative Execution For Batch Job

2023-01-10 Thread yuxia
+1 (non-binding). Best regards, Yuxia - 原始邮件 - 发件人: "Zhu Zhu" 收件人: "dev" 发送时间: 星期二, 2023年 1 月 10日 下午 5:50:39 主题: Re: [VOTE] FLIP-281: Sink Supports Speculative Execution For Batch Job +1 (binding) Thanks, Zhu Biao Liu 于2023年1月5日周四 10:37写道: > > Hi Martijn, > > Sure, thanks for the

Re: [Discussion] externalize Hive connector

2023-01-10 Thread yuxia
HI, Peter. Thanks for your work. Flink devs must benefit from it. +1 for make it a separate part so that other flink connectors can also make use of it witthout including hive connector. But since we try to externalize the external connectors from Flink repo and it's related to HMS, I think

Re: [Discussion] externalize Hive connector

2023-01-10 Thread yuxia
Hi, Chen. Appreciated your efforts. I would like to shared my thonghts about it. I do think it's not a good timing to externalize Hive connector right now. As you know, the Hive connector relies some internal methods in flink-table-planner. The Flink devs can modify them with more freedom as

[jira] [Created] (FLINK-30621) Violation of main thread constraint detected

2023-01-10 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-30621: - Summary: Violation of main thread constraint detected Key: FLINK-30621 URL: https://issues.apache.org/jira/browse/FLINK-30621 Project: Flink Issue Type:

[jira] [Created] (FLINK-30620) Table Store Hive catalog supports specifying custom Hive metastore client

2023-01-10 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-30620: --- Summary: Table Store Hive catalog supports specifying custom Hive metastore client Key: FLINK-30620 URL: https://issues.apache.org/jira/browse/FLINK-30620 Project:

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

2023-01-10 Thread Yun Tang
Congratulations, Lincoln! Best Yun Tang From: Jing Ge Sent: Tuesday, January 10, 2023 18:45 To: dev@flink.apache.org Subject: Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee congrats! Best regards Jing On Tue, Jan 10, 2023 at 9:39 AM Ryan Skraba

[jira] [Created] (FLINK-30619) AdaptiveSchedulerTest.testStatusMetrics is not stable

2023-01-10 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-30619: - Summary: AdaptiveSchedulerTest.testStatusMetrics is not stable Key: FLINK-30619 URL: https://issues.apache.org/jira/browse/FLINK-30619 Project: Flink

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

2023-01-10 Thread Jing Ge
congrats! Best regards Jing On Tue, Jan 10, 2023 at 9:39 AM Ryan Skraba wrote: > Congratulations Lincoln! > > All my best, Ryan > > On Tue, Jan 10, 2023 at 9:37 AM yh z wrote: > > > Congratulations, Lincoln! > > > > Best regards, > > Yunhong Zheng > > > > Biao Liu 于2023年1月10日周二 15:02写道: > >

[jira] [Created] (FLINK-30618) flink-connector-pulsar not retrievable from Apache's Snapshot Maven repository

2023-01-10 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-30618: - Summary: flink-connector-pulsar not retrievable from Apache's Snapshot Maven repository Key: FLINK-30618 URL: https://issues.apache.org/jira/browse/FLINK-30618

Re: [VOTE] FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs

2023-01-10 Thread weijie guo
+1 (non-binding) Best regards, Weijie yuxia 于2023年1月10日周二 18:06写道: > +1 (non-binding) > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Zhu Zhu" > 收件人: "dev" > 发送时间: 星期二, 2023年 1 月 10日 下午 5:49:58 > 主题: Re: [VOTE] FLIP-283: Use adaptive batch scheduler as default scheduler > for batch

[jira] [Created] (FLINK-30617) Wrong data type for null literal in Hive dialect

2023-01-10 Thread luoyuxia (Jira)
luoyuxia created FLINK-30617: Summary: Wrong data type for null literal in Hive dialect Key: FLINK-30617 URL: https://issues.apache.org/jira/browse/FLINK-30617 Project: Flink Issue Type:

Re: [Discussion] externalize Hive connector

2023-01-10 Thread Péter Váry
Hi Team, Somewhat, but not strictly related: - We would like to use delegation tokens to connect from the IcebergFilesCommitter tasks to kerberized Hive Metastore servers when committing changes in the Iceberg connector [1]. Gabor Somogyi is working on generalizing token support [2]. I am

Re: [VOTE] FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs

2023-01-10 Thread yuxia
+1 (non-binding) Best regards, Yuxia - 原始邮件 - 发件人: "Zhu Zhu" 收件人: "dev" 发送时间: 星期二, 2023年 1 月 10日 下午 5:49:58 主题: Re: [VOTE] FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs +1 (binding) Thanks, Zhu Lijie Wang 于2023年1月10日周二 17:48写道: > > +1 (binding) > >

Re: [VOTE] FLIP-280: Introduce EXPLAIN PLAN_ADVICE to provide SQL advice

2023-01-10 Thread Jark Wu
+1 (binding) Thanks for the detailed design! Best, Jark > 2023年1月10日 17:38,Lincoln Lee 写道: > > +1 (binding) > > Best, > Lincoln Lee > > > godfrey he 于2023年1月10日周二 15:12写道: > >> +1 (binding) >> >> Best, >> Godfrey >> >> Jingsong Li 于2023年1月10日周二 09:56写道: >>> >>> +1 (binding) >>> >>>

Re: [DISCUSS] FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs

2023-01-10 Thread Junrui Lee
Hi, all. I have started voting[1] for this FLIP. If you have any questions, you can reply in this discussion thread. [1]: https://lists.apache.org/thread/gdymm7pr2slzy9gqkfo97vn73496w0cj Best regards, Junrui Xintong Song 于2023年1月4日周三 18:16写道: > Thanks for the proposal. > > Another potential

Re: [VOTE] FLIP-281: Sink Supports Speculative Execution For Batch Job

2023-01-10 Thread Zhu Zhu
+1 (binding) Thanks, Zhu Biao Liu 于2023年1月5日周四 10:37写道: > > Hi Martijn, > > Sure, thanks for the reminder about the holiday period. > Looking forward to your feedback! > > Thanks, > Biao /'bɪ.aʊ/ > > > > On Thu, 5 Jan 2023 at 03:07, Martijn Visser > wrote: > > > Hi Biao, > > > > To be honest,

Re: [VOTE] FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs

2023-01-10 Thread Zhu Zhu
+1 (binding) Thanks, Zhu Lijie Wang 于2023年1月10日周二 17:48写道: > > +1 (binding) > > Best, > Lijie > > Junrui Lee 于2023年1月10日周二 11:31写道: > > > Hi all, > > > > Thanks for all the feedback so far. > > Based on the discussion[1], we have come to a consensus, > > so I would like to start a vote on

Re: [VOTE] FLIP-274: Introduce metric group for OperatorCoordinator

2023-01-10 Thread Zhu Zhu
+1 (binding) Thanks, Zhu Jark Wu 于2023年1月10日周二 17:43写道: > > +1 (binding) > > Best, > Jark > > > 2023年1月10日 12:02,Qingsheng Ren 写道: > > > > Thanks for the FLIP! > > > > +1 (binding) > > > > Best, > > Qingsheng > > > > On Wed, Jan 4, 2023 at 10:08 AM Hang Ruan wrote: > > > >> Hi all, > >> > >>

Re: [VOTE] FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs

2023-01-10 Thread Lijie Wang
+1 (binding) Best, Lijie Junrui Lee 于2023年1月10日周二 11:31写道: > Hi all, > > Thanks for all the feedback so far. > Based on the discussion[1], we have come to a consensus, > so I would like to start a vote on FLIP-283: Use adaptive > batch scheduler as default scheduler for batch jobs[2]. > > The

Re: [VOTE] FLIP-274: Introduce metric group for OperatorCoordinator

2023-01-10 Thread Jark Wu
+1 (binding) Best, Jark > 2023年1月10日 12:02,Qingsheng Ren 写道: > > Thanks for the FLIP! > > +1 (binding) > > Best, > Qingsheng > > On Wed, Jan 4, 2023 at 10:08 AM Hang Ruan wrote: > >> Hi all, >> >> Thanks for all the feedback so far. >> Based on the discussion[1], we have come to a

Re: [VOTE] FLIP-282: Introduce Delete & Update API

2023-01-10 Thread Jark Wu
+1 (binding) Best, Jark > 2023年1月10日 17:39,Lincoln Lee 写道: > > +1 (binding) > > Best, > Lincoln Lee > > > Jingsong Li 于2023年1月10日周二 09:56写道: > >> +1 binding >> >> On Mon, Jan 9, 2023 at 6:14 PM Samrat Deb wrote: >>> >>> +1 (non binding ) >>> >>> thank you for driving >>> >>> >>>

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

2023-01-10 Thread Jark Wu
+1 for the option and the default value. Best, Jark > 2023年1月10日 16:24,yh z 写道: > > Hi all, > > Thanks for yours reply. > > After receiving your comments and making targeted modifications. The > conclusion is that the option > "table.optimizer.bushy-join-reorder-threshold" can be added.

Re: [VOTE] FLIP-282: Introduce Delete & Update API

2023-01-10 Thread Lincoln Lee
+1 (binding) Best, Lincoln Lee Jingsong Li 于2023年1月10日周二 09:56写道: > +1 binding > > On Mon, Jan 9, 2023 at 6:14 PM Samrat Deb wrote: > > > > +1 (non binding ) > > > > thank you for driving > > > > > > > > On Mon, 9 Jan 2023 at 3:36 PM, yuxia > wrote: > > > > > Hi, all. > > > > > > I'd like

Re: [VOTE] FLIP-280: Introduce EXPLAIN PLAN_ADVICE to provide SQL advice

2023-01-10 Thread Lincoln Lee
+1 (binding) Best, Lincoln Lee godfrey he 于2023年1月10日周二 15:12写道: > +1 (binding) > > Best, > Godfrey > > Jingsong Li 于2023年1月10日周二 09:56写道: > > > > +1 (binding) > > > > On Mon, Jan 9, 2023 at 6:19 PM Jane Chan wrote: > > > > > > Hi all, > > > > > > Thanks for all the feedback so far. > > >

[VOTE] Apache Flink Table Store 0.3.0, release candidate #1

2023-01-10 Thread Jingsong Li
Hi everyone, Please review and vote on the release candidate #1 for the version 0.3.0 of Apache Flink Table Store, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of the

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

2023-01-10 Thread Ryan Skraba
Congratulations Lincoln! All my best, Ryan On Tue, Jan 10, 2023 at 9:37 AM yh z wrote: > Congratulations, Lincoln! > > Best regards, > Yunhong Zheng > > Biao Liu 于2023年1月10日周二 15:02写道: > > > Congratulations, Lincoln! > > > > Thanks, > > Biao /'bɪ.aʊ/ > > > > > > > > On Tue, 10 Jan 2023 at

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

2023-01-10 Thread yh z
Congratulations, Lincoln! Best regards, Yunhong Zheng Biao Liu 于2023年1月10日周二 15:02写道: > Congratulations, Lincoln! > > Thanks, > Biao /'bɪ.aʊ/ > > > > On Tue, 10 Jan 2023 at 14:59, Hang Ruan wrote: > > > Congratulations, Lincoln! > > > > Best, > > Hang > > > > Biao Geng 于2023年1月10日周二 14:57写道:

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

2023-01-10 Thread yh z
Hi all, Thanks for yours reply. After receiving your comments and making targeted modifications. The conclusion is that the option "table.optimizer.bushy-join-reorder-threshold" can be added. Relevant PR [1] has been submitted. Sincerely welcome to review it. Thank you. This discussion will be