Re: [Discuss] FLIP-355: Add parent dir of files to classpath using yarn.provided.lib.dirs

2023-08-23 Thread Yang Wang
+1 for this FLIP Maybe a FLIP is an overkill for this enhancement. Best, Yang Venkatakrishnan Sowrirajan 于2023年8月23日周三 01:44写道: > Thanks for the FLIP, Archit. > > This is definitely quite a useful addition to the *yarn.provided.lib.dirs* > . +1. > > IMO, except for the fact that

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-23 Thread Matt Wang
Sorry for the late reply, I still have a small question here: Does the interface function of handlerRecommendedParallelism in AutoScalerEventHandler conflict with handlerScalingFailure/handlerScalingReport (one of the handles the event of scale failure, and the other handles the event of scale

[jira] [Created] (FLINK-32949) Allow specifying the ServerSocket port for the collect function when accessing the TaskManager from the client.

2023-08-23 Thread JiaJian He (Jira)
JiaJian He created FLINK-32949: -- Summary: Allow specifying the ServerSocket port for the collect function when accessing the TaskManager from the client. Key: FLINK-32949 URL:

Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-23 Thread Venkatakrishnan Sowrirajan
Becket and Jark, Deprecate all the other > methods except tryApplyFilters() and tryApplyProjections(). For *SupportsProjectionPushDown*, we still need a *supportsNestedProjections* API on the table source as some of the table sources might not be able to handle nested fields and therefore the

Re: Flink 1.17.2 planned?

2023-08-23 Thread Jing Ge
Hi Christian, Thanks for your understanding. We will take a look at 1.17.2, once the 1.18 release is done. In the meantime, there might be someone in the community who volunteers to be the 1.17.2 release manager. You will see related email threads on the Dev. Stay tuned please :-) Best regards,

[jira] [Created] (FLINK-32948) Minimize visibility of parameterized tests

2023-08-23 Thread Jiabao Sun (Jira)
Jiabao Sun created FLINK-32948: -- Summary: Minimize visibility of parameterized tests Key: FLINK-32948 URL: https://issues.apache.org/jira/browse/FLINK-32948 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32947) Autoscaler standalone mode supports the metric report

2023-08-23 Thread Rui Fan (Jira)
Rui Fan created FLINK-32947: --- Summary: Autoscaler standalone mode supports the metric report Key: FLINK-32947 URL: https://issues.apache.org/jira/browse/FLINK-32947 Project: Flink Issue Type: New

[DISCUSS][2.0] Deprecating Accumulator in favor of MetricsGroup

2023-08-23 Thread Matthias Pohl
Hi everyone, I was looking into serializing the ArchivedExecutionGraph for another FLIP and came across Accumulators [1] (don't mix that one up with the window accumulators of the Table/SQL API). Accumulators were introduced in Flink quite a while ago in Statosphere PR #340 [2]. I had a brief

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Rui Fan
Thanks for the quick response! Best, Rui On Wed, Aug 23, 2023 at 9:46 PM Jing Ge wrote: > yes please > > On Wed, Aug 23, 2023 at 3:35 PM Rui Fan <1996fan...@gmail.com> wrote: > > > Hi Jing, > > > > Thanks for the effort and update! > > > > It means the PR of flink-1.19 can be merged to master

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Jing Ge
yes please On Wed, Aug 23, 2023 at 3:35 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Jing, > > Thanks for the effort and update! > > It means the PR of flink-1.19 can be merged to master branch, right? > > Best, > Rui > > On Wed, Aug 23, 2023 at 9:29 PM Jing Ge > wrote: > > > Hi devs, The

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Rui Fan
Hi Jing, Thanks for the effort and update! It means the PR of flink-1.19 can be merged to master branch, right? Best, Rui On Wed, Aug 23, 2023 at 9:29 PM Jing Ge wrote: > Hi devs, The release-1.18 branch has been forked out from the master > branch, with commit ID

[ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Jing Ge
Hi devs, The release-1.18 branch has been forked out from the master branch, with commit ID cfa4a9c35563fd8a5973ec2f35251190e365be14. The version on the master branch has been upgraded to 1.19-SNAPSHOT. From now on, for PRs that should be presented in 1.18.0, please make sure: Merge the PR into

Re: [DISCUSS] FLIP-319: Integrating with Kafka’s proper support for 2PC participation (KIP-939).

2023-08-23 Thread Gyula Fóra
Hi Gordon! Thank you for preparing the detailed FLIP, I think this is a huge improvement that enables the exactly-once Kafka sink in many environments / use-cases where this was previously unfeasible due to the limitations highlighted in the FLIP. Big +1 Cheers, Gyula On Fri, Aug 18, 2023 at

[jira] [Created] (FLINK-32946) Start End of Life discussion thread for now outdated Flink minor version

2023-08-23 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32946: - Summary: Start End of Life discussion thread for now outdated Flink minor version Key: FLINK-32946 URL: https://issues.apache.org/jira/browse/FLINK-32946 Project:

[jira] [Created] (FLINK-32945) NullPointerException when executing TopSpeedWindowing example with checkpointing enabled

2023-08-23 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32945: - Summary: NullPointerException when executing TopSpeedWindowing example with checkpointing enabled Key: FLINK-32945 URL: https://issues.apache.org/jira/browse/FLINK-32945

Re: [DISCUSS] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

2023-08-23 Thread Becket Qin
Hi Weihua, Just want to clarify. "client.attached.after.submission" is going to be a pure client side configuration. On the cluster side, it is only "execution.shutdown-on-attached-exit" controlling whether the cluster will shutdown or not when an attached client is disconnected. In order to

[jira] [Created] (FLINK-32944) Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.

2023-08-23 Thread Tan Kim (Jira)
Tan Kim created FLINK-32944: --- Summary: Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue. Key: FLINK-32944 URL:

Re: [DISCUSS] Update Flink Roadmap

2023-08-23 Thread Jing Ge
Thanks Jark, +1 for the OLAP :-) Best regards, Jing On Sun, Aug 20, 2023 at 5:04 PM Jark Wu wrote: > Hi all, > > I have addressed another bunch of comments on the Google doc (mainly about > the OLAP roadmap). > And I have opened a pull request for the website: >

[jira] [Created] (FLINK-32943) sometime run batch tasks concurrently, the tasks still in the initialization status

2023-08-23 Thread zhu (Jira)
zhu created FLINK-32943: --- Summary: sometime run batch tasks concurrently, the tasks still in the initialization status Key: FLINK-32943 URL: https://issues.apache.org/jira/browse/FLINK-32943 Project: Flink

Re: Heartbeat of TaskManager with id container

2023-08-23 Thread xiangyu feng
Hi Nagireddy, I'm not sure how you monitoring kafka lag. AFAIK, you can check the metadata of the topic in your Kafka cluster to see the actual lag by following command. ./kafka-consumer-groups.sh --bootstrap-server 192.168.0.107:39092 --group --describe This tool is provided with Kafka

Fwd: [Discussion] Slack Channel

2023-08-23 Thread Jing Ge
Hi devs, Thanks Giannis for your suggestion. It seems that the last email wasn't sent to the dev ML. It is also an interesting topic for devs and user-zh. Best regards, Jing -- Forwarded message - From: Giannis Polyzos Date: Tue, Aug 22, 2023 at 11:11 AM Subject: [Discussion]

Re: [DISCUSS] Status of Statefun Project

2023-08-23 Thread Filip Karnicki
Hi Gordon Any chance we could get this reviewed and released to the central repo? We're currently forced to use a Flink version that has a nasty bug causing an operational nightmare Many thanks Fil On Sat, 19 Aug 2023 at 01:38, Galen Warren via user wrote: > Gotcha, makes sense as to the

[jira] [Created] (FLINK-32942) JUnit5 parameter tests

2023-08-23 Thread Jiabao Sun (Jira)
Jiabao Sun created FLINK-32942: -- Summary: JUnit5 parameter tests Key: FLINK-32942 URL: https://issues.apache.org/jira/browse/FLINK-32942 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

2023-08-23 Thread Weihua Hu
Hi, Jiangjie Thanks for the clarification. My key point is the meaning of the "submission" in "client.attached.after.submission". At first glance, I thought only job submissions were taken into account. After your clarification, this option also works for cluster submissions. It's fine for me.