Re: [VOTE] FLIP-379: Dynamic source parallelism inference for batch jobs

+1 (binding) Best, Lijie Zhu Zhu 于2023年11月30日周四 13:13写道： > +1 > > Thanks, > Zhu > > Xia Sun 于2023年11月30日周四 11:41写道： > > > Hi everyone, > > > > I'd like to start a vote on FLIP-379: Dynamic source parallelism > inference > > for batch jobs[1] which has been discussed in this thread [2]. > > >

[DISCUSS][2.0] FLIP-382: Unify the Provision of Diverse Metadata for Context-like APIs

Hi devs, I would like to start a discussion on FLIP-382: Unify the Provision of Diverse Metadata for Context-like APIs [1]. In the Flink project, the context-like APIs are interfaces annotated by @Public and supply runtime metadata and functionalities to its modules and components.

Re: [ANNOUNCE] Experimental Java 21 support now available on master

Thanks Yun Tang One question to clarify: since the scala version was also bumped for java 17, shouldn't there be a similar task for java 17? On Thu, Nov 30, 2023 at 3:43 AM Yun Tang wrote: > Hi Sergey, > > You can leverage all tests extending SnapshotMigrationTestBase[1] to > verify the logic.

Re: [VOTE] Release flink-connector-aws, v4.2.0 release candidate #1

Hi danny , +1 (non binding) Release notes look good - Signatures and checksums verified - pom versions checked - Source build and tests verified Bests, Samrat On Wed, 29 Nov 2023 at 12:11 PM, Leonard Xu wrote: > Thanks Danny for driving this, sorry for late verification. > > +1 (binding) > >

Re: [VOTE] FLIP-379: Dynamic source parallelism inference for batch jobs

+1 Thanks, Zhu Xia Sun 于2023年11月30日周四 11:41写道： > Hi everyone, > > I'd like to start a vote on FLIP-379: Dynamic source parallelism inference > for batch jobs[1] which has been discussed in this thread [2]. > > The vote will be open for at least 72 hours unless there is an objection or > not

[VOTE] FLIP-379: Dynamic source parallelism inference for batch jobs

Hi everyone, I'd like to start a vote on FLIP-379: Dynamic source parallelism inference for batch jobs[1] which has been discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1]

[jira] [Created] (FLINK-33700) CustomSSLEngineProvider supports SHA-256

Bo Cui created FLINK-33700: -- Summary: CustomSSLEngineProvider supports SHA-256 Key: FLINK-33700 URL: https://issues.apache.org/jira/browse/FLINK-33700 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-33699) Verify the snapshot migration on Java21

Yun Tang created FLINK-33699: Summary: Verify the snapshot migration on Java21 Key: FLINK-33699 URL: https://issues.apache.org/jira/browse/FLINK-33699 Project: Flink Issue Type: Sub-task

Re: [ANNOUNCE] Experimental Java 21 support now available on master

Hi Sergey, You can leverage all tests extending SnapshotMigrationTestBase[1] to verify the logic. I believe all binary _metadata existing in the resources folder[2] were built by JDK8. I also create a ticket FLINK-33699[3] to track this. [1]

[ANNOUNCE] Apache Flink 1.16.3 released

The Apache Flink community is very happy to announce the release of Apache Flink 1.16.3, which is the third bugfix release for the Apache Flink 1.16 series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data

[jira] [Created] (FLINK-33698) Fix the backoff time calculation in ExponentialBackoffDelayRetryStrategy

xiangyu feng created FLINK-33698: Summary: Fix the backoff time calculation in ExponentialBackoffDelayRetryStrategy Key: FLINK-33698 URL: https://issues.apache.org/jira/browse/FLINK-33698 Project:

[jira] [Created] (FLINK-33696) FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

Piotr Nowojski created FLINK-33696: -- Summary: FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter Key: FLINK-33696 URL: https://issues.apache.org/jira/browse/FLINK-33696

[jira] [Created] (FLINK-33697) FLIP-386: Support adding custom metrics in Recovery Spans

Piotr Nowojski created FLINK-33697: -- Summary: FLIP-386: Support adding custom metrics in Recovery Spans Key: FLINK-33697 URL: https://issues.apache.org/jira/browse/FLINK-33697 Project: Flink

[jira] [Created] (FLINK-33695) FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

Piotr Nowojski created FLINK-33695: -- Summary: FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces Key: FLINK-33695 URL: https://issues.apache.org/jira/browse/FLINK-33695

Re: [VOTE] FLIP-386: Support adding custom metrics in Recovery Spans

Thank you all for voting, I will post the results soon! Best, Piotrek śr., 29 lis 2023 o 10:47 Stefan Richter napisał(a): > +1 (binding) > > Best, > Stefan > > > > On 23. Nov 2023, at 09:49, Roman Khachatryan wrote: > > > > +1 (binding) > > > > Regards, > > Roman > > > > > > On Wed, Nov 22,

[RESULT][VOTE] FLIP-386: Support adding custom metrics in Recovery Spans

I'm pleased to announce that FLIP-386 has been accepted. Binding votes +1: - Rui Fan - Roman Khachatryan - Stefan Richter There were no votes against. Best, Piotrek

[RESULT][VOTE] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

I'm pleased to announce that FLIP-385 has been accepted. Binding votes +1: - Rui Fan - Hangxiang Yu - Roman Khachatryan - Jing Ge - Stefan Richter There were no votes against. Best, Piotrek

Re: [VOTE] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

Thank you all for voting, I will post the results soon. Best, Piotrek śr., 29 lis 2023 o 10:47 Stefan Richter napisał(a): > +1 (binding) > > Best, > Stefan > > > > On 22. Nov 2023, at 12:34, Jing Ge wrote: > > > > +1(binding) > > Thanks! > > > > Best Regards, > > Jing > > > > On Wed, Nov 22,

[RESULT][VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

I'm pleased to announce that FLIP-384 has been accepted. Binding votes +1: - Jing Ge - Rui Fan - Hangxiang Yu - Roman Khachatryan - David Morávek - Stefan Richter Non-binding votes +1: - Zakelly Lan - David Radley There were no votes against. Best, Piotrek

Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

Thank you all for voting. I will post the results soon. Best, Piotrek śr., 29 lis 2023 o 11:10 David Radley napisał(a): > +1(non-binding) > > From: Stefan Richter > Date: Wednesday, 29 November 2023 at 09:44 > To: dev@flink.apache.org > Subject: [EXTERNAL] Re: [VOTE] FLIP-384: Introduce

Re: Python connector question

Hi Peter, Thanks for picking this up! Please find my thoughts inline. BR, G On Mon, Nov 27, 2023 at 4:05 PM Péter Váry wrote: > Hi Team, > > During some, mostly unrelated work we come to realize that during the > externalization of the connector's python modules and the related tests are >

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

With regards to Alex' concerns on hardware disparity: I did a bit more digging on that one. I added my findings in a hardware section to FLIP-396 [1]. It appears that the hardware is more or less the same between the different hosts. Apache INFRA's runners have more disk space (1TB in comparison

[jira] [Created] (FLINK-33694) GCS filesystem does not respect gs.storage.root.url config option

Patrick Lucas created FLINK-33694: - Summary: GCS filesystem does not respect gs.storage.root.url config option Key: FLINK-33694 URL: https://issues.apache.org/jira/browse/FLINK-33694 Project: Flink

Re: Sending a hi!

Welcome to the community. :) Some good starting points aside from reading the mailing lists [1] are the documentation about contributing [2] and setting up your IDE [3]. You can look for Jira issues labeled as starter [4]. Matthias [1]

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

Thanks for your feedback Alex. I responded to your comments below: This is mentioned in the "Limitations of GitHub Actions in the past" > section of the FLIP. Does this also apply to the Apache INFRA setup or can > we expect contributors' runs executed there too? Workflow runs on Flink forks

Re: [ANNOUNCE] Experimental Java 21 support now available on master

thanks for the response >I feel doubt about the conclusion that "don't try to load a savepoint from a Java 8/11/17 build due to bumping to scala-2.12.18", since the snapshotted state (operator/keyed state-backend), and most key/value serializer snapshots are generated by pure-java code. >The

Re: [DISCUSS] Resolve diamond inheritance of Sink.createWriter

Thanks, Martijn and Peter. In terms of the concrete issue: - I am following up with the author of FLIP-321 [1] (Becket) to update the docs [2] to reflect the right state. - I see two reasonable approaches in terms of proceeding with the specific changeset: 1. We allow the

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

Thanks for your comments, Xintong. See my answers below. > I think it would be helpful if we can at the end migrate the CI to an > ASF-managed Github Action, as long as it provides us a similar computation > capacity and stability. The current test runs in my Flink fork (using the

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

Thanks for driving this Mathhias! +1 for joining the INFRA trial. > Apache Infra did some experimenting on self-hosted runners in collaboration > with Apache Airflow (see ashb/runner with releases/pr-security-options branch) > where they only allow certain groups of users (e.g. committers) to

[jira] [Created] (FLINK-33693) The disable unaligned checkpoint isn't respected when the aligned checkpoint timeout is enabled

Rui Fan created FLINK-33693: --- Summary: The disable unaligned checkpoint isn't respected when the aligned checkpoint timeout is enabled Key: FLINK-33693 URL: https://issues.apache.org/jira/browse/FLINK-33693

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

Thanks for the efforts, Matthias. I think it would be helpful if we can at the end migrate the CI to an ASF-managed Github Action, as long as it provides us a similar computation capacity and stability. Given that the proposal is only to start a trial and investigate whether the migration is

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

Hi all, The user mail[1] has started for 13 days, and it collected one useful suggestion. > Given that the new default feels more complex than the current behavior, if we decide to do this I think it will be important to include the rationale you've shared in the documentation. I will add the

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

Ok, Thanks for the update and the explanations. Best, Yuxin Matthias Pohl 于2023年11月29日周三 15:43写道： > > > > According to the Flip, the new tests will support arm env. > > I believe that's good news for arm users. I have a minor > > question here. Will it be a blocker before migrating the new >

Re: [ANNOUNCE] Experimental Java 21 support now available on master

Thanks Sergey for the great work. I feel doubt about the conclusion that "don't try to load a savepoint from a Java 8/11/17 build due to bummping to scala-2.12.18", since the snapshotted state (operator/keyed state-backend), and most key/value serializer snapshots are generated by pure-java

Re: [VOTE] Release flink-shaded 18.0, release candidate #1

+1 (binding) * Downloaded all resources * Extracts sources and compilation on these sources * Diff of git tag checkout with downloaded sources * Verifies SHA512 checksums & GPG certification * Checks that all POMs have the right expected version * Generated diffs to compare pom file changes with

[DISCUSS] Release flink-connector-parent v1.01

Hi all, I would like to discuss making a v1.0.1 release of flink-connector-parent. Since last release, there were only 2 changes: - https://github.com/apache/flink-connector-shared-utils/pull/19 (spotless addition) - https://github.com/apache/flink-connector-shared-utils/pull/26 (surefire

RE: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

+1(non-binding) From: Stefan Richter Date: Wednesday, 29 November 2023 at 09:44 To: dev@flink.apache.org Subject: [EXTERNAL] Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces +1 (binding) Best, Stefan > On 22. Nov 2023, at 11:20, Roman

Re: [DISCUSSION] flink-connector-shared-utils release process

Thanks Leonard for the review. Best, Etienne Le 29/11/2023 à 08:25, Leonard Xu a écrit : Thanks Etienne for update the wiki, I just checked the change by comparing with history version, LGTM Best, Leonard 2023年11月28日 下午10:43，Etienne Chauchot 写道： Hi all, I just updated the

Re: [VOTE] FLIP-386: Support adding custom metrics in Recovery Spans

+1 (binding) Best, Stefan > On 23. Nov 2023, at 09:49, Roman Khachatryan wrote: > > +1 (binding) > > Regards, > Roman > > > On Wed, Nov 22, 2023 at 12:55 PM Rui Fan <1996fan...@gmail.com > > wrote: > >> +1(binding) >> >> Thanks for driving this proposal! >>

Re: [VOTE] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

+1 (binding) Best, Stefan > On 22. Nov 2023, at 12:34, Jing Ge wrote: > > +1(binding) > Thanks! > > Best Regards, > Jing > > On Wed, Nov 22, 2023 at 11:21 AM Roman Khachatryan > wrote: > >> +1 (binding) >> >> Regards, >> Roman >> >> On Wed, Nov 22, 2023, 7:30 AM

Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

+1 (binding) Best, Stefan > On 22. Nov 2023, at 11:20, Roman Khachatryan wrote: > > +1 (binding) > > Regards, > Roman > > On Wed, Nov 22, 2023, 7:08 AM Zakelly Lan > wrote: > >> +1(non-binding) >> >> Best, >> Zakelly >> >> On Wed, Nov 22, 2023 at 3:04 PM

[jira] [Created] (FLINK-33692) FlinkKafkaProducer could miss super.close

zhuming created FLINK-33692: --- Summary: FlinkKafkaProducer could miss super.close Key: FLINK-33692 URL: https://issues.apache.org/jira/browse/FLINK-33692 Project: Flink Issue Type: Bug

Re: [VOTE] Release flink-shaded 18.0, release candidate #1

Sorry, it's non-binding. On Wed, Nov 29, 2023 at 5:19 PM Rui Fan <1996fan...@gmail.com> wrote: > Thanks Matthias for the clarification! > > After I import the latest KEYS, it works fine. > > +1(binding) > > - Validated checksum hash > - Verified signature > - Verified that no binaries exist in

Re: [VOTE] Release flink-shaded 18.0, release candidate #1

Thanks Matthias for the clarification! After I import the latest KEYS, it works fine. +1(binding) - Validated checksum hash - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven and jdk8 - Verified licenses - Verified web PRs, and left a

Re: [VOTE] Release flink-shaded 18.0, release candidate #1

The key is the last key in the KEYS file. It's just having a different format with spaces being added (due to different gpg versions?): F752 9FAE 2481 1A5C 0DF3 CA74 1596 BBF0 7268 35D8 On Wed, Nov 29, 2023 at 9:41 AM Rui Fan <1996fan...@gmail.com> wrote: > Hey Sergey, > > Thank you for driving

[jira] [Created] (FLINK-33691) Support agg push down for 'count(*)/count(1)/count(column not null)'

Yunhong Zheng created FLINK-33691: - Summary: Support agg push down for 'count(*)/count(1)/count(column not null)' Key: FLINK-33691 URL: https://issues.apache.org/jira/browse/FLINK-33691 Project:

Re: [ANNOUNCE] Experimental Java 21 support now available on master

Thanks Sergey for the great work! Best, Rui On Wed, Nov 29, 2023 at 4:42 PM Leonard Xu wrote: > Cool ! > > Thanks Sergey for the great effort and all involved. > > > Best, > Leonard > > > 2023年11月29日 下午4:31，Swapnal Varma 写道： > > > > Congratulations Sergey, and everyone involved! > > > >

Re: [ANNOUNCE] Experimental Java 21 support now available on master

Cool ! Thanks Sergey for the great effort and all involved. Best, Leonard > 2023年11月29日 下午4:31，Swapnal Varma 写道： > > Congratulations Sergey, and everyone involved! > > Excited to work with and on this! > > Best, > Swapnal > > > On Wed, 29 Nov 2023, 13:58 Sergey Nuyanzin, wrote: > >>

Re: [VOTE] Release flink-shaded 18.0, release candidate #1

Hey Sergey, Thank you for driving this release. I try to check this signature, the whole key is F7529FAE24811A5C0DF3CA741596BBF0726835D8, it matches your 1596BBF0726835D8, but I cannot find it from the Flink KEYS[1]. Please correct me if my operation is wrong, thanks~ [1]

Re: [ANNOUNCE] Experimental Java 21 support now available on master

Congratulations Sergey, and everyone involved! Excited to work with and on this! Best, Swapnal On Wed, 29 Nov 2023, 13:58 Sergey Nuyanzin, wrote: > The master branch now builds and runs with Java 21 out-of-the-box. > > Notes: > - a nightly cron build was set up. > - In Java 21 builds, Scala

[ANNOUNCE] Experimental Java 21 support now available on master

The master branch now builds and runs with Java 21 out-of-the-box. Notes: - a nightly cron build was set up. - In Java 21 builds, Scala is being bumped to 2.12.18 which causes incompatibilities within Flink; i.e. don't try to load a savepoint from a Java 8/11/17 build - All the tests that are