Re: [VOTE] FLIP-372: Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable

2024-01-02 Thread Tzu-Li (Gordon) Tai
+1 (binding) looks good to me overall thank you for revising the FLIP and continuing to drive the decision, Peter! On Wed, Dec 27, 2023 at 7:16 AM Martijn Visser wrote: > Hi Peter, > > It would be good if Gordon can take a look, but overall this looks good to > me +1 > > Best regards, > >

[ANNOUNCE] Apache Flink Kafka Connectors 3.0.2 released

2023-12-01 Thread Tzu-Li (Gordon) Tai
The Apache Flink community is very happy to announce the release of Apache Flink Kafka Connectors 3.0.2. This release is compatible with the Apache Flink 1.17.x and 1.18.x release series. Apache Flink® is an open-source stream processing framework for distributed, high-performing,

Re: [RESULT][VOTE] Apache Flink Kafka Connectors v3.0.2, RC #1

2023-11-30 Thread Tzu-Li (Gordon) Tai
gt; > > - build source maven 3.8.6 and jdk11 > > > > - verified web PR > > > > > > > > Best regards, > > > > Jing > > > > > > > > On Sat, Nov 25, 2023 at 6:44 AM Rui Fan <1996fan...@gmail.com> > wrote: >

Re: [VOTE] Apache Flink Kafka Connectors v3.0.2, RC #1

2023-11-24 Thread Tzu-Li (Gordon) Tai
:compile [INFO] +- org.apache.logging.log4j:log4j-slf4j-impl:jar:2.17.1:runtime [INFO] +- org.apache.logging.log4j:log4j-api:jar:2.17.1:runtime [INFO] \- org.apache.logging.log4j:log4j-core:jar:2.17.1:runtime ``` On Fri, Nov 24, 2023 at 9:19 AM Tzu-Li (Gordon) Tai wrote: > Hi everyone, > > Plea

[VOTE] Apache Flink Kafka Connectors v3.0.2, RC #1

2023-11-24 Thread Tzu-Li (Gordon) Tai
Hi everyone, Please review and vote on release candidate #1 for version 3.0.2 of the Apache Flink Kafka Connector, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Compared to the previous hotfix release, this release only additionally

Re: [VOTE] Release flink-connector-opensearch v1.1.0, release candidate #1

2023-11-23 Thread Tzu-Li (Gordon) Tai
Hi Danny, Thanks for starting a RC for this. >From the looks of the staged POMs for 1.1.0-1.18, the flink versions for Flink dependencies still point to 1.17.1. My understanding is that this is fine, as those provided scope dependencies (e.g. flink-streaming-java) will have their versions

[ANNOUNCE] Apache Flink Kafka Connectors 3.0.1 released

2023-10-31 Thread Tzu-Li (Gordon) Tai
The Apache Flink community is very happy to announce the release of Apache Flink Kafka Connectors 3.0.1. This release is compatible with the Apache Flink 1.17.x and 1.18.x release series. Apache Flink® is an open-source stream processing framework for distributed, high-performing,

[RESULT] [VOTE] Apache Flink Kafka connector version 3.0.1, RC1

2023-10-30 Thread Tzu-Li (Gordon) Tai
I'm happy to announce that we have unanimously approved this release. There are 10 approving votes, 4 of which are binding: * Qingsheng Ren (binding) * Martijn Visser (binding) * Xianxun Ye * Mystic Lama * Leonard Xu (binding) * Ahmed Hamdy * Samrat Deb * Tzu-Li (Gordon) Tai (binding) * Sergey

Re: [VOTE] Apache Flink Kafka connector version 3.0.1, RC1

2023-10-30 Thread Tzu-Li (Gordon) Tai
ason > > > > On Mon, Oct 30, 2023 at 9:27 AM Sergey Nuyanzin > > wrote: > > > >> +1 (non-binding) > >> * Verified hashes and checksums > >> * Built from source > >> * Checked release tag > >> * Reviewed the web PR > >> > &g

Re: [DISCUSS] AWS Connectors v4.2.0 release + 1.18 support

2023-10-30 Thread Tzu-Li (Gordon) Tai
+1 On Mon, Oct 30, 2023 at 9:00 AM Danny Cranmer wrote: > Hey, > > > Did you mean skip 4.1.1, since 4.1.0 has already been released? > > I meant skip "4.1.0-1.18" since we could release this with the existing > source. We will additionally skip 4.1.1 and jump to 4.2.0 since this > version has

Re: [VOTE] Apache Flink Kafka connector version 3.0.1, RC1

2023-10-30 Thread Tzu-Li (Gordon) Tai
> >>> - Verified licenses > > > >>> - Verified web PR > > > >>> - Started a cluster and the Flink SQL client, successfully read and > > > >>> wrote with the Kafka connector to Confluent Cloud with AVRO and > > Schema > > &

[VOTE] Apache Flink Kafka connector version 3.0.1, RC1

2023-10-25 Thread Tzu-Li (Gordon) Tai
Hi everyone, Please review and vote on release candidate #1 for version 3.0.1 of the Apache Flink Kafka Connector, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) This release contains important changes for the following: - Supports

Re: [DISCUSS] Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable

2023-10-24 Thread Tzu-Li (Gordon) Tai
to migrate as soon as possible. On Tue, Oct 24, 2023 at 5:00 PM Tzu-Li (Gordon) Tai wrote: > Hi Peter, > > Thanks a lot for starting this FLIP! > > I agree that the current TwoPhaseCommittingSink interfaces is limiting in > that it assumes 1) committers have the same parallelism

Re: [DISCUSS] Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable

2023-10-24 Thread Tzu-Li (Gordon) Tai
Hi Peter, Thanks a lot for starting this FLIP! I agree that the current TwoPhaseCommittingSink interfaces is limiting in that it assumes 1) committers have the same parallelism as writers, and 2) writers immediately produce finalized committables. This FLIP captures the problem pretty well, and

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

2023-10-23 Thread Tzu-Li (Gordon) Tai
Hi David, Just to follow-up on that last question: I can confirm that there are no regressions for the Flink Kafka connector working with Flink 1.18. The previous nightly build failures were caused by breaking changes in test code, which has been resolved by now. I'll be creating new releases

Re: [VOTE] FLIP-371: Provide initialization context for Committer creation in TwoPhaseCommittingSink

2023-10-16 Thread Tzu-Li (Gordon) Tai
+1 (binding) On Mon, Oct 16, 2023 at 4:57 AM Martijn Visser wrote: > +1 (binding) > > On Mon, Oct 16, 2023 at 1:14 PM Leonard Xu wrote: > > > > No more comments, give my +1(binding) > > > > Best, > > Leonard > > > > > 2023年10月16日 下午6:20,Péter Váry 写道: > > > > > > Any more comments? > > > > >

Re: [DISCUSS] FLIP-371: SinkV2 Committer imporvements

2023-10-05 Thread Tzu-Li (Gordon) Tai
Thanks Peter for starting the FLIP. Overall, this seems pretty straightforward and overdue, +1. Two quick question / comments: 1. Can you rename the FLIP to something less generic? Perhaps "Provide initialization context for Committer creation in TwoPhaseCommittingSink"? 2. Can you

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

2023-09-29 Thread Tzu-Li (Gordon) Tai
Hi everyone, It’s been a while since this topic was last discussed, but nevertheless, it still remains very desirable to figure out a clear path towards making SinkV2 @Public. There’s a new thread [1] that has a pretty good description on missing features in SinkV2 from the Iceberg connector’s

Re: [DISCUSS] FLIP-319: Integrating with Kafka’s proper support for 2PC participation (KIP-939).

2023-08-31 Thread Tzu-Li (Gordon) Tai
bles the exactly-once Kafka sink in many > environments / > > use-cases where this was previously unfeasible due to the limitations > > highlighted in the FLIP. > > > > Big +1 > > > > Cheers, > > Gyula > > > > On Fri, Aug 18, 2023 at 7:54 PM Tzu-Li (Gor

Re: [DISCUSS] Status of Statefun Project

2023-08-18 Thread Tzu-Li (Gordon) Tai
solve the copying problem by requiring the two projects to be > siblings in the file system and by pre-copying the local build artifacts > into a location accessible by the existing Docker contexts. This would > still leave us with the need to have two PRs and releases instead of one, > t

Re: [DISCUSS] Status of Statefun Project

2023-08-18 Thread Tzu-Li (Gordon) Tai
default vs. local mode and what versions of Flink and Statefun > should be referenced, and then you can build a run the local examples > without any additional steps. Does that sound like a reasonable approach? > > > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai > wrote: > >

Re: [DISCUSS] Status of Statefun Project

2023-08-18 Thread Tzu-Li (Gordon) Tai
elp with Statefun releases. >>>> >>>> Best regards, >>>> >>>> Martijn >>>> >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren >>>> wrote: >>>> >>>>> Thanks. >>>>> >>>>> Marti

[DISCUSS] FLIP-319: Integrating with Kafka’s proper support for 2PC participation (KIP-939).

2023-08-18 Thread Tzu-Li (Gordon) Tai
Hi Flink devs, I’d like to officially start a discussion for FLIP-319: Integrating with Kafka’s proper support for 2PC participation (KIP-939) [1]. This is the “sister” joint FLIP for KIP-939 [2] [3]. It has been a long-standing issue that Flink’s Kafka connector doesn’t work fully correctly

[jira] [Created] (FLINK-32455) Breaking change in TypeSerializerUpgradeTestBase prevents flink-connector-kafka from building against 1.18-SNAPSHOT

2023-06-27 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-32455: --- Summary: Breaking change in TypeSerializerUpgradeTestBase prevents flink-connector-kafka from building against 1.18-SNAPSHOT Key: FLINK-32455 URL: https

[jira] [Created] (FLINK-32453) flink-connector-kafka does not build against Flink 1.18-SNAPSHOT

2023-06-27 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-32453: --- Summary: flink-connector-kafka does not build against Flink 1.18-SNAPSHOT Key: FLINK-32453 URL: https://issues.apache.org/jira/browse/FLINK-32453

Re: [DISCUSS] Status of Statefun Project

2023-06-17 Thread Tzu-Li (Gordon) Tai
> Perhaps he could weigh in on whether the combination of automated tests plus those smoke tests should be sufficient for testing with new Flink versions What we usually did at the bare minimum for new StateFun releases was the following: 1. Build tests (including the smoke tests in the e2e

Re: [VOTE] FLIP-287: Extend Sink#InitContext to expose TypeSerializer, ObjectReuse and JobID

2023-06-16 Thread Tzu-Li (Gordon) Tai
+1 (binding) On Fri, Jun 16, 2023, 09:53 Jing Ge wrote: > +1(binding) > > Best Regards, > Jing > > On Fri, Jun 16, 2023 at 10:10 AM Lijie Wang > wrote: > > > +1 (binding) > > > > Thanks for driving it, Joao. > > > > Best, > > Lijie > > > > Joao Boto 于2023年6月16日周五 15:53写道: > > > > > Hi all, >

Re: [VOTE] FLIP-246: Dynamic Kafka Source (originally Multi Cluster Kafka Source)

2023-06-16 Thread Tzu-Li (Gordon) Tai
+1 (binding) +1 for either DynamicKafkaSource or DiscoveringKafkaSource Cheers, Gordon On Thu, Jun 15, 2023, 10:56 Mason Chen wrote: > Hi all, > > Thank you to everyone for the feedback on FLIP-246 [1]. Based on the > discussion thread [2], we have come to a consensus on the design and are >

Re: [DISCUSS] FLIP-246: Multi Cluster Kafka Source

2023-06-14 Thread Tzu-Li (Gordon) Tai
gt; the FLIP, lossless consumer migration only works with a strategy that > > involves intermittent parallel consumption of old and new clusters to > drain > > and switch.) > > > > I think the "Table" in the name of those SQL connectors should avoid > > confusion

Re: [DISCUSS] FLIP-246: Multi Cluster Kafka Source

2023-06-09 Thread Tzu-Li (Gordon) Tai
ou mentioned, to use this in single cluster > mode (see *[1] *above). DynamicKafkaSource may be confusing because it is > really similar to the KafkaDynamicSource/Sink (table connectors). > > Best, > Mason > > On Wed, Jun 7, 2023 at 10:40 AM Tzu-Li (Gordon) Tai > wrote: > > &g

Re: [DISCUSS] FLIP-246: Multi Cluster Kafka Source

2023-06-07 Thread Tzu-Li (Gordon) Tai
Hi Mason, Thanks for updating the FLIP. In principle, I believe this would be a useful addition. Some comments so far: 1. In this sequence diagram [1], why is there a need for a GetMetadataUpdateEvent from the MultiClusterSourceReader going to the MultiClusterSourceEnumerator? Shouldn't the

Re: [DISCUSS] FLIP-287: Extend Sink#InitContext to expose ExecutionConfig and JobID

2023-05-30 Thread Tzu-Li (Gordon) Tai
Hi, > I think we can get the serializer directly in InitContextImpl through `getOperatorConfig().getTypeSerializerIn(0, getUserCodeClassloader()).duplicate()`. This should work, yes. +1 to the updated FLIP so far. Thank you, Joao, for being on top of this! Thanks, Gordon On Tue, May 30, 2023

[ANNOUNCE] Apache Flink Kafka Connectors 3.0.0 released

2023-04-21 Thread Tzu-Li (Gordon) Tai
The Apache Flink community is very happy to announce the release of Apache Flink Kafka Connectors 3.0.0. This release is compatible with the Apache Flink 1.17.x release series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and

Re: [VOTE] FLIP-288: Enable Dynamic Partition Discovery by Default in Kafka Source

2023-04-21 Thread Tzu-Li (Gordon) Tai
+1 On Thu, Apr 20, 2023 at 11:52 PM Hongshun Wang wrote: > Dear Flink Developers, > > > Thank you for providing feedback on FLIP-288: Enable Dynamic Partition > Discovery by Default in Kafka Source[1] on the discussion thread[2]. > > The goal of the FLIP is to enable partition discovery by

Re: [DISCUSS] FLIP-288:Enable Dynamic Partition Discovery by Default in Kafka Source

2023-04-21 Thread Tzu-Li (Gordon) Tai
> I have already modified FLIP-288 to provide a newDiscoveryOffsetsInitializer in the KafkaSourceBuilder and KafkaSourceEnumerator. Users can use KafkaSourceBuilder#setNewDiscoveryOffsets to change the strategy for new partitions. Thanks for addressing my comment Hongshun. > Considering these

Re: [DISCUSS] FLIP-287: Extend Sink#InitContext to expose ExecutionConfig and JobID

2023-04-21 Thread Tzu-Li (Gordon) Tai
ady, > while mutating the values at runtime is actually an undefined behavior. > > Thanks, > Zhu > > Tzu-Li (Gordon) Tai 于2023年4月18日周二 01:02写道: > > > > Hi, > > > > Sorry for chiming in late. > > > > I'm not so sure that exposing ExecutionConfig / ReadExe

Re: [VOTE] Release flink-connector-kafka 3.0.0 for Flink 1.17, release candidate #2

2023-04-20 Thread Tzu-Li (Gordon) Tai
We have unanimously approved this release. There are 6 approving votes, 3 of which are binding: * Alexander Sorokoumov * Martijn Visser (binding) * Tzu-Li (Gordon) Tai (binding) * Danny Cranmer (binding) * Ahmed Hamdy * Mason Chen Thanks so much everyone for testing and voting! I will now

Re: [VOTE] Release flink-connector-kafka 3.0.0 for Flink 1.17, release candidate #2

2023-04-18 Thread Tzu-Li (Gordon) Tai
gt; enabled > > On Fri, Apr 14, 2023 at 12:24 AM Alexander Sorokoumov > wrote: > > > +1 (nb). > > > > Checked: > > > >- checksums are correct > >- source code builds (JDK 8+11) > >- release notes are correct > > >

Re: [DISCUSS] FLIP-287: Extend Sink#InitContext to expose ExecutionConfig and JobID

2023-04-17 Thread Tzu-Li (Gordon) Tai
Hi, Sorry for chiming in late. I'm not so sure that exposing ExecutionConfig / ReadExecutionConfig directly through Sink#InitContext is the right thing to do. 1. A lot of the read-only getter methods on ExecutionConfig are irrelevant for sinks. Expanding the scope of the InitContext interface

Re: [VOTE] Release flink-connector-kafka, release candidate #1

2023-04-12 Thread Tzu-Li (Gordon) Tai
and will create a new RC >> shortly over the next day or two. >> >> Thanks, >> Gordon >> >> On Wed, Apr 5, 2023 at 9:32 PM Mason Chen wrote: >> >>> +1 for new RC! >>> >>> Best, >>> Mason >>> &g

Re: [VOTE] Release flink-connector-kafka 3.0.0 for Flink 1.17, release candidate #2

2023-04-12 Thread Tzu-Li (Gordon) Tai
v3.0.0. Thanks, Gordon [1] https://lists.apache.org/thread/r97y5qt8x0c72460vs5cjm5c729ljmh6 [2] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352577 On Wed, Apr 12, 2023 at 4:55 PM Tzu-Li (Gordon) Tai wrote: > Hi everyone, > > Please review and vote on release

[VOTE] Release flink-connector-kafka 3.0.0 for Flink 1.17, release candidate #2

2023-04-12 Thread Tzu-Li (Gordon) Tai
Hi everyone, Please review and vote on release candidate #2 for version 3.0.0 of the Apache Flink Kafka Connector, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which

Re: [VOTE] Release flink-connector-kafka, release candidate #1

2023-04-11 Thread Tzu-Li (Gordon) Tai
, 2023 at 11:32 AM Tzu-Li (Gordon) Tai > wrote: > > > Hi all, > > > > I've ported the critical fixes I mentioned to v3.0 and v4.0 branches of > > apache/flink-connector-kafka now. > > > > @martijnvis...@apache.org let me know if > you'd > > need h

[jira] [Created] (FLINK-31740) Allow setting boundedness for upsert-kafka SQL connector

2023-04-05 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-31740: --- Summary: Allow setting boundedness for upsert-kafka SQL connector Key: FLINK-31740 URL: https://issues.apache.org/jira/browse/FLINK-31740 Project: Flink

Re: [VOTE] Release flink-connector-kafka, release candidate #1

2023-04-04 Thread Tzu-Li (Gordon) Tai
at 11:21 PM Konstantin Knauf wrote: > +1. Thanks, Gordon! > > Am Mo., 3. Apr. 2023 um 06:37 Uhr schrieb Tzu-Li (Gordon) Tai < > tzuli...@apache.org>: > > > Hi Martijn, > > > > Since this RC vote was opened, we had three critical bug fixes that

Re: [VOTE] Release flink-connector-kafka, release candidate #1

2023-04-02 Thread Tzu-Li (Gordon) Tai
already, it doesn't seem like we're throwing away too much effort that has already been done if we start a new RC with these critical fixes included. What do you think? Thanks, Gordon On Thu, Feb 9, 2023 at 3:26 PM Tzu-Li (Gordon) Tai wrote: > +1 (binding) > > - Verified legals (licens

Re: [DISCUSS] FLIP-288:Enable Dynamic Partition Discovery by Default in Kafka Source

2023-03-29 Thread Tzu-Li (Gordon) Tai
Hi Hongshun, Thank you for drafting the FLIP for this. Overall, the intent of the FLIP makes sense to me. I'm actually surprised that the partition discovery feature works as it is in the KafkaSource today - in older versions of the Kafka source connector (implemented on source v1), newly

Re: [ANNOUNCE] Kafka Connector Code Removal from apache/flink:main branch and code freezing

2023-03-27 Thread Tzu-Li (Gordon) Tai
gt; > can help me review it in your free time. > > > > > > [1] https://github.com/apache/flink-connector-kafka/pull/10 > > > > > > Best Regards, > > > Ran Tao > > > > > > > > > Tzu-Li (Gordon) Tai 于2023年3月24日周五 23:21写道: >

Re: [ANNOUNCE] Kafka Connector Code Removal from apache/flink:main branch and code freezing

2023-03-24 Thread Tzu-Li (Gordon) Tai
k/pull/21965 > > Best regards, > Jing > > On Fri, Mar 24, 2023 at 4:06 PM Tzu-Li (Gordon) Tai > wrote: > > > Hi all, > > > > Now that Flink 1.17 has been released, and given that we've already > synced > > the latest Kafka connector code up to Flink 1.1

[ANNOUNCE] Kafka Connector Code Removal from apache/flink:main branch and code freezing

2023-03-24 Thread Tzu-Li (Gordon) Tai
Hi all, Now that Flink 1.17 has been released, and given that we've already synced the latest Kafka connector code up to Flink 1.17 to the apache/flink-connector-kafka repo (thanks to Mason and Martijn for most of the effort!), we're now in the final step of completely removing the Kafka

Re: [VOTE] Release flink-connector-parent10.0, release candidate #1

2023-03-16 Thread Tzu-Li (Gordon) Tai
+1 (binding) - Verified hashes and signature - Read web PR - Staging Maven artifacts looks good Thanks, Gordon On Thu, Mar 16, 2023 at 1:24 PM Martijn Visser wrote: > +1 (binding) > > - Validated hashes > - Verified signature > - Verified that no binaries exist in the source archive > -

Re: [DISCUSS] Enabling dynamic partition discovery by default in Kafka source

2023-02-10 Thread Tzu-Li (Gordon) Tai
PM Tzu-Li (Gordon) Tai wrote: > re: @Mason about TimestampOffsetInitializer default offset reset strategy > > Should the OffsetsInitializer even be respected for partitions from new > discoveries? By "new discoveries", I mean partitions that are discovered > outside of

Re: [DISCUSS] Enabling dynamic partition discovery by default in Kafka source

2023-02-10 Thread Tzu-Li (Gordon) Tai
heckpoints, and for the first set of partitions that already exist at startup time. In all other scenarios, we required dedicated ways to handle them which we probably shouldn't expose to the user for configuring. On Fri, Feb 10, 2023 at 1:34 PM Tzu-Li (Gordon) Tai wrote: > Joined the discussion a bit late,

Re: [DISCUSS] Enabling dynamic partition discovery by default in Kafka source

2023-02-10 Thread Tzu-Li (Gordon) Tai
Joined the discussion a bit late, but just want to add in my +1 as well :-) Historically, when dynamic partition discovery was implemented in the earlier versions of the FlinkKafkaConsumer, it was implemented such that multiple source subtasks would in parallel query Kafka brokers for

Re: [VOTE] Release flink-connector-kafka, release candidate #1

2023-02-09 Thread Tzu-Li (Gordon) Tai
+1 (binding) - Verified legals (license headers and root LICENSE / NOTICE file). AFAICT no dependencies require explicit acknowledgement in the NOTICE files. - No binaries in staging area - Built source with tests - Verified signatures and hashes - Web PR changes LGTM Thanks Martijn! Cheers,

Re: Stateful Functions with Flink 1.15 and onwards

2022-11-04 Thread Tzu-Li (Gordon) Tai
to get the release out ASAP. [1] https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Stateful+Functions+Release On Fri, Nov 4, 2022 at 5:54 AM Galen Warren wrote: > Thanks Gordon. > > What is the next step here? > > On Thu, Nov 3, 2022 at 1:45 PM Tzu-Li (Gordo

Re: Stateful Functions with Flink 1.15 and onwards

2022-11-03 Thread Tzu-Li (Gordon) Tai
FYI, release-3.3 branch has been cut and is ready for the release process for StateFun 3.3.0: https://github.com/apache/flink-statefun/tree/release-3.3 On Tue, Nov 1, 2022 at 10:21 AM Tzu-Li (Gordon) Tai wrote: > Btw, I'll assume that we're using this thread to gather consensus for >

Re: Stateful Functions with Flink 1.15 and onwards

2022-11-01 Thread Tzu-Li (Gordon) Tai
ing the release branch for 3.3.x, unless anyone raises any objections before that. Thanks, Gordon On Tue, Nov 1, 2022 at 10:09 AM Galen Warren wrote: > Thanks Gordon and Filip, I appreciate your help on this one. > > On Tue, Nov 1, 2022 at 1:07 PM Tzu-Li (Gordon) Tai > wrote: > >> P

Re: Stateful Functions with Flink 1.15 and onwards

2022-11-01 Thread Tzu-Li (Gordon) Tai
years back didn't have access to keyed state) > >>>>>> > >>>>>> > >>>>>> P.S. Please note that there is already a pull request that has > >>>>>> something to do wtih Flink 1.15, albeit wi

Re: [VOTE] FLIP-263: Improve resolving schema compatibility

2022-10-31 Thread Tzu-Li (Gordon) Tai
+1. Thanks Hangxiang, the proposed plan looks good! On Sun, Oct 30, 2022 at 3:38 AM Yu Li wrote: > +1 (binding) > > Thanks Hangxiang for driving this and thanks all for the thorough > discussion. > > Best Regards, > Yu > > > On Fri, 28 Oct 2022 at 16:01, Dawid Wysakowicz > wrote: > > > +1, > >

Re: [VOTE] Drop TypeSerializerConfigSnapshot and savepoint support from Flink versions < 1.8.0

2022-10-28 Thread Tzu-Li (Gordon) Tai
+1 On Fri, Oct 28, 2022 at 10:21 AM Konstantin Knauf wrote: > +1 (binding) > > Am Fr., 28. Okt. 2022 um 16:58 Uhr schrieb Piotr Nowojski < > pnowoj...@apache.org>: > > > Hi, > > > > As discussed on the dev mailing list [0] I would like to start a vote to > > drop support of older savepoint

Re: Stateful Functions with Flink 1.15 and onwards

2022-10-26 Thread Tzu-Li (Gordon) Tai
Hi Filip, Thanks for bringing this up. The hard truth is that committers who were previously active on the StateFun subproject, including myself, all currently have other focuses. Indeed, we may need to discuss with the community on how to proceed if there seems to be no continued committer

Re: [VOTE] FLIP-252: Amazon DynamoDB Sink Connector

2022-07-20 Thread Tzu-Li (Gordon) Tai
+1 On Wed, Jul 20, 2022 at 6:13 AM Danny Cranmer wrote: > Hi there, > > After the discussion in [1], I’d like to open a voting thread for FLIP-252 > [2], which proposes the addition of an Amazon DynamoDB sink based on the > Async Sink [3]. > > The vote will be open until July 23rd earliest

Re: [VOTE] Apache Flink Stateful Functions 3.2.0, release candidate #1

2022-01-28 Thread Tzu-Li (Gordon) Tai
+1 (binding) - verified checksums and signature - built from source with e2e tests - Ran Js greeter example - Checked new updated Dockerfile to be released Thanks, Gordon On Fri, Jan 28, 2022 at 8:58 AM Seth Wiesman wrote: > +1 (non-binding) > > - verified checksums and signatures > - built

Re: [DISCUSS] Stateful Functions 3.2.0 release?

2022-01-24 Thread Tzu-Li (Gordon) Tai
+1 On Mon, Jan 24, 2022 at 9:43 AM Igal Shilman wrote: > +1 and thanks for volunteering to be the release manager! > > Cheers, > Igal. > > On Mon, Jan 24, 2022 at 4:13 PM Seth Wiesman wrote: > > > +1 > > > > These are already a useful set of features to release to our users. > > > > Seth > > >

Re: [VOTE] Stateful functions 3.1.1 release

2021-12-20 Thread Tzu-Li (Gordon) Tai
+1 (binding) - Checked hash and signatures - Checked diff contains Flink upgrade - mvn clean install with e2e Thanks, Gordon On Mon, Dec 20, 2021, 13:55 Seth Wiesman wrote: > +1 (non-binding) > > - Verified signatures > - Checked diff > - Checked site PR > - Build from source and ran e2e

Re: [DISCUSS] Immediate dedicated StateFun releases for log4j vulnerability

2021-12-16 Thread Tzu-Li (Gordon) Tai
+1 On Thu, Dec 16, 2021, 10:30 Igal Shilman wrote: > Hi All, > > Following the recent Apache Flink releases due to the log4j vulnerability, > I'd like to propose an immediate StateFun release - 3.1.1. > This release is basically the same as 3.1 but updates the Flink dependency > to 1.13.3. > >

Re: [VOTE] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-14 Thread Tzu-Li (Gordon) Tai
+1 (binding) - verified hashes and signatures - checked that diff of all RCs contain only the log4j version upgrade On Tue, Dec 14, 2021 at 4:06 AM Yun Gao wrote: > +1 (non-binding) > > * Reviewed the blog post. > * Verified each version could run normally with example jobs. > * Checked each

Re: [ANNOUNCE] Apache Flink Stateful Functions 3.1.0 released

2021-08-31 Thread Tzu-Li (Gordon) Tai
Congrats on the release! And thank you for driving this release, Igal. Cheers Gordon On Tue, Aug 31, 2021, 23:13 Igal Shilman wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink Stateful Functions (StateFun) 3.1.0. > > StateFun is a cross-platform stack

Re: [VOTE] Apache Flink Stateful Functions 3.1.0, release candidate #1

2021-08-26 Thread Tzu-Li (Gordon) Tai
+1 (binding) - Built from source with Java 11 and Java 8 (mvn clean install -Prun-e2e-tests) - verified signatures and hashes - verified NOTICE files of Maven artifacts properly list actual bundled dependencies - Ran GoLang greeter and showcase with the proposed Dockerfiles for 3.1.0 - Ran a

Re: [NOTICE] StateFun docs have been migrated to new ASF Buildbot at ci2.apache.org

2021-08-22 Thread Tzu-Li (Gordon) Tai
gt; On 16/08/2021 08:59, Tzu-Li (Gordon) Tai wrote: > > Hi all, > > > > Just a quick announcement that we've officially migrated the StateFun > docs > > to ASF's new buildbot at https://ci2.apache.org/. > > > > There are a few other notable changes that too

[jira] [Created] (FLINK-23853) Update StateFun's Flink dependency to 1.13.2

2021-08-18 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-23853: --- Summary: Update StateFun's Flink dependency to 1.13.2 Key: FLINK-23853 URL: https://issues.apache.org/jira/browse/FLINK-23853 Project: Flink

Re: [DISCUSS] Stateful Functions 3.1.0 release?

2021-08-16 Thread Tzu-Li (Gordon) Tai
Hi Igal, Thanks a lot for starting this discussion! The list of features sounds like this will be a great release. As the list is already quite packed with major features, +1 to feature freeze on August 20 so we can move towards 3.1.0. Thanks, Gordon On Mon, Aug 16, 2021, 18:57 Igal Shilman

[NOTICE] StateFun docs have been migrated to new ASF Buildbot at ci2.apache.org

2021-08-16 Thread Tzu-Li (Gordon) Tai
Hi all, Just a quick announcement that we've officially migrated the StateFun docs to ASF's new buildbot at https://ci2.apache.org/. There are a few other notable changes that took part in this migration: - The buildbot configuration file now using Git, and lives at

[jira] [Created] (FLINK-23762) Revisit RequestReplyFunctionTest unit tests

2021-08-13 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-23762: --- Summary: Revisit RequestReplyFunctionTest unit tests Key: FLINK-23762 URL: https://issues.apache.org/jira/browse/FLINK-23762 Project: Flink

[jira] [Created] (FLINK-23718) Manage StateFun Python SDK as a Maven module

2021-08-10 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-23718: --- Summary: Manage StateFun Python SDK as a Maven module Key: FLINK-23718 URL: https://issues.apache.org/jira/browse/FLINK-23718 Project: Flink

[jira] [Created] (FLINK-23717) Allow setting configs using plain strings in StatefulFunctionsAppContainers

2021-08-10 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-23717: --- Summary: Allow setting configs using plain strings in StatefulFunctionsAppContainers Key: FLINK-23717 URL: https://issues.apache.org/jira/browse/FLINK-23717

[jira] [Created] (FLINK-23714) Expose both master and worker logs when using StatefulFunctionsAppContainers

2021-08-10 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-23714: --- Summary: Expose both master and worker logs when using StatefulFunctionsAppContainers Key: FLINK-23714 URL: https://issues.apache.org/jira/browse/FLINK-23714

[jira] [Created] (FLINK-23709) Remove SanityVerificationE2E and ExactlyOnceRemoteE2E

2021-08-10 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-23709: --- Summary: Remove SanityVerificationE2E and ExactlyOnceRemoteE2E Key: FLINK-23709 URL: https://issues.apache.org/jira/browse/FLINK-23709 Project: Flink

Re: [VOTE] FLIP-180: Adjust StreamStatus and Idleness definition

2021-08-09 Thread Tzu-Li (Gordon) Tai
+1 (binding) On Tue, Aug 10, 2021 at 12:15 AM Piotr Nowojski wrote: > +1 (binding) > > Piotrek > > pon., 9 sie 2021 o 18:00 Stephan Ewen napisał(a): > > > +1 (binding) > > > > On Mon, Aug 9, 2021 at 12:08 PM Till Rohrmann > > wrote: > > > > > +1 (binding) > > > > > > Cheers, > > > Till > > >

[jira] [Created] (FLINK-23600) Rework StateFun's remote module parsing and binding

2021-08-03 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-23600: --- Summary: Rework StateFun's remote module parsing and binding Key: FLINK-23600 URL: https://issues.apache.org/jira/browse/FLINK-23600 Project: Flink

[jira] [Created] (FLINK-23296) Add RequestReplyClientFactory as a pluggable extension

2021-07-07 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-23296: --- Summary: Add RequestReplyClientFactory as a pluggable extension Key: FLINK-23296 URL: https://issues.apache.org/jira/browse/FLINK-23296 Project: Flink

[jira] [Created] (FLINK-23295) Introduce extension module SPI to StateFun

2021-07-07 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-23295: --- Summary: Introduce extension module SPI to StateFun Key: FLINK-23295 URL: https://issues.apache.org/jira/browse/FLINK-23295 Project: Flink

[jira] [Created] (FLINK-23293) Support pluggable / extendable HTTP transport for StateFun remote invocations

2021-07-07 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-23293: --- Summary: Support pluggable / extendable HTTP transport for StateFun remote invocations Key: FLINK-23293 URL: https://issues.apache.org/jira/browse/FLINK-23293

Re: [DISCUSS] Definition of idle partitions

2021-06-09 Thread Tzu-Li (Gordon) Tai
Forgot to provide the link to the [1] reference: [1] https://issues.apache.org/jira/browse/FLINK-5017 On Thu, Jun 10, 2021 at 11:43 AM Tzu-Li (Gordon) Tai wrote: > Hi everyone, > > Sorry for chiming in late here. > > Regarding the topic of changing the definition of StreamStatus

Re: [DISCUSS] Definition of idle partitions

2021-06-09 Thread Tzu-Li (Gordon) Tai
Hi everyone, Sorry for chiming in late here. Regarding the topic of changing the definition of StreamStatus and changing the name as well: After digging into some of the roots of this implementation [1], initially the StreamStatus was actually defined to mark "watermark idleness", and not

Re: [VOTE] Watermark propagation with Sink API

2021-06-09 Thread Tzu-Li (Gordon) Tai
+1 On Thu, Jun 10, 2021 at 9:04 AM jincheng sun wrote: > +1 (binding) // Sorry for the late reply. > > Best, > Jincheng > > > Piotr Nowojski 于2021年6月9日周三 下午10:04写道: > > > Thanks for driving this Eron, and sorry for causing the delay. > > > > +1 (binding) from my side > > > > Piotrek > > > >

[jira] [Created] (FLINK-22529) StateFun Kinesis ingresses should support configs that are available via FlinkKinesisConsumer's ConsumerConfigConstants

2021-04-29 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-22529: --- Summary: StateFun Kinesis ingresses should support configs that are available via FlinkKinesisConsumer's ConsumerConfigConstants Key: FLINK-22529 URL: https

[ANNOUNCE] Apache Flink Stateful Functions 3.0.0 released

2021-04-15 Thread Tzu-Li (Gordon) Tai
The Apache Flink community is very happy to announce the release of Apache Flink Stateful Functions (StateFun) 3.0.0. StateFun is a cross-platform stack for building Stateful Serverless applications, making it radically simpler to develop scalable, consistent, and elastic distributed

Re: [PSA] Documentation is currently not being updated

2021-04-14 Thread Tzu-Li (Gordon) Tai
The same thing happened to the StateFun builds as well. They also have not been running since April 7th. After asking infra to reset the builds, the issue persisted. I recently switched the node to asf945_ubuntu for the StateFun doc builds because of this. On Thu, Apr 15, 2021, 12:48 AM Chesnay

Re: [VOTE] Apache Flink Stateful Functions 3.0.0, release candidate #1

2021-04-08 Thread Tzu-Li (Gordon) Tai
gt; +1 (binding) > > on this release. Thanks a lot for driving this! > > > On Thu, Apr 8, 2021 at 5:19 AM Tzu-Li (Gordon) Tai > wrote: > >> @Robert Metzger >> >> Sorry, this is the correct link to the class file you are referring to >> (previous link I mentioned i

Re: [VOTE] Apache Flink Stateful Functions 3.0.0, release candidate #1

2021-04-07 Thread Tzu-Li (Gordon) Tai
at 11:17 AM Tzu-Li (Gordon) Tai wrote: > @Robert Metzger > > I assume the com/google/protobuf classfile you found is this one: > > https://github.com/apache/flink-statefun/blob/master/statefun-shaded/statefun-protobuf-shaded/src/main/java/org/apache/flink/statefun/sdk/shaded/com/

Re: [VOTE] Apache Flink Stateful Functions 3.0.0, release candidate #1

2021-04-07 Thread Tzu-Li (Gordon) Tai
ll -Prun-e2e-tests (java 8) from source > > > > - python3 -m unittest tests > > > > - spin up Statefun Cluster on EKS with an image built from the > > > Dockerfiles > > > > of [1] > > > > - run Python & Java Greeter example on AWS Lambd

Re: [VOTE] Apache Flink Stateful Functions 3.0.0, release candidate #1

2021-03-31 Thread Tzu-Li (Gordon) Tai
nded the smoke test to include the remote sdks (1 function in the > Java SDK, 1 function in the Python SDK), and it passes. > - deployed to kubernetes with minio as an S3 replacement. > > > On Tue, Mar 30, 2021 at 12:29 PM Tzu-Li (Gordon) Tai > wrote: > > > Hi everyone, &

Re: [DISCUSS] Releasing Stateful Functions 3.0.0

2021-03-30 Thread Tzu-Li (Gordon) Tai
describes the > > breaking changes/migration in more detail in order to prepare? > > > > Cheers, > > > > Konstantin > > > > On Thu, Mar 25, 2021 at 9:27 AM Tzu-Li (Gordon) Tai > > > wrote: > > > > > Hi everyone, > > &g

[VOTE] Apache Flink Stateful Functions 3.0.0, release candidate #1

2021-03-30 Thread Tzu-Li (Gordon) Tai
Hi everyone, Please review and vote on the release candidate #1 for the version 3.0.0 of Apache Flink Stateful Functions, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Testing Guideline** You can find here [1] a page in the

[jira] [Created] (FLINK-22023) Remove outdated StateFun quickstart archetype

2021-03-30 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-22023: --- Summary: Remove outdated StateFun quickstart archetype Key: FLINK-22023 URL: https://issues.apache.org/jira/browse/FLINK-22023 Project: Flink

[DISCUSS] Releasing Stateful Functions 3.0.0

2021-03-25 Thread Tzu-Li (Gordon) Tai
Hi everyone, We'd like to prepare to release StateFun 3.0.0 over the next few days, ideally starting the first release candidate early next week. This is a version bump from 2.x to 3.x, with some major new features and reworks: - New request-reply protocol: the protocol has been reworked

[jira] [Created] (FLINK-21904) parseJmJvmArgsAndExportLogs: command not found warning when starting StateFun

2021-03-21 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-21904: --- Summary: parseJmJvmArgsAndExportLogs: command not found warning when starting StateFun Key: FLINK-21904 URL: https://issues.apache.org/jira/browse/FLINK-21904

[jira] [Created] (FLINK-21901) Update StateFun version to 3.0-SNAPSHOT

2021-03-21 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-21901: --- Summary: Update StateFun version to 3.0-SNAPSHOT Key: FLINK-21901 URL: https://issues.apache.org/jira/browse/FLINK-21901 Project: Flink Issue

  1   2   3   4   5   6   7   8   9   >