Re: [DISCUSS] Connector Externalization Retrospective

[Muhammet Orazov]

Hello Danny,

Thanks for the starting the discussion.

-1 for mono-repo, and -+1 for dropping Flink version.

I have mixed opinion with dropping the Flink version. Usually, large
production migrations happen on Flink versions and users want also
naturally update the connectors compatible for that Flink version.


which is a burden on the community.


Maybe this is another point we should address?

I agree with Sergey's point to have CI builds with SNAPSHOT versions,
which would make updating the versions easily. We could start updating
builds to include SNAPSHOT version if they are missing.

Another suggestion would be to have a dedicated owners (PMC/committers)
of set of connectors that are responsible for these regular update
tasks together with volunteers. Maybe this should be decided similar
to release managers before each planned release.

Best,
Muhammet


On 2024-06-10 16:25, Danny Cranmer wrote:

Hello Flink community,

It has been over 2 years [1] since we started externalizing the Flink
connectors to dedicated repositories from the main Flink code base. The
past discussions can be found here [2]. The community decided to
externalize the connectors to primarily 1/ improve stability and speed 
of
the CI, and 2/ decouple version and release lifecycle to allow the 
projects
to evolve independently. The outcome of this has resulted in each 
connector
requiring a dedicated release per Flink minor version, which is a 
burden on

the community. Flink 1.19.0 was released on 2024-03-18 [3], the first
supported connector followed roughly 2.5 months later on 2024-06-06 [4]
(MongoDB). There are still 5 connectors that do not support Flink 1.19 
[5].


Two decisions contribute to the high lag between releases. 1/ creating 
one
repository per connector instead of a single flink-connector mono-repo 
and

2/ coupling the Flink version to the connector version [6]. A single
connector repository would reduce the number of connector releases from 
N

to 1, but would couple the connector CI and reduce release flexibility.
Decoupling the connector versions from Flink would eliminate the need 
to
release each connector for each new Flink minor version, but we would 
need

a new compatibility mechanism.

I propose that from each next connector release we drop the coupling on 
the
Flink version. For example, instead of 3.4.0-1.20 (.) 
we

would release 3.4.0 (). We can model a compatibility matrix
within the Flink docs to help users pick the correct versions. This 
would
mean we would usually not need to release a new connector version per 
Flink
version, assuming there are no breaking changes. Worst case, if 
breaking

changes impact all connectors we would still need to release all
connectors. However, for Flink 1.17 and 1.18 there were only a handful 
of
issues (breaking changes), and mostly impacting tests. We could decide 
to

align this with Flink 2.0, however I see no compelling reason to do so.
This was discussed previously [2] as a long term goal once the 
connector
APIs are stable. But I think the current compatibility rules support 
this

change now.

I would prefer to not create a connector mono-repo. Separate repos 
gives

each connector more flexibility to evolve independently, and removing
unnecessary releases will significantly reduce the release effort.

I would like to hear opinions and ideas from the community. In 
particular,

are there any other issues you have observed that we should consider
addressing?

Thanks,
Danny.

[1]
https://github.com/apache/flink-connector-elasticsearch/commit/3ca2e625e3149e8864a4ad478773ab4a82720241
[2] https://lists.apache.org/thread/8k1xonqt7hn0xldbky1cxfx3fzh6sj7h
[3]
https://flink.apache.org/2024/03/18/announcing-the-release-of-apache-flink-1.19/
[4] https://flink.apache.org/downloads/#apache-flink-connectors-1
[5] https://issues.apache.org/jira/browse/FLINK-35131
[6]
https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development#ExternalizedConnectordevelopment-Examples

Re: [VOTE] FLIP-459: Support Flink hybrid shuffle integration with Apache Celeborn

[Muhammet Orazov]

+1 (non-binding)

Thanks Yuxin for driving this!

Best,
Muhammet


On 2024-06-07 08:02, Yuxin Tan wrote:

Hi everyone,

Thanks for all the feedback about the FLIP-459 Support Flink
hybrid shuffle integration with Apache Celeborn[1].
The discussion thread is here [2].

I'd like to start a vote for it. The vote will be open for at least
72 hours unless there is an objection or insufficient votes.

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-459%3A+Support+Flink+hybrid+shuffle+integration+with+Apache+Celeborn
[2] https://lists.apache.org/thread/gy7sm7qrf7yrv1rl5f4vtk5fo463ts33

Best,
Yuxin

Re: [ANNOUNCE] New Apache Flink PMC Member - Fan Rui

[Muhammet Orazov]

Congratulations Rui, well deserved!

Best,
Muhammet

On 2024-06-05 10:01, Piotr Nowojski wrote:

Hi everyone,

On behalf of the PMC, I'm very happy to announce another new Apache 
Flink

PMC Member - Fan Rui.

Rui has been active in the community since August 2019. During this 
time he

has contributed a lot of new features. Among others:
  - Decoupling Autoscaler from Kubernetes Operator, and supporting
Standalone Autoscaler
  - Improvements to checkpointing, flamegraphs, restart strategies,
watermark alignment, network shuffles
  - Optimizing the memory and CPU usage of large operators, greatly
reducing the risk and probability of TaskManager OOM

He reviewed a significant amount of PRs and has been active both on the
mailing lists and in Jira helping to both maintain and grow Apache 
Flink's

community. He is also our current Flink 1.20 release manager.

In the last 12 months, Rui has been the most active contributor in the
Flink Kubernetes Operator project, while being the 2nd most active 
Flink

contributor at the same time.

Please join me in welcoming and congratulating Fan Rui!

Best,
Piotrek (on behalf of the Flink PMC)

[jira] [Created] (FLINK-35491) [JUnit5 Migration] Module: Flink CDC modules

[Muhammet Orazov (Jira)]

Muhammet Orazov created FLINK-35491:
---

 Summary: [JUnit5 Migration] Module: Flink CDC modules
 Key: FLINK-35491
 URL: https://issues.apache.org/jira/browse/FLINK-35491
 Project: Flink
  Issue Type: Improvement
  Components: Flink CDC
Reporter: Muhammet Orazov


Migrate Junit4 tests to Junit5 for the following modules:
 * flink-cdc-common
 * flink-cdc-composer
 * flink-cdc-runtime
 * flink-cdc-connect/flink-cdc-pipeline-connectors
 * flink-cdc-e2e-tests



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-35490) [JUnit5 Migration] Module: Flink CDC flink-cdc-connect/flink-cdc-source-connectors

[Muhammet Orazov (Jira)]

Muhammet Orazov created FLINK-35490:
---

 Summary: [JUnit5 Migration] Module: Flink CDC 
flink-cdc-connect/flink-cdc-source-connectors
 Key: FLINK-35490
 URL: https://issues.apache.org/jira/browse/FLINK-35490
 Project: Flink
  Issue Type: Improvement
  Components: Flink CDC
Reporter: Muhammet Orazov


Migrate Junit4 tests to Junit5 in the Flink CDC following modules:

 

- flink-cdc-connect/flink-cdc-source-connectors



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] Flink CDC 3.1.1 Release

[Muhammet Orazov]

Thanks Xiqian,

It is good idea thanks, +1

Best,
Muhammet

On 2024-05-28 11:43, Xiqian YU wrote:

Hi devs,

I would like to make a proposal about creating a new Flink CDC 3.1 
patch release (3.1.1). It’s been a week since the last CDC version 
3.1.0 got released [1], and since then, 7 tickets have been closed, 4 
of them are of high priority.


Currently, there are 5 items open at the moment: 1 of them is a 
blocker, which stops users from restoring with existed checkpoints 
after upgrading [2]. There’s a PR ready and will be merged soon. Other 
4 of them have approved PRs, and will be merged soon [3][4][5][6]. I 
propose that a patch version could be released after all pending 
tickets closed.


Please reply if there are any unresolved blocking issues you’d like to 
include in this release.


Regards,
Xiqian

[1] 
https://flink.apache.org/2024/05/17/apache-flink-cdc-3.1.0-release-announcement/

[2] https://issues.apache.org/jira/browse/FLINK-35464
[3] https://issues.apache.org/jira/browse/FLINK-35149
[4] https://issues.apache.org/jira/browse/FLINK-35323
[5] https://issues.apache.org/jira/browse/FLINK-35430
[6] https://issues.apache.org/jira/browse/FLINK-35447

Re: [DISCUSS] Flink CDC 3.2 Release Planning

[Muhammet Orazov]

Hey Xiqian,

+1 to the change of schedule

Thanks and best,
Muhammet

On 2024-05-27 02:36, Xiqian YU wrote:

Hi devs,

Glad to see Flink CDC 3.1’s successful release! Also noticed that it’s 
a delayed release due to license compliance chores, which took more 
than half a month to process.


In 3.2 KO plannings, it is expected to add some strongly demanded 
feature to meet YAML pipeline job users’ needs [1]. However, 0 out of 
10 3.2 feature tickets have been completed till now, and it’s very 
unlikely for us to catch up with planned feature freeze (May 25th) and 
release deadline (June 1st) [2].


I hereby suggest postponing release schedule by 2 weeks, extending 
feature freeze day to June 8th and release day to June 15th. 
Considering its wide affect, please leave your comments & concerns 
about this suggestion.


Regards,
Xiqian

[1] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=607
[2] 
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794651



De : Peter Huang 
Date : jeudi, 9 mai 2024 à 23:31
À : dev@flink.apache.org 
Objet : Re: [DISCUSS] Flink CDC 3.2 Release Planning
Thanks Qingsheng for driving the release!

+1. I also would like to provide some help on CDC 3.2.


Best Regards
Peter Huang

On Thu, May 9, 2024 at 3:21 AM Xiqian YU  wrote:


Thanks Qingsheng for driving the release!

+1. Would love to provide my help on CDC 3.2.

Regards,
Xiqian

De : Hang Ruan 
Date : jeudi, 9 mai 2024 à 17:50
À : dev@flink.apache.org 
Objet : Re: [DISCUSS] Flink CDC 3.2 Release Planning
Thanks Qinsheng for driving.

I would like to provide some helps for this verison too. +1.

Best,
Hang

Hongshun Wang  于2024年5月9日周四 14:16写道：

> Thanks Qinsheng for driving,
> +1 from my side.
>
> Besi,
> Hongshun
>
> On Wed, May 8, 2024 at 11:41 PM Leonard Xu  wrote:
>
> > +1 for the proposal code freeze date and RM candidate.
> >
> > Best,
> > Leonard
> >
> > > 2024年5月8日 下午10:27，gongzhongqiang  写道：
> > >
> > > Hi Qingsheng
> > >
> > > Thank you for driving the release.
> > > Agree with the goal and I'm willing to help.
> > >
> > > Best,
> > > Zhongqiang Gong
> > >
> > > Qingsheng Ren  于2024年5月8日周三 14:22写道：
> > >
> > >> Hi devs,
> > >>
> > >> As we are in the midst of the release voting process for Flink CDC
> > 3.1.0, I
> > >> think it's a good time to kick off the upcoming Flink CDC 3.2
release
> > >> cycle.
> > >>
> > >> In this release cycle I would like to focus on the stability of
Flink
> > CDC,
> > >> especially for the newly introduced YAML-based data integration
> > >> framework. To ensure we can iterate and improve swiftly, I propose
to
> > make
> > >> 3.2 a relatively short release cycle, targeting a feature freeze by
> May
> > 24,
> > >> 2024.
> > >>
> > >> For developers that are interested in participating and contributing
> new
> > >> features in this release cycle, please feel free to list your
planning
> > >> features in the wiki page [1].
> > >>
> > >> I'm happy to volunteer as a release manager and of course open to
work
> > >> together with someone on this.
> > >>
> > >> What do you think?
> > >>
> > >> Best,
> > >> Qingsheng
> > >>
> > >> [1]
> > >>
> https://cwiki.apache.org/confluence/display/FLINK/Flink+CDC+3.2+Release
> > >>
> >
> >
>

Re: [VOTE] Release flink-connector-cassandra v3.2.0, release candidate #1

[Muhammet Orazov]

Hey all,

Could we please get some more votes to proceed with the release?

Thanks and best,
Muhammet

On 2024-04-22 13:04, Danny Cranmer wrote:

Hi everyone,

Please review and vote on release candidate #1 for
flink-connector-cassandra v3.2.0, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

This release supports Flink 1.18 and 1.19.

The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org 
[2],

which are signed with the key with fingerprint 125FD8DB [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag v3.2.0-rc1 [5],
* website pull request listing the new release [6].
* CI build of the tag [7].

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Danny

[1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353148
[2]
https://dist.apache.org/repos/dist/dev/flink/flink-connector-cassandra-3.2.0-rc1
[3] https://dist.apache.org/repos/dist/release/flink/KEYS
[4] 
https://repository.apache.org/content/repositories/orgapacheflink-1722

[5]
https://github.com/apache/flink-connector-cassandra/releases/tag/v3.2.0-rc1
[6] https://github.com/apache/flink-web/pull/737
[7]
https://github.com/apache/flink-connector-cassandra/actions/runs/8784310241

Re: [ANNOUNCE] Apache Flink CDC 3.1.0 released

[Muhammet Orazov]

Amazing, congrats!

Thanks for your efforts!

Best,
Muhammet

On 2024-05-17 09:32, Qingsheng Ren wrote:

The Apache Flink community is very happy to announce the release of
Apache Flink CDC 3.1.0.

Apache Flink CDC is a distributed data integration tool for real time
data and batch data, bringing the simplicity and elegance of data
integration via YAML to describe the data movement and transformation
in a data pipeline.

Please check out the release blog post for an overview of the release:
https://flink.apache.org/2024/05/17/apache-flink-cdc-3.1.0-release-announcement/

The release is available for download at:
https://flink.apache.org/downloads.html

Maven artifacts for Flink CDC can be found at:
https://search.maven.org/search?q=g:org.apache.flink%20cdc

The full release notes are available in Jira:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12354387

We would like to thank all contributors of the Apache Flink community
who made this release possible!

Regards,
Qingsheng Ren

Re: [DISCUSS] FLIP-453: Promote Unified Sink API V2 to Public and Deprecate SinkFunction

[Muhammet Orazov]

Thanks Ahmed!

I have reviewed your PubSub and RabbitMQ PRs, please have a look.

Best,
Muhammet

Re: [VOTE] FLIP-453: Promote Unified Sink API V2 to Public and Deprecate SinkFunction

[Muhammet Orazov]

Thanks Martijn driving this! +1 (non-binding)

Best,
Muhammet

On 2024-05-14 06:43, Martijn Visser wrote:

Hi everyone,

With no more discussions being open in the thread [1] I would like to 
start

a vote on FLIP-453: Promote Unified Sink API V2 to Public and Deprecate
SinkFunction [2]

The vote will be open for at least 72 hours unless there is an 
objection or

insufficient votes.

Best regards,

Martijn

[1] https://lists.apache.org/thread/hod6bg421bzwhbfv60lwsck7r81dvo59
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-453%3A+Promote+Unified+Sink+API+V2+to+Public+and+Deprecate+SinkFunction

Re: [DISCUSS] FLIP-XXX: Improve JDBC connector extensibility for Table API

[Muhammet Orazov]

Hey Lorenzo,

Thanks for driving this FLIP! +1

It will improve the user experience of using JDBC based
connectors and help developers to build with different drivers.

Best,
Muhammet

On 2024-05-13 10:20, lorenzo.affe...@ververica.com.INVALID wrote:

Hello dev!

I want to share a draft of my FLIP to refactor the JDBC connector to 
improve its extensibility [1].
The goal is to allow implementers to write new connectors on top of the 
JDBC one for Table API with clean and maintainable code.


Any feedback from the community is more and welcome.

[1] 
https://docs.google.com/document/d/1kl_AikMlqPUI-LNiPBraAFVZDRg1LF4bn6uiNtR4dlY/edit?usp=sharing

Re: [VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

[Muhammet Orazov]

Thanks Ahmed, +1 (non-binding)

Best,
Muhammet

On 2024-05-13 09:50, Ahmed Hamdy wrote:

Hi all,

Thanks for the feedback on the discussion thread[1], I would like to 
start

a vote on FLIP-451[2]: Introduce timeout configuration to AsyncSink

The vote will be open for at least 72 hours unless there is an 
objection or

insufficient votes.

1-https://lists.apache.org/thread/ft7wcw7kyftvww25n5fm4l925tlgdfg0
2-
https://cwiki.apache.org/confluence/display/FLINK/FLIP-451%3A+Introduce+timeout+configuration+to+AsyncSink+API
Best Regards
Ahmed Hamdy

Re: [DISCUSS] FLIP-451: Refactor Async sink API

[Muhammet Orazov]

Great, thanks for clarifying!

Best,
Muhammet


On 2024-05-06 13:40, Ahmed Hamdy wrote:

Hi Muhammet,
Thanks for the feedback.


Could you please add more here why it is harder? Would the
`completeExceptionally`
method be related to it? Maybe you can add usage example for it also.



this is mainly due to the current implementation of fatal exception
failures which depends on base `getFatalExceptionConsumer` method that 
is
decoupled from the actual called method `submitRequestEntries`, Since 
this

is now not the primary concern of the FLIP, I have removed it from the
motivation so that the scope is defined around introducing the timeout
configuration.

Should we add a list of possible connectors that this FLIP would 
improve?


Good call, I have added under migration plan.

Best Regards
Ahmed Hamdy


On Mon, 6 May 2024 at 08:49, Muhammet Orazov  
wrote:



Hey Ahmed,

Thanks for the FLIP! +1 (non-binding)

> Additionally the current interface for passing fatal exceptions and
> retrying records relies on java consumers which makes it harder to
> understand.

Could you please add more here why it is harder? Would the
`completeExceptionally`
method be related to it? Maybe you can add usage example for it also.

> we should proceed by adding support in all supporting connector repos.

Should we add list of possible connectors that this FLIP would 
improve?


Best,
Muhammet


On 2024-04-29 14:08, Ahmed Hamdy wrote:
> Hi all,
> I would like to start a discussion on FLIP-451[1]
> The proposal comes on encountering a couple of issues while working
> with
> implementers for Async Sink.
> The FLIP mainly proposes a new API similar to AsyncFunction and
> ResultFuture as well as introducing timeout handling for AsyncSink
> requests.
> The FLIP targets 1.20 with backward compatible changes and we should
> proceed by adding support in all supporting connector repos.
>
> 1-
>
https://cwiki.apache.org/confluence/display/FLINK/FLIP-451%3A+Refactor+Async+Sink+API
> Best Regards
> Ahmed Hamdy

Re: [VOTE] Apache Flink CDC Release 3.1.0, release candidate #2

[Muhammet Orazov]

Hey Qingsheng,

Thanks, I have run the following steps:

- Checked sha512sum hash
- Checked GPG signature
- Built the source with JDK 11 & 8
- Checked that src doesn't contain binary files

GitHub web PR is as before with suggestions requested.

Best,
Muhammet

On 2024-05-10 09:59, Qingsheng Ren wrote:

Hi everyone,

Please review and vote on the release candidate #2 for the version 
3.1.0 of

Apache Flink CDC, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

**Release Overview**

As an overview, the release consists of the following:
a) Flink CDC source release to be deployed to dist.apache.org
b) Maven artifacts to be deployed to the Maven Central Repository

**Staging Areas to Review**

The staging areas containing the above mentioned artifacts are as 
follows,

for your review:
* All artifacts for a) can be found in the corresponding dev repository 
at

dist.apache.org [1], which are signed with the key with fingerprint
A1BD477F79D036D2C30CA7DBCA8AEEC2F6EB040B [2]
* All artifacts for b) can be found at the Apache Nexus Repository [3]

Other links for your review:
* JIRA release notes [4]
* Source code tag "release-3.1.0-rc2" with commit hash
f3ce608903014d410eecc8ad62c7a9d9a27a6a19 [5]
* PR for release announcement blog post of Flink CDC 3.1.0 in flink-web 
[6]


**Vote Duration**

The voting time will run for at least 72 hours, adopted by majority
approval with at least 3 PMC affirmative votes.

Thanks,
Qingsheng Ren

[1] https://dist.apache.org/repos/dist/dev/flink/flink-cdc-3.1.0-rc2/
[2] https://dist.apache.org/repos/dist/release/flink/KEYS
[3] 
https://repository.apache.org/content/repositories/orgapacheflink-1732

[4]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12354387
[5] https://github.com/apache/flink-cdc/releases/tag/release-3.1.0-rc2
[6] https://github.com/apache/flink-web/pull/739

Re: [VOTE] FLIP-449: Reorganization of flink-connector-jdbc

[Muhammet Orazov]

Thanks João for your efforts and driving this!

+1 (non-binding)

Best,
Muhammet

On 2024-05-09 12:01, Joao Boto wrote:

Hi everyone,

Thanks for all the feedback, I'd like to start a vote on the FLIP-449:
Reorganization of flink-connector-jdbc [1].
The discussion thread is here [2].

The vote will be open for at least 72 hours unless there is an 
objection or

insufficient votes.

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-449%3A+Reorganization+of+flink-connector-jdbc
[2] https://lists.apache.org/thread/jc1yvvo35xwqzlxl5mj77qw3hq6f5sgr

Best
Joao Boto

Re: [DISCUSS] Flink CDC 3.2 Release Planning

[Muhammet Orazov]

Hey Qingsheng,

Thanks for your efforts, agreed!

I would be happy to help.

Best,
Muhammet

On 2024-05-08 06:21, Qingsheng Ren wrote:

Hi devs,

As we are in the midst of the release voting process for Flink CDC 
3.1.0, I
think it's a good time to kick off the upcoming Flink CDC 3.2 release 
cycle.


In this release cycle I would like to focus on the stability of Flink 
CDC,

especially for the newly introduced YAML-based data integration
framework. To ensure we can iterate and improve swiftly, I propose to 
make
3.2 a relatively short release cycle, targeting a feature freeze by May 
24,

2024.

For developers that are interested in participating and contributing 
new

features in this release cycle, please feel free to list your planning
features in the wiki page [1].

I'm happy to volunteer as a release manager and of course open to work
together with someone on this.

What do you think?

Best,
Qingsheng

[1] 
https://cwiki.apache.org/confluence/display/FLINK/Flink+CDC+3.2+Release

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

[Muhammet Orazov]

Thanks João for pointing it out. I didn't know about the PR, I am going 
to check it.


Best,
Muhammet


On 2024-05-06 14:45, João Boto wrote:

Hi Muhammet,

Have you had a chance to review the recently merged pull request [1]?
We've introduced a new feature allowing users to include ad hoc 
configurations in the 'JdbcConnectionOptions' class.

```
 new JdbcConnectionOptions.JdbcConnectionOptionsBuilder()
.withUrl(FakeDBUtils.TEST_DB_URL)
.withProperty("keyA", "valueA")
.build();
```

This provides flexibility by enabling users to specify additional 
configuration parameters dynamically.


[1] https://github.com/apache/flink-connector-jdbc/pull/115/files

Best

On 2024/05/06 07:34:06 Muhammet Orazov wrote:

Morning João,

Recently we had a case where the JDBC drivers authentication was
different than username authentication. For it to work, 
certain

hacks required, there interface would have been helpful.

But I agree maybe the interface module separation is not required at 
the

moment.

Thanks for your efforts!

Best,
Muhammet


On 2024-05-03 12:25, João Boto wrote:
> Hi Muhammet,
>
> While I generally agree, given our current usage, I'm struggling to
> discern any clear advantage. We already have abstract implementations
> that cover all necessary interfaces and offer essential functionality,
> complemented by a robust set of reusable tests to streamline
> implementation.
>
> With this established infrastructure in place, coupled with the added
> import overhead of introducing another module, I find it difficult to
> identify any distinct benefits at this point.
>
> Best
>
> On 2024/04/26 02:18:52 Muhammet Orazov wrote:
>> Hey João,
>>
>> Thanks for FLIP proposal!
>>
>> Since proposal is to introduce modules, would it make sense
>> to have another module for APIs (flink-jdbc-connector-api)?
>>
>> For this I would suggest to move all public interfaces (e.g,
>> JdbcRowConverter, JdbcConnectionProvider). And even convert
>> some classes into interface with their default implementations,
>> for example, JdbcSink, JdbcConnectionOptions.
>>
>> This way users would have clear interfaces to build their own
>> JDBC based Flink connectors.
>>
>> Here I am not suggesting to introduce new interfaces, only
>> suggest also to separate the API from the core implementation.
>>
>> What do you think?
>>
>> Best,
>> Muhammet
>>
>>
>> On 2024-04-25 08:54, Joao Boto wrote:
>> > Hi all,
>> >
>> > I'd like to start a discussion on FLIP-449: Reorganization of
>> > flink-connector-jdbc [1].
>> > As Flink continues to evolve, we've noticed an increasing level of
>> > complexity within the JDBC connector.
>> > The proposed solution is to address this complexity by separating the
>> > core
>> > functionality from individual database components, thereby streamlining
>> > the
>> > structure into distinct modules.
>> >
>> > Looking forward to your feedback and suggestions, thanks.
>> > Best regards,
>> > Joao Boto
>> >
>> > [1]
>> > 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-449%3A+Reorganization+of+flink-connector-jdbc
>>

Re: [VOTE] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

[Muhammet Orazov]

Thanks for your efforts! +1 (non-binding)

Best,
Muhammet

On 2024-05-06 02:53, yue ma wrote:

Hi everyone,

Thanks for all the feedback, I'd like to start a vote on the FLIP-447:
Upgrade FRocksDB from 6.20.3 to 8.10.0 [1]. The discussion thread is 
here

[2].

The vote will be open for at least 72 hours unless there is an 
objection or

insufficient votes.

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-447%3A+Upgrade+FRocksDB+from+6.20.3++to+8.10.0
[2] https://lists.apache.org/thread/lrxjfpjjwlq4sjzm1oolx58n1n8r48hw

Re: [DISCUSS] FLIP-451: Refactor Async sink API

[Muhammet Orazov]

Hey Ahmed,

Thanks for the FLIP! +1 (non-binding)


Additionally the current interface for passing fatal exceptions and
retrying records relies on java consumers which makes it harder to 
understand.


Could you please add more here why it is harder? Would the 
`completeExceptionally`

method be related to it? Maybe you can add usage example for it also.


we should proceed by adding support in all supporting connector repos.


Should we add list of possible connectors that this FLIP would improve?

Best,
Muhammet


On 2024-04-29 14:08, Ahmed Hamdy wrote:

Hi all,
I would like to start a discussion on FLIP-451[1]
The proposal comes on encountering a couple of issues while working 
with

implementers for Async Sink.
The FLIP mainly proposes a new API similar to AsyncFunction and
ResultFuture as well as introducing timeout handling for AsyncSink 
requests.

The FLIP targets 1.20 with backward compatible changes and we should
proceed by adding support in all supporting connector repos.

1-
https://cwiki.apache.org/confluence/display/FLINK/FLIP-451%3A+Refactor+Async+Sink+API
Best Regards
Ahmed Hamdy

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

[Muhammet Orazov]

Morning João,

Recently we had a case where the JDBC drivers authentication was 
different than username authentication. For it to work, certain 
hacks required, there interface would have been helpful.


But I agree maybe the interface module separation is not required at the 
moment.


Thanks for your efforts!

Best,
Muhammet


On 2024-05-03 12:25, João Boto wrote:

Hi Muhammet,

While I generally agree, given our current usage, I'm struggling to 
discern any clear advantage. We already have abstract implementations 
that cover all necessary interfaces and offer essential functionality, 
complemented by a robust set of reusable tests to streamline 
implementation.


With this established infrastructure in place, coupled with the added 
import overhead of introducing another module, I find it difficult to 
identify any distinct benefits at this point.


Best

On 2024/04/26 02:18:52 Muhammet Orazov wrote:

Hey João,

Thanks for FLIP proposal!

Since proposal is to introduce modules, would it make sense
to have another module for APIs (flink-jdbc-connector-api)?

For this I would suggest to move all public interfaces (e.g,
JdbcRowConverter, JdbcConnectionProvider). And even convert
some classes into interface with their default implementations,
for example, JdbcSink, JdbcConnectionOptions.

This way users would have clear interfaces to build their own
JDBC based Flink connectors.

Here I am not suggesting to introduce new interfaces, only
suggest also to separate the API from the core implementation.

What do you think?

Best,
Muhammet


On 2024-04-25 08:54, Joao Boto wrote:
> Hi all,
>
> I'd like to start a discussion on FLIP-449: Reorganization of
> flink-connector-jdbc [1].
> As Flink continues to evolve, we've noticed an increasing level of
> complexity within the JDBC connector.
> The proposed solution is to address this complexity by separating the
> core
> functionality from individual database components, thereby streamlining
> the
> structure into distinct modules.
>
> Looking forward to your feedback and suggestions, thanks.
> Best regards,
> Joao Boto
>
> [1]
> 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-449%3A+Reorganization+of+flink-connector-jdbc

Re: [VOTE] Apache Flink CDC Release 3.1.0, release candidate #1

[Muhammet Orazov]

Hey Qingsheng,

Thanks a lot! +1 (non-binding)

- Checked sha512sum hash
- Checked GPG signature
- Reviewed release notes
- Reviewed GitHub web pr (added minor suggestions)
- Built the source with JDK 11 & 8
- Checked that src doesn't contain binary files

Best,
Muhammet

On 2024-04-30 05:11, Qingsheng Ren wrote:

Hi everyone,

Please review and vote on the release candidate #1 for the version 
3.1.0 of

Apache Flink CDC, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

**Release Overview**

As an overview, the release consists of the following:
a) Flink CDC source release to be deployed to dist.apache.org
b) Maven artifacts to be deployed to the Maven Central Repository

**Staging Areas to Review**

The staging areas containing the above mentioned artifacts are as 
follows,

for your review:
* All artifacts for a) can be found in the corresponding dev repository 
at

dist.apache.org [1], which are signed with the key with fingerprint
A1BD477F79D036D2C30CA7DBCA8AEEC2F6EB040B [2]
* All artifacts for b) can be found at the Apache Nexus Repository [3]

Other links for your review:
* JIRA release notes [4]
* Source code tag "release-3.1.0-rc1" with commit hash
63b42cb937d481f558209ab3c8547959cf039643 [5]
* PR for release announcement blog post of Flink CDC 3.1.0 in flink-web 
[6]


**Vote Duration**

The voting time will run for at least 72 hours, adopted by majority
approval with at least 3 PMC affirmative votes.

Thanks,
Qingsheng Ren

[1] https://dist.apache.org/repos/dist/dev/flink/flink-cdc-3.1.0-rc1/
[2] https://dist.apache.org/repos/dist/release/flink/KEYS
[3] 
https://repository.apache.org/content/repositories/orgapacheflink-1731

[4]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12354387
[5] https://github.com/apache/flink-cdc/releases/tag/release-3.1.0-rc1
[6] https://github.com/apache/flink-web/pull/739

Re: [DISCUSS] FLIP-444: Native file copy support

[Muhammet Orazov]

Hey Piotr,

Thanks for the proposal! It would be great improvement!

Some questions from my side:


In order to configure s5cmd Flink’s user would need
to specify path to the s5cmd binary.


Could you please also add the configuration property
for this? An example showing how users would set this
parameter would be helpful.

Would this affect any filesystem connectors that use
FileSystem[1][2] dependencies?

Best,
Muhammet

[1]: 
https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/filesystems/s3/
[2]: 
https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/filesystem/


On 2024-04-30 13:15, Piotr Nowojski wrote:

Hi all!

I would like to put under discussion:

FLIP-444: Native file copy support
https://cwiki.apache.org/confluence/x/rAn9EQ

This proposal aims to speed up Flink recovery times, by speeding up 
state

download times. However in the future, the same mechanism could be also
used to speed up state uploading (checkpointing/savepointing).

I'm curious to hear your thoughts.

Best,
Piotrek

Re: [DISCUSS] FLIP-453: Promote Unified Sink API V2 to Public and Deprecate SinkFunction

[Muhammet Orazov]

Got it, thanks!

On 2024-05-02 06:53, Martijn Visser wrote:

Hi Muhammet,

Thanks for joining the discussion! The changes in this FLIP would be
targeted for Flink 1.19, since it's only a matter of changing the
annotation.

Best regards,

Martijn

On Thu, May 2, 2024 at 7:26 AM Muhammet Orazov 
wrote:


Hello Martijn,

Thanks for the FLIP and detailed history of changes, +1.

Would FLIP changes target for 2.0? I think it would be good
to have clear APIs on 2.0 release.

Best,
Muhammet

On 2024-05-01 15:30, Martijn Visser wrote:
> Hi everyone,
>
> I would like to start a discussion on FLIP-453: Promote Unified Sink
> API V2
> to Public and Deprecate SinkFunction
> https://cwiki.apache.org/confluence/x/rIobEg
>
> This FLIP proposes to promote the Unified Sink API V2 from
> PublicEvolving
> to Public and to mark the SinkFunction as Deprecated.
>
> I'm looking forward to your thoughts.
>
> Best regards,
>
> Martijn

Re: [Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding

[Muhammet Orazov]

Hey Alan,

Thanks for the proposal, +1!

The `isDeterministic()`[1] function is mentioned in the documentation,
I would suggest to add maybe a section for `supportsConstantFolding()`,
with short description and examples use cases (similar to the
motivation of the FLIP) where this could be useful in UDFs.

Best,
Muhammet

[1]: 
https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/table/functions/udfs/#evaluation-methods



On 2024-04-29 22:57, Alan Sheinberg wrote:
I'd like to start a discussion of FLIP-452: Allow Skipping Invocation 
of

Function Calls While Constant-folding [1]

This feature proposes adding a new
method FunctionDefinition.allowConstantFolding() as part of the Flink
Table/SQL API.  This would be used to determine whether an expression
containing this function should have constant-folding logic run on it,
invoking the function at planning time.

The current behavior of always doing constant-folding on function calls 
is

problematic for UDFs which invoke RPCs or have other side effects in
external systems.  In these cases, you either don’t want these actions 
to

occur during planning time, or it may be important to happen on a per
result row basis.

Note that this is a bit different than
FunctionDefinition.isDeterministic(), and can exist along-side it.

Looking forward to your feedback and suggestions.

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-452%3A+Allow+Skipping+Invocation+of+Function+Calls+While+Constant-folding


Thanks,
Alan

Re: [DISCUSS] FLIP-453: Promote Unified Sink API V2 to Public and Deprecate SinkFunction

[Muhammet Orazov]

Hello Martijn,

Thanks for the FLIP and detailed history of changes, +1.

Would FLIP changes target for 2.0? I think it would be good
to have clear APIs on 2.0 release.

Best,
Muhammet

On 2024-05-01 15:30, Martijn Visser wrote:

Hi everyone,

I would like to start a discussion on FLIP-453: Promote Unified Sink 
API V2

to Public and Deprecate SinkFunction
https://cwiki.apache.org/confluence/x/rIobEg

This FLIP proposes to promote the Unified Sink API V2 from 
PublicEvolving

to Public and to mark the SinkFunction as Deprecated.

I'm looking forward to your thoughts.

Best regards,

Martijn

Re: [VOTE] FLIP-436: Introduce Catalog-related Syntax

[Muhammet Orazov]

Hey Yubin, +1 (non-binding)

Looks good, thanks!

Best,
Muhammet

On 2024-04-24 08:56, Yubin Li wrote:

Hi everyone,

During the implementation of the "describe catalog" syntax, it was
found that the original output style needed to be improved.
```
desc catalog extended cat2;
+--+-+
| catalog_description_item |
catalog_description_value |
+--+-+
| Name |
 cat2 |
| Type |
generic_in_memory |
|  Comment |
  |
|   Properties | ('default-database','db'),
('type','generic_in_memory') |
+--+-+
4 rows in set
```
After offline discussions with Jane Chan and Jark Wu, we suggest
improving it to the following form:
```
desc catalog extended cat2;
+-+---+
|   info name |info value |
+-+---+
|name |  cat2 |
|type | generic_in_memory |
| comment |   |
| option:default-database |db |
+-+---+
4 rows in set
```

For the following reasons:
1. The title should be consistent with engines such as Databricks for
easy understanding, and it should also be consistent with Flink's own
naming style. Therefore, the title adopts "info name", "info value",
and the key name should be unified in lowercase, so "Name" is replaced
by "name".
Note: Databricks output style [1] as follows:
```

DESCRIBE CATALOG main;

 info_name info_value
   
 Catalog Name  main
  Comment   Main catalog (auto-created)
Owner metastore-admin-users
 Catalog Type   Regular
```
2. There may be many attributes of the catalog, and it is very poor in
readability when displayed in one line. It should be expanded into
multiple lines, and the key name is prefixed with "option:" to
identify that this is an attribute row. And since `type` is an
important information of the catalog, even if `extended` is not
specified, it should also be displayed, and correspondingly,
"option:type" should be removed to avoid redundancy.

WDYT? Looking forward to your reply!

[1] 
https://learn.microsoft.com/zh-tw/azure/databricks/sql/language-manual/sql-ref-syntax-aux-describe-catalog


Best,
Yubin

On Wed, Mar 20, 2024 at 2:15 PM Benchao Li  
wrote:


+1 (binding)

gongzhongqiang  于2024年3月20日周三 11:40写道：
>
> +1 (non-binding)
>
> Best,
> Zhongqiang Gong
>
> Yubin Li  于2024年3月19日周二 18:03写道：
>
> > Hi everyone,
> >
> > Thanks for all the feedback, I'd like to start a vote on the FLIP-436:
> > Introduce Catalog-related Syntax [1]. The discussion thread is here
> > [2].
> >
> > The vote will be open for at least 72 hours unless there is an
> > objection or insufficient votes.
> >
> > [1]
> > 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-436%3A+Introduce+Catalog-related+Syntax
> > [2] https://lists.apache.org/thread/10k1bjb4sngyjwhmfqfky28lyoo7sv0z
> >
> > Best regards,
> > Yubin
> >



--

Best,
Benchao Li

Re: [VOTE] FLIP-445: Support dynamic parallelism inference for HiveSource

[Muhammet Orazov]

Hey Xia,

+1 (non-binding)

Thanks and best,
Muhammet

On 2024-04-26 02:21, Xia Sun wrote:

Hi everyone,

I'd like to start a vote on FLIP-445: Support dynamic parallelism 
inference

for HiveSource[1] which has been discussed in this thread [2].

The vote will be open for at least 72 hours unless there is an 
objection or

not enough votes.


[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-445%3A+Support+dynamic+parallelism+inference+for+HiveSource
[2] https://lists.apache.org/thread/4k1qx6lodhbkknkhjyl0lq9bx8fcpjvn


Best,
Xia

Re: [DISCUSSION] FLIP-449: Reorganization of flink-connector-jdbc

[Muhammet Orazov]

Hey João,

Thanks for FLIP proposal!

Since proposal is to introduce modules, would it make sense
to have another module for APIs (flink-jdbc-connector-api)?

For this I would suggest to move all public interfaces (e.g,
JdbcRowConverter, JdbcConnectionProvider). And even convert
some classes into interface with their default implementations,
for example, JdbcSink, JdbcConnectionOptions.

This way users would have clear interfaces to build their own
JDBC based Flink connectors.

Here I am not suggesting to introduce new interfaces, only
suggest also to separate the API from the core implementation.

What do you think?

Best,
Muhammet


On 2024-04-25 08:54, Joao Boto wrote:

Hi all,

I'd like to start a discussion on FLIP-449: Reorganization of
flink-connector-jdbc [1].
As Flink continues to evolve, we've noticed an increasing level of
complexity within the JDBC connector.
The proposed solution is to address this complexity by separating the 
core
functionality from individual database components, thereby streamlining 
the

structure into distinct modules.

Looking forward to your feedback and suggestions, thanks.
Best regards,
Joao Boto

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-449%3A+Reorganization+of+flink-connector-jdbc

Re: [SUMMARY] Flink 1.20 Release Sync 04/23/2024

[Muhammet Orazov]

Hey Rui,

Thanks for the summary!

Could you please write what time are you meeting on 5th May?

Thanks and best,
Muhammet

On 2024-04-23 09:13, Rui Fan wrote:

Dear devs,

Today is the second meeting for Flink 1.20 release cycle. I'd like to 
share

the information synced in the meeting.

- Features:
  There're 8 weeks until the feature freeze date, so far we've had 6
  flips/features and the status looks good. It is encouraged to 
continuously

  updating the 1.20 wiki page[1] for contributors.

- Blockers:
  - [Closed] Change on getTransitivePredecessors breaks connectors
FLINK-35009
  - [Doing] 2 tests fail FLINK-35041, FLINK-34997
  - [Doing] 2 benchmarks performance regression FLINK-35040, 
FLINK-35215


- Sync meeting[2]:
 The next meeting is 05/07/2024, please feel free to join us.

Lastly, we encourage attendees to fill out the topics to be discussed 
at

the bottom of 1.20 wiki page[1] a day in advance, to make it easier for
everyone to understand the background of the topics.

[1] https://cwiki.apache.org/confluence/display/FLINK/1.20+Release
[2] https://meet.google.com/mtj-huez-apu

Best,

Weijie, Ufuk, Robert and Rui

Re: [VOTE] Release flink-connector-cassandra v3.2.0, release candidate #1

[Muhammet Orazov]

Thanks! +1 (non-binding)

- Checked 512 hash
- Checked gpg signature
- Reviewed web pr & release notes
- Built the source with JDK 11 & 8

Best,
Muhammet

On 2024-04-22 13:04, Danny Cranmer wrote:

Hi everyone,

Please review and vote on release candidate #1 for
flink-connector-cassandra v3.2.0, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

This release supports Flink 1.18 and 1.19.

The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org 
[2],

which are signed with the key with fingerprint 125FD8DB [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag v3.2.0-rc1 [5],
* website pull request listing the new release [6].
* CI build of the tag [7].

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Danny

[1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353148
[2]
https://dist.apache.org/repos/dist/dev/flink/flink-connector-cassandra-3.2.0-rc1
[3] https://dist.apache.org/repos/dist/release/flink/KEYS
[4] 
https://repository.apache.org/content/repositories/orgapacheflink-1722

[5]
https://github.com/apache/flink-connector-cassandra/releases/tag/v3.2.0-rc1
[6] https://github.com/apache/flink-web/pull/737
[7]
https://github.com/apache/flink-connector-cassandra/actions/runs/8784310241

Re: [VOTE] Release flink-connector-kafka v3.2.0, release candidate #1

[Muhammet Orazov]

Thanks Danny, +1 (non-binding)

- Checked 512 hash
- Checked gpg signature
- Reviewed pr
- Built the source with JDK 11 & 8

Best,
Muhammet

On 2024-04-22 13:55, Danny Cranmer wrote:

Hi everyone,

Please review and vote on release candidate #1 for 
flink-connector-kafka

v3.2.0, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

This release supports Flink 1.18 and 1.19.

The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org 
[2],

which are signed with the key with fingerprint 125FD8DB [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag v3.2.0-rc1 [5],
* website pull request listing the new release [6].
* CI build of the tag [7].

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Danny

[1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12354209
[2]
https://dist.apache.org/repos/dist/dev/flink/flink-connector-kafka-3.2.0-rc1
[3] https://dist.apache.org/repos/dist/release/flink/KEYS
[4] 
https://repository.apache.org/content/repositories/orgapacheflink-1723
[5] 
https://github.com/apache/flink-connector-kafka/releases/tag/v3.2.0-rc1

[6] https://github.com/apache/flink-web/pull/738
[7] https://github.com/apache/flink-connector-kafka

Re: [VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

[Muhammet Orazov]

Thanks Xuannan! +1 (non-binding)

Best,
Muhammet

On 2024-04-17 05:00, Xuannan Su wrote:

Hi everyone,

Thanks for all the feedback about the FLIP-442: General Improvement to
Configuration for Flink 2.0 [1] [2].

I'd like to start a vote for it. The vote will be open for at least 72
hours(excluding weekends,until APR 22, 12:00AM GMT) unless there is an
objection or an insufficient number of votes.

[1] 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-442%3A+General+Improvement+to+Configuration+for+Flink+2.0

[2] https://lists.apache.org/thread/15k0stwyoknhxvd643ctwjw3fd17pqwk


Best regards,
Xuannan

Re: [DISCUSS] FLIP-445: Support dynamic parallelism inference for HiveSource

[Muhammet Orazov]

Hello Xia,

Thanks for the FLIP!

Since we are introducing the mode as a configuration option,
could it make sense to have `InferMode.NONE` option also?
The `NONE` option would disable the inference.

This way we deprecate the `table.exec.hive.infer-source-parallelism`
and no additional `table.exec.hive.infer-source-parallelism.enabled`
option is required.

What do you think?

Best,
Muhammet

On 2024-04-16 07:07, Xia Sun wrote:

Hi everyone,
I would like to start a discussion on FLIP-445: Support dynamic 
parallelism

inference for HiveSource[1].

FLIP-379[2] has introduced dynamic source parallelism inference for 
batch
jobs, which can utilize runtime information to more accurately decide 
the
source parallelism. As a follow-up task, we plan to implement the 
dynamic
parallelism inference interface for HiveSource, and also switch the 
default

static parallelism inference to dynamic parallelism inference.

Looking forward to your feedback and suggestions, thanks.

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-445%3A+Support+dynamic+parallelism+inference+for+HiveSource
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-379%3A+Dynamic+source+parallelism+inference+for+batch+jobs

Best regards,
Xia

Re: [DISCUSS] Connector releases for Flink 1.19

[Muhammet Orazov]

Thanks Sergey and Danny for clarifying, indeed it
requires committer to go through the process.

Anyway, please let me know if I can be any help.

Best,
Muhammet


On 2024-04-16 11:19, Danny Cranmer wrote:

Hello,

I have opened the VOTE thread for the AWS connectors release [1].

If I'm not mistaking (please correct me if I'm wrong) this request is 
not

about version update it is about new releases for connectors

Yes, correct. If there are any other code changes required then help
would be appreciated.


Are you going to create an umbrella issue for it?


We do not usually create JIRA issues for releases. That being said it
sounds like a good idea to have one place to track the status of the
connector releases and pre-requisite code changes.


I would like to work on this task, thanks for initiating it!


The actual release needs to be performed by a committer. However, help
getting the connectors building against Flink 1.19 and testing the RC 
is

appreciated.

Thanks,
Danny

[1] https://lists.apache.org/thread/0nw9smt23crx4gwkf6p1dd4jwvp1g5s0



On Tue, Apr 16, 2024 at 6:34 AM Sergey Nuyanzin  
wrote:



Thanks for volunteering Muhammet!
And thanks Danny for starting the activity.

If I'm not mistaking (please correct me if I'm wrong)

this request is not about version update it is about new releases for
connectors
btw for jdbc connector support of 1.19 and 1.20-SNAPSHOT is already 
done


I would volunteer for Opensearch connector since currently I'm working 
on

support of Opensearch v2
and I think it would make sense to have a release after it is done

On Tue, Apr 16, 2024 at 4:29 AM Muhammet Orazov
 wrote:


Hello Danny,

I would like to work on this task, thanks for initiating it!

I could update the versions on JDBC and Pulsar connectors.

Are you going to create an umbrella issue for it?

Best,
Muhammet

On 2024-04-15 13:44, Danny Cranmer wrote:
> Hello all,
>
> Flink 1.19 was released on 2024-03-18 [1] and the connectors have not
> yet
> caught up. I propose we start releasing the connectors with support for
> Flink 1.19 as per the connector support guidelines [2].
>
> I will make a start on flink-connector-aws, then pickup others in the
> coming days. Please respond to the thread if you are/want to work on a
> particular connector to avoid duplicate work.
>
> Thanks,
> Danny
>
> [1]
>
https://flink.apache.org/2024/03/18/announcing-the-release-of-apache-flink-1.19/
> [2]
>
https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development#ExternalizedConnectordevelopment-Flinkcompatibility
> [3] https://github.com/apache/flink-connector-aws




--
Best regards,
Sergey

Re: [DISCUSS] Connector releases for Flink 1.19

[Muhammet Orazov]

Hello Danny,

I would like to work on this task, thanks for initiating it!

I could update the versions on JDBC and Pulsar connectors.

Are you going to create an umbrella issue for it?

Best,
Muhammet

On 2024-04-15 13:44, Danny Cranmer wrote:

Hello all,

Flink 1.19 was released on 2024-03-18 [1] and the connectors have not 
yet

caught up. I propose we start releasing the connectors with support for
Flink 1.19 as per the connector support guidelines [2].

I will make a start on flink-connector-aws, then pickup others in the
coming days. Please respond to the thread if you are/want to work on a
particular connector to avoid duplicate work.

Thanks,
Danny

[1]
https://flink.apache.org/2024/03/18/announcing-the-release-of-apache-flink-1.19/
[2]
https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development#ExternalizedConnectordevelopment-Flinkcompatibility
[3] https://github.com/apache/flink-connector-aws

Re: [Vote] FLIP-438: Amazon SQS Sink Connector

[Muhammet Orazov]

+1 (non-binding)

Thanks Priya for the FLIP and driving it!

Best,
Muhammet

On 2024-04-12 21:56, Dhingra, Priya wrote:

Hi devs,



Thank you to everyone for the feedback on FLIP-438: Amazon SQS Sink 
Connector




I would like to start a vote for it. The vote will be open for at least 
72


hours unless there is an objection or not enough votes.



https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector

Regards
Priya

Re: [DISCUSS] FLIP-438: Amazon SQS Sink Connector

[Muhammet Orazov]

Hey Priya,

Having local dockerized integration testing would help devs to run the 
integration tests when they don't have AWS service credentials.


You could have a look to the DynamoDB [1] or Kinesis [2] connector 
integration tests. DynamoDBContainer extends interface from 
Testcontainers [3].


Best,
Muhammet

[1]: 
https://github.com/apache/flink-connector-aws/blob/main/flink-connector-aws/flink-connector-dynamodb/src/test/java/org/apache/flink/connector/dynamodb/sink/DynamoDbSinkITCase.java#L74-L78
[2]: 
https://github.com/apache/flink-connector-aws/blob/main/flink-connector-aws/flink-connector-kinesis/src/test/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisITCase.java#L87-L89
[3]: 
https://github.com/apache/flink-connector-aws/blob/main/flink-connector-aws/flink-connector-dynamodb/src/test/java/org/apache/flink/connector/dynamodb/testutils/DynamoDbContainer.java



On 2024-04-15 20:31, Dhingra, Priya wrote:

Thanks Danny/ Muhammet for the feedback. Updated the FLIP accordingly.

I am not familiar with docker based integration testing but if that is 
what we have used for other AWS services, I can follow the same here. 
If you can share any existing sample code link that would be very 
helpful.


Thanks
Priya

On 4/15/24, 5:22 AM, "Danny Cranmer" <mailto:dannycran...@apache.org>> wrote:



CAUTION: This email originated from outside of the organization. Do not 
click links or open attachments unless you can confirm the sender and 
know the content is safe.







Thanks for the FLIP Priya. I have seen many custom implementations of 
SQS

sinks, so having an Apache supported connector will be great.


I agree on the points about implementation detail in the FLIP. There is
still too much code really, we will review that in the PR. The FLIP is
mainly to agree we should add this connector, we can leave the
implementation details for later.


nit: Wrt to versioning, it is actually Flink 1.18 and 1.19 now, since 
1.17
is out of support. But since we are using the flink-connector-aws repo 
we

will use the supported version there.



Would it make sense to use here docker based integration testing using

Testcontainer


+1, we typically use localstack for the Integration tests and hit AWS 
in

the e2e test packages. Having coverage of both will be great.


Thanks,
Danny


On Mon, Apr 15, 2024 at 3:23 AM Muhammet Orazov
mailto:mor+fl...@morazow.com.inva>lid> 
wrote:




Hey Priya,

Thanks for the FLIP and driving it!

One question from my side on test plan:

> End to end integration tests that hit the real Amazon SQS service.
> These tests will be enabled when credentials are defined.

Would it make sense to use here docker based integration testing using
Testcontainer[1], for example, with Localstack[2]? Seems like the SQS
service is provided on the free tier.

[1]: https://java.testcontainers.org/modules/localstack/ 
<https://java.testcontainers.org/modules/localstack/>
[2]: https://docs.localstack.cloud/user-guide/aws/sqs/ 
<https://docs.localstack.cloud/user-guide/aws/sqs/>


On 2024-04-05 18:16, Dhingra, Priya wrote:
> Hi Dev,
>
> I would like to start a discussion about FLIP-438: Amazon SQS Sink
> Connector<
https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector> 
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector;>.


> This FLIP is proposing to add support for AWS SQS sink in
> flink-connector-aws repo.
>
> For more details, see FLIP-438. Looking forward to your feedback.
>
https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector 
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector>

>
> Thanks,
> Priya

Re: [ANNOUNCE] New Apache Flink Committer - Zakelly Lan

[Muhammet Orazov]

Congratulations Zakelly!

Best,
Muhammet


On 2024-04-15 02:50, Yuan Mei wrote:

Hi everyone,

On behalf of the PMC, I'm happy to let you know that Zakelly Lan has 
become

a new Flink Committer!

Zakelly has been continuously contributing to the Flink project since 
2020,
with a focus area on Checkpointing, State as well as frocksdb (the 
default

on-disk state db).

He leads several FLIPs to improve checkpoints and state APIs, including
File Merging for Checkpoints and configuration/API reorganizations. He 
is
also one of the main contributors to the recent efforts of 
"disaggregated

state management for Flink 2.0" and drives the entire discussion in the
mailing thread, demonstrating outstanding technical depth and breadth 
of

knowledge.

Beyond his technical contributions, Zakelly is passionate about helping 
the
community in numerous ways. He spent quite some time setting up the 
Flink
Speed Center and rebuilding the benchmark pipeline after the original 
one

was out of lease. He helps build frocksdb and tests for the upcoming
frocksdb release (bump rocksdb from 6.20.3->8.10).

Please join me in congratulating Zakelly for becoming an Apache Flink
committer!

Best,
Yuan (on behalf of the Flink PMC)

Re: [DISCUSS] FLIP-438: Amazon SQS Sink Connector

[Muhammet Orazov]

Hey Priya,

Thanks for the FLIP and driving it!

One question from my side on test plan:

End to end integration tests that hit the real Amazon SQS service. 
These tests will be enabled when credentials are defined.


Would it make sense to use here docker based integration testing using 
Testcontainer[1], for example, with Localstack[2]? Seems like the SQS 
service is provided on the free tier.


[1]: https://java.testcontainers.org/modules/localstack/
[2]: https://docs.localstack.cloud/user-guide/aws/sqs/

On 2024-04-05 18:16, Dhingra, Priya wrote:

Hi Dev,

I would like to start a discussion about FLIP-438: Amazon SQS Sink 
Connector. 
This FLIP is proposing to add support for AWS SQS sink in 
flink-connector-aws repo.


For more details, see FLIP-438. Looking forward to your feedback.
https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector

Thanks,
Priya

Re: [ANNOUNCE] New Apache Flink PMC Member - Lincoln Lee

[Muhammet Orazov]

Congratulations Lincoln! Well deserved!

Best,
Muhammet


On 2024-04-12 07:59, Jark Wu wrote:

Hi everyone,

On behalf of the PMC, I'm very happy to announce that Lincoln Lee has
joined the Flink PMC!

Lincoln has been an active member of the Apache Flink community for
many years. He mainly works on Flink SQL component and has driven
/pushed many FLIPs around SQL, including FLIP-282/373/415/435 in
the recent versions. He has a great technical vision of Flink SQL and
participated in plenty of discussions in the dev mailing list. Besides
that,
he is community-minded, such as being the release manager of 1.19,
verifying releases, managing release syncs, writing the release
announcement etc.

Congratulations and welcome Lincoln!

Best,
Jark (on behalf of the Flink PMC)

Re: [ANNOUNCE] New Apache Flink PMC Member - Jing Ge

[Muhammet Orazov]

Congratulations Jing! Well deserved!

Best,
Muhammet

On 2024-04-12 08:02, Jark Wu wrote:

Hi everyone,

On behalf of the PMC, I'm very happy to announce that Jing Ge has
joined the Flink PMC!

Jing has been contributing to Apache Flink for a long time. He 
continuously

works on SQL, connectors, Source, and Sink APIs, test, and document
modules while contributing lots of code and insightful discussions. He 
is
one of the maintainers of Flink CI infra. He is also willing to help a 
lot

in the
community work, such as being the release manager for both 1.18 and 
1.19,
verifying releases, and answering questions on the mailing list. 
Besides

that,
he is continuously helping with the expansion of the Flink community 
and

has
given several talks about Flink at many conferences, such as Flink 
Forward

2022 and 2023.

Congratulations and welcome Jing!

Best,
Jark (on behalf of the Flink PMC)

[jira] [Created] (FLINK-35078) Include Flink CDC pipeline connector sub-modules in the CI runs

[Muhammet Orazov (Jira)]

Muhammet Orazov created FLINK-35078:
---

 Summary: Include Flink CDC pipeline connector sub-modules in the 
CI runs
 Key: FLINK-35078
 URL: https://issues.apache.org/jira/browse/FLINK-35078
 Project: Flink
  Issue Type: Improvement
  Components: Flink CDC
Reporter: Muhammet Orazov


At the moment, on the the Flink CDC pipeline connector parent is build using 
the GHA workflow. This misses the maven plugin checks on the sub-modules.

Similar to the other modules we should list the pipeline connector modules 
separately in the CI built.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

[Muhammet Orazov]

Hey Rui,

+1 (non-binding).

Thanks for driving it!

Best,
Muhammet

On 2024-04-10 04:36, Rui Fan wrote:

Hi devs,

Thank you to everyone for the feedback on FLIP-441: Show
the JobType and remove Execution Mode on Flink WebUI[1]
which has been discussed in this thread [2].

I would like to start a vote for it. The vote will be open for at least 
72

hours unless there is an objection or not enough votes.

[1] https://cwiki.apache.org/confluence/x/agrPEQ
[2] https://lists.apache.org/thread/0s52w17w24x7m2zo6ogl18t1fy412vcd

Best,
Rui

Re: [DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

[Muhammet Orazov]

Hey Xuannan,

Thanks for the FLIP and your efforts!

Minor clarification from my side:


We will relocate these ConfigOptions to a class that is included
in the documentation generation.


Would it make sense to define also in the FLIP the options class for
these variables? For example, GPUDriverOptions?

Best,
Muhammet

On 2024-04-09 08:20, Xuannan Su wrote:

Hi all,

I'd like to start a discussion on FLIP-442: General Improvement to
Configuration for Flink 2.0 [1]. As Flink moves toward 2.0, we aim to
provide users with a better experience with the existing
configuration. This FLIP proposes several general improvements to the
current configuration.

Looking forward to everyone's feedback and suggestions. Thank you!

Best regards,
Xuannan

[1] 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-442%3A+General+Improvement+to+Configuration+for+Flink+2.0

Re: Inquiry Regarding Azure Pipelines

[Muhammet Orazov]

Hey Robert,

Thanks for fixing the flinkbot issue. Thanks for your efforts (including 
Mathias and Lorenzo)!


Best,
Muhammet

p.s. Mandatory XKCD: https://xkcd.com/2347/

On 2024-04-03 16:52, Robert Metzger wrote:

Hi Yisha,

flinkbot is currently not active, so new PRs are not triggering any AZP
builds. We hope to restore the service soon.

AZP is still the source of truth for CI builds.


On Wed, Apr 3, 2024 at 11:34 AM Yisha Zhou 


wrote:


Hi devs,

I hope this email finds you well. I am writing to seek clarification
regarding the status of Azure Pipelines within the Apache community 
and

seek assistance with a specific issue I encountered.

Today, I made some new commits to a pull request in one of the Apache
repositories. However, I noticed that even after approximately six 
hours,
there were no triggers initiated for the Azure Pipeline. I have a 
couple of

questions regarding this matter:

1. Is the Apache community still utilizing Azure Pipelines for CI/CD
purposes? I came across an issue discussing the migration from Azure 
to
GitHub Actions, but I am uncertain about the timeline for 
discontinuing the

use of Azure Pipelines.

2. If Azure Pipelines are still in use, where can I find information 
about

the position of my commits in the CI queue, awaiting execution?

I would greatly appreciate any insights or guidance you can provide
regarding these questions. Thank you for your time and attention.

My PR link is https://github.com/apache/flink/pull/24567 <
https://github.com/apache/flink/pull/24567>.

Best regards,
Yisha

Re: [VOTE] FLIP-399: Flink Connector Doris

[Muhammet Orazov]

Hey Wudi,

Thanks for your efforts.

+1 (non-binding)

Best,
Muhammet

On 2024-04-09 02:47, wudi wrote:

Hi devs,

I would like to start a vote about FLIP-399 [1]. The FLIP is about 
contributing the Flink Doris Connector[2] to the Flink community. 
Discussion thread [3].


The vote will be open for at least 72 hours unless there is an 
objection or

insufficient votes.


Thanks,
Di.Wu


[1] 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-399%3A+Flink+Connector+Doris

[2] https://github.com/apache/doris-flink-connector
[3] https://lists.apache.org/thread/p3z4wsw3ftdyfs9p2wd7bbr2gfyl3xnh

Re: [DISCUSS] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

[Muhammet Orazov]

Hey Rui,

Thanks for the detailed explanation and updating the FLIP!

It is much clearer definitely, thanks for the proposal.

Best,
Muhammet

On 2024-03-28 07:37, Rui Fan wrote:

Hi Muhammet,

Thanks for your reply!


The execution mode is also used for the DataStream API [1],
would that also affect/hide the DataStream execution mode
if we remove it from the WebUI?


Sorry, I didn't describe it clearly in FLIP-441[2], I have updated it.
Let me clarify the Execution Mode here:

1. Flink 1.19 website[3] also mentions the Execution mode, but it
actually matches the JobType[4] in the Flink code. Both of them
have 2 types: STREAMING and BATCH.
2. execution.runtime-mode can be set to 3 types: STREAMING,
BATCH and AUTOMATIC. But the jobType will be inferred as
STREAMING or BATCH when execution.runtime-mode is set
to AUTOMATIC.
3. The ExecutionMode I describe is: code link[5] , as we can
see, ExecutionMode has 4 enums: PIPELINED,
PIPELINED_FORCED, BATCH and BATCH_FORCED.
And we can see a flink streaming job from Flink WebUI,
the Execution mode is PIPELINE instead of STREAMING.
I attached a screenshot to the FLIP doc[2], you can see it there.
4. What this proposal wants to do is to remove the ExecutionMode
with four enumerations on Flink WebUI and introduce the
JobType with two enumerations (STREAMING or BATCH).
STREAMING or BATCH is clearer and more accurate for users.

Please let me know if it's not clear or anything is wrong, thanks a 
lot!


[1]
https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/dev/datastream/execution_mode/
[2] https://cwiki.apache.org/confluence/x/agrPEQ
[3]
https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/datastream/execution_mode/
[4]
https://github.com/apache/flink/blob/f31c128bfc457b64dd7734f71123b74faa2958ba/flink-runtime/src/main/java/org/apache/flink/runtime/jobgraph/JobType.java#L22
[5]
https://github.com/apache/flink/blob/f31c128bfc457b64dd7734f71123b74faa2958ba/flink-core/src/main/java/org/apache/flink/api/common/ExecutionMode.java#L54

Best,
Rui

On Thu, Mar 28, 2024 at 1:33 AM Venkatakrishnan Sowrirajan 


wrote:


Rui,

I assume the current proposal would also handle the case of mixed mode
(BATCH + STREAMING within the same app) in the future, right?

Regards
Venkat

On Wed, Mar 27, 2024 at 10:15 AM Venkatakrishnan Sowrirajan <
vsowr...@asu.edu> wrote:

This will be a very useful addition to Flink UI. Thanks Rui for 
starting

a FLIP for this improvement.

Regards
Venkata krishnan


On Wed, Mar 27, 2024 at 4:49 AM Muhammet Orazov
 wrote:


Hello Rui,

Thanks for the proposal! It looks good!

I have minor clarification from my side:

The execution mode is also used for the DataStream API [1],
would that also affect/hide the DataStream execution mode
if we remove it from the WebUI?

Best,
Muhammet

[1]:

https://urldefense.com/v3/__https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/dev/datastream/execution_mode/__;!!IKRxdwAv5BmarQ!eFyqVJyje_8hu1vMSUwKGBsj8vqsFDisEvJ5AxPV0LduhhHWF3rPKYEEE-09biA0unFbfMy5AVQZMgBv1AOa5oTHmcYlkUE$


On 2024-03-27 06:23, Rui Fan wrote:
> Hi flink developers,
>
> I'd like to start a discussion to discuss FLIP-441:
> Show the JobType and remove Execution Mode on Flink WebUI[1].
>
> Currently, the jobType has 2 types in Flink: STREAMING and BATCH.
> They work on completely different principles, such as: scheduler,
> shuffle, join, etc. These differences lead to different troubleshooting
> processes, so when users are maintaining a job or troubleshooting,
> it's needed to know whether the current job is a STREAMING or
> BATCH job. Unfortunately, Flink WebUI doesn't expose it to the
> users so far.
>
> Also, Execution Mode is related to DataSet api, it has been marked
> as @Deprecated in FLINK-32258 (1.18), but it's still shown in Flink
> WebUI.
>
> Looking forward to hearing more thoughts about it! Thank you~
>
> [1]
https://urldefense.com/v3/__https://cwiki.apache.org/confluence/x/agrPEQ__;!!IKRxdwAv5BmarQ!eFyqVJyje_8hu1vMSUwKGBsj8vqsFDisEvJ5AxPV0LduhhHWF3rPKYEEE-09biA0unFbfMy5AVQZMgBv1AOa5oTHayPyFj8$
> [2]
https://urldefense.com/v3/__https://issues.apache.org/jira/browse/FLINK-32558__;!!IKRxdwAv5BmarQ!eFyqVJyje_8hu1vMSUwKGBsj8vqsFDisEvJ5AxPV0LduhhHWF3rPKYEEE-09biA0unFbfMy5AVQZMgBv1AOa5oTHftYeOLE$
>
> Best,
> Rui

Re: [DISCUSS] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

[Muhammet Orazov]

Hello Rui,

Thanks for the proposal! It looks good!

I have minor clarification from my side:

The execution mode is also used for the DataStream API [1],
would that also affect/hide the DataStream execution mode
if we remove it from the WebUI?

Best,
Muhammet

[1]: 
https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/dev/datastream/execution_mode/



On 2024-03-27 06:23, Rui Fan wrote:

Hi flink developers,

I'd like to start a discussion to discuss FLIP-441:
Show the JobType and remove Execution Mode on Flink WebUI[1].

Currently, the jobType has 2 types in Flink: STREAMING and BATCH.
They work on completely different principles, such as: scheduler,
shuffle, join, etc. These differences lead to different troubleshooting
processes, so when users are maintaining a job or troubleshooting,
it's needed to know whether the current job is a STREAMING or
BATCH job. Unfortunately, Flink WebUI doesn't expose it to the
users so far.

Also, Execution Mode is related to DataSet api, it has been marked
as @Deprecated in FLINK-32258 (1.18), but it's still shown in Flink 
WebUI.


Looking forward to hearing more thoughts about it! Thank you~

[1] https://cwiki.apache.org/confluence/x/agrPEQ
[2] https://issues.apache.org/jira/browse/FLINK-32558

Best,
Rui

Re: [ANNOUNCE] Apache Flink Kubernetes Operator 1.8.0 released

[Muhammet Orazov]

Great! Thanks Maximilian and everyone involved for the effort and 
release!


Best,
Muhammet

On 2024-03-25 10:35, Maximilian Michels wrote:

The Apache Flink community is very happy to announce the release of
the Apache Flink Kubernetes Operator version 1.8.0.

The Flink Kubernetes Operator allows users to manage their Apache
Flink applications on Kubernetes through all aspects of their
lifecycle.

Release highlights:
- Flink Autotuning automatically adjusts TaskManager memory
- Flink Autoscaling metrics and decision accuracy improved
- Improve standalone Flink Autoscaling
- Savepoint trigger nonce for savepoint-based restarts
- Operator stability improvements for cluster shutdown

Blog post: 
https://flink.apache.org/2024/03/21/apache-flink-kubernetes-operator-1.8.0-release-announcement/


The release is available for download at:
https://flink.apache.org/downloads.html

Maven artifacts for Flink Kubernetes Operator can be found at:
https://search.maven.org/artifact/org.apache.flink/flink-kubernetes-operator

Official Docker image for Flink Kubernetes Operator can be found at:
https://hub.docker.com/r/apache/flink-kubernetes-operator

The full release notes are available in Jira:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12353866=12315522

We would like to thank the Apache Flink community and its contributors
who made this release possible!

Cheers,
Max

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

[Muhammet Orazov]

Congratulations! Thanks for the great effort!

Best,
Muhammet Orazov

On 2024-03-20 13:34, Leonard Xu wrote:

Hi devs and users,

We are thrilled to announce that the donation of Flink CDC as a 
sub-project of Apache Flink has completed. We invite you to explore the 
new resources available:


- GitHub Repository: https://github.com/apache/flink-cdc
- Flink CDC Documentation: 
https://nightlies.apache.org/flink/flink-cdc-docs-stable


After Flink community accepted this donation[1], we have completed 
software copyright signing, code repo migration, code cleanup, website 
migration, CI migration and github issues migration etc.
Here I am particularly grateful to Hang Ruan, Zhongqaing Gong, 
Qingsheng Ren, Jiabao Sun, LvYanquan, loserwang1024 and other 
contributors for their contributions and help during this process！



For all previous contributors: The contribution process has slightly 
changed to align with the main Flink project. To report bugs or suggest 
new features, please open tickets
Apache Jira (https://issues.apache.org/jira).  Note that we will no 
longer accept GitHub issues for these purposes.



Welcome to explore the new repository and documentation. Your feedback 
and contributions are invaluable as we continue to improve Flink CDC.


Thanks everyone for your support and happy exploring Flink CDC!

Best，
Leonard
[1] https://lists.apache.org/thread/cw29fhsp99243yfo95xrkw82s5s418ob