Re: [ANNOUNCE] New Apache Flink Committer - Zhongqiang Gong

[Samrat Deb]

Congratulations Zhongqiang Gong !

Bests,
Samrat

On Mon, Jun 17, 2024 at 8:50 AM Leonard Xu  wrote:

> Hi everyone,
> On behalf of the PMC, I'm happy to announce that Zhongqiang Gong has
> become a new Flink Committer!
>
> Zhongqiang has been an active Flink community member since November 2021,
> contributing numerous PRs to both the Flink and Flink CDC repositories. As
> a core contributor to Flink CDC, he developed the Oracle and SQL Server CDC
> Connectors and managed essential website and CI migrations during the
> donation of Flink CDC to Apache Flink.
>
> Beyond his technical contributions, Zhongqiang actively participates in
> discussions on the Flink dev mailing list and responds to threads on the
> user and user-zh mailing lists. As an Apache StreamPark (incubating)
> Committer, he promotes Flink SQL and Flink CDC technologies at meetups and
> within the StreamPark community.
>
> Please join me in congratulating Zhongqiang Gong for becoming an Apache
> Flink committer!
>
> Best,
> Leonard (on behalf of the Flink PMC)

Re: [ANNOUNCE] New Apache Flink Committer - Hang Ruan

[Samrat Deb]

Congratulations Hang Ruan !

Bests,
Samrat

On Mon, Jun 17, 2024 at 8:47 AM Leonard Xu  wrote:

> Hi everyone,
> On behalf of the PMC, I'm happy to let you know that Hang Ruan has become
> a new Flink Committer !
>
> Hang Ruan has been continuously contributing to the Flink project since
> August 2021. Since then, he has continuously contributed to Flink, Flink
> CDC, and various Flink connector repositories, including
> flink-connector-kafka, flink-connector-elasticsearch, flink-connector-aws,
> flink-connector-rabbitmq, flink-connector-pulsar, and
> flink-connector-mongodb. Hang Ruan focuses on the improvements related to
> connectors and catalogs and initiated FLIP-274. He is most recognized as a
> core contributor and maintainer for the Flink CDC project, contributing
> many features such as MySQL CDC newly table addition and the Schema
> Evolution feature.
>
> Beyond his technical contributions, Hang Ruan is an active member of the
> Flink community. He regularly engages in discussions on the Flink dev
> mailing list and the user-zh and user mailing lists, participates in FLIP
> discussions, assists with user Q, and consistently volunteers for release
> verifications.
>
> Please join me in congratulating Hang Ruan for becoming an Apache Flink
> committer!
>
> Best,
> Leonard (on behalf of the Flink PMC)

Re: [ANNOUNCE] New Apache Flink PMC Member - Fan Rui

[Samrat Deb]

Congratulations Rui

Bests,
Samrat

On Thu, 6 Jun 2024 at 7:45 AM, Yuxin Tan  wrote:

> Congratulations, Rui!
>
> Best,
> Yuxin
>
>
> Xuannan Su  于2024年6月6日周四 09:58写道：
>
> > Congratulations!
> >
> > Best regards,
> > Xuannan
> >
> > On Thu, Jun 6, 2024 at 9:53 AM Hangxiang Yu  wrote:
> > >
> > > Congratulations, Rui !
> > >
> > > On Thu, Jun 6, 2024 at 9:18 AM Lincoln Lee 
> > wrote:
> > >
> > > > Congratulations, Rui!
> > > >
> > > > Best,
> > > > Lincoln Lee
> > > >
> > > >
> > > > Lijie Wang  于2024年6月6日周四 09:11写道：
> > > >
> > > > > Congratulations, Rui!
> > > > >
> > > > > Best,
> > > > > Lijie
> > > > >
> > > > > Rodrigo Meneses  于2024年6月5日周三 21:35写道：
> > > > >
> > > > > > All the best
> > > > > >
> > > > > > On Wed, Jun 5, 2024 at 5:56 AM xiangyu feng <
> xiangyu...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Congratulations, Rui!
> > > > > > >
> > > > > > > Regards,
> > > > > > > Xiangyu Feng
> > > > > > >
> > > > > > > Feng Jin  于2024年6月5日周三 20:42写道：
> > > > > > >
> > > > > > > > Congratulations, Rui!
> > > > > > > >
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Feng Jin
> > > > > > > >
> > > > > > > > On Wed, Jun 5, 2024 at 8:23 PM Yanfei Lei <
> fredia...@gmail.com
> > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Congratulations, Rui!
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Yanfei
> > > > > > > > >
> > > > > > > > > Luke Chen  于2024年6月5日周三 20:08写道：
> > > > > > > > > >
> > > > > > > > > > Congrats, Rui!
> > > > > > > > > >
> > > > > > > > > > Luke
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 5, 2024 at 8:02 PM Jiabao Sun <
> > > > jiabao...@apache.org>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Congrats, Rui. Well-deserved!
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Jiabao
> > > > > > > > > > >
> > > > > > > > > > > Zhanghao Chen  于2024年6月5日周三
> > > > > 19:29写道：
> > > > > > > > > > >
> > > > > > > > > > > > Congrats, Rui!
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Zhanghao Chen
> > > > > > > > > > > > 
> > > > > > > > > > > > From: Piotr Nowojski 
> > > > > > > > > > > > Sent: Wednesday, June 5, 2024 18:01
> > > > > > > > > > > > To: dev ; rui fan <
> > > > > 1996fan...@gmail.com>
> > > > > > > > > > > > Subject: [ANNOUNCE] New Apache Flink PMC Member - Fan
> > Rui
> > > > > > > > > > > >
> > > > > > > > > > > > Hi everyone,
> > > > > > > > > > > >
> > > > > > > > > > > > On behalf of the PMC, I'm very happy to announce
> > another
> > > > new
> > > > > > > Apache
> > > > > > > > > Flink
> > > > > > > > > > > > PMC Member - Fan Rui.
> > > > > > > > > > > >
> > > > > > > > > > > > Rui has been active in the community since August
> 2019.
> > > > > During
> > > > > > > this
> > > > > > > > > time
> > > > > > > > > > > he
> > > > > > > > > > > > has contributed a lot of new features. Among others:
> > > > > > > > > > > >   - Decoupling Autoscaler from Kubernetes Operator,
> and
> > > > > > > supporting
> > > > > > > > > > > > Standalone Autoscaler
> > > > > > > > > > > >   - Improvements to checkpointing, flamegraphs,
> restart
> > > > > > > strategies,
> > > > > > > > > > > > watermark alignment, network shuffles
> > > > > > > > > > > >   - Optimizing the memory and CPU usage of large
> > operators,
> > > > > > > greatly
> > > > > > > > > > > > reducing the risk and probability of TaskManager OOM
> > > > > > > > > > > >
> > > > > > > > > > > > He reviewed a significant amount of PRs and has been
> > active
> > > > > > both
> > > > > > > on
> > > > > > > > > the
> > > > > > > > > > > > mailing lists and in Jira helping to both maintain
> and
> > grow
> > > > > > > Apache
> > > > > > > > > > > Flink's
> > > > > > > > > > > > community. He is also our current Flink 1.20 release
> > > > manager.
> > > > > > > > > > > >
> > > > > > > > > > > > In the last 12 months, Rui has been the most active
> > > > > contributor
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > > > > Flink Kubernetes Operator project, while being the
> 2nd
> > most
> > > > > > > active
> > > > > > > > > Flink
> > > > > > > > > > > > contributor at the same time.
> > > > > > > > > > > >
> > > > > > > > > > > > Please join me in welcoming and congratulating Fan
> Rui!
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Piotrek (on behalf of the Flink PMC)
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best,
> > > Hangxiang.
> >
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo

[Samrat Deb]

Congratulations

Bests,
Samrat

On Thu, 6 Jun 2024 at 7:28 AM, Xuannan Su  wrote:

> Congratulations!
>
> Best regards,
> Xuannan
>
> On Thu, Jun 6, 2024 at 9:20 AM Jiabao Sun  wrote:
> >
> > Congratulations, Weijie!
> >
> > Best,
> > Jiabao
> >
> > Lincoln Lee  于2024年6月6日周四 09:17写道：
> >
> > > Congratulations, Weijie!
> > >
> > > Best,
> > > Lincoln Lee
> > >
> > >
> > > ConradJam  于2024年6月5日周三 10:46写道：
> > >
> > > > Congratulations！
> > > >
> > > > Biao Geng  于2024年6月5日周三 10:44写道：
> > > >
> > > > > Congratulations, Weijie!
> > > > > Best,
> > > > > Biao Geng
> > > > >
> > > > >
> > > > > Yun Tang  于2024年6月5日周三 10:42写道：
> > > > >
> > > > > > Congratulations, Weijie!
> > > > > >
> > > > > > Best
> > > > > > Yun Tang
> > > > > > 
> > > > > > From: Hangxiang Yu 
> > > > > > Sent: Wednesday, June 5, 2024 10:00
> > > > > > To: dev@flink.apache.org 
> > > > > > Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
> > > > > >
> > > > > > Congratulations, Weijie!
> > > > > >
> > > > > > On Tue, Jun 4, 2024 at 11:40 PM Zhanghao Chen <
> > > > zhanghao.c...@outlook.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Congrats, Weijie!
> > > > > > >
> > > > > > > Best,
> > > > > > > Zhanghao Chen
> > > > > > > 
> > > > > > > From: Hang Ruan 
> > > > > > > Sent: Tuesday, June 4, 2024 16:37
> > > > > > > To: dev@flink.apache.org 
> > > > > > > Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie
> Guo
> > > > > > >
> > > > > > > Congratulations Weijie!
> > > > > > >
> > > > > > > Best,
> > > > > > > Hang
> > > > > > >
> > > > > > > Yanfei Lei  于2024年6月4日周二 16:24写道：
> > > > > > >
> > > > > > > > Congratulations!
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Yanfei
> > > > > > > >
> > > > > > > > Leonard Xu  于2024年6月4日周二 16:20写道：
> > > > > > > > >
> > > > > > > > > Congratulations！
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Leonard
> > > > > > > > >
> > > > > > > > > > 2024年6月4日 下午4:02，Yangze Guo  写道：
> > > > > > > > > >
> > > > > > > > > > Congratulations!
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Yangze Guo
> > > > > > > > > >
> > > > > > > > > > On Tue, Jun 4, 2024 at 4:00 PM Weihua Hu <
> > > > huweihua@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > >>
> > > > > > > > > >> Congratulations, Weijie!
> > > > > > > > > >>
> > > > > > > > > >> Best,
> > > > > > > > > >> Weihua
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >> On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan <
> > > > > tanyuxinw...@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > >>
> > > > > > > > > >>> Congratulations, Weijie!
> > > > > > > > > >>>
> > > > > > > > > >>> Best,
> > > > > > > > > >>> Yuxin
> > > > > > > > > >>>
> > > > > > > > > >>>
> > > > > > > > > >>> Yuepeng Pan  于2024年6月4日周二
> 14:57写道：
> > > > > > > > > >>>
> > > > > > > > >  Congratulations !
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > >  Best,
> > > > > > > > >  Yuepeng Pan
> > > > > > > > > 
> > > > > > > > >  At 2024-06-04 14:45:45, "Xintong Song" <
> > > > tonysong...@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > > Hi everyone,
> > > > > > > > > >
> > > > > > > > > > On behalf of the PMC, I'm very happy to announce that
> > > > Weijie
> > > > > > Guo
> > > > > > > > has
> > > > > > > > >  joined
> > > > > > > > > > the Flink PMC!
> > > > > > > > > >
> > > > > > > > > > Weijie has been an active member of the Apache Flink
> > > > > community
> > > > > > > for
> > > > > > > > many
> > > > > > > > > > years. He has made significant contributions in many
> > > > > > components,
> > > > > > > > > >>> including
> > > > > > > > > > runtime, shuffle, sdk, connectors, etc. He has
> driven /
> > > > > > > > participated in
> > > > > > > > > > many FLIPs, authored and reviewed hundreds of PRs,
> been
> > > > > > > > consistently
> > > > > > > > >  active
> > > > > > > > > > on mailing lists, and also helped with release
> management
> > > > of
> > > > > > 1.20
> > > > > > > > and
> > > > > > > > > > several other bugfix releases.
> > > > > > > > > >
> > > > > > > > > > Congratulations and welcome Weijie!
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > >
> > > > > > > > > > Xintong (on behalf of the Flink PMC)
> > > > > > > > > 
> > > > > > > > > >>>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best,
> > > > > > Hangxiang.
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best
> > > >
> > > > ConradJam
> > > >
> > >
>

Re: [VOTE] FLIP-399: Flink Connector Doris

[Samrat Deb]

+1 (non binding)

On Mon, 15 Apr 2024 at 2:16 PM, Aleksandr Pilipenko 
wrote:

> +1 (non-binding),
>
> Best,
> Aleksandr
>
> On Mon, 15 Apr 2024 at 03:03, Yanquan Lv  wrote:
>
> > +1 (non-binding), thanks for it.
> >
> > wudi <676366...@qq.com.invalid> 于2024年4月9日周二 10:48写道：
> >
> > > Hi devs,
> > >
> > > I would like to start a vote about FLIP-399 [1]. The FLIP is about
> > > contributing the Flink Doris Connector[2] to the Flink community.
> > > Discussion thread [3].
> > >
> > > The vote will be open for at least 72 hours unless there is an
> objection
> > or
> > > insufficient votes.
> > >
> > >
> > > Thanks,
> > > Di.Wu
> > >
> > >
> > > [1]
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-399%3A+Flink+Connector+Doris
> > > [2] https://github.com/apache/doris-flink-connector
> > > [3] https://lists.apache.org/thread/p3z4wsw3ftdyfs9p2wd7bbr2gfyl3xnh
> > >
> > >
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Zakelly Lan

[Samrat Deb]

Congratulations Zakelly!

 Bests,
 Samrat

On Mon, 15 Apr 2024 at 1:19 PM, Sergey Nuyanzin  wrote:

> Congratulations,  Zakelly Lan!
>
> On Mon, Apr 15, 2024 at 9:46 AM Jinzhong Li 
> wrote:
>
> > Congratulations, Zakelly!
> >
> > Best,
> > Jinzhong
> >
> > On Mon, Apr 15, 2024 at 2:56 PM Hangxiang Yu 
> wrote:
> >
> > > Congratulations, Zakelly!
> > >
> > > On Mon, Apr 15, 2024 at 1:58 PM Yun Tang  wrote:
> > >
> > > > Congratulations, Zakelly!
> > > >
> > > > Best
> > > > Yun Tang
> > > > 
> > > > From: Yanquan Lv 
> > > > Sent: Monday, April 15, 2024 13:23
> > > > To: dev@flink.apache.org 
> > > > Subject: Re: [ANNOUNCE] New Apache Flink Committer - Zakelly Lan
> > > >
> > > > Congratulations, Zakelly!
> > > >
> > > > Best,
> > > > YanQuan
> > > >
> > > > Yuan Mei  于2024年4月15日周一 10:51写道：
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > On behalf of the PMC, I'm happy to let you know that Zakelly Lan
> has
> > > > become
> > > > > a new Flink Committer!
> > > > >
> > > > > Zakelly has been continuously contributing to the Flink project
> since
> > > > 2020,
> > > > > with a focus area on Checkpointing, State as well as frocksdb (the
> > > > default
> > > > > on-disk state db).
> > > > >
> > > > > He leads several FLIPs to improve checkpoints and state APIs,
> > including
> > > > > File Merging for Checkpoints and configuration/API reorganizations.
> > He
> > > is
> > > > > also one of the main contributors to the recent efforts of
> > > "disaggregated
> > > > > state management for Flink 2.0" and drives the entire discussion in
> > the
> > > > > mailing thread, demonstrating outstanding technical depth and
> breadth
> > > of
> > > > > knowledge.
> > > > >
> > > > > Beyond his technical contributions, Zakelly is passionate about
> > helping
> > > > the
> > > > > community in numerous ways. He spent quite some time setting up the
> > > Flink
> > > > > Speed Center and rebuilding the benchmark pipeline after the
> original
> > > one
> > > > > was out of lease. He helps build frocksdb and tests for the
> upcoming
> > > > > frocksdb release (bump rocksdb from 6.20.3->8.10).
> > > > >
> > > > > Please join me in congratulating Zakelly for becoming an Apache
> Flink
> > > > > committer!
> > > > >
> > > > > Best,
> > > > > Yuan (on behalf of the Flink PMC)
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best,
> > > Hangxiang.
> > >
> >
>
>
> --
> Best regards,
> Sergey
>

Re: [Vote] FLIP-438: Amazon SQS Sink Connector

[Samrat Deb]

+1 (non-binding)

Bests,
Samrat

On Sun, 14 Apr 2024 at 3:38 PM, Ahmed Hamdy  wrote:

> Hi Priya
> +1 (non-binding)
> Best Regards
> Ahmed Hamdy
>
>
> On Fri, 12 Apr 2024 at 22:56, Dhingra, Priya 
> wrote:
>
> > Hi devs,
> >
> >
> >
> > Thank you to everyone for the feedback on FLIP-438: Amazon SQS Sink
> > Connector<
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> > >
> >
> >
> >
> > I would like to start a vote for it. The vote will be open for at least
> 72
> >
> > hours unless there is an objection or not enough votes.
> >
> >
> >
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> >
> > Regards
> > Priya
> >
>

Re: [VOTE] FLIP-399: Flink Connector Doris

[Samrat Deb]

+1 ( non binding )

Bests,
Samrat

On Fri, 12 Apr 2024 at 1:07 PM, Hang Ruan  wrote:

> +1 (non-binding)
>
> Best,
> Hang
>
> Martijn Visser  于2024年4月12日周五 05:39写道：
>
> > +1 (binding)
> >
> > On Wed, Apr 10, 2024 at 4:34 AM Jing Ge 
> > wrote:
> >
> > > +1(binding)
> > >
> > > Best regards,
> > > Jing
> > >
> > > On Tue, Apr 9, 2024 at 8:54 PM Feng Jin  wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > Best,
> > > > Feng
> > > >
> > > > On Tue, Apr 9, 2024 at 5:56 PM gongzhongqiang <
> > gongzhongqi...@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Best,
> > > > >
> > > > > Zhongqiang Gong
> > > > >
> > > > > wudi <676366...@qq.com.invalid> 于2024年4月9日周二 10:48写道：
> > > > >
> > > > > > Hi devs,
> > > > > >
> > > > > > I would like to start a vote about FLIP-399 [1]. The FLIP is
> about
> > > > > > contributing the Flink Doris Connector[2] to the Flink community.
> > > > > > Discussion thread [3].
> > > > > >
> > > > > > The vote will be open for at least 72 hours unless there is an
> > > > objection
> > > > > or
> > > > > > insufficient votes.
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > Di.Wu
> > > > > >
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-399%3A+Flink+Connector+Doris
> > > > > > [2] https://github.com/apache/doris-flink-connector
> > > > > > [3]
> > https://lists.apache.org/thread/p3z4wsw3ftdyfs9p2wd7bbr2gfyl3xnh
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Lincoln Lee

[Samrat Deb]

Congratulations, Lincoln!


On Fri, 12 Apr 2024 at 2:16 PM, Jiabao Sun  wrote:

> Congratulations, Lincoln!
>
> Best,
> Jiabao
>
> Sergey Nuyanzin  于2024年4月12日周五 16:41写道：
>
> > Congratulations, Lincoln!
> >
> >
> >
> > On Fri, Apr 12, 2024 at 10:30 AM Rui Fan <1996fan...@gmail.com> wrote:
> >
> > > Congratulations, Lincoln!
> > >
> > > Best,
> > > Rui
> > >
> > > On Fri, Apr 12, 2024 at 3:59 PM Jark Wu  wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > On behalf of the PMC, I'm very happy to announce that Lincoln Lee has
> > > > joined the Flink PMC!
> > > >
> > > > Lincoln has been an active member of the Apache Flink community for
> > > > many years. He mainly works on Flink SQL component and has driven
> > > > /pushed many FLIPs around SQL, including FLIP-282/373/415/435 in
> > > > the recent versions. He has a great technical vision of Flink SQL and
> > > > participated in plenty of discussions in the dev mailing list.
> Besides
> > > > that,
> > > > he is community-minded, such as being the release manager of 1.19,
> > > > verifying releases, managing release syncs, writing the release
> > > > announcement etc.
> > > >
> > > > Congratulations and welcome Lincoln!
> > > >
> > > > Best,
> > > > Jark (on behalf of the Flink PMC)
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Sergey
> >
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Jing Ge

[Samrat Deb]

Congratulations, Jing!


On Fri, 12 Apr 2024 at 2:15 PM, Jiabao Sun  wrote:

> Congratulations, Jing!
>
> Best,
> Jiabao
>
> Sergey Nuyanzin  于2024年4月12日周五 16:41写道：
>
> > Congratulations, Jing!
> >
> > On Fri, Apr 12, 2024 at 10:29 AM Rui Fan <1996fan...@gmail.com> wrote:
> >
> > > Congratulations, Jing~
> > >
> > > Best,
> > > Rui
> > >
> > > On Fri, Apr 12, 2024 at 4:28 PM Yun Tang  wrote:
> > >
> > > > Congratulations, Jing!
> > > >
> > > > Best
> > > > Yun Tang
> > > > 
> > > > From: Jark Wu 
> > > > Sent: Friday, April 12, 2024 16:02
> > > > To: dev 
> > > > Cc: gej...@gmail.com 
> > > > Subject: [ANNOUNCE] New Apache Flink PMC Member - Jing Ge
> > > >
> > > > Hi everyone,
> > > >
> > > > On behalf of the PMC, I'm very happy to announce that Jing Ge has
> > > > joined the Flink PMC!
> > > >
> > > > Jing has been contributing to Apache Flink for a long time. He
> > > continuously
> > > > works on SQL, connectors, Source, and Sink APIs, test, and document
> > > > modules while contributing lots of code and insightful discussions.
> He
> > is
> > > > one of the maintainers of Flink CI infra. He is also willing to help
> a
> > > lot
> > > > in the
> > > > community work, such as being the release manager for both 1.18 and
> > 1.19,
> > > > verifying releases, and answering questions on the mailing list.
> > Besides
> > > > that,
> > > > he is continuously helping with the expansion of the Flink community
> > and
> > > > has
> > > > given several talks about Flink at many conferences, such as Flink
> > > Forward
> > > > 2022 and 2023.
> > > >
> > > > Congratulations and welcome Jing!
> > > >
> > > > Best,
> > > > Jark (on behalf of the Flink PMC)
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Sergey
> >
>

Re: [DISCUSS] FLIP-438: Amazon SQS Sink Connector

[Samrat Deb]

vour of this addition in general. I have a couple of
> > > comments/questions on the FLIP.
> > >
> > > - I am not sure why we need to suppress warnings in the sink example in
> > the
> > > FLIP?
> > > - You provided the sink example as it is the Public Interface, however
> > the
> > > actual AsyncSink logic is mostly in the writer, so would be helpful to
> > > provide a brief of the writer or the "submitRequestEntries"
> > > - I am not sure what customDimensions are or how are they going to be
> > used
> > > by the client, (that's why I thought the writer example should be
> > helpful).
> > > - Are we going to use the existing aws client providers to handle the
> > > authentication and async client creation similar to Kinesis/Firehose
> and
> > > DDB. I would strongly recommend we do.
> > > - Given you suggest implementing this in "flink-connector-aws" repo, it
> > > should probably follow the same versioning of AWS connectors, hence
> > > targeting 4.3.0. Also I am not sure why we are targeting support for
> 1.16
> > > given that it is out of support and 4.2 supports 1.17+.
> > > - Are we planning to support Table API & SQL as well? This should not
> be
> > > much of an effort IMO so I think we should.
> > > - It would be great if we also added an implementation of Element
> > converter
> > > given that SQS message bodies are mainly Strings if I remember
> correctly.
> > > We can extend it to other types for MessageAttributeValue augmentation,
> > > this should be more valuable on table API as well to use it as default
> > > Element Converter.
> > >
> > > I would love to assist with the implementation and reviews if this FLIP
> > was
> > > accepted.
> > > Best Regards
> > > Ahmed Hamdy
> > >
> > >
> > > On Fri, 5 Apr 2024 at 19:19, Dhingra, Priya  <mailto:dhipr...@amazon.com.inva>
> > <mailto:dhipr...@amazon.com.inva <mailto:dhipr...@amazon.com.inva>>
> <mailto:dhipr...@amazon.com.inva <mailto:dhipr...@amazon.com.inva>
> > <mailto:dhipr...@amazon.com.inva <mailto:dhipr...@amazon.com.inva>>>lid>
> > > wrote:
> > >
> > > > Hi Dev,
> > > >
> > > > I would like to start a discussion about FLIP-438: Amazon SQS Sink
> > > > Connector<
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> <
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> >
> > <
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector>
> <
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> ;>
> > <
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector>
> <
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> ;>
> > <
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> <
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connectorgt
> >
> > ;>
> > > >.
> > > > This FLIP is proposing to add support for AWS SQS sink in
> > > > flink-connector-aws repo.
> > > >
> > > > For more details, see FLIP-438. Looking forward to your feedback.
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> <
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> >
> > <
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector>
> <
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> ;>
> > <
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector>
> <
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> ;>
> > <
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> <
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connectorgt
> >
> > ;>
> > > >
> > > > Thanks,
> > > > Priya
> > > >
> > >
> >
> >
&g

Re: [VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

[Samrat Deb]

+1 (non binding)

Bests,
Samrat

On Wed, 10 Apr 2024 at 10:26 AM, Feifan Wang  wrote:

>
>
> +1 （non-binding）
>
>
>
>
>
>
>
> --
>
>
>
>
> ——
>
> Best regards,
>
> Feifan Wang
>
>
>
>
> At 2024-04-10 12:36:00, "Rui Fan" <1996fan...@gmail.com> wrote:
> >Hi devs,
> >
> >Thank you to everyone for the feedback on FLIP-441: Show
> >the JobType and remove Execution Mode on Flink WebUI[1]
> >which has been discussed in this thread [2].
> >
> >I would like to start a vote for it. The vote will be open for at least 72
> >hours unless there is an objection or not enough votes.
> >
> >[1] https://cwiki.apache.org/confluence/x/agrPEQ
> >[2] https://lists.apache.org/thread/0s52w17w24x7m2zo6ogl18t1fy412vcd
> >
> >Best,
> >Rui
>

Re: [DISCUSS] FLIP-438: Amazon SQS Sink Connector

[Samrat Deb]

Hi Priya,

Thank you for the FLIP. sqs connector would be a great addition to the
flink connector aws.

+1 for all the queries raised by Ahmed.

Adding to Ahmed's queries, I have a few more:

1. Different connectors provide different types of fault
tolerant guarantees[1]. What type of fault tolerant sink guarantees
flink-connector-sqs will provide ?
Could you elaborate on the fault-tolerant capabilities that the
flink-connector-sqs will provide?

2. Can you help with what the minimal configuration required for
instantiating the sink ?

3. Amazon SQS offers various data types [2]. Could you outline the types of
SQS data the sink plans to support?

[1]
https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/guarantees/
[2]
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_Types.html

Bests,
Samrat

On Sat, Apr 6, 2024 at 6:27 PM Ahmed Hamdy  wrote:

> Hi Dhingra, thanks for raising the FLIP
> I am in favour of this addition in general. I have a couple of
> comments/questions on the FLIP.
>
> - I am not sure why we need to suppress warnings in the sink example in the
> FLIP?
> - You provided the sink example as it is the Public Interface, however the
> actual AsyncSink logic is mostly in the writer, so would be helpful to
> provide a brief of the writer or the "submitRequestEntries"
> - I am not sure what customDimensions are or how are they going to be used
> by the client, (that's why I thought the writer example should be helpful).
> - Are we going to use the existing aws client providers to handle the
> authentication and async client creation similar to Kinesis/Firehose and
> DDB. I would strongly recommend we do.
> - Given you suggest implementing this in "flink-connector-aws" repo, it
> should probably follow the same versioning of AWS connectors, hence
> targeting 4.3.0. Also I am not sure why we are targeting support for 1.16
> given that it is out of support and  4.2 supports 1.17+.
> - Are we planning to support Table API & SQL as well? This should not be
> much of an effort IMO so I think we should.
> - It would be great if we also added an implementation of Element converter
> given that SQS message bodies are mainly Strings if I remember correctly.
> We can extend it to other types for MessageAttributeValue augmentation,
> this should be more valuable on table API as well to use it as default
> Element Converter.
>
> I would love to assist with the implementation and reviews if this FLIP was
> accepted.
> Best Regards
> Ahmed Hamdy
>
>
> On Fri, 5 Apr 2024 at 19:19, Dhingra, Priya 
> wrote:
>
> > Hi Dev,
> >
> > I would like to start a discussion about FLIP-438: Amazon SQS Sink
> > Connector<
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> >.
> > This FLIP is proposing to add support for AWS SQS sink in
> > flink-connector-aws repo.
> >
> > For more details, see FLIP-438. Looking forward to your feedback.
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector
> >
> > Thanks,
> > Priya
> >
>

Re: [ANNOUNCE] Apache Paimon is graduated to Top Level Project

[Samrat Deb]

Congratulations !
Great news

Bests,
Samrat

On Thu, 28 Mar 2024 at 3:00 PM, Ahmed Hamdy  wrote:

> Congratulations!
> Best Regards
> Ahmed Hamdy
>
>
> On Thu, 28 Mar 2024 at 08:16, Paul Lam  wrote:
>
> > Congrats!
> >
> > Best,
> > Paul Lam
> >
> > > 2024年3月28日 16:03，Rui Fan <1996fan...@gmail.com> 写道：
> > >
> > > Congratulations~
> > >
> > > Best,
> > > Rui
> > >
> > > On Thu, Mar 28, 2024 at 3:55 PM Yu Li  wrote:
> > >
> > >> CC the Flink user and dev mailing list.
> > >>
> > >> Paimon originated within the Flink community, initially known as Flink
> > >> Table Store, and all our incubating mentors are members of the Flink
> > >> Project Management Committee. I am confident that the bonds of
> > >> enduring friendship and close collaboration will continue to unite the
> > >> two communities.
> > >>
> > >> And congratulations all!
> > >>
> > >> Best Regards,
> > >> Yu
> > >>
> > >> On Wed, 27 Mar 2024 at 20:35, Guojun Li 
> > wrote:
> > >>>
> > >>> Congratulations!
> > >>>
> > >>> Best,
> > >>> Guojun
> > >>>
> > >>> On Wed, Mar 27, 2024 at 5:24 PM wulin  wrote:
> > >>>
> >  Congratulations~
> > 
> > > 2024年3月27日 15:54，王刚  写道：
> > >
> > > Congratulations~
> > >
> > >> 2024年3月26日 10:25，Jingsong Li  写道：
> > >>
> > >> Hi Paimon community,
> > >>
> > >> I’m glad to announce that the ASF board has approved a resolution
> to
> > >> graduate Paimon into a full Top Level Project. Thanks to everyone
> > >> for
> > >> your help to get to this point.
> > >>
> > >> I just created an issue to track the things we need to modify [2],
> > >> please comment on it if you feel that something is missing. You
> can
> > >> refer to apache documentation [1] too.
> > >>
> > >> And, we already completed the GitHub repo migration [3], please
> > >> update
> > >> your local git repo to track the new repo [4].
> > >>
> > >> You can run the following command to complete the remote repo
> > >> tracking
> > >> migration.
> > >>
> > >> git remote set-url origin https://github.com/apache/paimon.git
> > >>
> > >> If you have a different name, please change the 'origin' to your
> > >> remote
> >  name.
> > >>
> > >> Please join me in celebrating!
> > >>
> > >> [1]
> > 
> > >>
> >
> https://incubator.apache.org/guides/transferring.html#life_after_graduation
> > >> [2] https://github.com/apache/paimon/issues/3091
> > >> [3] https://issues.apache.org/jira/browse/INFRA-25630
> > >> [4] https://github.com/apache/paimon
> > >>
> > >> Best,
> > >> Jingsong Lee
> > 
> > 
> > >>
> >
> >
>

Re: [VOTE] FLIP-433: State Access on DataStream API V2

[Samrat Deb]

+1 (non binding)

Bests,
Samrat

On Sun, 24 Mar 2024 at 3:59 PM, Ahmed Hamdy  wrote:

> +1 (non-binding)
>
> Best Regards
> Ahmed Hamdy
>
>
> On Thu, 21 Mar 2024 at 10:45, Jinzhong Li 
> wrote:
>
> > +1 (non-binding)
> >
> > Best,
> > Jinzhong
> >
> > On Thu, Mar 21, 2024 at 6:15 PM Zakelly Lan 
> wrote:
> >
> > > +1 non-binding
> > >
> > >
> > > Best,
> > > Zakelly
> > >
> > > On Thu, Mar 21, 2024 at 5:34 PM Gyula Fóra 
> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Gyula
> > > >
> > > > On Thu, Mar 21, 2024 at 3:33 AM Rui Fan <1996fan...@gmail.com>
> wrote:
> > > >
> > > > > +1(binding)
> > > > >
> > > > > Thanks to Weijie for driving this proposal, which solves the
> problem
> > > > that I
> > > > > raised in FLIP-359.
> > > > >
> > > > > Best,
> > > > > Rui
> > > > >
> > > > > On Thu, Mar 21, 2024 at 10:10 AM Hangxiang Yu  >
> > > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > On Thu, Mar 21, 2024 at 10:04 AM Xintong Song <
> > tonysong...@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Best,
> > > > > > >
> > > > > > > Xintong
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 20, 2024 at 8:30 PM weijie guo <
> > > > guoweijieres...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi everyone,
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for all the feedback about the FLIP-433: State Access
> on
> > > > > > > > DataStream API V2 [1]. The discussion thread is here [2].
> > > > > > > >
> > > > > > > >
> > > > > > > > The vote will be open for at least 72 hours unless there is
> an
> > > > > > > > objection or insufficient votes.
> > > > > > > >
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > >
> > > > > > > > Weijie
> > > > > > > >
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-433%3A+State+Access+on+DataStream+API+V2
> > > > > > > >
> > > > > > > > [2]
> > > > https://lists.apache.org/thread/8ghzqkvt99p4k7hjqxzwhqny7zb7xrwo
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best,
> > > > > > Hangxiang.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

[Samrat Deb]

Congratulations !

Bests
Samrat

On Thu, 21 Mar 2024 at 5:52 PM, Ahmed Hamdy  wrote:

> Congratulations, great work and great news.
> Best Regards
> Ahmed Hamdy
>
>
> On Thu, 21 Mar 2024 at 11:41, Benchao Li  wrote:
>
> > Congratulations, and thanks for the great work!
> >
> > Yuan Mei  于2024年3月21日周四 18:31写道：
> > >
> > > Thanks for driving these efforts!
> > >
> > > Congratulations
> > >
> > > Best
> > > Yuan
> > >
> > > On Thu, Mar 21, 2024 at 4:35 PM Yu Li  wrote:
> > >
> > > > Congratulations and look forward to its further development!
> > > >
> > > > Best Regards,
> > > > Yu
> > > >
> > > > On Thu, 21 Mar 2024 at 15:54, ConradJam  wrote:
> > > > >
> > > > > Congrattulations!
> > > > >
> > > > > Leonard Xu  于2024年3月20日周三 21:36写道：
> > > > >
> > > > > > Hi devs and users,
> > > > > >
> > > > > > We are thrilled to announce that the donation of Flink CDC as a
> > > > > > sub-project of Apache Flink has completed. We invite you to
> explore
> > > > the new
> > > > > > resources available:
> > > > > >
> > > > > > - GitHub Repository: https://github.com/apache/flink-cdc
> > > > > > - Flink CDC Documentation:
> > > > > > https://nightlies.apache.org/flink/flink-cdc-docs-stable
> > > > > >
> > > > > > After Flink community accepted this donation[1], we have
> completed
> > > > > > software copyright signing, code repo migration, code cleanup,
> > website
> > > > > > migration, CI migration and github issues migration etc.
> > > > > > Here I am particularly grateful to Hang Ruan, Zhongqaing Gong,
> > > > Qingsheng
> > > > > > Ren, Jiabao Sun, LvYanquan, loserwang1024 and other contributors
> > for
> > > > their
> > > > > > contributions and help during this process！
> > > > > >
> > > > > >
> > > > > > For all previous contributors: The contribution process has
> > slightly
> > > > > > changed to align with the main Flink project. To report bugs or
> > > > suggest new
> > > > > > features, please open tickets
> > > > > > Apache Jira (https://issues.apache.org/jira).  Note that we will
> > no
> > > > > > longer accept GitHub issues for these purposes.
> > > > > >
> > > > > >
> > > > > > Welcome to explore the new repository and documentation. Your
> > feedback
> > > > and
> > > > > > contributions are invaluable as we continue to improve Flink CDC.
> > > > > >
> > > > > > Thanks everyone for your support and happy exploring Flink CDC!
> > > > > >
> > > > > > Best，
> > > > > > Leonard
> > > > > > [1]
> > https://lists.apache.org/thread/cw29fhsp99243yfo95xrkw82s5s418ob
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > Best
> > > > >
> > > > > ConradJam
> > > >
> >
> >
> >
> > --
> >
> > Best,
> > Benchao Li
> >
>

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

[Samrat Deb]

+1 (non-binding )

Bests,
Samrat

On Thu, Mar 21, 2024 at 12:55 PM Hang Ruan  wrote:

> +1 (non-binding)
>
> Best,
> Hang
>
> Őrhidi Mátyás  于2024年3月21日周四 00:00写道：
>
> > +1 (binding)
> >
> > On Wed, Mar 20, 2024 at 8:37 AM Gabor Somogyi  >
> > wrote:
> >
> > > +1 (binding)
> > >
> > > G
> > >
> > >
> > > On Wed, Mar 20, 2024 at 3:59 PM Gyula Fóra 
> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Thanks!
> > > > Gyula
> > > >
> > > > On Wed, Mar 20, 2024 at 3:36 PM Mate Czagany 
> > wrote:
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Thank you,
> > > > > Mate
> > > > >
> > > > > Ferenc Csaky  ezt írta (időpont: 2024.
> > > márc.
> > > > > 20., Sze, 15:11):
> > > > >
> > > > > > Hello devs,
> > > > > >
> > > > > > I would like to start a vote about FLIP-439 [1]. The FLIP is
> about
> > to
> > > > > > externalize the Kudu
> > > > > > connector from the recently retired Apache Bahir project [2] to
> > keep
> > > it
> > > > > > maintainable and
> > > > > > make it up to date as well. Discussion thread [3].
> > > > > >
> > > > > > The vote will be open for at least 72 hours (until 2024 March 23
> > > 14:03
> > > > > > UTC) unless there
> > > > > > are any objections or insufficient votes.
> > > > > >
> > > > > > Thanks,
> > > > > > Ferenc
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-439%3A+Externalize+Kudu+Connector+from+Bahir
> > > > > > [2] https://attic.apache.org/projects/bahir.html
> > > > > > [3]
> > https://lists.apache.org/thread/oydhcfkco2kqp4hdd1glzy5vkw131rkz
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] Apache Flink 1.19.0 released

[Samrat Deb]

Congratulations !

On Mon, 18 Mar 2024 at 2:07 PM, Jingsong Li  wrote:

> Congratulations!
>
> On Mon, Mar 18, 2024 at 4:30 PM Rui Fan <1996fan...@gmail.com> wrote:
> >
> > Congratulations, thanks for the great work!
> >
> > Best,
> > Rui
> >
> > On Mon, Mar 18, 2024 at 4:26 PM Lincoln Lee 
> wrote:
> >>
> >> The Apache Flink community is very happy to announce the release of
> Apache Flink 1.19.0, which is the fisrt release for the Apache Flink 1.19
> series.
> >>
> >> Apache Flink® is an open-source stream processing framework for
> distributed, high-performing, always-available, and accurate data streaming
> applications.
> >>
> >> The release is available for download at:
> >> https://flink.apache.org/downloads.html
> >>
> >> Please check out the release blog post for an overview of the
> improvements for this bugfix release:
> >>
> https://flink.apache.org/2024/03/18/announcing-the-release-of-apache-flink-1.19/
> >>
> >> The full release notes are available in Jira:
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353282
> >>
> >> We would like to thank all contributors of the Apache Flink community
> who made this release possible!
> >>
> >>
> >> Best,
> >> Yun, Jing, Martijn and Lincoln
>

Re: [VOTE] Release 1.19.0, release candidate #2

[Samrat Deb]

+1 (non binding)

- verified signatures and checksums
- ASF headers are present in all expected file
- No unexpected binaries files found in the source
- Build successful locally
- tested basic word count example




Bests,
Samrat

On Mon, 11 Mar 2024 at 7:33 PM, Ahmed Hamdy  wrote:

> Hi Lincoln
> +1 (non-binding) from me
>
> - Verified Checksums & Signatures
> - Verified Source dists don't contain binaries
> - Built source successfully
> - reviewed web PR
>
>
> Best Regards
> Ahmed Hamdy
>
>
> On Mon, 11 Mar 2024 at 15:18, Lincoln Lee  wrote:
>
> > Hi Robin,
> >
> > Thanks for helping verifying the release note[1], FLINK-14879 should not
> > have been included, after confirming this
> > I moved all unresolved non-blocker issues left over from 1.19.0 to 1.20.0
> > and reconfigured the release note [1].
> >
> > Best,
> > Lincoln Lee
> >
> > [1]
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353282
> >
> >
> > Robin Moffatt  于2024年3月11日周一 19:36写道：
> >
> > > Looking at the release notes [1] it lists `DESCRIBE DATABASE`
> > (FLINK-14879)
> > > and `DESCRIBE CATALOG` (FLINK-14690).
> > > When I try these in 1.19 RC2 the behaviour is as in 1.18.1, i.e. it is
> > not
> > > supported:
> > >
> > > ```
> > > [INFO] Execute statement succeed.
> > >
> > > Flink SQL> show catalogs;
> > > +-+
> > > |catalog name |
> > > +-+
> > > |   c_new |
> > > | default_catalog |
> > > +-+
> > > 2 rows in set
> > >
> > > Flink SQL> DESCRIBE CATALOG c_new;
> > > [ERROR] Could not execute SQL statement. Reason:
> > > org.apache.calcite.sql.validate.SqlValidatorException: Column 'c_new'
> not
> > > found in any table
> > >
> > > Flink SQL> show databases;
> > > +--+
> > > |database name |
> > > +--+
> > > | default_database |
> > > +--+
> > > 1 row in set
> > >
> > > Flink SQL> DESCRIBE DATABASE default_database;
> > > [ERROR] Could not execute SQL statement. Reason:
> > > org.apache.calcite.sql.validate.SqlValidatorException: Column
> > > 'default_database' not found in
> > > any table
> > > ```
> > >
> > > Is this an error in the release notes, or my mistake in interpreting
> > them?
> > >
> > > thanks, Robin.
> > >
> > >
> > > [1]
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353282
> > >
> > > On Thu, 7 Mar 2024 at 10:01, Lincoln Lee 
> wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > Please review and vote on the release candidate #2 for the version
> > > 1.19.0,
> > > > as follows:
> > > > [ ] +1, Approve the release
> > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > >
> > > > The complete staging area is available for your review, which
> includes:
> > > >
> > > > * JIRA release notes [1], and the pull request adding release note
> for
> > > > users [2]
> > > > * the official Apache source release and binary convenience releases
> to
> > > be
> > > > deployed to dist.apache.org [3], which are signed with the key with
> > > > fingerprint E57D30ABEE75CA06  [4],
> > > > * all artifacts to be deployed to the Maven Central Repository [5],
> > > > * source code tag "release-1.19.0-rc2" [6],
> > > > * website pull request listing the new release and adding
> announcement
> > > blog
> > > > post [7].
> > > >
> > > > The vote will be open for at least 72 hours. It is adopted by
> majority
> > > > approval, with at least 3 PMC affirmative votes.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353282
> > > > [2] https://github.com/apache/flink/pull/24394
> > > > [3] https://dist.apache.org/repos/dist/dev/flink/flink-1.19.0-rc2/
> > > > [4] https://dist.apache.org/repos/dist/release/flink/KEYS
> > > > [5]
> > > https://repository.apache.org/content/repositories/orgapacheflink-1709
> > > > [6] https://github.com/apache/flink/releases/tag/release-1.19.0-rc2
> > > > [7] https://github.com/apache/flink-web/pull/721
> > > >
> > > >
> > > > Best,
> > > > Yun, Jing, Martijn and Lincoln
> > > >
> > >
> >
>

Re: [VOTE] Accept Flink CDC into Apache Flink

[Samrat Deb]

+1 (non binding)

Bests,
Samrat

[jira] [Created] (FLINK-33995) Add test in test_file_sink.sh s3 StreamingFileSink for csv

[Samrat Deb (Jira)]

Samrat Deb created FLINK-33995:
--

 Summary: Add test in test_file_sink.sh s3 StreamingFileSink for 
csv 
 Key: FLINK-33995
 URL: https://issues.apache.org/jira/browse/FLINK-33995
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / FileSystem
Reporter: Samrat Deb


test_file_sink.sh s3 StreamingFileSink doesnt have coverage for csv format . 

this task will add new test case to cover when format is `csv`



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New Apache Flink Committer - Alexander Fedulov

[Samrat Deb]

Congratulations Alex

Re: [VOTE] FLIP-396: Trial to test GitHub Actions as an alternative for Flink's current Azure CI infrastructure

[Samrat Deb]

+1 (non binding)
Thanks for driving

On Tue, 12 Dec 2023 at 11:59 AM, Sergey Nuyanzin 
wrote:

> +1 (binding)
>
> Thanks for driving this
>
> On Tue, Dec 12, 2023, 07:22 Rui Fan <1996fan...@gmail.com> wrote:
>
> > +1(binding)
> >
> > Best,
> > Rui
> >
> > On Tue, Dec 12, 2023 at 11:58 AM weijie guo 
> > wrote:
> >
> > > Thanks Matthias for this efforts.
> > >
> > > +1(binding)
> > >
> > >
> > > Best regards,
> > >
> > > Weijie
> > >
> > >
> > > Matthias Pohl  于2023年12月11日周一 21:51写道：
> > >
> > > > Hi everyone,
> > > > I'd like to start a vote on FLIP-396 [1]. It covers enabling GitHub
> > > Actions
> > > > (GHA) in Apache Flink. This means that GHA workflows will run aside
> > from
> > > > the usual Azure CI workflows in a trial phase (which ends earliest
> with
> > > the
> > > > release of Flink 1.19). Azure CI will still serve as the project's
> > ground
> > > > of truth until the community decides in a final vote to switch to GHA
> > or
> > > > stick to Azure CI.
> > > >
> > > > The related discussion thread can be found in [2].
> > > >
> > > > The vote will remain open for at least 72 hours and only concluded if
> > > there
> > > > are no objections and enough (i.e. at least 3) binding votes.
> > > >
> > > > Matthias
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-396%3A+Trial+to+test+GitHub+Actions+as+an+alternative+for+Flink%27s+current+Azure+CI+infrastructure
> > > > [2] https://lists.apache.org/thread/h4cmv7l3y8mxx2t435dmq4ltco4sbrgb
> > > >
> > > > --
> > > >
> > > > [image: Aiven] 
> > > >
> > > > *Matthias Pohl*
> > > > Opensource Software Engineer, *Aiven*
> > > > matthias.p...@aiven.io|  +49 170 9869525
> > > > aiven.io    |   <
> > > https://www.facebook.com/aivencloud
> > > > >
> > > >      <
> > > > https://twitter.com/aiven_io>
> > > > *Aiven Deutschland GmbH*
> > > > Alexanderufer 3-7, 10117 Berlin
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > >
> >
>

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

[Samrat Deb]

That's really cool :)
+1 for the great addition

Bests,
Samrat

On Thu, 7 Dec 2023 at 9:20 AM, Jingsong Li  wrote:

> Wow, Cool, Nice
>
> CDC is playing an increasingly important role.
>
> +1
>
> Best,
> Jingsong
>
> On Thu, Dec 7, 2023 at 11:25 AM Leonard Xu  wrote:
> >
> > Dear Flink devs,
> >
> > As you may have heard, we at Alibaba (Ververica) are planning to donate
> CDC Connectors for the Apache Flink project[1] to the Apache Flink
> community.
> >
> > CDC Connectors for Apache Flink comprise a collection of source
> connectors designed specifically for Apache Flink. These connectors[2]
> enable the ingestion of changes from various databases using Change Data
> Capture (CDC), most of these CDC connectors are powered by Debezium[3].
> They support both the DataStream API and the Table/SQL API, facilitating
> the reading of database snapshots and continuous reading of transaction
> logs with exactly-once processing, even in the event of failures.
> >
> >
> > Additionally, in the latest version 3.0, we have introduced many
> long-awaited features. Starting from CDC version 3.0, we've built a
> Streaming ELT Framework available for streaming data integration. This
> framework allows users to write their data synchronization logic in a
> simple YAML file, which will automatically be translated into a Flink
> DataStreaming job. It emphasizes optimizing the task submission process and
> offers advanced functionalities such as whole database synchronization,
> merging sharded tables, and schema evolution[4].
> >
> >
> > I believe this initiative is a perfect match for both sides. For the
> Flink community, it presents an opportunity to enhance Flink's competitive
> advantage in streaming data integration, promoting the healthy growth and
> prosperity of the Apache Flink ecosystem. For the CDC Connectors project,
> becoming a sub-project of Apache Flink means being part of a neutral
> open-source community, which can attract a more diverse pool of
> contributors.
> >
> > Please note that the aforementioned points represent only some of our
> motivations and vision for this donation. Specific future operations need
> to be further discussed in this thread. For example, the sub-project name
> after the donation; we hope to name it Flink-CDC aiming to streaming data
> intergration through Apache Flink, following the naming convention of
> Flink-ML; And this project is managed by a total of 8 maintainers,
> including 3 Flink PMC members and 1 Flink Committer. The remaining 4
> maintainers are also highly active contributors to the Flink community,
> donating this project to the Flink community implies that their permissions
> might be reduced. Therefore, we may need to bring up this topic for further
> discussion within the Flink PMC. Additionally, we need to discuss how to
> migrate existing users and documents. We have a user group of nearly 10,000
> people and a multi-version documentation site need to migrate. We also need
> to plan for the migration of CI/CD processes and other specifics.
> >
> >
> > While there are many intricate details that require implementation, we
> are committed to progressing and finalizing this donation process.
> >
> >
> > Despite being Flink’s most active ecological project (as evaluated by
> GitHub metrics), it also boasts a significant user base. However, I believe
> it's essential to commence discussions on future operations only after the
> community reaches a consensus on whether they desire this donation.
> >
> >
> > Really looking forward to hear what you think!
> >
> >
> > Best,
> > Leonard (on behalf of the Flink CDC Connectors project maintainers)
> >
> > [1] https://github.com/ververica/flink-cdc-connectors
> > [2]
> https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-connectors.html
> > [3] https://debezium.io
> > [4]
> https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-pipeline.html
>

Re: [VOTE] Release flink-connector-aws, v4.2.0 release candidate #1

[Samrat Deb]

Hi danny ,

+1 (non binding)

Release notes look good
- Signatures and checksums verified
- pom versions checked
- Source build and tests verified

Bests,
Samrat

On Wed, 29 Nov 2023 at 12:11 PM, Leonard Xu  wrote:

> Thanks Danny for driving this, sorry for late verification.
>
> +1 (binding)
>
> - built from source code succeeded
> - verified signatures
> - verified hashsums
> - checked the contents contains jar and pom files in apache repo
> - checked Github release tag
> - reviewed the web PR with minor comment
>
> Best,
> Leonard
>
>
>
> > 2023年11月27日 上午1:13，Ahmed Hamdy  写道：
> >
> > Hi Danny
> > +1 (non-binding)
> >
> >
> > - Verified signatures and checksums
> > - verified no binaries exists in archive
> > - built source
> > - reviewed web PR
> > - Ran E2E example with Kinesis, Firehose & DynamoDB datastream
> connectors.
> >
> > Best Regards
> > Ahmed Hamdy
> >
> >
> > On Fri, 24 Nov 2023 at 08:38, Martijn Visser 
> > wrote:
> >
> >> Hi Danny,
> >>
> >> Thanks for driving this.
> >>
> >> +1 (binding)
> >>
> >> - Validated hashes
> >> - Verified signature
> >> - Verified that no binaries exist in the source archive
> >> - Build the source with Maven
> >> - Verified licenses
> >> - Verified web PRs
> >>
> >> On Mon, Nov 20, 2023 at 12:29 PM Jiabao Sun
> >>  wrote:
> >>>
> >>> Thanks Danny for driving the release,
> >>>
> >>> +1 (non-binding)
> >>>
> >>> - built from source code succeeded
> >>> - verified signatures
> >>> - verified hashsums
> >>> - checked release notes
> >>>
> >>> Best,
> >>> Jiabao
> >>>
> >>>
>  2023年11月20日 19:11，Danny Cranmer  写道：
> 
>  Hello all,
> 
>  +1 (binding).
> 
>  - Release notes look good
>  - Signatures and checksums match
>  - There are no binaries in the source archive
>  - pom versions are correct
>  - Tag is present in Github
>  - CI passes against FLink 1.17 and 1.18
>  - Source build and tests pass
> 
>  Thanks,
>  Danny
> 
>  On Wed, Nov 1, 2023 at 1:15 AM mystic lama 
> >> wrote:
> 
> > +1 (non-binding)
> >
> > - validated shasum
> > - verified build
> >  - Java 8   - build good and all test cases pass
> >  - Java 11 - build good and all test cases pass
> >
> > Observations: got test failures with Java 17, something to look for
> in
> > future
> >
> > On Tue, 31 Oct 2023 at 08:42, Danny Cranmer  >
> > wrote:
> >
> >> Hi everyone,
> >>
> >> Please review and vote on release candidate #1 for the version
> >> 4.2.0, as
> >> follows:
> >> [ ] +1, Approve the release
> >> [ ] -1, Do not approve the release (please provide specific
> comments)
> >>
> >> The complete staging area is available for your review, which
> >> includes:
> >> * JIRA release notes [1],
> >> * the official Apache source release to be deployed to
> >> dist.apache.org
> >> [2],
> >> which are signed with the key with fingerprint
> >> 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
> >> * all artifacts to be deployed to the Maven Central Repository [4],
> >> * source code tag v4.2.0-rc1 [5],
> >> * website pull request listing the new release [6].
> >> * A link to the CI run on the release tag [7]
> >>
> >> The vote will be open for at least 72 hours. It is adopted by
> >> majority
> >> approval, with at least 3 PMC affirmative votes.
> >>
> >> Thanks,
> >> Danny
> >>
> >> [1]
> >>
> >>
> >
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353011
> >> [2]
> >>
> >
> >>
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-aws-4.2.0-rc1
> >> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> >> [4]
> >>
> >> https://repository.apache.org/content/repositories/orgapacheflink-1665/
> >> [5]
> >
> https://github.com/apache/flink-connector-aws/releases/tag/v4.2.0-rc1
> >> [6] https://github.com/apache/flink-web/pull/693
> >> [7]
> >
> https://github.com/apache/flink-connector-aws/actions/runs/6707962074
> >>
> >
> >>>
> >>
>
>

Re: [VOTE] FLIP-381: Deprecate configuration getters/setters that return/set complex Java objects

[Samrat Deb]

+1 (non binding)

Bests,
Samrat

On Mon, 13 Nov 2023 at 9:10 AM, Yangze Guo  wrote:

> +1 (binding)
>
> Best,
> Yangze Guo
>
> On Mon, Nov 13, 2023 at 11:35 AM weijie guo 
> wrote:
> >
> > +1(binding)
> >
> > Best regards,
> >
> > Weijie
> >
> >
> > Lijie Wang  于2023年11月13日周一 10:40写道：
> >
> > > +1 (binding)
> > >
> > > Best,
> > > Lijie
> > >
> > > Yuepeng Pan  于2023年11月10日周五 18:32写道：
> > >
> > > > +1(non-binding)
> > > >
> > > > Best,
> > > > Roc
> > > >
> > > > On 2023/11/10 03:58:10 Junrui Lee wrote:
> > > > > Hi everyone,
> > > > >
> > > > > Thank you to everyone for the feedback on FLIP-381: Deprecate
> > > > configuration
> > > > > getters/setters that return/set complex Java objects[1] which has
> been
> > > > > discussed in this thread [2].
> > > > >
> > > > > I would like to start a vote for it. The vote will be open for at
> least
> > > > 72
> > > > > hours (excluding weekends) unless there is an objection or not
> enough
> > > > votes.
> > > > >
> > > > > [1]
> > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=278464992
> > > > > [2]
> https://lists.apache.org/thread/y5owjkfxq3xs9lmpdbl6d6jmqdgbjqxo
> > > > >
> > > >
> > >
>

Re: [VOTE] Release flink-connector-gcp-pubsub v3.0.2, release candidate #1

[Samrat Deb]

+1(non-binding)

- Checked release notes
- Verified checksums and signatures
- Verified no binaries in release
- Build connector from source

Bests,
Samrat

On Mon, 6 Nov 2023 at 8:20 PM, Ryan Skraba 
wrote:

> Hello! +1 (non-binding)
>
> One note: the parent pom still has 1.16.0 for the Maven property of
> flink.version for both 1.17 and 1.18 releases.
>
> I've validated the source for the RC1:
> flink-connector-gcp-pubsub-3.0.2-src.tgz at r65060
> * The sha512 checksum is OK.
> * The source file is signed correctly.
> * The signature 0F79F2AFB2351BC29678544591F9C1EC125FD8DB is found in the
> KEYS file, and on https://keyserver.ubuntu.com/
> * The source file is consistent with the GitHub tag v3.0.2-rc1, which
> corresponds to commit 4c6be836e6c0f36ef5711f12d7b935254e7d248d
> - The files explicitly excluded by create_pristine_sources (such as
> .gitignore and the submodule tools/releasing/shared) are not present.
> * Has a LICENSE file and a NOTICE file
> * Does not contain any compiled binaries.
>
> * The sources can be compiled and unit tests pass with flink.version 1.17.1
> and flink.version 1.18.0
>
> * Nexus has two staged artifact ids for 3.0.2-1.17 and 3.0.2-1.18
> - flink-connector-gcp-pubsub (.jar, -javadoc.jar, -sources.jar and .pom)
> - flink-connector-gcp-pubsub-parent (only .pom)
>
> I did a simple smoke test on an emulated Pub/Sub with the 1.18 version.
>
> All my best, Ryan Skraba
>

[jira] [Created] (FLINK-33451) Implement FileSytemAutoScalerStateStore

[Samrat Deb (Jira)]

Samrat Deb created FLINK-33451:
--

 Summary: Implement FileSytemAutoScalerStateStore
 Key: FLINK-33451
 URL: https://issues.apache.org/jira/browse/FLINK-33451
 Project: Flink
  Issue Type: Sub-task
Reporter: Samrat Deb


Implement a filesystem autoscaler state store that will be useful in autoscaler 
support for yarn 

- HDFS support 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS][FLINK-33240] Document deprecated options as well

[Samrat Deb]

Thanks for the proposal ,
+1 for adding deprecated identifier

[Thought] Can we have seperate section / page for deprecated configs ? Wdut
?


Bests,
Samrat


On Tue, 31 Oct 2023 at 3:44 PM, Alexander Fedulov <
alexander.fedu...@gmail.com> wrote:

> Hi Zhanghao,
>
> Thanks for the proposition.
> In general +1, this sounds like a good idea as long it is clear that the
> usage of these settings is discouraged.
> Just one minor concern - the configuration page is already very long, do
> you have a rough estimate of how many more options would be added with this
> change?
>
> Best,
> Alexander Fedulov
>
> On Mon, 30 Oct 2023 at 18:24, Matthias Pohl  .invalid>
> wrote:
>
> > Thanks for your proposal, Zhanghao Chen. I think it adds more
> transparency
> > to the configuration documentation.
> >
> > +1 from my side on the proposal
> >
> > On Wed, Oct 11, 2023 at 2:09 PM Zhanghao Chen  >
> > wrote:
> >
> > > Hi Flink users and developers,
> > >
> > > Currently, Flink won't generate doc for the deprecated options. This
> > might
> > > confuse users when upgrading from an older version of Flink: they have
> to
> > > either carefully read the release notes or check the source code for
> > > upgrade guidance on deprecated options.
> > >
> > > I propose to document deprecated options as well, with a "(deprecated)"
> > > tag placed at the beginning of the option description to highlight the
> > > deprecation status [1].
> > >
> > > Looking forward to your feedbacks on it.
> > >
> > > [1] https://issues.apache.org/jira/browse/FLINK-33240
> > >
> > > Best,
> > > Zhanghao Chen
> > >
> >
>

[jira] [Created] (FLINK-33420) Run e2e test fails Intermetiently with ClientCoordinationHandler : Unhandled exception

[Samrat Deb (Jira)]

Samrat Deb created FLINK-33420:
--

 Summary: Run e2e test fails Intermetiently with 
ClientCoordinationHandler : Unhandled exception
 Key: FLINK-33420
 URL: https://issues.apache.org/jira/browse/FLINK-33420
 Project: Flink
  Issue Type: Bug
Reporter: Samrat Deb


 

```
Oct 31 08:49:37 2023-10-31 08:49:33,348 ERROR 
org.apache.flink.runtime.rest.handler.job.coordination.ClientCoordinationHandler
 [] - Unhandled exception.
Oct 31 08:49:37 org.apache.flink.runtime.messages.FlinkJobNotFoundException: 
Could not find Flink job (8528fbf0d50c0f038653f6815d56f6fd)
Oct 31 08:49:37 at 
org.apache.flink.runtime.dispatcher.Dispatcher.getJobMasterGateway(Dispatcher.java:1450)
 ~[flink-dist-1.18-SNAPSHOT.jar:1.18-SNAPSHOT]
Oct 31 08:49:37 at 
org.apache.flink.runtime.dispatcher.Dispatcher.performOperationOnJobMasterGateway(Dispatcher.java:1465)
 ~[flink-dist-1.18-SNAPSHOT.jar:1.18-SNAPSHOT]
Oct 31 08:49:37 at 
org.apache.flink.runtime.dispatcher.Dispatcher.deliverCoordinationRequestToCoordinator(Dispatcher.java:1088)
 ~[flink-dist-1.18-SNAPSHOT.jar:1.18-SNAPSHOT]
Oct 31 08:49:37 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method) ~[?:1.8.0_382] 
```

log Link : 
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=54212=logs=af184cdd-c6d8-5084-0b69-7e9c67b35f7a=0f3adb59-eefa-51c6-2858-3654d9e0749d=ae4f8708-9994-57d3-c2d7-b892156e7812



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] Kubernetes Operator 1.7.0 release planning

[Samrat Deb]

Thank you Gyula

(+1 non-binding) in support of you taking on the role of release manager.

> I think this is reasonable as I am not aware of any big features / bug
fixes being worked on right now. Given the size of the changes related to
the autoscaler module refactor we should try to focus the remaining time on
testing.

I completely agree with you. Since the changes are quite extensive, it's
crucial to allocate more time for thorough testing and verification.

Regarding working with you for the release, I might not have the necessary
privileges for that.

However, I'd be more than willing to assist with testing the changes,
validating various features, and checking for any potential regressions in
the flink-kubernetes-operator.
Just let me know how I can support the testing efforts.

Bests,
Samrat


On Tue, 31 Oct 2023 at 12:59 AM, Gyula Fóra  wrote:

> Hi all!
>
> I would like to kick off the release planning for the operator 1.7.0
> release. We have added quite a lot of new functionality over the last few
> weeks and I think the operator is in a good state to kick this off.
>
> Based on the original release schedule we had Nov 1 as the proposed feature
> freeze date and Nov 7 as the date for the release cut / rc1.
>
> I think this is reasonable as I am not aware of any big features / bug
> fixes being worked on right now. Given the size of the changes related to
> the autoscaler module refactor we should try to focus the remaining time on
> testing.
>
> I am happy to volunteer as a release manager but I am of course open to
> working together with someone on this.
>
> What do you think?
>
> Cheers,
> Gyula
>

Re: [DISCUSS] AWS Connectors v4.2.0 release + 1.18 support

[Samrat Deb]

Hi Danny ,

Thank you for driving it.

+1 (non binding )


>  I am proposing we skip 4.1.0 for Flink 1.18 and go
straight to 4.2.0.

Does this imply that the 4.1.x series will be reserved for Flink 1.17, and
the 4.2.x series will correspond to Flink 1.18?

Bests,
Samrat


On Mon, Oct 30, 2023 at 7:32 PM Jing Ge  wrote:

> Hi Danny,
>
> +1 Thanks for driving it. Did you mean skip 4.1.1, since 4.1.0 has already
> been released?
>
> Best regards,
> Jing
>
> On Mon, Oct 30, 2023 at 11:49 AM Danny Cranmer 
> wrote:
>
> > Hello all,
> >
> > I would like to start the discussion to release Apache Flink AWS
> connectors
> > v4.2.0. We released v4.1.0 over six months ago on 2023-04-03. Since then
> we
> > have resolved 23 issues [1]. Additionally now Flink 1.18 is live we need
> to
> > add support for this. I am proposing we skip 4.1.0 for Flink 1.18 and go
> > straight to 4.2.0. The CI is stable [2].
> >
> > I volunteer myself as the release manager.
> >
> > Thanks,
> > Danny
> >
> > [1]
> >
> >
> https://issues.apache.org/jira/browse/FLINK-33021?jql=statusCategory%20%3D%20done%20AND%20project%20%3D%2012315522%20AND%20fixVersion%20%3D%2012353011%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
> > [2] https://github.com/apache/flink-connector-aws/actions
> >
>

Re: [VOTE] Apache Flink Kafka connector version 3.0.1, RC1

[Samrat Deb]

+1 (non-binding)

- Verified signatures
- Verified Checksum
- Build with Java 8 /11 - build success
- Started MSK cluster and EMR cluster with flink, successfully ran some
examples to read and write data to MSK.
- Checked release tag exists


Bests,
Samrat

On Mon, Oct 30, 2023 at 3:47 PM Ahmed Hamdy  wrote:

> +1 (non-binding)
> - Verified Singatures
> - Verified Checksum
> - Build source successfully
> - Checked release tag exists
> - Reviewed the web PR
> Best Regards
> Ahmed Hamdy
>
>
> On Sun, 29 Oct 2023 at 08:02, Leonard Xu  wrote:
>
> > +1 (binding)
> >
> > - Verified signatures
> > - Verified hashsums
> > - Checked Github release tag
> > - Built from source code succeeded
> > - Checked release notes
> > - Reviewed the web PR
> >
> > Best,
> > Leonard
> >
> >
> > > 2023年10月29日 上午11:34，mystic lama  写道：
> > >
> > > +1 (non-binding)
> > >
> > > - verified signatures
> > > - build with Java 8 and Java 11 - build success
> > >
> > > Minor observation
> > > - RAT check flagged that README.md is missing ASL
> > >
> > > On Fri, 27 Oct 2023 at 23:40, Xianxun Ye 
> > wrote:
> > >
> > >> +1(non-binding)
> > >>
> > >> - Started a local Flink 1.18 cluster, read and wrote with Kafka and
> > Upsert
> > >> Kafka connector successfully to Kafka 2.2 cluster
> > >>
> > >> One minor question: should we update the dependency manual of these
> two
> > >> documentations[1][2]?
> > >>
> > >> [1]
> > >>
> >
> https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/kafka/#dependencies
> > >> [2]
> > >>
> >
> https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/upsert-kafka/#dependencies
> > >>
> > >> Best regards,
> > >> Xianxun
> > >>
> > >>> 2023年10月26日 16:12，Martijn Visser  写道：
> > >>>
> > >>> +1 (binding)
> > >>>
> > >>> - Validated hashes
> > >>> - Verified signature
> > >>> - Verified that no binaries exist in the source archive
> > >>> - Build the source with Maven via mvn clean install
> > >>> -Pcheck-convergence -Dflink.version=1.18.0
> > >>> - Verified licenses
> > >>> - Verified web PR
> > >>> - Started a cluster and the Flink SQL client, successfully read and
> > >>> wrote with the Kafka connector to Confluent Cloud with AVRO and
> Schema
> > >>> Registry enabled
> > >>>
> > >>> On Thu, Oct 26, 2023 at 5:09 AM Qingsheng Ren 
> > wrote:
> > 
> >  +1 (binding)
> > 
> >  - Verified signature and checksum
> >  - Verified that no binary exists in the source archive
> >  - Built from source with Java 8 using -Dflink.version=1.18
> >  - Started a local Flink 1.18 cluster, submitted jobs with SQL client
> >  reading from and writing (with exactly-once) to Kafka 3.2.3 cluster
> >  - Nothing suspicious in LICENSE and NOTICE file
> >  - Reviewed web PR
> > 
> >  Thanks for the effort, Gordon!
> > 
> >  Best,
> >  Qingsheng
> > 
> >  On Thu, Oct 26, 2023 at 5:13 AM Tzu-Li (Gordon) Tai <
> > >> tzuli...@apache.org>
> >  wrote:
> > 
> > > Hi everyone,
> > >
> > > Please review and vote on release candidate #1 for version 3.0.1 of
> > the
> > > Apache Flink Kafka Connector, as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific
> comments)
> > >
> > > This release contains important changes for the following:
> > > - Supports Flink 1.18.x series
> > > - [FLINK-28303] EOS violation when using LATEST_OFFSETS startup
> mode
> > > - [FLINK-33231] Memory leak causing OOM when there are no offsets
> to
> > >> commit
> > > back to Kafka
> > > - [FLINK-28758] FlinkKafkaConsumer fails to stop with savepoint
> > >
> > > The release candidate contains the source release as well as JAR
> > >> artifacts
> > > to be released to Maven, built against Flink 1.17.1 and 1.18.0.
> > >
> > > The complete staging area is available for your review, which
> > includes:
> > > * JIRA release notes [1],
> > > * the official Apache source release to be deployed to
> > dist.apache.org
> > > [2],
> > > which are signed with the key with fingerprint
> > > 1C1E2394D3194E1944613488F320986D35C33D6A [3],
> > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > * source code tag v3.0.1-rc1 [5],
> > > * website pull request listing the new release [6].
> > >
> > > The vote will be open for at least 72 hours. It is adopted by
> > majority
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Gordon
> > >
> > > [1]
> > >
> > >
> > >>
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352910
> > > [2]
> > >
> > >
> > >>
> >
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-kafka-3.0.1-rc1/
> > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > > [4]
> > >>
> https://repository.apache.org/content/repositories/orgapacheflink-1664

[jira] [Created] (FLINK-33385) Flink connector redshift Table API implementation

[Samrat Deb (Jira)]

Samrat Deb created FLINK-33385:
--

 Summary: Flink connector redshift Table API implementation
 Key: FLINK-33385
 URL: https://issues.apache.org/jira/browse/FLINK-33385
 Project: Flink
  Issue Type: Sub-task
Reporter: Samrat Deb






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[Discuss] Proposal to add TTL Configuration in JobResultStore for cleanJobResults

[Samrat Deb]

Hi All,

The JobResultStore interface includes an Abstract implementation known as
`AbstractThreadsafeJobResultStore`.
This implementation is extended by two concrete implementations:
`FileSystemJobResultStore` and `EmbeddedJobResultStore`.
Both of these concrete implementations handle two types of job results:
clean and dirty.
The `EmbeddedJobResultStore` is an in-memory, thread-safe implementation
for managing job results. It currently allows the storage of both clean and
dirty jobs for further use.
However, it retains entries for clean job results indefinitely, which can
lead to unnecessary memory consumption.

To address this, we propose adding a new configuration option to allow
users to define a TTL (time-to-live) for map entries.
We are eager to hear feedback and thoughts from the Flink community
regarding introducing new public config.

During the review of PR[1] corresponding to JIRA[2], Matthias Pohl had an
idea to extend the clean-up functionality to both
`FileSystemJobResultStore` and `EmbeddedJobResultStore`.(Please check
review comments in PR[1] for more detail)
If the community finds this enhancement valuable, we plan to work on
implementing this clean-up improvement and potentially introduce a FLIP
(Flink Improvement Proposal) if required.

Your insights and opinions on this proposal are highly appreciated.

Bests,
Samrat

[1] https://github.com/apache/flink/pull/23531
[2] https://issues.apache.org/jira/browse/FLINK-32806

Re: [ANNOUNCE] Apache Flink 1.18.0 released

[Samrat Deb]

Congratulations on the great release

Bests,
Samrat

On Fri, 27 Oct 2023 at 7:59 AM, Yangze Guo  wrote:

> Great work! Congratulations to everyone involved!
>
> Best,
> Yangze Guo
>
> On Fri, Oct 27, 2023 at 10:23 AM Qingsheng Ren  wrote:
> >
> > Congratulations and big THANK YOU to everyone helping with this release!
> >
> > Best,
> > Qingsheng
> >
> > On Fri, Oct 27, 2023 at 10:18 AM Benchao Li 
> wrote:
> >>
> >> Great work, thanks everyone involved!
> >>
> >> Rui Fan <1996fan...@gmail.com> 于2023年10月27日周五 10:16写道：
> >> >
> >> > Thanks for the great work!
> >> >
> >> > Best,
> >> > Rui
> >> >
> >> > On Fri, Oct 27, 2023 at 10:03 AM Paul Lam 
> wrote:
> >> >
> >> > > Finally! Thanks to all!
> >> > >
> >> > > Best,
> >> > > Paul Lam
> >> > >
> >> > > > 2023年10月27日 03:58，Alexander Fedulov 
> 写道：
> >> > > >
> >> > > > Great work, thanks everyone!
> >> > > >
> >> > > > Best,
> >> > > > Alexander
> >> > > >
> >> > > > On Thu, 26 Oct 2023 at 21:15, Martijn Visser <
> martijnvis...@apache.org>
> >> > > > wrote:
> >> > > >
> >> > > >> Thank you all who have contributed!
> >> > > >>
> >> > > >> Op do 26 okt 2023 om 18:41 schreef Feng Jin <
> jinfeng1...@gmail.com>
> >> > > >>
> >> > > >>> Thanks for the great work! Congratulations
> >> > > >>>
> >> > > >>>
> >> > > >>> Best,
> >> > > >>> Feng Jin
> >> > > >>>
> >> > > >>> On Fri, Oct 27, 2023 at 12:36 AM Leonard Xu 
> wrote:
> >> > > >>>
> >> > >  Congratulations, Well done!
> >> > > 
> >> > >  Best,
> >> > >  Leonard
> >> > > 
> >> > >  On Fri, Oct 27, 2023 at 12:23 AM Lincoln Lee <
> lincoln.8...@gmail.com>
> >> > >  wrote:
> >> > > 
> >> > > > Thanks for the great work! Congrats all!
> >> > > >
> >> > > > Best,
> >> > > > Lincoln Lee
> >> > > >
> >> > > >
> >> > > > Jing Ge  于2023年10月27日周五 00:16写道：
> >> > > >
> >> > > >> The Apache Flink community is very happy to announce the
> release of
> >> > > > Apache
> >> > > >> Flink 1.18.0, which is the first release for the Apache
> Flink 1.18
> >> > > > series.
> >> > > >>
> >> > > >> Apache Flink® is an open-source unified stream and batch data
> >> > >  processing
> >> > > >> framework for distributed, high-performing,
> always-available, and
> >> > > > accurate
> >> > > >> data applications.
> >> > > >>
> >> > > >> The release is available for download at:
> >> > > >> https://flink.apache.org/downloads.html
> >> > > >>
> >> > > >> Please check out the release blog post for an overview of the
> >> > > > improvements
> >> > > >> for this release:
> >> > > >>
> >> > > >>
> >> > > >
> >> > > 
> >> > > >>>
> >> > > >>
> >> > >
> https://flink.apache.org/2023/10/24/announcing-the-release-of-apache-flink-1.18/
> >> > > >>
> >> > > >> The full release notes are available in Jira:
> >> > > >>
> >> > > >>
> >> > > >
> >> > > 
> >> > > >>>
> >> > > >>
> >> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352885
> >> > > >>
> >> > > >> We would like to thank all contributors of the Apache Flink
> >> > > >> community
> >> > >  who
> >> > > >> made this release possible!
> >> > > >>
> >> > > >> Best regards,
> >> > > >> Konstantin, Qingsheng, Sergey, and Jing
> >> > > >>
> >> > > >
> >> > > 
> >> > > >>>
> >> > > >>
> >> > >
> >> > >
> >>
> >>
> >>
> >> --
> >>
> >> Best,
> >> Benchao Li
>

Re: [VOTE] Release 1.18.0, release candidate #3

[Samrat Deb]

+1(non-binding)

- Downloaded artifacts from dist[1]
- Verified SHA512 checksums
- Verified GPG signatures
- Build the source with java 8 and 11

[1] https://dist.apache.org/repos/dist/dev/flink/flink-1.18.0-rc3/

Bests,
Samrat

On Tue, Oct 24, 2023 at 10:44 AM Jingsong Li  wrote:

> +1 (binding)
>
> - verified signatures & hash
> - built from source code succeeded
> - started SQL Client, used Paimon connector to write and read, the
> result is expected
>
> Best,
> Jingsong
>
> On Tue, Oct 24, 2023 at 12:15 PM Yuxin Tan  wrote:
> >
> > +1(non-binding)
> >
> > - Verified checksum
> > - Build from source code
> > - Verified signature
> > - Started a local cluster and run Streaming & Batch wordcount job, the
> > result is expected
> > - Verified web PR
> >
> > Best,
> > Yuxin
> >
> >
> > Qingsheng Ren  于2023年10月24日周二 11:19写道：
> >
> > > +1 (binding)
> > >
> > > - Verified checksums and signatures
> > > - Built from source with Java 8
> > > - Started a standalone cluster and submitted a Flink SQL job that read
> and
> > > wrote with Kafka connector and CSV / JSON format
> > > - Reviewed web PR and release note
> > >
> > > Best,
> > > Qingsheng
> > >
> > > On Mon, Oct 23, 2023 at 10:40 PM Leonard Xu  wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > - verified signatures
> > > > - verified hashsums
> > > > - built from source code succeeded
> > > > - checked all dependency artifacts are 1.18
> > > > - started SQL Client, used MySQL CDC connector to read changelog from
> > > > database , the result is expected
> > > > - reviewed the web PR, left minor comments
> > > > - reviewed the release notes PR, left minor comments
> > > >
> > > >
> > > > Best,
> > > > Leonard
> > > >
> > > > > 2023年10月21日 下午7:28，Rui Fan <1996fan...@gmail.com> 写道：
> > > > >
> > > > > +1(non-binding)
> > > > >
> > > > > - Downloaded artifacts from dist[1]
> > > > > - Verified SHA512 checksums
> > > > > - Verified GPG signatures
> > > > > - Build the source with java-1.8 and verified the licenses together
> > > > > - Verified web PR
> > > > >
> > > > > [1] https://dist.apache.org/repos/dist/dev/flink/flink-1.18.0-rc3/
> > > > >
> > > > > Best,
> > > > > Rui
> > > > >
> > > > > On Fri, Oct 20, 2023 at 10:31 PM Martijn Visser <
> > > > martijnvis...@apache.org>
> > > > > wrote:
> > > > >
> > > > >> +1 (binding)
> > > > >>
> > > > >> - Validated hashes
> > > > >> - Verified signature
> > > > >> - Verified that no binaries exist in the source archive
> > > > >> - Build the source with Maven
> > > > >> - Verified licenses
> > > > >> - Verified web PR
> > > > >> - Started a cluster and the Flink SQL client, successfully read
> and
> > > > >> wrote with the Kafka connector to Confluent Cloud with AVRO and
> Schema
> > > > >> Registry enabled
> > > > >>
> > > > >> On Fri, Oct 20, 2023 at 2:55 PM Matthias Pohl
> > > > >>  wrote:
> > > > >>>
> > > > >>> +1 (binding)
> > > > >>>
> > > > >>> * Downloaded artifacts
> > > > >>> * Built Flink from sources
> > > > >>> * Verified SHA512 checksums GPG signatures
> > > > >>> * Compared checkout with provided sources
> > > > >>> * Verified pom file versions
> > > > >>> * Verified that there are no pom/NOTICE file changes since RC1
> > > > >>> * Deployed standalone session cluster and ran WordCount example
> in
> > > > batch
> > > > >>> and streaming: Nothing suspicious in log files found
> > > > >>>
> > > > >>> On Thu, Oct 19, 2023 at 3:00 PM Piotr Nowojski <
> pnowoj...@apache.org
> > > >
> > > > >> wrote:
> > > > >>>
> > > >  +1 (binding)
> > > > 
> > > >  Best,
> > > >  Piotrek
> > > > 
> > > >  czw., 19 paź 2023 o 09:55 Yun Tang 
> napisał(a):
> > > > 
> > > > > +1 (non-binding)
> > > > >
> > > > >
> > > > >  *   Build from source code
> > > > >  *   Verify the pre-built jar packages were built with JDK8
> > > > >  *   Verify FLIP-291 with a standalone cluster, and it works
> fine
> > > > >> with
> > > > > StateMachine example.
> > > > >  *   Checked the signature
> > > > >  *   Viewed the PRs.
> > > > >
> > > > > Best
> > > > > Yun Tang
> > > > > 
> > > > > From: Cheng Pan 
> > > > > Sent: Thursday, October 19, 2023 14:38
> > > > > To: dev@flink.apache.org 
> > > > > Subject: RE: [VOTE] Release 1.18.0, release candidate #3
> > > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > We(the Apache Kyuubi community), verified that the Kyuubi Flink
> > > > >> engine
> > > > > works well[1] with Flink 1.18.0 RC3.
> > > > >
> > > > > [1] https://github.com/apache/kyuubi/pull/5465
> > > > >
> > > > > Thanks,
> > > > > Cheng Pan
> > > > >
> > > > >
> > > > > On 2023/10/19 00:26:24 Jing Ge wrote:
> > > > >> Hi everyone,
> > > > >>
> > > > >> Please review and vote on the release candidate #3 for the
> version
> > > > >> 1.18.0, as follows:
> > > > >> [ ] +1, Approve the release
> > > > >> [ ] -1, Do not approve the 

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

[Samrat Deb]

+1 (non binding)

- Verified checksum and  signatures,
- checked Helm repo
- Installed operator,
- tested woed count and state machine example

Bests,
Samrat


On Mon, 23 Oct 2023 at 9:35 PM, Mate Czagany  wrote:

> +1 (non-binding)
>
> - Verified checksums, signatures, no binary found in source
> - Verified Helm chart and Docker images
> - Tested autoscaler on 1.18 with reactive scaling
>
> Regards,
> Mate
>
> Gyula Fóra  ezt írta (időpont: 2023. okt. 23., H,
> 9:45):
>
> > +1 (binding)
> >
> > - Verified checksums, signatures, source release content
> > - Helm repo works correctly and points to the correct image / version
> > - Installed operator, ran stateful example
> >
> > Gyula
> >
> > On Sat, Oct 21, 2023 at 1:43 PM Rui Fan <1996fan...@gmail.com> wrote:
> >
> > > +1(non-binding)
> > >
> > > - Downloaded artifacts from dist
> > > - Verified SHA512 checksums
> > > - Verified GPG signatures
> > > - Build the source with java-11 and verified the licenses together
> > > - Verified that all POM files point to the same version.
> > > - Verified that chart and appVersion matches the target release
> > > - Verified that helm chart / values.yaml points to the RC docker image
> > > - Verified that RC repo works as Helm repo (helm repo add
> > > flink-operator-repo-1.6.1-rc1
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/flink/flink-kubernetes-operator-1.6.1-rc1/
> > > )
> > > - Verified Helm chart can be installed  (helm install
> > > flink-kubernetes-operator
> > > flink-operator-repo-1.6.1-rc1/flink-kubernetes-operator --set
> > > webhook.create=false)
> > > - Submitted the autoscaling demo, the autoscaler works well (kubectl
> > apply
> > > -f autoscaling.yaml)
> > > - Triggered a manual savepoint (update the yaml: savepointTriggerNonce:
> > > 101)
> > >
> > > Best,
> > > Rui
> > >
> > > On Sat, Oct 21, 2023 at 7:33 PM Rui Fan <1996fan...@gmail.com> wrote:
> > >
> > > > Hi Everyone,
> > > >
> > > > Please review and vote on the release candidate #1 for the version
> > 1.6.1
> > > of
> > > > Apache Flink Kubernetes Operator,
> > > > as follows:
> > > > [ ] +1, Approve the release
> > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > >
> > > > **Release Overview**
> > > >
> > > > As an overview, the release consists of the following:
> > > > a) Kubernetes Operator canonical source distribution (including the
> > > > Dockerfile), to be deployed to the release repository at
> > dist.apache.org
> > > > b) Kubernetes Operator Helm Chart to be deployed to the release
> > > repository
> > > > at dist.apache.org
> > > > c) Maven artifacts to be deployed to the Maven Central Repository
> > > > d) Docker image to be pushed to dockerhub
> > > >
> > > > **Staging Areas to Review**
> > > >
> > > > The staging areas containing the above mentioned artifacts are as
> > > follows,
> > > > for your review:
> > > > * All artifacts for a,b) can be found in the corresponding dev
> > repository
> > > > at dist.apache.org [1]
> > > > * All artifacts for c) can be found at the Apache Nexus Repository
> [2]
> > > > * The docker image for d) is staged on github [3]
> > > >
> > > > All artifacts are signed with the
> > > > key B2D64016B940A7E0B9B72E0D7D0528B28037D8BC [4]
> > > >
> > > > Other links for your review:
> > > > * source code tag "release-1.6.1-rc1" [5]
> > > > * PR to update the website Downloads page to
> > > > include Kubernetes Operator links [6]
> > > > * PR to update the doc version of flink-kubernetes-operator[7]
> > > >
> > > > **Vote Duration**
> > > >
> > > > The voting time will run for at least 72 hours.
> > > > It is adopted by majority approval, with at least 3 PMC affirmative
> > > votes.
> > > >
> > > > **Note on Verification**
> > > >
> > > > You can follow the basic verification guide here[8].
> > > > Note that you don't need to verify everything yourself, but please
> make
> > > > note of what you have tested together with your +- vote.
> > > >
> > > > [1]
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/flink/flink-kubernetes-operator-1.6.1-rc1/
> > > > [2]
> > > >
> > https://repository.apache.org/content/repositories/orgapacheflink-1663/
> > > > [3]
> > > >
> > >
> >
> https://github.com/apache/flink-kubernetes-operator/pkgs/container/flink-kubernetes-operator/139454270?tag=51eeae1
> > > > [4] https://dist.apache.org/repos/dist/release/flink/KEYS
> > > > [5]
> > > >
> > >
> >
> https://github.com/apache/flink-kubernetes-operator/tree/release-1.6.1-rc1
> > > > [6] https://github.com/apache/flink-web/pull/690
> > > > [7] https://github.com/apache/flink-kubernetes-operator/pull/687
> > > > [8]
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/Verifying+a+Flink+Kubernetes+Operator+Release
> > > >
> > > > Best,
> > > > Rui
> > > >
> > >
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Jane Chan

[Samrat Deb]

Congratulations Jane Chan :)

On Mon, 16 Oct 2023 at 7:30 AM, tison  wrote:

> Congrats! I noticed Jane has been around for a while; well-deserved.
>
> Best,
> tison.
>
>
> Jark Wu  于2023年10月16日周一 09:58写道：
>
> > Hi, everyone
> >
> > On behalf of the PMC, I'm very happy to announce Jane Chan as a new Flink
> > Committer.
> >
> > Jane started code contribution in Jan 2021 and has been active in the
> Flink
> > community since. She authored more than 60 PRs and reviewed more than 40
> > PRs. Her contribution mainly revolves around Flink SQL, including Plan
> > Advice (FLIP-280), operator-level state TTL (FLIP-292), and ALTER TABLE
> > statements (FLINK-21634). Jane participated deeply in development
> > discussions and also helped answer user question emails. Jane was also a
> > core contributor of Flink Table Store (now Paimon) when the project was
> in
> > the early days.
> >
> > Please join me in congratulating Jane Chan for becoming a Flink
> Committer!
> >
> > Best,
> > Jark Wu (on behalf of the Flink PMC)
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Ron Liu

[Samrat Deb]

Congratulations Ron Liu :)

On Mon, 16 Oct 2023 at 7:29 AM, tison  wrote:

> Congrats! Glad to see more and more committers on board :D
>
> Enjoy your journey ;-)
>
> Best,
> tison.
>
>
> Jark Wu  于2023年10月16日周一 09:57写道：
>
> > Hi, everyone
> >
> > On behalf of the PMC, I'm very happy to announce Ron Liu as a new Flink
> > Committer.
> >
> > Ron has been continuously contributing to the Flink project for many
> years,
> > authored and reviewed a lot of codes. He mainly works on Flink SQL parts
> > and drove several important FLIPs, e.g., USING JAR (FLIP-214), Operator
> > Fusion CodeGen (FLIP-315), Runtime Filter (FLIP-324). He has a great
> > knowledge of the Batch SQL and improved a lot of batch performance in the
> > past several releases. He is also quite active in mailing lists,
> > participating in discussions and answering user questions.
> >
> > Please join me in congratulating Ron Liu for becoming a Flink Committer!
> >
> > Best,
> > Jark Wu (on behalf of the Flink PMC)
> >
>

Re: [VOTE] FLIP-366: Support standard YAML for FLINK configuration

[Samrat Deb]

+1 (non binding )

On Fri, 13 Oct 2023 at 7:47 AM, Rui Fan <1996fan...@gmail.com> wrote:

> +1(binding)
>
> Best,
> Rui Fan
>
> On Fri, Oct 13, 2023 at 10:12 AM Junrui Lee  wrote:
>
> > Hi all,
> >
> > Thank you to everyone for the feedback on FLIP-366[1]: Support standard
> > YAML for FLINK configuration in the discussion thread [2].
> > I would like to start a vote for it. The vote will be open for at least
> 72
> > hours (excluding weekends, unless there is an objection or an
> insufficient
> > number of votes).
> >
> > Thanks,
> > Junrui
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-366%3A+Support+standard+YAML+for+FLINK+configuration
> > [2]https://lists.apache.org/thread/qfhcm7h8r5xkv38rtxwkghkrcxg0q7k5
> >
>

Re: [VOTE] FLIP-239: Port JDBC Connector to FLIP-27-143

[Samrat Deb]

+1 (non binding )

Bests,
Samrat

On Thu, Oct 12, 2023 at 11:14 AM Jing Ge  wrote:

> +1(binding) Thanks!
>
> Best regards,
> Jing
>
> On Wed, Oct 11, 2023 at 9:34 AM Yuepeng Pan  wrote:
>
> > +1(non-binding)
> > Thanks for your driving the voting thread.
> >
> > Best Regards.
> > Yuepeng Pan
> >
> > On 2023/10/06 16:33:40 Joao Boto wrote:
> > > Hi all, Thank you to everyone for the feedback on FLIP-239[1]. Based on
> > the
> > > discussion thread [2] and some offline discussions, we have come to a
> > > consensus on the design and are ready to take a vote to contribute this
> > to
> > > Flink. I'd like to start a vote for it. The vote will be open for at
> > least
> > > 72 hours(excluding weekends, unless there is an objection or an
> > > insufficient number of votes. [1]
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217386271
> > > [2]https://lists.apache.org/thread/yx833h5h3fjlyor0bfm32chy3sjw8hwt
> > Best,
> > > Joao Boto
> > >
> >
>

Re: [VOTE] FLIP-367: Support Setting Parallelism for Table/SQL Sources

[Samrat Deb]

+1 (non binding)

On Wed, 11 Oct 2023 at 1:29 PM, Martijn Visser 
wrote:

> +1 (binding)
>
> On Wed, Oct 11, 2023 at 9:15 AM Sergey Nuyanzin 
> wrote:
> >
> > +1 (binding)
> >
> > that's a nice improvement
> > thanks for driving that
> >
> > On Wed, Oct 11, 2023 at 8:42 AM Lincoln Lee 
> wrote:
> >
> > > +1 (binding)
> > >
> > > Best,
> > > Lincoln Lee
> > >
> > >
> > > Leonard Xu  于2023年10月10日周二 09:57写道：
> > >
> > > > +1（binding）
> > > >
> > > > Best,
> > > > Leonard
> > > >
> > > > > On Oct 9, 2023, at 9:45 PM, Jing Ge 
> > > wrote:
> > > > >
> > > > > +1(binding)
> > > > >
> > > > > Best Regards,
> > > > > Jing
> > > > >
> > > > > On Mon, Oct 9, 2023 at 10:40 AM Ahmed Hamdy 
> > > > wrote:
> > > > >
> > > > >> +1 (non-binding)
> > > > >> Best Regards
> > > > >> Ahmed Hamdy
> > > > >>
> > > > >>
> > > > >> On Mon, 9 Oct 2023 at 09:38, xiangyu feng 
> > > wrote:
> > > > >>
> > > > >>> +1 (non-binding)
> > > > >>>
> > > > >>> Feng Jin  于2023年10月9日周一 16:00写道：
> > > > >>>
> > > >  +1 (non-binding)
> > > > 
> > > >  Best,
> > > >  Feng
> > > > 
> > > >  On Mon, Oct 9, 2023 at 3:12 PM Yangze Guo 
> > > wrote:
> > > > 
> > > > > +1 (binding)
> > > > >
> > > > > Best,
> > > > > Yangze Guo
> > > > >
> > > > > On Mon, Oct 9, 2023 at 2:46 PM Yun Tang 
> wrote:
> > > > >>
> > > > >> +1 (binding)
> > > > >>
> > > > >> Best
> > > > >> Yun Tang
> > > > >> 
> > > > >> From: Weihua Hu 
> > > > >> Sent: Monday, October 9, 2023 12:03
> > > > >> To: dev@flink.apache.org 
> > > > >> Subject: Re: [VOTE] FLIP-367: Support Setting Parallelism for
> > > > >>> Table/SQL
> > > > > Sources
> > > > >>
> > > > >> +1 (binding)
> > > > >>
> > > > >> Best,
> > > > >> Weihua
> > > > >>
> > > > >>
> > > > >> On Mon, Oct 9, 2023 at 11:47 AM Shammon FY  >
> > > > >>> wrote:
> > > > >>
> > > > >>> +1 (binding)
> > > > >>>
> > > > >>>
> > > > >>> On Mon, Oct 9, 2023 at 11:12 AM Benchao Li <
> libenc...@apache.org
> > > > >>>
> > > > > wrote:
> > > > >>>
> > > >  +1 (binding)
> > > > 
> > > >  Zhanghao Chen  于2023年10月9日周一
> > > > >> 10:20写道：
> > > > >
> > > > > Hi All,
> > > > >
> > > > > Thanks for all the feedback on FLIP-367: Support Setting
> > > > > Parallelism
> > > > >>> for
> > > >  Table/SQL Sources [1][2].
> > > > >
> > > > > I'd like to start a vote for FLIP-367. The vote will be
> open
> > > >  until
> > > > > Oct
> > > >  12th 12:00 PM GMT) unless there is an objection or
> insufficient
> > > > > votes.
> > > > >
> > > > > [1]
> > > > 
> > > > >>>
> > > > >
> > > > 
> > > > >>>
> > > > >>
> > > >
> > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263429150
> > > > > [2]
> > > > >
> https://lists.apache.org/thread/gtpswl42jzv0c9o3clwqskpllnw8rh87
> > > > >
> > > > > Best,
> > > > > Zhanghao Chen
> > > > 
> > > > 
> > > > 
> > > >  --
> > > > 
> > > >  Best,
> > > >  Benchao Li
> > > > 
> > > > >>>
> > > > >
> > > > 
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Sergey
>

Re: [VOTE] FLIP-374: Adding a separate configuration for specifying Java Options of the SQL Gateway

[Samrat Deb]

+1 (non binding )

On Wed, 11 Oct 2023 at 12:02 PM, Jing Ge  wrote:

> +1(binding)
>
> Best Regards,
> Jing
>
> On Wed, Oct 11, 2023 at 5:36 AM Zakelly Lan  wrote:
>
> > +1(non-binding)
> >
> > Thanks for driving this.
> >
> > Best,
> > Zakelly
> >
> > On Wed, Oct 11, 2023 at 11:22 AM Weihua Hu 
> wrote:
> > >
> > > +1(binding)
> > >
> > > Best,
> > > Weihua
> > >
> > >
> > > On Wed, Oct 11, 2023 at 10:56 AM xiangyu feng 
> > wrote:
> > >
> > > > +1(non-binding)
> > > >
> > > > Regards,
> > > > Xiangyu
> > > >
> > > > Shammon FY  于2023年10月11日周三 10:30写道：
> > > >
> > > > > +1(binding), good job!
> > > > >
> > > > > Best,
> > > > > Shammon FY
> > > > >
> > > > > On Wed, Oct 11, 2023 at 10:18 AM Benchao Li 
> > > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Rui Fan <1996fan...@gmail.com> 于2023年10月11日周三 10:17写道：
> > > > > > >
> > > > > > > +1(binding)
> > > > > > >
> > > > > > > Best,
> > > > > > > Rui
> > > > > > >
> > > > > > > On Wed, Oct 11, 2023 at 10:07 AM Yangze Guo <
> karma...@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi everyone,
> > > > > > > >
> > > > > > > > I'd like to start the vote of FLIP-374 [1]. This FLIP is
> > discussed
> > > > in
> > > > > > > > the thread [2].
> > > > > > > >
> > > > > > > > The vote will be open for at least 72 hours. Unless there is
> an
> > > > > > > > objection, I'll try to close it by October 16, 2023 if we
> have
> > > > > > > > received sufficient votes.
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > >
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-374%3A+Adding+a+separate+configuration+for+specifying+Java+Options+of+the+SQL+Gateway
> > > > > > > > [2]
> > > > https://lists.apache.org/thread/g4vl8mgnwgl7vjyvjy6zrc8w54b2lthv
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Yangze Guo
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Best,
> > > > > > Benchao Li
> > > > > >
> > > > >
> > > >
> >
>

[jira] [Created] (FLINK-33134) Flink Connector redshift E2E test

[Samrat Deb (Jira)]

Samrat Deb created FLINK-33134:
--

 Summary: Flink Connector redshift E2E test 
 Key: FLINK-33134
 URL: https://issues.apache.org/jira/browse/FLINK-33134
 Project: Flink
  Issue Type: Sub-task
Reporter: Samrat Deb






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-33133) Flink Document Update

[Samrat Deb (Jira)]

Samrat Deb created FLINK-33133:
--

 Summary: Flink Document Update 
 Key: FLINK-33133
 URL: https://issues.apache.org/jira/browse/FLINK-33133
 Project: Flink
  Issue Type: Sub-task
Reporter: Samrat Deb






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-33132) Flink Connector Redshift Sink Implementation

[Samrat Deb (Jira)]

Samrat Deb created FLINK-33132:
--

 Summary: Flink Connector Redshift Sink Implementation 
 Key: FLINK-33132
 URL: https://issues.apache.org/jira/browse/FLINK-33132
 Project: Flink
  Issue Type: Sub-task
Reporter: Samrat Deb






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [Discuss] FLIP-366: Support standard YAML for FLINK configuration

[Samrat Deb]

Hello Junrui,

+1 for the proposal.


Bests,
Samrat

On Fri, Sep 22, 2023 at 10:18 AM Shammon FY  wrote:

> +1 for the proposal, thanks for driving.
>
> Bet,
> Shammon FY
>
> On Fri, Sep 22, 2023 at 12:41 PM Yangze Guo  wrote:
>
> > Thanks for driving this, +1 for the proposal.
> >
> > Best,
> > Yangze Guo
> >
> >
> > On Fri, Sep 22, 2023 at 11:59 AM Lijie Wang 
> > wrote:
> > >
> > > Hi Junrui,
> > >
> > > +1 for this proposal, thanks for driving.
> > >
> > > Best,
> > > Lijie
> > >
> > > ConradJam  于2023年9月22日周五 10:07写道：
> > >
> > > > +1 Support for standard YAML format facilitates specification
> > > >
> > > > Jing Ge  于2023年9月22日周五 02:23写道：
> > > >
> > > > > Hi Junrui,
> > > > >
> > > > > +1 for following the standard. Thanks for your effort!
> > > > >
> > > > > Best regards,
> > > > > Jing
> > > > >
> > > > > On Thu, Sep 21, 2023 at 5:09 AM Junrui Lee 
> > wrote:
> > > > >
> > > > > > Hi Jane,
> > > > > >
> > > > > > Thank you for your valuable feedback and suggestions.
> > > > > > I agree with your point about differentiating between
> > > > "flink-config.yaml"
> > > > > > and "flink-conf.yaml" to determine the standard syntax at a
> glance.
> > > > > >
> > > > > > While I understand your suggestion of using
> > "flink-conf-default.yaml"
> > > > to
> > > > > > represent the default YAML file for Flink 1.x, I have been
> > considering
> > > > > > the option of using "flink-configuration.yaml" as the file name
> > for the
> > > > > > new configuration file.
> > > > > > This name "flink-configuration.yaml" provides a clear distinction
> > > > between
> > > > > > the new and old configuration files based on their names, and it
> > does
> > > > not
> > > > > > introduce any additional semantics. Moreover, this name
> > > > > > "flink-configuration.yaml" can continue to be used in future
> > versions
> > > > > > FLINK-2.0.
> > > > > >
> > > > > > WDYT? If we can reach a consensus on this, I will update the FLIP
> > > > > > documentation
> > > > > > accordingly.
> > > > > >
> > > > > > Best regards,
> > > > > > Junrui
> > > > > >
> > > > > > Jane Chan  于2023年9月20日周三 23:38写道：
> > > > > >
> > > > > > > Hi Junrui,
> > > > > > >
> > > > > > > Thanks for driving this FLIP. +1 for adoption of the standard
> > YAML
> > > > > > syntax.
> > > > > > > I just have one minor suggestion. It's a little bit challenging
> > to
> > > > > > > differentiate between `flink-config.yaml` and `flink-conf.yaml`
> > to
> > > > > > > determine which one uses the standard syntax at a glance. How
> > about
> > > > > > > using `flink-conf-default.yaml` to represent the default yaml
> > file
> > > > for
> > > > > > > Flink 1.x?
> > > > > > >
> > > > > > > Best,
> > > > > > > Jane
> > > > > > >
> > > > > > > On Wed, Sep 20, 2023 at 11:06 AM Junrui Lee <
> jrlee@gmail.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi devs,
> > > > > > > >
> > > > > > > > I would like to start a discussion about FLIP-366:
> > > > > > > > Support standard YAML for FLINK configuration[1]
> > > > > > > >
> > > > > > > > The current flink-conf.yaml parser in FLINK is not a standard
> > YAML
> > > > > > > parser,
> > > > > > > > which has some shortcomings.
> > > > > > > > Firstly, it does not support nested structure configuration
> > items
> > > > and
> > > > > > > only
> > > > > > > > supports key-value pairs, resulting in poor readability.
> > Secondly,
> > > > if
> > > > > > the
> > > > > > > > value is a collection type, such as a List or Map, users are
> > > > required
> > > > > > to
> > > > > > > > write the value in a FLINK-specific pattern, which is
> > inconvenient
> > > > to
> > > > > > > use.
> > > > > > > > Additionally, the parser of FLINK has some differences in
> > syntax
> > > > > > compared
> > > > > > > > to the standard YAML parser, such as the syntax for parsing
> > > > comments
> > > > > > and
> > > > > > > > null values. These inconsistencies can cause confusion for
> > users,
> > > > as
> > > > > > seen
> > > > > > > > in FLINK-15358 and FLINK-32740.
> > > > > > > >
> > > > > > > > By supporting standard YAML, these issues can be resolved,
> and
> > > > users
> > > > > > can
> > > > > > > > create a Flink configuration file using third-party tools and
> > > > > leverage
> > > > > > > > some advanced YAML features. Therefore, we propose to support
> > > > > standard
> > > > > > > > YAML for FLINK configuration.
> > > > > > > >
> > > > > > > > You can find more details in the FLIP-366[1]. Looking forward
> > to
> > > > your
> > > > > > > > feedback.
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-366%3A+Support+standard+YAML+for+FLINK+configuration
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Junrui
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best
> > > >
> > > > ConradJam
> > > >
> >
>

[RESULT][VOTE] FLIP-307: Flink Connector Redshift

[Samrat Deb]

Hi Everyone,

The proposal, FLIP-307: Flink Connector Redshift, has been accepted
with 5 votes (4 binding) .

+1 votes:
- Danny Cranmer (binding)
- Jing Ge (binding)
- Ahmed Hamdy (non-binding)
- Martijn Visser (binding)
- Leonard Xu (binding)


Bests,
Samrat

Re: [VOTE] FLIP-307: Flink Connector Redshift

[Samrat Deb]

Thank you ,
closing the vote.
I will share the result in a separate thread.

Bests,
Samrat


On Wed, Sep 20, 2023 at 7:22 PM Leonard Xu  wrote:

> +1 (binding)
>
>
> Best,
> Leonard
>
> > On Sep 18, 2023, at 11:53 PM, Ahmed Hamdy  wrote:
> >
> > +1 (non-binding)
> > Best Regards
> > Ahmed Hamdy
> >
> >
> > On Mon, 18 Sept 2023 at 16:52, Jing Ge 
> wrote:
> >
> >> +1(binding). Thanks!
> >>
> >> Best regards,
> >> Jing
> >>
> >> On Mon, Sep 18, 2023 at 5:26 PM Danny Cranmer 
> >> wrote:
> >>
> >>> Thanks for driving this Samrat!
> >>>
> >>> +1 (binding)
> >>>
> >>> Thanks,
> >>> Danny
> >>>
> >>> On Mon, Sep 18, 2023 at 4:17 AM Samrat Deb 
> >> wrote:
> >>>
> >>>> Hi All,
> >>>>
> >>>> Thanks for all the feedback on FLIP-307: Flink Connector Redshift
> >> [1][2]
> >>>>
> >>>> I'd like to start a vote for FLIP-307. The vote will be open for at
> >> least
> >>>> 72
> >>>> hours unless there is an objection or insufficient votes.
> >>>>
> >>>> Bests,
> >>>> Samrat
> >>>>
> >>>> [1]
> >>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
> >>>> [2] https://lists.apache.org/thread/wsz4jgdpnlyw1x781f9qpk7y416b45dj
> >>>>
> >>>
> >>
>
>

Re: [VOTE] FLIP-312: Prometheus Sink Connector

[Samrat Deb]

Thank you ,

+1 (non binding)

Bests,
Samrat

On Thu, Sep 21, 2023 at 9:43 AM Leonard Xu  wrote:

> Thanks Lorenzo for driving this.
>
> +1(binding)
>
> Best,
> Leonard
>
> > On Sep 21, 2023, at 11:47 AM, Yun Tang  wrote:
> >
> > +1 (binding)
> >
> > Thanks for driving this, Lorenzo.
> >
> > Best
> > Yun Tang
> > 
> > From: Hong 
> > Sent: Thursday, September 21, 2023 1:22
> > To: dev@flink.apache.org 
> > Subject: Re: [VOTE] FLIP-312: Prometheus Sink Connector
> >
> > +1 (binding)
> >
> > Thanks Lorenzo.
> >
> > Hong
> >
> >> On 20 Sep 2023, at 17:49, Danny Cranmer 
> wrote:
> >>
> >> +1 binding.
> >>
> >> Thanks for picking this up Lorenzo!
> >>
> >> Danny
> >>
> >>
> >>> On Wed, 20 Sept 2023, 16:33 Jing Ge, 
> wrote:
> >>>
> >>> +1(binding) Thanks!
> >>>
> >>> Best regards,
> >>> Jing
> >>>
> >>> On Wed, Sep 20, 2023 at 3:20 PM Martijn Visser <
> martijnvis...@apache.org>
> >>> wrote:
> >>>
>  +1 (binding)
> 
>  Thanks for driving this. Cheers, M
> 
>  On Mon, Sep 18, 2023 at 1:51 PM Lorenzo Nicora <
> lorenzo.nic...@gmail.com
> 
>  wrote:
> >
> > Hi All,
> >
> > Thanks for the feedback on FLIP-312: Prometheus Sink Connector [1].
> > We updated the FLIP accordingly [2].
> >
> > I would like to open the vote on FLIP-312.
> > The vote will be open for at least 72 hours unless there is an
> >>> objection
>  or
> > insufficient votes.
> >
> >
> > [1] https://lists.apache.org/thread/tm4qqfb4fxr7bc6nq5mwty1fqz8sj39x
> > [2]
> >
> 
> >>>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-312%3A+Prometheus+Sink+Connector
> >
> > Regards
> > Lorenzo Nicora
> 
> >>>
> >
>
>

[VOTE] FLIP-307: Flink Connector Redshift

[Samrat Deb]

Hi All,

Thanks for all the feedback on FLIP-307: Flink Connector Redshift [1][2]

I'd like to start a vote for FLIP-307. The vote will be open for at least 72
hours unless there is an objection or insufficient votes.

Bests,
Samrat

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
[2] https://lists.apache.org/thread/wsz4jgdpnlyw1x781f9qpk7y416b45dj

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi ,

I've made updates to the FLIP[1] by incorporating relevant changes to avoid
using the Flink connector JDBC. This decision was based on the following
reasons:

AWS Redshift utilizes its specialized JDBC driver[2]. Given that their JDBC
driver may undergo evolutions over time, using the Flink connector JDBC
might face compatibility issues.
Considering the dependency issues mentioned in the thread.
Additionally, a Proof of Concept (POC) has been implemented for a
DynamicSink for Redshift[3]. This showcases that utilizing
flink-connector-jdbc for Redshift doesn't provide significant benefits.
The JDBC mode offers more flexibility by allowing direct use of the JDBC
driver, enabling Flink connector Redshift to evolve independently.

[1] -
https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
[2] -
https://docs.aws.amazon.com/redshift/latest/mgmt/jdbc20-download-driver.html
[3] -
https://github.com/Samrat002/flink-connector-aws/tree/redshift-connector

Bests,
Samrat




On Sat, Sep 16, 2023 at 10:46 AM Samrat Deb  wrote:

> Hello Martijn,
>
> I apologize for the delay in responding.
>
> Regarding your question about integrating Redshift directly into the JDBC
> connector, we are planning to offer two modes: JDBC and UNLOAD. Through our
> internal benchmarking, we have observed good performance in the UNLOAD
> flow. Additionally, there is a need for both flows based on different user
> use cases.
>
> If we were to explicitly add the JDBC mode to the flink-connector-jdbc, we
> would have two options:
>
> 1. Include flink-connector-redshift in flink-connector-jdbc: This would
> involve incorporating the Redshift connector into the JDBC connector. Since
> Redshift is an AWS proprietary product, some authentication utilities can
> be utilized from flink-connector-aws-base. If additional utilities are
> required from the Redshift connector, they could be added to
> flink-connector-aws-base. In my opinion, this approach is favorable as it
> keeps everything related to AWS in flink-connector-aws.
>
> 2. Implement JDBC mode for Redshift sink in flink-connector-jdbc and
> UNLOAD in flink-connector-aws: This alternative is not advisable as it
> could lead to maintenance challenges and complexities.
>
>
> Furthermore, it's important to highlight that Redshift has its own
> customized JDBC driver[1], specifically optimized for compatibility with
> Redshift. While I cannot confirm this definitively, there is a possibility
> that the Redshift JDBC driver [1] might have differences in compatibility
> when compared to the JDBC driver used in flink-connector-jdbc. This
> suggests that if flink-connector-redshift were to rely on the JDBC
> connector, it could potentially lead to future compatibility issues.
>
> Given these considerations, it seems prudent to maintain the
> Redshift-related functionality within flink-connector-aws and keep the
> Redshift connector independent of the JDBC connector. This approach can
> help ensure that the Redshift connector remains flexible and adaptable to
> any potential changes in JDBC compatibility.
>
> I will update the FLIP[2] to remove dependencies on flink-connector-jdbc.
>
> [1]
> https://docs.aws.amazon.com/redshift/latest/mgmt/jdbc20-download-driver.html
> [2]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
>
> Bests,
> Samrat
>
>
> On Mon, Sep 11, 2023 at 1:21 PM Martijn Visser 
> wrote:
>
>> Hi Samrat,
>>
>> I'm still having doubts about the dependency on the JDBC connector. When a
>> user specifies 'read mode', it will use the JDBC connector under the hood.
>> Why not integrate Redshift then directly in the JDBC connector itself? It
>> removes the need for a dependency on the JDBC driver, especially keeping
>> in
>> mind that this driver uses the old SourceFunction/SinkFunction interfaces
>> because it hasn't been migrated yet.
>>
>> Best regards,
>>
>> Martijn
>>
>> On Mon, Sep 11, 2023 at 8:54 AM Samrat Deb  wrote:
>>
>> > Hi Leonard,
>> >
>> > > Do we have to rely on the latest version of JDBC Connector here?
>> >
>> > No, there's no need for us to depend on the latest version of the JDBC
>> > Connector. Redshift has its dedicated JDBC driver [1], which includes
>> > custom modifications tailored to Redshift's specific implementation
>> needs.
>> > This driver is the most suitable choice for our purposes.
>> >
>> >
>> > > Could you collect the APIs that Redshift generally needs to use?
>> >
>> > I am actively working on it and making progress towards creating the
>> POC.
>> >
>> > Bests,

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hello Martijn,

I apologize for the delay in responding.

Regarding your question about integrating Redshift directly into the JDBC
connector, we are planning to offer two modes: JDBC and UNLOAD. Through our
internal benchmarking, we have observed good performance in the UNLOAD
flow. Additionally, there is a need for both flows based on different user
use cases.

If we were to explicitly add the JDBC mode to the flink-connector-jdbc, we
would have two options:

1. Include flink-connector-redshift in flink-connector-jdbc: This would
involve incorporating the Redshift connector into the JDBC connector. Since
Redshift is an AWS proprietary product, some authentication utilities can
be utilized from flink-connector-aws-base. If additional utilities are
required from the Redshift connector, they could be added to
flink-connector-aws-base. In my opinion, this approach is favorable as it
keeps everything related to AWS in flink-connector-aws.

2. Implement JDBC mode for Redshift sink in flink-connector-jdbc and UNLOAD
in flink-connector-aws: This alternative is not advisable as it could lead
to maintenance challenges and complexities.


Furthermore, it's important to highlight that Redshift has its own
customized JDBC driver[1], specifically optimized for compatibility with
Redshift. While I cannot confirm this definitively, there is a possibility
that the Redshift JDBC driver [1] might have differences in compatibility
when compared to the JDBC driver used in flink-connector-jdbc. This
suggests that if flink-connector-redshift were to rely on the JDBC
connector, it could potentially lead to future compatibility issues.

Given these considerations, it seems prudent to maintain the
Redshift-related functionality within flink-connector-aws and keep the
Redshift connector independent of the JDBC connector. This approach can
help ensure that the Redshift connector remains flexible and adaptable to
any potential changes in JDBC compatibility.

I will update the FLIP[2] to remove dependencies on flink-connector-jdbc.

[1]
https://docs.aws.amazon.com/redshift/latest/mgmt/jdbc20-download-driver.html
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift

Bests,
Samrat


On Mon, Sep 11, 2023 at 1:21 PM Martijn Visser 
wrote:

> Hi Samrat,
>
> I'm still having doubts about the dependency on the JDBC connector. When a
> user specifies 'read mode', it will use the JDBC connector under the hood.
> Why not integrate Redshift then directly in the JDBC connector itself? It
> removes the need for a dependency on the JDBC driver, especially keeping in
> mind that this driver uses the old SourceFunction/SinkFunction interfaces
> because it hasn't been migrated yet.
>
> Best regards,
>
> Martijn
>
> On Mon, Sep 11, 2023 at 8:54 AM Samrat Deb  wrote:
>
> > Hi Leonard,
> >
> > > Do we have to rely on the latest version of JDBC Connector here?
> >
> > No, there's no need for us to depend on the latest version of the JDBC
> > Connector. Redshift has its dedicated JDBC driver [1], which includes
> > custom modifications tailored to Redshift's specific implementation
> needs.
> > This driver is the most suitable choice for our purposes.
> >
> >
> > > Could you collect the APIs that Redshift generally needs to use?
> >
> > I am actively working on it and making progress towards creating the POC.
> >
> > Bests,
> > Samrat
> >
> > [1]
> >
> >
> https://docs.aws.amazon.com/redshift/latest/mgmt/jdbc20-download-driver.html
> >
> > On Mon, Sep 11, 2023 at 12:02 PM Samrat Deb 
> wrote:
> >
> > > Hello Danny,
> > >
> > > I wanted to express my gratitude for your valuable feedback and
> > insightful
> > > suggestions.
> > >
> > > I will be revising the FLIP to incorporate all of your queries and
> review
> > > suggestions. Additionally, I plan to provide a Proof of Concept (POC)
> for
> > > the connector by the end of this week. This POC will address the points
> > > you've raised and ensure that the FLIP aligns with your
> recommendations.
> > >
> > > Thank you once again for your input.
> > >
> > > Bests,
> > > Samrat
> > >
> > > On Thu, Sep 7, 2023 at 10:21 PM Danny Cranmer  >
> > > wrote:
> > >
> > >> Hello Leonard,
> > >>
> > >> > Do we have to rely on the latest version of JDBC Connector here? I
> > >> understand that as long as the version of flink minor is the same as
> the
> > >> JDBC Connector, Could you collect the APIs that Redshift generally
> needs
> > >> to
> > >> use?
> > >>
> > >> I agree we do not necessar

Re: [VOTE] FLIP-361: Improve GC Metrics

[Samrat Deb]

+1 ( non binding)

These improved GC metrics will be a great addition.

Bests,
Samrat

On Wed, 13 Sep 2023 at 7:58 PM, ConradJam  wrote:

> +1 (non-binding)
> gc metrics help with autoscale tuning features
>
> Chen Zhanghao  于2023年9月13日周三 22:16写道：
>
> > +1 (unbinding). Looking forward to it
> >
> > Best,
> > Zhanghao Chen
> > 
> > 发件人: Gyula Fóra 
> > 发送时间: 2023年9月13日 21:16
> > 收件人: dev 
> > 主题: [VOTE] FLIP-361: Improve GC Metrics
> >
> > Hi All!
> >
> > Thanks for all the feedback on FLIP-361: Improve GC Metrics [1][2]
> >
> > I'd like to start a vote for it. The vote will be open for at least 72
> > hours unless there is an objection or insufficient votes.
> >
> > Cheers,
> > Gyula
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-361%3A+Improve+GC+Metrics
> > [2] https://lists.apache.org/thread/qqqv54vyr4gbp63wm2d12q78m8h95xb2
> >
>
>
> --
> Best
>
> ConradJam
>

Re: [VOTE] FLIP-334: Decoupling autoscaler and kubernetes and support the Standalone Autoscaler

[Samrat Deb]

+1 (non binding )

Best,
Samrat

On Wed, 13 Sep 2023 at 7:08 PM, Feng Jin  wrote:

> Thanks for driving this, looking forward to this feature.
>
>
> +1 (non-binding)
>
> Best,
> Feng
>
> On Wed, Sep 13, 2023 at 9:11 PM Chen Zhanghao 
> wrote:
>
> > Thanks for driving this. +1 (non-binding)
> >
> > Best,
> > Zhanghao Chen
> > 
> > 发件人: Rui Fan <1996fan...@gmail.com>
> > 发送时间: 2023年9月13日 10:52
> > 收件人: dev 
> > 主题: [VOTE] FLIP-334: Decoupling autoscaler and kubernetes and support the
> > Standalone Autoscaler
> >
> > Hi all,
> >
> > Thanks for all the feedback about the FLIP-334:
> > Decoupling autoscaler and kubernetes and
> > support the Standalone Autoscaler[1].
> > This FLIP was discussed in [2].
> >
> > I'd like to start a vote for it. The vote will be open for at least 72
> > hours (until Sep 16th 11:00 UTC+8) unless there is an objection or
> > insufficient votes.
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-334+%3A+Decoupling+autoscaler+and+kubernetes+and+support+the+Standalone+Autoscaler
> > [2] https://lists.apache.org/thread/kmm03gls1vw4x6vk1ypr9ny9q9522495
> >
> > Best,
> > Rui
> >
>

Re: [VOTE] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

[Samrat Deb]

Thank you for driving this FLIP,

+1 (non-binding)

Bests,
Samrat

On Tue, Sep 12, 2023 at 2:27 PM Ahmed Hamdy  wrote:

> Hi Allison
> Thanks for the proposal.
> + (non-binding)
> Best Regards
> Ahmed Hamdy
>
>
> On Mon, 11 Sept 2023 at 03:48, Weihua Hu  wrote:
>
> > +1 (binding)
> >
> > Best,
> > Weihua
> >
> >
> > On Mon, Sep 11, 2023 at 3:16 AM Jing Ge 
> > wrote:
> >
> > > +1(binding)
> > >
> > > Best Regards,
> > > Jing
> > >
> > > On Sun, Sep 10, 2023 at 10:17 AM Dong Lin  wrote:
> > >
> > > > Thanks Allison for proposing the FLIP.
> > > >
> > > > +1 (binding)
> > > >
> > > > On Fri, Sep 8, 2023 at 4:21 AM Allison Chang
> > >  > > > >
> > > > wrote:
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > Would like to start the VOTE for FLIP-323<
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-323%3A+Support+Attached+Execution+on+Flink+Application+Completion+for+Batch+Jobs
> > > > >
> > > > > which proposes to introduce attached execution for batch jobs. The
> > > > > discussion thread can be found here<
> > > > > https://lists.apache.org/thread/d3toldk6qqjh2fnbmqthlfkj9rc6lwgl>:
> > > > >
> > > > >
> > > > > Best,
> > > > >
> > > > > Allison Chang
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Leonard,

> Do we have to rely on the latest version of JDBC Connector here?

No, there's no need for us to depend on the latest version of the JDBC
Connector. Redshift has its dedicated JDBC driver [1], which includes
custom modifications tailored to Redshift's specific implementation needs.
This driver is the most suitable choice for our purposes.


> Could you collect the APIs that Redshift generally needs to use?

I am actively working on it and making progress towards creating the POC.

Bests,
Samrat

[1]
https://docs.aws.amazon.com/redshift/latest/mgmt/jdbc20-download-driver.html

On Mon, Sep 11, 2023 at 12:02 PM Samrat Deb  wrote:

> Hello Danny,
>
> I wanted to express my gratitude for your valuable feedback and insightful
> suggestions.
>
> I will be revising the FLIP to incorporate all of your queries and review
> suggestions. Additionally, I plan to provide a Proof of Concept (POC) for
> the connector by the end of this week. This POC will address the points
> you've raised and ensure that the FLIP aligns with your recommendations.
>
> Thank you once again for your input.
>
> Bests,
> Samrat
>
> On Thu, Sep 7, 2023 at 10:21 PM Danny Cranmer 
> wrote:
>
>> Hello Leonard,
>>
>> > Do we have to rely on the latest version of JDBC Connector here? I
>> understand that as long as the version of flink minor is the same as the
>> JDBC Connector, Could you collect the APIs that Redshift generally needs
>> to
>> use?
>>
>> I agree we do not necessarily need to rely on the latest patch version,
>> only the same minor. The main issue for me is the dependency introduces a
>> blocker following a new Flink version. For example, when Flink 1.18.0 is
>> released we cannot release the AWS connectors until the JDBC is complete.
>> But I think this is a good tradeoff.
>>
>> > Splitting a separate redshift repository does not solve this coupling
>> problem
>>
>> Arguably it solves the AWS<>JDBC coupling problem, but creates a new, more
>> complex one!
>>
>> Thanks,
>>
>> On Thu, Sep 7, 2023 at 5:26 PM Leonard Xu  wrote:
>>
>> > Thanks Samrat and  Danny for driving this FLIP.
>> >
>> > >> an effective approach is to utilize the latest version of
>> > flink-connector-jdbc
>> > > as a Maven dependency
>> > >
>> > > When we have stable source/sink APIs and the connector versions are
>> > > decoupled from Flink this makes sense. But right now this would mean
>> that
>> > > the JDBC connector will block the AWS connector for each new Flink
>> > version
>> > > support release (1.18, 1.19, 1.20, 2.0 etc). That being said, I cannot
>> > > think of a cleaner alternative, without pulling the core JDBC bits out
>> > into
>> > > a dedicated project that is decoupled from and released independently
>> of
>> > > Flink. Splitting flink-connector-redshift into a dedicated repo would
>> > > decouple AWS/JDBC, but obviously introduce a new connector that is
>> > blocked
>> > > by both AWS and JDBC.
>> >
>> > Do we have to rely on the latest version of JDBC Connector here? I
>> > understand that as long as the version of flink minor is the same as the
>> > JDBC Connector, Could you collect the APIs that Redshift generally
>> needs to
>> > use?
>> >
>> > Assuming that AWS Connector(Redshift) depends on JDBC Connector and
>> wants
>> > a higher version of JDBC Connector, I understand that the correct
>> approach
>> > is to promote the release of JDBC Connector and looks like we have no
>> more
>> > options.
>> >
>> > Splitting a separate redshift repository does not solve this coupling
>> > problem, from a user perspective, redshift should also be in the AWS
>> > Connector repo.
>> >
>> > Best,
>> > Leonard
>>
>

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hello Danny,

I wanted to express my gratitude for your valuable feedback and insightful
suggestions.

I will be revising the FLIP to incorporate all of your queries and review
suggestions. Additionally, I plan to provide a Proof of Concept (POC) for
the connector by the end of this week. This POC will address the points
you've raised and ensure that the FLIP aligns with your recommendations.

Thank you once again for your input.

Bests,
Samrat

On Thu, Sep 7, 2023 at 10:21 PM Danny Cranmer 
wrote:

> Hello Leonard,
>
> > Do we have to rely on the latest version of JDBC Connector here? I
> understand that as long as the version of flink minor is the same as the
> JDBC Connector, Could you collect the APIs that Redshift generally needs to
> use?
>
> I agree we do not necessarily need to rely on the latest patch version,
> only the same minor. The main issue for me is the dependency introduces a
> blocker following a new Flink version. For example, when Flink 1.18.0 is
> released we cannot release the AWS connectors until the JDBC is complete.
> But I think this is a good tradeoff.
>
> > Splitting a separate redshift repository does not solve this coupling
> problem
>
> Arguably it solves the AWS<>JDBC coupling problem, but creates a new, more
> complex one!
>
> Thanks,
>
> On Thu, Sep 7, 2023 at 5:26 PM Leonard Xu  wrote:
>
> > Thanks Samrat and  Danny for driving this FLIP.
> >
> > >> an effective approach is to utilize the latest version of
> > flink-connector-jdbc
> > > as a Maven dependency
> > >
> > > When we have stable source/sink APIs and the connector versions are
> > > decoupled from Flink this makes sense. But right now this would mean
> that
> > > the JDBC connector will block the AWS connector for each new Flink
> > version
> > > support release (1.18, 1.19, 1.20, 2.0 etc). That being said, I cannot
> > > think of a cleaner alternative, without pulling the core JDBC bits out
> > into
> > > a dedicated project that is decoupled from and released independently
> of
> > > Flink. Splitting flink-connector-redshift into a dedicated repo would
> > > decouple AWS/JDBC, but obviously introduce a new connector that is
> > blocked
> > > by both AWS and JDBC.
> >
> > Do we have to rely on the latest version of JDBC Connector here? I
> > understand that as long as the version of flink minor is the same as the
> > JDBC Connector, Could you collect the APIs that Redshift generally needs
> to
> > use?
> >
> > Assuming that AWS Connector(Redshift) depends on JDBC Connector and wants
> > a higher version of JDBC Connector, I understand that the correct
> approach
> > is to promote the release of JDBC Connector and looks like we have no
> more
> > options.
> >
> > Splitting a separate redshift repository does not solve this coupling
> > problem, from a user perspective, redshift should also be in the AWS
> > Connector repo.
> >
> > Best,
> > Leonard
>

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

[Samrat Deb]

Hi Max,

> are we planning to add an alternative implementation
against the new interfaces?

Yes, we are simultaneously working on the YARN implementation using the
interface. During the initial interface design, we encountered some
anomalies while implementing it in YARN.

Once the interfaces are finalized, we will proceed to raise a pull request
(PR) for YARN as well.

Our initial approach was to create a decoupled interface as part of
FLIP-334 and then implement it for YARN in the subsequent phase.
However, if you recommend combining both phases, we can certainly consider
that option.

We look forward to hearing your thoughts on whether to have YARN
implementation as part of FLIP-334 or seperate one ?

Bests
Samrat



On Tue, Sep 5, 2023 at 8:41 PM Maximilian Michels  wrote:

> Thanks Rui for the update!
>
> Alongside with the refactoring to decouple autoscaler logic from the
> deployment logic, are we planning to add an alternative implementation
> against the new interfaces? I think the best way to get the interfaces
> right, is to have an alternative implementation in addition to
> Kubernetes. YARN or a standalone mode implementation were already
> mentioned. Ultimately, this is the reason we are doing the
> refactoring. Without a new implementation, it becomes harder to
> justify the refactoring work.
>
> Cheers,
> Max
>
> On Tue, Sep 5, 2023 at 9:48 AM Rui Fan  wrote:
> >
> > After discussing this FLIP-334[1] offline with Gyula and Max,
> > I updated the FLIP based on the latest conclusion.
> >
> > Big thanks to Gyula and Max for their professional advice!
> >
> > > Does the interface function of handlerRecommendedParallelism
> > > in AutoScalerEventHandler conflict with
> > > handlerScalingFailure/handlerScalingReport (one of the
> > > handles the event of scale failure, and the other handles
> > > the event of scale success).
> > Hi Matt,
> >
> > You can take a look at the FLIP, I think the issue has been fixed.
> > Currently, we introduced the ScalingRealizer and
> > AutoScalerEventHandler interface.
> >
> > The ScalingRealizer handles scaling action.
> >
> > The AutoScalerEventHandler  interface handles loggable events.
> >
> >
> > Looking forward to your feedback, thanks!
> >
> > [1] https://cwiki.apache.org/confluence/x/x4qzDw
> >
> > Best,
> > Rui
> >
> > On Thu, Aug 24, 2023 at 10:55 AM Matt Wang  wrote:
> >>
> >> Sorry for the late reply, I still have a small question here:
> >> Does the interface function of handlerRecommendedParallelism
> >> in AutoScalerEventHandler conflict with
> >> handlerScalingFailure/handlerScalingReport (one of the
> >> handles the event of scale failure, and the other handles
> >> the event of scale success).
> >>
> >>
> >>
> >> --
> >>
> >> Best,
> >> Matt Wang
> >>
> >>
> >>  Replied Message 
> >> | From | Rui Fan<1996fan...@gmail.com> |
> >> | Date | 08/21/2023 17:41 |
> >> | To |  |
> >> | Cc | Maximilian Michels ,
> >> Gyula Fóra ,
> >> Matt Wang |
> >> | Subject | Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and
> kubernetes |
> >> Hi Max, Gyula and Matt,
> >>
> >> Do you have any other comments?
> >>
> >> The flink-kubernetes-operator 1.6 has been released recently,
> >> it's a good time to kick off this FLIP.
> >>
> >> Please let me know if you have any questions or concerns,
> >> looking forward to your feedback, thanks!
> >>
> >> Best,
> >> Rui
> >>
> >> On Wed, Aug 9, 2023 at 11:55 AM Rui Fan <1996fan...@gmail.com> wrote:
> >>
> >> Hi Matt Wang,
> >>
> >> Thanks for your discussion here.
> >>
> >> it is recommended to unify the descriptions of AutoScalerHandler
> >> and AutoScalerEventHandler in the FLIP
> >>
> >> Good catch, I have updated all AutoScalerHandler to
> >> AutoScalerEventHandler.
> >>
> >> Can it support the use of zookeeper (zookeeper is a relatively
> >> common use of flink HA)?
> >>
> >> In my opinion, it's a good suggestion. However, I prefer we
> >> implement other state stores in the other FLINK JIRA, and
> >> this FLIP focus on the decoupling and implementing the
> >> necessary state store. Does that make sense?
> >>
> >> Regarding each scaling information, can it be persisted in
> >> the shared file system through the filesystem? I think it will
> >> be a more valuable requirement to support viewing
> >> Autoscaling info on the UI in the future, which can provide
> >> some foundations in advance;
> >>
> >> This is a good suggestion as well. It's useful for users to check
> >> the scaling information. I propose to add a CompositeEventHandler,
> >> it can include multiple EventHandlers.
> >>
> >> However, as the last question, I prefer we implement other
> >> event handler in the other FLINK JIRA. What do you think?
> >>
> >> A solution mentioned in FLIP is to initialize the
> >> AutoScalerEventHandler object every time an event is
> >> processed.
> >>
> >> No, the FLIP mentioned `The AutoScalerEventHandler  object is shared for
> >> all flink jobs`,
> >> So the AutoScalerEventHandler is only initialized once.
> >>
> >> 

Re: [VOTE] Release flink-connector-hbase v3.0.0, release candidate 2

[Samrat Deb]

Hi, 

+1 (non-binding)

Verified NOTICE files 
Verified CheckSum and signatures 
Glanced through  PR[1] , Looks good to me 

Bests, 
Samrat 

[1]https://github.com/apache/flink-web/pull/591


> On 04-Sep-2023, at 2:22 PM, Ahmed Hamdy  wrote:
> 
> Hi Martijn,
> +1 (non-binding)
> 
> - verified Checksums and signatures
> - no binaries in source
> - Checked NOTICE files contains migrated artifacts
> - tag is correct
> - Approved Web PR
> 
> Best Regards
> Ahmed Hamdy
> 
> 
> On Fri, 1 Sept 2023 at 15:35, Martijn Visser 
> wrote:
> 
>> Hi everyone,
>> 
>> Please review and vote on the release candidate #2 for the version 3.0.0,
>> as follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>> 
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org
>> [2],
>> which are signed with the key with fingerprint
>> A5F3BCE4CBE993573EC5966A65321B8382B219AF [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag v3.0.0-rc2 [5],
>> * website pull request listing the new release [6].
>> 
>> This replaces the old, cancelled vote of RC1 [7]. This version is the
>> externalized version which is compatible with Flink 1.16 and 1.17.
>> 
>> The vote will be open for at least 72 hours. It is adopted by majority
>> approval, with at least 3 PMC affirmative votes.
>> 
>> Thanks,
>> Release Manager
>> 
>> [1]
>> 
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352578
>> [2]
>> 
>> https://dist.apache.org/repos/dist/dev/flink/flink-connector-hbase-3.0.0-rc2
>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
>> [4] https://repository.apache.org/content/repositories/orgapacheflink-1650
>> [5]
>> https://github.com/apache/flink-connector-hbase/releases/tag/v3.0.0-rc2
>> [6] https://github.com/apache/flink-web/pull/591
>> [7] https://lists.apache.org/thread/wbl6sc86q9s5mmz5slx4z09svh91cpr0
>> 

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Martijn,
I fully comprehend your apprehensions regarding AWS's reliance on JDBC and
the resultant impact on the release cycle. However, an effective approach
is to utilize the latest version of flink-connector-jdbc as a Maven
dependency. In this arrangement, flink-connector-aws can be associated with
an older version of flink-connector-jdbc. It's not obligatory for
flink-connector-aws to constantly align with the most current version. This
strategy disengages direct dependencies between the two in their respective
release cycles. By adopting this approach, flink-connector-aws can maintain
its specific dependencies on JDBC and the preferred version paralleling its
approach with the AWS SDK's handling of dependencies.

Bests,
Samrat

On Tue, Jul 4, 2023 at 7:09 PM Martijn Visser 
wrote:

> Hi Samrat,
>
> The dependency between JDBC and AWS worries me a lot: we're already seeing
> that coupling causes a lot of issues down the line. Why can't we decouple
> these?
>
> Best regards,
>
> Martijn
>
> On Tue, Jul 4, 2023 at 3:35 PM Samrat Deb  wrote:
>
> > Hi Leonard,
> >
> > Sorry for the late reply.
> >
> > > 1 Reusing the capabilities of JDBC and Filesystem in the Redshift
> > connector generally makes sense to me. However, since all of them are
> > managed in different repositories and depend on Flink dependency, could
> you
> > explain how you establish the versioning, release, and dependency
> > management process?
> >
> > We intend to maintain the current release cycle for the Flink Connector
> > AWS. However, we will be incorporating a dependency of Flink Connector
> JDBC
> > to most recent version, which will synchronize between the flink
> connector
> > AWS and flink connector JDBC connectors releases for new Flink version
> > support. Additionally, the Flink Connector Redshift will exclusively
> > utilize the public API, minimizing the occurrence of immediate breaking
> > changes.
> >
> > > 2 Some configuration option names can be improved to match the naming
> > style of existing configuration options, for example:
> > table -> table-name
> > query -> scan.query
> > aws-iam-role -> aws.iam-role
> > read.mode -> scan.read.mode: similar to scan.startup.mode , and maybe we
> > will have lookup.read.mode
> > write.mode -> sink.write.mode
> >
> > Updated the FLIP-307 [1]
> >
> > > 3 The source of Redshift connector supports JDBC queries, IIUC, we can
> > also support the LookupTableSource as well?
> >
> > we will support `LookupTableSource` in redshift connector source
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
> >
> > Bests,
> > Samrat
> >
> > On Wed, Jun 21, 2023 at 4:47 PM Leonard Xu  wrote:
> >
> > > Thanks Samrat for driving this FLIP.
> > >
> > > Since the community has already built a set of basic components for the
> > > connector, I only have three comments.
> > >
> > > 1 Reusing the capabilities of JDBC and Filesystem in the Redshift
> > > connector generally makes sense to me. However, since all of them are
> > > managed in different repositories and depend on Flink dependency, could
> > you
> > > explain how you establish the versioning, release, and dependency
> > > management process?
> > >
> > > 2 Some configuration option names can be improved to match the naming
> > > style of existing configuration options, for example:
> > > table -> table-name
> > > query -> scan.query
> > > aws-iam-role -> aws.iam-role
> > > read.mode -> scan.read.mode: similar to scan.startup.mode , and maybe
> we
> > > will have lookup.read.mode
> > > write.mode -> sink.write.mode
> > >
> > > 3 The source of Redshift connector supports JDBC queries, IIUC, we can
> > > also support the LookupTableSource as well?
> > >
> > > Best,
> > > Leonard
> > >
> > > > On Jun 21, 2023, at 4:57 PM, Samrat Deb 
> wrote:
> > > >
> > > > Hi Martijn,
> > > >
> > > > Thank you for sharing your thoughts on the matter.
> > > > I understand that you don't have a strong opinion on whether to
> support
> > > > exactly-once processing from the beginning or at a later stage.
> > > > For initial implementation I will go ahead with at-least-once
> > semantics.
> > > >
> > > >> The only consideration that I could think of is that
> > > > if you start wit

Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

[Samrat Deb]

Congratulations, Matthias

Bests,
Samrat

On Wed, Aug 9, 2023 at 4:18 PM Benchao Li  wrote:

> Congratulations, Matthias!
>
> Maximilian Michels  于2023年8月8日周二 17:54写道：
>
> > Congrats, well done, and welcome to the PMC Matthias!
> >
> > -Max
> >
> > On Tue, Aug 8, 2023 at 8:36 AM yh z  wrote:
> > >
> > > Congratulations, Matthias!
> > >
> > > Best,
> > > Yunhong Zheng (Swuferhong)
> > >
> > > Ryan Skraba  于2023年8月7日周一 21:39写道：
> > >
> > > > Congratulations Matthias -- very well-deserved, the community is
> lucky
> > to
> > > > have you <3
> > > >
> > > > All my best, Ryan
> > > >
> > > > On Mon, Aug 7, 2023 at 3:04 PM Lincoln Lee 
> > wrote:
> > > >
> > > > > Congratulations!
> > > > >
> > > > > Best,
> > > > > Lincoln Lee
> > > > >
> > > > >
> > > > > Feifan Wang  于2023年8月7日周一 20:13写道：
> > > > >
> > > > > > Congrats Matthias!
> > > > > >
> > > > > >
> > > > > >
> > > > > > ——
> > > > > > Name: Feifan Wang
> > > > > > Email: zoltar9...@163.com
> > > > > >
> > > > > >
> > > > > >  Replied Message 
> > > > > > | From | Matthias Pohl |
> > > > > > | Date | 08/7/2023 16:16 |
> > > > > > | To |  |
> > > > > > | Subject | Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias
> > Pohl
> > > > |
> > > > > > Thanks everyone. :)
> > > > > >
> > > > > > On Mon, Aug 7, 2023 at 3:18 AM Andriy Redko 
> > wrote:
> > > > > >
> > > > > > Congrats Matthias, well deserved!!
> > > > > >
> > > > > > DC> Congrats Matthias!
> > > > > >
> > > > > > DC> Very well deserved, thankyou for your continuous, consistent
> > > > > > contributions.
> > > > > > DC> Welcome.
> > > > > >
> > > > > > DC> Thanks,
> > > > > > DC> Danny
> > > > > >
> > > > > > DC> On Fri, Aug 4, 2023 at 9:30 AM Feng Jin <
> jinfeng1...@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > Congratulations, Matthias!
> > > > > >
> > > > > > Best regards
> > > > > >
> > > > > > Feng
> > > > > >
> > > > > > On Fri, Aug 4, 2023 at 4:29 PM weijie guo <
> > guoweijieres...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > Congratulations, Matthias!
> > > > > >
> > > > > > Best regards,
> > > > > >
> > > > > > Weijie
> > > > > >
> > > > > >
> > > > > > Wencong Liu  于2023年8月4日周五 15:50写道：
> > > > > >
> > > > > > Congratulations, Matthias!
> > > > > >
> > > > > > Best,
> > > > > > Wencong Liu
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > At 2023-08-04 11:18:00, "Xintong Song" 
> > > > > > wrote:
> > > > > > Hi everyone,
> > > > > >
> > > > > > On behalf of the PMC, I'm very happy to announce that Matthias
> Pohl
> > > > > > has
> > > > > > joined the Flink PMC!
> > > > > >
> > > > > > Matthias has been consistently contributing to the project since
> > > > > > Sep
> > > > > > 2020,
> > > > > > and became a committer in Dec 2021. He mainly works in Flink's
> > > > > > distributed
> > > > > > coordination and high availability areas. He has worked on many
> > > > > > FLIPs
> > > > > > including FLIP195/270/285. He helped a lot with the release
> > > > > > management,
> > > > > > being one of the Flink 1.17 release managers and also very active
> > > > > > in
> > > > > > Flink
> > > > > > 1.18 / 2.0 efforts. He also contributed a lot to improving the
> > > > > > build
> > > > > > stability.
> > > > > >
> > > > > > Please join me in congratulating Matthias!
> > > > > >
> > > > > > Best,
> > > > > >
> > > > > > Xintong (on behalf of the Apache Flink PMC)
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> >
>
>
> --
>
> Best,
> Benchao Li
>

Re: [ANNOUNCE] New Apache Flink Committer - Hong Teoh

[Samrat Deb]

Congratulations Hong

Bests,
Samrat

On Fri, Aug 4, 2023 at 2:37 PM Dan Zou  wrote:

> Congrats, Hong!
>
> Best,
> Dan Zou
>
>
>
>
>
> > 2023年8月4日 15:30，Lijie Wang  写道：
> >
> > Congratulations, Hong!
> >
> > Best,
> > Lijie
> >
> > Paul Lam  于2023年8月4日周五 15:18写道：
> >
> >> Congrats, Hong!
> >>
> >> Best,
> >> Paul Lam
> >>
> >>> 2023年8月4日 14:59，Jark Wu  写道：
> >>>
> >>> Congratulations, Hong!
> >>>
> >>> Best,
> >>> Jark
> >>>
> >>> On Fri, 4 Aug 2023 at 14:24, Sergey Nuyanzin 
> >> wrote:
> >>>
> >>>> Congratulations, Hong!
> >>>>
> >>>> On Fri, Aug 4, 2023 at 7:25 AM Shammon FY  wrote:
> >>>>
> >>>>> Congratulations, Hong!
> >>>>>
> >>>>> Best,
> >>>>> Shammon FY
> >>>>>
> >>>>> On Fri, Aug 4, 2023 at 12:33 PM Jing Ge 
> >>>>> wrote:
> >>>>>
> >>>>>> congrats! Hong!
> >>>>>>
> >>>>>> Best regards,
> >>>>>> Jing
> >>>>>>
> >>>>>> On Fri, Aug 4, 2023 at 11:48 AM Qingsheng Ren 
> >>>>> wrote:
> >>>>>>
> >>>>>>> Congratulations and welcome aboard, Hong!
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Qingsheng
> >>>>>>>
> >>>>>>> On Fri, Aug 4, 2023 at 11:04 AM Matt Wang  wrote:
> >>>>>>>
> >>>>>>>> Congratulations, Hong!
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Matt Wang
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>  Replied Message 
> >>>>>>>> | From | Weihua Hu |
> >>>>>>>> | Date | 08/4/2023 10:55 |
> >>>>>>>> | To |  |
> >>>>>>>> | Subject | Re: [ANNOUNCE] New Apache Flink Committer - Hong Teoh
> |
> >>>>>>>> Congratulations, Hong!
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Weihua
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, Aug 4, 2023 at 10:49 AM Samrat Deb  >
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Congratulations, Hong Teoh
> >>>>>>>>
> >>>>>>>> On Fri, 4 Aug 2023 at 7:52 AM, Benchao Li 
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Congratulations, Hong!
> >>>>>>>>
> >>>>>>>> yuxia  于2023年8月4日周五 09:23写道：
> >>>>>>>>
> >>>>>>>> Congratulations, Hong Teoh!
> >>>>>>>>
> >>>>>>>> Best regards,
> >>>>>>>> Yuxia
> >>>>>>>>
> >>>>>>>> - 原始邮件 -
> >>>>>>>> 发件人: "Matthias Pohl" 
> >>>>>>>> 收件人: "dev" 
> >>>>>>>> 发送时间: 星期四, 2023年 8 月 03日 下午 11:24:39
> >>>>>>>> 主题: Re: [ANNOUNCE] New Apache Flink Committer - Hong Teoh
> >>>>>>>>
> >>>>>>>> Congratulations, Hong! :)
> >>>>>>>>
> >>>>>>>> On Thu, Aug 3, 2023 at 3:39 PM Leonard Xu 
> >>>> wrote:
> >>>>>>>>
> >>>>>>>> Congratulations, Hong!
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Leonard
> >>>>>>>>
> >>>>>>>> On Aug 3, 2023, at 8:45 PM, Jiabao Sun  >>>>>>>> .INVALID>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Congratulations, Hong Teoh!
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Jiabao Sun
> >>>>>>>>
> >>>>>>>> 2023年8月3日 下午7:32，Danny Cranmer  写道：
> >>>>>>>>
> >>>>>>>> On behalf of the PMC, I'm very happy to announce Hong Teoh as a
> >>>>>>>> new
> >>>>>>>> Flink
> >>>>>>>> Committer.
> >>>>>>>>
> >>>>>>>> Hong has been active in the Flink community for over 1 year and
> >>>>>>>> has
> >>>>>>>> played
> >>>>>>>> a key role in developing and maintaining AWS integrations, core
> >>>>>>>> connector
> >>>>>>>> APIs and more recently, improvements to the Flink REST API.
> >>>>>>>> Additionally,
> >>>>>>>> Hong is a very active community member, supporting users and
> >>>>>>>> participating
> >>>>>>>> in discussions on the mailing lists, Flink slack channels and
> >>>>>>>> speaking
> >>>>>>>> at
> >>>>>>>> conferences.
> >>>>>>>>
> >>>>>>>> Please join me in congratulating Hong for becoming a Flink
> >>>>>>>> Committer!
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Danny Cranmer (on behalf of the Flink PMC)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Benchao Li
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best regards,
> >>>> Sergey
> >>>>
> >>
> >>
>
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

[Samrat Deb]

Congrats, Matthias!


On Fri, 4 Aug 2023 at 10:13 AM, Benchao Li  wrote:

> Congratulations, Matthias!
>
> Jing Ge  于2023年8月4日周五 12:35写道：
>
> > Congrats! Matthias!
> >
> > Best regards,
> > Jing
> >
> > On Fri, Aug 4, 2023 at 12:09 PM Yangze Guo  wrote:
> >
> > > Congrats, Matthias!
> > >
> > > Best,
> > > Yangze Guo
> > >
> > > On Fri, Aug 4, 2023 at 11:44 AM Qingsheng Ren 
> wrote:
> > > >
> > > > Congratulations, Matthias! This is absolutely well deserved.
> > > >
> > > > Best,
> > > > Qingsheng
> > > >
> > > > On Fri, Aug 4, 2023 at 11:31 AM Rui Fan <1996fan...@gmail.com>
> wrote:
> > > >
> > > > > Congratulations Matthias, well deserved!
> > > > >
> > > > > Best,
> > > > > Rui Fan
> > > > >
> > > > > On Fri, Aug 4, 2023 at 11:30 AM Leonard Xu 
> > wrote:
> > > > >
> > > > > > Congratulations,  Matthias.
> > > > > >
> > > > > > Well deserved ^_^
> > > > > >
> > > > > > Best,
> > > > > > Leonard
> > > > > >
> > > > > >
> > > > > > > On Aug 4, 2023, at 11:18 AM, Xintong Song <
> tonysong...@gmail.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > Hi everyone,
> > > > > > >
> > > > > > > On behalf of the PMC, I'm very happy to announce that Matthias
> > > Pohl has
> > > > > > > joined the Flink PMC!
> > > > > > >
> > > > > > > Matthias has been consistently contributing to the project
> since
> > > Sep
> > > > > > 2020,
> > > > > > > and became a committer in Dec 2021. He mainly works in Flink's
> > > > > > distributed
> > > > > > > coordination and high availability areas. He has worked on many
> > > FLIPs
> > > > > > > including FLIP195/270/285. He helped a lot with the release
> > > management,
> > > > > > > being one of the Flink 1.17 release managers and also very
> active
> > > in
> > > > > > Flink
> > > > > > > 1.18 / 2.0 efforts. He also contributed a lot to improving the
> > > build
> > > > > > > stability.
> > > > > > >
> > > > > > > Please join me in congratulating Matthias!
> > > > > > >
> > > > > > > Best,
> > > > > > >
> > > > > > > Xintong (on behalf of the Apache Flink PMC)
> > > > > >
> > > > > >
> > > > >
> > >
> >
>
>
> --
>
> Best,
> Benchao Li
>

Re: [ANNOUNCE] New Apache Flink Committer - Hong Teoh

[Samrat Deb]

Congratulations, Hong Teoh

On Fri, 4 Aug 2023 at 7:52 AM, Benchao Li  wrote:

> Congratulations, Hong!
>
> yuxia  于2023年8月4日周五 09:23写道：
>
> > Congratulations, Hong Teoh!
> >
> > Best regards,
> > Yuxia
> >
> > - 原始邮件 -
> > 发件人: "Matthias Pohl" 
> > 收件人: "dev" 
> > 发送时间: 星期四, 2023年 8 月 03日 下午 11:24:39
> > 主题: Re: [ANNOUNCE] New Apache Flink Committer - Hong Teoh
> >
> > Congratulations, Hong! :)
> >
> > On Thu, Aug 3, 2023 at 3:39 PM Leonard Xu  wrote:
> >
> > > Congratulations, Hong!
> > >
> > >
> > > Best,
> > > Leonard
> > >
> > > > On Aug 3, 2023, at 8:45 PM, Jiabao Sun  > .INVALID>
> > > wrote:
> > > >
> > > > Congratulations, Hong Teoh!
> > > >
> > > > Best,
> > > > Jiabao Sun
> > > >
> > > >> 2023年8月3日 下午7:32，Danny Cranmer  写道：
> > > >>
> > > >> On behalf of the PMC, I'm very happy to announce Hong Teoh as a new
> > > Flink
> > > >> Committer.
> > > >>
> > > >> Hong has been active in the Flink community for over 1 year and has
> > > played
> > > >> a key role in developing and maintaining AWS integrations, core
> > > connector
> > > >> APIs and more recently, improvements to the Flink REST API.
> > > Additionally,
> > > >> Hong is a very active community member, supporting users and
> > > participating
> > > >> in discussions on the mailing lists, Flink slack channels and
> speaking
> > > at
> > > >> conferences.
> > > >>
> > > >> Please join me in congratulating Hong for becoming a Flink
> Committer!
> > > >>
> > > >> Thanks,
> > > >> Danny Cranmer (on behalf of the Flink PMC)
> > > >
> > >
> > >
> >
>
>
> --
>
> Best,
> Benchao Li
>

Re: Kubernetes Operator 1.6.0 release planning

[Samrat Deb]

thank you gyula ,
for driving it.
+1(non binding)


Bests,
Samrat

On Thu, 20 Jul 2023 at 8:02 AM, Rui Fan <1996fan...@gmail.com> wrote:

> Thanks Gyula for driving this release.
>
> +1 for the timeline
>
> Best,
> Rui Fan
>
> On Wed, Jul 19, 2023 at 11:03 PM Gyula Fóra  wrote:
>
> > Hi Devs!
> >
> > Based on our release schedule, it is about time for the next Flink K8s
> > Operator minor release.
> >
> > There are still some minor work items to be completed this week, but I
> > suggest aiming for next Wednesday (July 26th) as the 1.6.0 release-cut -
> > RC1 date.
> >
> > I am volunteering as the release manager but if someone else wants to do
> > it, I would also be happy to simply give assistance :)
> >
> > Please let me know if you agree or disagree with the suggested timeline.
> >
> > Cheers,
> > Gyula
> >
>

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Leonard,

Sorry for the late reply.

> 1 Reusing the capabilities of JDBC and Filesystem in the Redshift
connector generally makes sense to me. However, since all of them are
managed in different repositories and depend on Flink dependency, could you
explain how you establish the versioning, release, and dependency
management process?

We intend to maintain the current release cycle for the Flink Connector
AWS. However, we will be incorporating a dependency of Flink Connector JDBC
to most recent version, which will synchronize between the flink connector
AWS and flink connector JDBC connectors releases for new Flink version
support. Additionally, the Flink Connector Redshift will exclusively
utilize the public API, minimizing the occurrence of immediate breaking
changes.

> 2 Some configuration option names can be improved to match the naming
style of existing configuration options, for example:
table -> table-name
query -> scan.query
aws-iam-role -> aws.iam-role
read.mode -> scan.read.mode: similar to scan.startup.mode , and maybe we
will have lookup.read.mode
write.mode -> sink.write.mode

Updated the FLIP-307 [1]

> 3 The source of Redshift connector supports JDBC queries, IIUC, we can
also support the LookupTableSource as well?

we will support `LookupTableSource` in redshift connector source

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift

Bests,
Samrat

On Wed, Jun 21, 2023 at 4:47 PM Leonard Xu  wrote:

> Thanks Samrat for driving this FLIP.
>
> Since the community has already built a set of basic components for the
> connector, I only have three comments.
>
> 1 Reusing the capabilities of JDBC and Filesystem in the Redshift
> connector generally makes sense to me. However, since all of them are
> managed in different repositories and depend on Flink dependency, could you
> explain how you establish the versioning, release, and dependency
> management process?
>
> 2 Some configuration option names can be improved to match the naming
> style of existing configuration options, for example:
> table -> table-name
> query -> scan.query
> aws-iam-role -> aws.iam-role
> read.mode -> scan.read.mode: similar to scan.startup.mode , and maybe we
> will have lookup.read.mode
> write.mode -> sink.write.mode
>
> 3 The source of Redshift connector supports JDBC queries, IIUC, we can
> also support the LookupTableSource as well?
>
> Best,
> Leonard
>
> > On Jun 21, 2023, at 4:57 PM, Samrat Deb  wrote:
> >
> > Hi Martijn,
> >
> > Thank you for sharing your thoughts on the matter.
> > I understand that you don't have a strong opinion on whether to support
> > exactly-once processing from the beginning or at a later stage.
> > For initial implementation I will go ahead with at-least-once semantics.
> >
> >> The only consideration that I could think of is that
> > if you start with at-least-once, you could consider using the ASync API,
> > but I don't think the ASync API yet supports exactly-once.
> >
> > Noted. It's a valid consideration to start compatibility with the Async
> > API.
> >
> > Bests,
> > Samrat
> >
> >
> > On Mon, Jun 19, 2023 at 5:28 PM Martijn Visser  >
> > wrote:
> >
> >> Hi Samrat,
> >>
> >> I have no strong opinion on whether to support exactly-once from the
> start
> >> or potentially later. The only consideration that I could think of is
> that
> >> if you start with at-least-once, you could consider using the ASync API,
> >> but I don't think the ASync API yet supports exactly-once.
> >>
> >> Thanks,
> >>
> >> Martijn
> >>
> >> On Fri, Jun 9, 2023 at 7:22 PM Jing Ge 
> wrote:
> >>
> >>> Hi Samrat,
> >>>
> >>> The FLIP looks good, thanks!
> >>>
> >>> Best regards,
> >>> Jing
> >>>
> >>>
> >>> On Tue, Jun 6, 2023 at 8:16 PM Samrat Deb 
> wrote:
> >>>
> >>>> Hi Jing,
> >>>>
> >>>>> I would suggest adding that information into the
> >>>> FLIP.
> >>>>
> >>>> Updated now, please review the new version of flip whenever time.
> >>>>
> >>>>> +1 Looking forward to your PR :-)
> >>>> I will request for your review once m ready with PR :-)
> >>>>
> >>>> Bests,
> >>>> Samrat
> >>>>
> >>>> On Tue, Jun 6, 2023 at 11:43 PM Samrat Deb 
> >>> wrote:
> >>>>
> >>>>> Hi Martijn,
> >>>>>
> 

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Martijn,

Thank you for sharing your thoughts on the matter.
I understand that you don't have a strong opinion on whether to support
exactly-once processing from the beginning or at a later stage.
For initial implementation I will go ahead with at-least-once semantics.

>  The only consideration that I could think of is that
if you start with at-least-once, you could consider using the ASync API,
but I don't think the ASync API yet supports exactly-once.

Noted. It's a valid consideration to start compatibility with the Async
API.

Bests,
Samrat


On Mon, Jun 19, 2023 at 5:28 PM Martijn Visser 
wrote:

> Hi Samrat,
>
> I have no strong opinion on whether to support exactly-once from the start
> or potentially later. The only consideration that I could think of is that
> if you start with at-least-once, you could consider using the ASync API,
> but I don't think the ASync API yet supports exactly-once.
>
> Thanks,
>
> Martijn
>
> On Fri, Jun 9, 2023 at 7:22 PM Jing Ge  wrote:
>
> > Hi Samrat,
> >
> > The FLIP looks good, thanks!
> >
> > Best regards,
> > Jing
> >
> >
> > On Tue, Jun 6, 2023 at 8:16 PM Samrat Deb  wrote:
> >
> > > Hi Jing,
> > >
> > > >  I would suggest adding that information into the
> > > FLIP.
> > >
> > > Updated now, please review the new version of flip whenever time.
> > >
> > > > +1 Looking forward to your PR :-)
> > > I will request for your review once m ready with PR :-)
> > >
> > > Bests,
> > > Samrat
> > >
> > > On Tue, Jun 6, 2023 at 11:43 PM Samrat Deb 
> > wrote:
> > >
> > > > Hi Martijn,
> > > >
> > > > > If I understand this correctly, the Redshift sink
> > > > would not be able to support exactly-once, is that correct?
> > > >
> > > > As I delve deeper into the study of Redshift's capabilities, I have
> > > > discovered that it does support "merge into" operations [1] and some
> > > > merge into examples [2].
> > > > This opens up the possibility of implementing exactly-once semantics
> > with
> > > > the connector.
> > > > However, I believe it would be prudent to start with a more focused
> > scope
> > > > for the initial phase of implementation and defer the exact-once
> > support
> > > > for subsequent iterations.
> > > >
> > > > Before finalizing the approach, I would greatly appreciate your
> > thoughts
> > > > and suggestions on this matter.
> > > > Should we prioritize the initial implementation without exactly-once
> > > > support, or would you advise incorporating it right from the start?
> > > > Your insights and experiences would be immensely valuable in making
> > this
> > > > decision.
> > > >
> > > >
> > > > [1]
> > > >
> > >
> >
> https://docs.aws.amazon.com/redshift/latest/dg/t_updating-inserting-using-staging-tables-.html
> > > > [2]
> https://docs.aws.amazon.com/redshift/latest/dg/merge-examples.html
> > > >
> > > > Bests,
> > > > Samrat
> > > >
> > > > On Mon, Jun 5, 2023 at 7:09 PM Jing Ge 
> > > wrote:
> > > >
> > > >> Hi Samrat,
> > > >>
> > > >> Thanks for the feedback. I would suggest adding that information
> into
> > > the
> > > >> FLIP.
> > > >>
> > > >> +1 Looking forward to your PR :-)
> > > >>
> > > >> Best regards,
> > > >> Jing
> > > >>
> > > >> On Sat, Jun 3, 2023 at 9:19 PM Samrat Deb 
> > > wrote:
> > > >>
> > > >> > Hi Jing Ge,
> > > >> >
> > > >> > >>> Do you already have any prototype? I'd like to join the
> reviews.
> > > >> > The prototype is in progress. I will raise the dedicated PR for
> > review
> > > >> soon
> > > >> > also notify in this thread as well .
> > > >> >
> > > >> > >>> Will the Redshift connector provide additional features
> > > >> > beyond the mediator/wrapper of the jdbc connector?
> > > >> >
> > > >> > Here are the additional features that the Flink connector for AWS
> > > >> Redshift
> > > >> > can provide on top of using JDBC:
> > > >> >
> > >

[jira] [Created] (FLINK-32372) flink-connector-aws: build on pull request / compile_and_test doesn't support for Flink 1.16.2 and 1.17.1

[Samrat Deb (Jira)]

Samrat Deb created FLINK-32372:
--

 Summary: flink-connector-aws: build on pull request / 
compile_and_test doesn't support for Flink 1.16.2 and 1.17.1
 Key: FLINK-32372
 URL: https://issues.apache.org/jira/browse/FLINK-32372
 Project: Flink
  Issue Type: New Feature
  Components: Connectors / AWS
Reporter: Samrat Deb


*Recently* {*}1.16.2 and 1.17.1 flink version are released.{*}{*}{*}

*flink-connector-aws: build on pull request / compile_and_test currently 
doesn't support 1.16.2 and 1.17.1 flink version.* 

*Add the support to build and test for*  \{*}1.16.2 and 1.17.1 flink version on 
CI pipeline for flink-connector-aws.{*}{*}{*}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Jing,

>  I would suggest adding that information into the
FLIP.

Updated now, please review the new version of flip whenever time.

> +1 Looking forward to your PR :-)
I will request for your review once m ready with PR :-)

Bests,
Samrat

On Tue, Jun 6, 2023 at 11:43 PM Samrat Deb  wrote:

> Hi Martijn,
>
> > If I understand this correctly, the Redshift sink
> would not be able to support exactly-once, is that correct?
>
> As I delve deeper into the study of Redshift's capabilities, I have
> discovered that it does support "merge into" operations [1] and some
> merge into examples [2].
> This opens up the possibility of implementing exactly-once semantics with
> the connector.
> However, I believe it would be prudent to start with a more focused scope
> for the initial phase of implementation and defer the exact-once support
> for subsequent iterations.
>
> Before finalizing the approach, I would greatly appreciate your thoughts
> and suggestions on this matter.
> Should we prioritize the initial implementation without exactly-once
> support, or would you advise incorporating it right from the start?
> Your insights and experiences would be immensely valuable in making this
> decision.
>
>
> [1]
> https://docs.aws.amazon.com/redshift/latest/dg/t_updating-inserting-using-staging-tables-.html
> [2] https://docs.aws.amazon.com/redshift/latest/dg/merge-examples.html
>
> Bests,
> Samrat
>
> On Mon, Jun 5, 2023 at 7:09 PM Jing Ge  wrote:
>
>> Hi Samrat,
>>
>> Thanks for the feedback. I would suggest adding that information into the
>> FLIP.
>>
>> +1 Looking forward to your PR :-)
>>
>> Best regards,
>> Jing
>>
>> On Sat, Jun 3, 2023 at 9:19 PM Samrat Deb  wrote:
>>
>> > Hi Jing Ge,
>> >
>> > >>> Do you already have any prototype? I'd like to join the reviews.
>> > The prototype is in progress. I will raise the dedicated PR for review
>> soon
>> > also notify in this thread as well .
>> >
>> > >>> Will the Redshift connector provide additional features
>> > beyond the mediator/wrapper of the jdbc connector?
>> >
>> > Here are the additional features that the Flink connector for AWS
>> Redshift
>> > can provide on top of using JDBC:
>> >
>> > 1. Integration with AWS Redshift Workload Management (WLM): AWS Redshift
>> > allows you to configure WLM[1] to manage query prioritization and
>> resource
>> > allocation. The Flink connector for Redshift will be agnostic to the
>> > configured WLM and utilize it for scaling in and out for the sink. This
>> > means that the connector can leverage the WLM capabilities of Redshift
>> to
>> > optimize the execution of queries and allocate resources efficiently
>> based
>> > on your defined workload priorities.
>> >
>> > 2. Abstraction of AWS Redshift Quotas and Limits: AWS Redshift imposes
>> > certain quotas and limits[2] on various aspects such as the number of
>> > clusters, concurrent connections, queries per second, etc. The Flink
>> > connector for Redshift will provide an abstraction layer for users,
>> > allowing them to work with Redshift without having to worry about these
>> > specific limits. The connector will handle the management of connections
>> > and queries within the defined quotas and limits, abstracting away the
>> > complexity and ensuring compliance with Redshift's restrictions.
>> >
>> > These features aim to simplify the integration of Flink with AWS
>> Redshift,
>> > providing optimized resource utilization and transparent handling of
>> > Redshift-specific limitations.
>> >
>> > Bests,
>> > Samrat
>> >
>> > [1]
>> >
>> >
>> https://docs.aws.amazon.com/redshift/latest/dg/cm-c-implementing-workload-management.html
>> > [2]
>> >
>> >
>> https://docs.aws.amazon.com/redshift/latest/mgmt/amazon-redshift-limits.html
>> >
>> > On Sat, Jun 3, 2023 at 11:40 PM Samrat Deb 
>> wrote:
>> >
>> > > Hi Ahmed,
>> > >
>> > > >>> please let me know If you need any collaboration regarding
>> > integration
>> > > with
>> > > AWS connectors credential providers or regarding FLIP-171 I would be
>> more
>> > > than happy to assist.
>> > >
>> > > Sure, I will reach out incase of any hands required.
>> > >
>> > >
>> > >
>> > > On Fri, Jun 2, 2023 at 6:12 

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Martijn,

> If I understand this correctly, the Redshift sink
would not be able to support exactly-once, is that correct?

As I delve deeper into the study of Redshift's capabilities, I have
discovered that it does support "merge into" operations [1] and some
merge into examples [2].
This opens up the possibility of implementing exactly-once semantics with
the connector.
However, I believe it would be prudent to start with a more focused scope
for the initial phase of implementation and defer the exact-once support
for subsequent iterations.

Before finalizing the approach, I would greatly appreciate your thoughts
and suggestions on this matter.
Should we prioritize the initial implementation without exactly-once
support, or would you advise incorporating it right from the start?
Your insights and experiences would be immensely valuable in making this
decision.


[1]
https://docs.aws.amazon.com/redshift/latest/dg/t_updating-inserting-using-staging-tables-.html
[2] https://docs.aws.amazon.com/redshift/latest/dg/merge-examples.html

Bests,
Samrat

On Mon, Jun 5, 2023 at 7:09 PM Jing Ge  wrote:

> Hi Samrat,
>
> Thanks for the feedback. I would suggest adding that information into the
> FLIP.
>
> +1 Looking forward to your PR :-)
>
> Best regards,
> Jing
>
> On Sat, Jun 3, 2023 at 9:19 PM Samrat Deb  wrote:
>
> > Hi Jing Ge,
> >
> > >>> Do you already have any prototype? I'd like to join the reviews.
> > The prototype is in progress. I will raise the dedicated PR for review
> soon
> > also notify in this thread as well .
> >
> > >>> Will the Redshift connector provide additional features
> > beyond the mediator/wrapper of the jdbc connector?
> >
> > Here are the additional features that the Flink connector for AWS
> Redshift
> > can provide on top of using JDBC:
> >
> > 1. Integration with AWS Redshift Workload Management (WLM): AWS Redshift
> > allows you to configure WLM[1] to manage query prioritization and
> resource
> > allocation. The Flink connector for Redshift will be agnostic to the
> > configured WLM and utilize it for scaling in and out for the sink. This
> > means that the connector can leverage the WLM capabilities of Redshift to
> > optimize the execution of queries and allocate resources efficiently
> based
> > on your defined workload priorities.
> >
> > 2. Abstraction of AWS Redshift Quotas and Limits: AWS Redshift imposes
> > certain quotas and limits[2] on various aspects such as the number of
> > clusters, concurrent connections, queries per second, etc. The Flink
> > connector for Redshift will provide an abstraction layer for users,
> > allowing them to work with Redshift without having to worry about these
> > specific limits. The connector will handle the management of connections
> > and queries within the defined quotas and limits, abstracting away the
> > complexity and ensuring compliance with Redshift's restrictions.
> >
> > These features aim to simplify the integration of Flink with AWS
> Redshift,
> > providing optimized resource utilization and transparent handling of
> > Redshift-specific limitations.
> >
> > Bests,
> > Samrat
> >
> > [1]
> >
> >
> https://docs.aws.amazon.com/redshift/latest/dg/cm-c-implementing-workload-management.html
> > [2]
> >
> >
> https://docs.aws.amazon.com/redshift/latest/mgmt/amazon-redshift-limits.html
> >
> > On Sat, Jun 3, 2023 at 11:40 PM Samrat Deb 
> wrote:
> >
> > > Hi Ahmed,
> > >
> > > >>> please let me know If you need any collaboration regarding
> > integration
> > > with
> > > AWS connectors credential providers or regarding FLIP-171 I would be
> more
> > > than happy to assist.
> > >
> > > Sure, I will reach out incase of any hands required.
> > >
> > >
> > >
> > > On Fri, Jun 2, 2023 at 6:12 PM Jing Ge 
> > wrote:
> > >
> > >> Hi Samrat,
> > >>
> > >> Excited to see your proposal. Supporting data warehouses is one of the
> > >> major tracks for Flink. Thanks for driving it! Happy to see that we
> > >> reached
> > >> consensus to prioritize the Sink over Source in the previous
> discussion.
> > >> Do
> > >> you already have any prototype? I'd like to join the reviews.
> > >>
> > >> Just out of curiosity, speaking of JDBC mode, according to the FLIP,
> it
> > >> should be doable to directly use the jdbc connector with Redshift, if
> I
> > am
> > >> not mistaken. Will the Redshift conn

Re: [DISCUSS] FLIP 295: Support persistence of Catalog configuration and asynchronous registration

[Samrat Deb]

Hi Feng,

Thank you for providing the proposal. I believe this feature will be highly
valuable.

I have a couple of inquiries:

1. According to the documentation [1], the createCatalog method does not
mention any exceptions being thrown. However, I would like to confirm if it
is always true that there will be no failures in all scenarios. Please let
me know if there is any additional information I may have missed.

2. Regarding the registration process using the `table.catalog-store.kind`
configuration, could you please provide an exhaustive list of the supported
kinds?
   It would be great to have a comprehensive understanding of the options
available.

Bests,
Samrat

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-295%3A+Support+lazy+initialization+of+catalogs+and+persistence+of+catalog+configurations

On Sat, Jun 3, 2023 at 5:54 PM Feng Jin  wrote:

> Hi Hang and Jark
>
> Thank you very much for your reply.
>
> @Jark
> > 1. Could you move the CatalogStore registration API to the "Public
> Interface" section?
>
> Indeed, it is more reasonable.
>
> > 2. We should prefix "table." for the CatalogStore configuration.
>
> Sure,   the names that you provided are indeed more logical and easier to
> understand.
>
> > 3. About the open/close method in CatalogStoreFactory.
>
> I also agree with the proposed requirement scenario and design. I have
> already made modifications to the interface.
>
> @Hang
>
> > 1. The `CatalogStore` need the `open`/`close` methods to init or close
> the resource.
>
> I have already added the missing method.
>
> > 2. The `getCatalog` is misspelled as `optionalDescriptor`.
>
> Sorry for the misspelling, I updated this part.
>
> > 3.  Then the `CatalogStoreFactory` may need the `open`/`close` methods
> to  init or close its resource
>
> This is indeed a scenario that needs to be considered, and I have added
> relevant methods accordingly.
> Additionally, I have included some relevant description information in the
> documentation to help others understand
>  why the open/close methods need to be added.
>
>
> Best,
> Feng
>
> On Sat, Jun 3, 2023 at 12:35 PM Jark Wu  wrote:
>
> > Hi Jing,
> >
> > Thank you for the update.
> >
> > 1. Could you move the CatalogStore registration API to the "Public
> > Interface" section?
> > "Proposed Changes" is more like a place to describe the implementation
> > details.
> >
> > 2. We should prefix "table." for the CatalogStore configuration. Besides,
> > the config key
> > name should be hierarchical[1]. Therefore, it may be better to use:
> > "table.catalog-store.kind"
> > "table.catalog-store.file.path"
> >
> > 3. I think Hang's suggestions make sense.
> >
> > Others look good to me.
> >
> > Best,
> > Jark
> >
> > On Fri, 2 Jun 2023 at 17:28, Hang Ruan  wrote:
> >
> > > Hi, Feng.
> > >
> > > Thanks for the update.
> > > The current design looks good to me. I have some minor comments.
> > >
> > > 1. The `CatalogStore` need the `open`/`close` methods to init or close
> > the
> > > resource. For example, when storing the information in MySQL, the store
> > > needs to open and close the connections.
> > >
> > > 2. The `getCatalog` is misspelled as `optionalDescriptor`.
> > >
> > > 3. About the usage in the sql gateway.
> > > Considering the usage in sql gateway, the sql gateway may create a
> > > CatalogStore for each session.
> > > If we are using the MySqlCatalogStore, there would be so many
> > connections.
> > > How can we reuse the connection among these sessions?
> > > I think sql gateway need to maintain a connection pool in
> > > the CatalogStoreFactory and each session get its own connection from
> the
> > > pool when it is created.
> > > Then the `CatalogStoreFactory` may need the `open`/`close` methods to
> > init
> > > or close its resource.
> > > Or is there a better way?
> > >
> > > Best,
> > > Hang
> > >
> > > Feng Jin  于2023年6月2日周五 14:45写道：
> > >
> > > > Thanks Jingsong.
> > > >
> > > > > Just naming, maybe `createCatalog` in TableEnv
> > > >
> > > > +1 For this, I have already updated FLIP.
> > > >
> > > >
> > > > Best,
> > > > Feng
> > > >
> > > >
> > > > On Fri, Jun 2, 2023 at 11:32 AM Jingsong Li 
> > > > wrote:
> > > >
> > > > > Thanks Feng,
> > > > >
> > > > > Just naming, maybe `createCatalog` in TableEnv, I can see many
> > > > > functions are converted to createxxx from registerxxx.
> > > > >
> > > > > On Fri, Jun 2, 2023 at 11:04 AM Feng Jin 
> > > wrote:
> > > > > >
> > > > > > Hi jark, thanks for your suggestion.
> > > > > >
> > > > > > > 1. How to register the CatalogStore for Table API? I think the
> > > > > > CatalogStore should be immutable once TableEnv is created.
> > Otherwise,
> > > > > there
> > > > > > might be some data inconsistencies when CatalogStore is changed.
> > > > > >
> > > > > > Yes, We should initialize the CatalogStore when creating the
> > > TableEnv.
> > > > > > Therefore, my current proposal is to add a method to configure
> the
> > > > > > CatalogStore in EnvironmentSettings.
> > > > > >
> 

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Jing Ge,

>>> Do you already have any prototype? I'd like to join the reviews.
The prototype is in progress. I will raise the dedicated PR for review soon
also notify in this thread as well .

>>> Will the Redshift connector provide additional features
beyond the mediator/wrapper of the jdbc connector?

Here are the additional features that the Flink connector for AWS Redshift
can provide on top of using JDBC:

1. Integration with AWS Redshift Workload Management (WLM): AWS Redshift
allows you to configure WLM[1] to manage query prioritization and resource
allocation. The Flink connector for Redshift will be agnostic to the
configured WLM and utilize it for scaling in and out for the sink. This
means that the connector can leverage the WLM capabilities of Redshift to
optimize the execution of queries and allocate resources efficiently based
on your defined workload priorities.

2. Abstraction of AWS Redshift Quotas and Limits: AWS Redshift imposes
certain quotas and limits[2] on various aspects such as the number of
clusters, concurrent connections, queries per second, etc. The Flink
connector for Redshift will provide an abstraction layer for users,
allowing them to work with Redshift without having to worry about these
specific limits. The connector will handle the management of connections
and queries within the defined quotas and limits, abstracting away the
complexity and ensuring compliance with Redshift's restrictions.

These features aim to simplify the integration of Flink with AWS Redshift,
providing optimized resource utilization and transparent handling of
Redshift-specific limitations.

Bests,
Samrat

[1]
https://docs.aws.amazon.com/redshift/latest/dg/cm-c-implementing-workload-management.html
[2]
https://docs.aws.amazon.com/redshift/latest/mgmt/amazon-redshift-limits.html

On Sat, Jun 3, 2023 at 11:40 PM Samrat Deb  wrote:

> Hi Ahmed,
>
> >>> please let me know If you need any collaboration regarding integration
> with
> AWS connectors credential providers or regarding FLIP-171 I would be more
> than happy to assist.
>
> Sure, I will reach out incase of any hands required.
>
>
>
> On Fri, Jun 2, 2023 at 6:12 PM Jing Ge  wrote:
>
>> Hi Samrat,
>>
>> Excited to see your proposal. Supporting data warehouses is one of the
>> major tracks for Flink. Thanks for driving it! Happy to see that we
>> reached
>> consensus to prioritize the Sink over Source in the previous discussion.
>> Do
>> you already have any prototype? I'd like to join the reviews.
>>
>> Just out of curiosity, speaking of JDBC mode, according to the FLIP, it
>> should be doable to directly use the jdbc connector with Redshift, if I am
>> not mistaken. Will the Redshift connector provide additional features
>> beyond the mediator/wrapper of the jdbc connector?
>>
>> Best regards,
>> Jing
>>
>> On Thu, Jun 1, 2023 at 8:22 PM Ahmed Hamdy  wrote:
>>
>> > Hi Samrat
>> >
>> > Thanks for putting up this FLIP. I agree regarding the importance of the
>> > use case.
>> > please let me know If you need any collaboration regarding integration
>> with
>> > AWS connectors credential providers or regarding FLIP-171 I would be
>> more
>> > than happy to assist.
>> > I also like Leonard's proposal for starting with DataStreamSink and
>> > TableSink, It would be great to have some milestones delivered as soon
>> as
>> > ready.
>> > best regards
>> > Ahmed Hamdy
>> >
>> >
>> > On Wed, 31 May 2023 at 11:15, Samrat Deb  wrote:
>> >
>> > > Hi Liu Ron,
>> > >
>> > > > 1. Regarding the  `read.mode` and `write.mode`, you say here
>> provides
>> > two
>> > > modes, respectively, jdbc and `unload or copy`, What is the default
>> value
>> > > for `read.mode` and `write.mode?
>> > >
>> > > I have made an effort to make the configuration options `read.mode`
>> and
>> > > `write.mode` mandatory for the "flink-connector-redshift" according to
>> > > FLIP[1]. The rationale behind this decision is to empower users who
>> are
>> > > familiar with their Redshift setup and have specific expectations for
>> the
>> > > sink. By making these configurations mandatory, users can have more
>> > control
>> > > and flexibility in configuring the connector to meet their
>> requirements.
>> > >
>> > > However, I am open to receiving feedback on whether it would be
>> > beneficial
>> > > to make the configuration options non-mandatory and set default values
>> > for
>> &g

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Ahmed,

>>> please let me know If you need any collaboration regarding integration
with
AWS connectors credential providers or regarding FLIP-171 I would be more
than happy to assist.

Sure, I will reach out incase of any hands required.



On Fri, Jun 2, 2023 at 6:12 PM Jing Ge  wrote:

> Hi Samrat,
>
> Excited to see your proposal. Supporting data warehouses is one of the
> major tracks for Flink. Thanks for driving it! Happy to see that we reached
> consensus to prioritize the Sink over Source in the previous discussion. Do
> you already have any prototype? I'd like to join the reviews.
>
> Just out of curiosity, speaking of JDBC mode, according to the FLIP, it
> should be doable to directly use the jdbc connector with Redshift, if I am
> not mistaken. Will the Redshift connector provide additional features
> beyond the mediator/wrapper of the jdbc connector?
>
> Best regards,
> Jing
>
> On Thu, Jun 1, 2023 at 8:22 PM Ahmed Hamdy  wrote:
>
> > Hi Samrat
> >
> > Thanks for putting up this FLIP. I agree regarding the importance of the
> > use case.
> > please let me know If you need any collaboration regarding integration
> with
> > AWS connectors credential providers or regarding FLIP-171 I would be more
> > than happy to assist.
> > I also like Leonard's proposal for starting with DataStreamSink and
> > TableSink, It would be great to have some milestones delivered as soon as
> > ready.
> > best regards
> > Ahmed Hamdy
> >
> >
> > On Wed, 31 May 2023 at 11:15, Samrat Deb  wrote:
> >
> > > Hi Liu Ron,
> > >
> > > > 1. Regarding the  `read.mode` and `write.mode`, you say here provides
> > two
> > > modes, respectively, jdbc and `unload or copy`, What is the default
> value
> > > for `read.mode` and `write.mode?
> > >
> > > I have made an effort to make the configuration options `read.mode` and
> > > `write.mode` mandatory for the "flink-connector-redshift" according to
> > > FLIP[1]. The rationale behind this decision is to empower users who are
> > > familiar with their Redshift setup and have specific expectations for
> the
> > > sink. By making these configurations mandatory, users can have more
> > control
> > > and flexibility in configuring the connector to meet their
> requirements.
> > >
> > > However, I am open to receiving feedback on whether it would be
> > beneficial
> > > to make the configuration options non-mandatory and set default values
> > for
> > > them. If you believe there are advantages to having default values or
> any
> > > other suggestions, please share your thoughts. Your feedback is highly
> > > appreciated.
> > >
> > > >  2. For Source, does it both support batch read and streaming read?
> > >
> > > Redshift currently does not provide native support for streaming reads,
> > > although it does support streaming writes[2]. As part of the plan, I
> > intend
> > > to conduct a proof of concept and benchmarking to explore the
> > possibilities
> > > of implementing streaming reads using the Flink JDBC connector, as
> > Redshift
> > > is JDBC compatible.
> > > However, it is important to note that, in the initial phase of
> > > implementation, the focus will primarily be on supporting batch reads
> > > rather than streaming reads. This approach will allow us to deliver a
> > > robust and reliable solution for batch processing in phase 2 of the
> > > implementation.
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
> > > [2]
> > >
> > >
> >
> https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-streaming-ingestion.html
> > >
> > > Bests,
> > > Samrat
> > >
> > > On Wed, May 31, 2023 at 8:03 AM liu ron  wrote:
> > >
> > > > Hi, Samrat
> > > >
> > > > Thanks for driving this FLIP. It looks like supporting
> > > > flink-connector-redshift is very useful to Flink. I have two
> question:
> > > > 1. Regarding the  `read.mode` and `write.mode`, you say here provides
> > two
> > > > modes, respectively, jdbc and `unload or copy`, What is the default
> > value
> > > > for `read.mode` and `write.mode?
> > > > 2. For Source, does it both support batch read and streaming read?
> > > >
> > > >
> > 

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Liu Ron,

> 1. Regarding the  `read.mode` and `write.mode`, you say here provides two
modes, respectively, jdbc and `unload or copy`, What is the default value
for `read.mode` and `write.mode?

I have made an effort to make the configuration options `read.mode` and
`write.mode` mandatory for the "flink-connector-redshift" according to
FLIP[1]. The rationale behind this decision is to empower users who are
familiar with their Redshift setup and have specific expectations for the
sink. By making these configurations mandatory, users can have more control
and flexibility in configuring the connector to meet their requirements.

However, I am open to receiving feedback on whether it would be beneficial
to make the configuration options non-mandatory and set default values for
them. If you believe there are advantages to having default values or any
other suggestions, please share your thoughts. Your feedback is highly
appreciated.

>  2. For Source, does it both support batch read and streaming read?

Redshift currently does not provide native support for streaming reads,
although it does support streaming writes[2]. As part of the plan, I intend
to conduct a proof of concept and benchmarking to explore the possibilities
of implementing streaming reads using the Flink JDBC connector, as Redshift
is JDBC compatible.
However, it is important to note that, in the initial phase of
implementation, the focus will primarily be on supporting batch reads
rather than streaming reads. This approach will allow us to deliver a
robust and reliable solution for batch processing in phase 2 of the
implementation.

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
[2]
https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-streaming-ingestion.html

Bests,
Samrat

On Wed, May 31, 2023 at 8:03 AM liu ron  wrote:

> Hi, Samrat
>
> Thanks for driving this FLIP. It looks like supporting
> flink-connector-redshift is very useful to Flink. I have two question:
> 1. Regarding the  `read.mode` and `write.mode`, you say here provides two
> modes, respectively, jdbc and `unload or copy`, What is the default value
> for `read.mode` and `write.mode?
> 2. For Source, does it both support batch read and streaming read?
>
>
> Best,
> Ron
>
> Samrat Deb  于2023年5月30日周二 17:15写道：
>
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
> >
> > [note] Missed the trailing link for previous mail
> >
> >
> >
> > On Tue, May 30, 2023 at 2:43 PM Samrat Deb 
> wrote:
> >
> > > Hi Leonard,
> > >
> > > > and I’m glad to help review the design as well as the code review.
> > > Thank you so much. It would be really great and helpful to bring
> > > flink-connector-redshift for flink users :) .
> > >
> > > I have divided the implementation in 3 phases in the `Scope`
> Section[1].
> > > 1st phase is to
> > >
> > >- Integrate with Flink Sink API (*FLIP-171*
> > ><
> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink
> >
> > >)
> > >
> > >
> > > > About the implementation phases, How about prioritizing support for
> the
> > > Datastream Sink API and TableSink API in the first phase?
> > > I can completely resonate with you to prioritize support for Datastream
> > > Sink API and TableSink API in the first phase.
> > > I will update the FLIP[1] as you have suggested.
> > >
> > > > It seems that the primary use cases for the Redshift connector are
> > > acting as a sink for processed data by Flink.
> > > Yes, majority ask and requirement for Redshift connector is sink for
> > > processed data by Flink.
> > >
> > > Bests,
> > > Samrat
> > >
> > > On Tue, May 30, 2023 at 12:35 PM Leonard Xu  wrote:
> > >
> > >> Thanks @Samrat for bringing this discussion.
> > >>
> > >> It makes sense to me to introduce AWS Redshift connector for Apache
> > >> Flink, and I’m glad to help review the design as well as the code
> > review.
> > >>
> > >> About the implementation phases, How about prioritizing support for
> the
> > >> Datastream Sink API and TableSink API in the first phase? It seems
> that
> > the
> > >> primary use cases for the Redshift connector are acting as a sink for
> > >> processed data by Flink.
> > >>
> > >> Best,
> > >> Leonard
> > >>
> > >>
> > >> > On May 29, 2023, at 12:51 PM, Samrat Deb 
> >

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift

[note] Missed the trailing link for previous mail



On Tue, May 30, 2023 at 2:43 PM Samrat Deb  wrote:

> Hi Leonard,
>
> > and I’m glad to help review the design as well as the code review.
> Thank you so much. It would be really great and helpful to bring
> flink-connector-redshift for flink users :) .
>
> I have divided the implementation in 3 phases in the `Scope` Section[1].
> 1st phase is to
>
>- Integrate with Flink Sink API (*FLIP-171*
><https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink>
>)
>
>
> > About the implementation phases, How about prioritizing support for the
> Datastream Sink API and TableSink API in the first phase?
> I can completely resonate with you to prioritize support for Datastream
> Sink API and TableSink API in the first phase.
> I will update the FLIP[1] as you have suggested.
>
> > It seems that the primary use cases for the Redshift connector are
> acting as a sink for processed data by Flink.
> Yes, majority ask and requirement for Redshift connector is sink for
> processed data by Flink.
>
> Bests,
> Samrat
>
> On Tue, May 30, 2023 at 12:35 PM Leonard Xu  wrote:
>
>> Thanks @Samrat for bringing this discussion.
>>
>> It makes sense to me to introduce AWS Redshift connector for Apache
>> Flink, and I’m glad to help review the design as well as the code review.
>>
>> About the implementation phases, How about prioritizing support for the
>> Datastream Sink API and TableSink API in the first phase? It seems that the
>> primary use cases for the Redshift connector are acting as a sink for
>> processed data by Flink.
>>
>> Best,
>> Leonard
>>
>>
>> > On May 29, 2023, at 12:51 PM, Samrat Deb  wrote:
>> >
>> > Hello all ,
>> >
>> > Context:
>> > Amazon Redshift [1] is a fully managed, petabyte-scale data warehouse
>> > service in the cloud. It allows analyzing data without all of the
>> > configurations of a provisioned data warehouse. Resources are
>> automatically
>> > provisioned and data warehouse capacity is intelligently scaled to
>> deliver
>> > fast performance for even the most demanding and unpredictable
>> workloads.
>> > Redshift is one of the widely used warehouse solutions in the current
>> > market.
>> >
>> > Building flink connector redshift will allow flink users to have source
>> and
>> > sink directly to redshift. It will help flink to expand the scope to
>> > redshift as a new connector in the ecosystem.
>> >
>> > I would like to start a discussion on the FLIP-307: Flink connector
>> > redshift [2].
>> > Looking forward to comments, feedbacks and suggestions from the
>> community
>> > on the proposal.
>> >
>> > [1] https://docs.aws.amazon.com/redshift/latest/mgmt/welcome.html
>> > [2]
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
>> >
>> >
>> >
>> > Bests,
>> > Samrat
>>
>>

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Leonard,

> and I’m glad to help review the design as well as the code review.
Thank you so much. It would be really great and helpful to bring
flink-connector-redshift for flink users :) .

I have divided the implementation in 3 phases in the `Scope` Section[1].
1st phase is to

   - Integrate with Flink Sink API (*FLIP-171*
   <https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink>
   )


> About the implementation phases, How about prioritizing support for the
Datastream Sink API and TableSink API in the first phase?
I can completely resonate with you to prioritize support for Datastream
Sink API and TableSink API in the first phase.
I will update the FLIP[1] as you have suggested.

> It seems that the primary use cases for the Redshift connector are acting
as a sink for processed data by Flink.
Yes, majority ask and requirement for Redshift connector is sink for
processed data by Flink.

Bests,
Samrat

On Tue, May 30, 2023 at 12:35 PM Leonard Xu  wrote:

> Thanks @Samrat for bringing this discussion.
>
> It makes sense to me to introduce AWS Redshift connector for Apache Flink,
> and I’m glad to help review the design as well as the code review.
>
> About the implementation phases, How about prioritizing support for the
> Datastream Sink API and TableSink API in the first phase? It seems that the
> primary use cases for the Redshift connector are acting as a sink for
> processed data by Flink.
>
> Best,
> Leonard
>
>
> > On May 29, 2023, at 12:51 PM, Samrat Deb  wrote:
> >
> > Hello all ,
> >
> > Context:
> > Amazon Redshift [1] is a fully managed, petabyte-scale data warehouse
> > service in the cloud. It allows analyzing data without all of the
> > configurations of a provisioned data warehouse. Resources are
> automatically
> > provisioned and data warehouse capacity is intelligently scaled to
> deliver
> > fast performance for even the most demanding and unpredictable workloads.
> > Redshift is one of the widely used warehouse solutions in the current
> > market.
> >
> > Building flink connector redshift will allow flink users to have source
> and
> > sink directly to redshift. It will help flink to expand the scope to
> > redshift as a new connector in the ecosystem.
> >
> > I would like to start a discussion on the FLIP-307: Flink connector
> > redshift [2].
> > Looking forward to comments, feedbacks and suggestions from the community
> > on the proposal.
> >
> > [1] https://docs.aws.amazon.com/redshift/latest/mgmt/welcome.html
> > [2]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
> >
> >
> >
> > Bests,
> > Samrat
>
>

Re: [DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hi Yaroslav Tkachenko,

Thanks for pointing that out. I have added it in the limitation section.
For now its ok to put it in the limitation section as per initial
implementation.

Bests,
Samrat

On Mon, May 29, 2023 at 9:50 PM Yaroslav Tkachenko
 wrote:

> Hi Samrat,
>
> Regarding the sink, will it only support append-only tables (no changelog
> mode support)? It looks like it's the case, so IMO, it should be mentioned
> in the limitations section.
>
> On Sun, May 28, 2023 at 9:52 PM Samrat Deb  wrote:
>
> > Hello all ,
> >
> > Context:
> > Amazon Redshift [1] is a fully managed, petabyte-scale data warehouse
> > service in the cloud. It allows analyzing data without all of the
> > configurations of a provisioned data warehouse. Resources are
> automatically
> > provisioned and data warehouse capacity is intelligently scaled to
> deliver
> > fast performance for even the most demanding and unpredictable workloads.
> > Redshift is one of the widely used warehouse solutions in the current
> > market.
> >
> > Building flink connector redshift will allow flink users to have source
> and
> > sink directly to redshift. It will help flink to expand the scope to
> > redshift as a new connector in the ecosystem.
> >
> > I would like to start a discussion on the FLIP-307: Flink connector
> > redshift [2].
> > Looking forward to comments, feedbacks and suggestions from the community
> > on the proposal.
> >
> > [1] https://docs.aws.amazon.com/redshift/latest/mgmt/welcome.html
> > [2]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift
> >
> >
> >
> > Bests,
> > Samrat
> >
>

[DISCUSS] FLIP-307: Flink connector Redshift

[Samrat Deb]

Hello all ,

Context:
Amazon Redshift [1] is a fully managed, petabyte-scale data warehouse
service in the cloud. It allows analyzing data without all of the
configurations of a provisioned data warehouse. Resources are automatically
provisioned and data warehouse capacity is intelligently scaled to deliver
fast performance for even the most demanding and unpredictable workloads.
Redshift is one of the widely used warehouse solutions in the current
market.

Building flink connector redshift will allow flink users to have source and
sink directly to redshift. It will help flink to expand the scope to
redshift as a new connector in the ecosystem.

I would like to start a discussion on the FLIP-307: Flink connector
redshift [2].
Looking forward to comments, feedbacks and suggestions from the community
on the proposal.

[1] https://docs.aws.amazon.com/redshift/latest/mgmt/welcome.html
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift



Bests,
Samrat

[jira] [Created] (FLINK-32176) [CVE-2022-1471] Upgrade snake yaml to 2.0

[Samrat Deb (Jira)]

Samrat Deb created FLINK-32176:
--

 Summary: [CVE-2022-1471] Upgrade snake yaml to 2.0
 Key: FLINK-32176
 URL: https://issues.apache.org/jira/browse/FLINK-32176
 Project: Flink
  Issue Type: Bug
Affects Versions: 1.17.0
Reporter: Samrat Deb


 
 * *CVE ID:* {{CVE-2022-1471}}

 * *CWE:* CWE-502 Deserialization of Untrusted Data

 * {*}Severity{*}: Critical

{{pulsar-client-all-2.10.0.jar (shaded: org.yaml:snakeyaml:1.30)}}

 

{{SnakeYaml's Constructor() class does not restrict types which can be 
instantiated during deserialization. Deserializing yaml content provided by an 
attacker can lead to remote code execution. We recommend using SnakeYaml's 
SafeConsturctor when parsing untrusted content to restrict deserialization.}}

{{}}

{{More details : https://nvd.nist.gov/vuln/detail/CVE-2022-1471}}

{{{}{}}}{{{}{}}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release 1.16.2, release candidate #1

[Samrat Deb]

+1 (non-binding)

- Verified signature
- Verified hashes
- Build form source with mac
- Verify that the source archives do not contain any binaries
- Run streaming and batch job in sql-client successfully.

Thank you for driving this release candidate.

On Tue, 23 May 2023 at 1:51 PM, Qingsheng Ren  wrote:

> +1 (binding)
>
> - Verified checksums and signatures
> - Verified that no binary exist in the source archive
> - Build the source with Maven
> - Verified that README doesn't have unexpected contents
> - Verified that the commit ID on Flink UI matches with the tag
> - Run a job with SQL client that reads from a Kafka topic then writes to
> another one. Created a savepoint for the job and recovered from savepoint
> successfully
> - Reviewed release announcement PR
>
> Best regards,
> Qingsheng
>
> On Mon, May 22, 2023 at 8:34 PM Xingbo Huang  wrote:
>
> > +1 (binding)
> >
> > - verify signatures and checksums
> > - verify python wheel package contents
> > - pip install apache-flink-libraries and apache-flink wheel packages
> > - run example
> flink/flink-python/pyflink/examples/table/basic_operations.py
> > with Python 3.7
> > - reviewed release blog post
> >
> > Best,
> > Xingbo
> >
> > Xintong Song  于2023年5月21日周日 13:03写道：
> >
> > > +1 (binding)
> > >
> > > - verified signatures and checksums
> > > - built from source
> > > - tried example jobs with a standalone cluster, everything seems fine
> > > - review release announcement PR
> > >
> > > Best,
> > >
> > > Xintong
> > >
> > >
> > >
> > > On Sat, May 20, 2023 at 6:04 PM Jing Ge 
> > > wrote:
> > >
> > > > +1(non-binding)
> > > >
> > > > - reviewed Jira release notes
> > > > - built from source
> > > > - apache repos contain all necessary files
> > > > - verified signatures
> > > > - verified hashes
> > > > - verified tag
> > > > - reviewed PR
> > > >
> > > > Best regards,
> > > > Jing
> > > >
> > > > On Sat, May 20, 2023 at 11:51 AM Yun Tang  wrote:
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > >
> > > > >   *   Verified signatures
> > > > >   *   Reviewed the flink-web PR
> > > > >   *   Set up a standalone cluster from released binaries and check
> > the
> > > > git
> > > > > revision number.
> > > > >   *   Submit the statemachine example with RocksDB, and it works
> > fine.
> > > > >
> > > > > Best,
> > > > > Yun Tang
> > > > > 
> > > > > From: Yuxin Tan 
> > > > > Sent: Friday, May 19, 2023 17:41
> > > > > To: dev@flink.apache.org 
> > > > > Subject: Re: [VOTE] Release 1.16.2, release candidate #1
> > > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > - Verified signature
> > > > > - Verified hashes
> > > > > - Build form source with mac
> > > > > - Verify that the source archives do not contain any binaries
> > > > > - Run streaming and batch job in sql-client successfully.
> > > > >
> > > > > Thanks weijie for driving this release candidate.
> > > > >
> > > > > Best,
> > > > > Yuxin
> > > > >
> > > > >
> > > > > weijie guo  于2023年5月19日周五 16:19写道：
> > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > >
> > > > > > Please review and vote on the release candidate #1 for the
> version
> > > > > 1.16.2,
> > > > > >
> > > > > > as follows:
> > > > > >
> > > > > >
> > > > > > [ ] +1, Approve the release
> > > > > >
> > > > > > [ ] -1, Do not approve the release (please provide specific
> > comments)
> > > > > >
> > > > > >
> > > > > > The complete staging area is available for your review, which
> > > includes:
> > > > > >
> > > > > > * JIRA release notes [1],
> > > > > >
> > > > > > * the official Apache source release and binary convenience
> > releases
> > > to
> > > > > be
> > > > > >
> > > > > > deployed to dist.apache.org [2], which are signed with the key
> > with
> > > > > >
> > > > > > fingerprint 8D56AE6E7082699A4870750EA4E8C4C05EE6861F [3],
> > > > > >
> > > > > > * all artifacts to be deployed to the Maven Central Repository
> [4],
> > > > > >
> > > > > > * source code tag "release-1.16.2-rc1" [5],
> > > > > >
> > > > > > * website pull request listing the new release and adding
> > > announcement
> > > > > blog
> > > > > >
> > > > > > post [6].
> > > > > >
> > > > > >
> > > > > > The vote will be open for at least 72 hours. It is adopted by
> > > majority
> > > > > >
> > > > > > approval, with at least 3 PMC affirmative votes.
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Release Manager
> > > > > >
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352765
> > > > > >
> > > > > > [2]
> https://dist.apache.org/repos/dist/dev/flink/flink-1.16.2-rc1/
> > > > > >
> > > > > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > > > > >
> > > > > > [4]
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapacheflink-1634/
> > > > > >
> > > > > > [5]
> > https://github.com/apache/flink/releases/tag/release-1.16.2-rc1
> > > > > 

[jira] [Created] (FLINK-32096) Flink 1.17 doc points to non-existant branch in flink-training repo

[Samrat Deb (Jira)]

Samrat Deb created FLINK-32096:
--

 Summary: Flink 1.17 doc points to non-existant branch in 
flink-training repo
 Key: FLINK-32096
 URL: https://issues.apache.org/jira/browse/FLINK-32096
 Project: Flink
  Issue Type: Bug
  Components: Documentation, Documentation / Training / Exercises
Affects Versions: 1.17.0
Reporter: Samrat Deb


There is one broken link in Flink documentation which i think needs commiter or 
PMC privileges.
In 
[Hands-on|https://nightlies.apache.org/flink/flink-docs-stable/docs/learn-flink/etl/#hands-on]
 section

>  The hands-on exercise that goes with this section is the Rides and Fares .

[Rides and 
Fares|https://github.com/apache/flink-training/blob/release-1.17//rides-and-fares]
 points to non-existent branch in flink-training repo and leads to 404 (Not 
Found)  
[hyper-link|https://github.com/apache/flink-training/blob/release-1.17//rides-and-fares].
 This is due to missing `release-1.17` branch in 
[flink-training|https://github.com/apache/flink-training/] repo.
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-31979) Constructing Glue Client using AwsGeneralUtils

[Samrat Deb (Jira)]

Samrat Deb created FLINK-31979:
--

 Summary: Constructing Glue Client using AwsGeneralUtils
 Key: FLINK-31979
 URL: https://issues.apache.org/jira/browse/FLINK-31979
 Project: Flink
  Issue Type: Sub-task
Reporter: Samrat Deb






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-31975) default catalog failed to retrieve partition Spec

[Samrat Deb (Jira)]

Samrat Deb created FLINK-31975:
--

 Summary: default catalog failed to retrieve partition Spec
 Key: FLINK-31975
 URL: https://issues.apache.org/jira/browse/FLINK-31975
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Client
Affects Versions: 1.16.0
Reporter: Samrat Deb


Here is the attached Repro for the error . 
-  Flink 1.16.0 cluster 



 

 
{code:java}
Flink SQL> show current catalog
> ;
+--+
| current catalog name |
+--+
|      default_catalog |
+--+
1 row in set
Flink SQL> show tables;
+---+
|        table name |
+---+
| country_page_view |
|  page_view_source |
|        part_table |
+---+
3 rows in set
Flink SQL> drop table page_view_source;
[INFO] Execute statement succeed.
Flink SQL> drop table country_page_view;
[INFO] Execute statement succeed.
Flink SQL> CREATE TABLE  page_view_source (`user` STRING, `cnt` INT, `date` 
STRING, `country` STRING)
> WITH (
>   'connector' = 'datagen',  'number-of-rows' = '10'
> );
[INFO] Execute statement succeed.
Flink SQL> CREATE TABLE country_page_view (`user` STRING, `cnt` INT, `date` 
STRING, `country` STRING)
> PARTITIONED BY (`date`, `country`)
> WITH (
>
>    'format' = 'csv',
>    'path' = 's3://dbsamrat-emr-dev/glue-catalog/dbsamrat/country_page_view/',
>    'connector' = 'filesystem'
> )
> ;
[INFO] Execute statement succeed.
Flink SQL> INSERT INTO country_page_view PARTITION (`date`='2019-8-30', 
`country`='China')
>   SELECT `user`, `cnt` FROM page_view_source;
>
[INFO] Submitting SQL update statement to the cluster...
2023-04-29 09:51:36,133 INFO  
org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider [] - 
Connecting to ResourceManager at 
ip-172-31-38-72.us-west-2.compute.internal/172.31.38.72:8032
2023-04-29 09:51:36,134 INFO  org.apache.hadoop.yarn.client.AHSProxy            
           [] - Connecting to Application History server at 
ip-172-31-38-72.us-west-2.compute.internal/172.31.38.72:10200
2023-04-29 09:51:36,135 INFO  org.apache.flink.yarn.YarnClusterDescriptor       
           [] - No path for the flink jar passed. Using the location of class 
org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2023-04-29 09:51:36,135 WARN  org.apache.flink.yarn.YarnClusterDescriptor       
           [] - Neither the HADOOP_CONF_DIR nor the YARN_CONF_DIR environment 
variable is set.The Flink YARN Client needs one of these to be set to properly 
load the Hadoop configuration for accessing YARN.
2023-04-29 09:51:36,149 INFO  org.apache.flink.yarn.YarnClusterDescriptor       
           [] - Found Web Interface 
ip-172-31-39-51.us-west-2.compute.internal:36583 of application 
'application_1682266531513_0004'.
[INFO] SQL update statement has been successfully submitted to the cluster:
Job ID: 7c39db71be1f1b52e13a72831fed8105

Flink SQL> EXECUTE INSERT INTO country_page_view PARTITION (`date`='2019-8-30', 
`country`='China')
>   SELECT `user`, `cnt` FROM page_view_source;
[INFO] Submitting SQL update statement to the cluster...
2023-04-29 09:51:41,424 INFO  
org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider [] - 
Connecting to ResourceManager at 
ip-172-31-38-72.us-west-2.compute.internal/172.31.38.72:8032
2023-04-29 09:51:41,424 INFO  org.apache.hadoop.yarn.client.AHSProxy            
           [] - Connecting to Application History server at 
ip-172-31-38-72.us-west-2.compute.internal/172.31.38.72:10200
2023-04-29 09:51:41,424 INFO  org.apache.flink.yarn.YarnClusterDescriptor       
           [] - No path for the flink jar passed. Using the location of class 
org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2023-04-29 09:51:41,424 WARN  org.apache.flink.yarn.YarnClusterDescriptor       
           [] - Neither the HADOOP_CONF_DIR nor the YARN_CONF_DIR environment 
variable is set.The Flink YARN Client needs one of these to be set to properly 
load the Hadoop configuration for accessing YARN.
2023-04-29 09:51:41,427 INFO  org.apache.flink.yarn.YarnClusterDescriptor       
           [] - Found Web Interface 
ip-172-31-39-51.us-west-2.compute.internal:36583 of application 
'application_1682266531513_0004'.
[INFO] SQL update statement has been successfully submitted to the cluster:
Job ID: 69e18cb23f505528948a6398390ad070

Flink SQL> INSERT INTO country_page_view PARTITION (`date`='2019-8-30')
>   SELECT `user`, `cnt`, `country` FROM page_view_source;
[INFO] Submitting SQL update statement to the cluster...
2023-04-29 09:51:47,509 INFO  
org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider [] - 
Connecting to ResourceManager at 
ip-172-31-38-72.us-west-2.compute.internal/172.31.38.72:8032
2023-04-29 09:51:47,509 INFO  org.apache.hadoop.yarn.client.AHSProxy            
           

[jira] [Created] (FLINK-31973) Introduce Managed table support in glue catalog

[Samrat Deb (Jira)]

Samrat Deb created FLINK-31973:
--

 Summary: Introduce Managed table support in glue catalog
 Key: FLINK-31973
 URL: https://issues.apache.org/jira/browse/FLINK-31973
 Project: Flink
  Issue Type: Sub-task
Reporter: Samrat Deb


Hive-catalog support managed table. Similarly glue catalog can support managed 
table backed by glue metasore. This task focuses on introducing managed table 
support in glue catalog



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-31961) Implement ExpandedQuery and OriginalQuery methods for glue in catalog

[Samrat Deb (Jira)]

Samrat Deb created FLINK-31961:
--

 Summary: Implement ExpandedQuery and OriginalQuery methods for 
glue in catalog
 Key: FLINK-31961
 URL: https://issues.apache.org/jira/browse/FLINK-31961
 Project: Flink
  Issue Type: Sub-task
  Components: Connectors / AWS
Reporter: Samrat Deb






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-31926) Implement rename Table in GlueCatalog

[Samrat Deb (Jira)]

Samrat Deb created FLINK-31926:
--

 Summary: Implement rename Table in GlueCatalog
 Key: FLINK-31926
 URL: https://issues.apache.org/jira/browse/FLINK-31926
 Project: Flink
  Issue Type: Sub-task
Reporter: Samrat Deb


Glue catalog don't support renaming table. 
Currently marked as unsupported operation . 

This task intend to implement it later.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

[Samrat Deb]

congratulations

On Fri, 21 Apr 2023 at 9:44 PM, David Morávek  wrote:

> Congratulations, Leonard, well deserved!
>
> Best,
> D.
>
> On Fri 21. 4. 2023 at 16:40, Feng Jin  wrote:
>
> > Congratulations, Leonard
> >
> >
> > 
> > Best,
> > Feng Jin
> >
> > On Fri, Apr 21, 2023 at 8:38 PM Mang Zhang  wrote:
> >
> > > Congratulations, Leonard.
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Mang Zhang
> > >
> > >
> > >
> > >
> > >
> > > At 2023-04-21 19:47:52, "Jark Wu"  wrote:
> > > >Hi everyone,
> > > >
> > > >We are thrilled to announce that Leonard Xu has joined the Flink PMC!
> > > >
> > > >Leonard has been an active member of the Apache Flink community for
> many
> > > >years and became a committer in Nov 2021. He has been involved in
> > various
> > > >areas of the project, from code contributions to community building.
> His
> > > >contributions are mainly focused on Flink SQL and connectors,
> especially
> > > >leading the flink-cdc-connectors project to receive 3.8+K GitHub
> stars.
> > He
> > > >authored 150+ PRs, and reviewed 250+ PRs, and drove several FLIPs
> (e.g.,
> > > >FLIP-132, FLIP-162). He has participated in plenty of discussions in
> the
> > > >dev mailing list, answering questions about 500+ threads in the
> > > >user/user-zh mailing list. Besides that, he is community minded, such
> as
> > > >being the release manager of 1.17, verifying releases, managing
> release
> > > >syncs, etc.
> > > >
> > > >Congratulations and welcome Leonard!
> > > >
> > > >Best,
> > > >Jark (on behalf of the Flink PMC)
> > >
> >
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

[Samrat Deb]

congratulations !

On Fri, 21 Apr 2023 at 9:45 PM, David Morávek  wrote:

> Congratulations, Qingsheng, well deserved!
>
> Best,
> D.
>
> On Fri 21. 4. 2023 at 16:41, Feng Jin  wrote:
>
> > Congratulations, Qingsheng
> >
> >
> > 
> > Best,
> > Feng Jin
> >
> > On Fri, Apr 21, 2023 at 8:39 PM Mang Zhang  wrote:
> >
> > > Congratulations, Qingsheng.
> > >
> > >
> > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Mang Zhang
> > >
> > >
> > >
> > >
> > >
> > > At 2023-04-21 19:50:02, "Jark Wu"  wrote:
> > > >Hi everyone,
> > > >
> > > >We are thrilled to announce that Qingsheng Ren has joined the Flink
> PMC!
> > > >
> > > >Qingsheng has been contributing to Apache Flink for a long time. He is
> > the
> > > >core contributor and maintainer of the Kafka connector and
> > > >flink-cdc-connectors, bringing users stability and ease of use in both
> > > >projects. He drove discussions and implementations in FLIP-221,
> > FLIP-288,
> > > >and the connector testing framework. He is continuously helping with
> the
> > > >expansion of the Flink community and has given several talks about
> Flink
> > > >connectors at many conferences, such as Flink Forward Global and Flink
> > > >Forward Asia. Besides that, he is willing to help a lot in the
> community
> > > >work, such as being the release manager for both 1.17 and 1.18,
> > verifying
> > > >releases, and answering questions on the mailing list.
> > > >
> > > >Congratulations and welcome Qingsheng!
> > > >
> > > >Best,
> > > >Jark (on behalf of the Flink PMC)
> > >
> >
>

[jira] [Created] (FLINK-31854) Flink connector redshift

[Samrat Deb (Jira)]

Samrat Deb created FLINK-31854:
--

 Summary: Flink connector redshift 
 Key: FLINK-31854
 URL: https://issues.apache.org/jira/browse/FLINK-31854
 Project: Flink
  Issue Type: New Feature
  Components: Connectors / AWS
Reporter: Samrat Deb


Proposal :

Add new feature (submodule) flink connector redshift in flink-connector-aws.
This connector should be based on FLIP-27 



 

Next steps 
- Create a flip for the flink connector redshift 

 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release flink-connector-mongodb v1.0.1, release candidate #1

[Samrat Deb]

+1 (non binding)

On Fri, 14 Apr 2023 at 1:23 PM, Teoh, Hong 
wrote:

> Thanks Danny
>
> +1 (non-binding)
>
> * Hashes and Signatures look good
> * All required files on dist.apache.org
> * Tag is present in Github
> * Verified source archive does not contain any binary files
> * Source archive builds using maven
>
>
> Cheers,
> Hong
>
>
> > On 14 Apr 2023, at 06:36, Elphas Toringepi  wrote:
> >
> > CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
> >
> >
> >
> > Thanks Danny
> >
> > +1 (non-binding)
> >
> > * Checked JIRA release notes
> > * Verified signature and checksum for Apache source
> > * Checked source code tag
> > * Confirmed Github and Apache source releases are identical
> > * Source code builds (JDK 11)
> > * Approved website pull request
> >
> > Kind regards,
> > Elphas
> >
> >
> > On Thu, Apr 13, 2023 at 4:20 PM Danny Cranmer 
> > wrote:
> >
> >> Hi everyone,
> >> Please review and vote on the release candidate #${RC_NUM} for the
> version
> >> ${NEW_VERSION}, as follows:
> >> [ ] +1, Approve the release
> >> [ ] -1, Do not approve the release (please provide specific comments)
> >>
> >> This version supports both Flink 1.16.x and Flink 1.17.x
> >>
> >> The complete staging area is available for your review, which includes:
> >> * JIRA release notes [1],
> >> * the official Apache source release to be deployed to dist.apache.org
> >> [2],
> >> which are signed with the key with fingerprint
> >> 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
> >> * all artifacts to be deployed to the Maven Central Repository [4],
> >> * source code tag v1.0.1-rc1 [5],
> >> * website pull request listing the new release [6].
> >>
> >> The vote will be open for at least 72 hours. It is adopted by majority
> >> approval, with at least 3 PMC affirmative votes.
> >>
> >> Thanks,
> >> Danny
> >>
> >> [1]
> >>
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353094
> >> [2]
> >>
> >>
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-mongodb-1.0.1-rc1
> >> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> >> [4]
> >> https://repository.apache.org/content/repositories/orgapacheflink-1616/
> >> [5]
> >>
> https://github.com/apache/flink-connector-mongodb/releases/tag/v1.0.1-rc1
> >> [6] https://github.com/apache/flink-web/pull/640
> >>
>
>

Re: [VOTE] Release flink-connector-aws, 4.1.0 for Flink 1.17

[Samrat Deb]

+1 (non binding)


On Tue, 4 Apr 2023 at 1:42 AM, Elphas Tori  wrote:

> +1 (non-binding)
>
> + verified hashes and signatures
> + checked local build of website pull request and approved
>
> On 2023/04/03 16:19:00 Danny Cranmer wrote:
> > Hi everyone,
> > Please review and vote on the binaries for flink-connector-aws version
> > 4.1.0-1.17, as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The v4.1.0 source release has already been approved [1], this vote is to
> > distribute the binaries for Flink 1.17 support.
> >
> > The complete staging area is available for your review, which includes:
> > * all artifacts to be deployed to the Maven Central Repository [2], which
> > are signed with the key with fingerprint
> > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
> > * website pull request listing the new release [4].
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Danny
> >
> > [1] https://lists.apache.org/thread/7q3ysg9jz5cjwdgdmgckbnqhxh44ncmv
> > [2]
> https://repository.apache.org/content/repositories/orgapacheflink-1602/
> > [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > [4] https://github.com/apache/flink-web/pull/628
> >
>

Re: [VOTE] Release flink-connector-mongodb, release candidate #1

[Samrat Deb]

+1 (non-binding)


- checked Source archive builds using maven

- verified Hashes and Signatures

- verified Tag is present in Github

- checked dist.apache.org,  required files  present

- verified source archive


Bests,

Samrat

On Mon, Mar 20, 2023 at 2:08 PM Leonard Xu  wrote:

>
> > + 1 (non-binding)
> >
> > Best,
> > Jiaba
>
> Hi, Jiabao
>
> Thanks for help to verify the release candidate. But  we need to do the
> verification firstly and then post the verified items when  in a release
> candidate vote thread.
>
> The community does not require verification of all items in the document
> [1], each contributor can determine the items they need to verify based on
> the modules they are familiar with and the features they care about.
>
> Best,
> Leonard
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/Verifying+a+Flink+Release
>
>
>
>
>
> >
> >
> >> 2023年3月17日 上午4:32，Martijn Visser  写道：
> >>
> >> +1 (binding)
> >>
> >> - Validated hashes
> >> - Verified signature
> >> - Verified that no binaries exist in the source archive
> >> - Build the source with Maven
> >> - Verified licenses
> >> - Verified web PRs
> >>
> >> On Thu, Mar 16, 2023 at 11:21 AM Danny Cranmer  >
> >> wrote:
> >>
> >>> Hi everyone,
> >>> Please review and vote on the release candidate # for the version , as
> >>> follows:
> >>> [ ] +1, Approve the release
> >>> [ ] -1, Do not approve the release (please provide specific comments)
> >>>
> >>>
> >>> The complete staging area is available for your review, which includes:
> >>> * JIRA release notes [1],
> >>> * the official Apache source release to be deployed to dist.apache.org
> >>> [2],
> >>> which are signed with the key with fingerprint
> >>> 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
> >>> * all artifacts to be deployed to the Maven Central Repository [4],
> >>> * source code tag v1.0.0-rc1 [5],
> >>> * website pull request listing the new release [6].
> >>>
> >>> Given this is a new connector I will follow-up with a blog, but it
> will not
> >>> block the release.
> >>>
> >>> The vote will be open for at least 72 hours (ending 2023-03-21 11:00
> UTC).
> >>> It is adopted by majority approval, with at least 3 PMC affirmative
> votes.
> >>>
> >>> Thanks,
> >>> Danny
> >>>
> >>> [1]
> >>>
> >>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352386
> >>> [2]
> >>>
> >>>
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-mongodb-1.0.0-rc1/
> >>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> >>> [4]
> https://repository.apache.org/content/repositories/orgapacheflink-1598
> >>> [5]
> >>>
> https://github.com/apache/flink-connector-mongodb/releases/tag/v1.0.0-rc1
> >>> [6] https://github.com/apache/flink-web/pull/622
> >>>
> >
>
>

Re: [VOTE] Release flink-connector-aws 4.1.0, release candidate #1

[Samrat Deb]

+1 (non-binding)

- checked Source archive builds using maven
- verified Hashes and Signatures
- verified Tag is present in Github
- checked dist.apache.org,  required files  present
- verified source archive

Bests,
Samrat


On Tue, Mar 28, 2023 at 5:12 PM Teoh, Hong 
wrote:

> +1 (non-binding)
>
> * Hashes and Signatures look good
> * All required files on dist.apache.org
> * Tag is present in Github
> * Verified source archive does not contain any binary files
> * Source archive builds using maven
>
>
> Cheers,
> Hong
>
> > On 22 Mar 2023, at 07:21, Robert Metzger  wrote:
> >
> > CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
> >
> >
> >
> > +1 (binding)
> >
> > Note, in the NOTICE files, the copyright year is still only until 2022. I
> > don't think it's a blocker, but should be updated.
> >
> > + Checked the website update
> > + checked the source archive for copied code (no licenses are listed in
> the
> > root NOTICE file)
> > + src archive contents build locally
> >
> >
> > On Thu, Mar 16, 2023 at 1:11 PM Danny Cranmer 
> > wrote:
> >
> >> Hi everyone,
> >> Please review and vote on the flink-connector-aws release candidate #1
> for
> >> the version 4.1.0, as follows:
> >> [ ] +1, Approve the release
> >> [ ] -1, Do not approve the release (please provide specific comments)
> >>
> >>
> >> The complete staging area is available for your review, which includes:
> >> * JIRA release notes [1],
> >> * the official Apache source release to be deployed to dist.apache.org
> >> [2],
> >> which are signed with the key with fingerprint
> >> 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
> >> * all artifacts to be deployed to the Maven Central Repository [4],
> >> * source code tag v4.1.0-rc1 [5],
> >> * website pull request listing the new release [6].
> >>
> >> The vote will be open for at least 72 hours (ending 2023-03-21 13:00
> UTC).
> >> It is adopted by majority approval, with at least 3 PMC affirmative
> votes.
> >>
> >> Thanks,
> >> Danny
> >>
> >> [1]
> >>
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352646
> >> [2]
> >>
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-aws-4.1.0-rc1/
> >> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> >> [4]
> >> https://repository.apache.org/content/repositories/orgapacheflink-1599/
> >> [5]
> https://github.com/apache/flink-connector-aws/releases/tag/v4.1.0-rc1
> >> [6] https://github.com/apache/flink-web/pull/623
> >>
>
>

Re: [VOTE] FLIP-284: Making AsyncSinkWriter Flush triggers adjustable

[Samrat Deb]

Thank you Ahmed for driving the change.
This new feature will help flink developers with more control on their job
in SinkWriter.

+1 (non binding)

Bests,
Samrat




On Mon, Mar 27, 2023 at 12:34 PM Danny Cranmer 
wrote:

> Thanks for driving this Ahmed. This change will benefit future and current
> sink implementations by giving the developer more flexibility. A good
> addition following configurable rate limiting strategies [1].
>
> +1 (binding)
>
> Thanks,
> Danny
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-242%3A+Introduce+configurable+RateLimitingStrategy+for+Async+Sink
>
> On Sun, Mar 26, 2023 at 2:27 PM Ahmed Hamdy  wrote:
>
> > Dear Flink developers,
> >
> > I want to move discussion on this FLIP[1] to voting. Discussion thread[2]
> > has been inactive for more than 2 months. Please feel free to add your
> > feedback below.
> >
> > The vote will be open for at least 72
> > hours unless there is an objection or not enough votes.
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-284+%3A+Making+AsyncSinkWriter+Flush+triggers+adjustable
> > [2] https://lists.apache.org/thread/hmh2xoy34jm4r81txm3x1wv27d6vnpkw
> >
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Samrat Deb]

congratulations

Bests,
Samrat
On Mon, 27 Mar 2023 at 8:24 PM, Alexander Fedulov <
alexander.fedu...@gmail.com> wrote:

> Great to see this, congratulations!
>
> Best,
> Alex
>
> On Mon, 27 Mar 2023 at 11:24, Yu Li  wrote:
>
> > Dear Flinkers,
> >
> >
> >
> > As you may have noticed, we are pleased to announce that Flink Table
> Store has joined the Apache Incubator as a separate project called Apache
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a
> streaming data lake platform for high-speed data ingestion, change data
> tracking and efficient real-time analytics, with the vision of supporting a
> larger ecosystem and establishing a vibrant and neutral open source
> community.
> >
> >
> >
> > We would like to thank everyone for their great support and efforts for
> the Flink Table Store project, and warmly welcome everyone to join the
> development and activities of the new project. Apache Flink will continue
> to be one of the first-class citizens supported by Paimon, and we believe
> that the Flink and Paimon communities will maintain close cooperation.
> >
> >
> > 亲爱的Flinkers，
> >
> >
> > 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> > 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> > Apache
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
> >
> >
> > 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache
> Flink
> > 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
> >
> >
> > Best Regards,
> >
> > Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
> >
> >
> > 致礼，
> >
> > 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
> >
> >
> > [1] https://paimon.apache.org/
> >
> > [2] https://github.com/apache/incubator-paimon
> >
> > [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
> >
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Samrat Deb]

congratulations

Bests,
Samrat

On Mon, Mar 27, 2023 at 7:19 PM Yanfei Lei  wrote:

> Congratulations!
>
> Best Regards,
> Yanfei
>
> ramkrishna vasudevan  于2023年3月27日周一 21:46写道：
> >
> > Congratulations !!!
> >
> > On Mon, Mar 27, 2023 at 2:54 PM Yu Li  wrote:
> >>
> >> Dear Flinkers,
> >>
> >>
> >> As you may have noticed, we are pleased to announce that Flink Table
> Store has joined the Apache Incubator as a separate project called Apache
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a
> streaming data lake platform for high-speed data ingestion, change data
> tracking and efficient real-time analytics, with the vision of supporting a
> larger ecosystem and establishing a vibrant and neutral open source
> community.
> >>
> >>
> >> We would like to thank everyone for their great support and efforts for
> the Flink Table Store project, and warmly welcome everyone to join the
> development and activities of the new project. Apache Flink will continue
> to be one of the first-class citizens supported by Paimon, and we believe
> that the Flink and Paimon communities will maintain close cooperation.
> >>
> >>
> >> 亲爱的Flinkers，
> >>
> >>
> >> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2]
> [3]。新项目的名字是 Apache
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
> >>
> >>
> >> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache
> Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
> >>
> >>
> >> Best Regards,
> >>
> >> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
> >>
> >>
> >> 致礼，
> >>
> >> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
> >>
> >>
> >> [1] https://paimon.apache.org/
> >>
> >> [2] https://github.com/apache/incubator-paimon
> >>
> >> [3]
> https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Samrat Deb]

congratulations 拾

On Mon, 27 Mar 2023 at 6:02 PM, Leonard Xu  wrote:

> Congratulations!
>
>
> Best,
> Leonard
>
> > On Mar 27, 2023, at 5:23 PM, Yu Li  wrote:
> >
> > Dear Flinkers,
> >
> > As you may have noticed, we are pleased to announce that Flink Table
> Store has joined the Apache Incubator as a separate project called Apache
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a
> streaming data lake platform for high-speed data ingestion, change data
> tracking and efficient real-time analytics, with the vision of supporting a
> larger ecosystem and establishing a vibrant and neutral open source
> community.
> >
> > We would like to thank everyone for their great support and efforts for
> the Flink Table Store project, and warmly welcome everyone to join the
> development and activities of the new project. Apache Flink will continue
> to be one of the first-class citizens supported by Paimon, and we believe
> that the Flink and Paimon communities will maintain close cooperation.
> >
> > 亲爱的Flinkers，
> >
> > 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2]
> [3]。新项目的名字是 Apache
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
> >
> > 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache
> Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
> >
> > Best Regards,
> > Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
> >
> > 致礼，
> > 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
> >
> > [1] https://paimon.apache.org/ 
> > [2] https://github.com/apache/incubator-paimon <
> https://github.com/apache/incubator-paimon>
> > [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
> 
>

Re: [VOTE] FLIP-293: Introduce Flink Jdbc Driver For Sql Gateway

[Samrat Deb]

+1 ( non binding)

Bests,
Samrat

On Wed, 15 Mar 2023 at 8:54 PM, Dong Lin  wrote:

> +1 (binding)
>
> On Mon, Mar 13, 2023 at 1:47 PM Shammon FY  wrote:
>
> > Hi Devs,
> >
> > I'd like to start the vote on FLIP-293: Introduce Flink Jdbc Driver For
> Sql
> > Gateway [1].
> >
> > The FLIP was discussed in thread [2], and it aims to introduce Flink Jdbc
> > Driver module in Flink.
> >
> > The vote will last for at least 72 hours (03/16, 15:00 UTC+8) unless
> there
> > is an objection or insufficient vote. Thank you all.
> >
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-293%3A+Introduce+Flink+Jdbc+Driver+For+Sql+Gateway
> > [2] https://lists.apache.org/thread/d1owrg8zh77v0xygcpb93fxt0jpjdkb3
> >
> >
> > Best,
> > Shammon.FY
> >
>