Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Yuxin Tan]

Congratulations！

Best,
Yuxin


Guanghui Zhang  于2023年3月28日周二 11:06写道：

> Congratulations！
>
> Best,
> Zhang Guanghui
>
> Hang Ruan  于2023年3月28日周二 10:29写道：
>
> > Congratulations!
> >
> > Best,
> > Hang
> >
> > yu zelin  于2023年3月28日周二 10:27写道：
> >
> >> Congratulations!
> >>
> >> Best,
> >> Yu Zelin
> >>
> >> 2023年3月27日 17:23，Yu Li  写道：
> >>
> >> Dear Flinkers,
> >>
> >>
> >>
> >> As you may have noticed, we are pleased to announce that Flink Table
> Store has joined the Apache Incubator as a separate project called Apache
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a
> streaming data lake platform for high-speed data ingestion, change data
> tracking and efficient real-time analytics, with the vision of supporting a
> larger ecosystem and establishing a vibrant and neutral open source
> community.
> >>
> >>
> >>
> >> We would like to thank everyone for their great support and efforts for
> the Flink Table Store project, and warmly welcome everyone to join the
> development and activities of the new project. Apache Flink will continue
> to be one of the first-class citizens supported by Paimon, and we believe
> that the Flink and Paimon communities will maintain close cooperation.
> >>
> >>
> >> 亲爱的Flinkers，
> >>
> >>
> >> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> >> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> >> Apache
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
> >>
> >>
> >> 在这里我们要感谢大家对 Flink Table Store
> >> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon
> 支持的主力计算引擎之一，我们也相信
> >> Flink 和 Paimon 社区将继续保持密切合作。
> >>
> >>
> >> Best Regards,
> >> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
> >>
> >> 致礼，
> >> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
> >>
> >> [1] https://paimon.apache.org/
> >> [2] https://github.com/apache/incubator-paimon
> >> [3]
> https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
> >>
> >>
> >>
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Yuxin Tan]

Congratulations！

Best,
Yuxin


Guanghui Zhang  于2023年3月28日周二 11:06写道：

> Congratulations！
>
> Best,
> Zhang Guanghui
>
> Hang Ruan  于2023年3月28日周二 10:29写道：
>
> > Congratulations!
> >
> > Best,
> > Hang
> >
> > yu zelin  于2023年3月28日周二 10:27写道：
> >
> >> Congratulations!
> >>
> >> Best,
> >> Yu Zelin
> >>
> >> 2023年3月27日 17:23，Yu Li  写道：
> >>
> >> Dear Flinkers,
> >>
> >>
> >>
> >> As you may have noticed, we are pleased to announce that Flink Table
> Store has joined the Apache Incubator as a separate project called Apache
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a
> streaming data lake platform for high-speed data ingestion, change data
> tracking and efficient real-time analytics, with the vision of supporting a
> larger ecosystem and establishing a vibrant and neutral open source
> community.
> >>
> >>
> >>
> >> We would like to thank everyone for their great support and efforts for
> the Flink Table Store project, and warmly welcome everyone to join the
> development and activities of the new project. Apache Flink will continue
> to be one of the first-class citizens supported by Paimon, and we believe
> that the Flink and Paimon communities will maintain close cooperation.
> >>
> >>
> >> 亲爱的Flinkers，
> >>
> >>
> >> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> >> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> >> Apache
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
> >>
> >>
> >> 在这里我们要感谢大家对 Flink Table Store
> >> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon
> 支持的主力计算引擎之一，我们也相信
> >> Flink 和 Paimon 社区将继续保持密切合作。
> >>
> >>
> >> Best Regards,
> >> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
> >>
> >> 致礼，
> >> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
> >>
> >> [1] https://paimon.apache.org/
> >> [2] https://github.com/apache/incubator-paimon
> >> [3]
> https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
> >>
> >>
> >>
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Guanghui Zhang]

Congratulations！

Best,
Zhang Guanghui

Hang Ruan  于2023年3月28日周二 10:29写道：

> Congratulations!
>
> Best,
> Hang
>
> yu zelin  于2023年3月28日周二 10:27写道：
>
>> Congratulations!
>>
>> Best,
>> Yu Zelin
>>
>> 2023年3月27日 17:23，Yu Li  写道：
>>
>> Dear Flinkers,
>>
>>
>>
>> As you may have noticed, we are pleased to announce that Flink Table Store 
>> has joined the Apache Incubator as a separate project called Apache 
>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
>> streaming data lake platform for high-speed data ingestion, change data 
>> tracking and efficient real-time analytics, with the vision of supporting a 
>> larger ecosystem and establishing a vibrant and neutral open source 
>> community.
>>
>>
>>
>> We would like to thank everyone for their great support and efforts for the 
>> Flink Table Store project, and warmly welcome everyone to join the 
>> development and activities of the new project. Apache Flink will continue to 
>> be one of the first-class citizens supported by Paimon, and we believe that 
>> the Flink and Paimon communities will maintain close cooperation.
>>
>>
>> 亲爱的Flinkers，
>>
>>
>> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
>> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
>> Apache 
>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>>
>>
>> 在这里我们要感谢大家对 Flink Table Store
>> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信
>> Flink 和 Paimon 社区将继续保持密切合作。
>>
>>
>> Best Regards,
>> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>>
>> 致礼，
>> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>>
>> [1] https://paimon.apache.org/
>> [2] https://github.com/apache/incubator-paimon
>> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>>
>>
>>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Guanghui Zhang]

Congratulations！

Best,
Zhang Guanghui

Hang Ruan  于2023年3月28日周二 10:29写道：

> Congratulations!
>
> Best,
> Hang
>
> yu zelin  于2023年3月28日周二 10:27写道：
>
>> Congratulations!
>>
>> Best,
>> Yu Zelin
>>
>> 2023年3月27日 17:23，Yu Li  写道：
>>
>> Dear Flinkers,
>>
>>
>>
>> As you may have noticed, we are pleased to announce that Flink Table Store 
>> has joined the Apache Incubator as a separate project called Apache 
>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
>> streaming data lake platform for high-speed data ingestion, change data 
>> tracking and efficient real-time analytics, with the vision of supporting a 
>> larger ecosystem and establishing a vibrant and neutral open source 
>> community.
>>
>>
>>
>> We would like to thank everyone for their great support and efforts for the 
>> Flink Table Store project, and warmly welcome everyone to join the 
>> development and activities of the new project. Apache Flink will continue to 
>> be one of the first-class citizens supported by Paimon, and we believe that 
>> the Flink and Paimon communities will maintain close cooperation.
>>
>>
>> 亲爱的Flinkers，
>>
>>
>> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
>> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
>> Apache 
>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>>
>>
>> 在这里我们要感谢大家对 Flink Table Store
>> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信
>> Flink 和 Paimon 社区将继续保持密切合作。
>>
>>
>> Best Regards,
>> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>>
>> 致礼，
>> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>>
>> [1] https://paimon.apache.org/
>> [2] https://github.com/apache/incubator-paimon
>> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>>
>>
>>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Yun Tang]

Congratulations!
Unlike other data-lakes, Paimon might be the first one to act as a stream-first 
(not batch-first) data-lake.

Best
Yun Tang

From: Xianxun Ye 
Sent: Tuesday, March 28, 2023 10:52
To: d...@flink.apache.org 
Cc: Yu Li ; user ; user-zh 
; d...@flink.apache.org 
Subject: Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache 
Paimon(incubating)

Congratulations!

Best regards,

Xianxun

On 03/27/2023 22:51，Samrat 
Deb wrote：
congratulations

Bests,
Samrat

On Mon, Mar 27, 2023 at 7:19 PM Yanfei Lei  wrote:

Congratulations!

Best Regards,
Yanfei

ramkrishna vasudevan  于2023年3月27日周一 21:46写道：

Congratulations !!!

On Mon, Mar 27, 2023 at 2:54 PM Yu Li  wrote:

Dear Flinkers,


As you may have noticed, we are pleased to announce that Flink Table
Store has joined the Apache Incubator as a separate project called Apache
Paimon(incubating) [1] [2] [3]. The new project still aims at building a
streaming data lake platform for high-speed data ingestion, change data
tracking and efficient real-time analytics, with the vision of supporting a
larger ecosystem and establishing a vibrant and neutral open source
community.


We would like to thank everyone for their great support and efforts for
the Flink Table Store project, and warmly welcome everyone to join the
development and activities of the new project. Apache Flink will continue
to be one of the first-class citizens supported by Paimon, and we believe
that the Flink and Paimon communities will maintain close cooperation.


亲爱的Flinkers，


正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2]
[3]。新项目的名字是 Apache
Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。


在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache
Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。


Best Regards,

Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)


致礼，

李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）


[1] https://paimon.apache.org/

[2] https://github.com/apache/incubator-paimon

[3]
https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Yun Tang]

Congratulations!
Unlike other data-lakes, Paimon might be the first one to act as a stream-first 
(not batch-first) data-lake.

Best
Yun Tang

From: Xianxun Ye 
Sent: Tuesday, March 28, 2023 10:52
To: d...@flink.apache.org 
Cc: Yu Li ; user ; user-zh 
; d...@flink.apache.org 
Subject: Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache 
Paimon(incubating)

Congratulations!

Best regards,

Xianxun

On 03/27/2023 22:51，Samrat 
Deb wrote：
congratulations

Bests,
Samrat

On Mon, Mar 27, 2023 at 7:19 PM Yanfei Lei  wrote:

Congratulations!

Best Regards,
Yanfei

ramkrishna vasudevan  于2023年3月27日周一 21:46写道：

Congratulations !!!

On Mon, Mar 27, 2023 at 2:54 PM Yu Li  wrote:

Dear Flinkers,


As you may have noticed, we are pleased to announce that Flink Table
Store has joined the Apache Incubator as a separate project called Apache
Paimon(incubating) [1] [2] [3]. The new project still aims at building a
streaming data lake platform for high-speed data ingestion, change data
tracking and efficient real-time analytics, with the vision of supporting a
larger ecosystem and establishing a vibrant and neutral open source
community.


We would like to thank everyone for their great support and efforts for
the Flink Table Store project, and warmly welcome everyone to join the
development and activities of the new project. Apache Flink will continue
to be one of the first-class citizens supported by Paimon, and we believe
that the Flink and Paimon communities will maintain close cooperation.


亲爱的Flinkers，


正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2]
[3]。新项目的名字是 Apache
Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。


在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache
Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。


Best Regards,

Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)


致礼，

李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）


[1] https://paimon.apache.org/

[2] https://github.com/apache/incubator-paimon

[3]
https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Xianxun Ye]

Congratulations!






  

Best regards,Xianxun


 


On 03/27/2023 22:51，Samrat Deb wrote： 


congratulationsBests,SamratOn Mon, Mar 27, 2023 at 7:19 PM Yanfei Lei  wrote: Congratulations! Best Regards, Yanfei ramkrishna vasudevan  于2023年3月27日周一 21:46写道： Congratulations !!! On Mon, Mar 27, 2023 at 2:54 PM Yu Li  wrote: Dear Flinkers, As you may have noticed, we are pleased to announce that Flink Table Store has joined the Apache Incubator as a separate project called Apache Paimon(incubating) [1] [2] [3]. The new project still aims at building a streaming data lake platform for high-speed data ingestion, change data tracking and efficient real-time analytics, with the vision of supporting a larger ecosystem and establishing a vibrant and neutral open source community. We would like to thank everyone for their great support and efforts for the Flink Table Store project, and warmly welcome everyone to join the development and activities of the new project. Apache Flink will continue to be one of the first-class citizens supported by Paimon, and we believe that the Flink and Paimon communities will maintain close cooperation. 亲爱的Flinkers， 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2] [3]。新项目的名字是 Apache Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。 Best Regards, Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC) 致礼， 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC） [1] https://paimon.apache.org/ [2] https://github.com/apache/incubator-paimon [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Biao Liu]

Congrats!

Thanks,
Biao /'bɪ.aʊ/



On Tue, 28 Mar 2023 at 10:29, Hang Ruan  wrote:

> Congratulations!
>
> Best,
> Hang
>
> yu zelin  于2023年3月28日周二 10:27写道：
>
>> Congratulations!
>>
>> Best,
>> Yu Zelin
>>
>> 2023年3月27日 17:23，Yu Li  写道：
>>
>> Dear Flinkers,
>>
>>
>>
>> As you may have noticed, we are pleased to announce that Flink Table Store 
>> has joined the Apache Incubator as a separate project called Apache 
>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
>> streaming data lake platform for high-speed data ingestion, change data 
>> tracking and efficient real-time analytics, with the vision of supporting a 
>> larger ecosystem and establishing a vibrant and neutral open source 
>> community.
>>
>>
>>
>> We would like to thank everyone for their great support and efforts for the 
>> Flink Table Store project, and warmly welcome everyone to join the 
>> development and activities of the new project. Apache Flink will continue to 
>> be one of the first-class citizens supported by Paimon, and we believe that 
>> the Flink and Paimon communities will maintain close cooperation.
>>
>>
>> 亲爱的Flinkers，
>>
>>
>> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
>> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
>> Apache 
>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>>
>>
>> 在这里我们要感谢大家对 Flink Table Store
>> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信
>> Flink 和 Paimon 社区将继续保持密切合作。
>>
>>
>> Best Regards,
>> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>>
>> 致礼，
>> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>>
>> [1] https://paimon.apache.org/
>> [2] https://github.com/apache/incubator-paimon
>> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>>
>>
>>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Biao Liu]

Congrats!

Thanks,
Biao /'bɪ.aʊ/



On Tue, 28 Mar 2023 at 10:29, Hang Ruan  wrote:

> Congratulations!
>
> Best,
> Hang
>
> yu zelin  于2023年3月28日周二 10:27写道：
>
>> Congratulations!
>>
>> Best,
>> Yu Zelin
>>
>> 2023年3月27日 17:23，Yu Li  写道：
>>
>> Dear Flinkers,
>>
>>
>>
>> As you may have noticed, we are pleased to announce that Flink Table Store 
>> has joined the Apache Incubator as a separate project called Apache 
>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
>> streaming data lake platform for high-speed data ingestion, change data 
>> tracking and efficient real-time analytics, with the vision of supporting a 
>> larger ecosystem and establishing a vibrant and neutral open source 
>> community.
>>
>>
>>
>> We would like to thank everyone for their great support and efforts for the 
>> Flink Table Store project, and warmly welcome everyone to join the 
>> development and activities of the new project. Apache Flink will continue to 
>> be one of the first-class citizens supported by Paimon, and we believe that 
>> the Flink and Paimon communities will maintain close cooperation.
>>
>>
>> 亲爱的Flinkers，
>>
>>
>> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
>> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
>> Apache 
>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>>
>>
>> 在这里我们要感谢大家对 Flink Table Store
>> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信
>> Flink 和 Paimon 社区将继续保持密切合作。
>>
>>
>> Best Regards,
>> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>>
>> 致礼，
>> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>>
>> [1] https://paimon.apache.org/
>> [2] https://github.com/apache/incubator-paimon
>> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>>
>>
>>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Hang Ruan]

Congratulations!

Best,
Hang

yu zelin  于2023年3月28日周二 10:27写道：

> Congratulations!
>
> Best,
> Yu Zelin
>
> 2023年3月27日 17:23，Yu Li  写道：
>
> Dear Flinkers,
>
>
>
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
>
>
>
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
>
>
> 亲爱的Flinkers，
>
>
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>
>
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>
>
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>
> [1] https://paimon.apache.org/
> [2] https://github.com/apache/incubator-paimon
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>
>
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Hang Ruan]

Congratulations!

Best,
Hang

yu zelin  于2023年3月28日周二 10:27写道：

> Congratulations!
>
> Best,
> Yu Zelin
>
> 2023年3月27日 17:23，Yu Li  写道：
>
> Dear Flinkers,
>
>
>
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
>
>
>
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
>
>
> 亲爱的Flinkers，
>
>
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>
>
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>
>
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>
> [1] https://paimon.apache.org/
> [2] https://github.com/apache/incubator-paimon
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>
>
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[yu zelin]

Congratulations!

Best,
Yu Zelin

> 2023年3月27日 17:23，Yu Li  写道：
> 
> Dear Flinkers,
> 
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
> 
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
> 
> 亲爱的Flinkers，
> 
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2] 
> [3]。新项目的名字是 Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
> 
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
> 
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
> 
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
> 
> [1] https://paimon.apache.org/
> [2] https://github.com/apache/incubator-paimon
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[yu zelin]

Congratulations!

Best,
Yu Zelin

> 2023年3月27日 17:23，Yu Li  写道：
> 
> Dear Flinkers,
> 
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
> 
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
> 
> 亲爱的Flinkers，
> 
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2] 
> [3]。新项目的名字是 Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
> 
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
> 
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
> 
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
> 
> [1] https://paimon.apache.org/
> [2] https://github.com/apache/incubator-paimon
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal

flink sql upsert mysql问题

[小昌同学]

你好，我这边使用flink  sql实现四条流的关联，后续实现case 
when的逻辑，并且将数据插入到mysql，但是从结果数据来看，数据存在部分丢失，代码我粘贴再后面，麻烦各位老师指导，下面是sql【create 
function get_json_value as 'com.nesc.flink.udf.GetJsonValue';
set 'table.exec.sink.not-null-enforcer'='drop';
测试环境
CREATE TABLE dm_cust_oact_prog_ri (
 cust_id STRING COMMENT '客户id'
,cust_nme STRING COMMENT '客户姓名'
,cust_mob_tel STRING COMMENT '客户手机号'
,cust_curr_step STRING COMMENT '客户当前步骤'
,cust_curr_step_num INT COMMENT '客户当前步骤数字'
,cust_curr_step_occu_tm STRING COMMENT '客户当前步骤最近发生时间'
,user_id STRING COMMENT '开户时使用的user_id'
,tech_sys_time STRING COMMENT '技术字段，更新时间'
,primary key (user_id,cust_curr_step) not enforced
) WITH (
 'connector' = 'jdbc'
,'url' = 'jdbc:mysql://111/iap'
,'username' = 'db_iap'
,'password' = '加密内容2'
,'table-name' = 'dm_cust_oact_prog_ri'
);
CREATE TABLE dm_cust_oact_prog_ri_print (
 cust_id STRING COMMENT '客户id'
,cust_nme STRING COMMENT '客户姓名'
,cust_mob_tel STRING COMMENT '客户手机号'
,cust_curr_step STRING COMMENT '客户当前步骤'
,cust_curr_step_num INT COMMENT '客户当前步骤数字'
,cust_curr_step_occu_tm STRING COMMENT '客户当前步骤最近发生时间'
,user_id STRING COMMENT '开户时使用的user_id'
,tech_sys_time STRING COMMENT '技术字段，更新时间'
,primary key (user_id,cust_curr_step) not enforced
) WITH (
  'connector' = 'print'
);
CREATE TABLE dm_crh_cust_oact_rec_ri
(
 op_type string
,op_ts string
,`after` string
,current_ts string
,curr_datetime as get_json_value(`after`,'CURR_DATETIME')
,user_id as get_json_value(`after`,'USER_ID')
,request_no as get_json_value(`after`,'REQUEST_NO')
,business_flag as get_json_value(`after`,'BUSINESS_FLAG')
,proc_time as PROCTIME()
) WITH (
  'connector' = 'kafka',
  'topic' = 'BUSINFLOWRECORD',
  'properties.bootstrap.servers' = '111',
  'properties.group.id' = 'iap_dm_cust_oact_prog_ri_test_env',
  'format' = 'json',
  'scan.startup.mode' = 'latest-offset',
  --'csv.field-delimiter' = ','
  --'scan.startup.mode' = 'timestamp',
  --'scan.startup.timestamp-millis' = '167535360', --通过 
unix_timestamp('2023-02-03 00:00:00')*1000 获取开始时间点
  'json.fail-on-missing-field' = 'false', -- 字段丢失任务不失败
  'json.ignore-parse-errors' = 'true' -- 解析失败跳过
);
CREATE TABLE dm_crh_cust_info_ri
(
 op_type string
,op_ts string
,`after` string
,current_ts string
,client_name as get_json_value(`after`,'CLIENT_NAME')
,request_status as get_json_value(`after`,'REQUEST_STATUS')
,mobile_tel as get_json_value(`after`,'MOBILE_TEL')
,user_id as get_json_value(`after`,'USER_ID')
,active_datetime as get_json_value(`after`,'ACTIVE_DATETIME')
,channel_code as get_json_value(`after`,'CHANNEL_CODE')
,broker_code as get_json_value(`after`,'BROKER_CODE')
,user_gender as get_json_value(`after`,'USER_GENDER')
,birthday as get_json_value(`after`,'BIRTHDAY')
,client_id as get_json_value(`after`,'CLIENT_ID')
) WITH (
  'connector' = 'kafka',
  'topic' = 'USERQUERYEXTINFO',
  'properties.bootstrap.servers' = '111',
  'properties.group.id' = 'iap_dm_cust_oact_prog_ri_test_env',
  'format' = 'json',
  'scan.startup.mode' = 'latest-offset',
  --'csv.field-delimiter' = ','
  --'scan.startup.mode' = 'timestamp',
  --'scan.startup.timestamp-millis' = '167535360', --通过 
unix_timestamp('2023-02-03 00:00:00')*1000 获取开始时间点
  'json.fail-on-missing-field' = 'false', -- 字段丢失任务不失败
  'json.ignore-parse-errors' = 'true' -- 解析失败跳过
);
CREATE TABLE dm_crh_audit_rec_ri
(
 op_type string
,op_ts string
,`after` string
,current_ts string
,business_flag as get_json_value(`after`,'BUSINESS_FLAG')
,curr_datetime as get_json_value(`after`,'CURR_DATETIME')
,request_no as get_json_value(`after`,'REQUEST_NO')
) WITH (
  'connector' = 'kafka',
  'topic' = 'CRH_USER.BUSINFLOWAUDITRECORD',
  'properties.bootstrap.servers' = '111',
  'properties.group.id' = 'iap_dm_cust_oact_prog_ri_test_env',
  'format' = 'json',
  'scan.startup.mode' = 'latest-offset',
  --'csv.field-delimiter' = ','
  --'scan.startup.mode' = 'timestamp',
  --'scan.startup.timestamp-millis' = '167535360', --通过 
unix_timestamp('2023-02-03 00:00:00')*1000 获取开始时间点
  'json.fail-on-missing-field' = 'false', -- 字段丢失任务不失败
  'json.ignore-parse-errors' = 'true' -- 解析失败跳过
);
CREATE TABLE dm_crh_user_vidro_ri
(
 op_type string
,op_ts string
,`after` string
,current_ts string
,business_flag as get_json_value(`after`,'BUSINESS_FLAG')
,create_datetime as get_json_value(`after`,'CREATE_DATETIME')
,user_id as get_json_value(`after`,'USER_ID')
,join_position_str as get_json_value(`after`,'JOIN_POSITION_STR')
) WITH (
  'connector' = 'kafka',
  'topic' = 'CRH_USER.USERVIDEOFLOW',
  'properties.bootstrap.servers' = '111',
  'properties.group.id' = 'iap_dm_cust_oact_prog_ri_test_env',
  'format' = 'json',
  'scan.startup.mode' = 'latest-offset',
  --'csv.field-delimiter' = ','
  --'scan.startup.mode' = 'timestamp',
  --'scan.startup.timestamp-millis' = 

Re: Custom source

[Shammon FY]

Hi Lasse

Does your job table/sql or datastream? Here's the doc [1] for customized
source in table and there is an example of socket source.


[1]
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sourcessinks/

Best,
Shammon FY

On Mon, Mar 27, 2023 at 8:02 PM Lasse Nedergaard <
lassenedergaardfl...@gmail.com> wrote:

> Hi.
>
> I have to use data from a Rest API, very slow changing. Instead of doing
> an async i/o request I would like to create a source function that read
> from Rest Api so I can connect and enrich another stream without a lookup
> for each incoming row.
> Flink doc provide an explanation of how to do it but I find it pretty hard
> get all the concepts to work together and I haven’t been able to find a
> “simple” app example.
> Is there anyone out there knowing a good example or give a simple
> explanation of best practice for implementing your own source
>
> Med venlig hilsen / Best regards
> Lasse Nedergaard
>
>

Re: Issue with the flink version 1.10.1

[Shammon FY]

Hi Kiran

According to your description, it sounds like a configuration issue
prevented the job from being submitted successfully. You can check whether
there are any error messages from the submission log of yarn or your
submission system. You can also compare the configuration differences
between the new and old versions.

Best,
Shammon FY


On Mon, Mar 27, 2023 at 10:38 PM Kiran Kumar Kathe <
kirankumarkathe...@gmail.com> wrote:

> Even though when I run with the latest versions I am not able to see the
> jobs running in the flink in resource manager Web UI , but when I use flink
> less than 1.9.3 it is working fine , and if i replace the  1.9.3 flink_dist
> jar file which is in the lib folder of flink application and  in the higher
> version like 1.10.1 and above these are also working good and I am able to
> see the jobs in the Web UI. Can I use it like that or is there another way
> ?  And I am able to see the logs in versions below 1.9.3 like submitting
> application , Submitted application , deploying cluster , yarn application
> has been successfully deployed  but in higher versions it is not showing
> logs like that , i have attached screenshots have a look on that to get
> clear idea , please do suggest if there are any other ways  or if i can use
> flink_dist jar which is in lib for in higher versions.
>
> On Mon, 27 Mar 2023 at 18:06, Leonard Xu  wrote:
>
>> Hi, Kiran
>>
>> To be honest, both 1.10 and 1.9 are pretty old version, it’s hard to fix
>> and release a bugfix version for 1.10.1 even the community help
>> troubleshoot your issue.
>> So, could you try latest versions like Flink 1.16.1 or 1.17.0?
>>
>> Best,
>> Leonard
>>
>> > On Mar 27, 2023, at 8:28 PM, Kiran Kumar Kathe <
>> kirankumarkathe...@gmail.com> wrote:
>> >
>> > When I submit a job using flink version 1.10.1 ,  it is not upadating
>> the jobs that are running and completed successfully in the Web UI of YARN
>> resource manager . But When I use flink version 1.9.3 it is working fine
>> and I am able to see the jobs that are running and completed in
>> > YARN resource manager Web UI . And to find why this is happening I just
>> tried with replacing the application folders and in lib folder when I use
>> the flink_dist jar of version 1.9.3 in place of flink_dist of version
>> 1.10.1 it is running fine and I am able to see the jobs running and
>> completed. Is it the right way , if not will I face any compatible issues
>> in future with this change of flink_dist jar in lib folder.
>>
>>

Re: Flink CEP Resource Utilisation Optimisation

[Abhishek Singla]

Hi Geng,

I tried the same experiment with version 1.16.0 and was able to achieve the
same throughput with 3 nodes (24 parallelism) with CPU usage 60-70%

Regarding the checkpoint size issue: Since there were no input events, the
watermark was not advancing hence the events in CEP state were not getting
timed out.

Thanks again for helping out.

Regards,
Abhishek Singla

On Sun, Mar 26, 2023 at 10:33 PM Abhishek Singla <
abhisheksingla...@gmail.com> wrote:

> Thanks, yes there were a lot of keys in the test input. In fact, every
> event has a unique key which is not repeated in subsequent events.
>
> On Sun, Mar 26, 2023 at 10:26 PM Geng Biao  wrote:
>
>> I see your point. Are there lots of different keys in your test input? If
>> that is the case, CEP operator in 1.15.0 will not clean some intermediate
>> states(partial matches will be cleaned due to timeout but some
>> computation states are leaked). It is fixed in flink1.16(FLINK-31017) by
>> Juntao Hu.
>> Best,
>> Biao
>>
>> 获取 Outlook for iOS 
>> --
>> *发件人:* Abhishek Singla 
>> *发送时间:* Monday, March 27, 2023 12:38:59 AM
>> *收件人:* Geng Biao 
>> *抄送:* user@flink.apache.org 
>> *主题:* Re: Flink CEP Resource Utilisation Optimisation
>>
>> Thanks, Geng for the quick and actionable response.
>>
>> I will definitely try this with Flink version >= 1.16.0 and get back
>> with the observations.
>>
>> Regarding the checkpoint size issue, my concern is if there is no more
>> state, shouldn't the checkpoint size be way less than 2 GB? I mean I was
>> expecting it to be only a few MBs. Is there something I am missing here?
>>
>> Regards,
>> Abhishek Singla
>>
>> On Sun, Mar 26, 2023 at 9:56 PM Geng Biao  wrote:
>>
>> Hi Abhishek,
>>
>>
>>
>> Thanks for sharing the experiment! As for the performance question, I
>> believe you could give a try on Flink CEP with version >= 1.16.0, which
>> includes the optimization introduced in FLINK-23890
>> . This optimization
>> will reduce lots of timer registration which can increase the throughput
>> significantly. In our own experiment, given same papalism settings, the
>> same job in 1.16.0 will require much less CPU usage than that in 1.15.x.
>> (~100% -> ~30%). In fact, due to the implementation, the optimization
>> should make CEP 10x better.  If you must use Flink1.15.0 for some reason,
>> you may cherry-pick the relevant change and recompile the CEP library by
>> yourself. The change does not depend on some framework changes so it may
>> not cost much efforts.
>>
>> As for the checkpoint size issue, CEP Operator will store immediate
>> matching result in the state. So if there are no new events, then there are
>> no new partial matched and CEP Operator will not use more state.
>>
>>
>>
>> Best,
>> Biao Geng
>>
>>
>>
>> *From: *Abhishek Singla 
>> *Date: *Sunday, March 26, 2023 at 11:58 PM
>> *To: *user@flink.apache.org 
>> *Subject: *Flink CEP Resource Utilisation Optimisation
>>
>> Hi Team,
>>
>> *Flink Version:* 1.15.0
>> *Java Version:* 1.8
>> *Standalone Cluster*
>> *Task Manager:* AWS EC2 of Instance Type c5n.4xlarge (vCPU 16, Memory 42
>> Gb, 8 slots per TM)
>> *CEP Scenario:* Kafka Event A followed by Kafka Event B within 10 mins
>> *Throughput:* 20k events per second for Event A, 0 for Kafka Event B
>> *State Backend:* FsStateBackend
>> *Unaligned Checkpoints:* Enabled
>> *asynchronousSnapshots:* true
>>
>>
>>
>> While testing this (Kafka Event A followed by Kafka Event B within 10
>> mins) scenario on load environment, it took 20 nodes of TM to achieve this
>> throughput otherwise either CPU utilization would reach its peak or
>> backpressure would be observed because output buffers are full. The
>> checkpoint size is only 6.75 GB, the state stored within the CEP
>> operator would be much lesser as we do unaligned checkpointing.
>>
>>
>> I am looking for some input on if it takes this many resources to
>> archive this throughput, and if not what probably could be the issue here.
>>
>>
>>
>> There was one more issue that I found If the throughput of Event A goes
>> to zero, then also the checkpoint size stays around 2 GB even after hours.
>> Is this expected?
>>
>> Regards,
>> Abhishek Singla
>>
>>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Alexander Fedulov]

Great to see this, congratulations!

Best,
Alex

On Mon, 27 Mar 2023 at 11:24, Yu Li  wrote:

> Dear Flinkers,
>
>
>
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
>
>
>
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
>
>
> 亲爱的Flinkers，
>
>
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>
>
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>
>
> Best Regards,
>
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>
>
> 致礼，
>
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>
>
> [1] https://paimon.apache.org/
>
> [2] https://github.com/apache/incubator-paimon
>
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Yanfei Lei]

Congratulations!

Best Regards,
Yanfei

ramkrishna vasudevan  于2023年3月27日周一 21:46写道：
>
> Congratulations !!!
>
> On Mon, Mar 27, 2023 at 2:54 PM Yu Li  wrote:
>>
>> Dear Flinkers,
>>
>>
>> As you may have noticed, we are pleased to announce that Flink Table Store 
>> has joined the Apache Incubator as a separate project called Apache 
>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
>> streaming data lake platform for high-speed data ingestion, change data 
>> tracking and efficient real-time analytics, with the vision of supporting a 
>> larger ecosystem and establishing a vibrant and neutral open source 
>> community.
>>
>>
>> We would like to thank everyone for their great support and efforts for the 
>> Flink Table Store project, and warmly welcome everyone to join the 
>> development and activities of the new project. Apache Flink will continue to 
>> be one of the first-class citizens supported by Paimon, and we believe that 
>> the Flink and Paimon communities will maintain close cooperation.
>>
>>
>> 亲爱的Flinkers，
>>
>>
>> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2] 
>> [3]。新项目的名字是 Apache 
>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>>
>>
>> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 
>> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>>
>>
>> Best Regards,
>>
>> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>>
>>
>> 致礼，
>>
>> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>>
>>
>> [1] https://paimon.apache.org/
>>
>> [2] https://github.com/apache/incubator-paimon
>>
>> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Yanfei Lei]

Congratulations!

Best Regards,
Yanfei

ramkrishna vasudevan  于2023年3月27日周一 21:46写道：
>
> Congratulations !!!
>
> On Mon, Mar 27, 2023 at 2:54 PM Yu Li  wrote:
>>
>> Dear Flinkers,
>>
>>
>> As you may have noticed, we are pleased to announce that Flink Table Store 
>> has joined the Apache Incubator as a separate project called Apache 
>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
>> streaming data lake platform for high-speed data ingestion, change data 
>> tracking and efficient real-time analytics, with the vision of supporting a 
>> larger ecosystem and establishing a vibrant and neutral open source 
>> community.
>>
>>
>> We would like to thank everyone for their great support and efforts for the 
>> Flink Table Store project, and warmly welcome everyone to join the 
>> development and activities of the new project. Apache Flink will continue to 
>> be one of the first-class citizens supported by Paimon, and we believe that 
>> the Flink and Paimon communities will maintain close cooperation.
>>
>>
>> 亲爱的Flinkers，
>>
>>
>> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2] 
>> [3]。新项目的名字是 Apache 
>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>>
>>
>> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 
>> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>>
>>
>> Best Regards,
>>
>> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>>
>>
>> 致礼，
>>
>> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>>
>>
>> [1] https://paimon.apache.org/
>>
>> [2] https://github.com/apache/incubator-paimon
>>
>> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[ramkrishna vasudevan]

Congratulations !!!

On Mon, Mar 27, 2023 at 2:54 PM Yu Li  wrote:

> Dear Flinkers,
>
>
>
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
>
>
>
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
>
>
> 亲爱的Flinkers，
>
>
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>
>
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>
>
> Best Regards,
>
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>
>
> 致礼，
>
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>
>
> [1] https://paimon.apache.org/
>
> [2] https://github.com/apache/incubator-paimon
>
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[feng xiangyu]

Congrats Yu! Looking forward to contributing to Paimon!

Best Regards,
Xiangyu

yuxia  于2023年3月27日周一 21:01写道：

> congratulations!
>
> Best regards,
> Yuxia
>
>
> 发件人: "Andrew Otto" 
> 收件人: "Matthias Pohl" 
> 抄送: "Jing Ge" , "Leonard Xu" , "Yu
> Li" , "dev" , "User" <
> user@flink.apache.org>, "user-zh" 
> 发送时间: 星期一, 2023年 3 月 27日 下午 8:57:50
> 主题: Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache
> Paimon(incubating)
>
> Exciting!
>
> If this ends up working well, Wikimedia Foundation would love to try it
> out!
>
> On Mon, Mar 27, 2023 at 8:39 AM Matthias Pohl via user < [ mailto:
> user@flink.apache.org | user@flink.apache.org ] > wrote:
>
>
>
> Congratulations and good luck with pushing the project forward.
>
> On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user < [ mailto:
> user@flink.apache.org | user@flink.apache.org ] > wrote:
>
> BQ_BEGIN
>
> Congrats!
> Best regards,
> Jing
>
> On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu < [ mailto:xbjt...@gmail.com |
> xbjt...@gmail.com ] > wrote:
>
> BQ_BEGIN
>
> Congratulations!
>
> Best,
> Leonard
>
>
> BQ_BEGIN
>
> On Mar 27, 2023, at 5:23 PM, Yu Li < [ mailto:car...@gmail.com |
> car...@gmail.com ] > wrote:
>
> Dear Flinkers,
>
>
>
>
> As you may have noticed, we are pleased to announce that Flink Table Store
> has joined the Apache Incubator as a separate project called Apache
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a
> streaming data lake platform for high-speed data ingestion, change data
> tracking and efficient real-time analytics, with the vision of supporting a
> larger ecosystem and establishing a vibrant and neutral open source
> community.
>
>
>
>
> We would like to thank everyone for their great support and efforts for
> the Flink Table Store project, and warmly welcome everyone to join the
> development and activities of the new project. Apache Flink will continue
> to be one of the first-class citizens supported by Paimon, and we believe
> that the Flink and Paimon communities will maintain close cooperation.
>
>
>
>
> 亲爱的Flinkers，
>
>
>
>
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2]
> [3]。新项目的名字是 Apache
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>
>
>
>
>
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>
>
>
>
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>
>
>
>
> [1] [ https://paimon.apache.org/ | https://paimon.apache.org/ ]
> [2] [ https://github.com/apache/incubator-paimon |
> https://github.com/apache/incubator-paimon ]
> [3] [ https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
> | https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal ]
>
>
>
>
>
> BQ_END
>
>
> BQ_END
>
>
> BQ_END
>
>
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Weihua Hu]

Congratulations!

Best,
Weihua


On Mon, Mar 27, 2023 at 9:02 PM yuxia  wrote:

> congratulations!
>
> Best regards,
> Yuxia
>
>
> 发件人: "Andrew Otto" 
> 收件人: "Matthias Pohl" 
> 抄送: "Jing Ge" , "Leonard Xu" , "Yu
> Li" , "dev" , "User" <
> u...@flink.apache.org>, "user-zh" 
> 发送时间: 星期一, 2023年 3 月 27日 下午 8:57:50
> 主题: Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache
> Paimon(incubating)
>
> Exciting!
>
> If this ends up working well, Wikimedia Foundation would love to try it
> out!
>
> On Mon, Mar 27, 2023 at 8:39 AM Matthias Pohl via user < [ mailto:
> u...@flink.apache.org | u...@flink.apache.org ] > wrote:
>
>
>
> Congratulations and good luck with pushing the project forward.
>
> On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user < [ mailto:
> u...@flink.apache.org | u...@flink.apache.org ] > wrote:
>
> BQ_BEGIN
>
> Congrats!
> Best regards,
> Jing
>
> On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu < [ mailto:xbjt...@gmail.com |
> xbjt...@gmail.com ] > wrote:
>
> BQ_BEGIN
>
> Congratulations!
>
> Best,
> Leonard
>
>
> BQ_BEGIN
>
> On Mar 27, 2023, at 5:23 PM, Yu Li < [ mailto:car...@gmail.com |
> car...@gmail.com ] > wrote:
>
> Dear Flinkers,
>
>
>
>
> As you may have noticed, we are pleased to announce that Flink Table Store
> has joined the Apache Incubator as a separate project called Apache
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a
> streaming data lake platform for high-speed data ingestion, change data
> tracking and efficient real-time analytics, with the vision of supporting a
> larger ecosystem and establishing a vibrant and neutral open source
> community.
>
>
>
>
> We would like to thank everyone for their great support and efforts for
> the Flink Table Store project, and warmly welcome everyone to join the
> development and activities of the new project. Apache Flink will continue
> to be one of the first-class citizens supported by Paimon, and we believe
> that the Flink and Paimon communities will maintain close cooperation.
>
>
>
>
> 亲爱的Flinkers，
>
>
>
>
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2]
> [3]。新项目的名字是 Apache
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>
>
>
>
>
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>
>
>
>
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>
>
>
>
> [1] [ https://paimon.apache.org/ | https://paimon.apache.org/ ]
> [2] [ https://github.com/apache/incubator-paimon |
> https://github.com/apache/incubator-paimon ]
> [3] [ https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
> | https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal ]
>
>
>
>
>
> BQ_END
>
>
> BQ_END
>
>
> BQ_END
>
>
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[yuxia]

congratulations! 

Best regards, 
Yuxia 


发件人: "Andrew Otto"  
收件人: "Matthias Pohl"  
抄送: "Jing Ge" , "Leonard Xu" , "Yu Li" 
, "dev" , "User" 
, "user-zh"  
发送时间: 星期一, 2023年 3 月 27日 下午 8:57:50 
主题: Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache 
Paimon(incubating) 

Exciting! 

If this ends up working well, Wikimedia Foundation would love to try it out! 

On Mon, Mar 27, 2023 at 8:39 AM Matthias Pohl via user < [ 
mailto:user@flink.apache.org | user@flink.apache.org ] > wrote: 



Congratulations and good luck with pushing the project forward. 

On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user < [ 
mailto:user@flink.apache.org | user@flink.apache.org ] > wrote: 

BQ_BEGIN

Congrats! 
Best regards, 
Jing 

On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu < [ mailto:xbjt...@gmail.com | 
xbjt...@gmail.com ] > wrote: 

BQ_BEGIN

Congratulations! 

Best, 
Leonard 


BQ_BEGIN

On Mar 27, 2023, at 5:23 PM, Yu Li < [ mailto:car...@gmail.com | 
car...@gmail.com ] > wrote: 

Dear Flinkers, 




As you may have noticed, we are pleased to announce that Flink Table Store has 
joined the Apache Incubator as a separate project called Apache 
Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
streaming data lake platform for high-speed data ingestion, change data 
tracking and efficient real-time analytics, with the vision of supporting a 
larger ecosystem and establishing a vibrant and neutral open source community. 




We would like to thank everyone for their great support and efforts for the 
Flink Table Store project, and warmly welcome everyone to join the development 
and activities of the new project. Apache Flink will continue to be one of the 
first-class citizens supported by Paimon, and we believe that the Flink and 
Paimon communities will maintain close cooperation. 




亲爱的Flinkers， 




正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2] 
[3]。新项目的名字是 Apache 
Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
 




在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 
将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。 




Best Regards, 
Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC) 

致礼， 
李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC） 




[1] [ https://paimon.apache.org/ | https://paimon.apache.org/ ] 
[2] [ https://github.com/apache/incubator-paimon | 
https://github.com/apache/incubator-paimon ] 
[3] [ https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal | 
https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal ] 





BQ_END


BQ_END


BQ_END

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[yuxia]

congratulations! 

Best regards, 
Yuxia 


发件人: "Andrew Otto"  
收件人: "Matthias Pohl"  
抄送: "Jing Ge" , "Leonard Xu" , "Yu Li" 
, "dev" , "User" 
, "user-zh"  
发送时间: 星期一, 2023年 3 月 27日 下午 8:57:50 
主题: Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache 
Paimon(incubating) 

Exciting! 

If this ends up working well, Wikimedia Foundation would love to try it out! 

On Mon, Mar 27, 2023 at 8:39 AM Matthias Pohl via user < [ 
mailto:u...@flink.apache.org | u...@flink.apache.org ] > wrote: 



Congratulations and good luck with pushing the project forward. 

On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user < [ 
mailto:u...@flink.apache.org | u...@flink.apache.org ] > wrote: 

BQ_BEGIN

Congrats! 
Best regards, 
Jing 

On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu < [ mailto:xbjt...@gmail.com | 
xbjt...@gmail.com ] > wrote: 

BQ_BEGIN

Congratulations! 

Best, 
Leonard 


BQ_BEGIN

On Mar 27, 2023, at 5:23 PM, Yu Li < [ mailto:car...@gmail.com | 
car...@gmail.com ] > wrote: 

Dear Flinkers, 




As you may have noticed, we are pleased to announce that Flink Table Store has 
joined the Apache Incubator as a separate project called Apache 
Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
streaming data lake platform for high-speed data ingestion, change data 
tracking and efficient real-time analytics, with the vision of supporting a 
larger ecosystem and establishing a vibrant and neutral open source community. 




We would like to thank everyone for their great support and efforts for the 
Flink Table Store project, and warmly welcome everyone to join the development 
and activities of the new project. Apache Flink will continue to be one of the 
first-class citizens supported by Paimon, and we believe that the Flink and 
Paimon communities will maintain close cooperation. 




亲爱的Flinkers， 




正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2] 
[3]。新项目的名字是 Apache 
Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
 




在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 
将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。 




Best Regards, 
Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC) 

致礼， 
李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC） 




[1] [ https://paimon.apache.org/ | https://paimon.apache.org/ ] 
[2] [ https://github.com/apache/incubator-paimon | 
https://github.com/apache/incubator-paimon ] 
[3] [ https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal | 
https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal ] 





BQ_END


BQ_END


BQ_END

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Andrew Otto]

Exciting!

If this ends up working well, Wikimedia Foundation would love to try it out!

On Mon, Mar 27, 2023 at 8:39 AM Matthias Pohl via user <
user@flink.apache.org> wrote:

> Congratulations and good luck with pushing the project forward.
>
> On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user 
> wrote:
>
>> Congrats!
>>
>> Best regards,
>> Jing
>>
>> On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu  wrote:
>>
>>> Congratulations!
>>>
>>>
>>> Best,
>>> Leonard
>>>
>>> On Mar 27, 2023, at 5:23 PM, Yu Li  wrote:
>>>
>>> Dear Flinkers,
>>>
>>>
>>>
>>> As you may have noticed, we are pleased to announce that Flink Table Store 
>>> has joined the Apache Incubator as a separate project called Apache 
>>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
>>> streaming data lake platform for high-speed data ingestion, change data 
>>> tracking and efficient real-time analytics, with the vision of supporting a 
>>> larger ecosystem and establishing a vibrant and neutral open source 
>>> community.
>>>
>>>
>>>
>>> We would like to thank everyone for their great support and efforts for the 
>>> Flink Table Store project, and warmly welcome everyone to join the 
>>> development and activities of the new project. Apache Flink will continue 
>>> to be one of the first-class citizens supported by Paimon, and we believe 
>>> that the Flink and Paimon communities will maintain close cooperation.
>>>
>>>
>>> 亲爱的Flinkers，
>>>
>>>
>>> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
>>> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
>>> Apache 
>>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>>>
>>>
>>> 在这里我们要感谢大家对 Flink Table Store
>>> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信
>>> Flink 和 Paimon 社区将继续保持密切合作。
>>>
>>>
>>> Best Regards,
>>> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>>>
>>> 致礼，
>>> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>>>
>>> [1] https://paimon.apache.org/
>>> [2] https://github.com/apache/incubator-paimon
>>> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>>>
>>>
>>>

Re: Handling JSON Serialization without Kryo

[Andrew Otto]

Hi,

> The problem here is that the shape of the data can vary wildly and
dynamically. Some records may have properties unique to only that record,
which makes defining a POJO difficult

AFAIK, the only way to avoid POJOs in Flink is to use Row (DataStream) or
RowData (Table API).  These are Flink's 'schema' typesystem, and they allow
you to dynamically describe the shape of data.

At the Wikimedia Foundation, we also find it difficult to use JSON +
Flink.  We at least have JSONSchemas though, and that has allowed us to
write custom converters (serdes) for JSONSchema -> Row & RowData.

This might not be helpful for you (unless you also use JSONSchema), but
here's a couple of convos on this list with some relevant info:

- Flink, JSON, and JSONSchemas

- Converting ResolvedSchema to JSON and Protobuf Schemas

Good luck!
-Andrew Otto
 Wikimedia Foundation

On Wed, Mar 22, 2023 at 8:07 AM Rion Williams  wrote:

> Hi Ken,
>
> I’m going to profile the job today to try and get a better handle on where
> the bottleneck is. The job currently just passes around JsonObjects between
> the operators, which are relying on Kryo. The job also writes to Postgres,
> Kafka, and Elasticsearch so it’s possible that one of those is causing the
> back-pressure.
>
> I’m a bit shocked at the stunningly low speeds as well. Initially, the job
> would perform fine but checkpointing sizes would gradually build up (as
> would durations for them) until performance degraded to the borderline
> unusable 1-2 records/second.
>
> On Mar 21, 2023, at 2:35 PM, Ken Krugler 
> wrote:
>
> Hi Rion,
>
> I’m using Gson to deserialize to a Map.
>
> 1-2 records/second sounds way too slow, unless each record is enormous.
>
> — Ken
>
> On Mar 21, 2023, at 6:18 AM, Rion Williams  wrote:
>
> Hi Ken,
>
> Thanks for the response. I hadn't tried exploring the use of the Record
> class, which I'm assuming you're referring to a flink.types.Record, to read
> the JSON into. Did you handle this via using a mapper to read the
> properties in (e.g. Gson, Jackson) as fields or take a different approach?
> Additionally, how has your experience been with performance? Kryo with the
> existing job leveraging JsonObjects (via Gson) is horrific (~1-2
> records/second) and can't keep up with the speed of the producers, which is
> the impetus behind reevaluating the serialization.
>
> I'll explore this a bit more.
>
> Thanks,
>
> Rion
>
> On Mon, Mar 20, 2023 at 10:28 PM Ken Krugler 
> wrote:
>
>> Hi Rion,
>>
>> For my similar use case, I was able to make a simplifying assumption that
>> my top-level JSON object was a record.
>>
>> I then registered a custom Kryo serde that knew how to handle the handful
>> of JsonPrimitive types for the record entries.
>>
>> I recently looked at extending that to support arrays and nested records,
>> but haven’t had to do that.
>>
>> — Ken
>>
>>
>> On Mar 20, 2023, at 6:56 PM, Rion Williams  wrote:
>>
>> Hi Shammon,
>>
>> Unfortunately it’s a data stream job. I’ve been exploring a few options
>> but haven’t found anything I’ve decided on yet. I’m currently looking at
>> seeing if I can leverage some type of partial serialization to bind to the
>> properties that I know the job will use and retain the rest as a JSON blob.
>> I’ve also consider trying to store the fields as a large map of
>> string-object pairs and translating thay into a string prior to writing to
>> the sinks.
>>
>> Still accepting any/all ideas that I come across to see if I can handle
>> this in an efficient, reasonable way.
>>
>> Thanks,
>>
>> Rion
>>
>> On Mar 20, 2023, at 8:40 PM, Shammon FY  wrote:
>>
>> 
>> Hi Rion
>>
>> Is your job datastream or table/sql? If it is a table/sql job, and you
>> can define all the fields in json you need, then you can directly use json
>> format [1] to parse the data.
>>
>> You can also customize udf functions to parse json data into struct data,
>> such as map, row and other types supported by flink
>>
>>
>> [1]
>> https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/formats/json/
>>
>> Best,
>> Shammon FY
>>
>>
>> On Sun, Mar 19, 2023 at 7:44 AM Rion Williams 
>> wrote:
>>
>>> Hi all,
>>>
>>> I’m reaching out today for some suggestions (and hopefully a solution)
>>> for a Flink job that I’m working on. The job itself reads JSON strings from
>>> a Kafka topic and reads those into JSONObjects (currently via Gson), which
>>> are then operated against, before ultimately being written out to Kafka
>>> again.
>>>
>>> The problem here is that the shape of the data can vary wildly and
>>> dynamically. Some records may have properties unique to only that record,
>>> which makes defining a POJO difficult. In addition to this, the JSONObjects
>>> fall by to Kryo serialization which is leading to atrocious throughput.
>>>
>>> I basically need to read in JSON strings, enrich properties on 

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Matthias Pohl]

Congratulations and good luck with pushing the project forward.

On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user 
wrote:

> Congrats!
>
> Best regards,
> Jing
>
> On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu  wrote:
>
>> Congratulations!
>>
>>
>> Best,
>> Leonard
>>
>> On Mar 27, 2023, at 5:23 PM, Yu Li  wrote:
>>
>> Dear Flinkers,
>>
>>
>>
>> As you may have noticed, we are pleased to announce that Flink Table Store 
>> has joined the Apache Incubator as a separate project called Apache 
>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
>> streaming data lake platform for high-speed data ingestion, change data 
>> tracking and efficient real-time analytics, with the vision of supporting a 
>> larger ecosystem and establishing a vibrant and neutral open source 
>> community.
>>
>>
>>
>> We would like to thank everyone for their great support and efforts for the 
>> Flink Table Store project, and warmly welcome everyone to join the 
>> development and activities of the new project. Apache Flink will continue to 
>> be one of the first-class citizens supported by Paimon, and we believe that 
>> the Flink and Paimon communities will maintain close cooperation.
>>
>>
>> 亲爱的Flinkers，
>>
>>
>> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
>> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
>> Apache 
>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>>
>>
>> 在这里我们要感谢大家对 Flink Table Store
>> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信
>> Flink 和 Paimon 社区将继续保持密切合作。
>>
>>
>> Best Regards,
>> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>>
>> 致礼，
>> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>>
>> [1] https://paimon.apache.org/
>> [2] https://github.com/apache/incubator-paimon
>> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>>
>>
>>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Matthias Pohl via user]

Congratulations and good luck with pushing the project forward.

On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user 
wrote:

> Congrats!
>
> Best regards,
> Jing
>
> On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu  wrote:
>
>> Congratulations!
>>
>>
>> Best,
>> Leonard
>>
>> On Mar 27, 2023, at 5:23 PM, Yu Li  wrote:
>>
>> Dear Flinkers,
>>
>>
>>
>> As you may have noticed, we are pleased to announce that Flink Table Store 
>> has joined the Apache Incubator as a separate project called Apache 
>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
>> streaming data lake platform for high-speed data ingestion, change data 
>> tracking and efficient real-time analytics, with the vision of supporting a 
>> larger ecosystem and establishing a vibrant and neutral open source 
>> community.
>>
>>
>>
>> We would like to thank everyone for their great support and efforts for the 
>> Flink Table Store project, and warmly welcome everyone to join the 
>> development and activities of the new project. Apache Flink will continue to 
>> be one of the first-class citizens supported by Paimon, and we believe that 
>> the Flink and Paimon communities will maintain close cooperation.
>>
>>
>> 亲爱的Flinkers，
>>
>>
>> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
>> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
>> Apache 
>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>>
>>
>> 在这里我们要感谢大家对 Flink Table Store
>> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信
>> Flink 和 Paimon 社区将继续保持密切合作。
>>
>>
>> Best Regards,
>> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>>
>> 致礼，
>> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>>
>> [1] https://paimon.apache.org/
>> [2] https://github.com/apache/incubator-paimon
>> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>>
>>
>>

Re: Issue with the flink version 1.10.1

[Leonard Xu]

Hi, Kiran

To be honest, both 1.10 and 1.9 are pretty old version, it’s hard to fix and 
release a bugfix version for 1.10.1 even the community help troubleshoot your 
issue. 
So, could you try latest versions like Flink 1.16.1 or 1.17.0?

Best,
Leonard

> On Mar 27, 2023, at 8:28 PM, Kiran Kumar Kathe  
> wrote:
> 
> When I submit a job using flink version 1.10.1 ,  it is not upadating the 
> jobs that are running and completed successfully in the Web UI of YARN 
> resource manager . But When I use flink version 1.9.3 it is working fine and 
> I am able to see the jobs that are running and completed in 
> YARN resource manager Web UI . And to find why this is happening I just tried 
> with replacing the application folders and in lib folder when I use the 
> flink_dist jar of version 1.9.3 in place of flink_dist of version 1.10.1 it 
> is running fine and I am able to see the jobs running and completed. Is it 
> the right way , if not will I face any compatible issues in future with this 
> change of flink_dist jar in lib folder.

Re: Issue with the flink version 1.10.1

[Matthias Pohl via user]

Hi Kiran,
it's really hard to come up with an answer based on your description.
Usually, it helps to share some logs with the exact error that's appearing
and a clear description on what you're observing and what you're expecting.
A plain "no jobs are running" is too general to come up with a conclusion.
Sorry.

Additionally, let me state that Flink 1.10 and 1.9 are quite old versions.
The community doesn't support those versions anymore. It might be the case
that you're running into issues that are already fixed in newer versions.
Investigating code from years ago can be quite tedious.

Best,
Matthias

On Mon, Mar 27, 2023 at 2:29 PM Kiran Kumar Kathe <
kirankumarkathe...@gmail.com> wrote:

> When I submit a job using flink version 1.10.1 ,  it is not upadating the
> jobs that are running and completed successfully in the Web UI of YARN
> resource manager . But When I use flink version 1.9.3 it is working fine
> and I am able to see the jobs that are running and completed in
> YARN resource manager Web UI . And to find why this is happening I just
> tried with replacing the application folders and in lib folder when I use
> the flink_dist jar of version 1.9.3 in place of flink_dist of version
> 1.10.1 it is running fine and I am able to see the jobs running and
> completed. Is it the right way , if not will I face any compatible issues
> in future with this change of flink_dist jar in lib folder.
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Konstantin Knauf]

Congrats!

Am Mo., 27. März 2023 um 14:32 Uhr schrieb Leonard Xu :

> Congratulations!
>
>
> Best,
> Leonard
>
> On Mar 27, 2023, at 5:23 PM, Yu Li  wrote:
>
> Dear Flinkers,
>
>
>
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
>
>
>
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
>
>
> 亲爱的Flinkers，
>
>
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>
>
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>
>
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>
> [1] https://paimon.apache.org/
> [2] https://github.com/apache/incubator-paimon
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>
>
>

-- 
https://twitter.com/snntrable
https://github.com/knaufk

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Konstantin Knauf]

Congrats!

Am Mo., 27. März 2023 um 14:32 Uhr schrieb Leonard Xu :

> Congratulations!
>
>
> Best,
> Leonard
>
> On Mar 27, 2023, at 5:23 PM, Yu Li  wrote:
>
> Dear Flinkers,
>
>
>
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
>
>
>
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
>
>
> 亲爱的Flinkers，
>
>
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>
>
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>
>
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>
> [1] https://paimon.apache.org/
> [2] https://github.com/apache/incubator-paimon
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>
>
>

-- 
https://twitter.com/snntrable
https://github.com/knaufk

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Jing Ge]

Congrats!

Best regards,
Jing

On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu  wrote:

> Congratulations!
>
>
> Best,
> Leonard
>
> On Mar 27, 2023, at 5:23 PM, Yu Li  wrote:
>
> Dear Flinkers,
>
>
>
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
>
>
>
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
>
>
> 亲爱的Flinkers，
>
>
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>
>
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>
>
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>
> [1] https://paimon.apache.org/
> [2] https://github.com/apache/incubator-paimon
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>
>
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Jing Ge via user]

Congrats!

Best regards,
Jing

On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu  wrote:

> Congratulations!
>
>
> Best,
> Leonard
>
> On Mar 27, 2023, at 5:23 PM, Yu Li  wrote:
>
> Dear Flinkers,
>
>
>
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
>
>
>
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
>
>
> 亲爱的Flinkers，
>
>
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
> 孵化器独立孵化 [1] [2] [3]。新项目的名字是
> Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
>
>
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
>
>
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
>
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
>
> [1] https://paimon.apache.org/
> [2] https://github.com/apache/incubator-paimon
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal
>
>
>

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Leonard Xu]

Congratulations!


Best,
Leonard

> On Mar 27, 2023, at 5:23 PM, Yu Li  wrote:
> 
> Dear Flinkers,
> 
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
> 
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
> 
> 亲爱的Flinkers，
> 
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2] 
> [3]。新项目的名字是 Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
> 
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
> 
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
> 
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
> 
> [1] https://paimon.apache.org/ 
> [2] https://github.com/apache/incubator-paimon 
> 
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal 
> 

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Leonard Xu]

Congratulations!


Best,
Leonard

> On Mar 27, 2023, at 5:23 PM, Yu Li  wrote:
> 
> Dear Flinkers,
> 
> As you may have noticed, we are pleased to announce that Flink Table Store 
> has joined the Apache Incubator as a separate project called Apache 
> Paimon(incubating) [1] [2] [3]. The new project still aims at building a 
> streaming data lake platform for high-speed data ingestion, change data 
> tracking and efficient real-time analytics, with the vision of supporting a 
> larger ecosystem and establishing a vibrant and neutral open source community.
> 
> We would like to thank everyone for their great support and efforts for the 
> Flink Table Store project, and warmly welcome everyone to join the 
> development and activities of the new project. Apache Flink will continue to 
> be one of the first-class citizens supported by Paimon, and we believe that 
> the Flink and Paimon communities will maintain close cooperation.
> 
> 亲爱的Flinkers，
> 
> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache 孵化器独立孵化 [1] [2] 
> [3]。新项目的名字是 Apache 
> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。
> 
> 在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 
> 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。
> 
> Best Regards,
> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)
> 
> 致礼，
> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）
> 
> [1] https://paimon.apache.org/ 
> [2] https://github.com/apache/incubator-paimon 
> 
> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal 
> 

Issue with the flink version 1.10.1

[Kiran Kumar Kathe]

When I submit a job using flink version 1.10.1 ,  it is not upadating the
jobs that are running and completed successfully in the Web UI of YARN
resource manager . But When I use flink version 1.9.3 it is working fine
and I am able to see the jobs that are running and completed in
YARN resource manager Web UI . And to find why this is happening I just
tried with replacing the application folders and in lib folder when I use
the flink_dist jar of version 1.9.3 in place of flink_dist of version
1.10.1 it is running fine and I am able to see the jobs running and
completed. Is it the right way , if not will I face any compatible issues
in future with this change of flink_dist jar in lib folder.

Custom source

[Lasse Nedergaard]

Hi. 

I have to use data from a Rest API, very slow changing. Instead of doing an 
async i/o request I would like to create a source function that read from Rest 
Api so I can connect and enrich another stream without a lookup for each 
incoming row. 
Flink doc provide an explanation of how to do it but I find it pretty hard get 
all the concepts to work together and I haven’t been able to find a “simple” 
app example. 
Is there anyone out there knowing a good example or give a simple explanation 
of best practice for implementing your own source 

Med venlig hilsen / Best regards
Lasse Nedergaard

[ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Yu Li]

Dear Flinkers,


As you may have noticed, we are pleased to announce that Flink Table
Store has joined the Apache Incubator as a separate project called
Apache Paimon(incubating) [1] [2] [3]. The new project still aims at
building a streaming data lake platform for high-speed data ingestion,
change data tracking and efficient real-time analytics, with the
vision of supporting a larger ecosystem and establishing a vibrant and
neutral open source community.


We would like to thank everyone for their great support and efforts
for the Flink Table Store project, and warmly welcome everyone to join
the development and activities of the new project. Apache Flink will
continue to be one of the first-class citizens supported by Paimon,
and we believe that the Flink and Paimon communities will maintain
close cooperation.


亲爱的Flinkers，


正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
孵化器独立孵化 [1] [2] [3]。新项目的名字是
Apache 
Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。


在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。


Best Regards,

Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)


致礼，

李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）


[1] https://paimon.apache.org/

[2] https://github.com/apache/incubator-paimon

[3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal

[ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

[Yu Li]

Dear Flinkers,


As you may have noticed, we are pleased to announce that Flink Table
Store has joined the Apache Incubator as a separate project called
Apache Paimon(incubating) [1] [2] [3]. The new project still aims at
building a streaming data lake platform for high-speed data ingestion,
change data tracking and efficient real-time analytics, with the
vision of supporting a larger ecosystem and establishing a vibrant and
neutral open source community.


We would like to thank everyone for their great support and efforts
for the Flink Table Store project, and warmly welcome everyone to join
the development and activities of the new project. Apache Flink will
continue to be one of the first-class citizens supported by Paimon,
and we believe that the Flink and Paimon communities will maintain
close cooperation.


亲爱的Flinkers，


正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache
孵化器独立孵化 [1] [2] [3]。新项目的名字是
Apache 
Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。


在这里我们要感谢大家对 Flink Table Store 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink
将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 Flink 和 Paimon 社区将继续保持密切合作。


Best Regards,

Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC)


致礼，

李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC）


[1] https://paimon.apache.org/

[2] https://github.com/apache/incubator-paimon

[3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal

Re: [ANNOUNCE] Apache Flink 1.17.0 released

[Matthias Pohl via user]

Here are a few things I noticed from the 1.17 release retrospectively which
I want to share (other release managers might have a different view or
might disagree):

- Google Meet might not be the best choice for the release sync. We need to
be able to invite attendees even if the creator of the meeting isn't
available (maybe try Zoom or jitsi instead?)

- Release sync every 2 weeks and a switch to weekly after feature freeze
felt reasonable

- Slack worked well as a collaboration tool to document the monitoring
tasks (#builds [1], #flink-dev-benchmarks [2]) in a team with multiple
release managers

- The Slack Azure Pipeline bot seems to be buggy. It swallows some build
failures. It's not a severe issue, though. We created #builds-debug [3] to
monitor whether it's happening consistently. The issue is covered in
FLINK-30733 [4].

- Having dedicated people for monitoring the build failures helps getting a
more consistent picture of test instabilities

- We experienced occasional issues in the manual steps of the release
creation in the past (e.g. japicmp config was not properly pushed).
Creating Jira issues for the release helped to make the release creation
more transparent and made the steps more reviewable [5][6][7][8].
Additionally, it helped to distribute subtasks to different people with
Jira being the tool for documentation and synchronization. That's
especially helpful when there is more than one person in charge of creating
the release.

- We had backports/merges without PRs happening by committers occasionally
during the 1.17 release which broke master/release branches (probably,
changes were done locally before merging which were not part of the PR to
have a faster backport experience). It might make sense to remind everyone
that this should be avoided. Not sure whether we want/can restrict that.

- We observed a good response on fixing test instabilities by the end of
the release cycle but had some long running issues earlier in the cycle
which caused extra efforts on the release managers due to reoccurring test
failures.

- Release testing picked up “slowly”: Initially, we planned 2 weeks for
release testing. But there was not really any progress (tickets being
created and worked on) in the first week. In the end, we had to extend the
phase by another week resulting in 3 instead of 2 weeks of release testing.
I guess we could encourage the community to create release testing tasks
earlier and label them properly to be able to monitor the effort. That
would even enable us to do release testing for a certain feature after the
feature is done and not necessarily only at the end of the release cycle.

- Manual test data generation is tedious (FLINK-31593 [9]). But this should
be fixed in 1.18 with FLINK-27518 [10] being almost done.

- We started creating documentation for release management [11]. The goal
is to collect what tasks are there to help support a Flink release to
encourage newcomers to pick up the task.

I'm going to add these to the Flink 1.17 release documentation [12] as
feedback as well.

Best,
Matthias

[1] https://apache-flink.slack.com/archives/C03MR1HQHK2
[2] https://apache-flink.slack.com/archives/C0471S0DFJ9
[3] https://apache-flink.slack.com/archives/C04LZM3EE9E
[4] https://issues.apache.org/jira/browse/FLINK-30733
[5] https://issues.apache.org/jira/browse/FLINK-31146
[6] https://issues.apache.org/jira/browse/FLINK-31154
[7] https://issues.apache.org/jira/browse/FLINK-31562
[8] https://issues.apache.org/jira/browse/FLINK-31567
[9] https://issues.apache.org/jira/browse/FLINK-31593
[10] https://issues.apache.org/jira/browse/FLINK-27518
[11]
https://cwiki.apache.org/confluence/display/FLINK/Flink+Release+Management
[12] https://cwiki.apache.org/confluence/display/FLINK/1.17+Release

On Sat, Mar 25, 2023 at 8:29 AM Hang Ruan  wrote:

> Thanks for the great work ! Congrats all!
>
> Best,
> Hang
>
> Panagiotis Garefalakis  于2023年3月25日周六 03:22写道：
>
>> Congrats all! Well done!
>>
>> Cheers,
>> Panagiotis
>>
>> On Fri, Mar 24, 2023 at 2:46 AM Qingsheng Ren  wrote:
>>
>> > I'd like to say thank you to all contributors of Flink 1.17. Your
>> support
>> > and great work together make this giant step forward!
>> >
>> > Also like Matthias mentioned, feel free to leave us any suggestions and
>> > let's improve the releasing procedure together.
>> >
>> > Cheers,
>> > Qingsheng
>> >
>> > On Fri, Mar 24, 2023 at 5:00 PM Etienne Chauchot 
>> > wrote:
>> >
>> >> Congrats to all the people involved!
>> >>
>> >> Best
>> >>
>> >> Etienne
>> >>
>> >> Le 23/03/2023 à 10:19, Leonard Xu a écrit :
>> >> > The Apache Flink community is very happy to announce the release of
>> >> Apache Flink 1.17.0, which is the first release for the Apache Flink
>> 1.17
>> >> series.
>> >> >
>> >> > Apache Flink® is an open-source unified stream and batch data
>> >> processing framework for distributed, high-performing,
>> always-available,
>> >> and accurate data applications.
>> >> >
>> >> > The release is available 

Re: [ANNOUNCE] Apache Flink 1.17.0 released

[Matthias Pohl]

Here are a few things I noticed from the 1.17 release retrospectively which
I want to share (other release managers might have a different view or
might disagree):

- Google Meet might not be the best choice for the release sync. We need to
be able to invite attendees even if the creator of the meeting isn't
available (maybe try Zoom or jitsi instead?)

- Release sync every 2 weeks and a switch to weekly after feature freeze
felt reasonable

- Slack worked well as a collaboration tool to document the monitoring
tasks (#builds [1], #flink-dev-benchmarks [2]) in a team with multiple
release managers

- The Slack Azure Pipeline bot seems to be buggy. It swallows some build
failures. It's not a severe issue, though. We created #builds-debug [3] to
monitor whether it's happening consistently. The issue is covered in
FLINK-30733 [4].

- Having dedicated people for monitoring the build failures helps getting a
more consistent picture of test instabilities

- We experienced occasional issues in the manual steps of the release
creation in the past (e.g. japicmp config was not properly pushed).
Creating Jira issues for the release helped to make the release creation
more transparent and made the steps more reviewable [5][6][7][8].
Additionally, it helped to distribute subtasks to different people with
Jira being the tool for documentation and synchronization. That's
especially helpful when there is more than one person in charge of creating
the release.

- We had backports/merges without PRs happening by committers occasionally
during the 1.17 release which broke master/release branches (probably,
changes were done locally before merging which were not part of the PR to
have a faster backport experience). It might make sense to remind everyone
that this should be avoided. Not sure whether we want/can restrict that.

- We observed a good response on fixing test instabilities by the end of
the release cycle but had some long running issues earlier in the cycle
which caused extra efforts on the release managers due to reoccurring test
failures.

- Release testing picked up “slowly”: Initially, we planned 2 weeks for
release testing. But there was not really any progress (tickets being
created and worked on) in the first week. In the end, we had to extend the
phase by another week resulting in 3 instead of 2 weeks of release testing.
I guess we could encourage the community to create release testing tasks
earlier and label them properly to be able to monitor the effort. That
would even enable us to do release testing for a certain feature after the
feature is done and not necessarily only at the end of the release cycle.

- Manual test data generation is tedious (FLINK-31593 [9]). But this should
be fixed in 1.18 with FLINK-27518 [10] being almost done.

- We started creating documentation for release management [11]. The goal
is to collect what tasks are there to help support a Flink release to
encourage newcomers to pick up the task.

I'm going to add these to the Flink 1.17 release documentation [12] as
feedback as well.

Best,
Matthias

[1] https://apache-flink.slack.com/archives/C03MR1HQHK2
[2] https://apache-flink.slack.com/archives/C0471S0DFJ9
[3] https://apache-flink.slack.com/archives/C04LZM3EE9E
[4] https://issues.apache.org/jira/browse/FLINK-30733
[5] https://issues.apache.org/jira/browse/FLINK-31146
[6] https://issues.apache.org/jira/browse/FLINK-31154
[7] https://issues.apache.org/jira/browse/FLINK-31562
[8] https://issues.apache.org/jira/browse/FLINK-31567
[9] https://issues.apache.org/jira/browse/FLINK-31593
[10] https://issues.apache.org/jira/browse/FLINK-27518
[11]
https://cwiki.apache.org/confluence/display/FLINK/Flink+Release+Management
[12] https://cwiki.apache.org/confluence/display/FLINK/1.17+Release

On Sat, Mar 25, 2023 at 8:29 AM Hang Ruan  wrote:

> Thanks for the great work ! Congrats all!
>
> Best,
> Hang
>
> Panagiotis Garefalakis  于2023年3月25日周六 03:22写道：
>
>> Congrats all! Well done!
>>
>> Cheers,
>> Panagiotis
>>
>> On Fri, Mar 24, 2023 at 2:46 AM Qingsheng Ren  wrote:
>>
>> > I'd like to say thank you to all contributors of Flink 1.17. Your
>> support
>> > and great work together make this giant step forward!
>> >
>> > Also like Matthias mentioned, feel free to leave us any suggestions and
>> > let's improve the releasing procedure together.
>> >
>> > Cheers,
>> > Qingsheng
>> >
>> > On Fri, Mar 24, 2023 at 5:00 PM Etienne Chauchot 
>> > wrote:
>> >
>> >> Congrats to all the people involved!
>> >>
>> >> Best
>> >>
>> >> Etienne
>> >>
>> >> Le 23/03/2023 à 10:19, Leonard Xu a écrit :
>> >> > The Apache Flink community is very happy to announce the release of
>> >> Apache Flink 1.17.0, which is the first release for the Apache Flink
>> 1.17
>> >> series.
>> >> >
>> >> > Apache Flink® is an open-source unified stream and batch data
>> >> processing framework for distributed, high-performing,
>> always-available,
>> >> and accurate data applications.
>> >> >
>> >> > The release is available 

Re:Re: flink1.16 sql gateway hive2

[guanyq]

感谢两位老师





在 2023-03-27 09:58:24，"Shengkai Fang"  写道：
>方勇老师说的没错。我们在文档里面也加了如何配置 hiveserver2 endpoint 的文档[1]
>
>[1]
>https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/hive-compatibility/hiveserver2/#setting-up
>
>Shammon FY  于2023年3月27日周一 08:41写道：
>
>> Hi
>>
>>
>> 如果要启动hiveserver2协议的gateway，需要将jar包flink-connector-hive_${scala.binary.version}放入到gateway的lib目录
>>
>> Best,
>> Shammon FY
>>
>>
>> On Sun, Mar 26, 2023 at 12:07 PM guanyq  wrote:
>>
>> > 本地启动了flink及hive在启动sql gateway时有以下异常，请问还需要其他什么操作么
>> > ./bin/sql-gateway.sh start-foreground
>> > -Dsql-gateway.endpoint.type=hiveserver2
>> >
>> -Dsql-gateway.endpoint.hiveserver2.catalog.hive-conf-dir=/usr/local/app/apache-hive-3.1.2-bin/conf
>> >
>> >
>> > 异常信息
>> >
>> > Available factory identifiers are:
>> > rest
>> > at
>> >
>> org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:545)
>> > ~[flink-table-api-java-uber-1.16.0.jar:1.16.0]
>> > at
>> >
>> org.apache.flink.table.gateway.api.endpoint.SqlGatewayEndpointFactoryUtils.createSqlGatewayEndpoint(SqlGatewayEndpointFactoryUtils.java:65)
>> > ~[flink-table-api-java-uber-1.16.0.jar:1.16.0]
>> > at org.apache.flink.table.gateway.SqlGateway.start(SqlGateway.java:72)
>> > [flink-sql-gateway-1.16.0.jar:1.16.0]
>> > at
>> >
>> org.apache.flink.table.gateway.SqlGateway.startSqlGateway(SqlGateway.java:118)
>> > [flink-sql-gateway-1.16.0.jar:1.16.0]
>> > at org.apache.flink.table.gateway.SqlGateway.main(SqlGateway.java:98)
>> > [flink-sql-gateway-1.16.0.jar:1.16.0]
>> > Exception in thread "main"
>> > org.apache.flink.table.gateway.api.utils.SqlGatewayException: Failed to
>> > start the endpoints.
>> > at org.apache.flink.table.gateway.SqlGateway.start(SqlGateway.java:79)
>> > at
>> >
>> org.apache.flink.table.gateway.SqlGateway.startSqlGateway(SqlGateway.java:118)
>> > at org.apache.flink.table.gateway.SqlGateway.main(SqlGateway.java:98)
>> > Caused by: org.apache.flink.table.api.ValidationException: Could not find
>> > any factory for identifier 'hiveserver2' that implements
>> > 'SqlGatewayEndpointFactory' in the classpath.
>> > Available factory identifiers are:
>> > rest
>> > at
>> >
>> org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:545)
>> > at
>> >
>> org.apache.flink.table.gateway.api.endpoint.SqlGatewayEndpointFactoryUtils.createSqlGatewayEndpoint(SqlGatewayEndpointFactoryUtils.java:65)
>> > at org.apache.flink.table.gateway.SqlGateway.start(SqlGateway.java:72)
>> > ... 2 more
>> >
>> >
>>