[ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Yu Li
Dear Flinkers, As you may have noticed, we are pleased to announce that Flink Table Store has joined the Apache Incubator as a separate project called Apache Paimon(incubating) [1] [2] [3]. The new project still aims at building a streaming data lake platform for high-speed data ingestion,

Re: [ANNOUNCE] Apache Flink 1.17.0 released

2023-03-27 Thread Matthias Pohl via user
Here are a few things I noticed from the 1.17 release retrospectively which I want to share (other release managers might have a different view or might disagree): - Google Meet might not be the best choice for the release sync. We need to be able to invite attendees even if the creator of the

Custom source

2023-03-27 Thread Lasse Nedergaard
Hi. I have to use data from a Rest API, very slow changing. Instead of doing an async i/o request I would like to create a source function that read from Rest Api so I can connect and enrich another stream without a lookup for each incoming row. Flink doc provide an explanation of how to do

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread ramkrishna vasudevan
Congratulations !!! On Mon, Mar 27, 2023 at 2:54 PM Yu Li wrote: > Dear Flinkers, > > > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project called Apache > Paimon(incubating) [1] [2] [3]. The new project still

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Alexander Fedulov
Great to see this, congratulations! Best, Alex On Mon, 27 Mar 2023 at 11:24, Yu Li wrote: > Dear Flinkers, > > > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project called Apache > Paimon(incubating) [1] [2]

Issue with the flink version 1.10.1

2023-03-27 Thread Kiran Kumar Kathe
When I submit a job using flink version 1.10.1 , it is not upadating the jobs that are running and completed successfully in the Web UI of YARN resource manager . But When I use flink version 1.9.3 it is working fine and I am able to see the jobs that are running and completed in YARN resource

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Leonard Xu
Congratulations! Best, Leonard > On Mar 27, 2023, at 5:23 PM, Yu Li wrote: > > Dear Flinkers, > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project called Apache > Paimon(incubating) [1] [2] [3]. The new

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Matthias Pohl via user
Congratulations and good luck with pushing the project forward. On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user wrote: > Congrats! > > Best regards, > Jing > > On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu wrote: > >> Congratulations! >> >> >> Best, >> Leonard >> >> On Mar 27, 2023, at 5:23 PM,

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Konstantin Knauf
Congrats! Am Mo., 27. März 2023 um 14:32 Uhr schrieb Leonard Xu : > Congratulations! > > > Best, > Leonard > > On Mar 27, 2023, at 5:23 PM, Yu Li wrote: > > Dear Flinkers, > > > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a

Re: Issue with the flink version 1.10.1

2023-03-27 Thread Matthias Pohl via user
Hi Kiran, it's really hard to come up with an answer based on your description. Usually, it helps to share some logs with the exact error that's appearing and a clear description on what you're observing and what you're expecting. A plain "no jobs are running" is too general to come up with a

Re: Issue with the flink version 1.10.1

2023-03-27 Thread Leonard Xu
Hi, Kiran To be honest, both 1.10 and 1.9 are pretty old version, it’s hard to fix and release a bugfix version for 1.10.1 even the community help troubleshoot your issue. So, could you try latest versions like Flink 1.16.1 or 1.17.0? Best, Leonard > On Mar 27, 2023, at 8:28 PM, Kiran Kumar

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Jing Ge via user
Congrats! Best regards, Jing On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu wrote: > Congratulations! > > > Best, > Leonard > > On Mar 27, 2023, at 5:23 PM, Yu Li wrote: > > Dear Flinkers, > > > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread yuxia
congratulations! Best regards, Yuxia 发件人: "Andrew Otto" 收件人: "Matthias Pohl" 抄送: "Jing Ge" , "Leonard Xu" , "Yu Li" , "dev" , "User" , "user-zh" 发送时间: 星期一, 2023年 3 月 27日 下午 8:57:50 主题: Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating) Exciting!

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Yanfei Lei
Congratulations! Best Regards, Yanfei ramkrishna vasudevan 于2023年3月27日周一 21:46写道: > > Congratulations !!! > > On Mon, Mar 27, 2023 at 2:54 PM Yu Li wrote: >> >> Dear Flinkers, >> >> >> As you may have noticed, we are pleased to announce that Flink Table Store >> has joined the Apache

Re: Handling JSON Serialization without Kryo

2023-03-27 Thread Andrew Otto
Hi, > The problem here is that the shape of the data can vary wildly and dynamically. Some records may have properties unique to only that record, which makes defining a POJO difficult AFAIK, the only way to avoid POJOs in Flink is to use Row (DataStream) or RowData (Table API). These are

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Andrew Otto
Exciting! If this ends up working well, Wikimedia Foundation would love to try it out! On Mon, Mar 27, 2023 at 8:39 AM Matthias Pohl via user < user@flink.apache.org> wrote: > Congratulations and good luck with pushing the project forward. > > On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user >

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread feng xiangyu
Congrats Yu! Looking forward to contributing to Paimon! Best Regards, Xiangyu yuxia 于2023年3月27日周一 21:01写道: > congratulations! > > Best regards, > Yuxia > > > 发件人: "Andrew Otto" > 收件人: "Matthias Pohl" > 抄送: "Jing Ge" , "Leonard Xu" , "Yu > Li" , "dev" , "User" < > user@flink.apache.org>,

Re: Flink CEP Resource Utilisation Optimisation

2023-03-27 Thread Abhishek Singla
Hi Geng, I tried the same experiment with version 1.16.0 and was able to achieve the same throughput with 3 nodes (24 parallelism) with CPU usage 60-70% Regarding the checkpoint size issue: Since there were no input events, the watermark was not advancing hence the events in CEP state were not

Re: Issue with the flink version 1.10.1

2023-03-27 Thread Shammon FY
Hi Kiran According to your description, it sounds like a configuration issue prevented the job from being submitted successfully. You can check whether there are any error messages from the submission log of yarn or your submission system. You can also compare the configuration differences

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Xianxun Ye
Congratulations! Best regards,Xianxun On 03/27/2023 22:51,Samrat Deb wrote: congratulationsBests,SamratOn Mon, Mar 27, 2023 at 7:19 PM Yanfei Lei wrote:

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Guanghui Zhang
Congratulations! Best, Zhang Guanghui Hang Ruan 于2023年3月28日周二 10:29写道: > Congratulations! > > Best, > Hang > > yu zelin 于2023年3月28日周二 10:27写道: > >> Congratulations! >> >> Best, >> Yu Zelin >> >> 2023年3月27日 17:23,Yu Li 写道: >> >> Dear Flinkers, >> >> >> >> As you may have noticed, we are

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread yu zelin
Congratulations! Best, Yu Zelin > 2023年3月27日 17:23,Yu Li 写道: > > Dear Flinkers, > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project called Apache > Paimon(incubating) [1] [2] [3]. The new project still aims

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Hang Ruan
Congratulations! Best, Hang yu zelin 于2023年3月28日周二 10:27写道: > Congratulations! > > Best, > Yu Zelin > > 2023年3月27日 17:23,Yu Li 写道: > > Dear Flinkers, > > > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Yun Tang
Congratulations! Unlike other data-lakes, Paimon might be the first one to act as a stream-first (not batch-first) data-lake. Best Yun Tang From: Xianxun Ye Sent: Tuesday, March 28, 2023 10:52 To: d...@flink.apache.org Cc: Yu Li ; user ; user-zh ;

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Biao Liu
Congrats! Thanks, Biao /'bɪ.aʊ/ On Tue, 28 Mar 2023 at 10:29, Hang Ruan wrote: > Congratulations! > > Best, > Hang > > yu zelin 于2023年3月28日周二 10:27写道: > >> Congratulations! >> >> Best, >> Yu Zelin >> >> 2023年3月27日 17:23,Yu Li 写道: >> >> Dear Flinkers, >> >> >> >> As you may have noticed, we

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Yuxin Tan
Congratulations! Best, Yuxin Guanghui Zhang 于2023年3月28日周二 11:06写道: > Congratulations! > > Best, > Zhang Guanghui > > Hang Ruan 于2023年3月28日周二 10:29写道: > > > Congratulations! > > > > Best, > > Hang > > > > yu zelin 于2023年3月28日周二 10:27写道: > > > >> Congratulations! > >> > >> Best, > >> Yu Zelin

Re: Custom source

2023-03-27 Thread Shammon FY
Hi Lasse Does your job table/sql or datastream? Here's the doc [1] for customized source in table and there is an example of socket source. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sourcessinks/ Best, Shammon FY On Mon, Mar 27, 2023 at 8:02 PM Lasse Nedergaard <

Re:Re: flink1.16 sql gateway hive2

2023-03-27 Thread guanyq
感谢两位老师 在 2023-03-27 09:58:24,"Shengkai Fang" 写道: >方勇老师说的没错。我们在文档里面也加了如何配置 hiveserver2 endpoint 的文档[1] > >[1] >https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/hive-compatibility/hiveserver2/#setting-up > >Shammon FY 于2023年3月27日周一 08:41写道: > >> Hi >> >> >>

[ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Yu Li
Dear Flinkers, As you may have noticed, we are pleased to announce that Flink Table Store has joined the Apache Incubator as a separate project called Apache Paimon(incubating) [1] [2] [3]. The new project still aims at building a streaming data lake platform for high-speed data ingestion,

Re: [ANNOUNCE] Apache Flink 1.17.0 released

2023-03-27 Thread Matthias Pohl
Here are a few things I noticed from the 1.17 release retrospectively which I want to share (other release managers might have a different view or might disagree): - Google Meet might not be the best choice for the release sync. We need to be able to invite attendees even if the creator of the

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Weihua Hu
Congratulations! Best, Weihua On Mon, Mar 27, 2023 at 9:02 PM yuxia wrote: > congratulations! > > Best regards, > Yuxia > > > 发件人: "Andrew Otto" > 收件人: "Matthias Pohl" > 抄送: "Jing Ge" , "Leonard Xu" , "Yu > Li" , "dev" , "User" < > u...@flink.apache.org>, "user-zh" > 发送时间: 星期一, 2023年 3 月

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Jing Ge
Congrats! Best regards, Jing On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu wrote: > Congratulations! > > > Best, > Leonard > > On Mar 27, 2023, at 5:23 PM, Yu Li wrote: > > Dear Flinkers, > > > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Leonard Xu
Congratulations! Best, Leonard > On Mar 27, 2023, at 5:23 PM, Yu Li wrote: > > Dear Flinkers, > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project called Apache > Paimon(incubating) [1] [2] [3]. The new

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Konstantin Knauf
Congrats! Am Mo., 27. März 2023 um 14:32 Uhr schrieb Leonard Xu : > Congratulations! > > > Best, > Leonard > > On Mar 27, 2023, at 5:23 PM, Yu Li wrote: > > Dear Flinkers, > > > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Matthias Pohl
Congratulations and good luck with pushing the project forward. On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user wrote: > Congrats! > > Best regards, > Jing > > On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu wrote: > >> Congratulations! >> >> >> Best, >> Leonard >> >> On Mar 27, 2023, at 5:23 PM,

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread yuxia
congratulations! Best regards, Yuxia 发件人: "Andrew Otto" 收件人: "Matthias Pohl" 抄送: "Jing Ge" , "Leonard Xu" , "Yu Li" , "dev" , "User" , "user-zh" 发送时间: 星期一, 2023年 3 月 27日 下午 8:57:50 主题: Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating) Exciting!

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Yanfei Lei
Congratulations! Best Regards, Yanfei ramkrishna vasudevan 于2023年3月27日周一 21:46写道: > > Congratulations !!! > > On Mon, Mar 27, 2023 at 2:54 PM Yu Li wrote: >> >> Dear Flinkers, >> >> >> As you may have noticed, we are pleased to announce that Flink Table Store >> has joined the Apache

flink sql upsert mysql问题

2023-03-27 Thread 小昌同学
你好,我这边使用flink sql实现四条流的关联,后续实现case when的逻辑,并且将数据插入到mysql,但是从结果数据来看,数据存在部分丢失,代码我粘贴再后面,麻烦各位老师指导,下面是sql【create function get_json_value as 'com.nesc.flink.udf.GetJsonValue'; set 'table.exec.sink.not-null-enforcer'='drop'; 测试环境 CREATE TABLE dm_cust_oact_prog_ri ( cust_id STRING COMMENT

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Hang Ruan
Congratulations! Best, Hang yu zelin 于2023年3月28日周二 10:27写道: > Congratulations! > > Best, > Yu Zelin > > 2023年3月27日 17:23,Yu Li 写道: > > Dear Flinkers, > > > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Biao Liu
Congrats! Thanks, Biao /'bɪ.aʊ/ On Tue, 28 Mar 2023 at 10:29, Hang Ruan wrote: > Congratulations! > > Best, > Hang > > yu zelin 于2023年3月28日周二 10:27写道: > >> Congratulations! >> >> Best, >> Yu Zelin >> >> 2023年3月27日 17:23,Yu Li 写道: >> >> Dear Flinkers, >> >> >> >> As you may have noticed, we

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Guanghui Zhang
Congratulations! Best, Zhang Guanghui Hang Ruan 于2023年3月28日周二 10:29写道: > Congratulations! > > Best, > Hang > > yu zelin 于2023年3月28日周二 10:27写道: > >> Congratulations! >> >> Best, >> Yu Zelin >> >> 2023年3月27日 17:23,Yu Li 写道: >> >> Dear Flinkers, >> >> >> >> As you may have noticed, we are

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread yu zelin
Congratulations! Best, Yu Zelin > 2023年3月27日 17:23,Yu Li 写道: > > Dear Flinkers, > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project called Apache > Paimon(incubating) [1] [2] [3]. The new project still aims

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Yun Tang
Congratulations! Unlike other data-lakes, Paimon might be the first one to act as a stream-first (not batch-first) data-lake. Best Yun Tang From: Xianxun Ye Sent: Tuesday, March 28, 2023 10:52 To: d...@flink.apache.org Cc: Yu Li ; user ; user-zh ;

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Yuxin Tan
Congratulations! Best, Yuxin Guanghui Zhang 于2023年3月28日周二 11:06写道: > Congratulations! > > Best, > Zhang Guanghui > > Hang Ruan 于2023年3月28日周二 10:29写道: > > > Congratulations! > > > > Best, > > Hang > > > > yu zelin 于2023年3月28日周二 10:27写道: > > > >> Congratulations! > >> > >> Best, > >> Yu Zelin