Re: Flink kinesis connector 4.3.0 release estimated date

Hey, Vararu The kinesis connector 4.3.0 release is under vote phase and we hope to finalize the release work in this week if everything goes well. Best, Leonard > 2024年5月22日 下午11:51，Vararu, Vadim 写道： > > Hi guys, > > Any idea when the 4.3.0 kinesis connector is estimated to be released?

Re: [ANNOUNCE] Apache Flink CDC 3.1.0 released

Congratulations ! Thanks Qingsheng for the great work and all contributors involved !! Best, Leonard > 2024年5月17日 下午5:32，Qingsheng Ren 写道： > > The Apache Flink community is very happy to announce the release of > Apache Flink CDC 3.1.0. > > Apache Flink CDC is a distributed data integration

Re: [ANNOUNCE] Apache Flink CDC 3.1.0 released

Congratulations ! Thanks Qingsheng for the great work and all contributors involved !! Best, Leonard > 2024年5月17日 下午5:32，Qingsheng Ren 写道： > > The Apache Flink community is very happy to announce the release of > Apache Flink CDC 3.1.0. > > Apache Flink CDC is a distributed data integration

[ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

Hi devs and users, We are thrilled to announce that the donation of Flink CDC as a sub-project of Apache Flink has completed. We invite you to explore the new resources available: - GitHub Repository: https://github.com/apache/flink-cdc - Flink CDC Documentation:

[ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

Hi devs and users, We are thrilled to announce that the donation of Flink CDC as a sub-project of Apache Flink has completed. We invite you to explore the new resources available: - GitHub Repository: https://github.com/apache/flink-cdc - Flink CDC Documentation:

Re: [ANNOUNCE] Apache Flink 1.19.0 released

Congratulations, thanks release managers and all involved for the great work! Best, Leonard > 2024年3月18日 下午4:32，Jingsong Li 写道： > > Congratulations! > > On Mon, Mar 18, 2024 at 4:30 PM Rui Fan <1996fan...@gmail.com> wrote: >> >> Congratulations, thanks for the great work! >> >> Best, >>

Re: [ANNOUNCE] Apache Flink 1.19.0 released

Congratulations, thanks release managers and all involved for the great work! Best, Leonard > 2024年3月18日 下午4:32，Jingsong Li 写道： > > Congratulations! > > On Mon, Mar 18, 2024 at 4:30 PM Rui Fan <1996fan...@gmail.com> wrote: >> >> Congratulations, thanks for the great work! >> >> Best, >>

Re: 退订

可以发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org 取消订阅来自 user-zh@flink.apache.org 邮件列表的邮件，邮件列表的订阅管理，可以参考[1] 祝好， [1] https://flink.apache.org/zh/what-is-flink/community/ > 2024年2月20日 下午4:36，任香帅 写道： > > 退订

Re: [ANNOUNCE] Apache Flink 1.18.1 released

ors of the Apache Flink community who > made this release possible! Special thanks to @Qingsheng Ren @Leonard Xu > @Xintong Song @Matthias Pohl @Martijn Visser for the support during this > release. > > A Jira task series based on the Flink release wiki has been created for >

Re: [ANNOUNCE] Apache Flink 1.18.1 released

ors of the Apache Flink community who > made this release possible! Special thanks to @Qingsheng Ren @Leonard Xu > @Xintong Song @Matthias Pohl @Martijn Visser for the support during this > release. > > A Jira task series based on the Flink release wiki has been created for >

Re: [ANNOUNCE] Apache Flink 1.17.2 released

Thanks Yun for driving the release. Thanks a lot to everyone that has contributed with bug fixes and other improvements! Best, Leonard > 2023年11月29日 下午1:05，Yun Tang 写道： > > The Apache Flink community is very happy to announce the release of Apache > Flink 1.17.2, which is the second

Re: [ANNOUNCE] Apache Flink 1.17.2 released

Thanks Yun for driving the release. Thanks a lot to everyone that has contributed with bug fixes and other improvements! Best, Leonard > 2023年11月29日 下午1:05，Yun Tang 写道： > > The Apache Flink community is very happy to announce the release of Apache > Flink 1.17.2, which is the second

Re: dependency error with latest Kafka connector

> built a fat uber jar from quickstart with Flink 1.18.0 for > flink-streaming-java and flink-clients, and flink-connector-kafka version > 3.0.1-1.18 > then submitted to local Flink cluster 1.18.0. Things worked as expected and > the job ran fine. Hey，@Gordan I guess things may work as

Re: dependency error with latest Kafka connector

Hi, Gurnterh It seems a bug for me that 3.0.1-1.18 flink Kafka connector use flink 1.17 dependency which lead to your issue. I guess we need propose a new release for Kafka connector for fix this issue. CC: Gordan, Danny, Martijn Best, Leonard > 2023年11月14日 下午6:53，Alexey Novakov via user

Re: [ANNOUNCE] Apache Flink 1.18.0 released

Congratulations, Well done! Best, Leonard On Fri, Oct 27, 2023 at 12:23 AM Lincoln Lee wrote: > Thanks for the great work! Congrats all! > > Best, > Lincoln Lee > > > Jing Ge 于2023年10月27日周五 00:16写道： > > > The Apache Flink community is very happy to announce the release of > Apache > > Flink

Re: [ANNOUNCE] Apache Flink 1.18.0 released

Congratulations, Well done! Best, Leonard On Fri, Oct 27, 2023 at 12:23 AM Lincoln Lee wrote: > Thanks for the great work! Congrats all! > > Best, > Lincoln Lee > > > Jing Ge 于2023年10月27日周五 00:16写道： > > > The Apache Flink community is very happy to announce the release of > Apache > > Flink

Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

Congrats to all ! It will be helpful to promote Apache Flink if we can add a page to our website like others[2]. I’ve created an issue to improve this. Best, Leonard [1] https://issues.apache.org/jira/browse/FLINK-32555 [2] https://spark.apache.org/news/sigmod-system-award.html

Re: Unsubscribe

Please send email to user-unsubscr...@flink.apache.org if you want to unsubscribe the mail from user@flink.apache.org, and you can refer [1][2] for more details. 请发送任意内容的邮件到 user-unsubscr...@flink.apache.org 地址来取消订阅来自 user@flink.apache.org 邮件组的邮件，你可以参考[1][2] 管理你的邮件订阅。 Best, Leonard [1]

[ANNOUNCE] Apache flink-connector-pulsar v3.0.1 released

The Apache Flink community is very happy to announce the release of Apache flink-connector-pulsar v3.0.1. This release is compatible with Flink 1.16.x series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data

Re: 提问

(1)可以检查下是不是其他作业或者同步工具使用了对应的server-id (2) server-id 可以尝试用机器IP+时间戳来生成，这样能尽可能避免冲突 祝好， 雪尽 > On May 22, 2023, at 3:34 PM, 曹明勤 wrote: > >

Re: checkpoint Kafka Offset commit failed

可以发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org 取消订阅来自 user-zh@flink.apache.org 邮件列表的邮件，邮件列表的订阅管理，可以参考[1] 祝好， Leonard [1] https://flink.apache.org/zh/community/#%e9%82%ae%e4%bb%b6%e5%88%97%e8%a1%a8 > 2023年5月4日 下午9:00，wuzhongxiu 写道： > > 退订 > > > > | | > go574...@163.com > | > | >

Re: 退订

如果需要取消订阅 user-zh@flink.apache.org 邮件组，请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org ,参考[1] [1] https://flink.apache.org/zh/community/ > 2023年4月21日 上午10:52，琴师 <1129656...@qq.com.INVALID> 写道： > > 退订 > > > 琴师 > 1129656...@qq.com > > > >

Re: 取消订阅

可以发送任意内容的邮件到 user-unsubscr...@flink.apache.org 取消订阅来自 user@flink.apache.org 邮件列表的邮件，其他邮件列表的订阅和退订管理也类似，邮件列表的订阅管理，可以参考[1] 祝好， Leonard Xu [1] https://flink.apache.org/community.html#how-to-subscribe-to-a-mailing-list > On Apr 18, 2023, at 2:53 PM, wangw...@sina.cn wrote: > > 取消订阅

Re: Issue with the flink version 1.10.1

Hi, Kiran To be honest, both 1.10 and 1.9 are pretty old version, it’s hard to fix and release a bugfix version for 1.10.1 even the community help troubleshoot your issue. So, could you try latest versions like Flink 1.16.1 or 1.17.0? Best, Leonard > On Mar 27, 2023, at 8:28 PM, Kiran Kumar

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

Congratulations! Best, Leonard > On Mar 27, 2023, at 5:23 PM, Yu Li wrote: > > Dear Flinkers, > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project called Apache > Paimon(incubating) [1] [2] [3]. The new

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

Congratulations! Best, Leonard > On Mar 27, 2023, at 5:23 PM, Yu Li wrote: > > Dear Flinkers, > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the Apache Incubator as a separate project called Apache > Paimon(incubating) [1] [2] [3]. The new

Re: 项目中引入 flink-sql-connector-oracle-cdc-2.3.0.jar 后启动报解析配置异常

flink-sql-connector-xx 都是uber jar, 不应该在项目中直接uber jar，你在项目中应该引入 flink-connector-xx 依赖并自己管理。 Best, Leonard > On Mar 25, 2023, at 3:25 PM, casel.chen wrote: > > 项目中引入 flink-sql-connector-oracle-cdc-2.3.0.jar > 后启动过程中报如下异常，查了一下该jar下有oracle.xml.jaxp.JXDocumentBuilderFactory类，有什么办法解决么？ > > >

[ANNOUNCE] Apache Flink 1.17.0 released

The Apache Flink community is very happy to announce the release of Apache Flink 1.17.0, which is the first release for the Apache Flink 1.17 series. Apache Flink® is an open-source unified stream and batch data processing framework for distributed, high-performing, always-available, and

[ANNOUNCE] Apache Flink 1.17.0 released

The Apache Flink community is very happy to announce the release of Apache Flink 1.17.0, which is the first release for the Apache Flink 1.17 series. Apache Flink® is an open-source unified stream and batch data processing framework for distributed, high-performing, always-available, and

Re: IntervalJoin invisibly becomes a regular Join - why?

> > CREATE TEMPORARY VIEW filteredResults AS > SELECT * from suspiciousOrders WHERE small_ts > large_ts; Looks like after added the condition, the final expanded query should not match the condition[1] of an interval join that leads to the planner recognize it as an interval join. It’s not

Re: is there any detrimental side-effect if i set the max parallelismas 32768

> > 退订 请发送任意邮件到 user-unsubscr...@flink.apache.org 取消 订阅来自 user@flink.apache.org 邮件列表的邮件，发送到 user@flink.apache.org 是不会取消订阅的。 > 发自我的iPhone > > > -- Original -- > From: Tony Wei > Date: Tue,Mar 14,2023 1:11 PM > To: David Anderson > Cc: Hangxiang Yu , user >

[SUMMARY] Flink 1.17 Release Sync 3/7/2023

Hi devs and users, I'd like to share some highlights from Flink 1.17 release sync on 3/7/2023. 1.17 Blockers: - Currently, there is one blocker issue (FLINK-31351[1]) that needs to be resolved before we can create a votable RC1. Our contributors are working hard to fix it as soon as

Re: 退订

Please send an email to user-unsubscr...@flink.apache.org to unsubscribe > On Mar 3, 2023, at 8:42 AM, zhangjunjie wrote: > > 退订 > >

Re: [ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 released

Thanks Yanfei for driving the release ! ! Best, Leonard > On Jan 31, 2023, at 3:43 PM, Yun Tang wrote: > > Thanks Yuanfei for driving the frocksdb release! > > Best > Yun Tang > From: Yuan Mei > Sent: Tuesday, January 31, 2023 15:09 > To: Jing Ge > Cc: Yanfei Lei ; d...@flink.apache.org >

Re: [ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 released

Thanks Yanfei for driving the release ! ! Best, Leonard > On Jan 31, 2023, at 3:43 PM, Yun Tang wrote: > > Thanks Yuanfei for driving the frocksdb release! > > Best > Yun Tang > From: Yuan Mei > Sent: Tuesday, January 31, 2023 15:09 > To: Jing Ge > Cc: Yanfei Lei ; d...@flink.apache.org >

[SUMMARY] Flink 1.17 Release Sync 12/13/2022

Hi devs and users, I’d like to share the highlights about the 1.17 release sync on 12/13/2022. - Release tracking page: - 1.17 development is moving forward [1], we have 5 weeks remaining - @committers Please continuously update the the progress in the 1.17 page - Externalized

Re: FlinkCDC可以读到MySQL变更数据，但是插不到新的MySQL表里

> On Nov 4, 2022, at 2:34 PM, 左岩 <13520871...@163.com> wrote: > > tenv.executeSql("xxx); > env.execute(); 这样使用是不对的，你可以看下这两个方法的java doc 祝好， Leonard

Re: flink sql接cdc数据源按最新数据统计问题

> On Nov 29, 2022, at 8:32 AM, casel.chen wrote: > > 业务需求是mysql订单表按天按供应商实时统计交易金额，订单表会发生修改和删除，用flink > sql要如何实现呢？开窗取最新一条记录再聚合吗？如果遇到delete记录会不会减去相应的price呢？试着写了如下flink sql不知道对不对 会的，可以看下flink sql相关的原理文章，百度/谷歌一搜一大把。 祝好 Leonard > > > select > s.biddate, > s.supplier, > sum(s.price) >

Re: debezium-json数据timestamp类型时区问题

你在Oracle 数据库中的数据类型是TIMESTAMP 还是 TIMESTAMP WITH LOCAL TIME ZONE？ 我猜是后者，如果是后者直接在Flink SQL 里TIMESTAMP_LTZ 类型去映射就可以了 Oracle 的TIMESTAMP LTZ 类型和Flink SQL的TIMESTAMP LTZ类型含义和存储都是一致的语义，即epoch mills，存储时不需要时区。这两个类型都是在各自的系统中在在需要查看这些数据时，需要用 session 时区从epoch mills 转换成可读timestamp格式的字符串。 Oracle 设置session

Re: Weird Flink SQL error

Do not trust the line number from sql parser exception, you should use ROW in your DDL when you declare a composite row type, try the following: CREATE TABLE test_content_metrics ( dt STRING NOT NULL, `body` ROW< `platform_id` BIGINT, `content_id` STRING > ) PARTITIONED BY

[SUMMARY] Flink 1.17 Release Sync 11/15/2022

Hi devs and users, I’d like to share some highlights about the 1.17 release sync on 11/15/2022. - Release tracking page: - The community has collected some great features on the 1.17 page[1] - @committers Please continuously update the page in the coming week - JIRA account apply

Re: [ACCOUNCE] Apache Flink Elasticsearch Connector 3.0.0 released

Thanks Chesnay and Martijn for the great work! I believe the flink-connector-shared-utils[1] you built will help Flink connector developers a lot. Best, Leonard [1] https://github.com/apache/flink-connector-shared-utils > 2022年11月10日 下午9:53，Martijn Visser 写道： > > Really happy with the

Re: [ACCOUNCE] Apache Flink Elasticsearch Connector 3.0.0 released

Thanks Chesnay and Martijn for the great work! I believe the flink-connector-shared-utils[1] you built will help Flink connector developers a lot. Best, Leonard [1] https://github.com/apache/flink-connector-shared-utils > 2022年11月10日 下午9:53，Martijn Visser 写道： > > Really happy with the

Re: UDFs classloading changes in 1.16

Thanks Alexander for reporting this issue, Could you open a jira ticket as well? CC: Shengkai, please take a look this ticket, looks like an incompatibility change. Best, Leonard > 2022年11月4日 下午6:15，Alexander Fedulov 写道： > > Hi everyone, > > 1.16 introduced quite a lot of changes with

Re: flinkcdc 读不到mysql中数据

Flink CDC 社区有提供1.14支持的，2.2.1版本即可。你这个好像是没有开启checkpoint, 开启下就好了。 // enable checkpoint env.enableCheckpointing(1000); 祝好， Leonard > 2022年11月3日 上午11:34，左岩 <13520871...@163.com> 写道： > > 我用的是flink1.14 > ，因为官方没有匹配的版本，所以自己编译的flinkCDC，binlog也开启了，然后也没报错，读不到mysql的数据，idea控制台不报错也不输出数据，可能是什么原因呢（运行日志见附件） >

Re: Flink CDC 打宽表

是的，如果是双流join打宽，ttl设置得过短，state里的历史数据被清理掉了，后续的更新数据进入join节点关联不上就可能会下发nul 祝好， Leonard > 2022年11月2日 上午11:49，Fei Han 写道： > > 大家好！关于 Flink CDC 打宽表有如下疑问： > 启动一个任务后，刚开始个字段是有值的。但跑一段时间或者跨天后，字段无缘无故是null值了。用其他引擎跑数据却是正常的。 > 比如第一天启动任务，A字段是有值的。但是第二天发现A字段全部是NULL值了。但用presto查询却是正常的。但我猜测是不是和TTL设置有关系呢？我设置了1天。

Re: Flink CDC2.2.1 设置server id范围

l.chen 于2022年10月31日周一 16:50写道： > >> >> >> >> >> server-id配置范围对于后面修改并发度是不是不太友好？每改一次并发度就得重新调整server-id范围么？还是说先配置一个较大的server-id范围，在在这个较大的范围内调整并发度？ >> >> >> >> >> >> >> >> >> >> >> >> 在

Re: Flink CDC2.2.1 设置server id范围

> server-id配置范围对于后面修改并发度是不是不太友好？每改一次并发度就得重新调整server-id范围么？还是说先配置一个较大的server-id范围，在在这个较大的范围内调整并发度？ 作业起来后修改并发是需要调整的，建议这块可以放到平台里去设计，这样可以让写sql的用户知道with参数里参数的作用。 祝好， Leonard > > > > > > > > > > > 在 2022-10-31 16:04:32，"Leonard Xu" 写道： >

Re: flink sql client取消sql-clients-default.yaml后那些预置catalogs建议在哪里定义呢？

Hi, 我记得有个-i 参数可以指定初始化sql文件，你贴你的初始化sql在文件里加进去就可以了。 祝好， Leonard > 2022年10月31日 下午4:52，casel.chen 写道： > > flink新版本已经找不到sql-clients-default.yaml文件了，那么之前配置的那些预置catalogs建议在哪里定义呢？通过初始化sql么？

Re: OutOfMemoryError (java heap space) on small, local test

Hi, Matt I’ve checked your job is pretty simple, I've CC Xingbo who is a PyFlink expert to help take a quick look. Best， Leonard > 2022年10月31日 上午11:47，Matt Fysh 写道： > > Hi there, > > I am running a local test with: > * source = env.from_collection > * sink = datastream.execute_and_collect

Re: Flink CDC2.2.1 设置server id范围

Hi, 你5张表对应的source并发是多少呀？如果是多并发需要把server-id设置成一个范围，范围和并发数匹配，比如4个并发，应该类似’1101-1104’. 另外 server-id 是全局唯一的，你需要确保下你使用的server-id 和其他作业、其他同步工具都不冲突才可以。 Best, Leonard > 2022年10月31日 下午4:00，Fei Han 写道： > > 大家好！ > 现在我在 Flink CDC2.2.1设置了server id。有5张表且server id的范围都不同，通过Flink CDC >

Re: Performing left join between two streams

Hi, Lalwani Flink does not support outer join on two data streams now[1], you can use the DataStream API ds1.coGroup(ds2) as a workaround. Flink SQL support outer joins well, you can also try SQL way[2]. Best, Leonard [1] https://issues.apache.org/jira/browse/FLINK-4187 [2]

Re: Flink SQL 问题请教

你好，你的Flink 版本多少呀？我记得低版本才有这个问题。 另外SQL可以贴下嘛？ 祝好， Leonard > 2022年10月22日 上午11:11，邮件帮助中心 写道： > > 大家好！ >最近在开发一个项目时，在使用CDC表和维表表做Temporal Table > JOIN时，发现2个表Join时join字段的类型必须一致，否则提交时提示如下的错误 >The main method caused an error: Temporal table join requires an equality > condition on fields of

Re: [DISCUSS] Reverting sink metric name changes made in 1.15

Thanks Qingsheng for starting this thread. +1 on reverting sink metric name and releasing 1.15.3 to fix this inconsistent behavior. Best, Leonard > 2022年10月10日 下午3:06，Jark Wu 写道： > > Thanks for discovering this problem, Qingsheng! > > I'm also +1 for reverting the breaking changes. >

Re: 看了官方文档的Versioned Table，有一些小疑惑希望可以得到解答

> 2022年8月8日 下午3:34，林影 写道： > > 先上链接， Versioned Table > > 从文档描述中可知，以Upsert-Kafka作为Source，以debezium或canal作为format时，可被认为是Versioned > Table Source。 > > 1. 那么flink

Re: accuracy validation of streaming pipeline

Hi, vtygoss > I'm working on migrating from full-data-pipeline(with spark) to > incremental-data-pipeline(with flink cdc), and i met a problem about accuracy > validation between pipeline based flink and spark. Glad to hear that ! > For bounded data, it's simple to validate the two result

Re: table.local-time-zone not working

ng property as follows: > > Configuration tconf = tenv.getConfig().getConfiguration(); > tconf.setString("table.local-time-zone", "America/Los_Angeles"); > > tenv.getConfig().setLocalTimeZone(ZoneId.of("America/Los_Angeles")); &g

Re: table.local-time-zone not working

Hi, Lalwani Could you share how you set this property and your Flink version? Best, Leonard > 2022年5月10日 下午3:01，Surendra Lalwani 写道： > > Hi Team, > > I have tried using this property > table.local-time-zone > > But it seems like it is not making any impact, on calling functions like >

Re: Flink-SQL returning duplicate rows for some records

Hi Joost Could you share your flink version and the two records in debezium-json format which produced by two MS SQL UPDATE statement ? Best, Leonard > 2022年5月2日 下午9:59，Joost Molenaar 写道： > > Hello all, > > I'm trying to use Flink-SQL to monitor a Kafka topic that's populated by > Debezium,

Re: flink table store

项目是开源的[1], 最近快要发布第一个版本了，可以关注下 Best, Leonard [1] https://github.com/apache/flink-table-store > 2022年4月7日 上午9:54，Xianxun Ye 写道： > > 这里有 flink table store 的设计文档，你可以了解下。 >

Re: [ANNOUNCE] Apache Flink 1.1.4.4 released

Thanks a lot for being our release manager Konstantin and everyone who involved! Best, Leonard > 2022年3月15日 下午9:34，Martijn Visser 写道： > > Thank you Konstantin and everyone who contributed!

Re: [ANNOUNCE] Apache Flink 1.1.4.4 released

Thanks a lot for being our release manager Konstantin and everyone who involved! Best, Leonard > 2022年3月15日 下午9:34，Martijn Visser 写道： > > Thank you Konstantin and everyone who contributed!

Re: flinkcdc:slave with the same server_uuid/server_id as this slave has connected to the master;

Please see the FAQ document [1] Best, Leonard [1] https://github.com/ververica/flink-cdc-connectors/wiki/FAQ(ZH)#q10-%E4%BD%9C%E4%B8%9A%E6%8A%A5%E9%94%99-connectexception-a-slave-with-the-same-server_uuidserver_id-as-this-slave-has-connected-to-the-master%E6%80%8E%E4%B9%88%E5%8A%9E%E5%91%A2 >

Re: [Table API] [JDBC CDC] Caching Configuration from MySql instance needed in multiple Flink Jobs

Hello, Dan > 2022年2月21日 下午9:11，Dan Serb 写道： > 1.Have a processor that uses Flink JDBC CDC Connector over the table that > stores the information I need. (This is implemented currently - working) You mean you’ve implemented a Flink JDBC Connector？ Maybe the Flink CDC Connectors[1] would help

Re: CDC using Query

Hello, mohan > 1. Does flink have any support to track any missed source Jdbc CDC records ? Flink CDC Connector provides Exactly once semantics which means they won’t miss records. Tips: The Flink JDBC Connector only Scan the database once which can not continuously read CDC stream. > 2.

Re: [ANNOUNCE] Apache Flink 1.14.2 / 1.13.5 / 1.12.7 / 1.11.6 released

I guess this is related to publishers everywhere are updating their artifacts in response to the log4shell vulnerability[1]. All we can do and need to do is wait. ☕️ Best, Leonard [1] https://issues.sonatype.org/browse/OSSRH-76300 >

Re: flink结合历史数据怎么处理

MySQL CDC connector 支持并发读取的，读取过程也不会用锁，600万的数据量很小了，百亿级的分库分表我们和社区用户测试下都是ok的，你可以自己试试。 祝好， Leonard > 2021年12月6日 下午3:54，张阳 <705503...@qq.com.INVALID> 写道： > > 因为数据量有600w 所以担心初始化时间太长 或者性能问题 > > > > > --原始邮件-- > 发件人:

Re: flink hang : es_rejected_execution_exception导致的flink 假死问题

Hi, ren I think the root cause is you didn’t set proper FailureHandler for ElasticSearch connector, the `RetryRejectedExecutionFailureHandler` can resolve your issue, you can see ElasticSearch connector docs[1] for more information. You can also set 'connector.failure-handler to

Re: flink结合历史数据怎么处理

如果你的数据源是 数据库，你可以尝试下 Flink CDC Connectors[1], 这些Connector 就是 hybrid source， 先读历史全量数据，再读增量数据， 历史和增量阶段是无缝衔接的。 祝好， Leonard [1] https://ververica.github.io/flink-cdc-connectors/release-2.1/content/connectors/mysql-cdc.html > 2021年12月2日 下午2:40，张阳 写道： > > 统计的指标有大量的历史数据，怎么把历史的数据和今天的实时数据进行汇总呢。

Re: 退订

你好，取消订阅是发送到 user-zh-unsubscr...@flink.apache.org , 参考 https://flink.apache.org/zh/community.html#section 祝好 > 在 2021年11月24日，14:33，Gauler Tan 写道： > > 你好，已经发了很多次退订了，发啥还在源源不断的给我发邮件？ > > 谢谢

Re: FlinkSQL ES7连接器无法使用

这是个依赖问题，你检查下你环境中是否只使用sql connector 的jar，即 flink-sql-connector-elasticsearch7， 如果不是 datastream 作业是不需要 flink-connector-elasticsearch7 这个 jar包的。如果不是这个问题，你可以分析下你作业里使用的 es 相关依赖，可以参考异常栈确定类再去确定jar包，看下是不是多加了一些无用的jar。 祝好， Leonard > 在 2021年11月22日，12:30，mispower 写道： > > 你好，咨询一下后续你这个问题是如何解决的？ > > >

Re: 退订

如果需要取消订阅 user-zh@flink.apache.org 邮件组，请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org > 在 2021年11月2日，14:15，李芳奎 写道： > > 退订 > > felix > > felix_...@163.com

Re: flinksql 读取orc文件失败

图挂了，试着直接贴下日志文本，或者用图床工具。 > 在 2021年10月28日，16:54，陈卓宇 <2572805...@qq.com.INVALID> 写道： > > > > > > flink版本:1.12.2 > jdk:1.8 > 场景:flinksql 读取hdfs的orc文件 > 请教大神这个报错是什么原因 > > > 宇 >

Re: 一些关于flink rabbitmq connector的疑问

Hi, Peng There’s no doubt that RabbitMQ is a good open source community with active users. I understand what @renqschn means is that Flink RabbitMQ Connector is one connector with few users among the many connectors in the Flink project. From my observation, the connector that is used more

Re: window join in flink sql

Tips: The documentation in https://ci.apache.org/projects/flink is no longer updated any more, The new documentation site is https://nightlies.apache.org/flink/ , please use the new one. Best, Leonard > 在

Re: database as stream source issue

Hi, Qihua JDBC connector support Postgres dailect, but it is implemented as a bounded source which means it only captures the snapshot data(the existed records) and then finished its work, the new adding transaction log records (as known as MySql bin-log) won’t be captured. You should receive

Re: [ANNOUNCE] Apache Flink 1.13.3 released

Thanks to Chesnay & Martijn and everyone who made this release happen. > 在 2021年10月21日，20:08，Martijn Visser 写道： > > Thank you Chesnay, Leonard and all contributors! > > On Thu, 21 Oct 2021 at 13:40, Jingsong Li > wrote: > Thanks, Chesnay & Martijn > > 1.13.3

Re: [DISCUSS] Creating an external connector repository

Hi, all I understand very well that the maintainers of the community want to move the connector to an external system. Indeed, the development and maintenance of the connector requires a lot of energy, and these do not involve the Flink core framework, which can reduce the maintenance pressure

Re: 退订

如果需要取消订阅 user-zh@flink.apache.org 邮件组，请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org 即可 > 在 2021年9月27日，14:43，rzy1107 写道： > > 退订

Re: GroupWindowAggregate doesn't support consuming update and delete changes which is produced by node Deduplicate

hi, 报错详情可以在邮件贴下吗？ > 在 2021年9月27日，11:36，lzy139...@outlook.com 写道： > > 使用ROW_NUMBER过滤数据后，进行开窗聚合计算报错

Re: mysql cdc入hudi湖出错

Hi, chan 完整的日志能贴下吗？这个日志还看不出来。 > 在 2021年9月24日，18:23，casel.chen 写道： > > SELECT `id`, `name`, `birthday`, `ts`, DATE_FORMAT(`birthday`, 'MMdd') AS > `partition` FROM mysql_users;

Re: 退订

如果需要取消订阅 user-zh@flink.apache.org 邮件组，请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org 即可 Best, Leonard > 在 2021年9月26日，14:25，luoye <13033709...@163.com> 写道： > > 退订

Re: Can't access Debezium metadata fields in Kafka table

Hi, Harshvardhan The format debezium-avro-confluent doesn’t support read metadata yet[1], the supported formats including debezium-json, canal-json and maxwell-json, you can try the supported formats. Best, Leonard [1] https://issues.apache.org/jira/browse/FLINK-20454

Re: flink-1.12.0 ddl设置watermark error，但是1.13.2没有报错

这是个已知bug[1]， 在1.13.0 和 1.12.3上都修复了， 你可以用下flink 1.12.5 或 1.13.2的小版本 [1]https://issues.apache.org/jira/browse/FLINK-22082 祝好 > 在 2021年9月25日，21:29，kcz <573693...@qq.com.INVALID> 写道： > > SQL定义如下，当1.12.0将watermark语句移除之后，就不报错了。 > CREATE TABLE KafkaTable ( > test array gatherTime STRING, > log_ts

Re: flink cdc SQL2ES，GC overhead limit exceeded

应该和Flink CDC无关，CDC只是source，这个异常栈是从join节点抛出来的，你可以贴下你的SQL和配置 这样大家才好分析一点 Best, Leonard > 在 2021年9月15日，15:01，wukon...@foxmail.com 写道： > > hi LIYUAN: > 请描述下如何使用的flink，以及什么场景下 会造成这个报错， 这样方便大家帮助你定位问题。 > > > > wukon...@foxmail.com > > 发件人： LI YUAN > 发送时间： 2021-09-09 20:38 > 收件人： user-zh > 主题：

Re: Streaming SQL support for redis streaming connector

Hi, Osada Just want to offer some material here.The flink-cdc-connectors project [1] maybe also help you, we supports the document db MongoDB[2] recently. Best, Leonard [1] https://github.com/ververica/flink-cdc-connectors [2]

Re: flink-connector-postgres-cdc：no changes will be captured 无数据捕获到

gt; > > BR > Fisher > > > On Mon, Sep 6, 2021 at 5:55 PM Leonard Xu <mailto:xbjt...@gmail.com>> wrote: > Hello, Fisher > > 图挂了，可以用图床工具贴下吗？ > 我可以帮忙看看 > > 祝好， > Leonard > > > 在 2021年9月6日，17:48，Fisher Xiang > <mailto:fisherxia...@gmail

Re: flink-connector-postgres-cdc：no changes will be captured 无数据捕获到

Hello, Fisher 图挂了，可以用图床工具贴下吗？ 我可以帮忙看看 祝好， Leonard > 在 2021年9月6日，17:48，Fisher Xiang 写道： > > hi, > > 在使用 flink-connector-postgres-cdc时（版本从1.1.1 ~ 1.4.0都试过）， 出现一个警告： > WARN io.debezium.relational.RelationalDatabaseSchema - After applying the > include/exclude list filters, no changes will be

[ANNOUNCE] Flink mailing lists archive service has migrated to Apache Archive service

Hi, all The mailing list archive service Nabble Archive was broken at the end of June, the Flink community has migrated the mailing lists archives[1] to Apache Archive service by commit[2], you can refer [3] to know more mailing lists archives of Flink. Apache Archive service is maintained by

[ANNOUNCE] Flink mailing lists archive service has migrated to Apache Archive service

Hi, all The mailing list archive service Nabble Archive was broken at the end of June, the Flink community has migrated the mailing lists archives[1] to Apache Archive service by commit[2], you can refer [3] to know more mailing lists archives of Flink. Apache Archive service is maintained by

Re: 退订

Hi, Please send email to dev-unsubscr...@flink.apache.org if you want to unsubscribe the mail from d...@flink.apache.org . Please send email to user-unsubscr...@flink.apache.org

Re: 退订

Hi, Please send email to dev-unsubscr...@flink.apache.org if you want to unsubscribe the mail from d...@flink.apache.org . Please send email to user-unsubscr...@flink.apache.org

Re: 【Announce】Zeppelin 0.10.0 is released, Flink on Zeppelin Improved

Thanks Jeff for the great work ! Best, Leonard > 在 2021年8月25日，22:48，Jeff Zhang 写道： > > Hi Flink users, > > We (Zeppelin community) are very excited to announce Zeppelin 0.10.0 is > officially released. In this version, we made several improvements on Flink > interpreter. Here's the main

Re: 【Announce】Zeppelin 0.10.0 is released, Flink on Zeppelin Improved

Thanks Jeff for the great work ! Best, Leonard > 在 2021年8月25日，22:48，Jeff Zhang 写道： > > Hi Flink users, > > We (Zeppelin community) are very excited to announce Zeppelin 0.10.0 is > officially released. In this version, we made several improvements on Flink > interpreter. Here's the main

Re: mini-batch 设置后没效果

> 如何退订这个邮件订阅了 如果需要取消订阅 user-zh@flink.apache.org 邮件组，请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org 即可 Best, Leonard

Re: Flink SQL Api不支持TIMESTAMP(p) WITH TIME ZONE 类型的列

Hello, Flink 还不支持 TIMESTAMP WITH TIME ZONE 类型， 目前支持的有： TIMESTAMP WITHOUT TIME ZONE, 缩写为 TIMESTAMP TIMESTAMP WITH LOCAL TIME ZONE，缩写为TIMESTAMP_LTZ 祝好， Leonard > 在 2021年8月19日，20:51，changfeng 写道： > > ` TIMESTAMP(6) WITH TIME ZONE

Re: 请教下Flink时间戳问题

Hi, 你贴的图都挂了，需要传图可以用下图床工具，代码少可以直接贴代码。 TIMESTAMP 类型中 显示的T 没有任何含义，只是 format 一个时间戳时的一个分割符，你最终把 TIMESTAMP 写入到你的sink，你自己的sink（比如mysql）会有其自己的format。 第二个问题，看不到你的图，你看下你flink的版本，1.13后这个TIMESTAMP_LTZ类型支持才完善的。 祝好， Leonard > 在 2021年8月16日，10:27，Geoff nie 写道： > > 问题一：flink timestamp时间戳为何中间多了个T，怎么才能少去中间T呢？

Re: Flink SQL向下兼容吗？

这里的SQL是指DDL还是DML，通常 DML都是兼容的，且一般不会有不兼容的升级， DDL 语法 各家 SQL 方言都有自己的语法，这个比较灵活，FLINK SQL 的DDL 各个版本稍有不同，但 Flink SQL 新版本都是兼容老的 DDL的， 只是新版本上的DDL语法如果提供了更丰富的功能，那么老版本的DDL则不能提供 。 所以我理解你关心的兼容性问题是不存在的，但请注意如果你的SQL作业是有状态的，需要带状态升级，这些状态都是跨版本不兼容的。 祝好， Leonard > 在 2021年8月10日，11:44，Jason Lee 写道： > > 各位大佬好， > >

Re: 退订

如果需要取消订阅 user-zh@flink.apache.org 邮件组，请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org Best, Leonard > 在 2021年8月6日，10:49，汪嘉富 写道： > > 退订 >

Re: 退订

如果需要取消订阅 user-zh@flink.apache.org 邮件组，请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org Best, Leonard > 在 2021年8月11日，08:16，Lee2097 写道： > > 退订

  1   2   3   4   5   6   >