Re: [VOTE] Release 1.8.0, release candidate #3

2019-03-20 Thread Yu Li
-1, observed stably failure on streaming bucketing end-to-end test case in two different environments (Linux/MacOS) when running with both shaded hadoop-2.8.3 jar file

Re: [VOTE] Release 1.8.0, release candidate #3

2019-03-21 Thread Yu Li
ng sun wrote: > Thanks for the quick fix, Yu. the PR of FLINK-11972 > <https://issues.apache.org/jira/browse/FLINK-11972> has been merged. > > Cheers, > Jincheng > > Yu Li 于2019年3月21日周四 上午7:23写道: > >> -1, observed stably failure on streaming bucketing end-to-end

Re: End to End Performance Testing

2019-04-03 Thread Yu Li
A short answer is no, the test platform is not included in the blink branch. FWIW (to make it clear, I'm not the decision maker so just some of my own opinions), the test platform is for production usage and includes simulation of online jobs and probably some confidential stuff, so I don't think

Re: RocksDB local snapshot sliently disappears and cause checkpoint to fail

2019-03-28 Thread Yu Li
Hi Paul, Regarding "mistakenly uses the default filesystem scheme, which is specified to hdfs in the new cluster in my case", could you further clarify the configuration property and value you're using? Do you mean you're using an HDFS directory to store the local snapshot data? Thanks. Best

Re: RocksDB local snapshot sliently disappears and cause checkpoint to fail

2019-03-28 Thread Yu Li
cal snapshot still goes > to YARN local cache dirs. > > Hope that answers your question. > > Best, > Paul Lam > > 在 2019年3月28日,15:34,Yu Li 写道: > > Hi Paul, > > Regarding "mistakenly uses the default filesystem scheme, which is > specified to

Re: [VOTE] Release 1.8.0, release candidate #3

2019-03-21 Thread Yu Li
mment? > > I will also cancel this RC because of various issues. > > Best, > Aljoscha > > On 21. Mar 2019, at 12:23, Yu Li wrote: > > Thanks @jincheng > > @Aljoscha I've just opened FLINK-11990 > <https://issues.apache.org/jira/browse/FLINK-11990> for the HDF

Re: [ANNOUNCE] Apache Flink 1.8.0 released

2019-04-12 Thread Yu Li
Thanks Aljoscha and all for making this happen, I believe 1.8.0 will be a great and successful release. Best Regards, Yu On Fri, 12 Apr 2019 at 21:23, Patrick Lucas wrote: > The Docker images for 1.8.0 are now available. > > Note that like Flink itself, starting with Flink 1.8 we are no

[DISCUSS] Proposal to support disk spilling in HeapKeyedStateBackend

2019-05-24 Thread Yu Li
Hi All, As mentioned in our speak[1] given in FlinkForwardChina2018, we have improved HeapKeyedStateBackend to support disk spilling and put it in production here in Alibaba for last year's Singles' Day. Now we're ready to upstream our work and the design doc is up for review[2]. Please let us

Re: [ANNOUNCE] Rong Rong becomes a Flink committer

2019-07-11 Thread Yu Li
Congratulations Rong! Best Regards, Yu On Thu, 11 Jul 2019 at 22:54, zhijiang wrote: > Congratulations Rong! > > Best, > Zhijiang > > -- > From:Kurt Young > Send Time:2019年7月11日(星期四) 22:54 > To:Kostas Kloudas > Cc:Jark Wu ;

Re: Bandwidth throttling of checkpoints uploading to s3

2019-07-12 Thread Yu Li
Hi Pavel, Currently there's no such throttling functionality in Flink and I think it's a valid requirement. But before opening a JIRA for this, please allow me to ask for more details to better understand your scenario: 1. What kind of state backend are you using? Since you observe high load to

Re: Bandwidth throttling of checkpoints uploading to s3

2019-07-12 Thread Yu Li
problem. >4. I think it is about 50 Mbit/s. > > Thanks. > > 12.07.2019, 17:27, "Yu Li" : > > Hi Pavel, > > Currently there's no such throttling functionality in Flink and I think > it's a valid requirement. But before opening a JIRA for this, please allow >

Re: [Discuss] Semantics of event time for state TTL

2019-04-16 Thread Yu Li
Thanks for initiating the discussion and wrap-up the conclusion Andrey, and thanks all for participating. Just to confirm, that for the out-of-order case, the conclusion is to update the data and timestamp with the currently-being-processed record w/o checking whether it's an old data, right? In

Re: Making broadcast state queryable?

2019-08-13 Thread Yu Li
Hi Oytun, Sorry but TBH such support will probably not be added in the foreseeable future due to lack of committer bandwidth (not only support queryable broadcast state but all about QueryableState module) as pointed out in other threads [1] [2]. However, I think you could open a JIRA for this

Re: Making broadcast state queryable?

2019-08-14 Thread Yu Li
; Meanwhile, for the sake of opening our state to outside, we will put a > stupid simple operator in between to keep a *duplicate* of the state... > > Thanks again! > > > > > > --- > Oytun Tez > > *M O T A W O R D* > The World's Fastest Human Translation Platform. >

Re: [ANNOUNCE] Apache Flink 1.9.0 released

2019-08-22 Thread Yu Li
Thanks for the update Gordon, and congratulations! Great thanks to all for making this release possible, especially to our release managers! Best Regards, Yu On Thu, 22 Aug 2019 at 14:55, Xintong Song wrote: > Congratulations! > Thanks Gordon and Kurt for being the release managers, and

Re: [SURVEY] Is the default restart delay of 0s causing problems?

2019-09-01 Thread Yu Li
-1 on increasing the default delay to none zero, with below reasons: a) I could see some concerns about setting the delay to zero in the very original JIRA (FLINK-2993 ) but later on in FLINK-9158

Re: [ANNOUNCE] Kinesis connector becomes part of Flink releases

2019-09-01 Thread Yu Li
Great to know, thanks for the efforts Bowen! And I believe it worth a release note in the original JIRA, wdyt? Thanks. Best Regards, Yu On Sat, 31 Aug 2019 at 11:01, Bowen Li wrote: > Hi all, > > I'm glad to announce that, as #9494 > was merged

Re: Making broadcast state queryable?

2019-09-15 Thread Yu Li
l teams. > > --- > Oytun Tez > > *M O T A W O R D* > The World's Fastest Human Translation Platform. > oy...@motaword.com — www.motaword.com > > > On Wed, Aug 14, 2019 at 3:13 AM Yu Li wrote: > >> Good to know your thoughts and the coming talk in Flink Forward Be

Re: Re: [ANNOUNCE] Hequn becomes a Flink committer

2019-08-08 Thread Yu Li
Congratulations Hequn! Well deserved! Best Regards, Yu On Thu, 8 Aug 2019 at 03:53, Haibo Sun wrote: > Congratulations! > > Best, > Haibo > > At 2019-08-08 02:08:21, "Yun Tang" wrote: > >Congratulations Hequn. > > > >Best > >Yun Tang > > > >From: Rong Rong >

Re: Memory constrains running Flink on Kubernetes

2019-07-29 Thread Yu Li
For the memory usage of RocksDB, there's already some discussion in FLINK-7289 and a good suggestion

Re: Graceful Task Manager Termination and Replacement

2019-07-29 Thread Yu Li
Belated but FWIW, besides the region failover and best-efforts failover efforts, I believe stop with checkpoint as proposed in FLINK-12619 and FLIP-45 could also help here, FYI. W.r.t k8s, there're also some offline discussion about supporting local recovery with persistent volume even when task

Re: Capping RocksDb memory usage

2019-08-09 Thread Yu Li
Hi Cam, Which flink version are you using? Actually I don't think any existing flink release could take usage of the write buffer manager natively through some configuration magic, but requires some "developing" efforts, such as manually building flink with a higher version rocksdb to have the

Re: Capping RocksDb memory usage

2019-08-09 Thread Yu Li
r after we read that Jira. > One quick question, I noticed that our disk usage (SSD) for RocksDb is > always stay around %2 (or 2.2 GB), which is not the case before we enable > RocksDb state backend. So wondering what stoping it? > > Thanks, > Cam > > > > On Fri, Aug 9,

Re: [DISCUSS] Deprecate previous Python APIs

2019-06-14 Thread Yu Li
+1 on removing plus an explicit NOTE thread, to prevent any neglection due to the current title (deprecation). Best Regards, Yu On Fri, 14 Jun 2019 at 18:09, Stephan Ewen wrote: > Okay, so we seem to have consensus for at least deprecating them, with a > suggestion to even directly remove

Re: Queryable state and TTL

2019-07-03 Thread Yu Li
Thanks for the ping Andrey. For me the general answer is yes, but TBH it will probably not be added in the foreseeable future due to lack of committer bandwidth (not only QueryableState with TTL but all about QueryableState module) as per Aljoscha pointed out in another thread [1]. Although we

Re: Per Operator State Monitoring

2019-11-26 Thread Yu Li
Hi Aaron, I don't think we have such fine grained metrics on per operation state size, but from your description that "YARN kills containers who are exceeding their memory limits", I think the root cause is not the state size but related to the memory consumption of the state backend. My guess

Re: [PROPOSAL] Contribute Stateful Functions to Apache Flink

2019-10-12 Thread Yu Li
Hi Stephan, Big +1 for adding stateful functions to Flink. I believe a lot of user would be interested to try this out and I could imagine how this could contribute to reduce the TCO for business requiring both streaming processing and stateful functions. And my 2 cents is to put it into flink

[ANNOUNCE] Apache Flink 1.10.0 released

2020-02-12 Thread Yu Li
The Apache Flink community is very happy to announce the release of Apache Flink 1.10.0, which is the latest major release. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications. The release is

Re: FsStateBackend vs RocksDBStateBackend

2020-02-23 Thread Yu Li
Yes FsStateBackend would be the best fit for state access performance in this case. Just a reminder that FsStateBackend will upload the full dataset to DFS during checkpointing, so please watch the network bandwidth usage and make sure it won't become a new bottleneck. Best Regards, Yu On Fri,

Re: [DISCUSS] Drop Savepoint Compatibility with Flink 1.2

2020-02-23 Thread Yu Li
+1 for dropping savepoint compatibility with Flink 1.2. Best Regards, Yu On Sat, 22 Feb 2020 at 22:05, Ufuk Celebi wrote: > Hey Stephan, > > +1. > > Reading over the linked ticket and your description here, I think it makes > a lot of sense to go ahead with this. Since it's possible to

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-23 Thread Yu Li
Congratulations Jingsong! Well deserved. Best Regards, Yu On Mon, 24 Feb 2020 at 14:10, Congxian Qiu wrote: > Congratulations Jingsong! > > Best, > Congxian > > > jincheng sun 于2020年2月24日周一 下午1:38写道: > >> Congratulations Jingsong! >> >> Best, >> Jincheng >> >> >> Zhu Zhu 于2020年2月24日周一

Re: Status of FLINK-12692 (Support disk spilling in HeapKeyedStateBackend)

2020-01-30 Thread Yu Li
the package is ready. Best Regards, Yu [1] https://flink-packages.org/ On Thu, 30 Jan 2020 at 08:47, Ken Krugler wrote: > Hi Yu Li, > > It looks like this stalled out a bit, from May of last year, and won’t > make it into 1.10. > > I’m wondering if there’s a version in Blin

Re: Status of FLINK-12692 (Support disk spilling in HeapKeyedStateBackend)

2020-01-30 Thread Yu Li
/browse/FLINK-12697 [2] https://github.com/apache/flink/pull/9501 On Fri, 31 Jan 2020 at 15:27, Yu Li wrote: > Hi Ken, > > Thanks for watching this feature. > > Unfortunately yes this didn't make into 1.10, and we will try our best to > complete the upstreaming work in 1.

Re: [ANNOUNCE] Apache Flink 1.10.0 released

2020-02-12 Thread Yu Li
Hi Kristoff, Thanks for the question. About Java 11 support, please allow me to quote from our release note [1]: Lastly, note that the connectors for Cassandra, Hive, HBase, and Kafka 0.8–0.11 have not been tested with Java 11 because the respective projects did not provide Java 11 support at

Re: [ANNOUNCE] Apache Flink 1.10.0 released

2020-02-12 Thread Yu Li
gt; <https://issues.apache.org/jira/browse/FLINK-13884> J > > > > > > *From:* Yu Li > *Sent:* Wednesday, February 12, 2020 8:31 AM > *To:* dev ; user ; > annou...@apache.org > *Subject:* [ANNOUNCE] Apache Flink 1.10.0 released > > > > The Apache Flink co

Re: Streaming Job eventually begins failing during checkpointing

2020-04-27 Thread Yu Li
Sorry, just noticed this thread... @Stephan I cannot remember the discussion but I think it's an interesting topic, will find some time to consider it (unregister states). @Eleanore Glad to know that Beam community has fixed it and thanks for the reference. Best Regards, Yu On Sun, 26 Apr

[ANNOUNCE] Apache Flink 1.10.1 released

2020-05-13 Thread Yu Li
The Apache Flink community is very happy to announce the release of Apache Flink 1.10.1, which is the first bugfix release for the Apache Flink 1.10 series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming

Re: The question about the FLIP-45

2020-03-23 Thread Yu Li
Hi LakeShen, Sorry for the late response. For the first question, literally, the stop command should be used if one means to stop the job instead of canceling it. For the second one, since FLIP-45 is still under discussion [1] [2] (although a little bit stalled due to priority), we still don't

Re: Performance issue associated with managed RocksDB memory

2020-09-15 Thread Yu Li
Thanks for the follow up Juha, I've just assigned FLINK-19238 to you. Let's further track this on JIRA. Best Regards, Yu On Tue, 15 Sep 2020 at 15:04, Juha Mynttinen wrote: > Hey > > I created this one https://issues.apache.org/jira/browse/FLINK-19238. > > Regards, > Juha >

Re: flink checkpoint timeout

2020-10-05 Thread Yu Li
I'm not 100% sure but from the given information this might be related to FLINK-14498 [1] and partially relieved by FLINK-16645 [2]. @Omkar Could you try the 1.11.0 release out and see whether the issue disappeared? @zhijiang @yingjie could you also take a look here? Thanks. Best Regards, Yu

Re: Disable WAL in RocksDB recovery

2020-09-18 Thread Yu Li
Thanks for bringing this up Juha, and good catch. We actually are disabling WAL for routine writes by default when using RocksDB and never encountered segment fault issues. However, from history in FLINK-8922, segment fault issue occurs during restore if WAL is disabled, so I guess the root cause

Re: [ANNOUNCE] Apache Flink 1.11.2 released

2020-09-21 Thread Yu Li
Thanks Zhu Zhu for being our release manager and everyone else who made the release possible! Best Regards, Yu On Thu, 17 Sep 2020 at 13:29, Zhu Zhu wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.11.2, which is the second bugfix release for the

Re: Disable WAL in RocksDB recovery

2020-09-21 Thread Yu Li
> Regards, > Juha > ---------- > *From:* Yu Li > *Sent:* Friday, September 18, 2020 3:58 PM > *To:* Juha Mynttinen > *Cc:* user@flink.apache.org > *Subject:* Re: Disable WAL in RocksDB recovery > > Thanks for bringing this up Juha, and good catch. > > We actually are

Re: [DISCUSS] Upgrade HBase connector to 2.2.x

2020-06-19 Thread Yu Li
+1 on upgrading the HBase version of the connector, and 1.4.3 is indeed an old version. OTOH, AFAIK there're still quite some 1.x HBase clusters in production. We could also see that the HBase community is still maintaining 1.x release lines (with "stable-1 release" point to 1.4.13) [1] Please

Re: [DISCUSS] Upgrade HBase connector to 2.2.x

2020-06-19 Thread Yu Li
in production. Best Regards, Yu [1] http://mail-archives.apache.org/mod_mbox/hbase-dev/202003.mbox/%3c30180be2-bd93-d414-a158-16c9c8d01...@apache.org%3E On Fri, 19 Jun 2020 at 15:54, Yu Li wrote: > +1 on upgrading the HBase version of the connector, and 1.4.3 is indeed an > old v

Re: Performance issue associated with managed RocksDB memory

2020-06-26 Thread Yu Li
To clarify, that my questions were all against the very original issue instead of the WordCount job. The timers come from the window operator you mentioned as the source of the original issue: === bq. If I create a Flink job that has a single "heavy"

Re: Performance issue associated with managed RocksDB memory

2020-06-25 Thread Yu Li
Thanks for the ping Andrey. Hi Juha, Thanks for reporting the issue. I'd like to check the below things before further digging into it: 1. Could you let us know your configurations (especially memory related ones) when running the tests? 2. Did you watch the memory consumption before / after

Re: [ANNOUNCE] Apache Flink 1.10.3 released

2021-01-28 Thread Yu Li
Thanks Xintong for being our release manager and everyone else who made the release possible! Best Regards, Yu On Fri, 29 Jan 2021 at 15:05, Xintong Song wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.10.3, which is the third bugfix release for

Re: [ANNOUNCE] Apache Flink 1.13.2 released

2021-08-09 Thread Yu Li
Thanks Yun Tang for being our release manager and everyone else who made the release possible! Best Regards, Yu On Fri, 6 Aug 2021 at 13:52, Yun Tang wrote: > > The Apache Flink community is very happy to announce the release of Apache > Flink 1.13.2, which is the second bugfix release for

[ANNOUNCE] Call for Presentations for ApacheCon Asia 2022 streaming track

2022-05-18 Thread Yu Li
Hi everyone, ApacheCon Asia [1] will feature the Streaming track for the second year. Please don't hesitate to submit your proposal if there is an interesting project or Flink experience you would like to share with us! The conference will be online (virtual) and the talks will be pre-recorded.

[ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Yu Li
Dear Flinkers, As you may have noticed, we are pleased to announce that Flink Table Store has joined the Apache Incubator as a separate project called Apache Paimon(incubating) [1] [2] [3]. The new project still aims at building a streaming data lake platform for high-speed data ingestion,

Re: [ANNOUNCE] Apache Flink 1.19.0 released

2024-03-18 Thread Yu Li
Congrats and thanks all for the efforts! Best Regards, Yu On Tue, 19 Mar 2024 at 11:51, gongzhongqiang wrote: > > Congrats! Thanks to everyone involved! > > Best, > Zhongqiang Gong > > Lincoln Lee 于2024年3月18日周一 16:27写道: >> >> The Apache Flink community is very happy to announce the release of

Re: [ANNOUNCE] Apache Paimon is graduated to Top Level Project

2024-03-28 Thread Yu Li
CC the Flink user and dev mailing list. Paimon originated within the Flink community, initially known as Flink Table Store, and all our incubating mentors are members of the Flink Project Management Committee. I am confident that the bonds of enduring friendship and close collaboration will

[DISCUSS] State TTL支持EventTime

2019-04-09 Thread Yu Li
大家好, 也许大家已经注意到通过FLINK-12005[1]我们提出了在State中支持基于EventTime[2]的TTL[3]语义,有关这个特性我们希望向大家发起一个讨论/调查,包括下述问题: 1. 在您的应用场景下是否需要基于EventTime的state TTL?如果需要,请简单描述场景和需求原因,这是决定我们是否开发该特性的关键。 2. 在TTL定义中有两个关键的时间,一个是数据的时间戳,另外一个是检查数据是否过期的当前时间(currentTime)。由于流式计算的特点,数据有可能“乱序”到达,而Flink为了处理这种情况引入了watermark的概念。因此对于

Re: 关于在读和写频率都很高的情况下怎么优化rocksDB

2020-02-26 Thread Yu Li
Hi, 您给出的链接里的文档是面向最新发布版本,也就是1.10.0的,在之前的版本里RocksDB使用的不是Managed Memory,所以文档里的一些调优指导也不会有效。 建议先了解一下1.10.0里RocksDB内存管理的相关内容 [1],我们提供了一些配置以方便调节读写缓存的比例,了解之后可以试用1.10.0版本并尝试根据文档进行性能调优。 另外,如果升级并进行简单的调试依然无效,并且确认是由于state瓶颈导致作业反压,那么归根结底需要对RocksDB进行调优。建议参考RocksDB官方的相关文档 [2] 进行尝试。 希望这些信息可以帮到您。 Best Regards,

Re: flink rocksdb状态后端物理内存溢出的问题

2020-02-20 Thread Yu Li
建议升级到1.10.0版本,该版本默认对RocksDB backend的内存使用会有限制,更多资料请参考官方文档 [1]。 Best Regards, Yu [1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/state/state_backends.html#memory-management On Thu, 20 Feb 2020 at 17:42, chanamper wrote: > 请教一下,我采用flink >

Re: 咨询一下 RocksDB 状态后端的调优经验

2020-01-14 Thread Yu Li
Hi, 如唐云所述,FLINK-7289 [1] 所有的开发工作已经完成,目前剩余的工作是补充end-to-end测试以及完善文档,因此release-1.10分支的代码功能已经完全可用了 我们建议使用FLINK-7289实现的方式来控制rocksdb内存,这将极大的简化用户所需的配置,只需要设置"state.backend.rocksdb.memory.managed"为true并调整managed memory大小,或者通过"state.backend.rocksdb.memory.fixed-per-slot" 配置对应单个slot RocksDB可使用的最大内存即可

Re: 请问Flink-1.10.1 release可以在哪里下载?(无正文)

2020-04-16 Thread Yu Li
1.10.1还剩余最后一个blocker [1],解决之后将创建Release Candidate并启动投票,预计还需要1-2周时间,感谢关注。 Best Regards, Yu [1] https://issues.apache.org/jira/browse/FLINK-16662 On Thu, 16 Apr 2020 at 17:24, godfrey he wrote: > 目前社区已经在讨论 release-1.10.1 RC [1] 的发布 > > [1] > >

[ANNOUNCE] Apache Flink 1.10.1 released

2020-05-13 Thread Yu Li
The Apache Flink community is very happy to announce the release of Apache Flink 1.10.1, which is the first bugfix release for the Apache Flink 1.10 series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming

Re: The question about the FLIP-45

2020-03-23 Thread Yu Li
Hi LakeShen, Sorry for the late response. For the first question, literally, the stop command should be used if one means to stop the job instead of canceling it. For the second one, since FLIP-45 is still under discussion [1] [2] (although a little bit stalled due to priority), we still don't

Re: Flink对IoTDB的支持

2020-10-10 Thread Yu Li
暂时没有听到相关的计划,如果有相关需求,欢迎在社区发起讨论 @jincheng sun 金城老师看有没有什么补充?谢谢 Best Regards, Yu On Fri, 2 Oct 2020 at 19:21, milan183sansiro wrote: > 请问社区有无对IoTDB的source或sink的支持计划 > >

Re: [ANNOUNCE] Apache Flink 1.11.2 released

2020-09-21 Thread Yu Li
Thanks Zhu Zhu for being our release manager and everyone else who made the release possible! Best Regards, Yu On Thu, 17 Sep 2020 at 13:29, Zhu Zhu wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.11.2, which is the second bugfix release for the

Re: flink1.10.1/1.11.1 使用sql 进行group 和 时间窗口 操作后 状态越来越大

2020-08-06 Thread Yu Li
看到生产上使用的还是1.8.2版本,请问同样的作业使用1.8.2的表现是怎样的? Best Regards, Yu On Thu, 6 Aug 2020 at 16:29, op <520075...@qq.com> wrote: > 感谢回答 > 我之前用1.10也有同样的问题 > > > > > --原始邮件-- > 发件人: > "user-zh" >

Re: flink1.10.1/1.11.1 使用sql 进行group 和 时间窗口 操作后 状态越来越大

2020-08-06 Thread Yu Li
@鱼子酱 请问同样的作业,都使用RocksDB增量checkpoint,在1.8.2版本和1.11.1版本下的表现是否一致?还是说只有1.11.1版本下增量大小会单调增加? @op 类似的问题,请问使用FsStateBackend,是否在不同Flink版本下测试过?表现是否一致? 上述问题主要想确认一下新版本的表现和旧版本是否一致,如果不一致则有可能是新版本中引入的bug。谢谢。 Best Regards, Yu On Thu, 6 Aug 2020 at 13:52, Congxian Qiu wrote: > Hi >

Re: Flink 1.10.1版本StreamingFileSink写入HDFS失败

2020-08-11 Thread Yu Li
Hi 王剑, 我认为你的分析是正确的,相关代码在超过lease的soft limit之后应该主动调用一下recover lease的逻辑。你是否愿意提交一个patch来fix该问题?我在JIRA上也留言了,后续可以直接在JIRA上讨论。 另外,更正一下JIRA链接:https://issues.apache.org/jira/browse/FLINK-18592 Best Regards, Yu On Tue, 11 Aug 2020 at 15:16, Jian Wang wrote: > Hi all, > > 我们在使用flink-1.10.1 on YARN(版本:

Re: [ANNOUNCE] Apache Flink 1.10.3 released

2021-01-28 Thread Yu Li
Thanks Xintong for being our release manager and everyone else who made the release possible! Best Regards, Yu On Fri, 29 Jan 2021 at 15:05, Xintong Song wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.10.3, which is the third bugfix release for

Re: flink 1.12.2-rc2 被挖矿

2021-03-01 Thread Yu Li
Hi, 从上述描述上看,并不能确认是Flink的安全漏洞,建议进行更多测试,确认之后再向Flink社区报告。 近期已知的Flink安全漏洞包括CVE-2020-1960 , CVE-2020-17518 和 CVE-2020-17519

Re: flink 1.12.2-rc2 被挖矿

2021-02-28 Thread Yu Li
能再给一些细节吗?确认是Flink的问题导致的吗?怀疑的漏洞是哪个? 最近1.12.2 rc2正在release voting阶段,我们希望尽快确认是否存在安全漏洞并及时修复(如果有),谢谢。 Best Regards, Yu On Mon, 1 Mar 2021 at 13:26, macdoor wrote: > 我编译的flink 1.12.2-rc2 被挖矿,这个漏洞之前不是堵住了吗? > > > > -- > Sent from: http://apache-flink.147419.n8.nabble.com/ >

Re: [ANNOUNCE] Apache Flink 1.13.2 released

2021-08-09 Thread Yu Li
Thanks Yun Tang for being our release manager and everyone else who made the release possible! Best Regards, Yu On Fri, 6 Aug 2021 at 13:52, Yun Tang wrote: > > The Apache Flink community is very happy to announce the release of Apache > Flink 1.13.2, which is the second bugfix release for

[ANNOUNCE] Call for Presentations for ApacheCon Asia 2022 streaming track

2022-05-18 Thread Yu Li
Hi everyone, ApacheCon Asia [1] will feature the Streaming track for the second year. Please don't hesitate to submit your proposal if there is an interesting project or Flink experience you would like to share with us! The conference will be online (virtual) and the talks will be pre-recorded.

Re: Flink OLAP 与 Trino TPC-DS 对比

2022-05-06 Thread Yu Li
感谢大家的分享和分析,也期待Flink在相关方向的持续优化! Let's make Flink great together. :-) btw, 第5个引用的语雀文档链接已过期,建议使用google doc并更新一下链接 Best Regards, Yu On Sun, 1 May 2022 at 21:57, Zhilong Hong wrote: > Hello, > > 这段时间我们针对 LuNing 反馈的问题进行了深入的分析调研,在此将结论同步给社区。特别感谢 LuNing 反馈这一问题并与我们一起进行分析排查。 > > 根据我们的分析,造成 Flink 1.14

[ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-27 Thread Yu Li
Dear Flinkers, As you may have noticed, we are pleased to announce that Flink Table Store has joined the Apache Incubator as a separate project called Apache Paimon(incubating) [1] [2] [3]. The new project still aims at building a streaming data lake platform for high-speed data ingestion,

Re: [ANNOUNCE] Apache Flink 1.19.0 released

2024-03-18 Thread Yu Li
Congrats and thanks all for the efforts! Best Regards, Yu On Tue, 19 Mar 2024 at 11:51, gongzhongqiang wrote: > > Congrats! Thanks to everyone involved! > > Best, > Zhongqiang Gong > > Lincoln Lee 于2024年3月18日周一 16:27写道: >> >> The Apache Flink community is very happy to announce the release of