Re: [DISCUSS] FLIP-27: Refactor Source Interface

Hi Becket, Thanks for updating the progress! I have a question about the #OperatorCoordinator. Will there be any communication between different #OperatorCoordinators (or in the future plan)? Because in that way it may be able to cover some cases in FLIP-27[1] like initializing static data

Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC

Congratulations Jark! Best, Jiayi Liao Original Message Sender: Yun Gao Recipient: dev Date: Friday, Nov 8, 2019 18:37 Subject: Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC Congratulations Jark！ Best, Yun --

Re: RocksDB state on HDFS seems not being cleanned up

Hi Shuwen, The “shared” means that the state files are shared among multiple checkpoints, which happens when you enable incremental checkpointing[1]. Therefore, it’s reasonable that the size keeps growing if you set “state.checkpoint.num-retained” to be a big value. [1]

Re: [DISCUSS] Semantic and implementation of per-job mode

Hi all, Firstly thanks @tison for bring this up and strongly +1 for the overall design. I’d like to add one more example of "multiple jobs in one program" with what I’m currently working on. I’m trying to run a TPC-DS benchmark testing (including tens of sql query job) on Flink and

Re: [ANNOUNCE] Becket Qin joins the Flink PMC

Congratulations Becket! Best, Jiayi Liao Original Message Sender: Biao Liu Recipient: dev Date: Tuesday, Oct 29, 2019 12:10 Subject: Re: [ANNOUNCE] Becket Qin joins the Flink PMC Congrats Becket! Thanks, Biao /'bɪ.aʊ/ On Tue, 29 Oct 2019 at 12:07, jincheng sun wrote: > Congratulations

Re: [DISCUSS] Introduce a location-oriented two-stage query mechanism toimprove the queryable state.

Hi vino, +1 for improvement on queryable state feature. This reminds me of the state-processing-api module, which is very helpful when we analyze state in offline. However currently we don’t have many ways to know what is happening about the state inside a running application, which makes me

Re: Per Key Grained Watermark Support

Hi Congxian, Thanks but by doing that, we will lose some features like output of the late data. Original Message Sender: Congxian Qiu Recipient: Lasse Nedergaard Cc: 廖嘉逸; u...@flink.apache.org; dev@flink.apache.org Date: Monday, Sep 23, 2019 19:56 Subject: Re: Per Key Grained Watermark

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations! Best, Jiayi Liao Original Message Sender: Till Rohrmann Recipient: dev; user Date: Wednesday, Sep 11, 2019 17:22 Subject: [ANNOUNCE] Zili Chen becomes a Flink committer Hi everyone, I'm very happy to announce that Zili Chen (some of you might also know him as Tison Kun)

Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer tothe Flink project

Congratulations ! Best Regards, Jiayi Liao Original Message Sender:Hequn chengchenghe...@gmail.com Recipient:dev...@flink.apache.org Date:Thursday, Jul 18, 2019 17:51 Subject:Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer tothe Flink project Congratulations Becket! Best,

Re: Add relative path support in Savepoint Connector

of making savepoints relocatable. Best, Konstantin [1] https://issues.apache.org/jira/browse/FLINK-5763 On Wed, Jul 17, 2019 at 8:35 AM bupt_ljy bupt_...@163.com wrote: Hi again, Anyone has any opinion on this topic?Best Regards, Jiayi LiaoOriginal Message Sender:bupt_ljybupt_...@163

Re: Add relative path support in Savepoint Connector

Hi again, Anyone has any opinion on this topic? Best Regards, Jiayi Liao Original Message Sender:bupt_ljybupt_...@163.com Recipient:dev...@flink.apache.org Cc:Tzu-Li (Gordon) taitzuli...@apache.org Date:Tuesday, Jul 16, 2019 15:24 Subject:Add relative path support in Savepoint Connector Hi

Add relative path support in Savepoint Connector

Hi all, Firstly I appreciate Gordon and Seth’s effort on this feature, which is really helpful to our production use. Like you mentioned in the FLINK-12047, one of the production uses is that we use the existing state to derive new state. However, since the state handle is using the

Re: RE: [DISCUSS] Improve Queryable State and introduce aQueryServerProxy component

Hi vino, Big +1 for this. Glad to see new progress on this topic! I’ve left some comments on it. Best Regards, Jiayi Liao Original Message Sender:vino yangyanghua1...@gmail.com Recipient:Georgi stoyanovgstoya...@live.com Cc:dev...@flink.apache.org; useru...@flink.apache.org; Stefan

Re: A Question About Flink Network Stack

Hi Zhijiang, Thank you for the detailed explaination! Best Regards, Jiayi Liao Original Message Sender:zhijiangwangzhijiang...@aliyun.com.INVALID Recipient:dev...@flink.apache.org Date:Tuesday, Jun 18, 2019 17:34 Subject:Re: A Question About Flink Network Stack Hi Jiayi, Thanks for

A Question About Flink Network Stack

Hi all, I’m not very familiar with the network part of Flink. And a question occurs to me after I read most related source codes about network. I’ve seen that Flink uses the credit-based machanism to solve the blocking problem from receivers’ side, which means that one “slow” input channel

[jira] [Created] (FLINK-12668) Introduce fromParallelElements for generating DataStreamSource

bupt_ljy created FLINK-12668: Summary: Introduce fromParallelElements for generating DataStreamSource Key: FLINK-12668 URL: https://issues.apache.org/jira/browse/FLINK-12668 Project: Flink

Re: [DISCUSS] Improve Queryable State and introduce aQueryServerProxy component

Hi yang, +1 for this proposal. Queryable state is a very common usage in our scenarios when we debug and query the realtime status in streaming process like CEP. And we’ve done a lot to improve the “user experience” of this feature like exposing the taskmanager’s proxy port in TaskManagerInfo.

CEP - Support for multiple pattern

Hi, all It’s actually very common that we construct more than one rule on the same data source. And I’m developing some such kind of features for our businesses and some ideas come up. Do we have any plans for supporting multiple patterns in CEP? Best, Jiayi Liao

Re: [DISCUSS] Releasing Flink 1.7.1

Hi Chesnay, Thanks for these useful fixes. +1 for the release. Best, Jiayi Liao Original Message Sender:Hequn chengchenghe...@gmail.com Recipient:dev...@flink.apache.org Date:Thursday, Dec 13, 2018 09:37 Subject:Re: [DISCUSS] Releasing Flink 1.7.1 Hi Chesnay, Thanks for the efforts. +1

Re: [ANNOUNCE] New committer Gary Yao

Congratulations Gary! Jiayi Original Message Sender:vino yangyanghua1...@gmail.com Recipient:dev...@flink.apache.org Date:Saturday, Sep 8, 2018 10:11 Subject:Re: [ANNOUNCE] New committer Gary Yao Congratulations Gary! Chen Qin qinnc...@gmail.com 于2018年9月8日周六 上午2:07写道： Congrats! Chen

Re: [Proposal] Utilities for reading, transforming and creatingStreaming savepoints

Hi, +1, I think it will be a very great tool for Flink, especially the creating new state part. On production, we’re really worried about the availability of the savepoints, because the generating logic is inside Flink and we don’t have a good way to validate it. But with this tool, we can