Re: [ANNOUNCE] Progress of Apache Flink 1.10 #3

2020-01-03 Thread vino yang
Hi Yu, Thanks for your great work. Best, Vino Congxian Qiu 于2020年1月3日周五 下午6:18写道: > Thanks for the great work, and look forward to the first RC. > > Best, > Congxian > > > Jingsong Li 于2020年1月3日周五 下午12:58写道: > > > Thanks Yu and Gary, > > > > Table and hive layer have resolved all blockers

Re: Flink On K8s, build docker image very slowly, is there some way to make it faster?

2019-12-22 Thread vino yang
Hi Lake, Can you clearly count or identify which steps are taking a long time? Best, Vino LakeShen 于2019年12月23日周一 下午2:46写道: > Hi community , when I run the flink task on k8s , the first thing is that > to build the flink task jar to > Docker Image . I find that It would spend much time to

Re: [Question] How to use different filesystem between checkpoint data and user data sink

2019-12-18 Thread vino yang
Hi ouywl, *>>Thread.currentThread().getContextClassLoader();* What does this statement mean in your program? In addition, can you share your implementation of the customized file system plugin and the related exception? Best, Vino ouywl 于2019年12月18日周三 下午4:59写道: > Hi all, > We have

Re: Documentation tasks for release-1.10

2019-12-16 Thread vino yang
+1 for centralizing all the documentation issues so that the community can take more effective to fix them. Best, Vino Xintong Song 于2019年12月16日周一 下午6:02写道: > Thank you Kostas. > Big +1 for keeping all the documentation related issues at one place. > > I've added the documentation task for

Re: [ANNOUNCE] Zhu Zhu becomes a Flink committer

2019-12-15 Thread vino yang
Congratulations, Zhu Zhu! Best, Vino Yang Wang 于2019年12月16日周一 上午10:01写道: > Congratulations, Zhu Zhu! > > wenlong.lwl 于2019年12月16日周一 上午9:56写道: > > > Congratulations, Zhu Zhu! > > > > On Mon, 16 Dec 2019 at 09:14, Leonard Xu wrote: > > > > >

Re: [VOTE] Multi-topics consuming from KafkaTableSource

2019-12-04 Thread vino yang
Hi Leo, IMO, if you want to vote a proposal. It would be better to start a new VOTE ML thread. Best, Vino leo 于2019年12月5日周四 上午9:07写道: > Hi everyone, > > > Please vote for this FLINK-14729. The JIRA can be found here, > > https://issues.apache.org/jira/browse/FLINK-14729 > > The discussion can

Re: [DISCUSS] Drop Kafka 0.8/0.9

2019-12-04 Thread vino yang
+1 jincheng sun 于2019年12月5日周四 上午10:26写道: > +1 for drop it, and Thanks for bring up this discussion Chesnay! > > Best, > Jincheng > > Jark Wu 于2019年12月5日周四 上午10:19写道: > >> +1 for dropping, also cc'ed user mailing list. >> >> >> Best, >> Jark >> >> On Thu, 5 Dec 2019 at 03:39, Konstantin Knauf

Re: Building with Hadoop 3

2019-12-04 Thread vino yang
Hi Marton, Thanks for your explanation. Personally, I look forward to your contribution! Best, Vino Márton Balassi 于2019年12月4日周三 下午5:15写道: > Wearing my Cloudera hat I can tell you that we have done this exercise for > our distros of the 3.0 and 3.1 Hadoop versions. We have not contributed >

Re: Building with Hadoop 3

2019-12-03 Thread vino yang
cc @Chesnay Schepler to answer this question. Foster, Craig 于2019年12月4日周三 上午1:22写道: > Hi: > > I don’t see a JIRA for Hadoop 3 support. I see a comment on a JIRA here > from a year ago that no one is looking into Hadoop 3 support [1]. Is there > a document or JIRA that now exists which would

Re: [DISCUSS] Drop RequiredParameters and OptionType

2019-12-03 Thread vino yang
+1, One concern: these two classes are marked with `@publicEvolving` annotation. Shall we mark them with `@Deprecated` annotation firstly? Best, Vino Dian Fu 于2019年12月3日周二 下午8:56写道: > +1 to remove them. It seems that we should also drop the class Option as > it's currently only used in

Re: [DISCUSS] Remove old WebUI

2019-11-25 Thread vino yang
+1 Best, Vino Zhu Zhu 于2019年11月25日周一 下午2:06写道: > +1 > > Thanks, > Zhu Zhu > > Dian Fu 于2019年11月25日周一 上午11:09写道: > > > +1 to drop the old UI. > > > > > 在 2019年11月25日,上午10:59,Zhenghua Gao 写道: > > > > > > +1 to drop the old one. > > > > > > *Best Regards,* > > > *Zhenghua Gao* > > > > > > > > >

Re: I want to contribute to Flink!

2019-11-24 Thread vino yang
Hi, Currently, you do not need a JIRA permission. Only committers can assign a Jira ticket. Please read the official contribution guide[1] before contributing. Best, Vino [1]: https://flink.apache.org/contributing/contribute-code.html#consensus 470905446 <470905...@qq.com> 于2019年11月24日周日

Re: [DISCUSS] Remove old WebUI

2019-11-24 Thread vino yang
+1 Best, Vino Congxian Qiu 于2019年11月25日周一 上午10:48写道: > +1 to drop the old UI > Best, > Congxian > > > aihua li 于2019年11月25日周一 上午9:38写道: > > > +1 to drop the old UI. > > > > > 在 2019年11月21日,下午8:04,Chesnay Schepler 写道: > > > > > > Hello everyone, > > > > > > Flink 1.9 shipped with a new UI,

Re: [ANNOUNCE] Launch of flink-packages.org: A website to foster the Flink Ecosystem

2019-11-19 Thread vino yang
Hi Robert, Just added it under the "Tools" category[1]. [1]: https://flink-packages.org/packages/kylin-flink-cube-engine Best, Vino Robert Metzger 于2019年11月19日周二 下午4:33写道: > Thanks. > You can add Kylin whenever you think it is ready. > > On Tue, Nov 19, 2019 at 9:

Re: [DISCUSS] Release flink-shaded 9.0

2019-11-19 Thread vino yang
+1 Best, Vino Dian Fu 于2019年11月19日周二 下午4:17写道: > Hi Chesnay, > > Thanks a lot for kicking off this release. +1 to release flink-shaded 9.0. > > I'm willing to help on the release. Please feel free to let me know > if there is anything I could help. > > Regards, > Dian > > On Mon, Nov 18, 2019

Re: [ANNOUNCE] Launch of flink-packages.org: A website to foster the Flink Ecosystem

2019-11-19 Thread vino yang
Thanks Robert. Great job! The web site looks great. In the future, we can also add my Kylin Flink cube engine[1] to the ecosystem projects list. [1]: https://github.com/apache/kylin/tree/engine-flink Best, Vino Oytun Tez 于2019年11月19日周二 上午12:09写道: > Congratulations! This is exciting. > > >

Re: [VOTE] FLIP-83: Flink End-to-end Performance Testing Framework

2019-11-18 Thread vino yang
+1 (non-binding) Best, Vino jincheng sun 于2019年11月18日周一 下午2:31写道: > +1 (binding) > > OpenInx 于2019年11月18日周一 下午12:09写道: > > > +1 (non-binding) > > > > On Mon, Nov 18, 2019 at 11:54 AM aihua li wrote: > > > > > +1 (non-binding) > > > > > > Thanks Yu Li for driving on this. > > > > > > > 在

Re: Flink on Yarn resource arrangement

2019-11-13 Thread vino yang
Hi Alex, Which Flink version are you using? AFAIK, since Flink 1.8+, the config option: "-yn" for Flink on YARN job cluster mode does not take effect(always 1 and would be overridden). So, the config option "-ys" and "-p" will decide the number of TM. The first example: -p(20)/-ys(3) should be

Re: [VOTE] FLIP-79: Flink Function DDL Support (1.10 Release Feature Only)

2019-11-11 Thread vino yang
+1 (non-binding) Best, Vino Xuefu Z 于2019年11月12日周二 上午3:27写道: > +1 (non-binding) > > On Mon, Nov 11, 2019 at 9:54 AM Shuyi Chen wrote: > > > +1 (binding) > > > > On Sat, Nov 9, 2019 at 11:17 PM Kurt Young wrote: > > > > > +1 (binding) > > > > > > Best, > > > Kurt > > > > > > > > > On Sun, Nov

Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC

2019-11-09 Thread vino yang
Congratulations, Jark. Best, Vino Leonard Xu 于2019年11月8日周五 下午8:40写道: > Congratulations, Jark. > Thanks for your contribution and help. > > Best, > Leonard Xu > > > On 2019年11月8日, at 下午6:37, Yun Gao wrote: > > > > Congratulations Jark! > > > > Best, > > Yun > > > > > >

Re: Checkpoint in FlinkSQL

2019-11-04 Thread vino yang
Hi Simon, Absolutely, yes. Before using Flink SQL, you need to initialize a StreamExecutionEnvirnoment instance[1], then call StreamExecutionEnvirnoment#setStateBackend or StreamExecutionEnvirnoment#enableCheckpointing to specify the information what you want. [1]:

Re: Unit testing for ProcessAllWindowFunction

2019-11-04 Thread vino yang
.com/ctechworks> < > https://www.facebook.com/CriticalTechWorks/> > > > On 01/11/2019, 03:38, "vino yang" wrote: > > Hi Diogo, > > In order to test ProcessAllWindowFunction, you need to do a little > more. > > You can r

Re: FW: Unit testing for ProcessAllWindowFunction

2019-10-31 Thread vino yang
Hi Diogo, In order to test ProcessAllWindowFunction, you need to do a little more. You can refer to Flink's own test code.[1] In short: 1) Write a Flink Streaming job that uses your UDF (here is ProcessAllWindowFunction) and return a DataStream. 2) Get OneInputTransformation through

Re: [DISCUSS] Introduce a location-oriented two-stage query mechanism toimprove the queryable state.

2019-10-31 Thread vino yang
[2] https://s.apache.org/r8k8a > > Best Regards, > Yu > > > On Fri, 25 Oct 2019 at 23:59, vino yang wrote: > >> Hi David, >> >> I know that in some scenarios, the queryable state has some limitations. >> But in some scenarios, it is valuable, such as in

Re: [VOTE] Move flink-orc to flink-formats from flink-connectors

2019-10-30 Thread vino yang
+1 (non-binding) Best, Vino Dawid Wysakowicz 于2019年10月30日周三 下午4:07写道: > +1 > > Best, > > Dawid > > On 30/10/2019 04:49, Jingsong Li wrote: > > Hi all: > > > > We already have the parent model of formats. we have put other > > formats(flink-avro, flink-json, flink-parquet, flink-json,

Re: [ANNOUNCE] Becket Qin joins the Flink PMC

2019-10-28 Thread vino yang
Congratulations Becket! Best, Vino Wei Zhong 于2019年10月29日周二 上午9:42写道: > Congratulations Becket! > > Best, > Wei > > > 在 2019年10月29日,09:36,Paul Lam 写道: > > > > Congrats Becket! > > > > Best, > > Paul Lam > > > >> 在 2019年10月29日,02:18,Xingcan Cui 写道: > >> > >> Congratulations, Becket! > >> > >>

Re: [DISCUSS] Introduce a location-oriented two-stage query mechanism toimprove the queryable state.

2019-10-25 Thread vino yang
reflects an earlier point in time than what may've been previously > fetched. > > (4) The state that is wanted (e.g., window state, or operator state) > isn't queryable. > > Best, > David > > On Fri, Oct 25, 2019 at 9:51 AM vino yang wrote: > > > > Hi Jiayi, >

Re: [VOTE] FLIP-81: Executor-related new ConfigOptions

2019-10-24 Thread vino yang
+1 (non-binding) Best, Vino Yang Wang 于2019年10月23日周三 下午9:05写道: > We could benefit a lot from unifying the cli options and config options. > > +1(non-binding) > > Best, > Yang > > Aljoscha Krettek 于2019年10月23日周三 下午5:24写道: > > > +1 (binding) > > > >

Re: [VOTE] Accept Stateful Functions into Apache Flink

2019-10-22 Thread vino yang
+1 (non-binding) Best, Vino Zhu Zhu 于2019年10月22日周二 下午4:33写道: > +1 (non-binding) > > Thanks, > Zhu Zhu > > Biao Liu 于2019年10月22日周二 上午11:06写道: > > > +1 (non-binding) > > > > Thanks, > > Biao /'bɪ.aʊ/ > > > > > > > > On Tue, 22 Oct 2019 at 10:26, Jark Wu wrote: > > > > > +1 (non-binding) > > >

[DISCUSS] Introduce a location-oriented two-stage query mechanism to improve the queryable state.

2019-10-22 Thread vino yang
Hi guys, Currently, queryable state's client is hard to use. Because it requires users to know the address of TaskManager and the port of the proxy. Actually, most users who do not have good knowledge about the Flink's inner and runtime in production. The queryable state clients directly

Re: [NOTICE] Binary licensing is now auto-generated

2019-10-17 Thread vino yang
Thanks Chesnay! Great job! Best, Vino Jark Wu 于2019年10月18日周五 上午9:51写道: > Thanks Chesnay! This is really a great job! > > Best, > Jark > > > 在 2019年10月17日,22:03,未来阳光 <2217232...@qq.com> 写道: > > > > Thanks for this improvement Chesnay ! > > > > > > > > > > ---Original--- > > From: "Chesnay

Re: [DISCUSS] Stateful Functions - in which form to contribute? (same or different repository)

2019-10-16 Thread vino yang
Hi, Fast release cycles seems a good viewpoint to support keeping it in a separate repository. IMO, the placement of documentation should keep consistency with the repository. Best, Vino Timo Walther 于2019年10月16日周三 下午4:02写道: > Hi Stephan, > > +1 for keeping it in a separate repository for

Re: [VOTE] Drop Python 2 support for 1.10

2019-10-16 Thread vino yang
+1 Bowen Li 于2019年10月16日周三 上午5:12写道: > +1 > > On Sun, Oct 13, 2019 at 10:54 PM Hequn Cheng wrote: > > > +1 > > > > Thanks a lot for driving this, Dian! > > > > On Mon, Oct 14, 2019 at 1:46 PM jincheng sun > > wrote: > > > > > +1 > > > > > > Dian Fu 于2019年10月14日周一 下午1:21写道: > > > > > > > Hi

Re: [VOTE] FLIP-64: Support for Temporary Objects in Table module

2019-10-15 Thread vino yang
+1 Best, Vino Aljoscha Krettek 于2019年10月15日周二 下午4:31写道: > +1 > > Best, > Aljoscha > > > On 14. Oct 2019, at 14:55, Kurt Young wrote: > > > > +1 > > > > Best, > > Kurt > > > > > > On Fri, Oct 11, 2019 at 1:39 PM Dawid Wysakowicz > > > wrote: > > > >> Hi everyone, > >> I would like to start a

Re: [VOTE] FLIP-74: Flink JobClient API

2019-10-15 Thread vino yang
+1 (non-binding) Best, Vino Aljoscha Krettek 于2019年10月15日周二 下午2:59写道: > +1 (binding) > > Best, > Aljoscha > > > On 15. Oct 2019, at 04:01, Zili Chen wrote: > > > > Hi all, > > > > +1 from my side. > > > > Given the current state of this voting thread, FLIP-74 is accepted > > with 3 binding

Re: Ignore operator failure

2019-10-14 Thread vino yang
Hi Dom, If you consider ignoring checkpoint failures, you can use this API: setTolerableCheckpointFailureNumber[1]. But for Jobs with checkpoints enabled and failed operators containing states, Flink can't ignore these failures without restarting Jobs. Subsequent regional recovery may be

Re: [PROPOSAL] Contribute Stateful Functions to Apache Flink

2019-10-14 Thread vino yang
+1 to add Stateful Function to flink core to let it stay in the Flink repository. Best, Vino Stephan Ewen 于2019年10月14日周一 下午7:29写道: > Thank you all for the encouraging feedback! So far the reaction to add this > to Flink was exclusively positive, which is really great to see! > > To make this

Call for approving Elasticsearch 7.x connector

2019-09-09 Thread vino yang
Hi guys, There is an issue about supporting Elasticsearch 7.x.[1] Based on our validation and discussion. We found that Elasticsearch 7.x does not guarantee API compatibility. Therefore, it does not have the ability to provide a universal connector like Kafka. It seems that we have to provide a

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

2019-09-07 Thread vino yang
Congratulations Klou! Best, Vino zhijiang 于2019年9月7日周六 上午12:22写道: > Congratulations Klou! > > Best, > Zhijiang > -- > From:Zhu Zhu > Send Time:2019年9月6日(星期五) 17:19 > To:dev > Subject:Re: [ANNOUNCE] Kostas Kloudas joins the Flink

Re: [VOTE] FLIP-62: Set default restart delay for FixedDelay- and FailureRateRestartStrategy to 1s

2019-09-05 Thread vino yang
+1 (non-binding) Best, Vino Yu Li 于2019年9月6日周五 上午2:13写道: > +1 (non-binding) > > Best Regards, > Yu > > > On Thu, 5 Sep 2019 at 00:23, zhijiang > wrote: > > > +1 > > > > Best, > > Zhijiang > > -- > > From:Jark Wu > > Send

Re: [ANNOUNCE] Java 11 cron builds activated on master

2019-09-05 Thread vino yang
Great news Chesnay! Thanks for your hard work and effort! Best, Vino Chesnay Schepler 于2019年9月5日周四 下午9:58写道: > Hello everyone, > > I just wanted to inform everyone that we now run Java 11 builds on > Travis as part of the cron jobs, subsuming the existing Java 9 tests. > All existing Java 9

Re: Re: [DISCUSS] Support JSON functions in Flink SQL

2019-09-05 Thread vino yang
+1 to have JSON functions in Flink SQL JingsongLee 于2019年9月5日周四 下午4:46写道: > +1 > Nice document. I think it is easier to do after expression reworking[1]. > By the way, which planner do you want to start? > > [1] >

Re: [VOTE] FLIP-61 Simplify Flink's cluster level RestartStrategy configuration

2019-09-05 Thread vino yang
+1 (non-binding) Zili Chen 于2019年9月5日周四 上午10:55写道: > +1 > > > zhijiang 于2019年9月5日周四 上午12:36写道: > > > +1 > > -- > > From:Till Rohrmann > > Send Time:2019年9月4日(星期三) 13:39 > > To:dev > > Cc:Zhu Zhu > > Subject:Re: [VOTE] FLIP-61

Re: [ANNOUNCE] Kinesis connector becomes part of Flink releases

2019-09-03 Thread vino yang
Good news! Thanks for your efforts, Bowen! Best, Vino Yu Li 于2019年9月2日周一 上午6:04写道: > Great to know, thanks for the efforts Bowen! > > And I believe it worth a release note in the original JIRA, wdyt? Thanks. > > Best Regards, > Yu > > > On Sat, 31 Aug 2019 at 11:01, Bowen Li wrote: > >> Hi

Re: State of FLIPs

2019-09-03 Thread vino yang
Hi Chesnay, I created FLIP-44 two months ago. Originally, the discussion of local aggregation feature happened in the ML thread.[1] After many people agree with this feature, I created an umbrella issue and split into some subtasks and opened one PR. However, @Aljoscha Krettek thinks it's a big

Re: [DISCUSS] FLIP-60: Restructure the Table API & SQL documentation

2019-09-02 Thread vino yang
Agree with Dawid's suggestion about function. Having a Functions section to unify the built-in function and UDF would be better. Dawid Wysakowicz 于2019年8月30日周五 下午7:43写道: > +1 to the idea of restructuring the docs. > > My only suggestion to consider is how about moving the >

Re: [VOTE] FLIP-54: Evolve ConfigOption and Configuration

2019-09-02 Thread vino yang
+1 Dawid Wysakowicz 于2019年8月30日周五 下午7:34写道: > +1 to the design > > On 29/08/2019 15:53, Timo Walther wrote: > > I converted the mentioned Google doc into a wiki page: > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-54%3A+Evolve+ConfigOption+and+Configuration > > > > > > The

Re: [DISCUSS] Use Java's Duration instead of Flink's Time

2019-08-23 Thread vino yang
+1 to replace the Time class provided by Flink with Java's Duration: - Java's Duration has better representation than the Flink's Time class; - As a built-in Java class, Duration class has a clear advantage over Java's Time class when interacting with other Java APIs and third-party

Re: CiBot Update

2019-08-21 Thread vino yang
+1 for "@flinkbot run travis", it is very convenient. Chesnay Schepler 于2019年8月21日周三 下午9:12写道: > Hi everyone, > > this is an update on recent changes to the CI bot. > > > The bot now cancels builds if a new commit was added to a PR, and > cancels all builds if the PR was closed. > (This was

Re: [VOTE] Apache Flink 1.9.0, release candidate #3

2019-08-20 Thread vino yang
+1 (non-binding) - checkout source code and build successfully - started a local cluster and ran some example jobs successfully - verified signatures and hashes - checked release notes and post Best, Vino Stephan Ewen 于2019年8月21日周三 上午4:20写道: > +1 (binding) > > - Downloaded the binary release

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

2019-08-14 Thread vino yang
Congratulations Andrey! Best, Vino Yun Gao 于2019年8月15日周四 上午10:49写道: > Congratulations Andrey! > > Best, > Yun > > > -- > From:Congxian Qiu > Send Time:2019 Aug. 15 (Thu.) 10:28 > To:dev@flink.apache.org > Subject:Re: [ANNOUNCE]

Re: [VOTE] Flink Project Bylaws

2019-08-13 Thread vino yang
+1 Tzu-Li (Gordon) Tai 于2019年8月13日周二 下午6:32写道: > +1 > > On Tue, Aug 13, 2019, 12:31 PM Hequn Cheng wrote: > > > +1 (non-binding) > > > > Thanks a lot for driving this! Good job. @Becket Qin < > becket@gmail.com > > > > > > > Best, Hequn > > > > On Tue, Aug 13, 2019 at 6:26 PM Stephan Ewen

Re: [ANNOUNCE] Hequn becomes a Flink committer

2019-08-07 Thread vino yang
Congratulations! highfei2...@126.com 于2019年8月7日周三 下午7:09写道: > Congrats Hequn! > > Best, > Jeff Yang > > > Original Message > Subject: Re: [ANNOUNCE] Hequn becomes a Flink committer > From: Piotr Nowojski > To: JingsongLee > CC: Biao Liu ,Zhu Zhu ,Zili Chen ,Jeff Zhang ,Paul

Re: [VOTE] Publish the PyFlink into PyPI

2019-08-01 Thread vino yang
+1 (non-binding) Jeff Zhang 于2019年8月1日周四 下午4:33写道: > +1 (non-binding) > > Stephan Ewen 于2019年8月1日周四 下午4:29写道: > > > +1 (binding) > > > > On Thu, Aug 1, 2019 at 9:52 AM Dian Fu wrote: > > > > > Hi Jincheng, > > > > > > Thanks a lot for driving this. > > > +1 (non-binding). > > > > > > Regards,

Re: [ANNOUNCE] Kete Young is now part of the Flink PMC

2019-07-23 Thread vino yang
Congratulations Kurt! Bo WANG 于2019年7月23日周二 下午7:13写道: > Congratulations Kurt! > > > Best, > > Bo WANG > > > On Tue, Jul 23, 2019 at 5:24 PM Robert Metzger > wrote: > > > Hi all, > > > > On behalf of the Flink PMC, I'm happy to announce that Kete Young is now > > part of the Apache Flink

Re: Re: [ANNOUNCE] Zhijiang Wang has been added as a committer to the Flink project

2019-07-22 Thread vino yang
Congratulations Zhijiang! Haibo Sun 于2019年7月23日周二 上午10:48写道: > Congrats, Zhejiang! > > > Best, > Haibo > 在 2019-07-23 10:26:20,"Yun Tang" 写道: > >Congratulations Zhijiang, well deserved. > > > >Best > > > >From: Yingjie Cao > >Sent: Tuesday, July 23, 2019 10:23

Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project

2019-07-18 Thread vino yang
Congratulations! Best, Vino Yun Gao 于2019年7月18日周四 下午5:31写道: > Congratulations! > > Best, > Yun > > > -- > From:Kostas Kloudas > Send Time:2019 Jul. 18 (Thu.) 17:30 > To:dev > Subject:Re: [ANNOUNCE] Jiangjie (Becket) Qin has been

Re: [ANNOUNCE] Rong Rong becomes a Flink committer

2019-07-11 Thread vino yang
congratulations Rong Rong! Fabian Hueske 于2019年7月11日周四 下午10:25写道: > Hi everyone, > > I'm very happy to announce that Rong Rong accepted the offer of the Flink > PMC to become a committer of the Flink project. > > Rong has been contributing to Flink for many years, mainly working on SQL > and

Re: [DISCUSS] Make window state queryable

2019-07-04 Thread vino yang
. > +1 from my side. > > > 在 2019年7月4日,11:41,vino yang 写道: > > > > Hi folks, > > > > Currently, the queryable state is not widely used in production. IMO, > there > > are two key reasons caused this result. 1) the client of the queryable > &g

Re: [VOTE] Migrate to sponsored Travis account

2019-07-04 Thread vino yang
+1 Dian Fu 于2019年7月4日周四 下午7:09写道: > +1. Thanks Chesnay and Bowen for pushing this forward. > > Regards, > Dian > > > 在 2019年7月4日,下午6:28,zhijiang 写道: > > > > +1 and thanks for Chesnay' work on this. > > > > Best, > > Zhijiang > > > >

Re: [DISCUSS] Flink framework and user log separation

2019-07-04 Thread vino yang
ing that can just be done by the right logging framework and > configuration? > > Like having a log framework with two targets, one filtered on > "org.apache.flink" and the other one filtered on "my.company.project" or > so? > > On Fri, Mar 1, 2019 at 3:

Re: RE: [DISCUSS] Improve Queryable State and introduce aQueryServerProxy component

2019-07-04 Thread vino yang
ome comments on it. > > > Best Regards, > > Jiayi Liao > > Original Message > *Sender:* vino yang > *Recipient:* Georgi Stoyanov > *Cc:* dev; user; Stefan > Richter; Aljoscha Krettek; > kklou...@gmail.com; Stephan Ewen; > l...@apache.org; Tzu-Li (Gordon) Tai &

[DISCUSS] Make window state queryable

2019-07-03 Thread vino yang
Hi folks, Currently, the queryable state is not widely used in production. IMO, there are two key reasons caused this result. 1) the client of the queryable state is hard to use. Because it requires users to know the address of TaskManager and the port of the proxy. Actually, most business users

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-07-03 Thread vino yang
ign, and we should first focus to > discuss around this. Previously, we actually had a wrong target, which > focused on how to do proper *local aggregation*. > > If the community is OK with this changes, we can further discuss how do we > do the changes step by step. If the community

Re: RE: [DISCUSS] Improve Queryable State and introduce a QueryServerProxy component

2019-07-02 Thread vino yang
> It will be very useful if someone from the committers joins the topic and > give us some insights what’s going to happen with that feature. > > > > > > Kind Regards, > > Georgi > > > > > > > > *From:* vino yang > *Sent:* Thursday, April 2

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-30 Thread vino yang
t; state to support local aggregation as Kurt and others said before). It will > be easier to push a thin FLIP forward. > > At last, I think "localKeyBy()" returning "LocalKeyedStream" may be a good > way to start. However, we still need to figure out how to app

Re: [ANNOUNCE] Weekly Community Update 2019/25

2019-06-28 Thread vino yang
Hi Robert, I like this idea. Having a Flink "monthly community update" could let the users and developers know more about the community. So Big +1 from my side. Robert Metzger 于2019年6月28日周五 下午3:51写道: > Thank you for the update! I always enjoy reading them. > > Maybe we could use these threads

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-27 Thread vino yang
y().process()` is something called "local process", > not just "local aggregate". > It needs more discussion about local keyed state, and I would like to put > it out of this FLIP. > > > Regards, > Jark > > > On Thu, 27 Jun 2019 at 13:03, vino yang w

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-26 Thread vino yang
examples in the "Public Interfaces" section of the FLIP-44 documentation. Let us first know the API through its use examples. Any feedback and questions please let me know. Best, Vino vino yang 于2019年6月27日周四 下午12:51写道: > Hi Jark, > > `DataStream.localKeyBy().process()` has

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-26 Thread vino yang
ion. Having >> "stream.enableLocalAggregation(Trigger)” that might add some implicit >> operators in the job graph can cause problems with savepoint/checkpoint >> compatibility. >> >> However I haven’t thought about/looked into the details of the Vino’s API >&

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-26 Thread vino yang
EventTimeTrigger.of(Time.of(Time.minutes(1)))". Then it will > be optmized into >local operator + final operator, and local operator will combine records > every minute on event time. > - In this way, there is only one line added, and the output is the same > with befo

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-25 Thread vino yang
80% of your FLIP content is actually describing how do we support > local > > keyed state. I don't know if this is necessary > > to introduce at the first step and we should also involve committers work > > on state backend to share their thoughts. > > > > Best, >

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-24 Thread vino yang
data ? Or we have a > mechanism to handle this ? > > > Thanks, > Simon > On 06/25/2019 10:56,vino yang wrote: > Hi Simon, > > Good question! > > For event time semantics, we reuse the window operator can keep the correct > behavior which is the same as the curr

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-24 Thread vino yang
it mean that > The final aggregation will wait forever ? > > > Thanks, > Simon > > > On 06/18/2019 10:06,vino yang wrote: > Hi Jark, > > We have done a comparative test. The effect is obvious. > > From our observation, the optimized effect mainly depends on two factors: &

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

2019-06-24 Thread vino yang
Congratulations Jincheng! Jark Wu 于2019年6月25日周二 上午9:28写道: > Congratulations Jincheng! Well deserved! > > On Tue, 25 Jun 2019 at 01:38, Bowen Li wrote: > > > Congratulations! > > > > On Mon, Jun 24, 2019 at 9:53 AM Yun Tang wrote: > > > > > Congratulations Jincheng! > > > > > > Best > > > Yun

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-24 Thread vino yang
nk we should definitely more thoughts > about this topic. > > I also try to loop in Stephan for this discussion. > > Best, > Kurt > > > On Mon, Jun 24, 2019 at 3:26 PM vino yang wrote: > > > Hi all, > > > > I am happy we have a wonderful discussion

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-24 Thread vino yang
a configurable API for users to choose how to handle intermediate result; The above three points have been updated in the design doc. Any questions, please let me know. @Aljoscha Krettek What do you think? Any further comments? Best, Vino vino yang 于2019年6月20日周四 下午2:02写道: > Hi Kurt, > &g

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-06-21 Thread vino yang
essfully on > 2019-05-10. Everything goes well until 2019-05-10 and no more emails come > again from the mailing list. > > Best, Hequn > > On Fri, Jun 21, 2019 at 12:56 PM vino yang wrote: > > > Hi Kurt, > > > > I have copied my reply to the Jira issue o

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-06-20 Thread vino yang
you also share this information with apache > INFRA? Maybe we can find a workable solution together. > You can try to leave comments in this jira: > https://issues.apache.org/jira/browse/INFRA-18249) > > Best, > Kurt > > > On Fri, Jun 21, 2019 at 11:45 AM vino yang wrote: &

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-06-20 Thread vino yang
to the QQ mailbox to facilitate the location problem. The attached file contains a sample of spam that was rejected and returned to the QQ mailbox. Best, Vino vino yang 于2019年6月21日周五 上午10:16写道: > Hi Robert, > > Yes, QQ mail product belongs to Tencent and I work at Tencent. > > I a

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-06-20 Thread vino yang
o Tencent, right? > As far as I know, we have some active contributors working at Tencent (Vino > Yang). Maybe he or other employees from Tencent following this mailing > list, could help to make a connection to the QQ teams to resolve that > problem? > > > > On Thu, Jun 20,

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-20 Thread vino yang
ion engine, or we > just want to introduce a new concept called LocalKeyBy? > > [1]: https://github.com/apache/flink/pull/4626 > > Best, > Kurt > > > On Wed, Jun 19, 2019 at 5:13 PM vino yang wrote: > > > Hi Hequn, > > > > Thanks for your comments! > &g

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-19 Thread vino yang
enable all > functionalities for localKeyBy from KeyedStream? > > Great to also have your opinions. > > Best, Hequn > > > > > On Wed, Jun 19, 2019 at 10:24 AM vino yang wrote: > > > Hi Kurt and Piotrek, > > > > Thanks for your comments. > > > > I ag

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-18 Thread vino yang
on > > > need > > > to do is output all the partial results, it's purely cpu workload, not > > > introducing any IO. I want to point out that even if we have this > > > cost, we reduced another barrier align cost of the operator, which is > the > > > syn

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-18 Thread vino yang
mprove the aggregate performance. > > Let me know if you have any other questions. > > Best, > Kurt > > > On Tue, Jun 18, 2019 at 1:29 PM vino yang wrote: > > > Hi Kurt, > > > > Thanks for your reply. > > > > Actually, I am not against you to raise yo

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-17 Thread vino yang
> > > local aggregation operator but it's not performed as well as the later > > one, > > > and also effect the barrie > > > alignment time. The later one is fairly simple and more efficient. > > > > > > I would highly suggest you to consider

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-17 Thread vino yang
alKeyBy(0).countWindow(5).sum(1).keyBy(0).sum(1)", have you > done > > some benchmark? > > Because I'm curious about how much performance improvement can we get by > > using count window as the local operator. > > > > Best, > > Jark > > > > > >

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-17 Thread vino yang
ot;input.localKeyBy(0).countWindow(5).sum(1).keyBy(0).sum(1)", have you done > some benchmark? > Because I'm curious about how much performance improvement can we get by > using count window as the local operator. > > Best, > Jark > > > > On Mon, 17 Jun 2019 at

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-17 Thread vino yang
um(1) > > Would also be great if we can add this into the document. Thank you very > much. > > Best, Hequn > > > On Fri, Jun 14, 2019 at 11:34 AM vino yang wrote: > > > Hi Aljoscha, > > > > I have looked at the "*Process*" section of FLIP wiki p

Re: About Deprecating split/select for DataStream API

2019-06-17 Thread vino yang
Hi, I also think it is valuable and reasonable to keep the split/select APIs. They are very convenient and widely used in our platform. I think they are also used in other users' jobs. If the community has doubts about this, IMHO, it would be better to start a user survey. Best, Vino SHI

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-13 Thread vino yang
1008284.n3.nabble.com/DISCUSS-Support-Local-Aggregation-in-Flink-td29307.html#a29308 leesf 于2019年6月13日周四 上午9:19写道: > +1 for the FLIP, thank vino for your efforts. > > Best, > Leesf > > vino yang 于2019年6月12日周三 下午5:46写道: > > > Hi folks, > > > > I wou

Re: [DISCUSS] Support Local Aggregation in Flink

2019-06-12 Thread vino yang
to the new thread. Best, Vino vino yang 于2019年6月12日周三 上午11:33写道: > Hi Aljoscha, > > I am happy to create a FLIP and have a voting process for this feature. I > have already sent a mail to apply for the wiki permissions. > > Once I get the permission, will start the next step. Whe

Re: [DISCUSS] Deprecate previous Python APIs

2019-06-12 Thread vino yang
+1 from my side Best, Vino Terry Wang 于2019年6月12日周三 下午5:45写道: > +1 for deprecation. It’s very reasonable. > > 在 2019年6月12日,下午5:32,Till Rohrmann 写道: > > +1 for deprecation. > > Cheers, > Till > > On Wed, Jun 12, 2019 at 4:31 AM Hequn Cheng wrote: > >> +1 on the proposal! >> Maintaining only

[DISCUSS] FLIP-44: Support Local Aggregation in Flink

2019-06-12 Thread vino yang
Hi folks, I would like to start the FLIP discussion thread about supporting local aggregation in Flink. In short, this feature can effectively alleviate data skew. This is the FLIP: https://cwiki.apache.org/confluence/display/FLINK/FLIP-44%3A+Support+Local+Aggregation+in+Flink *Motivation*

Re: Apply for confluence Wiki create and edit permission

2019-06-12 Thread vino yang
Hi Aljoscha, Thanks! Aljoscha Krettek 于2019年6月12日周三 下午3:25写道: > I gave you the permissions  > > > On 12. Jun 2019, at 05:26, vino yang wrote: > > > > Hi, > > > > I am going to create a new FLIP for Proposal of supporting local > > aggregati

Re: [DISCUSS] Support Local Aggregation in Flink

2019-06-11 Thread vino yang
is a bad proposal, just that I don’t > know yet, and others might not know as well. > > Best, > Aljoscha > > [1] > https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals > > On 10. Jun 2019, at 04:30, vino yang wrote: > > Hi all, > >

Apply for confluence Wiki create and edit permission

2019-06-11 Thread vino yang
Hi, I am going to create a new FLIP for Proposal of supporting local aggregation in Flink. The discussion thread in the Flink dev mailing list is here.[1] Could you please give me the create and edit permission for this page[2]. My ID is: yanghua Best, Vino [1]:

Re: [DISCUSS] Allow at-most-once delivery in case of failures

2019-06-11 Thread vino yang
+1 from my side to support this feature in Flink. Best, Vino Biao Liu 于2019年6月11日周二 下午6:14写道: > Hi Piotrek, > I agree with you that there are strained resources of community to support > such a feature. I was planing to start a similar discussion after 1.9 > released. Anyway we don't have

Re: [DISCUSS] Support Local Aggregation in Flink

2019-06-09 Thread vino yang
Hi all, Thanks for all the feedback and comments. Since the thread of this feature has been presented about one week in the dev mailing list and has got much support from the community, I have created a new JIRA feature issue[1] to track it and I will split subtasks soon. We can move further

Re: [DISCUSS] Support Local Aggregation in Flink

2019-06-05 Thread vino yang
Hi Aljoscha, What do you think about this feature and design document? Best, Vino vino yang 于2019年6月5日周三 下午4:18写道: > Hi Dian, > > I still think your implementation is similar to the window operator, you > mentioned the scalable trigger mechanism, the window API also can customi

  1   2   >