Re: Re: [DISCUSS] New Flink Writer Proposal

2021-01-08 Thread Danny Chan
er is like an app, just like the delta >> > streamer. If we want to build another Flink writer, we can still share >> the >> > same flink client right? Does the flink client also have to use the new >> > feature only available on Flink 1.12? >> > >> > Than

Re: Re: [DISCUSS] New Flink Writer Proposal

2021-01-07 Thread Danny Chan
> > same flink client right? Does the flink client also have to use the new > > feature only available on Flink 1.12? > > > > Thanks, > > Gary Li > > ____________ > > From: Danny Chan > > Sent: Thursday, January 7, 2021 10:19 AM >

Re: Re: [DISCUSS] New Flink Writer Proposal

2021-01-07 Thread vino yang
available on Flink 1.12? > > Thanks, > Gary Li > > From: Danny Chan > Sent: Thursday, January 7, 2021 10:19 AM > To: dev@hudi.apache.org > Subject: Re: Re: [DISCUSS] New Flink Writer Proposal > > Thanks vino yang ~ > > IMO, we shou

Re: Re: [DISCUSS] New Flink Writer Proposal

2021-01-07 Thread Vinoth Chandar
; > Thanks, > Gary Li > > From: Danny Chan > Sent: Thursday, January 7, 2021 10:19 AM > To: dev@hudi.apache.org > Subject: Re: Re: [DISCUSS] New Flink Writer Proposal > > Thanks vino yang ~ > > IMO, we should not

Re: Re: [DISCUSS] New Flink Writer Proposal

2021-01-07 Thread Gary Li
From: Danny Chan Sent: Thursday, January 7, 2021 10:19 AM To: dev@hudi.apache.org Subject: Re: Re: [DISCUSS] New Flink Writer Proposal Thanks vino yang ~ IMO, we should not put much put too much energy for current Flink writer, it is not production-ready in the long run

Re: Re: [DISCUSS] New Flink Writer Proposal

2021-01-06 Thread Danny Chan
Thanks vino yang ~ IMO, we should not put much put too much energy for current Flink writer, it is not production-ready in the long run. There are so many features need to add/support for the Flink write/read(MOR write, COR read, MOR read, the new index), we should focus on one version first,

Re: Re: [DISCUSS] New Flink Writer Proposal

2021-01-05 Thread vino yang
Hi Danny, As we discussed in the doc, we should agree on if we should be compatible with the version less than Flink 1.11/1.12. We all know that there are some bottlenecks in the current plan. You proposed some improvements, yes it is great, but it radically uses the newer features provided by

Re: Re: [DISCUSS] New Flink Writer Proposal

2021-01-05 Thread vino yang
Hi Danny, You should have cwiki edit permission now. Any problems let me know. Best, Vino Danny Chan 于2021年1月6日周三 下午12:05写道: > Sorry ~ > > Forget to say that my Confluence ID is danny0405. > > It would be nice if any of you can help on this. > > Best, > Danny Chan > > Danny Chan 于2021年1月6日周三

Re: Re: [DISCUSS] New Flink Writer Proposal

2021-01-05 Thread Danny Chan
Sorry ~ Forget to say that my Confluence ID is danny0405. It would be nice if any of you can help on this. Best, Danny Chan Danny Chan 于2021年1月6日周三 下午12:00写道: > Hi, can someone give me the CWIKI permission so that i can update the > design details to that (maybe as a new RFC though ~). > >

Re: Re: [DISCUSS] New Flink Writer Proposal

2021-01-05 Thread Danny Chan
Hi, can someone give me the CWIKI permission so that i can update the design details to that (maybe as a new RFC though ~). wangxianghu 于2021年1月5日周二 下午2:43写道: > + 1, Thanks Danny! > I believe this new feature OperatorConrdinator in flink-1.11 will help > improve the current implementation > >