Re: [DISCUSS] [RFC] Hudi bundle standards

2022-10-24 Thread Shiyan Xu
Thanks Xinyao for raising the problem. Let's align more on the RFC to help clarify usage. Agree on the importance - the bundle artifacts are the user-facing components from this project. On Mon, Oct 10, 2022 at 4:44 PM 田昕峣 (Xinyao Tian) wrote: > Hi Shiyan, > > > Having carefully read the

Re: [Discuss] SCD-2 Payload

2022-10-24 Thread Alexey Kudinkin
Hey, hey, Fengjian! With the landing of the RFC-46 we'll be kick-starting a process of phasing out HoodieRecordPayload as an abstraction and instead migrating to HoodieRecordMerger interface. I'd recommend to base your design considerations off the new HoodieRecordMerger interface instead of

Re: [Discuss] SCD-2 Payload

2022-10-24 Thread Shiyan Xu
Interesting thoughts. Not sure if I fully understand this part: "generate 2 records in combineAndGetUpdateValue". the API is defined to return just 1 record? On Fri, Oct 21, 2022 at 1:07 AM 冯健 wrote: > Hi guys, > After reading this article with respect to how to implement SCD-2 with > Hudi

Re: [DISCUSS] Build tool upgrade

2022-10-24 Thread Shiyan Xu
Thank you all for the valuable inputs! I think we can close this topic for now, given the majority is leaning towards continuing with maven. On Mon, Oct 17, 2022 at 8:48 PM zhaojing yu wrote: > I have experienced some gradle development projects and want to share some > thoughts. > > The

Re: [ANNOUNCE] Apache Hudi 0.12.1 released

2022-10-24 Thread Shiyan Xu
Congrats! On Sun, Oct 23, 2022 at 3:57 PM Zhuoluo Yang wrote: > Congrats! > > Thanks, > Zhuoluo > > > leesf 于2022年10月20日周四 09:03写道: > > > Great job! > > > > Alexey Kudinkin 于2022年10月20日周四 03:45写道: > > > > > Thanks Zhaojing for masterfully navigating this release! > > > > > > On Wed, Oct 19,

Re: [PSA] CI flakiness in master branch

2022-10-24 Thread Shiyan Xu
CI is back to stably passing now. Thank you all. On Fri, Oct 21, 2022 at 11:20 PM Shiyan Xu wrote: > Amendment: cancel CI jobs for non-urgent PRs *when needed* > > On Fri, Oct 21, 2022 at 11:18 PM Shiyan Xu > wrote: > >> Hi all, >> >> We have seen CI passing rate being very low due to

Re: [Discuss] SCD-2 Payload

2022-10-24 Thread 冯健
to Raymond: now combineAndGetUpdateValue can only return one IndexedRecord, but in the case of SCD-2, both old and new records need to be stored. to Alexey: yeah, this feature should be designed on top of RFC-46. Can HoodieRecordMerger return 2 HoodieRecord in this case? On Tue, 25 Oct 2022