Thanks for bubbling this up, Siva! Merge has not happened yet.
What you're saying makes sense to me -- after merging RFC-46 everyone will likely have to rebase their changes since there it touches quite a few core components, so it'd make it harder to cherry-pick 0.12.2 changes. What do other folks think? On Tue, Dec 6, 2022 at 2:06 PM Sivabalan <n.siv...@gmail.com> wrote: > Hey folks, > Has this happened already? If not, with 0.12.2 underway, is it > possible to delay this until end of this week. Not sure if code freeze for > 0.12.2 will be this Friday, but atleast we can do our best and ask RM to > cherry-pick the commits until Friday. Sorry, I also don't want to drag > RFC-46 anymore, open to hear others thoughts. > > On Mon, 5 Dec 2022 at 09:56, Alexey Kudinkin <ale...@onehouse.ai> wrote: > > > Hey, everyone! > > > > Long-awaited RFC-46 update: It took the crew quite a bit more time than > > originally anticipated to stabilize and make sure everything is working > as > > expected and now we believe it's finally ready for prime time! > > > > As such, we're going to execute on a following plan: > > > > 1. As called out originally, to move forward with the PR of this > > magnitude we will be instituting *temporary code-freeze* on master > > starting at *00:00 (midnight) 12/06 PST* and lasting for *24h. * > > 2. During that window we're planning to do one final rebase and land > it > > on master. > > 3. We will lift the code-freeze as soon as PR lands, but reserve 24h > for > > us to have ample buffer just in case. > > > > Let me know if you have any questions or concerns with this plan > > > > Alexey, on behalf of RFC-46 team > > > > On Wed, Sep 28, 2022 at 9:17 AM Alexey Kudinkin <ale...@onehouse.ai> > > wrote: > > > > > @Ken yes, that's the plan eventually -- to rely on Execution (Query) > > > Engines to provide their own representation that Hudi will be handling > > w/o > > > any intermediate transitions. If you're curious to learn more i'd > > encourage > > > you (and everyone) to check out the RFC-46 itself. > > > > > > In the Phase 1 though, we're only focusing on Spark integration for now > > > (as proof of concept) and then later on after the new infra is hardened > > > enough we can expand it to other engines as well. > > > > > > On Tue, Sep 27, 2022 at 7:07 PM Gary Li <yanjia.gary...@gmail.com> > > wrote: > > > > > >> Great work! Really excited about this feature. Kudos to RFC-46 team. > > >> > > >> Best, > > >> Gary > > >> > > >> On Wed, Sep 28, 2022 at 7:22 AM Ken Krugler < > > kkrugler_li...@transpac.com> > > >> wrote: > > >> > > >> > Hi Alexey, > > >> > > > >> > Thanks for the update! > > >> > > > >> > So for maximum performance when writing to Hudi from a low-level > > >> > (DataStream, not Table) Flink workflow, we’d be creating RowData > > >> records? > > >> > > > >> > — Ken > > >> > > > >> > > > >> > > On Sep 27, 2022, at 2:08 PM, Alexey Kudinkin <ale...@onehouse.ai> > > >> wrote: > > >> > > > > >> > > Hello, everyone! > > >> > > > > >> > > As you might be aware, community has been very busy at work on > > RFC-46 > > >> > > aiming to bring long-awaited cutting edge level of performance to > > >> Hudi by > > >> > > avoiding using Avro as an intermediate representation, instead > > >> relying on > > >> > > individual engines to host data in their own formats (InternalRow > > for > > >> > > Spark, RowData for Flink, etc) > > >> > > > > >> > > We wanted to share an update in terms of where we are and what are > > the > > >> > next > > >> > > steps from here: > > >> > > > > >> > > - We're very close to completing the work and are already > > preparing > > >> to > > >> > > be landing complete implementation of the Phase 1 of the RFC-46 > > >> > currently > > >> > > being developed in a feature branch > > >> > > <https://github.com/apache/hudi/tree/release-feature-rfc46> > > >> > > - To be able to successfully merge the change of such scale, we > > will > > >> > > have to do a *code freeze* for the master branch barring any > > >> changes to > > >> > > land before we're able to merge the feature-branch. > > >> > > - To make sure that this activity doesn't interrupt the 0.12.1 > > >> release > > >> > > that is currently in progress we're tentatively planning to > > schedule > > >> > this > > >> > > code-freeze *after* successful finalization of the release > process > > >> with > > >> > > RC branch being cut and validated for release. As of now, > provided > > >> RC > > >> > > candidate will be cut tomorrow on 09/28 we're aiming to > schedule a > > >> > merge > > >> > > attempt somewhere mid to late next week. > > >> > > - We will follow-up on this thread separately at least *24h* > > before > > >> the > > >> > > scheduled code-freeze with an exact date and time frame for it. > > Stay > > >> > tuned. > > >> > > > > >> > > > > >> > > Alexey, on behalf of the RFC-46 group > > >> > > > >> > -------------------------- > > >> > Ken Krugler > > >> > http://www.scaleunlimited.com > > >> > Custom big data solutions > > >> > Flink, Pinot, Solr, Elasticsearch > > >> > > > >> > > > >> > > > >> > > > >> > > > > > > > > -- > Regards, > -Sivabalan >
