Hi Folks,
We are continuing to work on CI stabilization and will cut the release once we
stabilize the builds hopefully tonight/tomorrow.
Thanks,Balaji.V
On Tuesday, August 11, 2020, 09:15:05 PM PDT, Vinoth Chandar
<vin...@apache.org> wrote:
Hello all,
Update on this. We have landed most of the blockers for the 0.6.0 release
and I am currently working on the last major blocker, HUDI-1013.
We are working through some unexpected CI flakiness. We hope to stabilize
master, cut the RC, and then open up master for regular PR merges.
ETA for this is tomorrow night PST (Aug 12, PST).
We will keep this thread posted!
Thanks
Vinoth
On Tue, Aug 4, 2020 at 9:47 PM Vinoth Chandar <vin...@apache.org> wrote:
> Small correction:
>
> >> Vinoth working on code review, tests for PR 1876,
> This is landed!
>
>
> On Tue, Aug 4, 2020 at 9:44 PM Bhavani Sudha <bhavanisud...@gmail.com>
> wrote:
>
>> Hello all,
>>
>> We are targeting the end of this week to cut RC. Here is an update of
>> where
>> we are at release blockers.
>>
>> 0.6.0 Release blocker status (board
>> <
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=397&projectKey=HUDI&view=detail&selectedIssue=HUDI-69
>> >)
>> ,
>>
>> - Spark Datasource/MOR https://github.com/apache/hudi/pull/1848 needs
>> to
>> be tested by gary/balaji (About to land)
>> - Hive Sync restructuring (Review done, about to land)
>> - Bootstrap
>> - Vinoth working on code review, tests for PR 1876,
>> - then udit will rework PR 1702 (In Code review)
>> - then we will review, land PR 1870, 1869
>> - Bulk insert V2 PR 1834, lower risk, independent PR, well tested
>> already
>> - Dependent PR 1149 to be landed,
>> - and modes to be respected in V2 impl as well (At risk)
>> - Upgrade Downgrade Hooks, PR 1858 : (In Code review)
>> - HUDI-1054- Marker list perf improvement, Udit has a PR out
>> - HUDI-115 : Overwrite with... ordering issue, Sudha has a PR nearing
>> landing
>> - HUDI-1098 : Marker file issue with non-existent files. (In Code
>> review)
>> - Spark Streaming + Async Compaction , test complete, code review
>> comments and land PR 1752 (About to land)
>> - Spark DataSource/Hive MOR Incremental Query HUDI-920 (At risk)
>> - Flink/Multi Engine refactor, will need a large rebase and rework,
>> review, land (At risk for 0.6.0)
>> - BloomIndex V2 - Global index implementation. (At risk)
>> - HUDI-845 : Parallel writing i.e allow multiple writers (Pushed out of
>> 0.6.0)
>> - HUDI-860 : Small File Handling without memory caching (Pushed out of
>> 0.6.0)
>>
>>
>> Thanks,
>> Sudha
>>
>> On Mon, Aug 3, 2020 at 3:41 PM Vinoth Chandar <vin...@apache.org> wrote:
>>
>> > +1 (we need to formalize this well)
>> > But having just blockers land first, would help not just with rebasing,
>> but
>> > also wind down towards cutting an RC by end of week.
>> >
>> >
>> > On Mon, Aug 3, 2020 at 2:53 PM Bhavani Sudha <bhavanisud...@gmail.com>
>> > wrote:
>> >
>> > > Hello all,
>> > >
>> > > As we are all hustling towards getting the blockers in, I wanted to
>> > propose
>> > > a code/merge freeze until we cut a release for 0.6.0 and restrict it
>> to
>> > > only merging blockers identified for this release. It would reduce
>> > rebasing
>> > > time for blockers in progress. If we feel some issue is a serious
>> blocker
>> > > we can discuss it here and bump it's priority.
>> > >
>> > > Please share your thoughts or concerns.
>> > >
>> > > Thanks,
>> > > Sudha
>> > >
>> > >
>> > > On Mon, Aug 3, 2020 at 8:19 AM Vinoth Chandar <vin...@apache.org>
>> wrote:
>> > >
>> > > > Given enough time has passed, Sudha can be our RM for 0.6.0.
>> > > >
>> > > > On the release blocker progress, we landed few blockers over the
>> > weekend,
>> > > > with some almost ready for landing
>> > > >
>> > > > Will send out a status update again tomorrow night PST!
>> > > >
>> > > > On Mon, Aug 3, 2020 at 8:17 AM Vinoth Chandar <vin...@apache.org>
>> > wrote:
>> > > >
>> > > > > Hi anton.
>> > > > >
>> > > > > We were hoping to cut a release by last weekend. New target is
>> this
>> > > > > weekend!
>> > > > > (tbh we were thrown off a bit due to COVID in Q2, given a lot of
>> > > > > PMC/Committers had additional kid care duties. Now we are back to
>> > > normal
>> > > > > cadence)
>> > > > >
>> > > > > Going forward, I plan to start a discussion around planning,
>> > > prioritizing
>> > > > > and other release processes after 0.6.0. Would be great to have
>> the
>> > > > > community weigh in even more in these things upfront.
>> > > > >
>> > > > > Thanks
>> > > > > Vinoth
>> > > > >
>> > > > > On Fri, Jul 31, 2020 at 6:49 PM Anton Zuyeu <
>> anton.zu...@gmail.com>
>> > > > wrote:
>> > > > >
>> > > > >> Hi All,
>> > > > >>
>> > > > >> I apologize for possibly dumb question but when was 0.6.0
>> planned to
>> > > be
>> > > > >> released? Can't find any dates on Hudi related pages.
>> > > > >>
>> > > > >> On Thu, Jul 30, 2020 at 10:36 AM Vinoth Chandar <
>> vin...@apache.org>
>> > > > >> wrote:
>> > > > >>
>> > > > >> > Is anyone able to help with the at risk items? :)
>> > > > >> >
>> > > > >> > On Thu, Jul 30, 2020 at 7:07 AM leesf <leesf0...@gmail.com>
>> > wrote:
>> > > > >> >
>> > > > >> > > @Vinoth Chandar <vin...@apache.org> Thanks for the reminder,
>> > > marked
>> > > > >> to
>> > > > >> > > blocker, and next week would be ok to me.
>> > > > >> > >
>> > > > >> > > Vinoth Chandar <vin...@apache.org> 于2020年7月30日周四 上午11:35写道：
>> > > > >> > >
>> > > > >> > > > @leesf <leesf0...@gmail.com> can we please mark the
>> relevant
>> > > > >> > ticket(s)
>> > > > >> > > > with blocker priority, so it's easier to track?
>> > > > >> > > >
>> > > > >> > > > Looks like we are nearing a choice for RM.
>> > > > >> > > > Any more thoughts on timelines? Looks like everyone so far
>> is
>> > > > >> leaning
>> > > > >> > > > towards completeness of the release over doing it sooner?
>> > > > >> > > >
>> > > > >> > > > On Wed, Jul 29, 2020 at 6:36 PM vino yang <
>> > > yanghua1...@gmail.com>
>> > > > >> > wrote:
>> > > > >> > > >
>> > > > >> > > > > +1 on Sudha being RM for the release. And looking
>> forward to
>> > > > >> 0.6.0.
>> > > > >> > > > >
>> > > > >> > > > > Best,
>> > > > >> > > > > Vino
>> > > > >> > > > >
>> > > > >> > > > > leesf <leesf0...@gmail.com> 于2020年7月30日周四 上午9:15写道：
>> > > > >> > > > >
>> > > > >> > > > > > +1 on Sudha on being RM, and PR#1810
>> > > > >> > > > > > https://github.com/apache/hudi/pull/1810 (abstract
>> hive
>> > > sync
>> > > > >> > module)
>> > > > >> > > > > would
>> > > > >> > > > > > also goes to this release.
>> > > > >> > > > > >
>> > > > >> > > > > > Sivabalan <n.siv...@gmail.com> 于2020年7月30日周四 上午2:18写道：
>> > > > >> > > > > >
>> > > > >> > > > > > > +1 on Sudha being RM for the release. Makes sense to
>> > push
>> > > > the
>> > > > >> > > release
>> > > > >> > > > > by
>> > > > >> > > > > > a
>> > > > >> > > > > > > week.
>> > > > >> > > > > > >
>> > > > >> > > > > > > On Wed, Jul 29, 2020 at 1:35 AM vbal...@apache.org <
>> > > > >> > > > vbal...@apache.org
>> > > > >> > > > > >
>> > > > >> > > > > > > wrote:
>> > > > >> > > > > > >
>> > > > >> > > > > > > > +1 on Sudha on being RM for this release. Also
>> agree
>> > on
>> > > > >> > pushing
>> > > > >> > > > the
>> > > > >> > > > > > > > release date by a week.
>> > > > >> > > > > > > > Balaji.V
>> > > > >> > > > > > > > On Tuesday, July 28, 2020, 10:08:41 PM PDT,
>> > Bhavani
>> > > > >> Sudha <
>> > > > >> > > > > > > > bhavanisud...@gmail.com> wrote:
>> > > > >> > > > > > > >
>> > > > >> > > > > > > > Thanks Vinoth for the update. I can volunteer to
>> RM
>> > > this
>> > > > >> > > release.
>> > > > >> > > > > > > >
>> > > > >> > > > > > > > Understand 0.6.0 release is delayed than what we
>> > > > originally
>> > > > >> > > > > discussed.
>> > > > >> > > > > > Q2
>> > > > >> > > > > > > > has been really hard with COVID and everything
>> going
>> > on.
>> > > > >> Given
>> > > > >> > > that
>> > > > >> > > > > we
>> > > > >> > > > > > > are
>> > > > >> > > > > > > > at this point, I feel by delaying the RC by a week
>> or
>> > so
>> > > > >> more
>> > > > >> > if
>> > > > >> > > we
>> > > > >> > > > > can
>> > > > >> > > > > > > get
>> > > > >> > > > > > > > some of the 'At risk' items in, I would vote for
>> that.
>> > > > That
>> > > > >> is
>> > > > >> > > just
>> > > > >> > > > > my
>> > > > >> > > > > > > > personal opinion. I ll let others chime in.
>> > > > >> > > > > > > >
>> > > > >> > > > > > > > Thanks,
>> > > > >> > > > > > > > Sudha
>> > > > >> > > > > > > >
>> > > > >> > > > > > > > On Tue, Jul 28, 2020 at 9:48 PM Vinoth Chandar <
>> > > > >> > > vin...@apache.org>
>> > > > >> > > > > > > wrote:
>> > > > >> > > > > > > >
>> > > > >> > > > > > > > > Hello all,
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > > > Just wanted to kickstart a thread to firm up the
>> RC
>> > > cut
>> > > > >> date
>> > > > >> > > for
>> > > > >> > > > > > 0.6.0
>> > > > >> > > > > > > > and
>> > > > >> > > > > > > > > pick a RM. (any volunteers?, if not I self
>> nominate
>> > > > >> myself)
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > > > Here's an update on where we are at with the
>> > remaining
>> > > > >> > release
>> > > > >> > > > > > > blockers.
>> > > > >> > > > > > > > I
>> > > > >> > > > > > > > > have marked items as "At risk" assuming we cut RC
>> > > > sometime
>> > > > >> > next
>> > > > >> > > > > week.
>> > > > >> > > > > > > > > Please chime in with your thoughts. Ideally, we
>> > don't
>> > > > take
>> > > > >> > any
>> > > > >> > > > more
>> > > > >> > > > > > > > > blockers. If we also want to knock off the at
>> risk
>> > > > items,
>> > > > >> > then
>> > > > >> > > we
>> > > > >> > > > > > would
>> > > > >> > > > > > > > > at-least push dates by another week (my guess).
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > > > 0.6.0 Release blocker status (board
>> > > > >> > > > > > > > > <
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > >
>> > > > >> > > > > > >
>> > > > >> > > > > >
>> > > > >> > > > >
>> > > > >> > > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=397&projectKey=HUDI&view=detail&selectedIssue=HUDI-69
>> > > > >> > > > > > > > > >)
>> > > > >> > > > > > > > > ,
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > > > - Spark Datasource/MOR
>> > > > >> > > > https://github.com/apache/hudi/pull/1848
>> > > > >> > > > > > > needs
>> > > > >> > > > > > > > > to
>> > > > >> > > > > > > > > be tested by gary/balaji
>> > > > >> > > > > > > > > - Bootstrap
>> > > > >> > > > > > > > > - Vinoth working on code review, tests for
>> PR
>> > > 1876,
>> > > > >> > > > > > > > > - then udit will rework PR 1702
>> > > > >> > > > > > > > > - then we will review, land PR 1870, 1869
>> > > > >> > > > > > > > > - Also need to fix HUDI-999, HUDI-1021
>> > > > >> > > > > > > > > - Bulk insert V2 PR 1834, lower risk,
>> independent
>> > > PR,
>> > > > >> well
>> > > > >> > > > > tested
>> > > > >> > > > > > > > > already
>> > > > >> > > > > > > > > - Dependent PR 1149 to be landed,
>> > > > >> > > > > > > > > - and modes to be respected in V2 impl as
>> well
>> > > (At
>> > > > >> risk)
>> > > > >> > > > > > > > > - Upgrade Downgrade Hooks, PR 1858 : Siva has
>> a
>> > PR
>> > > > out,
>> > > > >> > code
>> > > > >> > > > > > > > completing
>> > > > >> > > > > > > > > this week
>> > > > >> > > > > > > > > - HUDI-1054- Marker list perf improvement,
>> Udit
>> > > has a
>> > > > >> PR
>> > > > >> > out
>> > > > >> > > > > > > > > - HUDI-115 : Overwrite with... ordering issue,
>> > > Sudha
>> > > > >> has a
>> > > > >> > > PR
>> > > > >> > > > > > > nearing
>> > > > >> > > > > > > > > landing
>> > > > >> > > > > > > > > - HUDI-1098 : Marker file issue with
>> non-existent
>> > > > >> files.
>> > > > >> > > Siva
>> > > > >> > > > to
>> > > > >> > > > > > > begin
>> > > > >> > > > > > > > > impl
>> > > > >> > > > > > > > > - Spark Streaming + Async Compaction , test
>> > > complete,
>> > > > >> code
>> > > > >> > > > > review
>> > > > >> > > > > > > > > comments and land PR 1752
>> > > > >> > > > > > > > > - Spark DataSource/Hive MOR Incremental Query
>> > > > HUDI-920
>> > > > >> (At
>> > > > >> > > > risk)
>> > > > >> > > > > > > > > - Flink/Multi Engine refactor, will need a
>> large
>> > > > rebase
>> > > > >> > and
>> > > > >> > > > > > rework,
>> > > > >> > > > > > > > > review, land (At risk for 0.6.0, high scope,
>> may
>> > > not
>> > > > >> have
>> > > > >> > > > enough
>> > > > >> > > > > > > time)
>> > > > >> > > > > > > > > - BloomIndex V2 - Global index implementation.
>> > (At
>> > > > >> risk)
>> > > > >> > > > > > > > > - HUDI-845 : Parallel writing i.e allow
>> multiple
>> > > > >> writers
>> > > > >> > (At
>> > > > >> > > > > risk)
>> > > > >> > > > > > > > > - HUDI-860 : Small File Handling without
>> memory
>> > > > caching
>> > > > >> > (At
>> > > > >> > > > > risk)
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > > > Thanks
>> > > > >> > > > > > > > > Vinoth
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > >
>> > > > >> > > > > > >
>> > > > >> > > > > > >
>> > > > >> > > > > > >
>> > > > >> > > > > > > --
>> > > > >> > > > > > > Regards,
>> > > > >> > > > > > > -Sivabalan
>> > > > >> > > > > > >
>> > > > >> > > > > >
>> > > > >> > > > >
>> > > > >> > > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
