Re: DISCUSS RFC 6 - Add indexing support to the log file

ew tables or > >> when hudi version is stable > >> > >> > >>> On Sun, Oct 27, 2019 at 12:13 AM Vinoth Chandar > wrote: > >>> > >>> > >>> > >> > https://cwiki.apache.org/confluence/display/HUDI/RFC-6+Add+indexing+support+to+the+log+file > >>> > >>> > >>> Feedback welcome, on this RFC tackling HUDI-86 > >>> > >> > -- Regards, -Sivabalan

Re: [Discuss] Convenient time for weekly sync meeting

; > > > > > > > > Mon, Tue, Thursday - 9 p.m to 12 a.m PST > > > > > > > > > > > > > Wed - 5:00 to 6:00 am and 9:30 p.m to 12 a.m PST > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:31 PM Vinoth Chandar < > > > > > > > > vin...@apache.org> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Interested. > > > > > > > > > > > > > > Mon-Thu 5AM-6:30AM PST > > > > > > > > > > > > > > Mon-Thu 9PM-10:30PM PST > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:28 PM Bhavani Sudha < > > > > > > > > > > bhavanisud...@gmail.com > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hello all, > > > > > > > > > > > > > > > Currently the weekly sync meeting is scheduled > to > > > > run on > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Tuesdays > > > > > > > > > > from > > > > > > > > > > > > > > 9pm > > > > > > > > > > > > > > > PST to 10 pm PST. Given our users are from > > multiple > > > > time > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > zones, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > we > > > > > > > > > > can > > > > > > > > > > > > > > try > > > > > > > > > > > > > > > to see if there is any overlapping time that > > works > > > > best. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > chime > > > > > > > > > > > > > in > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > what would be a suitable time for you if you > are > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > interested in > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > attending > > > > > > > > > > > > > > > the weekly meetings. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > Sudha > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Regards, -Sivabalan

Re: EMR + HUDI

5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0=1m4UXNjVbRS_Qp6qjI1zUHFGQ8u3bUZyCQo-KkrDgLM=_B0PqvZladwQZwEFjj1rwFU1hkjz65hKZaY3lywdTv8= > > > > > > Thanks to our wonderful contributors from AWS (Udit & team) for > making it > > > happen > > > > > > Thanks > > > Vinoth > > > > > > -- Regards, -Sivabalan

Re: Re: Checkstyle changes?

t; best, > lamber-ken > > At 2019-12-13 03:49:12, "Sivabalan" wrote: > >thanks lamber-ken. Sorry, I wasn't paying close attention to these > changes. > >Don't we make a separate PR (with just the changes pertaining to new check > >style rules across entire repo

Re: Re: Re: [DISCUSS] Refactor of the configuration framework of hudi project

; >> Best, > >> >> lamber-ken > >> >> > >> >> > >> >> > >> >> At 2019-12-10 11:51:22, "vino yang" wrote: > >> >> >Hi Lamber, > >> >> > > >> >> >Thanks for the proposal. +1 from my side. > >> >> > > >> >> >When it comes to configuration, it will involve how we handle > >> deprecated > >> >> >configuration items in the future. In my opinion, we need to take > this > >> >> into > >> >> >consideration when designing. There are already some successful > >> practices > >> >> >for our reference. For example, Flink defines some deprecated > >> >> >configurations as FallbackKey[1]. Maybe we can learn from these > >> designs. > >> >> > > >> >> >WDYT？ > >> >> > > >> >> >[1]: > >> >> > > >> >> > >> > https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/configuration/FallbackKey.java > >> >> > > >> >> >Best, > >> >> >Vino > >> >> > > >> >> >lamberken 于2019年12月9日周一 下午11:19写道： > >> >> > > >> >> >> > >> >> >> > >> >> >> Hi, all > >> >> >> > >> >> >> > >> >> >> Currently, many configuration items and their default values are > >> >> dispersed > >> >> >> in the config file like HoodieWriteConfig. It’s very confused for > >> >> >> developers, and it's easy for developers to use them in a wrong > place > >> >> >> especially when there are more and more configuration items. If we > >> can > >> >> >> solve this, developers will benefit from it and the code structure > >> will > >> >> be > >> >> >> more concise. > >> >> >> > >> >> >> > >> >> >> I had create a JIRA[1] and a under discuss RFC[2] to explain how > to > >> >> solve > >> >> >> the problem, if you are interested in this, you can visit jira and > >> RFC > >> >> for > >> >> >> detail. Any comments and feedback are welcome, WDYT? > >> >> >> > >> >> >> > >> >> >> Best, > >> >> >> lamber-ken > >> >> >> > >> >> >> > >> >> >> [1] https://issues.apache.org/jira/projects/HUDI/issues/HUDI-375 > >> >> >> [2] > >> >> >> > >> >> > >> > https://cwiki.apache.org/confluence/display/HUDI/RFC-11+%3A+Refactor+of+the+configuration+framework+of+hudi+project > >> >> > >> > -- Regards, -Sivabalan

Re: [QUESTION] Handle record partition change

, > Raymond > -- Regards, -Sivabalan

Checkstyle changes?

I build: https://travis-ci.org/apache/incubator-hudi/jobs/624228722?utm_medium=notification_source=github_status - Do others face this issue or it is just me? - Can someone give some pointers on how to go about fixing this? -- Regards, -Sivabalan

Re: Checkstyle changes?

already fixed like 20 files and the list keeps growing. On Thu, Dec 12, 2019 at 10:14 AM lamberken wrote: > > > Hi, @Sivabalan > > The new ImportOrder rule split import statements into groups and groups > are separated by one blank line. > These groups are 1) org.apache.

Re: Re:Re: [DISCUSS] RFC-12 : Efficient migration of large parquet tables to Apache Hudi

t;> Please find the link for the RFC below. > >> > >> > >> > https://cwiki.apache.org/confluence/display/HUDI/RFC+-+12+%3A+Efficient+Migration+of+Large+Parquet+Tables+to+Apache+Hudi > >> > >> Please review and let me know your thoughts. > >> > >> Thanks, > >> Balaji.V > >> > -- Regards, -Sivabalan

Re: Re: Checkstyle changes?

Sure. Will do that. On Sat, Dec 14, 2019 at 8:10 AM Vinoth Chandar wrote: > Is this worth documenting for everyone? > > On Fri, Dec 13, 2019 at 4:19 PM Sivabalan wrote: > > > Thanks. I had some issue w/ local intellij check style configuration. > After > > fi

Re: Execute presto query from file in docker

Let me know if there is an option to pipe in commands from file w/o copying the file to container. On Wed, Oct 23, 2019 at 7:43 AM Sivabalan wrote: > I got it working. I had to copy the file to the container. thanks. > > On Wed, Oct 23, 2019 at 6:30 AM Bhavani Sudha > wrote:

Execute presto query from file in docker

-coordinator-1:8090 -f file.sql // file.sql contains multiple presto queries. -- Regards, -Sivabalan

Dec 17, 2019 Meeting notes

to join hangouts. Leesf will coordinate with other folks to figure a way out so that we can use some video conf tool where in all of us can join. -- Regards, -Sivabalan

Re: [QUESTION] Handle record partition change

wrote: > Hi Sivabalan, > > Sorry for the late reply. I now see that GLOBAL_BLOOM allows records to be > looked up in different partitions. This is indeed helpful in the situation > where the same record key gets updated on its partition path. > > Now I'm thinking when w

Re: [DISCUSS] Jira integration in slack ?

ult in a valid JIRA.. So far, > we > > > have > > > > > > created JIRAs when finding worthy issues there.. > > > > > > > > > > > > I was actually thinking about a slack to mailing list (only for > > > > #general) > > > > > > integration, where threads are mirrored to the mailing list for > > ease > > > of > > > > > > discovery.. Not sure if thats doable or even a good idea.. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Feb 4, 2020 at 2:38 PM Bhavani Sudha < > > > bhavanisud...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I was wondering if we should look into Slack Jira integration > to > > > > > capture > > > > > > > activities in Slack. Often times slack has a lot of useful > > > > > conversations > > > > > > > that can be tracked in a Jira for future reference. Any ideas > on > > > how > > > > > > other > > > > > > > projects handle this ? > > > > > > > > > > > > > > Thanks, > > > > > > > Sudha > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Regards, -Sivabalan

Re: Test coverage is now integrated to codecov.io

> latest master. > > > 2. Code coverage ran but reported as zero. Three was one diff (#1350) > > where > > > we saw this issue yesterday. This in general shouldn't happen. Could > have > > > been due to an outage in codecov website. I will be monitoring upcoming > > > diffs for the near future to see if this problem persists. Please ping > me > > > in the diff if you have any questions/concerns regarding code coverage. > > > > > > Thanks, > > > Ram > > > > > > -- Regards, -Sivabalan

20200301 Sync meeting

https://cwiki.apache.org/confluence/display/HUDI/20200303+Weekly+Sync+Minutes -- Regards, -Sivabalan

Re: Need clarity on these test cases in TestHoodieDeltaStreamer

wrote: >> >> > >> > This change was done as part of adding delete API support : >> > >> https://github.com/apache/incubator-hudi/commit/7031445eb3cae5a4557786c7eb080944320609aa >> > >> > I don't remember the reason behind this. >> &g

[DISCUSS] Adding common errors and solutions to FAQs

ping in slack if they don't find them already in that section. Also, it gives us an opportunity to think about improvising hudi when we look back at some of the common errors that users encounter. -- Regards, -Sivabalan

Repro travis CI build locally by filtering for an individual test/package

gation. -- Regards, -Sivabalan

Re: Re: Please welcome our new PPMCs and Committer

> > > > > >> Congratulations to Vino Yang. > > > > >> --原始邮件-- > > > > >> 发件人:"vbal...@apache.org" > > > >> 发送时间:2020年2月15日(星期六) 凌晨5:11 > > > > >> 收件人:"dev" >

[DISCUSS] RFC - 08 : Record level indexing mechanisms for Hudi datasets

ord+level+indexing+mechanisms+for+Hudi+datasets> is the link to RFC. Appreciate your review and thoughts. -- Regards, -Sivabalan

Right Dependency for spark and scala in latest master

ConfPropagated(SQLExecution.scala:160) at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:87) at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:828) at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:309) at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:293) at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:236) -- Regards, -Sivabalan

Re: Right Dependency for spark and scala in latest master

M Vinoth Chandar wrote: > > > I will triage this tonight and get back! > > > > On Mon, Jan 13, 2020 at 2:28 PM Sivabalan wrote: > > > > > Yes, that is what I tried. Is there any recommended version. I tried > with > > > 2.4.4. (My local spark from which I

Re: Right Dependency for spark and scala in latest master

rk-avro_2.11: .. With this change, we > effectively dropped support for spark versions older than 2.4. > Are you running on a older spark version? > > > > On Mon, Jan 13, 2020 at 10:03 AM Sivabalan wrote: > > > Hey folks, > >I am running into scala dependency issue w/ latest m

Re: Right Dependency for spark and scala in latest master

Sorry, sent before attaching screen shots. On Tue, Jan 14, 2020 at 4:26 PM Sivabalan wrote: > 3.x is not available under spark-avro_2.11. It is available only with 2.12 > and since 2.12 is not recommended, we are good. I verified that 2.4.4 works > for me if both spark shell and

Re: Right Dependency for spark and scala in latest master

irm that if you match the spark version (version > of spark-shell) with the version of spark-avro, things work for both 2.4.4 > and 3.x? Else this is a release blocker. > > On Tue, Jan 14, 2020 at 6:45 AM Sivabalan wrote: > > > cool, thanks for the assistance Sudha. We hav

Re: [SUPPORT] hudi-integ-test fails after adding licenses

d ([presto, --server, presto-coordinator-1:8090, --catalog, hive, > > --schema, default, -f, /usr/hive/data/input/presto-table-check.commands]) > > expected to succeed. Exit (1) expected:<0> but was:<1> > > > > and the detail log link > https://api.travis-ci.org/v3/job/640489853/log.txt > > > > Any ideas? thanks. > > > > Best, > > Leesf > > > -- Regards, -Sivabalan

Re: Re: IDE setup for code formatting

t; > >> > > > > >> 3, The code of the current project has met the checkstyle rules, > we > > > just > > > > >> need handle the incremental codes. > > > > >> > > > > >> > > > > >> 4, here are some useful tools which can be placed on contributing > > > guide. > > > > >> 1) mvn scalastyle:check > > > > >> 2) mvn checkstyle:check > > > > >> 3) https://checkstyle.sourceforge.io/index.html > > > > >> 4) http://www.scalastyle.org/rules-1.0.0.html > > > > >> > > > > >> > > > > >> best, > > > > >> lamber-ken > > > > >> On 12/23/2019 13:03，Vinoth Chandar wrote： > > > > >> Hello all, > > > > >> > > > > >> I know a bunch of work has happened to format the code base, > closer > > to > > > > what > > > > >> other project are doing.. > > > > >> > > > > >> While working through some checkstyle violations today, I noticed > > that > > > > the > > > > >> IDE formatting is now out of date with the checkstyle enforced? > > > Manually > > > > >> fixing these checkstyle issues are not a very productive use of > time > > > > IMHO. > > > > >> > > > > >> http://hudi.apache.org/contributing.html#ide-setup > > > > >> > > > > >> Can we make the instructions here work with the checkstyle rules > we > > > > already > > > > >> have? Goal should be that formatting code in IntelliJ (or other > > IDEs) > > > > >> should autofix it so that checkstyle passes.. > > > > >> > > > > >> thoughts? > > > > >> > > > > >> Thanks > > > > >> Vinoth > > > > >> > > > > > > > > > > -- Regards, -Sivabalan

Re: Contribution guidelines

community, I have been > > spending > > > a lot of time cleaning up JIRAs and also writing up explicitly, the > > > processes around contributions that we have been adopting.. > > > > > > Please give this is a quick read > > > http://hudi.apache.org/contributing.html#life-of-a-contributor > > > > > > Thanks > > > Vinoth > > > > > > -- Regards, -Sivabalan

Re: [QUESTION] Encountering exceptions while upserting with Deltastreamer

; > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.spark.scheduler.DAGSch

Re: Issue related to [HUDI-377] Adding Delete() support to DeltaStreamer

https://github.com/apache/incubator-hudi/pull/1395 On Tue, Mar 10, 2020 at 12:40 PM Prashant Wason wrote: > Thanks for the update Sivabalan. I will wait for your fix. > > On Tue, Mar 10, 2020 at 12:36 PM Sivabalan wrote: > > > thanks for bringing this to my attention Pras

Re: Hudi logo not found in apache projects logos

sure. On Tue, Mar 10, 2020 at 7:38 PM Vinoth Chandar wrote: > Logos are here > https://drive.google.com/drive/folders/1a5CngkGjvc-hp72XtOy2zcsSG81V_qbe > > Siva, can you take up the task of adding them to asf logos page :) > > On Tue, Mar 10, 2020 at 12:52 PM Sivabalan wrote

Re: [VOTE] Release 0.5.2-incubating, release candidate #1

/ReleaseNote.jspa?projectId=12322822=12346606 > > > > [2] > > > https://dist.apache.org/repos/dist/dev/incubator/hudi/hudi-0.5.2-incubating-rc1/ > > > > [3] https://dist.apache.org/repos/dist/release/incubator/hudi/KEYS > > > > [4] > https://repository.apache.org/content/repositories/orgapachehudi-1018/ > > > > [5] > > > https://github.com/apache/incubator-hudi/tree/release-0.5.2-incubating-rc1 > > > > > > > -- Regards, -Sivabalan

Re: New PPMC Member : Bhavani Sudha

> > > > > > > > I am very excited to share that we have new PPMC member - Sudha. > > She > > > > has > > > > > > been a great champion for the project for almost couple years > now, > > > > > driving > > > > > > a lot of presto/query engine facing changes and most of all being > > the > > > > > face > > > > > > of our community to new users on Slack, over the past few months. > > > > > > > > > > > > Please join me in congratulating her! > > > > > > > > > > > > On behalf of Hudi PPMC, > > > > > > Vinoth > > > > > > > > > > > > > > > > > > > > > -- Regards, -Sivabalan

Re: New Committer: lamber-ken

to become > a > > > > > > committer > > > > > > > > and we are pleased to announce that he has accepted. > > > > > > > > > > > > > > > > lamber-ken has had a large impact by in hudi, with some > > sustained > > > > > > efforts > > > > > > > > in the past several months. He has rebuilt our site ground > up, > > > > > > automated > > > > > > > > doc workflows, helped fixed a lot of bugs and also been super > > > > helpful > > > > > > for > > > > > > > > the community at large. > > > > > > > > > > > > > > > > Congratulations lamber-ken !! Please join me in recognizing > his > > > > > > efforts! > > > > > > > > > > > > > > > > On behalf of PPMC, > > > > > > > > Vinoth > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Regards, -Sivabalan

Re: [VOTE] Release 0.5.2-incubating, release candidate #1

thanks. On Thu, Mar 12, 2020 at 10:47 PM Vinoth Chandar wrote: > This is some standard steps I do. (plus run my own little jobs) > > https://gist.github.com/vinothchandar/68b34f3051e41752ebffd6a3edeb042b > > > On Thu, Mar 12, 2020 at 8:54 PM Sivabalan wro

Re: [VOTE] Release 0.5.2-incubating, release candidate #1

t; > > > > > > [ ] +1, Approve the release > > > > > > > > [ ] -1, Do not approve the release (please provide specific comments) > > > > > > > > The complete staging area is available for your review, which > includes: > > > > > > > > * JIRA release notes [1], > > > > * the official Apache source release and binary convenience releases > to > > > be > > > > deployed to dist.apache.org [2], which are signed with the key with > > > > fingerprint C3A96EC77149571AE89F82764C86684D047DE03C [3], > > > > > > > > * all artifacts to be deployed to the Maven Central Repository [4], > > > > * source code tag "release-0.5.2-incubating-rc1" [5], > > > > > > > > The vote will be open for at least 72 hours. It is adopted by > majority > > > > approval, with at least 3 PMC affirmative votes. > > > > > > > > Thanks, > > > > > > > > Vino > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12322822=12346606 > > > > > > > > [2] > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/hudi/hudi-0.5.2-incubating-rc1/ > > > > > > > > [3] https://dist.apache.org/repos/dist/release/incubator/hudi/KEYS > > > > > > > > [4] > > > https://repository.apache.org/content/repositories/orgapachehudi-1018/ > > > > > > > > [5] > > > > > > > > > > https://github.com/apache/incubator-hudi/tree/release-0.5.2-incubating-rc1 > > > > > > > > > > -- Regards, -Sivabalan

Re: [ATTN] JUnit 5 adoption

; > - To PR submitters: for newly added test classes, please start using > > > JUnit > > > > 5 APIs (org.junit.jupiter.*) > > > > - To PR reviewers: please help look out for the JUnit adopt in the > new > > > test > > > > classes > > > > > > > > Really appreciate the coordination efforts on this matter. > > > > > > > > Thank you. > > > > > > > > Regards, > > > > Raymond > > > > > > > > > > -- Regards, -Sivabalan

Re: [DISCUSS] Next Release timeline

gt; might be a good timeline? > > > > > > > > > I can volunteer to be the release manager if the community is okay with > > it. > > > Please share your thoughts. > > > > > > Thanks, > > > Sudha > > > > > > -- Regards, -Sivabalan

Re: [DISCUSS] Bug bash?

there be interest in doing a bug bash for a week, where we > >aggressively close out some pesky bugs that have been lingering around.. > If > >enough committers and contributors are around, we can move the needle. We > >could time this a week before cutting RC for next release. > > > >Thanks > >Vinoth > -- Regards, -Sivabalan

Re: [DISCUSS] Bug bash?

ot click links or open attachments unless you can confirm the sender > and > > > know the content is safe. > > > > > > > > > > > > +1 > > > > > > Shiyan Xu 于2020年4月24日周五 上午9:11写道： > > > > > > >

Re: [DISCUSS] Readiness for graduation to TLP

re [1] > > > > - Migrated developer conversations to the list at > dev@hudi.apache.org > > > > - Migrated all issue tracking to JIRA [2] > > > > - Apache Hudi name search has been approved [3] > > > > - We have built a meritocratic, open collaborative process, the > Apache > > > way > > > > - Our PMC is diverse and consists of members from ~10 organizations > > > > > > > > Please chime in with your thoughts. > > > > > > > > Thanks > > > > Vinoth > > > > > > > > > > -- Regards, -Sivabalan

Re: Checking out the asf svn repo

>> > >> > > > >> > >> > > > >> Best, > >> > > > >> Lamber-Ken > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> At 2020-04-17 08:24:01, "Vinoth Chandar" > >> wrote: > >> > > > >> >Hello all, > >> > > > >> > > >> > > > >> >Can anyone here (potentially from prior experience with other > >> apache > >> > > > >> >projects) point me, to how I can checkout the apache svn repo > >> here? > >> > > > >> > > >> > > > >> > >> > > > >> > https://svn.apache.org/viewvc/incubator/public/trunk/content/projects/hudi.xml?view=log > >> > > > >> > > >> > > > >> > > >> > > > >> >Would like to make some edits to our status file.. > >> Specifically, I am > >> > > > >> >trying to understand how I authenticate to commit the changes? > >> (I > >> > > have > >> > > > >> not > >> > > > >> >used svn in years. So apologize if I am asking some basic qs) > >> > > > >> > > >> > > > >> >Thanks > >> > > > >> >Vinoth > >> > > > >> > >> > > > > > >> > > > > >> > > > >> > > >> > > > -- Regards, -Sivabalan

Re: [DISCUSS] Logos on project front page.

tes are meant as suggestions for projects. PMCs > > are > > > > free to adopt (or not) any of these suggestions for their sites.". > > > > > > > > But I would prefer to play by the best practices if we can.. > > > > > > > > Can you all chime in with your thoughts? > > > > > > > > > > > > > > > > [1] https://www.apache.org/foundation/marks/linking > > > > > > > > > > -- Regards, -Sivabalan

Re: [DISCUSS] Logos on project front page.

All green :) On Wed, May 13, 2020 at 12:54 PM Vinoth Chandar wrote: > https://svn.apache.org/repos/asf/comdev/project-logos/originals/ > Checked our logo svg in. This must be the last issue on the whimsy page. > > On Wed, May 13, 2020 at 8:23 AM Sivabalan wrote: > > >

Re: Clarification on PRECOMBINE_FIELD_OPT_KEY

T_KEY > operation and logcailly handles batch 2 updates. Because my batch 2 will > have 2 updates and I need to logically combine them and upsert HDFS. > > > > PRECOMBINE_FIELD_OPT_KEY seems to be mandateone and it throws error, if I > don’t have ts column in my table and also don't want Hudi to do > precombine as it could eliminate some of my genuine records. What is > your suggestion on handling this scenario? > > > > Thanks, > > Selva > -- Regards, -Sivabalan

Re: [VOTE] Apache Hudi graduation to top level project

> > > > > > > > * Shaofeng Li > > > > > > > > * Steve Blackmon > > > > > > > > * Suneel Marthi > > > > > > > > * Thomas Weise > > > > > > > > * Vino Yang > > > > > > > > * Vinoth Chandar > > > > > > > > NOW, THEREFORE, BE IT FURTHER RESOLVED, that Vinoth Chandar be > > appointed > > > to > > > > the office of Vice President, Apache Hudi, to serve in accordance > with > > > and > > > > subject to the direction of the Board of Directors and the Bylaws of > > the > > > > Foundation until death, resignation, retirement, removal of > > > > disqualification, or until a successor is appointed; and > > > > > > > > be it further > > > > > > > > RESOLVED, that the Apache Hudi Project be and hereby is tasked with > the > > > > migration and rationalization of the Apache Incubator Hudi podling; > and > > > > > > > > be it further > > > > > > > > RESOLVED, that all responsibilities pertaining to the Apache > Incubator > > > Hudi > > > > podling encumbered upon the Apache Incubator PMC are hereafter > > > discharged. > > > > > > > > > > -- Regards, -Sivabalan

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

> > > > I am wondering if we should do a 0.5.3 release by backporting all minor > to > > medium bug fixes (that are in master already) to 0.5.2 and do a minor > > release ? That way we can use some time to reserve 0.6.0 release for all > > major features that are upcoming and/or almost there. Please share your > > thoughts. If you agree also please share the list of fixes that you know > of > > that can go into 0.5.3. > > > > Thanks, > > Sudha > > > -- Regards, -Sivabalan

Bug Bash 0.6.0

As we discussed earlier in our mailing list, here we are commencing our bug bash for 0.6.0. This is a tracking ticket for the same. Thanks to all who have volunteered to participate. We are looking to time it for 10 days. Planning to wrap it up by

Re: PR backlog

handar wrote: > > > Hello all, > > > > As we approach 0.5.3 code freeze, there is a lot of PR backlog. If you > are > > assigned PRs as reviewer, please take a look at them and drive them > towards > > completion. > > > > Thanks > > Vinoth > > > -- Regards, -Sivabalan

Re: Bug Bash 0.6.0

pick up one at this time. > > On Fri, May 15, 2020 at 6:31 AM Sivabalan wrote: > > > As we discussed earlier in our mailing list, here we are commencing our > bug > > bash for 0.6.0. This is a tracking ticket > > <https://issues.apache.org/jira/browse/HUDI-901> for the

Re: [DISCUSS] Logos on project front page.

l) > > We need to upload the logo, which I assume Siva you are taking care of ? > > On Wed, May 13, 2020 at 7:45 AM Vinoth Chandar wrote: > > > https://whimsy.apache.org/pods/project/hudi > > We need to get this in order, looks like.. > > > > On Wed, May 13, 2020

Re: [VOTE] Release 0.5.2-incubating, release candidate #2

Apache > > > > Hudi > > > > > > > > > (incubating). The version is: 0.5.2-incubating-rc2. Please > > > > review and > > > > > > > > vote > > > > > > > > > on the release candidate #2 for the version 0.5.2, as > > follows: > > > > > > > > > > > > > > > > > > [ ] +1, Approve the release > > > > > > > > > > > > > > > > > > [ ] -1, Do not approve the release (please provide specific > > > > comments) > > > > > > > > > > > > > > > > > > The complete staging area is available for your review, > which > > > > > > includes: > > > > > > > > > > > > > > > > > > * JIRA release notes [1], > > > > > > > > > * the official Apache source release and binary convenience > > > > releases > > > > > > to > > > > > > > > be > > > > > > > > > deployed to dist.apache.org [2], which are signed with the > > key > > > > with > > > > > > > > > fingerprint C3A96EC77149571AE89F82764C86684D047DE03C [3], > > > > > > > > > > > > > > > > > > * all artifacts to be deployed to the Maven Central > > Repository > > > > [4], > > > > > > > > > * source code tag "release-0.5.2-incubating-rc2" [5], > > > > > > > > > > > > > > > > > > The vote will be open for at least 72 hours. It is adopted > by > > > > > > majority > > > > > > > > > approval, with at least 3 PMC affirmative votes. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Vino > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12322822=12346606 > > > > > > > > > > > > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/hudi/hudi-0.5.2-incubating-rc2/ > > > > > > > > > > > > > > > > > > [3] > > > > https://dist.apache.org/repos/dist/release/incubator/hudi/KEYS > > > > > > > > > > > > > > > > > > [4] > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachehudi-1019 > > > > > > > > > > > > > > > > > > [5] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-hudi/tree/release-0.5.2-incubating-rc2 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Regards, -Sivabalan

Re: Issue related to [HUDI-377] Adding Delete() support to DeltaStreamer

added in the checks above? Is this > related to the existingKeys.remove() which does not remove the records from > the Map? > > I have left these comments on the PR itself so they are easier to read. > > Thanks > Prashant > -- Regards, -Sivabalan

Re: Hudi logo not found in apache projects logos

cool. thnx. On Tue, Mar 10, 2020 at 12:46 PM Prashant Wason wrote: > Found this link on that page: > http://apache.org/logos/about.html > > > On Tue, Mar 10, 2020 at 12:42 PM Sivabalan wrote: > > > Hi folks, > >Do you guys know to add hudi log in

Hudi logo not found in apache projects logos

Hi folks, Do you guys know to add hudi log in the list of apache logos here <http://apache.org/logos/#>. I do see other incubating projects as well. -- Regards, -Sivabalan

Re: [DISCUSS] Bug bash?

bash would be tickets which are > easily reproducible, have isolated fixes, doable in a day or two. > > There you go :) > > On Wed, Apr 29, 2020 at 9:45 AM Sivabalan wrote: > > > I could lend a hand if we need any help in organizing this. LMK. > > > > On Tue, Apr

Re: Bug Bash 0.6.0

wrote: > Hi siva, > > Can we do a quick update on progress so far? So everyone gets a sense of > how this is going :) > > > > On Fri, May 15, 2020 at 5:19 PM Sivabalan wrote: > > > Sure Ethan. Will keep you posted if something comes up. > > > > &g

Re: Re: Apache Hudi Graduation vote on general@incubator

>> > >> Hello all, > >> > >> Just started the VOTE on the IPMC general list [1] > >> > >> If you are an IPMC member, you do a *binding *vote > >> If you are not, you can still do a *non-binding* vote > >> > >> Please take a moment to vote. > >> > >> [1] > >> > >> > >> > https://lists.apache.org/thread.html/r8039c8eece636df8c81a24c26965f5c1556a3c6404de02912d6455b4%40%3Cgeneral.incubator.apache.org%3E > >> > >> Thanks > >> Vinoth > >> > >> > -- Regards, -Sivabalan

Re: Bug Bash 0.6.0

Status update: 6 : closed. 15 : patches available. Good job folks. Appreciate it. Do tag your reviewers or follow up with feedback if need be. We plan to wrap up bug bash in 5 days. On Wed, May 20, 2020 at 12:01 AM Sivabalan wrote: > This is the status for now: > > 5 closed/res

Re: [DISCUSS] Formalizing the release process

new items, re-prioritizations > >- Contributors are welcome to commit more features/asks, (with due > >process) > > > > I would love to hear +1s, -1s and also any new, completely different > ideas > > as well. Let's use this thread to align ourselves. > > > > Once we align ourselves, there are some release certification tools that > > need to be built out. Hopefully, we can do this together. :) > > > > > > Thanks > > Vinoth > > > -- Regards, -Sivabalan

Re: Congrats to our newest committers!

> *Pratyaksh Sharma:* This is yet another great example of user -> > > > > contributor -> committer. Pratyaksh has been a great champion for the > > > > project, over the past year or so, steadily contributing many > > > improvements > > > > around the Delta Streamer tool. > > > > > > > > Please join me in, congratulating them on this well deserved milestone! > > > > > > > > Onwards and upwards, > > > > Vinoth > > > > > > > > > -- Regards, -Sivabalan

Re: Congrats to our newest committers!

your continued > > contributions. > > Balaji.V > > On Thursday, September 3, 2020, 07:19:45 PM PDT, Sivabalan < > > n.siv...@gmail.com> wrote: > > > > Congrats to all 3. Much deserved and really excited to see more > committers > >  > > &

Re: schema compatibility check and change column type

","default":null},{"name":"_hoodie_record_key","type":["null","string"],"doc":"","default":null},{"name":"_hoodie_partition_path","type":["null","string"],"doc":"","default":null},{"name":"_hoodie_file_name","type":["null","string"],"doc":"","default":null},{"name":"a","type":"long"},{"name":"b","type":"string"},{"name":"__row_key","type":"int"},{"name":"__row_version","type":"int"}]}, > > > table schema > > > > > > :{"type":"record","name":"foo_record","namespace":"hoodie.foo","fields":[{"name":"_hoodie_commit_time","type":["null","string"],"doc":"","default":null},{"name":"_hoodie_commit_seqno","type":["null","string"],"doc":"","default":null},{"name":"_hoodie_record_key","type":["null","string"],"doc":"","default":null},{"name":"_hoodie_partition_path","type":["null","string"],"doc":"","default":null},{"name":"_hoodie_file_name","type":["null","string"],"doc":"","default":null},{"name":"a","type":"int"},{"name":"b","type":"string"},{"name":"__row_key","type":"int"},{"name":"__row_version","type":"int"}]}, > > > base path :file:///jfs/cadl/hudi_data/schema/foo > > > at > > org.apache.hudi.table.HoodieTable.validateSchema(HoodieTable.java:564) > > > at > > > > > > org.apache.hudi.table.HoodieTable.validateUpsertSchema(HoodieTable.java:570) > > > ... 94 more. > > > -- Regards, -Sivabalan

Re: schema compatibility check and change column type

Sivabalan wrote: > Hudi relies on avro's Schema compatability check. Looks like as per avro > SchemaCompatability, INT can't be evolved to a LONG, but LONG to INT is > allowed. > > Check line no 339 here > <https://github.com/apache/avro/blob/master/lang/java/avro/src/main

Re: HUDI Table Primary Key - UUID or Custom For Better Performance

performance in writing if I will have the UUID vs composite domain > keys. > > I believe read is not impacted as per the Primary Key as its not being > considered ? > > Please suggest > > -- Regards, -Sivabalan

Re: Adding recent talks to site

ou please add any recent talks to our powered by/talks page? > I know we had an ApacheCon and may be one more talk? > > I opened https://github.com/apache/hudi/pull/2155 for the PrestoCon panel > > Thanks > Vinoth > -- Regards, -Sivabalan

Re: [ANNOUNCE] Apache Hudi 0.6.0 released

d the Apache > > Software Foundation for enabling this release and we look forward to > > continued collaboration. We welcome your help and feedback. For more > > information on how to report problems, and to get involved, visit the > > project website at: > > http://hudi.apache.org/ > > > > Thanks to everyone involved! > > - Bhavani Sudha > > > -- Regards, -Sivabalan

Re: [DISCUSS] Release 0.6.0 timelines

That would be of great help Allen. Much appreciated. On Wed, Aug 19, 2020 at 9:30 AM Allen Underwood wrote: > Thanks Sivabalan, > > That's definitely the issue I had to resolve when introducing Joda time. > I'll have a look back at my code to see how I got around it. If I remember

Re: [DISCUSS] Release 0.6.0 timelines

. >> > > > > >> > > > >> > > > > >> > > > On the release blocker progress, we landed few blockers >> over >> > > the >> > > > > >> > weekend, >> > > > > >> > > &g

Re: [DISCUSS] Codestyle: force multiline indentation

.com > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > I noticed that throughout the codebase, when method > > > arguments > > > > > > wrap > > > > > > > > to a > > > > > > > > > > new > > > > > > > > > > > line, there are cases where indentation is 4 and other > > > cases > > > > > > align > > > > > > > > the > > > > > > > > > > > wrapped line to the previous line of argument. > > > > > > > > > > > > > > > > > > > > > > The latter is caused by intelliJ settings of "Align > when > > > > > > multiline" > > > > > > > > > > > enabled. This won't be flagged by checkstyle due to not > > > > setting > > > > > > > > > > > *forceStrictCondition* to *true* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://checkstyle.sourceforge.io/config_misc.html#Indentation_Properties > > > > > > > > > > > > > > > > > > > > > > I'm suggesting setting this to true to avoid the > > > discrepancy > > > > > and > > > > > > > > > > redundant > > > > > > > > > > > diffs in PR caused by individual IDE settings. People > who > > > > have > > > > > > set > > > > > > > > > "Align > > > > > > > > > > > when multiline" will need to disable it to pass the > > > > checkstyle > > > > > > > > > > validation. > > > > > > > > > > > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > Raymond > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Regards, -Sivabalan

Re: [DISCUSS] Support for `_hoodie_record_key` as a virtual column

ted data. > > > > > > Proposal: > > > In the interest of improving storage efficiency, we could add confs / > > > abstract classes that can construct the _hoodie_record_key given other > > > columns. That way we do not have to store duplicated data on DFS. > > > > > > Any thoughts on this? > > > > > > Best, > > > Modi > > > > > > -- Regards, -Sivabalan

Re: [DISCUSS] Support for `_hoodie_record_key` as a virtual column

Aah, yes. That’s right. On Sat, Aug 22, 2020 at 2:43 AM Vinoth Chandar wrote: > All of the remaining meta fields compress very very nicely. They have > > almost no overhead. > > > > On Fri, Aug 21, 2020 at 12:00 PM Abhishek Modi > > wrote: > > > > >

DevX, Test infra Rgdn

in this regard. -- Regards, -Sivabalan

Re: [DISSCUSS] Trigger a Travis-CI rebuild without pushing a commit

t;run pr validation" > > https://github.com/apache/bookkeeper/pull/2158 > > > > But, I can't find a effective solution from Github and Travis's > > documentation[1], any thoughts or opinions? > > > > Best, > > Lamber-Ken > > > > [1] https://docs.travis-ci.comhttps://support.github.com > > > -- Regards, -Sivabalan

Re: Progress on 0.5.3 release

rior graduation, one needs to send emails to general@incubator. After graduation, is there a diff address, or just annou...@apache.org would suffice. On Wed, May 27, 2020 at 8:54 PM Sivabalan wrote: > thanks leesf. I will reach out if I need any help. I will share the > updated release guide o

Re: Progress on 0.5.3 release

e-calibrated again.. > > > > > > Ideally, someone who has done the release before, can help Siva? > Anyone > > > wants to volunteer? > > > > > > Thanks > > > VInoth > > > > > > On Mon, May 25, 2020 at 5:48 AM Sivabalan w

Re: Progress on 0.5.3 release

Hey folks, Please hold on to review the guide. I happened to trim out "incubator"/"incubating" texts only where ever required. Will compare w/ flink's release guide and add/edit sections as appropriate. Will update the thread once I have something. On Wed, May 27, 2020 a

Re: Progress on 0.5.3 release

Here <https://cwiki.apache.org/confluence/display/HUDI/Apache+Hudi+-+Release+Guide+Updated> is the updated guide. I have added comments at places where I had doubts or wasn't clear. Would appreciate if someone can review it and help me out. On Wed, May 27, 2020 at 11:23 PM Sivabalan

Progress on 0.5.3 release

on to creating a release branch in apache/hudi and creating release notes. - Also, I might need someone to help on any new changes or process required after graduating. As of now, I am following this <https://cwiki.apache.org/confluence/display/HUDI/Apache+Hudi+-+Release+Guide> . -- Regards, -Sivabalan

Re: Hudi Global Bloom Index Issue

s3:////trgt.avsc > > hoodie.deltastreamer.schemaprovider.source.schema.file=s3:///XXX/src.avsc > > hoodie.datasource.hive_sync.partition_extractor_class=org.apache.hudi.hive.MultiPartKeysValueExtractor > hoodie.deltastreamer.transformer.sql=select > Op,src_received_ts,tran_id,tran_date,store_id,store_city,store_state,item_code,quantity > from > hoodie.datasource.hive_sync.jdbcurl=jdbc:hive2://XXX: > hoodie.deltastreamer.source.dfs.root=s3:/// XXX > hoodie.index.type=GLOBAL_BLOOM > > > Thanks, > Raghu > > -- Regards, -Sivabalan

Re: Apache Hudi Graduation vote on general@incubator

he Apache board! > > > Congratulations everyone! :) > > > > > > More to come, as we prepare to exit the incubator. Stay tuned!! > > > > > > On Tue, May 19, 2020 at 7:33 PM Balaji Varadarajan > > > wrote: > >

Re: [DISCUSS] Release 0.6.0 timelines

and rework, > > review, land (At risk for 0.6.0, high scope, may not have enough time) > >- BloomIndex V2 - Global index implementation. (At risk) > >- HUDI-845 : Parallel writing i.e allow multiple writers (At risk) > >- HUDI-860 : Small File Handling without memory caching (At risk) > > > > > > Thanks > > Vinoth > > > -- Regards, -Sivabalan

Recording link to Apache Hudi - Design/Code Walkthrough Session

Hey folks, Last week we had a session from Vinoth Chandar on design and code walkthrough session for Apache Hudi. Here <https://www.youtube.com/watch?v=N2eDfU_rQ_U> is the recording link. -- Regards, -Sivabalan

Re: DISCUSS code, config, design walk through sessions

ones/edt> > > > > >> > > > > > > > > > > > UTC-4 > > > > >> > > > > > > > > > > > > > hours > > > > >> > > > > > > > > > > > > > New Delhi < > > > > >> > > > > > > > > > > > https://www.timeanddate.com/worldclock/india/new-delhi> > > > > >> > > > > > > > > > > > > (India > > > > >> > > > > > > > > > > > > > - > > > > >> > > > > > > > > > > > > > Delhi) 8:30:00 pm IST < > > > > >> > > > > > > > > https://www.timeanddate.com/time/zones/ist> > > > > >> > > > > > > > > > > > > > UTC+5:30 > > > > >> > > > > > > > > > > > > > hours > > > > >> > > > > > > > > > > > > > Shanghai < > > > > >> > > > > > > > > https://www.timeanddate.com/worldclock/china/shanghai > > > > > > > >> > > > > > > > > > > > (China > > > > >> > > > > > > > > > > > > - > > > > >> > > > > > > > > > > > > > Shanghai Municipality) 11:00:00 pm CST > > > > >> > > > > > > > > > > > > > < > > > > >> https://www.timeanddate.com/time/zones/cst-china> > > > > >> > > > UTC+8 > > > > >> > > > > > > hours > > > > >> > > > > > > > > > > > > > London < > > > > >> > > > https://www.timeanddate.com/worldclock/uk/london > > > > >> > > > > > > > > > >> > > > > > > > > (United > > > > >> > > > > > > > > > > > > Kingdom > > > > >> > > > > > > > > > > > > > - > > > > >> > > > > > > > > > > > > > England) 4:00:00 pm BST < > > > > >> > > > > > > > > > https://www.timeanddate.com/time/zones/bst> > > > > >> > > > > > > > > > > > > UTC+1 > > > > >> > > > > > > > > > > > > > hour > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Please speak up if this does not work > for > > > > >> anyone.. > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Thanks > > > > >> > > > > > > > > > > > > > Vinoth > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > On Sun, Jul 12, 2020 at 4:12 PM > Ranganath > > > > >> Tirumala > > > > >> > < > > > > >> > > > > > > > > > > > > > ranganath.tirum...@gmail.com> wrote: > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > So, Is this confirmed for 14th July > > 9:30pm > > > > >> PST? > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > On Sat, 11 Jul 2020 at 14:32, Gurudatt > > > > >> Kulkarni < > > > > >> > > > > > > > > > > guruak...@gmail.com > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > If possible recoding of these > sessions > > > > >> would be > > > > >> > > > > great, > > > > >> > > > > > to > > > > >> > > > > > > > > fill > > > > >> > > > > > > > > > > the > > > > >> > > > > > > > > > > > > > > timezone > > > > >> > > > > > > > > > > > > > > > gap. > > > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > On Friday, July 10, 2020, Pratyaksh > > > > Sharma < > > > > >> > > > > > > > > > > pratyaks...@gmail.com> > > > > >> > > > > > > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > > > > @Vinoth Chandar < > vin...@apache.org> > > > > Time > > > > >> > zones > > > > >> > > > are > > > > >> > > > > > > > indeed > > > > >> > > > > > > > > > > > tricky. > > > > >> > > > > > > > > > > > > > > Maybe > > > > >> > > > > > > > > > > > > > > > we > > > > >> > > > > > > > > > > > > > > > > can do a poll again to decide on > the > > > > time > > > > >> for > > > > >> > > > these > > > > >> > > > > > > > > sessions > > > > >> > > > > > > > > > > > given > > > > >> > > > > > > > > > > > > > the > > > > >> > > > > > > > > > > > > > > > > community size has increased much > > more > > > > >> now as > > > > >> > > > > > compared > > > > >> > > > > > > to > > > > >> > > > > > > > > > last > > > > >> > > > > > > > > > > > time > > > > >> > > > > > > > > > > > > > we > > > > >> > > > > > > > > > > > > > > > > decided on weekly sync timings? > This > > > > might > > > > >> > help > > > > >> > > > all > > > > >> > > > > > the > > > > >> > > > > > > > new > > > > >> > > > > > > > > > > > members > > > > >> > > > > > > > > > > > > > of > > > > >> > > > > > > > > > > > > > > > our > > > > >> > > > > > > > > > > > > > > > > community as well. :) > > > > >> > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > On Fri, Jul 10, 2020 at 8:45 AM > Adam > > > > >> Feldman > > > > >> > < > > > > >> > > > > > > > > > > > afeldm...@gmail.com> > > > > >> > > > > > > > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> Yea, time zones are tough. That's > > > > >> midnight > > > > >> > in > > > > >> > > > EST > > > > >> > > > > > and > > > > >> > > > > > > > the > > > > >> > > > > > > > > > > middle > > > > >> > > > > > > > > > > > > of > > > > >> > > > > > > > > > > > > > > the > > > > >> > > > > > > > > > > > > > > > >> night if anyone is in Western > > > Europe... > > > > >> > > > > > > > > > > > > > > > >> > > > > >> > > > > > > > > > > > > > > > >> On Thu, Jul 9, 2020, 23:08 wei > li < > > > > >> > > > > > > > lw309637...@gmail.com> > > > > >> > > > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > > > >> > > > > >> > > > > > > > > > > > > > > > >> > +1 > > > > >> > > > > > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > >> > On 2020/07/06 03:30:43, Vinoth > > > > Chandar > > > > >> < > > > > >> > > > > > > > > vin...@apache.org > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > wrote: > > > > >> > > > > > > > > > > > > > > > >> > > Hi all, > > > > >> > > > > > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > >> > > As we scale the community, > its > > > > >> important > > > > >> > > > that > > > > >> > > > > > more > > > > >> > > > > > > > of > > > > >> > > > > > > > > us > > > > >> > > > > > > > > > > are > > > > >> > > > > > > > > > > > > > able > > > > >> > > > > > > > > > > > > > > to > > > > >> > > > > > > > > > > > > > > > >> help > > > > >> > > > > > > > > > > > > > > > >> > > users, users becoming > > > contributors. > > > > >> > > > > > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > >> > > In the past, we have drafted > > > faqs, > > > > >> > trouble > > > > >> > > > > > > shooting > > > > >> > > > > > > > > > > guides. > > > > >> > > > > > > > > > > > > But > > > > >> > > > > > > > > > > > > > I > > > > >> > > > > > > > > > > > > > > > feel > > > > >> > > > > > > > > > > > > > > > >> > > sometimes, more hands on walk > > > > through > > > > >> > > > sessions > > > > >> > > > > > > over > > > > >> > > > > > > > > > video > > > > >> > > > > > > > > > > > > could > > > > >> > > > > > > > > > > > > > > > help. > > > > >> > > > > > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > >> > > I am happy to spend 2 hours > > each > > > on > > > > >> > > > > > code/configs, > > > > >> > > > > > > > > > > > > > > > >> > design/perf/architecture. > > > > >> > > > > > > > > > > > > > > > >> > > Have the session be recorded > as > > > > well > > > > >> for > > > > >> > > > > future. > > > > >> > > > > > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > >> > > What does everyone think? > > > > >> > > > > > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > >> > > Thanks > > > > >> > > > > > > > > > > > > > > > >> > > Vinoth > > > > >> > > > > > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > >> > > > > >> > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > -- > > > > >> > > > > > > > > > > > > > > Regards, > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Ranganath Tirumala > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > -- > > Regards, > > > > Ranganath Tirumala > > > -- Regards, -Sivabalan

Re: DISCUSS code, config, design walk through sessions

s, trouble shooting guides. But I > > feel > > > > > sometimes, more hands on walk through sessions over video could > help. > > > > > > > > > > I am happy to spend 2 hours each on code/configs, > > > > design/perf/architecture. > > > > > Have the session be recorded as well for future. > > > > > > > > > > What does everyone think? > > > > > > > > > > Thanks > > > > > Vinoth > > > > > > > > > > > > > > > > > > -- > > > Regards, > > > > > > Ranganath Tirumala > > > > > > -- Regards, -Sivabalan

Re: [DISCUSS] Publishing benchmarks for releases

ormance reports > > On Mon, 22 Jun 2020, 02:41 vino yang, wrote: > > > +1 as well, > > > > it would be helpful to measure the performance between different > versions. > > > > Shiyan Xu 于2020年6月22日周一 上午8:37写道： > > > > > +1 definitely u

Re: Re:Re: [DISCUSS] Regarding nightly builds

PM Bhavani Sudha < > bhavanisud...@gmail.com > wrote: > > Hello all, > > Should we have nightly builds that way we can point users to > those builds > for the latest features introduced, instead of being blocked on > the next > release. Also this kind of gives an early feedback on new > features or > fixes > if any further improvements are needed. Does anyone > know if and how > other > Apache projects handle nightly builds? > > Thanks, > Sudha > > -- Regards, -Sivabalan

[DISCUSS] Publishing benchmarks for releases

r more operations and for every release, we could publish some benchmark numbers. -- Regards, -Sivabalan

Re: [DISCUSS] Introduce a write committed callback hook

one of our scene requirements, and it will be a very useful feature > combined with the incremental query, it can make the incremental processing > more timely. > > We hope to hear what the community thinks of this proposal. Any comments > and opinions are appreciated. > > Best, > Vino > -- Regards, -Sivabalan

[ANNOUNCE] Apache Hudi 0.5.3 released

and feedback. For more information on how to report problems, and to get involved, visit the project website at: http://hudi.apache.org/ Kind regards, Sivabalan Narayanan (Hudi 0.5.3 Release Manager) On behalf of the Apache Hudi

Re: [VOTE] Release 0.5.3, release candidate #2

Release label:emr-5.29.0 > Hadoop distribution:Amazon 2.8.5 > Applications:Spark 2.4.4, Hive 2.3.6, Tez 0.9.2, Presto 0.227 > Upsert to COW table ... ok > Hive sync ... ok > HiveQL select ... ok > > On Fri, Jun 12, 2020 at 5:09 AM Yajun Luo wrote: > > > +1 > >

[RESULT] [VOTE] Release 0.5.3, release candidate #2

* Gary Li * Yajun Luo * Raymond There are no disapproving votes. Link to Voting thread: https://lists.apache.org/thread.html/ree0d4b93a6c1b974836babec647ad38d1ab47eb5ae7b60314e472253%40%3Cdev.hudi.apache.org%3E Thanks everyone! -- Regards, -Sivabalan

Re: [DISCUSS] Make delete marker configurable?

eld for delete marker and `_hoodie_is_deleted` remains as > > default. > > > > Regards, > > Raymond > > > -- Regards, -Sivabalan

0.5.3 release status update

intermittently. Have filed a ticket <https://issues.apache.org/jira/browse/HUDI-1017> for the same. -- Regards, -Sivabalan

[VOTE] Release 0.5.3, release candidate #2

Hi everyone, Please review and vote on the release candidate #2 for the version 0.5.3, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1],

[VOTE] Release 0.5.3, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the version 0.5.3, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes

Re: [PMCs] Ramping up for Google Summer of Code 2021: invitation to participate

t; > >, > > > > > > the participating students will be working on the project as a > > > > full-time > > > > > > job for 12 weeks. So roughly we could expect 40 hours work per > > week. > > > > > > > > > > > > Also agree on your poin

Re: [DISCUSS] 0.7.0 release timelines

0. We already have a few large features in > master/pending PRs (spark3, flink, replace/overwrite etc..) > Option 2: Wait till December end to cut RC, with all the originally > planned feature set. > > Please chime in with your thoughts. > > Thanks > Vinoth > -- Regards, -Sivabalan

  1   2   3   >