Sg. Please share it for review comments whenever you are ready On Thu, Feb 28, 2019 at 10:07 PM nishith agarwal <n3.nas...@gmail.com> wrote:
> Vinoth, > > I've got a template for HIP (Hudi Improvement Plan) going on the google > drive. Working on writing up a description of what each component of the > template means (which we can put on cWiki). > PS : I've borrowed majorly from KIP. > > Thanks > > On Thu, Feb 28, 2019 at 4:26 PM Vinoth Chandar <vin...@apache.org> wrote: > > > Hi Nishith, > > > > Curious if you are further along on this. I also want to start one around > > easy migration of non Hudi datasets.. > > > > Thanks > > Vinoth > > > > On Mon, Feb 25, 2019 at 7:10 AM Vinoth Chandar <vin...@apache.org> > wrote: > > > > > My suggestion would be to first define the HIP template and the > process, > > > document , push the site out.. > > > Then we can use the GlobalIndex doc to test drive the process. > > > Are there other specific things you want to test out on cWiki itself? > > > > > > On Sun, Feb 24, 2019 at 10:09 AM nishith agarwal <n3.nas...@gmail.com> > > > wrote: > > > > > >> Okay, great. Yeah, I'll look into the templates and guidelines around > > >> writing new proposals/designs. > > >> In parallel, I'll share the GlobalIndex document on google drive and > we > > >> can > > >> try to put that up on cWiki to test things out ? (or any other > document > > >> works fine too). > > >> > > >> -Nishith > > >> > > >> On Fri, Feb 22, 2019 at 12:53 PM Vinoth Chandar <vin...@apache.org> > > >> wrote: > > >> > > >> > Nishith, that google account is already owned by private@ . It's > > >> folders > > >> > are publicly viewable.. > > >> > So we should be good to go. > > >> > > > >> > I think next step is to define a standard HIP (Hudi Improvement > > >> Proposal) > > >> > template and guidelines around writing a new proposal.. > > >> > This is what you were originally trying to do, IIRC? > > >> > > > >> > On Fri, Feb 22, 2019 at 12:40 PM nishith agarwal < > n3.nas...@gmail.com > > > > > >> > wrote: > > >> > > > >> > > Great! Thanks Thomas. > > >> > > > > >> > > Vinoth, > > >> > > Can we use our existing hoodie google drive and share it with > @dev ? > > >> > > > > >> > > -Nishith > > >> > > > > >> > > On Thu, Feb 21, 2019 at 9:52 PM Thomas Weise <t...@apache.org> > > wrote: > > >> > > > > >> > > > I created the space and added user vinoth as space admin. > > >> > > > > > >> > > > https://cwiki.apache.org/confluence/display/HUDI/Home > > >> > > > > > >> > > > Thomas > > >> > > > > > >> > > > On Tue, Feb 19, 2019 at 4:25 PM Vinoth Chandar < > vin...@apache.org > > > > > >> > > wrote: > > >> > > > > > >> > > > > I think its listed here? > > >> > > > > https://cwiki.apache.org/confluence/display/INFRA/Cwiki > > >> > > > > https://selfserve.apache.org ? > > >> > > > > > > >> > > > > On Mon, Feb 18, 2019 at 9:13 PM Thomas Weise <t...@apache.org> > > >> wrote: > > >> > > > > > > >> > > > > > The confluence wiki space should be created in any case. I > > >> believe > > >> > > this > > >> > > > > can > > >> > > > > > be done by mentors through the infra self serve portal, but > > >> don't > > >> > > have > > >> > > > it > > >> > > > > > in front of me at the moment. > > >> > > > > > > > >> > > > > > -- > > >> > > > > > sent from mobile > > >> > > > > > > > >> > > > > > On Mon, Feb 18, 2019, 5:51 PM Vinoth Chandar < > > vin...@apache.org > > >> > > > >> > > > wrote: > > >> > > > > > > > >> > > > > > > Bumping this thread up. We have a few big designs > upcoming.. > > >> > > > > > > Love to get this going, using this new setup. > > >> > > > > > > > > >> > > > > > > On Wed, Feb 13, 2019 at 8:54 PM nishith agarwal < > > >> > > n3.nas...@gmail.com > > >> > > > > > > >> > > > > > > wrote: > > >> > > > > > > > > >> > > > > > > > I can go ahead and setup the google drive as well as > > cWiki. > > >> > > > > > > > Mentors, > > >> > > > > > > > I created a new account at https://cwiki.apache.org/. > > >> Should I > > >> > > > > create > > >> > > > > > a > > >> > > > > > > > new > > >> > > > > > > > space with name "Hudi" where we could start adding the > > final > > >> > > > drafts ? > > >> > > > > > > > > > >> > > > > > > > Thanks, > > >> > > > > > > > Nishith > > >> > > > > > > > > > >> > > > > > > > On Sun, Feb 10, 2019 at 11:08 AM Prasanna < > > >> > > prasanna....@gmail.com> > > >> > > > > > > wrote: > > >> > > > > > > > > > >> > > > > > > > > +1 > > >> > > > > > > > > > > >> > > > > > > > > On Fri, Feb 8, 2019, 7:46 PM Suneel Marthi < > > >> > smar...@apache.org > > >> > > > > > >> > > > > > wrote: > > >> > > > > > > > > > > >> > > > > > > > > > +1 > > >> > > > > > > > > > > > >> > > > > > > > > > On Fri, Feb 8, 2019 at 6:39 PM Vinoth Chandar < > > >> > > > vin...@apache.org > > >> > > > > > > > >> > > > > > > > wrote: > > >> > > > > > > > > > > > >> > > > > > > > > > > >>Generally, project artifacts should be hosted on > > ASF > > >> > > > > > > > infrastructure. > > >> > > > > > > > > > > +1 on this. The Google doc => wiki workflow has > also > > >> > worked > > >> > > > > well > > >> > > > > > in > > >> > > > > > > > > most > > >> > > > > > > > > > > companies for me. > > >> > > > > > > > > > > > > >> > > > > > > > > > > Looks like we have consensus? > > >> > > > > > > > > > > > > >> > > > > > > > > > > On Fri, Feb 8, 2019 at 2:46 PM Suneel Marthi < > > >> > > > > > > > suneel.mar...@gmail.com> > > >> > > > > > > > > > > wrote: > > >> > > > > > > > > > > > > >> > > > > > > > > > > > Most of the Flink FLIPs r initially done in a > > >> working > > >> > > > google > > >> > > > > > doc > > >> > > > > > > > and > > >> > > > > > > > > > the > > >> > > > > > > > > > > > finally draft is captured on wiki. > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > Example ---> > > >> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-24+-+SQL+Client > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > could do the same with Hudi --> as long as the > > >> google > > >> > doc > > >> > > > is > > >> > > > > > > > editable > > >> > > > > > > > > > by > > >> > > > > > > > > > > > one and all > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > On Fri, Feb 8, 2019 at 5:29 PM Thomas Weise < > > >> > > > t...@apache.org> > > >> > > > > > > > wrote: > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > Generally, project artifacts should be hosted > on > > >> ASF > > >> > > > > > > > > infrastructure. > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > Google drive is good for collaboration, but > not > > a > > >> > > > > replacement > > >> > > > > > > for > > >> > > > > > > > > > > project > > >> > > > > > > > > > > > > documentation. > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > On Fri, Feb 8, 2019 at 2:05 PM Vinoth Chandar > < > > >> > > > > > > vin...@apache.org > > >> > > > > > > > > > > >> > > > > > > > > > > wrote: > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > I think there is good search on gdrive. Let > me > > >> > > confirm > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > On Thu, Feb 7, 2019 at 7:28 AM Prasanna < > > >> > > > > > prasa...@apache.org > > >> > > > > > > > > > >> > > > > > > > > > wrote: > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > How could we search through a bunch of > > Google > > >> > docs? > > >> > > > Is > > >> > > > > > > there > > >> > > > > > > > a > > >> > > > > > > > > > way > > >> > > > > > > > > > > to > > >> > > > > > > > > > > > > do > > >> > > > > > > > > > > > > > > this? > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > If not may be moving the final specs into > > >> cwiki > > >> > and > > >> > > > > > > deleting > > >> > > > > > > > > the > > >> > > > > > > > > > > > Google > > >> > > > > > > > > > > > > > doc > > >> > > > > > > > > > > > > > > may be better. > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > -Prasanna > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > On Thu, Feb 7, 2019, 2:08 AM Vinoth > Chandar > > < > > >> > > > > > > > vin...@apache.org > > >> > > > > > > > > > > > >> > > > > > > > > > > > wrote: > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > I dont mind calling them HIPs at all :) > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > On Thu, Feb 7, 2019 at 2:06 AM Suneel > > >> Marthi < > > >> > > > > > > > > > smar...@apache.org > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > wrote: > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > using google docs is fine, > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > u might want to look at how Kafka KIPs > > and > > >> > > Flink > > >> > > > > > FLIPs > > >> > > > > > > r > > >> > > > > > > > > done > > >> > > > > > > > > > > to > > >> > > > > > > > > > > > > see > > >> > > > > > > > > > > > > > > how > > >> > > > > > > > > > > > > > > > > other projects r doing it now and its > > been > > >> > > > working > > >> > > > > > well > > >> > > > > > > > > > there. > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Start HIPs ??? > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > On Thu, Feb 7, 2019 at 11:04 AM Vinoth > > >> > Chandar > > >> > > < > > >> > > > > > > > > > > > > > > > > mail.vinoth.chan...@gmail.com> wrote: > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > +1 on using Google docs for sharing > > >> design > > >> > > > docs. > > >> > > > > > IIRC > > >> > > > > > > > > even > > >> > > > > > > > > > > > spark > > >> > > > > > > > > > > > > > has > > >> > > > > > > > > > > > > > > > used > > >> > > > > > > > > > > > > > > > > > google docs for collaborating on > > >> designs. > > >> > We > > >> > > > > > already > > >> > > > > > > > have > > >> > > > > > > > > > the > > >> > > > > > > > > > > > > gmail > > >> > > > > > > > > > > > > > > > > account > > >> > > > > > > > > > > > > > > > > > shared with private@. > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > Now, once the designs are finalized, > > do > > >> we > > >> > > move > > >> > > > > > them > > >> > > > > > > to > > >> > > > > > > > > > cwiki > > >> > > > > > > > > > > > or > > >> > > > > > > > > > > > > > just > > >> > > > > > > > > > > > > > > > > keep > > >> > > > > > > > > > > > > > > > > > in gdoc itself? For eg, kafka has > all > > >> the > > >> > > > design > > >> > > > > > docs > > >> > > > > > > > in > > >> > > > > > > > > > > > cwiki. I > > >> > > > > > > > > > > > > > am > > >> > > > > > > > > > > > > > > > okay > > >> > > > > > > > > > > > > > > > > > with just sticking to gdocs > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > Any thoughts? Others? > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > On Sun, Feb 3, 2019 at 11:32 PM > > nishith > > >> > > > agarwal < > > >> > > > > > > > > > > > > > nagar...@apache.org > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > wrote: > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > Hi All, > > >> > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > I wanted to start a discussion > > around > > >> > > > > > contributing > > >> > > > > > > > and > > >> > > > > > > > > > > > > > > collaborating > > >> > > > > > > > > > > > > > > > on > > >> > > > > > > > > > > > > > > > > > > design for large features. I've > > >> noticed > > >> > > that > > >> > > > > for > > >> > > > > > > some > > >> > > > > > > > > of > > >> > > > > > > > > > > the > > >> > > > > > > > > > > > > > > projects > > >> > > > > > > > > > > > > > > > > > such > > >> > > > > > > > > > > > > > > > > > > as apache beam, google docs is > > chosen > > >> as > > >> > a > > >> > > > > means > > >> > > > > > to > > >> > > > > > > > > > > > collaborate > > >> > > > > > > > > > > > > > on > > >> > > > > > > > > > > > > > > > > design > > >> > > > > > > > > > > > > > > > > > > documents and large changes (Find > it > > >> here > > >> > > > > > > > > > > > > > > > > > > < > > https://beam.apache.org/contribute/ > > >> >. > > >> > > Read > > >> > > > > > under > > >> > > > > > > > > "Share > > >> > > > > > > > > > > > your > > >> > > > > > > > > > > > > > > > intent" > > >> > > > > > > > > > > > > > > > > > > section). > > >> > > > > > > > > > > > > > > > > > > I've researched and seen some > other > > >> > > projects > > >> > > > > > (such > > >> > > > > > > as > > >> > > > > > > > > > > Hadoop) > > >> > > > > > > > > > > > > use > > >> > > > > > > > > > > > > > > > PDF's > > >> > > > > > > > > > > > > > > > > > to > > >> > > > > > > > > > > > > > > > > > > share design documents which may > be > > is > > >> > the > > >> > > > > final > > >> > > > > > > > design > > >> > > > > > > > > > but > > >> > > > > > > > > > > > > > unsure > > >> > > > > > > > > > > > > > > > how > > >> > > > > > > > > > > > > > > > > > > collaboration happened on those. > > >> > > > > > > > > > > > > > > > > > > Some projects like apache spark > > >> mention > > >> > the > > >> > > > use > > >> > > > > > of > > >> > > > > > > > > > mailing > > >> > > > > > > > > > > > > lists > > >> > > > > > > > > > > > > > > for > > >> > > > > > > > > > > > > > > > > > design > > >> > > > > > > > > > > > > > > > > > > discussions. (Find it here < > > >> > > > > > > > > > > > > > > > > > https://spark.apache.org/contributing.html > > >> > > > > > > > > > > > > > > > > >. > > >> > > > > > > > > > > > > > > > > > > Read under Contributing bug > > reports). > > >> > > > > > > > > > > > > > > > > > > Traditionally, for Hudi, we've > used > > >> > google > > >> > > > docs > > >> > > > > > to > > >> > > > > > > > > > > > collaborate > > >> > > > > > > > > > > > > > and > > >> > > > > > > > > > > > > > > it > > >> > > > > > > > > > > > > > > > > has > > >> > > > > > > > > > > > > > > > > > > been very useful. We also have > some > > >> > > existing > > >> > > > > > > > > outstanding > > >> > > > > > > > > > > > design > > >> > > > > > > > > > > > > > > > > documents > > >> > > > > > > > > > > > > > > > > > > in google docs and would favor > using > > >> the > > >> > > same > > >> > > > > > going > > >> > > > > > > > > > forward > > >> > > > > > > > > > > > for > > >> > > > > > > > > > > > > > > > apache > > >> > > > > > > > > > > > > > > > > > > hudi. > > >> > > > > > > > > > > > > > > > > > > I'd like to gather your thoughts > on > > >> this. > > >> > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > Thanks, > > >> > > > > > > > > > > > > > > > > > > Nishith > > >> > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > >
