Vinoth, I've got a template for HIP (Hudi Improvement Plan) going on the google drive. Working on writing up a description of what each component of the template means (which we can put on cWiki). PS : I've borrowed majorly from KIP.
Thanks On Thu, Feb 28, 2019 at 4:26 PM Vinoth Chandar <vin...@apache.org> wrote: > Hi Nishith, > > Curious if you are further along on this. I also want to start one around > easy migration of non Hudi datasets.. > > Thanks > Vinoth > > On Mon, Feb 25, 2019 at 7:10 AM Vinoth Chandar <vin...@apache.org> wrote: > > > My suggestion would be to first define the HIP template and the process, > > document , push the site out.. > > Then we can use the GlobalIndex doc to test drive the process. > > Are there other specific things you want to test out on cWiki itself? > > > > On Sun, Feb 24, 2019 at 10:09 AM nishith agarwal <n3.nas...@gmail.com> > > wrote: > > > >> Okay, great. Yeah, I'll look into the templates and guidelines around > >> writing new proposals/designs. > >> In parallel, I'll share the GlobalIndex document on google drive and we > >> can > >> try to put that up on cWiki to test things out ? (or any other document > >> works fine too). > >> > >> -Nishith > >> > >> On Fri, Feb 22, 2019 at 12:53 PM Vinoth Chandar <vin...@apache.org> > >> wrote: > >> > >> > Nishith, that google account is already owned by private@ . It's > >> folders > >> > are publicly viewable.. > >> > So we should be good to go. > >> > > >> > I think next step is to define a standard HIP (Hudi Improvement > >> Proposal) > >> > template and guidelines around writing a new proposal.. > >> > This is what you were originally trying to do, IIRC? > >> > > >> > On Fri, Feb 22, 2019 at 12:40 PM nishith agarwal <n3.nas...@gmail.com > > > >> > wrote: > >> > > >> > > Great! Thanks Thomas. > >> > > > >> > > Vinoth, > >> > > Can we use our existing hoodie google drive and share it with @dev ? > >> > > > >> > > -Nishith > >> > > > >> > > On Thu, Feb 21, 2019 at 9:52 PM Thomas Weise <t...@apache.org> > wrote: > >> > > > >> > > > I created the space and added user vinoth as space admin. > >> > > > > >> > > > https://cwiki.apache.org/confluence/display/HUDI/Home > >> > > > > >> > > > Thomas > >> > > > > >> > > > On Tue, Feb 19, 2019 at 4:25 PM Vinoth Chandar <vin...@apache.org > > > >> > > wrote: > >> > > > > >> > > > > I think its listed here? > >> > > > > https://cwiki.apache.org/confluence/display/INFRA/Cwiki > >> > > > > https://selfserve.apache.org ? > >> > > > > > >> > > > > On Mon, Feb 18, 2019 at 9:13 PM Thomas Weise <t...@apache.org> > >> wrote: > >> > > > > > >> > > > > > The confluence wiki space should be created in any case. I > >> believe > >> > > this > >> > > > > can > >> > > > > > be done by mentors through the infra self serve portal, but > >> don't > >> > > have > >> > > > it > >> > > > > > in front of me at the moment. > >> > > > > > > >> > > > > > -- > >> > > > > > sent from mobile > >> > > > > > > >> > > > > > On Mon, Feb 18, 2019, 5:51 PM Vinoth Chandar < > vin...@apache.org > >> > > >> > > > wrote: > >> > > > > > > >> > > > > > > Bumping this thread up. We have a few big designs upcoming.. > >> > > > > > > Love to get this going, using this new setup. > >> > > > > > > > >> > > > > > > On Wed, Feb 13, 2019 at 8:54 PM nishith agarwal < > >> > > n3.nas...@gmail.com > >> > > > > > >> > > > > > > wrote: > >> > > > > > > > >> > > > > > > > I can go ahead and setup the google drive as well as > cWiki. > >> > > > > > > > Mentors, > >> > > > > > > > I created a new account at https://cwiki.apache.org/. > >> Should I > >> > > > > create > >> > > > > > a > >> > > > > > > > new > >> > > > > > > > space with name "Hudi" where we could start adding the > final > >> > > > drafts ? > >> > > > > > > > > >> > > > > > > > Thanks, > >> > > > > > > > Nishith > >> > > > > > > > > >> > > > > > > > On Sun, Feb 10, 2019 at 11:08 AM Prasanna < > >> > > prasanna....@gmail.com> > >> > > > > > > wrote: > >> > > > > > > > > >> > > > > > > > > +1 > >> > > > > > > > > > >> > > > > > > > > On Fri, Feb 8, 2019, 7:46 PM Suneel Marthi < > >> > smar...@apache.org > >> > > > > >> > > > > > wrote: > >> > > > > > > > > > >> > > > > > > > > > +1 > >> > > > > > > > > > > >> > > > > > > > > > On Fri, Feb 8, 2019 at 6:39 PM Vinoth Chandar < > >> > > > vin...@apache.org > >> > > > > > > >> > > > > > > > wrote: > >> > > > > > > > > > > >> > > > > > > > > > > >>Generally, project artifacts should be hosted on > ASF > >> > > > > > > > infrastructure. > >> > > > > > > > > > > +1 on this. The Google doc => wiki workflow has also > >> > worked > >> > > > > well > >> > > > > > in > >> > > > > > > > > most > >> > > > > > > > > > > companies for me. > >> > > > > > > > > > > > >> > > > > > > > > > > Looks like we have consensus? > >> > > > > > > > > > > > >> > > > > > > > > > > On Fri, Feb 8, 2019 at 2:46 PM Suneel Marthi < > >> > > > > > > > suneel.mar...@gmail.com> > >> > > > > > > > > > > wrote: > >> > > > > > > > > > > > >> > > > > > > > > > > > Most of the Flink FLIPs r initially done in a > >> working > >> > > > google > >> > > > > > doc > >> > > > > > > > and > >> > > > > > > > > > the > >> > > > > > > > > > > > finally draft is captured on wiki. > >> > > > > > > > > > > > > >> > > > > > > > > > > > Example ---> > >> > > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > >> > > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-24+-+SQL+Client > >> > > > > > > > > > > > > >> > > > > > > > > > > > could do the same with Hudi --> as long as the > >> google > >> > doc > >> > > > is > >> > > > > > > > editable > >> > > > > > > > > > by > >> > > > > > > > > > > > one and all > >> > > > > > > > > > > > > >> > > > > > > > > > > > On Fri, Feb 8, 2019 at 5:29 PM Thomas Weise < > >> > > > t...@apache.org> > >> > > > > > > > wrote: > >> > > > > > > > > > > > > >> > > > > > > > > > > > > Generally, project artifacts should be hosted on > >> ASF > >> > > > > > > > > infrastructure. > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > Google drive is good for collaboration, but not > a > >> > > > > replacement > >> > > > > > > for > >> > > > > > > > > > > project > >> > > > > > > > > > > > > documentation. > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > On Fri, Feb 8, 2019 at 2:05 PM Vinoth Chandar < > >> > > > > > > vin...@apache.org > >> > > > > > > > > > >> > > > > > > > > > > wrote: > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > I think there is good search on gdrive. Let me > >> > > confirm > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > On Thu, Feb 7, 2019 at 7:28 AM Prasanna < > >> > > > > > prasa...@apache.org > >> > > > > > > > > >> > > > > > > > > > wrote: > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > How could we search through a bunch of > Google > >> > docs? > >> > > > Is > >> > > > > > > there > >> > > > > > > > a > >> > > > > > > > > > way > >> > > > > > > > > > > to > >> > > > > > > > > > > > > do > >> > > > > > > > > > > > > > > this? > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > If not may be moving the final specs into > >> cwiki > >> > and > >> > > > > > > deleting > >> > > > > > > > > the > >> > > > > > > > > > > > Google > >> > > > > > > > > > > > > > doc > >> > > > > > > > > > > > > > > may be better. > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > -Prasanna > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > On Thu, Feb 7, 2019, 2:08 AM Vinoth Chandar > < > >> > > > > > > > vin...@apache.org > >> > > > > > > > > > > >> > > > > > > > > > > > wrote: > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > I dont mind calling them HIPs at all :) > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > On Thu, Feb 7, 2019 at 2:06 AM Suneel > >> Marthi < > >> > > > > > > > > > smar...@apache.org > >> > > > > > > > > > > > > >> > > > > > > > > > > > > > wrote: > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > using google docs is fine, > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > u might want to look at how Kafka KIPs > and > >> > > Flink > >> > > > > > FLIPs > >> > > > > > > r > >> > > > > > > > > done > >> > > > > > > > > > > to > >> > > > > > > > > > > > > see > >> > > > > > > > > > > > > > > how > >> > > > > > > > > > > > > > > > > other projects r doing it now and its > been > >> > > > working > >> > > > > > well > >> > > > > > > > > > there. > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Start HIPs ??? > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > On Thu, Feb 7, 2019 at 11:04 AM Vinoth > >> > Chandar > >> > > < > >> > > > > > > > > > > > > > > > > mail.vinoth.chan...@gmail.com> wrote: > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > +1 on using Google docs for sharing > >> design > >> > > > docs. > >> > > > > > IIRC > >> > > > > > > > > even > >> > > > > > > > > > > > spark > >> > > > > > > > > > > > > > has > >> > > > > > > > > > > > > > > > used > >> > > > > > > > > > > > > > > > > > google docs for collaborating on > >> designs. > >> > We > >> > > > > > already > >> > > > > > > > have > >> > > > > > > > > > the > >> > > > > > > > > > > > > gmail > >> > > > > > > > > > > > > > > > > account > >> > > > > > > > > > > > > > > > > > shared with private@. > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > Now, once the designs are finalized, > do > >> we > >> > > move > >> > > > > > them > >> > > > > > > to > >> > > > > > > > > > cwiki > >> > > > > > > > > > > > or > >> > > > > > > > > > > > > > just > >> > > > > > > > > > > > > > > > > keep > >> > > > > > > > > > > > > > > > > > in gdoc itself? For eg, kafka has all > >> the > >> > > > design > >> > > > > > docs > >> > > > > > > > in > >> > > > > > > > > > > > cwiki. I > >> > > > > > > > > > > > > > am > >> > > > > > > > > > > > > > > > okay > >> > > > > > > > > > > > > > > > > > with just sticking to gdocs > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > Any thoughts? Others? > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > On Sun, Feb 3, 2019 at 11:32 PM > nishith > >> > > > agarwal < > >> > > > > > > > > > > > > > nagar...@apache.org > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > wrote: > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > Hi All, > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > I wanted to start a discussion > around > >> > > > > > contributing > >> > > > > > > > and > >> > > > > > > > > > > > > > > collaborating > >> > > > > > > > > > > > > > > > on > >> > > > > > > > > > > > > > > > > > > design for large features. I've > >> noticed > >> > > that > >> > > > > for > >> > > > > > > some > >> > > > > > > > > of > >> > > > > > > > > > > the > >> > > > > > > > > > > > > > > projects > >> > > > > > > > > > > > > > > > > > such > >> > > > > > > > > > > > > > > > > > > as apache beam, google docs is > chosen > >> as > >> > a > >> > > > > means > >> > > > > > to > >> > > > > > > > > > > > collaborate > >> > > > > > > > > > > > > > on > >> > > > > > > > > > > > > > > > > design > >> > > > > > > > > > > > > > > > > > > documents and large changes (Find it > >> here > >> > > > > > > > > > > > > > > > > > > < > https://beam.apache.org/contribute/ > >> >. > >> > > Read > >> > > > > > under > >> > > > > > > > > "Share > >> > > > > > > > > > > > your > >> > > > > > > > > > > > > > > > intent" > >> > > > > > > > > > > > > > > > > > > section). > >> > > > > > > > > > > > > > > > > > > I've researched and seen some other > >> > > projects > >> > > > > > (such > >> > > > > > > as > >> > > > > > > > > > > Hadoop) > >> > > > > > > > > > > > > use > >> > > > > > > > > > > > > > > > PDF's > >> > > > > > > > > > > > > > > > > > to > >> > > > > > > > > > > > > > > > > > > share design documents which may be > is > >> > the > >> > > > > final > >> > > > > > > > design > >> > > > > > > > > > but > >> > > > > > > > > > > > > > unsure > >> > > > > > > > > > > > > > > > how > >> > > > > > > > > > > > > > > > > > > collaboration happened on those. > >> > > > > > > > > > > > > > > > > > > Some projects like apache spark > >> mention > >> > the > >> > > > use > >> > > > > > of > >> > > > > > > > > > mailing > >> > > > > > > > > > > > > lists > >> > > > > > > > > > > > > > > for > >> > > > > > > > > > > > > > > > > > design > >> > > > > > > > > > > > > > > > > > > discussions. (Find it here < > >> > > > > > > > > > > > > > > > > https://spark.apache.org/contributing.html > >> > > > > > > > > > > > > > > > > >. > >> > > > > > > > > > > > > > > > > > > Read under Contributing bug > reports). > >> > > > > > > > > > > > > > > > > > > Traditionally, for Hudi, we've used > >> > google > >> > > > docs > >> > > > > > to > >> > > > > > > > > > > > collaborate > >> > > > > > > > > > > > > > and > >> > > > > > > > > > > > > > > it > >> > > > > > > > > > > > > > > > > has > >> > > > > > > > > > > > > > > > > > > been very useful. We also have some > >> > > existing > >> > > > > > > > > outstanding > >> > > > > > > > > > > > design > >> > > > > > > > > > > > > > > > > documents > >> > > > > > > > > > > > > > > > > > > in google docs and would favor using > >> the > >> > > same > >> > > > > > going > >> > > > > > > > > > forward > >> > > > > > > > > > > > for > >> > > > > > > > > > > > > > > > apache > >> > > > > > > > > > > > > > > > > > > hudi. > >> > > > > > > > > > > > > > > > > > > I'd like to gather your thoughts on > >> this. > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > Thanks, > >> > > > > > > > > > > > > > > > > > > Nishith > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >
