Hi Nishith, Curious if you are further along on this. I also want to start one around easy migration of non Hudi datasets..
Thanks Vinoth On Mon, Feb 25, 2019 at 7:10 AM Vinoth Chandar <vin...@apache.org> wrote: > My suggestion would be to first define the HIP template and the process, > document , push the site out.. > Then we can use the GlobalIndex doc to test drive the process. > Are there other specific things you want to test out on cWiki itself? > > On Sun, Feb 24, 2019 at 10:09 AM nishith agarwal <n3.nas...@gmail.com> > wrote: > >> Okay, great. Yeah, I'll look into the templates and guidelines around >> writing new proposals/designs. >> In parallel, I'll share the GlobalIndex document on google drive and we >> can >> try to put that up on cWiki to test things out ? (or any other document >> works fine too). >> >> -Nishith >> >> On Fri, Feb 22, 2019 at 12:53 PM Vinoth Chandar <vin...@apache.org> >> wrote: >> >> > Nishith, that google account is already owned by private@ . It's >> folders >> > are publicly viewable.. >> > So we should be good to go. >> > >> > I think next step is to define a standard HIP (Hudi Improvement >> Proposal) >> > template and guidelines around writing a new proposal.. >> > This is what you were originally trying to do, IIRC? >> > >> > On Fri, Feb 22, 2019 at 12:40 PM nishith agarwal <n3.nas...@gmail.com> >> > wrote: >> > >> > > Great! Thanks Thomas. >> > > >> > > Vinoth, >> > > Can we use our existing hoodie google drive and share it with @dev ? >> > > >> > > -Nishith >> > > >> > > On Thu, Feb 21, 2019 at 9:52 PM Thomas Weise <t...@apache.org> wrote: >> > > >> > > > I created the space and added user vinoth as space admin. >> > > > >> > > > https://cwiki.apache.org/confluence/display/HUDI/Home >> > > > >> > > > Thomas >> > > > >> > > > On Tue, Feb 19, 2019 at 4:25 PM Vinoth Chandar <vin...@apache.org> >> > > wrote: >> > > > >> > > > > I think its listed here? >> > > > > https://cwiki.apache.org/confluence/display/INFRA/Cwiki >> > > > > https://selfserve.apache.org ? >> > > > > >> > > > > On Mon, Feb 18, 2019 at 9:13 PM Thomas Weise <t...@apache.org> >> wrote: >> > > > > >> > > > > > The confluence wiki space should be created in any case. I >> believe >> > > this >> > > > > can >> > > > > > be done by mentors through the infra self serve portal, but >> don't >> > > have >> > > > it >> > > > > > in front of me at the moment. >> > > > > > >> > > > > > -- >> > > > > > sent from mobile >> > > > > > >> > > > > > On Mon, Feb 18, 2019, 5:51 PM Vinoth Chandar <vin...@apache.org >> > >> > > > wrote: >> > > > > > >> > > > > > > Bumping this thread up. We have a few big designs upcoming.. >> > > > > > > Love to get this going, using this new setup. >> > > > > > > >> > > > > > > On Wed, Feb 13, 2019 at 8:54 PM nishith agarwal < >> > > n3.nas...@gmail.com >> > > > > >> > > > > > > wrote: >> > > > > > > >> > > > > > > > I can go ahead and setup the google drive as well as cWiki. >> > > > > > > > Mentors, >> > > > > > > > I created a new account at https://cwiki.apache.org/. >> Should I >> > > > > create >> > > > > > a >> > > > > > > > new >> > > > > > > > space with name "Hudi" where we could start adding the final >> > > > drafts ? >> > > > > > > > >> > > > > > > > Thanks, >> > > > > > > > Nishith >> > > > > > > > >> > > > > > > > On Sun, Feb 10, 2019 at 11:08 AM Prasanna < >> > > prasanna....@gmail.com> >> > > > > > > wrote: >> > > > > > > > >> > > > > > > > > +1 >> > > > > > > > > >> > > > > > > > > On Fri, Feb 8, 2019, 7:46 PM Suneel Marthi < >> > smar...@apache.org >> > > > >> > > > > > wrote: >> > > > > > > > > >> > > > > > > > > > +1 >> > > > > > > > > > >> > > > > > > > > > On Fri, Feb 8, 2019 at 6:39 PM Vinoth Chandar < >> > > > vin...@apache.org >> > > > > > >> > > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > > >>Generally, project artifacts should be hosted on ASF >> > > > > > > > infrastructure. >> > > > > > > > > > > +1 on this. The Google doc => wiki workflow has also >> > worked >> > > > > well >> > > > > > in >> > > > > > > > > most >> > > > > > > > > > > companies for me. >> > > > > > > > > > > >> > > > > > > > > > > Looks like we have consensus? >> > > > > > > > > > > >> > > > > > > > > > > On Fri, Feb 8, 2019 at 2:46 PM Suneel Marthi < >> > > > > > > > suneel.mar...@gmail.com> >> > > > > > > > > > > wrote: >> > > > > > > > > > > >> > > > > > > > > > > > Most of the Flink FLIPs r initially done in a >> working >> > > > google >> > > > > > doc >> > > > > > > > and >> > > > > > > > > > the >> > > > > > > > > > > > finally draft is captured on wiki. >> > > > > > > > > > > > >> > > > > > > > > > > > Example ---> >> > > > > > > > > > > > >> > > > > > > > > >> > > > > > >> > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-24+-+SQL+Client >> > > > > > > > > > > > >> > > > > > > > > > > > could do the same with Hudi --> as long as the >> google >> > doc >> > > > is >> > > > > > > > editable >> > > > > > > > > > by >> > > > > > > > > > > > one and all >> > > > > > > > > > > > >> > > > > > > > > > > > On Fri, Feb 8, 2019 at 5:29 PM Thomas Weise < >> > > > t...@apache.org> >> > > > > > > > wrote: >> > > > > > > > > > > > >> > > > > > > > > > > > > Generally, project artifacts should be hosted on >> ASF >> > > > > > > > > infrastructure. >> > > > > > > > > > > > > >> > > > > > > > > > > > > Google drive is good for collaboration, but not a >> > > > > replacement >> > > > > > > for >> > > > > > > > > > > project >> > > > > > > > > > > > > documentation. >> > > > > > > > > > > > > >> > > > > > > > > > > > > On Fri, Feb 8, 2019 at 2:05 PM Vinoth Chandar < >> > > > > > > vin...@apache.org >> > > > > > > > > >> > > > > > > > > > > wrote: >> > > > > > > > > > > > > >> > > > > > > > > > > > > > I think there is good search on gdrive. Let me >> > > confirm >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > On Thu, Feb 7, 2019 at 7:28 AM Prasanna < >> > > > > > prasa...@apache.org >> > > > > > > > >> > > > > > > > > > wrote: >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > > How could we search through a bunch of Google >> > docs? >> > > > Is >> > > > > > > there >> > > > > > > > a >> > > > > > > > > > way >> > > > > > > > > > > to >> > > > > > > > > > > > > do >> > > > > > > > > > > > > > > this? >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > If not may be moving the final specs into >> cwiki >> > and >> > > > > > > deleting >> > > > > > > > > the >> > > > > > > > > > > > Google >> > > > > > > > > > > > > > doc >> > > > > > > > > > > > > > > may be better. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > -Prasanna >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > On Thu, Feb 7, 2019, 2:08 AM Vinoth Chandar < >> > > > > > > > vin...@apache.org >> > > > > > > > > > >> > > > > > > > > > > > wrote: >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > I dont mind calling them HIPs at all :) >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > On Thu, Feb 7, 2019 at 2:06 AM Suneel >> Marthi < >> > > > > > > > > > smar...@apache.org >> > > > > > > > > > > > >> > > > > > > > > > > > > > wrote: >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > using google docs is fine, >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > u might want to look at how Kafka KIPs and >> > > Flink >> > > > > > FLIPs >> > > > > > > r >> > > > > > > > > done >> > > > > > > > > > > to >> > > > > > > > > > > > > see >> > > > > > > > > > > > > > > how >> > > > > > > > > > > > > > > > > other projects r doing it now and its been >> > > > working >> > > > > > well >> > > > > > > > > > there. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Start HIPs ??? >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > On Thu, Feb 7, 2019 at 11:04 AM Vinoth >> > Chandar >> > > < >> > > > > > > > > > > > > > > > > mail.vinoth.chan...@gmail.com> wrote: >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > +1 on using Google docs for sharing >> design >> > > > docs. >> > > > > > IIRC >> > > > > > > > > even >> > > > > > > > > > > > spark >> > > > > > > > > > > > > > has >> > > > > > > > > > > > > > > > used >> > > > > > > > > > > > > > > > > > google docs for collaborating on >> designs. >> > We >> > > > > > already >> > > > > > > > have >> > > > > > > > > > the >> > > > > > > > > > > > > gmail >> > > > > > > > > > > > > > > > > account >> > > > > > > > > > > > > > > > > > shared with private@. >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > Now, once the designs are finalized, do >> we >> > > move >> > > > > > them >> > > > > > > to >> > > > > > > > > > cwiki >> > > > > > > > > > > > or >> > > > > > > > > > > > > > just >> > > > > > > > > > > > > > > > > keep >> > > > > > > > > > > > > > > > > > in gdoc itself? For eg, kafka has all >> the >> > > > design >> > > > > > docs >> > > > > > > > in >> > > > > > > > > > > > cwiki. I >> > > > > > > > > > > > > > am >> > > > > > > > > > > > > > > > okay >> > > > > > > > > > > > > > > > > > with just sticking to gdocs >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > Any thoughts? Others? >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > On Sun, Feb 3, 2019 at 11:32 PM nishith >> > > > agarwal < >> > > > > > > > > > > > > > nagar...@apache.org >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > wrote: >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > Hi All, >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > I wanted to start a discussion around >> > > > > > contributing >> > > > > > > > and >> > > > > > > > > > > > > > > collaborating >> > > > > > > > > > > > > > > > on >> > > > > > > > > > > > > > > > > > > design for large features. I've >> noticed >> > > that >> > > > > for >> > > > > > > some >> > > > > > > > > of >> > > > > > > > > > > the >> > > > > > > > > > > > > > > projects >> > > > > > > > > > > > > > > > > > such >> > > > > > > > > > > > > > > > > > > as apache beam, google docs is chosen >> as >> > a >> > > > > means >> > > > > > to >> > > > > > > > > > > > collaborate >> > > > > > > > > > > > > > on >> > > > > > > > > > > > > > > > > design >> > > > > > > > > > > > > > > > > > > documents and large changes (Find it >> here >> > > > > > > > > > > > > > > > > > > <https://beam.apache.org/contribute/ >> >. >> > > Read >> > > > > > under >> > > > > > > > > "Share >> > > > > > > > > > > > your >> > > > > > > > > > > > > > > > intent" >> > > > > > > > > > > > > > > > > > > section). >> > > > > > > > > > > > > > > > > > > I've researched and seen some other >> > > projects >> > > > > > (such >> > > > > > > as >> > > > > > > > > > > Hadoop) >> > > > > > > > > > > > > use >> > > > > > > > > > > > > > > > PDF's >> > > > > > > > > > > > > > > > > > to >> > > > > > > > > > > > > > > > > > > share design documents which may be is >> > the >> > > > > final >> > > > > > > > design >> > > > > > > > > > but >> > > > > > > > > > > > > > unsure >> > > > > > > > > > > > > > > > how >> > > > > > > > > > > > > > > > > > > collaboration happened on those. >> > > > > > > > > > > > > > > > > > > Some projects like apache spark >> mention >> > the >> > > > use >> > > > > > of >> > > > > > > > > > mailing >> > > > > > > > > > > > > lists >> > > > > > > > > > > > > > > for >> > > > > > > > > > > > > > > > > > design >> > > > > > > > > > > > > > > > > > > discussions. (Find it here < >> > > > > > > > > > > > > > > > https://spark.apache.org/contributing.html >> > > > > > > > > > > > > > > > > >. >> > > > > > > > > > > > > > > > > > > Read under Contributing bug reports). >> > > > > > > > > > > > > > > > > > > Traditionally, for Hudi, we've used >> > google >> > > > docs >> > > > > > to >> > > > > > > > > > > > collaborate >> > > > > > > > > > > > > > and >> > > > > > > > > > > > > > > it >> > > > > > > > > > > > > > > > > has >> > > > > > > > > > > > > > > > > > > been very useful. We also have some >> > > existing >> > > > > > > > > outstanding >> > > > > > > > > > > > design >> > > > > > > > > > > > > > > > > documents >> > > > > > > > > > > > > > > > > > > in google docs and would favor using >> the >> > > same >> > > > > > going >> > > > > > > > > > forward >> > > > > > > > > > > > for >> > > > > > > > > > > > > > > > apache >> > > > > > > > > > > > > > > > > > > hudi. >> > > > > > > > > > > > > > > > > > > I'd like to gather your thoughts on >> this. >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > Thanks, >> > > > > > > > > > > > > > > > > > > Nishith >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
