Re: [PROPOSAL] Hudi Web UI
Thanks for doing it! Will review sometime this week. On Mon, Sep 23, 2019 at 5:43 PM vino yang wrote: > Thanks Taher, great job! Will have another look soon. Best, Vino On > 09/24/2019 02:15, Taher Koitawala wrote: Hi All, Hip has been > migrated to confluence. Please take a look. > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130027233 > Regards, Taher Koitawala On Mon, Sep 23, 2019 at 10:17 PM Taher Koitawala < > [email protected]> wrote: > Yup got it. Thanks Vinoth > > On Mon, Sep > 23, 2019, 10:02 PM Vinoth Chandar wrote: > >> Now? >> > >> On Mon, Sep 23, 2019 at 9:05 AM Taher Koitawala > >> wrote: >> >> > Hey Vinoth still the same. No perms >> > >> > On Mon, Sep > 23, 2019 at 9:24 PM Vinoth Chandar >> wrote: >> > >> > > > thats correct. you should have perms now. >> > > >> > > On Sun, Sep 22, > 2019 at 11:06 PM Taher Koitawala >> > > wrote: >> > > > >> > > > Hi Vinoth, >> > > >I do not have access on > confluence to create a page >> in >> > the >> > > > HIP section or even > copy the HIP template. Please can you give >> access? >> > My >> > > > id > is taherk77. >> > > > >> > > > Regards, >> > > > Taher Koitawala >> > > > > >> > > > On Sun, Sep 22, 2019 at 6:34 PM Vinoth Chandar > >> > > wrote: >> > > > >> > > > > Taher, can we please move the HIP to the > cWiki space as documented >> > here >> > > > > >> > > > > >> > > > >> > > > >> > >> > https://cwiki.apache.org/confluence/display/HUDI/Hudi+Improvement+Plan+Details+and+Process > >> > > > > >> > > > > >> > > > > Would love to take a pass at it. This will > definitely improve >> > > usability.. >> > > > > >> > > > > @leesf I think > we can use the standalone timeline server bundle >> you >> > > > worked >> > > > > > on, and have a central timeline server for all Hudi jobs within >> > your >> > > > org.. >> > > > > Then, if you host an UI endpoint on that > server, we can visualize >> > much >> > > of >> > > > > what Tahen put up > on the doc. thoughts? >> > > > > >> > > > > On Sun, Sep 22, 2019 at 4:07 AM > Taher Koitawala < >> [email protected]> >> > > > > wrote: >> > > > > >> > > > > > > Hi Leesf, >> > > > > >Thank you for your interest. > HIP already has been >> > > > implemented >> > > > > in >> > > > > > terms > of design and components we need to see. The link is given >> > > below. >> > > > > > > Please free to chime in, to add and implement. >> > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > >> > > > > > >> > > > > > Regards, >> > > > > > Taher Koitawala >> > > > > > > >> > > > > > On Sun, Sep 22, 2019, 3:26 PM leesf > >> wrote: >> > > > > > >> > > > > > > + 1 to this valuable HIP. >> > > > > > > > And I am also very interested, perhaps together to implement >> this >> > > > > HIP. >> > > > > > > >> > > > > > > Best, >> > > > > > > Leesf >> > > > > > > > >> > > > > > > Taher Koitawala 于2019年9月22日周日 > 下午2:09写道： >> > > > > > > >> > > > > > > > Guys it not only includes tables > views and admin kind of >> > > > > > > views(Compactions) >> > > > > > > > > but it also includes 'Metadata lineage' which can help users >> > know >> > > > > how >> > > > > > > this >> > > > > > > > Hudi dataset merged and also > another strong feature is >> creating >> > > > > > > > DeltaStreamer jobs > through the webui and having >> DeltaStreamer >> > > > > > > > > templates(Makes sharing jobs easy). I think those are really >> > > really > >> > > > > > > strong >> > > > > > > > features. >> > > > > > > > >> > > > > > > > > >> > > > > > > > Regards, >> > > > > > > > Taher Koitawala >> > > > > > > > > >> > > > > > > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha > Saktheeswaran >> > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > +1 for adding web ui. The web ui viz for table > configs >> would >> > be >> > > > > > pretty >> > > > > > > > > useful for > easy debugging. >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > > >> > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth > Chandar < >> > > > [email protected]> >> > > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > +1 will take a look at the doc for specifics > in a few >> days. >> > > > > > > > > > >> > > > > > > > > > On Sat, Sep 21, > 2019 at 7:18 PM vino yang < >> > > > [email protected] >> > > > > > > >> > > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > > +1 to > introduce Hudi web UI. Great suggestion! On >> > > 09/21/2019 >> > > > > > > 12:24, >> > > > > > > > > Minh >> > > > > > > > > > > Pham wrote: +1. I > think an admin UI will help with >> > > > reusability >> > > > > > > alot. > >> > > > > > > > On >> > > > > > > > > > > Fri, Sep 20, 2019 at 8:32 PM > Vinay Patil < >> > > > > > [email protected]> >> > > > > > > > > > > wrote: >> > > > > > > > > > > > Hi Taher, > > I really liked this idea, > these >> details >> > > will >> > > > be >> > > > >
Re: [PROPOSAL] Hudi Web UI
Thanks Taher, great job! Will have another look soon. Best, Vino On 09/24/2019 02:15, Taher Koitawala wrote: Hi All, Hip has been migrated to confluence. Please take a look. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130027233 Regards, Taher Koitawala On Mon, Sep 23, 2019 at 10:17 PM Taher Koitawala wrote: > Yup got it. Thanks Vinoth > > On Mon, Sep 23, 2019, 10:02 PM Vinoth Chandar wrote: > >> Now? >> >> On Mon, Sep 23, 2019 at 9:05 AM Taher Koitawala >> wrote: >> >> > Hey Vinoth still the same. No perms >> > >> > On Mon, Sep 23, 2019 at 9:24 PM Vinoth Chandar >> wrote: >> > >> > > thats correct. you should have perms now. >> > > >> > > On Sun, Sep 22, 2019 at 11:06 PM Taher Koitawala >> > > wrote: >> > > >> > > > Hi Vinoth, >> > > > I do not have access on confluence to create a page >> in >> > the >> > > > HIP section or even copy the HIP template. Please can you give >> access? >> > My >> > > > id is taherk77. >> > > > >> > > > Regards, >> > > > Taher Koitawala >> > > > >> > > > On Sun, Sep 22, 2019 at 6:34 PM Vinoth Chandar >> > > wrote: >> > > > >> > > > > Taher, can we please move the HIP to the cWiki space as documented >> > here >> > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/HUDI/Hudi+Improvement+Plan+Details+and+Process >> > > > > >> > > > > >> > > > > Would love to take a pass at it. This will definitely improve >> > > usability.. >> > > > > >> > > > > @leesf I think we can use the standalone timeline server bundle >> you >> > > > worked >> > > > > on, and have a central timeline server for all Hudi jobs within >> your >> > > > org.. >> > > > > Then, if you host an UI endpoint on that server, we can visualize >> > much >> > > of >> > > > > what Tahen put up on the doc. thoughts? >> > > > > >> > > > > On Sun, Sep 22, 2019 at 4:07 AM Taher Koitawala < >> [email protected]> >> > > > > wrote: >> > > > > >> > > > > > Hi Leesf, >> > > > > > Thank you for your interest. HIP already has been >> > > > implemented >> > > > > in >> > > > > > terms of design and components we need to see. The link is given >> > > below. >> > > > > > Please free to chime in, to add and implement. >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing >> > > > > > >> > > > > > Regards, >> > > > > > Taher Koitawala >> > > > > > >> > > > > > On Sun, Sep 22, 2019, 3:26 PM leesf >> wrote: >> > > > > > >> > > > > > > + 1 to this valuable HIP. >> > > > > > > And I am also very interested, perhaps together to implement >> this >> > > > HIP. >> > > > > > > >> > > > > > > Best, >> > > > > > > Leesf >> > > > > > > >> > > > > > > Taher Koitawala 于2019年9月22日周日 下午2:09写道： >> > > > > > > >> > > > > > > > Guys it not only includes tables views and admin kind of >> > > > > > > views(Compactions) >> > > > > > > > but it also includes 'Metadata lineage' which can help users >> > know >> > > > how >> > > > > > > this >> > > > > > > > Hudi dataset merged and also another strong feature is >> creating >> > > > > > > > DeltaStreamer jobs through the webui and having >> DeltaStreamer >> > > > > > > > templates(Makes sharing jobs easy). I think those are really >> > > really >> > > > > > > strong >> > > > > > > > features. >> > > > > > > > >> > > > > > > > >> > > > > > > > Regards, >> > > > > > > > Taher Koitawala >> > > > > > > > >> > > > > > > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > +1 for adding web ui. The web ui viz for table configs >> would >> > be >> > > > > > pretty >> > > > > > > > > useful for easy debugging. >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar < >> > > > [email protected]> >> > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > > +1 will take a look at the doc for specifics in a few >> days. >> > > > > > > > > > >> > > > > > > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang < >> > > > [email protected] >> > > > > > >> > > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > > +1 to introduce Hudi web UI. Great suggestion! On >> > > 09/21/2019 >> > > > > > 12:24, >> > > > > > > > > Minh >> > > > > > > > > > > Pham wrote: +1. I think an admin UI will help with >> > > > reusability >> > > > > > > alot. >> > > > > > > > On >> > > > > > > > > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil < >> > > > > > [email protected]> >> > > > > > > > > > wrote: >> > > > > > > > > > > > Hi Taher, > > I really liked this idea, these >> details >> > > will >> > > > be >> > > > > > > > > valuable >> > > > > > > > > > to >> > > > > > > > > > > checkout on Web > UI. > > +1 > > Regards, > Vinay >> Patil > >> > > > > >> > > > > On >> > > > > > > Fri, >> > > > > > > > > Sep >
Re: [PROPOSAL] Hudi Web UI
Hi All, Hip has been migrated to confluence. Please take a look. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130027233 Regards, Taher Koitawala On Mon, Sep 23, 2019 at 10:17 PM Taher Koitawala wrote: > Yup got it. Thanks Vinoth > > On Mon, Sep 23, 2019, 10:02 PM Vinoth Chandar wrote: > >> Now? >> >> On Mon, Sep 23, 2019 at 9:05 AM Taher Koitawala >> wrote: >> >> > Hey Vinoth still the same. No perms >> > >> > On Mon, Sep 23, 2019 at 9:24 PM Vinoth Chandar >> wrote: >> > >> > > thats correct. you should have perms now. >> > > >> > > On Sun, Sep 22, 2019 at 11:06 PM Taher Koitawala >> > > wrote: >> > > >> > > > Hi Vinoth, >> > > >I do not have access on confluence to create a page >> in >> > the >> > > > HIP section or even copy the HIP template. Please can you give >> access? >> > My >> > > > id is taherk77. >> > > > >> > > > Regards, >> > > > Taher Koitawala >> > > > >> > > > On Sun, Sep 22, 2019 at 6:34 PM Vinoth Chandar >> > > wrote: >> > > > >> > > > > Taher, can we please move the HIP to the cWiki space as documented >> > here >> > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/HUDI/Hudi+Improvement+Plan+Details+and+Process >> > > > > >> > > > > >> > > > > Would love to take a pass at it. This will definitely improve >> > > usability.. >> > > > > >> > > > > @leesf I think we can use the standalone timeline server bundle >> you >> > > > worked >> > > > > on, and have a central timeline server for all Hudi jobs within >> your >> > > > org.. >> > > > > Then, if you host an UI endpoint on that server, we can visualize >> > much >> > > of >> > > > > what Tahen put up on the doc. thoughts? >> > > > > >> > > > > On Sun, Sep 22, 2019 at 4:07 AM Taher Koitawala < >> [email protected]> >> > > > > wrote: >> > > > > >> > > > > > Hi Leesf, >> > > > > >Thank you for your interest. HIP already has been >> > > > implemented >> > > > > in >> > > > > > terms of design and components we need to see. The link is given >> > > below. >> > > > > > Please free to chime in, to add and implement. >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing >> > > > > > >> > > > > > Regards, >> > > > > > Taher Koitawala >> > > > > > >> > > > > > On Sun, Sep 22, 2019, 3:26 PM leesf >> wrote: >> > > > > > >> > > > > > > + 1 to this valuable HIP. >> > > > > > > And I am also very interested, perhaps together to implement >> this >> > > > HIP. >> > > > > > > >> > > > > > > Best, >> > > > > > > Leesf >> > > > > > > >> > > > > > > Taher Koitawala 于2019年9月22日周日 下午2:09写道： >> > > > > > > >> > > > > > > > Guys it not only includes tables views and admin kind of >> > > > > > > views(Compactions) >> > > > > > > > but it also includes 'Metadata lineage' which can help users >> > know >> > > > how >> > > > > > > this >> > > > > > > > Hudi dataset merged and also another strong feature is >> creating >> > > > > > > > DeltaStreamer jobs through the webui and having >> DeltaStreamer >> > > > > > > > templates(Makes sharing jobs easy). I think those are really >> > > really >> > > > > > > strong >> > > > > > > > features. >> > > > > > > > >> > > > > > > > >> > > > > > > > Regards, >> > > > > > > > Taher Koitawala >> > > > > > > > >> > > > > > > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > +1 for adding web ui. The web ui viz for table configs >> would >> > be >> > > > > > pretty >> > > > > > > > > useful for easy debugging. >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar < >> > > > [email protected]> >> > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > > +1 will take a look at the doc for specifics in a few >> days. >> > > > > > > > > > >> > > > > > > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang < >> > > > [email protected] >> > > > > > >> > > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > > +1 to introduce Hudi web UI. Great suggestion! On >> > > 09/21/2019 >> > > > > > 12:24, >> > > > > > > > > Minh >> > > > > > > > > > > Pham wrote: +1. I think an admin UI will help with >> > > > reusability >> > > > > > > alot. >> > > > > > > > On >> > > > > > > > > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil < >> > > > > > [email protected]> >> > > > > > > > > > wrote: >> > > > > > > > > > > > Hi Taher, > > I really liked this idea, these >> details >> > > will >> > > > be >> > > > > > > > > valuable >> > > > > > > > > > to >> > > > > > > > > > > checkout on Web > UI. > > +1 > > Regards, > Vinay >> Patil > >> > > > > >> > > > > On >> > > > > > > Fri, >> > > > > > > > > Sep >> > > > > > > > > > > 20, 2019 at 3:28 PM Taher Koitawala < >> [email protected]> >> > > >> > > > > > wrote: >> > > > > > > > >> > > > > > > > > >> > > > > > > > > > >> >
Re: [PROPOSAL] Hudi Web UI
Yup got it. Thanks Vinoth On Mon, Sep 23, 2019, 10:02 PM Vinoth Chandar wrote: > Now? > > On Mon, Sep 23, 2019 at 9:05 AM Taher Koitawala > wrote: > > > Hey Vinoth still the same. No perms > > > > On Mon, Sep 23, 2019 at 9:24 PM Vinoth Chandar > wrote: > > > > > thats correct. you should have perms now. > > > > > > On Sun, Sep 22, 2019 at 11:06 PM Taher Koitawala > > > wrote: > > > > > > > Hi Vinoth, > > > >I do not have access on confluence to create a page in > > the > > > > HIP section or even copy the HIP template. Please can you give > access? > > My > > > > id is taherk77. > > > > > > > > Regards, > > > > Taher Koitawala > > > > > > > > On Sun, Sep 22, 2019 at 6:34 PM Vinoth Chandar > > > wrote: > > > > > > > > > Taher, can we please move the HIP to the cWiki space as documented > > here > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/HUDI/Hudi+Improvement+Plan+Details+and+Process > > > > > > > > > > > > > > > Would love to take a pass at it. This will definitely improve > > > usability.. > > > > > > > > > > @leesf I think we can use the standalone timeline server bundle you > > > > worked > > > > > on, and have a central timeline server for all Hudi jobs within > your > > > > org.. > > > > > Then, if you host an UI endpoint on that server, we can visualize > > much > > > of > > > > > what Tahen put up on the doc. thoughts? > > > > > > > > > > On Sun, Sep 22, 2019 at 4:07 AM Taher Koitawala < > [email protected]> > > > > > wrote: > > > > > > > > > > > Hi Leesf, > > > > > >Thank you for your interest. HIP already has been > > > > implemented > > > > > in > > > > > > terms of design and components we need to see. The link is given > > > below. > > > > > > Please free to chime in, to add and implement. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > > > > > > > > > > > > Regards, > > > > > > Taher Koitawala > > > > > > > > > > > > On Sun, Sep 22, 2019, 3:26 PM leesf wrote: > > > > > > > > > > > > > + 1 to this valuable HIP. > > > > > > > And I am also very interested, perhaps together to implement > this > > > > HIP. > > > > > > > > > > > > > > Best, > > > > > > > Leesf > > > > > > > > > > > > > > Taher Koitawala 于2019年9月22日周日 下午2:09写道： > > > > > > > > > > > > > > > Guys it not only includes tables views and admin kind of > > > > > > > views(Compactions) > > > > > > > > but it also includes 'Metadata lineage' which can help users > > know > > > > how > > > > > > > this > > > > > > > > Hudi dataset merged and also another strong feature is > creating > > > > > > > > DeltaStreamer jobs through the webui and having DeltaStreamer > > > > > > > > templates(Makes sharing jobs easy). I think those are really > > > really > > > > > > > strong > > > > > > > > features. > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > Taher Koitawala > > > > > > > > > > > > > > > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran > > > > > > > > wrote: > > > > > > > > > > > > > > > > > +1 for adding web ui. The web ui viz for table configs > would > > be > > > > > > pretty > > > > > > > > > useful for easy debugging. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 will take a look at the doc for specifics in a few > days. > > > > > > > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang < > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > +1 to introduce Hudi web UI. Great suggestion! On > > > 09/21/2019 > > > > > > 12:24, > > > > > > > > > Minh > > > > > > > > > > > Pham wrote: +1. I think an admin UI will help with > > > > reusability > > > > > > > alot. > > > > > > > > On > > > > > > > > > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil < > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > Hi Taher, > > I really liked this idea, these details > > > will > > > > be > > > > > > > > > valuable > > > > > > > > > > to > > > > > > > > > > > checkout on Web > UI. > > +1 > > Regards, > Vinay > Patil > > > > > > > > > > > On > > > > > > > Fri, > > > > > > > > > Sep > > > > > > > > > > > 20, 2019 at 3:28 PM Taher Koitawala < > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi All, > > A HIP has been created by me > > > proposing a > > > > > > Hudi > > > > > > > > Web > > > > > > > > > UI > > > > > > > > > > > with a lot > of > > helpful features. The major > > motivation > > > of > > > > > > this > > > > > > > > > > proposal > > > > > > > > > > > is that at the > > moment, users have depend on Spark's > > > > > > > configuration > > > > > > > > > to > >
Re: [PROPOSAL] Hudi Web UI
Now? On Mon, Sep 23, 2019 at 9:05 AM Taher Koitawala wrote: > Hey Vinoth still the same. No perms > > On Mon, Sep 23, 2019 at 9:24 PM Vinoth Chandar wrote: > > > thats correct. you should have perms now. > > > > On Sun, Sep 22, 2019 at 11:06 PM Taher Koitawala > > wrote: > > > > > Hi Vinoth, > > >I do not have access on confluence to create a page in > the > > > HIP section or even copy the HIP template. Please can you give access? > My > > > id is taherk77. > > > > > > Regards, > > > Taher Koitawala > > > > > > On Sun, Sep 22, 2019 at 6:34 PM Vinoth Chandar > > wrote: > > > > > > > Taher, can we please move the HIP to the cWiki space as documented > here > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/HUDI/Hudi+Improvement+Plan+Details+and+Process > > > > > > > > > > > > Would love to take a pass at it. This will definitely improve > > usability.. > > > > > > > > @leesf I think we can use the standalone timeline server bundle you > > > worked > > > > on, and have a central timeline server for all Hudi jobs within your > > > org.. > > > > Then, if you host an UI endpoint on that server, we can visualize > much > > of > > > > what Tahen put up on the doc. thoughts? > > > > > > > > On Sun, Sep 22, 2019 at 4:07 AM Taher Koitawala > > > > wrote: > > > > > > > > > Hi Leesf, > > > > >Thank you for your interest. HIP already has been > > > implemented > > > > in > > > > > terms of design and components we need to see. The link is given > > below. > > > > > Please free to chime in, to add and implement. > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > > > > > > > > > > Regards, > > > > > Taher Koitawala > > > > > > > > > > On Sun, Sep 22, 2019, 3:26 PM leesf wrote: > > > > > > > > > > > + 1 to this valuable HIP. > > > > > > And I am also very interested, perhaps together to implement this > > > HIP. > > > > > > > > > > > > Best, > > > > > > Leesf > > > > > > > > > > > > Taher Koitawala 于2019年9月22日周日 下午2:09写道： > > > > > > > > > > > > > Guys it not only includes tables views and admin kind of > > > > > > views(Compactions) > > > > > > > but it also includes 'Metadata lineage' which can help users > know > > > how > > > > > > this > > > > > > > Hudi dataset merged and also another strong feature is creating > > > > > > > DeltaStreamer jobs through the webui and having DeltaStreamer > > > > > > > templates(Makes sharing jobs easy). I think those are really > > really > > > > > > strong > > > > > > > features. > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > Taher Koitawala > > > > > > > > > > > > > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran > > > > > > > wrote: > > > > > > > > > > > > > > > +1 for adding web ui. The web ui viz for table configs would > be > > > > > pretty > > > > > > > > useful for easy debugging. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > +1 will take a look at the doc for specifics in a few days. > > > > > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 to introduce Hudi web UI. Great suggestion! On > > 09/21/2019 > > > > > 12:24, > > > > > > > > Minh > > > > > > > > > > Pham wrote: +1. I think an admin UI will help with > > > reusability > > > > > > alot. > > > > > > > On > > > > > > > > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil < > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > Hi Taher, > > I really liked this idea, these details > > will > > > be > > > > > > > > valuable > > > > > > > > > to > > > > > > > > > > checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > > > > > > > On > > > > > > Fri, > > > > > > > > Sep > > > > > > > > > > 20, 2019 at 3:28 PM Taher Koitawala > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi All, > > A HIP has been created by me > > proposing a > > > > > Hudi > > > > > > > Web > > > > > > > > UI > > > > > > > > > > with a lot > of > > helpful features. The major > motivation > > of > > > > > this > > > > > > > > > proposal > > > > > > > > > > is that at the > > moment, users have depend on Spark's > > > > > > configuration > > > > > > > > to > > > > > > > > > a > > > > > > > > > > metrics > collection > > system to see Hudi metrics. > > > > > > > > > > > > > > > > Also, > > > > > > > > > > a lot of metadata about Hudi that the user doesn't find > > > > > > > > easily > > > > > > > > after > > > > > > > > > > creating tables will be put up on the UI. Please check > the > > > > > > > > > > > > > > document > > > > > > > > > for > > > > > > > > > > a detailed explanation. > > > > > > > > > > > > > > > >
Re: [PROPOSAL] Hudi Web UI
Hey Vinoth still the same. No perms On Mon, Sep 23, 2019 at 9:24 PM Vinoth Chandar wrote: > thats correct. you should have perms now. > > On Sun, Sep 22, 2019 at 11:06 PM Taher Koitawala > wrote: > > > Hi Vinoth, > >I do not have access on confluence to create a page in the > > HIP section or even copy the HIP template. Please can you give access? My > > id is taherk77. > > > > Regards, > > Taher Koitawala > > > > On Sun, Sep 22, 2019 at 6:34 PM Vinoth Chandar > wrote: > > > > > Taher, can we please move the HIP to the cWiki space as documented here > > > > > > > > > https://cwiki.apache.org/confluence/display/HUDI/Hudi+Improvement+Plan+Details+and+Process > > > > > > > > > Would love to take a pass at it. This will definitely improve > usability.. > > > > > > @leesf I think we can use the standalone timeline server bundle you > > worked > > > on, and have a central timeline server for all Hudi jobs within your > > org.. > > > Then, if you host an UI endpoint on that server, we can visualize much > of > > > what Tahen put up on the doc. thoughts? > > > > > > On Sun, Sep 22, 2019 at 4:07 AM Taher Koitawala > > > wrote: > > > > > > > Hi Leesf, > > > >Thank you for your interest. HIP already has been > > implemented > > > in > > > > terms of design and components we need to see. The link is given > below. > > > > Please free to chime in, to add and implement. > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > > > > > > > > Regards, > > > > Taher Koitawala > > > > > > > > On Sun, Sep 22, 2019, 3:26 PM leesf wrote: > > > > > > > > > + 1 to this valuable HIP. > > > > > And I am also very interested, perhaps together to implement this > > HIP. > > > > > > > > > > Best, > > > > > Leesf > > > > > > > > > > Taher Koitawala 于2019年9月22日周日 下午2:09写道： > > > > > > > > > > > Guys it not only includes tables views and admin kind of > > > > > views(Compactions) > > > > > > but it also includes 'Metadata lineage' which can help users know > > how > > > > > this > > > > > > Hudi dataset merged and also another strong feature is creating > > > > > > DeltaStreamer jobs through the webui and having DeltaStreamer > > > > > > templates(Makes sharing jobs easy). I think those are really > really > > > > > strong > > > > > > features. > > > > > > > > > > > > > > > > > > Regards, > > > > > > Taher Koitawala > > > > > > > > > > > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran > > > > > > wrote: > > > > > > > > > > > > > +1 for adding web ui. The web ui viz for table configs would be > > > > pretty > > > > > > > useful for easy debugging. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > +1 will take a look at the doc for specifics in a few days. > > > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > +1 to introduce Hudi web UI. Great suggestion! On > 09/21/2019 > > > > 12:24, > > > > > > > Minh > > > > > > > > > Pham wrote: +1. I think an admin UI will help with > > reusability > > > > > alot. > > > > > > On > > > > > > > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > Hi Taher, > > I really liked this idea, these details > will > > be > > > > > > > valuable > > > > > > > > to > > > > > > > > > checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > > > > > On > > > > > Fri, > > > > > > > Sep > > > > > > > > > 20, 2019 at 3:28 PM Taher Koitawala > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi All, > > A HIP has been created by me > proposing a > > > > Hudi > > > > > > Web > > > > > > > UI > > > > > > > > > with a lot > of > > helpful features. The major motivation > of > > > > this > > > > > > > > proposal > > > > > > > > > is that at the > > moment, users have depend on Spark's > > > > > configuration > > > > > > > to > > > > > > > > a > > > > > > > > > metrics > collection > > system to see Hudi metrics. > > > > > > > > > > > > > Also, > > > > > > > > > a lot of metadata about Hudi that the user doesn't find > > > > > > easily > > > > > > > after > > > > > > > > > creating tables will be put up on the UI. Please check the > > > > > > > > > > > document > > > > > > > > for > > > > > > > > > a detailed explanation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4_edit-3Fusp-3Dsharing&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=aL8qpmWcgJ5Kv3y-xHJIfAXDqt-1yIXEwTidVJTcqHI&s=cF3VNnlSaNRsN4t2bSmA8XcNfZefvxZhd8KCbtxpqyE&e= > > > >
Re: [PROPOSAL] Hudi Web UI
thats correct. you should have perms now. On Sun, Sep 22, 2019 at 11:06 PM Taher Koitawala wrote: > Hi Vinoth, >I do not have access on confluence to create a page in the > HIP section or even copy the HIP template. Please can you give access? My > id is taherk77. > > Regards, > Taher Koitawala > > On Sun, Sep 22, 2019 at 6:34 PM Vinoth Chandar wrote: > > > Taher, can we please move the HIP to the cWiki space as documented here > > > > > https://cwiki.apache.org/confluence/display/HUDI/Hudi+Improvement+Plan+Details+and+Process > > > > > > Would love to take a pass at it. This will definitely improve usability.. > > > > @leesf I think we can use the standalone timeline server bundle you > worked > > on, and have a central timeline server for all Hudi jobs within your > org.. > > Then, if you host an UI endpoint on that server, we can visualize much of > > what Tahen put up on the doc. thoughts? > > > > On Sun, Sep 22, 2019 at 4:07 AM Taher Koitawala > > wrote: > > > > > Hi Leesf, > > >Thank you for your interest. HIP already has been > implemented > > in > > > terms of design and components we need to see. The link is given below. > > > Please free to chime in, to add and implement. > > > > > > > > > > > > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > > > > > > Regards, > > > Taher Koitawala > > > > > > On Sun, Sep 22, 2019, 3:26 PM leesf wrote: > > > > > > > + 1 to this valuable HIP. > > > > And I am also very interested, perhaps together to implement this > HIP. > > > > > > > > Best, > > > > Leesf > > > > > > > > Taher Koitawala 于2019年9月22日周日 下午2:09写道： > > > > > > > > > Guys it not only includes tables views and admin kind of > > > > views(Compactions) > > > > > but it also includes 'Metadata lineage' which can help users know > how > > > > this > > > > > Hudi dataset merged and also another strong feature is creating > > > > > DeltaStreamer jobs through the webui and having DeltaStreamer > > > > > templates(Makes sharing jobs easy). I think those are really really > > > > strong > > > > > features. > > > > > > > > > > > > > > > Regards, > > > > > Taher Koitawala > > > > > > > > > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran > > > > > wrote: > > > > > > > > > > > +1 for adding web ui. The web ui viz for table configs would be > > > pretty > > > > > > useful for easy debugging. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > +1 will take a look at the doc for specifics in a few days. > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > +1 to introduce Hudi web UI. Great suggestion! On 09/21/2019 > > > 12:24, > > > > > > Minh > > > > > > > > Pham wrote: +1. I think an admin UI will help with > reusability > > > > alot. > > > > > On > > > > > > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > Hi Taher, > > I really liked this idea, these details will > be > > > > > > valuable > > > > > > > to > > > > > > > > checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > > > On > > > > Fri, > > > > > > Sep > > > > > > > > 20, 2019 at 3:28 PM Taher Koitawala > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi All, > > A HIP has been created by me proposing a > > > Hudi > > > > > Web > > > > > > UI > > > > > > > > with a lot > of > > helpful features. The major motivation of > > > this > > > > > > > proposal > > > > > > > > is that at the > > moment, users have depend on Spark's > > > > configuration > > > > > > to > > > > > > > a > > > > > > > > metrics > collection > > system to see Hudi metrics. > > > > > > > > > > > Also, > > > > > > > > a lot of metadata about Hudi that the user doesn't find > > > > > easily > > > > > > after > > > > > > > > creating tables will be put up on the UI. Please check the > > > > > > > > document > > > > > > > for > > > > > > > > a detailed explanation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4_edit-3Fusp-3Dsharing&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=aL8qpmWcgJ5Kv3y-xHJIfAXDqt-1yIXEwTidVJTcqHI&s=cF3VNnlSaNRsN4t2bSmA8XcNfZefvxZhd8KCbtxpqyE&e= > > > > > > > > > > > > Regards, > > Taher Koitawala > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [PROPOSAL] Hudi Web UI
Hi Vinoth, I do not have access on confluence to create a page in the HIP section or even copy the HIP template. Please can you give access? My id is taherk77. Regards, Taher Koitawala On Sun, Sep 22, 2019 at 6:34 PM Vinoth Chandar wrote: > Taher, can we please move the HIP to the cWiki space as documented here > > https://cwiki.apache.org/confluence/display/HUDI/Hudi+Improvement+Plan+Details+and+Process > > > Would love to take a pass at it. This will definitely improve usability.. > > @leesf I think we can use the standalone timeline server bundle you worked > on, and have a central timeline server for all Hudi jobs within your org.. > Then, if you host an UI endpoint on that server, we can visualize much of > what Tahen put up on the doc. thoughts? > > On Sun, Sep 22, 2019 at 4:07 AM Taher Koitawala > wrote: > > > Hi Leesf, > >Thank you for your interest. HIP already has been implemented > in > > terms of design and components we need to see. The link is given below. > > Please free to chime in, to add and implement. > > > > > > > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > > > > Regards, > > Taher Koitawala > > > > On Sun, Sep 22, 2019, 3:26 PM leesf wrote: > > > > > + 1 to this valuable HIP. > > > And I am also very interested, perhaps together to implement this HIP. > > > > > > Best, > > > Leesf > > > > > > Taher Koitawala 于2019年9月22日周日 下午2:09写道： > > > > > > > Guys it not only includes tables views and admin kind of > > > views(Compactions) > > > > but it also includes 'Metadata lineage' which can help users know how > > > this > > > > Hudi dataset merged and also another strong feature is creating > > > > DeltaStreamer jobs through the webui and having DeltaStreamer > > > > templates(Makes sharing jobs easy). I think those are really really > > > strong > > > > features. > > > > > > > > > > > > Regards, > > > > Taher Koitawala > > > > > > > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran > > > > wrote: > > > > > > > > > +1 for adding web ui. The web ui viz for table configs would be > > pretty > > > > > useful for easy debugging. > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar > > > > wrote: > > > > > > > > > > > +1 will take a look at the doc for specifics in a few days. > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang > > > > > wrote: > > > > > > > > > > > > > +1 to introduce Hudi web UI. Great suggestion! On 09/21/2019 > > 12:24, > > > > > Minh > > > > > > > Pham wrote: +1. I think an admin UI will help with reusability > > > alot. > > > > On > > > > > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil < > > [email protected]> > > > > > > wrote: > > > > > > > > Hi Taher, > > I really liked this idea, these details will be > > > > > valuable > > > > > > to > > > > > > > checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > > On > > > Fri, > > > > > Sep > > > > > > > 20, 2019 at 3:28 PM Taher Koitawala > > > wrote: > > > > > > > > > > > > > > > > > > > > > > Hi All, > > A HIP has been created by me proposing a > > Hudi > > > > Web > > > > > UI > > > > > > > with a lot > of > > helpful features. The major motivation of > > this > > > > > > proposal > > > > > > > is that at the > > moment, users have depend on Spark's > > > configuration > > > > > to > > > > > > a > > > > > > > metrics > collection > > system to see Hudi metrics. > > > > > > > > > > Also, > > > > > > > a lot of metadata about Hudi that the user doesn't find > > > > easily > > > > > after > > > > > > > creating tables will be put up on the UI. Please check the > > > > > > document > > > > > > for > > > > > > > a detailed explanation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4_edit-3Fusp-3Dsharing&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=aL8qpmWcgJ5Kv3y-xHJIfAXDqt-1yIXEwTidVJTcqHI&s=cF3VNnlSaNRsN4t2bSmA8XcNfZefvxZhd8KCbtxpqyE&e= > > > > > > > > > > > Regards, > > Taher Koitawala > > > > > > > > > > > > > > > > > > > > > > > >
Re: [PROPOSAL] Hudi Web UI
Taher, can we please move the HIP to the cWiki space as documented here https://cwiki.apache.org/confluence/display/HUDI/Hudi+Improvement+Plan+Details+and+Process Would love to take a pass at it. This will definitely improve usability.. @leesf I think we can use the standalone timeline server bundle you worked on, and have a central timeline server for all Hudi jobs within your org.. Then, if you host an UI endpoint on that server, we can visualize much of what Tahen put up on the doc. thoughts? On Sun, Sep 22, 2019 at 4:07 AM Taher Koitawala wrote: > Hi Leesf, >Thank you for your interest. HIP already has been implemented in > terms of design and components we need to see. The link is given below. > Please free to chime in, to add and implement. > > > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > > Regards, > Taher Koitawala > > On Sun, Sep 22, 2019, 3:26 PM leesf wrote: > > > + 1 to this valuable HIP. > > And I am also very interested, perhaps together to implement this HIP. > > > > Best, > > Leesf > > > > Taher Koitawala 于2019年9月22日周日 下午2:09写道： > > > > > Guys it not only includes tables views and admin kind of > > views(Compactions) > > > but it also includes 'Metadata lineage' which can help users know how > > this > > > Hudi dataset merged and also another strong feature is creating > > > DeltaStreamer jobs through the webui and having DeltaStreamer > > > templates(Makes sharing jobs easy). I think those are really really > > strong > > > features. > > > > > > > > > Regards, > > > Taher Koitawala > > > > > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran > > > wrote: > > > > > > > +1 for adding web ui. The web ui viz for table configs would be > pretty > > > > useful for easy debugging. > > > > > > > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar > > > wrote: > > > > > > > > > +1 will take a look at the doc for specifics in a few days. > > > > > > > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang > > > wrote: > > > > > > > > > > > +1 to introduce Hudi web UI. Great suggestion! On 09/21/2019 > 12:24, > > > > Minh > > > > > > Pham wrote: +1. I think an admin UI will help with reusability > > alot. > > > On > > > > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil < > [email protected]> > > > > > wrote: > > > > > > > Hi Taher, > > I really liked this idea, these details will be > > > > valuable > > > > > to > > > > > > checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > On > > Fri, > > > > Sep > > > > > > 20, 2019 at 3:28 PM Taher Koitawala > > wrote: > > > > > > > > > > > > > > > > > > Hi All, > > A HIP has been created by me proposing a > Hudi > > > Web > > > > UI > > > > > > with a lot > of > > helpful features. The major motivation of > this > > > > > proposal > > > > > > is that at the > > moment, users have depend on Spark's > > configuration > > > > to > > > > > a > > > > > > metrics > collection > > system to see Hudi metrics. > > > > > > > > > Also, > > > > > > a lot of metadata about Hudi that the user doesn't find > > > easily > > > > after > > > > > > creating tables will be put up on the UI. Please check the > > > > > document > > > > > for > > > > > > a detailed explanation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4_edit-3Fusp-3Dsharing&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=aL8qpmWcgJ5Kv3y-xHJIfAXDqt-1yIXEwTidVJTcqHI&s=cF3VNnlSaNRsN4t2bSmA8XcNfZefvxZhd8KCbtxpqyE&e= > > > > > > > > > > Regards, > > Taher Koitawala > > > > > > > > > > > > > > > > > >
Re: [PROPOSAL] Hudi Web UI
Hi Leesf, Thank you for your interest. HIP already has been implemented in terms of design and components we need to see. The link is given below. Please free to chime in, to add and implement. https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing Regards, Taher Koitawala On Sun, Sep 22, 2019, 3:26 PM leesf wrote: > + 1 to this valuable HIP. > And I am also very interested, perhaps together to implement this HIP. > > Best, > Leesf > > Taher Koitawala 于2019年9月22日周日 下午2:09写道： > > > Guys it not only includes tables views and admin kind of > views(Compactions) > > but it also includes 'Metadata lineage' which can help users know how > this > > Hudi dataset merged and also another strong feature is creating > > DeltaStreamer jobs through the webui and having DeltaStreamer > > templates(Makes sharing jobs easy). I think those are really really > strong > > features. > > > > > > Regards, > > Taher Koitawala > > > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran > > wrote: > > > > > +1 for adding web ui. The web ui viz for table configs would be pretty > > > useful for easy debugging. > > > > > > > > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar > > wrote: > > > > > > > +1 will take a look at the doc for specifics in a few days. > > > > > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang > > wrote: > > > > > > > > > +1 to introduce Hudi web UI. Great suggestion! On 09/21/2019 12:24, > > > Minh > > > > > Pham wrote: +1. I think an admin UI will help with reusability > alot. > > On > > > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil > > > > wrote: > > > > > > Hi Taher, > > I really liked this idea, these details will be > > > valuable > > > > to > > > > > checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > On > Fri, > > > Sep > > > > > 20, 2019 at 3:28 PM Taher Koitawala > wrote: > > > > > > > > > > > > > > Hi All, > > A HIP has been created by me proposing a Hudi > > Web > > > UI > > > > > with a lot > of > > helpful features. The major motivation of this > > > > proposal > > > > > is that at the > > moment, users have depend on Spark's > configuration > > > to > > > > a > > > > > metrics > collection > > system to see Hudi metrics. > > > > > > > > Also, > > > > > a lot of metadata about Hudi that the user doesn't find > > easily > > > after > > > > > creating tables will be put up on the UI. Please check the > > > > document > > > > for > > > > > a detailed explanation. > > > > > > > > > > > > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4_edit-3Fusp-3Dsharing&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=aL8qpmWcgJ5Kv3y-xHJIfAXDqt-1yIXEwTidVJTcqHI&s=cF3VNnlSaNRsN4t2bSmA8XcNfZefvxZhd8KCbtxpqyE&e= > > > > > > > > > Regards, > > Taher Koitawala > > > > > > > > > > > > >
Re: [PROPOSAL] Hudi Web UI
+ 1 to this valuable HIP. And I am also very interested, perhaps together to implement this HIP. Best, Leesf Taher Koitawala 于2019年9月22日周日 下午2:09写道： > Guys it not only includes tables views and admin kind of views(Compactions) > but it also includes 'Metadata lineage' which can help users know how this > Hudi dataset merged and also another strong feature is creating > DeltaStreamer jobs through the webui and having DeltaStreamer > templates(Makes sharing jobs easy). I think those are really really strong > features. > > > Regards, > Taher Koitawala > > On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran > wrote: > > > +1 for adding web ui. The web ui viz for table configs would be pretty > > useful for easy debugging. > > > > > > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar > wrote: > > > > > +1 will take a look at the doc for specifics in a few days. > > > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang > wrote: > > > > > > > +1 to introduce Hudi web UI. Great suggestion! On 09/21/2019 12:24, > > Minh > > > > Pham wrote: +1. I think an admin UI will help with reusability alot. > On > > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil > > > wrote: > > > > > Hi Taher, > > I really liked this idea, these details will be > > valuable > > > to > > > > checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > On Fri, > > Sep > > > > 20, 2019 at 3:28 PM Taher Koitawala > wrote: > > > > > > > > > > Hi All, > > A HIP has been created by me proposing a Hudi > Web > > UI > > > > with a lot > of > > helpful features. The major motivation of this > > > proposal > > > > is that at the > > moment, users have depend on Spark's configuration > > to > > > a > > > > metrics > collection > > system to see Hudi metrics. > > > > > > > Also, > > > > a lot of metadata about Hudi that the user doesn't find > > easily > > after > > > > creating tables will be put up on the UI. Please check the > > > document > > > for > > > > a detailed explanation. > > > > > > > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4_edit-3Fusp-3Dsharing&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=aL8qpmWcgJ5Kv3y-xHJIfAXDqt-1yIXEwTidVJTcqHI&s=cF3VNnlSaNRsN4t2bSmA8XcNfZefvxZhd8KCbtxpqyE&e= > > > > > > > > Regards, > > Taher Koitawala > > > > > > > > >
Re: [PROPOSAL] Hudi Web UI
Guys it not only includes tables views and admin kind of views(Compactions) but it also includes 'Metadata lineage' which can help users know how this Hudi dataset merged and also another strong feature is creating DeltaStreamer jobs through the webui and having DeltaStreamer templates(Makes sharing jobs easy). I think those are really really strong features. Regards, Taher Koitawala On Sun, Sep 22, 2019, 9:52 AM Bhavani Sudha Saktheeswaran wrote: > +1 for adding web ui. The web ui viz for table configs would be pretty > useful for easy debugging. > > > > > On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar wrote: > > > +1 will take a look at the doc for specifics in a few days. > > > > On Sat, Sep 21, 2019 at 7:18 PM vino yang wrote: > > > > > +1 to introduce Hudi web UI. Great suggestion! On 09/21/2019 12:24, > Minh > > > Pham wrote: +1. I think an admin UI will help with reusability alot. On > > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil > > wrote: > > > > Hi Taher, > > I really liked this idea, these details will be > valuable > > to > > > checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > On Fri, > Sep > > > 20, 2019 at 3:28 PM Taher Koitawala > wrote: > > > > > > > Hi All, > > A HIP has been created by me proposing a Hudi Web > UI > > > with a lot > of > > helpful features. The major motivation of this > > proposal > > > is that at the > > moment, users have depend on Spark's configuration > to > > a > > > metrics > collection > > system to see Hudi metrics. > > > > > > Also, > > > a lot of metadata about Hudi that the user doesn't find > > easily > after > > > creating tables will be put up on the UI. Please check the > > document > > for > > > a detailed explanation. > > > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4_edit-3Fusp-3Dsharing&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=aL8qpmWcgJ5Kv3y-xHJIfAXDqt-1yIXEwTidVJTcqHI&s=cF3VNnlSaNRsN4t2bSmA8XcNfZefvxZhd8KCbtxpqyE&e= > > > > > > > Regards, > > Taher Koitawala > > > > > >
Re: [PROPOSAL] Hudi Web UI
+1 for adding web ui. The web ui viz for table configs would be pretty useful for easy debugging. On Sat, Sep 21, 2019 at 7:35 PM Vinoth Chandar wrote: > +1 will take a look at the doc for specifics in a few days. > > On Sat, Sep 21, 2019 at 7:18 PM vino yang wrote: > > > +1 to introduce Hudi web UI. Great suggestion! On 09/21/2019 12:24, Minh > > Pham wrote: +1. I think an admin UI will help with reusability alot. On > > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil > wrote: > > > Hi Taher, > > I really liked this idea, these details will be valuable > to > > checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > On Fri, Sep > > 20, 2019 at 3:28 PM Taher Koitawala > wrote: > > > > > Hi All, > > A HIP has been created by me proposing a Hudi Web UI > > with a lot > of > > helpful features. The major motivation of this > proposal > > is that at the > > moment, users have depend on Spark's configuration to > a > > metrics > collection > > system to see Hudi metrics. > > > > > Also, > > a lot of metadata about Hudi that the user doesn't find > > easily after > > creating tables will be put up on the UI. Please check the > > document > for > > a detailed explanation. > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4_edit-3Fusp-3Dsharing&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=aL8qpmWcgJ5Kv3y-xHJIfAXDqt-1yIXEwTidVJTcqHI&s=cF3VNnlSaNRsN4t2bSmA8XcNfZefvxZhd8KCbtxpqyE&e= > > > > > > Regards, > > Taher Koitawala > > > >
Re: [PROPOSAL] Hudi Web UI
+1 will take a look at the doc for specifics in a few days. On Sat, Sep 21, 2019 at 7:18 PM vino yang wrote: > +1 to introduce Hudi web UI. Great suggestion! On 09/21/2019 12:24, Minh > Pham wrote: +1. I think an admin UI will help with reusability alot. On > Fri, Sep 20, 2019 at 8:32 PM Vinay Patil wrote: > > Hi Taher, > > I really liked this idea, these details will be valuable to > checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > On Fri, Sep > 20, 2019 at 3:28 PM Taher Koitawala > wrote: > > > > Hi All, > > A HIP has been created by me proposing a Hudi Web UI > with a lot > of > > helpful features. The major motivation of this proposal > is that at the > > moment, users have depend on Spark's configuration to a > metrics > collection > > system to see Hudi metrics. > > > > Also, > a lot of metadata about Hudi that the user doesn't find > > easily after > creating tables will be put up on the UI. Please check the > > document for > a detailed explanation. > > > > > > > > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > > > > > Regards, > > Taher Koitawala > > >
Re: [PROPOSAL] Hudi Web UI
+1 to introduce Hudi web UI. Great suggestion! On 09/21/2019 12:24, Minh Pham wrote: +1. I think an admin UI will help with reusability alot. On Fri, Sep 20, 2019 at 8:32 PM Vinay Patil wrote: > Hi Taher, > > I really liked this idea, these details will be valuable to checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > On Fri, Sep 20, 2019 at 3:28 PM Taher Koitawala > wrote: > > > Hi All, > > A HIP has been created by me proposing a Hudi Web UI with a lot > of > > helpful features. The major motivation of this proposal is that at the > > moment, users have depend on Spark's configuration to a metrics > collection > > system to see Hudi metrics. > > > > Also, a lot of metadata about Hudi that the user doesn't find > > easily after creating tables will be put up on the UI. Please check the > > document for a detailed explanation. > > > > > > > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > > > > Regards, > > Taher Koitawala > > >
Re: [PROPOSAL] Hudi Web UI
+1. I think an admin UI will help with reusability alot. On Fri, Sep 20, 2019 at 8:32 PM Vinay Patil wrote: > Hi Taher, > > I really liked this idea, these details will be valuable to checkout on Web > UI. > > +1 > > Regards, > Vinay Patil > > > On Fri, Sep 20, 2019 at 3:28 PM Taher Koitawala > wrote: > > > Hi All, > > A HIP has been created by me proposing a Hudi Web UI with a lot > of > > helpful features. The major motivation of this proposal is that at the > > moment, users have depend on Spark's configuration to a metrics > collection > > system to see Hudi metrics. > > > > Also, a lot of metadata about Hudi that the user doesn't find > > easily after creating tables will be put up on the UI. Please check the > > document for a detailed explanation. > > > > > > > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > > > > Regards, > > Taher Koitawala > > >
Re: [PROPOSAL] Hudi Web UI
Hi Taher, I really liked this idea, these details will be valuable to checkout on Web UI. +1 Regards, Vinay Patil On Fri, Sep 20, 2019 at 3:28 PM Taher Koitawala wrote: > Hi All, > A HIP has been created by me proposing a Hudi Web UI with a lot of > helpful features. The major motivation of this proposal is that at the > moment, users have depend on Spark's configuration to a metrics collection > system to see Hudi metrics. > > Also, a lot of metadata about Hudi that the user doesn't find > easily after creating tables will be put up on the UI. Please check the > document for a detailed explanation. > > > https://docs.google.com/document/d/1oEjukuaK2ltqiD0sjVs5IUzvDzilWF0viwMASXTdEd4/edit?usp=sharing > > Regards, > Taher Koitawala >
