Re: [VOTE] Moving Submarine to a separate Apache project proposal
+1 Thanx for the proposal. I would even like to participate in the project. Please add me as well. -Ayush > On 03-Sep-2019, at 6:00 PM, Vinayakumar B wrote: > > +1 > > Thanks for the proposal. > Its very interesting project and looks very promising one by looking at the > participations from various companies and the speed of development. > > I would also like to participate in the project. > Please add me as well. > > Thanks, > -Vinay > > On Tue, 3 Sep 2019, 12:38 pm Rakesh Radhakrishnan, > wrote: > >> +1, Thanks for the proposal. >> >> I am interested to participate in this project. Please include me as well >> in the project. >> >> Thanks, >> Rakesh >> >> On Tue, Sep 3, 2019 at 11:59 AM zhankun tang >> wrote: >> >>> +1 >>> >>> Thanks for Wangda's proposal. >>> >>> The submarine project is born within Hadoop, but not limited to Hadoop. >> It >>> began with a trainer on YARN but it quickly realized that only a trainer >> is >>> not enough to meet the AI platform requirements. But now there's no >>> user-friendly open-source solution covers the whole AI pipeline like data >>> engineering, training, and serving. And the underlying data >> infrastructure >>> itself is also evolving, for instance, many people love k8s. Not >> mentioning >>> there're many AI domain problems in this area to be solved. >>> It's almost for sure that building such an ML platform would utilize >>> various other open-source components taking ML into consideration >>> initially. >>> >>> I see submarine grows rapidly towards an enterprise-grade ML platform >> which >>> could potentially enable AI ability for data engineer and scientist. This >>> is an exciting thing for both the community and the industry. >>> >>> BR, >>> Zhankun >>> >>> On Tue, 3 Sep 2019 at 13:34, Xun Liu wrote: +1 Hello everyone, I am a member of the submarine development team. I have been contributing to submarine for more than a year. I have seen the progress of submarine development very fast. In more than a year, there are 9 long-term developers of different companies. Contributing, submarine cumulative code has more than 200,000 lines of code, is >> growing very fast, and is used in the production environment of multiple companies. In the submarine development group, there are 5 PMCs and 7committer >>> members from Hadoop, spark, zeppelin projects. They are very familiar with the development process and specifications >> of the apache community, and can well grasp the project development progress and project >> quality. So I recommend submarine to be a TLP project directly. We will continue to contribute to the submarine project. :-) Xun Liu Regards > On Tue, 3 Sep 2019 at 12:01, Devaraj K wrote: > > +1 > > Thanks Wangda for the proposal. > I would like to participate in this project, Please add me also to >> the > project. > > Regards > Devaraj K > > On Mon, Sep 2, 2019 at 8:50 PM zac yuan >> wrote: > >> +1 >> >> Submarine will be a complete solution for AI service development. >> It can >> take advantage of two best cluster systems: yarn and k8s, which >> will help >> more and more people get AI ability. To be a separate Apache >> project, > will >> accelerate the procedure of development apparently. >> >> Look forward to a big success in submarine project~ >> >> 朱林浩 于2019年9月3日周二 上午10:38写道： >> >>> +1, >>> Hopefully, that will become the top project, >>> >>> I also hope to make more contributions to this project. >>> >>> >>> >>> >>> >>> >>> >>> >>> At 2019-09-03 09:26:53, "Naganarasimha Garla" < >> naganarasimha...@apache.org> >>> wrote: + 1, I would also like start participate in this project, hope to >>> get >>> myself added to the project. Thanks and Regards, + Naga On Tue, Sep 3, 2019 at 8:35 AM Wangda Tan > wrote: > Hi Sree, > > I put it to the proposal, please let me know what you think: > > The traditional path at Apache would have been to create an > incubator >> project, but the code is already being released by Apache >> and most >> of >>> the >> developers are familiar with Apache rules and guidelines. In >>> particular, >> the proposed PMC has 2 Apache TLP PMCs and proposed initial >> committers >> have 4 Apache TLP PMCs (from 3 different companies). They >> will >> provide >> oversight and guidance for the developers that are less > experienced >> in > the >> Apache Way. Therefore, the Submarine project would like to propose > becoming >> a Top Level Project
[jira] [Resolved] (SUBMARINE-166) [WORKBENCH] Adjust submarine-workbench module structure
[ https://issues.apache.org/jira/browse/SUBMARINE-166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xun Liu resolved SUBMARINE-166. --- Resolution: Fixed h3. What is this PR for? The original workbench consists of two modules, submarine-server and submarine-web. Now workbench needs to add notebook and interpreter(e.g., spark, python ...) modules, all need to re-adjust the engineering structure. h3. What type of PR is it? Improvement h3. What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-166 h3. How should this be tested? [CI Pass|https://travis-ci.org/liuxunorg/submarine/builds/580120787] h3. Screenshots (if appropriate) h3. Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No > [WORKBENCH] Adjust submarine-workbench module structure > --- > > Key: SUBMARINE-166 > URL: https://issues.apache.org/jira/browse/SUBMARINE-166 > Project: Hadoop Submarine > Issue Type: Sub-task >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > Labels: submarine-external-repo > > The original workbench consists of two modules, submarine-server and > submarine-web. > Now workbench needs to add notebook and interpreter(e.g., spark, python ...) > modules, all need to re-adjust the engineering structure. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (SUBMARINE-168) [WEB] Submarine workspace project manager webpage
Xun Liu created SUBMARINE-168: - Summary: [WEB] Submarine workspace project manager webpage Key: SUBMARINE-168 URL: https://issues.apache.org/jira/browse/SUBMARINE-168 Project: Hadoop Submarine Issue Type: Sub-task Reporter: Xun Liu Assignee: Xun Liu Adjust the existing workspace/project page to add the following features: 1. Add new project 2. delete project 3. modify project set -- This message was sent by Atlassian Jira (v8.3.2#803003)
Re: [VOTE] Moving Submarine to a separate Apache project proposal
+1, Thanks for the proposal. I am interested to participate in this project. Please include me as well in the project. Thanks, Rakesh On Tue, Sep 3, 2019 at 11:59 AM zhankun tang wrote: > +1 > > Thanks for Wangda's proposal. > > The submarine project is born within Hadoop, but not limited to Hadoop. It > began with a trainer on YARN but it quickly realized that only a trainer is > not enough to meet the AI platform requirements. But now there's no > user-friendly open-source solution covers the whole AI pipeline like data > engineering, training, and serving. And the underlying data infrastructure > itself is also evolving, for instance, many people love k8s. Not mentioning > there're many AI domain problems in this area to be solved. > It's almost for sure that building such an ML platform would utilize > various other open-source components taking ML into consideration > initially. > > I see submarine grows rapidly towards an enterprise-grade ML platform which > could potentially enable AI ability for data engineer and scientist. This > is an exciting thing for both the community and the industry. > > BR, > Zhankun > > > On Tue, 3 Sep 2019 at 13:34, Xun Liu wrote: > > > +1 > > > > Hello everyone, I am a member of the submarine development team. > > I have been contributing to submarine for more than a year. > > I have seen the progress of submarine development very fast. > > In more than a year, there are 9 long-term developers of different > > companies. Contributing, > > submarine cumulative code has more than 200,000 lines of code, is growing > > very fast, > > and is used in the production environment of multiple companies. > > > > In the submarine development group, there are 5 PMCs and 7committer > members > > from Hadoop, spark, zeppelin projects. > > They are very familiar with the development process and specifications of > > the apache community, > > and can well grasp the project development progress and project quality. > > So I recommend submarine to be a TLP project directly. > > > > We will continue to contribute to the submarine project. :-) > > > > Xun Liu > > Regards > > > > On Tue, 3 Sep 2019 at 12:01, Devaraj K wrote: > > > > > +1 > > > > > > Thanks Wangda for the proposal. > > > I would like to participate in this project, Please add me also to the > > > project. > > > > > > Regards > > > Devaraj K > > > > > > On Mon, Sep 2, 2019 at 8:50 PM zac yuan wrote: > > > > > > > +1 > > > > > > > > Submarine will be a complete solution for AI service development. It > > can > > > > take advantage of two best cluster systems: yarn and k8s, which will > > help > > > > more and more people get AI ability. To be a separate Apache project, > > > will > > > > accelerate the procedure of development apparently. > > > > > > > > Look forward to a big success in submarine project~ > > > > > > > > 朱林浩 于2019年9月3日周二 上午10:38写道： > > > > > > > > > +1, > > > > > Hopefully, that will become the top project, > > > > > > > > > > I also hope to make more contributions to this project. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > At 2019-09-03 09:26:53, "Naganarasimha Garla" < > > > > naganarasimha...@apache.org> > > > > > wrote: > > > > > >+ 1, > > > > > >I would also like start participate in this project, hope to > get > > > > > myself > > > > > >added to the project. > > > > > > > > > > > >Thanks and Regards, > > > > > >+ Naga > > > > > > > > > > > >On Tue, Sep 3, 2019 at 8:35 AM Wangda Tan > > > wrote: > > > > > > > > > > > >> Hi Sree, > > > > > >> > > > > > >> I put it to the proposal, please let me know what you think: > > > > > >> > > > > > >> The traditional path at Apache would have been to create an > > > incubator > > > > > >> > project, but the code is already being released by Apache and > > most > > > > of > > > > > the > > > > > >> > developers are familiar with Apache rules and guidelines. In > > > > > particular, > > > > > >> > the proposed PMC has 2 Apache TLP PMCs and proposed initial > > > > committers > > > > > >> > have 4 Apache TLP PMCs (from 3 different companies). They will > > > > provide > > > > > >> > oversight and guidance for the developers that are less > > > experienced > > > > in > > > > > >> the > > > > > >> > Apache Way. Therefore, the Submarine project would like to > > propose > > > > > >> becoming > > > > > >> > a Top Level Project at Apache. > > > > > >> > > > > > > >> > > > > > >> To me, putting to TLP has mostly pros, it is an easier process > > (same > > > > as > > > > > ORC > > > > > >> spin-off from Hive), much less overhead to both dev community > and > > > > Apache > > > > > >> side. > > > > > >> > > > > > >> Thanks, > > > > > >> Wangda > > > > > >> > > > > > >> On Sun, Sep 1, 2019 at 2:04 PM Sree Vaddi < > > sree_at_ch...@yahoo.com> > > > > > wrote: > > > > > >> > > > > > >> > +1 to move submarine to separate apache project. > > > > > >> > > > > > > >> > It is not clear in the proposal, if submarine majority
[jira] [Updated] (SUBMARINE-148) [Umbrella] Submarine-sdk tracking
[ https://issues.apache.org/jira/browse/SUBMARINE-148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevin su updated SUBMARINE-148: --- Summary: [Umbrella] Submarine-sdk tracking (was: [Umbrella] Submarine-sdk) > [Umbrella] Submarine-sdk tracking > - > > Key: SUBMARINE-148 > URL: https://issues.apache.org/jira/browse/SUBMARINE-148 > Project: Hadoop Submarine > Issue Type: New Feature >Reporter: kevin su >Priority: Major > Labels: submarine-external-repo > > The submarine-sdk is an API for logging parameters, code versions, metrics > > 1. Add the sumbrine-sdk/pysubmarine module. > 2. pysubmarine supports mysql/HDFS in two ways to store metric data. Priority > to do mysql, in order to performance requires python to directly write mysql > data, do not take the submarine-sever RESTful interface. > 3. Add metric, param instance to the submarine-server database and rest > 4. Add Metric tracking ui to submarine-web > 5. Add a metric graphical display to submarine-web -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (SUBMARINE-148) [Umbrella] Submarine-sdk
[ https://issues.apache.org/jira/browse/SUBMARINE-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921201#comment-16921201 ] kevin su commented on SUBMARINE-148: Thanks [~tangzhankun] for the reply I'm going to update my design doc, the document is incomplete make sense, this Jira is about submarine tracking, renamed the Jira name > [Umbrella] Submarine-sdk > > > Key: SUBMARINE-148 > URL: https://issues.apache.org/jira/browse/SUBMARINE-148 > Project: Hadoop Submarine > Issue Type: New Feature >Reporter: kevin su >Priority: Major > Labels: submarine-external-repo > > The submarine-sdk is an API for logging parameters, code versions, metrics > > 1. Add the sumbrine-sdk/pysubmarine module. > 2. pysubmarine supports mysql/HDFS in two ways to store metric data. Priority > to do mysql, in order to performance requires python to directly write mysql > data, do not take the submarine-sever RESTful interface. > 3. Add metric, param instance to the submarine-server database and rest > 4. Add Metric tracking ui to submarine-web > 5. Add a metric graphical display to submarine-web -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (SUBMARINE-167) [Umbrella] Submarine Workbench Notebook
Xun Liu created SUBMARINE-167: - Summary: [Umbrella] Submarine Workbench Notebook Key: SUBMARINE-167 URL: https://issues.apache.org/jira/browse/SUBMARINE-167 Project: Hadoop Submarine Issue Type: New Feature Reporter: Xun Liu Assignee: Xun Liu Submarine needs to add notebook front-end and background services. Let the workbench support notebook development. The following features are supported through the notebook: 1. spark & spark.sql for data processing 2. JDBC for data processing 3. Python for algorithm development -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (SUBMARINE-165) [Umbrella] Submarine Workbench
[ https://issues.apache.org/jira/browse/SUBMARINE-165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xun Liu updated SUBMARINE-165: -- Parent: (was: SUBMARINE-98) Issue Type: New Feature (was: Sub-task) > [Umbrella] Submarine Workbench > -- > > Key: SUBMARINE-165 > URL: https://issues.apache.org/jira/browse/SUBMARINE-165 > Project: Hadoop Submarine > Issue Type: New Feature >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > Labels: submarine-external-repo > > Submarine can be managed via submarine Workbench: > 1. Support multi-user and multi-role > 2. Support for allocation management of server resources > 3. Support data development, algorithm development, and model launch through > notebook > 4. Support workflow workflow orchestration > 5. Support service monitoring after the model goes online > 6. Support Notebook > 7. Support Python, Spark ... execute. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (SUBMARINE-165) [Umbrella] Submarine Workbench
[ https://issues.apache.org/jira/browse/SUBMARINE-165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xun Liu updated SUBMARINE-165: -- Parent: SUBMARINE-98 Issue Type: Sub-task (was: New Feature) > [Umbrella] Submarine Workbench > -- > > Key: SUBMARINE-165 > URL: https://issues.apache.org/jira/browse/SUBMARINE-165 > Project: Hadoop Submarine > Issue Type: Sub-task >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > Labels: submarine-external-repo > > Submarine can be managed via submarine Workbench: > 1. Support multi-user and multi-role > 2. Support for allocation management of server resources > 3. Support data development, algorithm development, and model launch through > notebook > 4. Support workflow workflow orchestration > 5. Support service monitoring after the model goes online > 6. Support Notebook > 7. Support Python, Spark ... execute. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (SUBMARINE-98) [Umbrella] Submarine web page
[ https://issues.apache.org/jira/browse/SUBMARINE-98?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xun Liu updated SUBMARINE-98: - Parent: (was: SUBMARINE-165) Issue Type: New Feature (was: Sub-task) > [Umbrella] Submarine web page > - > > Key: SUBMARINE-98 > URL: https://issues.apache.org/jira/browse/SUBMARINE-98 > Project: Hadoop Submarine > Issue Type: New Feature >Reporter: Xun Liu >Assignee: Xun Liu >Priority: Major > Labels: submarine-external-repo > > Submarine can be managed via submarine web: > 1. Support multi-user and multi-role > 2. Support for allocation management of server resources > 3. Support data development, algorithm development, and model launch through > notebook > 4. Support workflow workflow orchestration > 5. Support service monitoring after the model goes online -- This message was sent by Atlassian Jira (v8.3.2#803003)
