Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Eric Yang
AM To: Eric Yang Cc: Weiwei Yang , Xun Liu , Hadoop Common , "yarn-...@hadoop.apache.org" , Hdfs-dev , "mapreduce-...@hadoop.apache.org" Subject: Re: [DISCUSS] Making submarine to different release model like Ozone Thanks everyone for sharing thoughts! Eric, appreci

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Wangda Tan
doop Common , "yarn-...@hadoop.apache.org" > , Hdfs-dev , " > mapreduce-...@hadoop.apache.org" > *Subject: *Re: [DISCUSS] Making submarine to different release model like > Ozone > > > > Thanks everyone for sharing thoughts! > > > > Eric, appreciat

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Elek, Marton
+1. I like the idea. For me, submarine/ML-job-execution seems to be a natural extension of the existing Hadoop/Yarn capabilities. And like the proposed project structure / release lifecycle, too. I think it's better to be more modularized but keep the development in the same project. IMHO it

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Hanisha Koneru
This is a great proposal. +1. Thanks, Hanisha On 2/1/19, 11:04 AM, "Bharat Viswanadham" wrote: >Thank You Wangda for driving this discussion. >+1 for a separate release for submarine. >Having own release cadence will help iterate the project to grow at a faster >pace and also get the

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Bharat Viswanadham
Thank You Wangda for driving this discussion. +1 for a separate release for submarine. Having own release cadence will help iterate the project to grow at a faster pace and also get the new features in hand to the users, and get their feedback quickly. Thanks, Bharat On 2/1/19, 10:54 AM,

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Xiaoyu Yao
+1, thanks for bringing this up, Wangda. This will help expanding the Hadoop ecosystem by supporting new AI/ML workloads. Thanks, Xiaoyu On 2/1/19, 10:58 AM, "Dinesh Chitlangia" wrote: +1 This is a fantastic recommendation given the increasing interest in ML across the globe.

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Ajay Kumar
+1, Thanks for driving this. With rise of use cases running ML along with traditional applications this will be of great help. Thanks, Ajay On 2/1/19, 10:49 AM, "Suma Shivaprasad" wrote: +1. Thanks for bringing this up Wangda. Makes sense to have Submarine follow its own

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Wangda Tan
Thanks everyone for sharing thoughts! Eric, appreciate your suggestions. But there are many examples to have separate releases, like Hive's storage API, OZone, etc. For loosely coupled sub-projects, it gonna be great (at least for most of the users) to have separate releases so new features can

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Suma Shivaprasad
+1. Thanks for bringing this up Wangda. Makes sense to have Submarine follow its own release cadence given the good momentum/adoption so far. Also, making it run with older versions of Hadoop would drive higher adoption. Suma On Fri, Feb 1, 2019 at 9:40 AM Eric Yang wrote: > Submarine is an

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread John Zhuge
+1 Does Submarine support Jupyter? On Fri, Feb 1, 2019 at 8:54 AM Zhe Zhang wrote: > +1 on the proposal and looking forward to the progress of the project! > > On Thu, Jan 31, 2019 at 10:51 PM Weiwei Yang wrote: > > > Thanks for proposing this Wangda, my +1 as well. > > It is amazing to see

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Eric Yang
Submarine is an application built for YARN framework, but it does not have strong dependency on YARN development. For this kind of projects, it would be best to enter Apache Incubator cycles to create a new community. Apache commons is the only project other than Incubator that has

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Zhe Zhang
+1 on the proposal and looking forward to the progress of the project! On Thu, Jan 31, 2019 at 10:51 PM Weiwei Yang wrote: > Thanks for proposing this Wangda, my +1 as well. > It is amazing to see the progress made in Submarine last year, the > community grows fast and quiet collaborative. I

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread runlin zhang
+1, It is very necessary to use Submarine on older Hadoop. What's more, the development of deep learning is too fast, and Submarine must keep faster release iterate . > 在 2019年2月1日,上午2:53,Wangda Tan 写道: > > Hi devs, > > Since we started submarine-related effort last year, we received a

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Sunil G
+1 from me on this. ML/DL is one of the fast growing areas and a runtime on YARN helps customers to have ML/DL workloads to run on same cluster where the ETL or other traditional big data workloads ingest or mine data. Faster release cadence can pace up the development for Submarine and more agile

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Rohith Sharma K S
+1, Few interested ML/DL folks from Banglore asked about Submarine release for trying out TensorFlow on YARN. We told them wait for release since they were not ready to use trunk. I see agile release cycle for Submarine brings lot of added value. -Rohith Sharma K S On Fri, 1 Feb 2019 at 00:34,

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Arun Suresh
Thanks for bringing this up Wangda. +1 Makes a lot of sense to have Submarine follow its own release cadence - for all the reasons you outlined. I would one up this proposal to ask why shouldn't we allow YARN to have its own releases as well - but that is for a separate thread :) Cheers -Arun On

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Anu Engineer
>> I propose to adopt Ozone model: which is the same master branch, different >> release cycle, and different release branch. It is a great example to show >> agile release we can do (2 Ozone releases after Oct 2018) with less >> overhead to setup CI, projects, etc. I second this, especially

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Jonathan Hung
+1. This is important for improving the deep learning on hadoop story. There's recently a lot of momentum for this, and decoupling submarine/hadoop will help it continue. Jonathan Hung On Thu, Jan 31, 2019 at 11:04 AM Wangda Tan wrote: > Hi devs, > > Since we started submarine-related effort

[DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Wangda Tan
Hi devs, Since we started submarine-related effort last year, we received a lot of feedbacks, several companies (such as Netease, China Mobile, etc.) are trying to deploy Submarine to their Hadoop cluster along with big data workloads. Linkedin also has big interests to contribute a Submarine