Hi, hadoop submarine team I am the head of the development of the machine learning platform of the Dahua Research Institute of Dahua (https://www.dahuatech.com/). We use hadoop submarine as a machine learning engine. Through machine learning algorithms, Characterizing our image and video data,
Currently, hadoop submarine is not good enough for model serving. I hope that the version of hadoop submarine is updated faster. +1 ---------- Forwarded message --------- 发件人： Szilard Nemeth <snem...@cloudera.com<mailto:snem...@cloudera.com>> Date: 2019年7月17日周三 下午2:55 Subject: Re: Any thoughts making Submarine a separate Apache project? To: runlin zhang <runlin...@gmail.com<mailto:runlin...@gmail.com>> Cc: Xun Liu <liu...@apache.org<mailto:liu...@apache.org>>, Hadoop Common <common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>>, <yarn-...@hadoop.apache.org<mailto:yarn-...@hadoop.apache.org>>, Hdfs-dev <hdfs-...@hadoop.apache.org<mailto:hdfs-...@hadoop.apache.org>>, <mapreduce-...@hadoop.apache.org<mailto:mapreduce-...@hadoop.apache.org>>, <submarine-dev@hadoop.apache.org<mailto:submarine-dev@hadoop.apache.org>> +1, this is a very great idea. As Hadoop repository has already grown huge and contains many projects, I think in general it's a good idea to separate projects in the early phase. On Wed, Jul 17, 2019, 08:50 runlin zhang <runlin...@gmail.com<mailto:runlin...@gmail.com>> wrote: +1 ，That will be great ！ > 在 2019年7月10日，下午3:34，Xun Liu <liu...@apache.org<mailto:liu...@apache.org>> 写道： > > Hi all, > > This is Xun Liu contributing to the Submarine project for deep learning > workloads running with big data workloads together on Hadoop clusters. > > There are a bunch of integrations of Submarine to other projects are > finished or going on, such as Apache Zeppelin, TonY, Azkaban. The next step > of Submarine is going to integrate with more projects like Apache Arrow, > Redis, MLflow, etc. & be able to handle end-to-end machine learning use > cases like model serving, notebook management, advanced training > optimizations (like auto parameter tuning, memory cache optimizations for > large datasets for training, etc.), and make it run on other platforms like > Kubernetes or natively on Cloud. LinkedIn also wants to donate TonY project > to Apache so we can put Submarine and TonY together to the same codebase > (Page #30. > https://www.slideshare.net/xkrogen/hadoop-meetup-jan-2019-tony-tensorflow-on-yarn-and-beyond#30 > ). > > This expands the scope of the original Submarine project in exciting new > ways. Toward that end, would it make sense to create a separate Submarine > project at Apache? This can make faster adoption of Submarine, and allow > Submarine to grow to a full-blown machine learning platform. > > There will be lots of technical details to work out, but any initial > thoughts on this? > > Best Regards, > Xun Liu --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org<mailto:common-dev-unsubscr...@hadoop.apache.org> For additional commands, e-mail: common-dev-h...@hadoop.apache.org<mailto:common-dev-h...@hadoop.apache.org>
