hi, community

I updated the design documentation, 
https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit#
 
<https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit#>
added the interactive development of machine learning algorithms and the 
overall design of submitting Note packages to YARN for job execution.

I have completed the development of the pre-research part of the code, and 
established JIRA: https://issues.apache.org/jira/browse/ZEPPELIN-3856 
<https://issues.apache.org/jira/browse/ZEPPELIN-3856>

welcome comments, thank you!

> 在 2018年10月30日,下午10:11,liuxun <neliu...@163.com> 写道:
> 
> HI, community
> 
> 
> I updated the design document and submitted it to the hadoop community.
> https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit#
>  
> <https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit#>
> 
> If you have any questions, you can ask them in the email or directly in the 
> document. Thank you!
> 
> I developed the submarine interpreter module in zeppelin. Below is the 
> interface for the system to run.
> 
> 1. zeppelin submarine interpreter properties
> <submarine-interpreter.png>
> 1.1 System administrators can configure multiple interpreters, and different 
> interpreters can have different resource configurations.
> 1.2 Allow different users to use different interpreters for resource 
> allocation and management.
> 
> 2. zeppelin submarine Tensorflow interpreter
> <submarine-tensorflow.png>
> 2.1 Users use their own interpreter, First through the %submarine.tensorflow 
> paragraph, To write python code for tensorflow.
> 2.2 After the user writes the python code of tensorflow, Click the [RUN] 
> button and zeppelin will upload the python code to the specified HDFS 
> directory. The submarine is loaded into the docker container when it is 
> running.
> 
> 3. zeppelin submarine interpreter
> <submarine-command.png>
> 3.1 The user sets the call parameter value of the tensorflow python code, 
> Then enter the job run command.
> 3.2 The zeppelin submarine interpreter first checks that all parameters of 
> the submarine run are set completely. After the check is passed, the 
> implementation of YARN is performed via submarine.jar.
> 3.3 The progress and log information of the submarine will be displayed in 
> zeppelin's note.
> 3.4 You can enter submarine's other commands in the submarine interpreter, 
> view all the jobs in the submarine, etc.
> 
> 
>> 在 2018年10月30日,下午6:40,liuxun <neliu...@163.com <mailto:neliu...@163.com>> 写道:
>> 
>> 
>> HI,
>> 
>> 
>>      I updated the design document and submitted it to the hadoop community.
>>      
>> https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit#
>>  
>> <https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit#>
>>  
>> <https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit#
>>  
>> <https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit#>>
>>  
>>      If you have any questions, you can ask them in the email or directly in 
>> the document. Thank you!
>> 
>>      I developed the submarine interpreter module in zeppelin. Below is the 
>> interface for the system to run.
>> 
>> zeppelin submarine interpreter properties
>> 
>> System administrators can configure multiple interpreters, and different 
>> interpreters can have different resource configurations.
>> Allow different users to use different interpreters for resource allocation 
>> and management.
>> zeppelin submarine Tensorflow interpreter
>> 
>> Users use their own interpreter, First through the %submarine.tensorflow 
>> paragraph, To write python code for tensorflow.
>> After the user writes the python code of tensorflow, Click the [RUN] button 
>> and zeppelin will upload the python code to the specified HDFS directory. 
>> The submarine is loaded into the docker container when it is running.
>> 
>> zeppelin submarine interpreter
>> 
>> The user sets the call parameter value of the tensorflow python code, Then 
>> enter the job run command.
>> The zeppelin submarine interpreter first checks that all parameters of the 
>> submarine run are set completely. After the check is passed, the 
>> implementation of YARN is performed via submarine.jar.
>> The progress and log information of the submarine will be displayed in 
>> zeppelin's note.
>> You can enter submarine's other commands in the submarine interpreter, view 
>> all the jobs in the submarine, etc.
>> 
>> 
>> 
>>> 在 2018年10月21日,上午6:26,Felix Cheung <felixcheun...@hotmail.com 
>>> <mailto:felixcheun...@hotmail.com>> 写道:
>>> 
>>> Very cool!
>>> 
>>> 
>>> ________________________________
>>> From: Jeff Zhang <zjf...@gmail.com <mailto:zjf...@gmail.com>>
>>> Sent: Friday, October 19, 2018 7:14 AM
>>> To: d...@zeppelin.apache.org <mailto:d...@zeppelin.apache.org>
>>> Subject: Re: Zeppelin add hadoop submarine(machine learning framework) 
>>> interpreter
>>> 
>>> Thanks xun. This would be a great addon for zeppelin to support deep
>>> learning. I will check the design later.
>>> 
>>> 
>>> 
>>> liu xun <neliu...@163.com <mailto:neliu...@163.com>>于2018年10月19日周五 下午3:56写道:
>>> 
>>>> Hi,
>>>> Hadoop Submarine is the latest machine learning framework subproject in
>>>> the Hadoop 3.2 release. It allows Hadoop to support Tensorflow,
>>>> MXNet,Caffe, Spark, etc. A variety of deep learning frameworks provide a
>>>> full-featured system framework for machine learning algorithm development,
>>>> distributed model training, model management, and model publishing,
>>>> combined with hadoop's intrinsic data storage and data processing
>>>> capabilities to enable data scientists to Good mining and the value of the
>>>> data.
>>>> 
>>>> 
>>>> I was involved in the development of the hadoop submarine project. So I
>>>> plan to add the interpreter module of hadoop submarine to zeppelin.
>>>> Let zeppeline increase the development of deep learning. This is my design
>>>> document, let's see if there is any opinion, you can put it directly in the
>>>> document, thank you!
>>>> 
>>>> 
>>>> https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit?ts=5bc6bfdd
>>>>  
>>>> <https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit?ts=5bc6bfdd>
>>>> 
>>>> 
>>>> 
>> 
> 

Reply via email to