ave
>>>> experience with them?
>>>>
>>>> Thanks
>>>>
>>>> On Sat, May 4, 2019 at 8:01 PM Pat Ferrel
>>>> wrote:
>>>>
>>>>> @Riccardo
>>>>>
>>>>> Spark does not do th
t;
>>>> Thanks
>>>>
>>>> On Sat, May 4, 2019 at 8:01 PM Pat Ferrel
>>>> wrote:
>>>>
>>>>> @Riccardo
>>>>>
>>>>> Spark does not do the DL learning part of the pipeline (afaik) so it
>>>>
t of the pipeline (afaik) so it is
>>>> limited to data ingestion and transforms (ETL). It therefore is optional
>>>> and other ETL options might be better for you.
>>>>
>>>> Most of the technologies @Gourav mentions have their own scaling based
>>
;>> and other ETL options might be better for you.
>>>
>>> Most of the technologies @Gourav mentions have their own scaling based
>>> on their own compute engines specialized for their DL implementations, so
>>> be aware that Spark scaling has nothing t
t;
>> Reply: Gourav Sengupta
>>
>> Date: May 4, 2019 at 10:24:29 AM
>> To: Riccardo Ferrari
>> Cc: User
>> Subject: Re: Deep Learning with Spark, what is your experience?
>>
>> Try using MxNet and Horovod directly as well (I think that MXNet is w
do with scaling most of the DL
>> engines, they have their own solutions.
>>
>> From: Gourav Sengupta
>>
>> Reply: Gourav Sengupta
>>
>> Date: May 4, 2019 at 10:24:29 AM
>> To: Riccardo Ferrari
>> Cc: User
>> Subject: Re: De
Sengupta
>
> Date: May 4, 2019 at 10:24:29 AM
> To: Riccardo Ferrari
> Cc: User
> Subject: Re: Deep Learning with Spark, what is your experience?
>
> Try using MxNet and Horovod directly as well (I think that MXNet is worth
> a try as well):
> 1.
> https://medi
Learning with Spark, what is your experience?
Try using MxNet and Horovod directly as well (I think that MXNet is worth a
try as well):
1.
https://medium.com/apache-mxnet/distributed-training-using-apache-mxnet-with-horovod-44f98bf0e7b7
2.
https://docs.nvidia.com/deeplearning/dgx/mxnet-release
Try using MxNet and Horovod directly as well (I think that MXNet is worth a
try as well):
1.
https://medium.com/apache-mxnet/distributed-training-using-apache-mxnet-with-horovod-44f98bf0e7b7
2.
https://docs.nvidia.com/deeplearning/dgx/mxnet-release-notes/rel_19-01.html
3.
Hi list,
I am trying to undestand if ti make sense to leverage on Spark as enabling
platform for Deep Learning.
My open question to you are:
- Do you use Apache Spark in you DL pipelines?
- How do you use Spark for DL? Is it just a stand-alone stage in the
workflow (ie data preparation
10 matches
Mail list logo
