You could use Horovod to distribute your ML algorithm on a cluster,
while Horovod also supports Spark clusters.
Enrico
Am 06.09.20 um 15:30 schrieb Ankur Das:
Good Evening Sir/Madam,
Hope you are doing well, I am experimenting on some ML techniques
where I need to test it on a distributed environment.
For example a particular algorithm I want to run it on different nodes
at the same time and collect the results at the end in one single node
or the parent node.
So, I would like to know if it is possible or a good choice to use
spark for this.
Hope to hear from you soon, Stay safe and healthy
Thanking you in advance.
--
Regards,
Ankur J Das
Research Scholar @ Tezpur University
Tezpur, Assam
