Hi Jeremy,
Thanks for the reply.
We got Spark on our setup after a similar script was brought up to work with
LSF.
Really appreciate your help.
Will keep in touch on Twitter
Thanks,@sidkashyap :)
From: freeman.jer...@gmail.com
Subject: Re: Spark on an HPC setup
Date: Thu, 29 May 2014 00:37:54
Hi Sid,
We are successfully running Spark on an HPC, it works great. Here's info on our
setup / approach.
We have a cluster with 256 nodes running Scientific Linux 6.3 and scheduled by
Univa Grid Engine. The environment also has a DDN GridScalar running GPFS and
several EMC Isilon clusters se
Hi,
Has anyone tried to get Spark working on an HPC setup?If yes, can you please
share your learnings and how you went about doing it?
An HPC setup typically comes bundled with dynamically allocated cluster and a
very efficient scheduler.
Configuring Spark standalone in this mode of operation is
