subject:"RE\: Spark on an HPC setup"

RE: Spark on an HPC setup

2014-08-11 Thread Sidharth Kashyap

Hi Jeremy, Thanks for the reply. We got Spark on our setup after a similar script was brought up to work with LSF. Really appreciate your help. Will keep in touch on Twitter Thanks,@sidkashyap :) From: freeman.jer...@gmail.com Subject: Re: Spark on an HPC setup Date: Thu, 29 May 2014 00:37:54

Re: Spark on an HPC setup

2014-05-28 Thread Jeremy Freeman

Hi Sid, We are successfully running Spark on an HPC, it works great. Here's info on our setup / approach. We have a cluster with 256 nodes running Scientific Linux 6.3 and scheduled by Univa Grid Engine. The environment also has a DDN GridScalar running GPFS and several EMC Isilon clusters se