Hi,
I'm working with an infrastructure that already has its own web server set up
on EC2. I would like to set up a separate spark cluster on EC2 with the scripts
and have the web server submit jobs to this spark cluster.
Is it possible to do this? I'm getting some errors running the spark
The main issue with running a spark-shell locally is that it orchestrates
the actual computation, so you want it to be close to the actual Worker
nodes for latency reasons. Running a spark-shell on EC2 in the same region
as the Spark cluster avoids this problem.
The error you're seeing seems to
