peristent-hdfs server is set to 9010 port, instead of 9000. Does spark need more config for this?
On Thu, Jan 23, 2014 at 12:26 AM, Patrick Wendell <[email protected]>wrote: > > 1. It seems by default spark ec2 uses ephemeral hdfs, how to switch this > to > > persistent hdfs? > You can stop the ephemeral one using > > /root/ephemeral-hdfs/bin/stop-dfs.sh > > and start the persistent one using > > /root/persistent-hdfs/bin/start-dfs.sh > > > 2. By default persistent hdfs server is not up, is this meant to be like > > this? > > Yes - it starts only an ephemeral one: > > "The spark-ec2 script already sets up a HDFS instance for you. It’s > installed in /root/ephemeral-hdfs" >
