Hi all, Is it advisable to use nfs as shared storage for a small Spark cluster to process video and images? I have a total of 20 vms (2vCPU, 6GB Ram, 20GB Local Disk) connected to 500GB nfs shared storage (mounted the same in each of the vms) at my disposal and I'm wondering if I can avoid the need for hdfs and instead use the larger capacity nfs to work with my videos and images in Spark?
I have spun up a master node (using maven not sbt) and connected 1 slave to it but I haven't made any configuration changes to Spark. On masternode:4040/ I don't see anything under storage. Is that to be expected and if I do need to spin up hadoop can I double job the 20vms by running both hadoop and spark on all 20 machines or would the recommendation be I split them into separate hadoop and spark clusters Michael -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Apache-Spark-Video-Processing-from-NFS-Shared-storage-Advise-needed-tp27030.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org