Hi, I'm using spark on a cluster of 8 VMs, each with two cores and 3.5GB RAM.
But I need to run a shortest path algorithm on data of 500+GB(textfile, each line contains a node id and nodes it points to) I've tested it on the cluster, but the speed seems to be extremely slow, and haven't got any result yet. Is it natural to be so slow based on such cluster and data, or there is something wrong since the problem can be solved much efficiently?(say half an hour after reading the data?) Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/The-running-time-of-spark-tp12624.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
