Is there a technique for forcing the evaluation of an RDD? I have used actions to do so but even the most basic "count" has a non-negligible cost (even on a cached RDD, repeated calls to count take time).
My use case is for logging the execution time of the major components in my application. At the end of each component I have a statement like "rdd.cache().count()" and time how long it takes. Thanks in advance for any advice! Nick -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Force-RDD-evaluation-tp21748.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
