On Mon, Dec 29, 2014 at 12:00 AM, Eric Friedman <eric.d.fried...@gmail.com> wrote:
I'm not seeing RDDs or SRDDs cached in the Spark UI. That page remains > empty despite my calling cache(). > Just a small note on this, and perhaps you already know: Calling cache() is not enough to cache something and thus make it show up in the UI. Caching in Spark is lazy. You need to call cache() and then run some some action (like count()) to force the RDD into memory. Then you’ll be able to see it in the UI under storage. You should even be able to see the “cached fraction” increase with page refreshes as Spark gradually loads the RDD bit by bit. Nick
