SparkContext.getRDDStorageInfo
On Tue, Apr 29, 2014 at 12:34 PM, Andras Nemeth < [email protected]> wrote: > Hi, > > Is it possible to know from code about an RDD if it is cached, and more > precisely, how many of its partitions are cached in memory and how many are > cached on disk? I know I can get the storage level, but I also want to know > the current actual caching status. Knowing memory consumption would also be > awesome. :) > > Basically what I'm looking for is the information on the storage tab of > the UI, but accessible from the API. > > Thanks, > Andras >
