Thanks Koert, very useful!
On Tue, Apr 29, 2014 at 6:41 PM, Koert Kuipers <ko...@tresata.com> wrote: > SparkContext.getRDDStorageInfo > > > On Tue, Apr 29, 2014 at 12:34 PM, Andras Nemeth < > andras.nem...@lynxanalytics.com> wrote: > >> Hi, >> >> Is it possible to know from code about an RDD if it is cached, and more >> precisely, how many of its partitions are cached in memory and how many are >> cached on disk? I know I can get the storage level, but I also want to know >> the current actual caching status. Knowing memory consumption would also be >> awesome. :) >> >> Basically what I'm looking for is the information on the storage tab of >> the UI, but accessible from the API. >> >> Thanks, >> Andras >> > >
