Just a head's up, there is an active <https://github.com/apache/spark/pull/126>*pull requeust* that will automatically unpersist RDDs that are not in reference/scope from the application any more.
TD On Wed, Mar 19, 2014 at 6:58 PM, hequn cheng <[email protected]> wrote: > persist and unpersist. > unpersist:Mark the RDD as non-persistent, and remove all blocks for it > from memory and disk > > > 2014-03-19 16:40 GMT+08:00 林武康 <[email protected]>: > > Hi, can any one tell me about the lifecycle of an rdd? I search through >> the official website and still can't figure it out. Can I use an rdd in >> some stages and destroy it in order to release memory because that no >> stages ahead will use this rdd any more. Is it possible? >> >> Thanks! >> >> Sincerely >> Lin wukang >> > >
