Thanks Josh
On Wed, Nov 20, 2013 at 5:01 PM, Josh Rosen <[email protected]> wrote: > JavaPairRDD should have had a unpersist() method; we'll fix this bug in > 0.8.1 (see https://github.com/apache/incubator-spark/pull/103). In the > meantime, just call myJavaPairRDD.rdd().unpersist() (see > https://mail-archives.apache.org/mod_mbox/incubator-spark-user/201310.mbox/%3ccaoepxp5syqn7v9gdomjcvgqzvaad-epewxi025jpmu_neic...@mail.gmail.com%3E > ). > > > On Wed, Nov 20, 2013 at 4:39 PM, sasmita Patra <[email protected]>wrote: > >> Hi, >> I have two datasets that i load from HDFS file system. After loading the >> files, i cache the datasets. >> >> I have a requirement to do join on these datasets (LEFT/RIGHT INNER/OUTER >> JOIN) and apply some filter conditions and then run multiple queries on >> this joined filtered dataset. >> >> I have created JavaPairRDDs for the datasets, doing the join, applying >> filter and caching the dataset. >> >> After running the queries, i need to unpersist this dataset. Currently i >> don't see any unpersist api for JavaPairRDD. >> >> Is there any way to unpersist the JavaPairRDD? >> >> Thanks, >> Sasmita >> >> >
