I have a situation similar to the following and I get SPARK-13758 <https://issues.apache.org/jira/browse/SPARK-13758>.
I understand why I get this error, but I want to know what should be the approach in dealing with these situations. Thanks > var cached = ssc.sparkContext.parallelize(Seq("apple, banana")) > val words = ssc.socketTextStream(ip, port).flatMap(_.split(" ")) > words.foreachRDD((rdd: RDD[String]) => { > val res = rdd.map(word => (word, word.length)).collect() > println("words: " + res.mkString(", ")) > cached = cached.union(rdd) > cached.checkpoint() > println("cached words: " + cached.collect.mkString(", ")) > })