DStream.foreachRDD gives you an RDD[String] for each interval of course. I don't think it makes sense to say a DStream can be converted into one RDD since it is a stream. The past elements are inherently not supposed to stick around for a long time, and future elements aren't known. You may consider saving each RDD[String] to HDFS, and then simply loading it from HDFS as an RDD[String].
On Wed, Dec 3, 2014 at 7:45 AM, Hafiz Mujadid <hafizmujadi...@gmail.com> wrote: > Hi everyOne! > > I want to convert a DStream[String] into an RDD[String]. I could not find > how to do this. > > var data = KafkaUtils.createStream[Array[Byte], Array[Byte], DefaultDecoder, > DefaultDecoder](ssc, consumerConfig, topicMap, > StorageLevel.MEMORY_ONLY).map(_._2) > val streams = data.window(Seconds(interval), Seconds(interval)).map(x => > new String(x)) > > Now I want to convert this streams into a single RDD[String]. > > > Any help please. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/converting-DStream-String-into-RDD-String-in-spark-streaming-tp20253.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
