Re: Saving Data only if Dstream is not empty

2014-12-10 Thread manasdebashiskar
Can you do a countApprox as a condition to check non-empty RDD? ..Manas - Manas Kar -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Saving-Data-only-if-Dstream-is-not-empty-tp20587p20617.html Sent from the Apache Spark User List mailing list archive

Re: Saving Data only if Dstream is not empty

2014-12-09 Thread Gerard Maas
reachRDD(rdd=>rdd.foreach(println)) > > > > //what condition can be applied here to store only non empty DStream > > streams.saveAsTextFiles(sink, "msg") > > Thanks > > > > > > > > > > -- &

Re: Saving Data only if Dstream is not empty

2014-12-09 Thread Sean Owen
iles(sink, "msg") > Thanks > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Saving-Data-only-if-Dstream-is-not-empty-tp20587.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > >

Saving Data only if Dstream is not empty

2014-12-08 Thread Hafiz Mujadid
sg") Thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Saving-Data-only-if-Dstream-is-not-empty-tp20587.html Sent from the Apache Spark User List mailing list archive at Nabble.com.