Thanks Guys for reply, but I have found this piece of code in streaming:
def print() {
585 def foreachFunc = (rdd: RDD[T], time: Time) => {
586 //val first11 = rdd.take(11)
587 //val first100 = rdd.take(100)
588 val total = rdd.collect()
589 println ("-------------------------------------------")
590 println ("Time: " + time)
591 println ("-------------------------------------------")
592 //first100.take(100).foreach(println)
593 total.collect().foreach(println)
594 //if (first100.size > 99) println("(...)")
595 println()
596 }
597 new ForEachDStream(this,
context.sparkContext.clean(foreachFunc)).register()
598 }
I would like to print whole the RDD, I have seen I should use the
collect() function instead of take(), I have done but I have gotten a
compilation error.
does anyone help me?
BR
Em 3/25/14, 13:48, Sourav Chandra escreveu:
You can extend DStream and override print() method. Then you can
create your own DSTream extending from this.
On Tue, Mar 25, 2014 at 6:07 PM, Eduardo Costa Alfaia
<e.costaalf...@unibs.it <mailto:e.costaalf...@unibs.it>> wrote:
Hi Guys,
I think that I already did this question, but I don't remember if
anyone has answered me. I would like changing in the function
print() the quantity of words and the frequency number that are
sent to driver's screen. The default value is 10.
Anyone could help me with this?
Best Regards
--
Informativa sulla Privacy: http://www.unibs.it/node/8155
--
Sourav Chandra
Senior Software Engineer
· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·
sourav.chan...@livestream.com <mailto:sourav.chan...@livestream.com>
o: +91 80 4121 8723
m: +91 988 699 3746
skype: sourav.chandra
Livestream
"Ajmera Summit", First Floor, #3/D, 68 Ward, 3rd Cross, 7th C Main,
3rd Block, Koramangala Industrial Area,
Bangalore 560034
www.livestream.com <http://www.livestream.com/>
--
Informativa sulla Privacy: http://www.unibs.it/node/8155
