Re: Convert DStream to String

2014-10-29 Thread Sean Owen
Sure, that code looks like it does sort of what you describe but it's mixed up in a few ways. It looks like you only want to operate on words that start with SECRETWORD, but then you are prepending acct and _ in the code but expecting something appending in the result. You also seem like you want t

Re: Convert DStream to String

2014-10-29 Thread Harold Nguyen
Hi Sean, I'd just like to take the first "word" of every line, and use it as a variable for later. Is there a way to do that? Here's the gist of what I want to do: val lines = KafkaUtils.createStream(ssc, "localhost:2181", "test", Map("test" -> 10)).map(_._2) val words = lines.flatMap(_.spli

Re: Convert DStream to String

2014-10-29 Thread Sean Owen
What would it mean to make a DStream into a String? it's inherently a sequence of things over time, each of which might be a string but which are usually RDDs of things. On Wed, Oct 29, 2014 at 11:15 PM, Harold Nguyen wrote: > Hi all, > > How do I convert a DStream to a string ? > > For instance,

Convert DStream to String

2014-10-29 Thread Harold Nguyen
Hi all, How do I convert a DStream to a string ? For instance, I want to be able to: val myword = words.filter(word => word.startsWith("blah")) And use "myword" in other places, like tacking it onto (key, value) pairs, like so: val pairs = words.map(word => (myword+"_"+word, 1)) Thanks for an