Great! I don't know if I'll have time to try it soon but good to know that it's worth giving it another shot. Would be great if someone could report in if they have been able to get it up and running.
Mike On Tue, Jan 21, 2014 at 4:09 PM, Tathagata Das <[email protected]>wrote: > Actually, I just realised why it wasnt working. I just made some changes > in the examples and last night and they are in the 0.9-rc4 that is being > voted now. With those changes the streaming examples should work. > > The issue was that the thread running the main() of the examples was not > waiting for the streaming context to continue processing. So the processing > was being cancelled as soon as the streaming context was started. This has > been fixed with the examples, and running the twitter examples should be > the same as running the SparkPi example (though you have to set the > twitter4j.oauth.* properties for the twitter example to work). > > TD > > > On Tue, Jan 21, 2014 at 12:13 PM, Mike Percy <[email protected]> wrote: > >> Just FYI I got it working in standalone mode with no probs, so thanks for >> the help Tathagata. I was not able to get it working on YARN, I gave up. >> >> Thanks, >> Mike >> >> >> On Fri, Jan 10, 2014 at 6:17 PM, Tathagata Das < >> [email protected]> wrote: >> >>> Let me know if you have any trouble getting it running on the standalone >>> mode. That will be easier for us to help out with. >>> >>> TD >>> >>> >>> On Fri, Jan 10, 2014 at 5:47 PM, Mike Percy <[email protected]> wrote: >>> >>>> On Fri, Jan 10, 2014 at 6:35 AM, Neal Sidhwaney <[email protected]>wrote: >>>> >>>>> Are you setting the SPARK_JAVA_OPTS environment variable? Under 0.8.1, >>>>> we had to set that to -Dlog4j.configuration=<path to logging >>>>> properties file> to get STDOUT for workers. This was in standalone >>>>> mode, though not with YARN, so YMMV. >>>>> Neal >>>>> >>>> >>>> Thanks for the idea Neal. The docs say that just putting it into conf/ >>>> should do the trick, but apparently that's not the case. I tried setting >>>> that and it just stopped printing on the shell and had no effect on the >>>> deployed container. >>>> >>>> Too much struggling with this, I'm going to work on getting this >>>> running in standalone mode. >>>> >>>> Mike >>>> >>>> >>> >> >
