Thanks Akhil, Yes indeed this is why it works when using local[2] but I'm unclear of why it doesn't work when using standalone daemons?
Is there way to check what cores are being seen when running against standalone daemons? I'm running the master and worker on same ubuntu host. The Driver program is running from a windows machine. On ubuntu host command cat /proc/cpuinfo | grep processor | wc -l is giving 2 On Windows machine it is: NumberOfCores=2 NumberOfLogicalProcessors=4 On Wed, Mar 25, 2015 at 2:06 PM, Akhil Das <[email protected]> wrote: > Spark Streaming requires you to have minimum of 2 cores, 1 for receiving > your data and the other for processing. So when you say local[2] it > basically initialize 2 threads on your local machine, 1 for receiving data > from network and the other for your word count processing. > > Thanks > Best Regards > > On Wed, Mar 25, 2015 at 6:31 PM, James King <[email protected]> wrote: > >> I'm trying to run the Java NetwrokWordCount example against a simple >> spark standalone runtime of one master and one worker. >> >> But it doesn't seem to work, the text entered on the Netcat data server >> is not being picked up and printed to Eclispe console output. >> >> However if I use conf.setMaster("local[2]") it works, the correct text >> gets picked up and printed to Eclipse console. >> >> Any ideas why, any pointers? >> > >
