Re: broadcast in spark streaming

I get it. thank you On Fri, Jun 20, 2014 at 4:43 PM, Sourav Chandra < sourav.chan...@livestream.com> wrote: > From the StreamingContext object, you can get reference of SparkContext > using which you can create broadcast variables > > > On Fri, Jun 20, 2014 at 2:09 PM

broadcast in spark streaming

I want to use broadcast in spark streaming, but I found there is no this function. How can I use global variable in spark streaming? thanks

Re: Error when I use spark-streaming

I found it. I should run "nc -lk " at first and then run the NetworkWordCount. Thanks On Fri, Apr 11, 2014 at 4:13 PM, Schein, Sagi wrote: > I would check the DNS setting. > > Akka seems to pick configuration from FQDN on my system > > > >

Error when I use spark-streaming

hi all, When I run spark-streaming use NetworkWordCount in example, it always throw this Exception. I don't understand why it can't connect and I don't restrict port. 14/04/11 15:38:56 ERROR SocketReceiver: Error receiving data in receiver 0 java.net.ConnectException: Connection refuse

Re:

> @mayur_rustagi <https://twitter.com/mayur_rustagi> > > > > On Thu, Mar 27, 2014 at 4:07 AM, Hahn Jiang wrote: > >> I set "ulimit -n 10" in conf/spark-env.sh, is it too small? >> >> >> On Thu, Mar 27, 2014 at 3:36 PM, Sonal Goyal wrote: >

Re:

://mail-archives.apache.org/mod_mbox/spark-user/201402.mbox/%3ccangvg8qpn_wllsrcjegdb7hmza2ux7myxzhfvtz+b-sdxdk...@mail.gmail.com%3E > > > Sent from my iPad > > > On Mar 27, 2014, at 12:15 PM, Hahn Jiang > wrote: > > > > Hi, all > > > > I write a spark pro

[no subject]

Hi, all I write a spark program on yarn. When I use small size input file, my program can run well. But my job will failed if input size is more than 40G. the error log: java.io.FileNotFoundException (java.io.FileNotFoundException: /home/work/data12/yarn/nodemanager/usercache/appcache/application