value of broadcast variable is saved in a file on HTTP server in case of HttpBroadcast( default one). You can check files starting with "broadcast_" in /tmp ( or spark.local.dir) to verify number of files being created by broadcast variables in your case.
On 4 February 2014 18:03, Akhil Das <[email protected]> wrote: > Hi Sourav, > > If you do ulimit -n that will reset the limit for the current shell. > > If you are using Linux and you got the permission error, you will need to > raise the allowed limit in the /etc/limits.conf or > /etc/security/limits.conf file (where the file is located depends on your > specific Linux distribution). > > For example to allow anyone on the machine to raise their number of open > files up to 10000 add the line to the limits.conf file. > > * hard nofile 10000 > > Then logout and relogin to your system and you should be able to do: > > ulimit -n 10000 > > > - > AkhilDas > CodeBreach.in > > - in.linkedin.com/in/akhildas/ > >
