I am assuming you are using Spark 0.8.0 and not 0.6.0. Have you looked at
the worker logs to see what is happening there?
On Oct 30, 2013 2:21 AM, "Ramkumar Chokkalingam" <[email protected]>
wrote:
> Hello Community,
>
> I'm running Spark on a single Server in a stand-alone mode . The Server
> has *two cores* and so while initializing the SparkContext object, I
> used
>
> *var sc = new SparkContext("local[2]", "MyApp", SPARK_HOME)*
>
> [ As per this
> http://spark.incubator.apache.org/docs/0.6.0/scala-programming-guide.html#master-urls
> ]
>
> But, the program while processing files of size more than 60MB hangs. This
> without emitting any error message or anything(I have my Heap size
> increased to large size). Also, when I checked *the _temporary directory * in
> output location, I noticed that the program has stopped doing any further
> processing/parsing of my input directory.
>
> Any ideas on why it happens when I use "local[2]" ?
>
> Regards,
>
> Ramkumar Chokkalingam ,
> University of Washington.
> LinkedIn <http://www.linkedin.com/in/mynameisram>
>
>
>
>
