Hello Community,
I'm running Spark on a single Server in a stand-alone mode . The Server has
*two cores* and so while initializing the SparkContext object, I used
*var sc = new SparkContext("local[2]", "MyApp", SPARK_HOME)*
[ As per this
http://spark.incubator.apache.org/docs/0.6.0/scala-programming-guide.html#master-urls
]
But, the program while processing files of size more than 60MB hangs. This
without emitting any error message or anything(I have my Heap size
increased to large size). Also, when I checked *the _temporary directory * in
output location, I noticed that the program has stopped doing any further
processing/parsing of my input directory.
Any ideas on why it happens when I use "local[2]" ?
Regards,
Ramkumar Chokkalingam ,
University of Washington.
LinkedIn <http://www.linkedin.com/in/mynameisram>
