I am facing same issue, posted a new thread. Please respond. On Wed, Jul 9, 2014 at 1:56 AM, Rahul Bhojwani <rahulbhojwani2...@gmail.com> wrote:
> Hi, > > My code was running properly but then it suddenly gave this error. Can you > just put some light on it. > > ####################### > 0 KB, free: 38.7 MB) > 14/07/09 01:46:12 INFO BlockManagerMaster: Updated info of block rdd_2212_4 > 14/07/09 01:46:13 INFO PythonRDD: Times: total = 1486, boot = 698, init = > 626, finish = 162 > Exception in thread "stdin writer for python" 14/07/09 01:46:14 INFO > MemoryStore: ensureFreeSpace(61480) called with cur > Mem=270794224, maxMem=311387750 > java.lang.OutOfMemoryError: Java heap space > at java.io.BufferedOutputStream.<init>(Unknown Source) > at > org.apache.spark.api.python.PythonRDD$$anon$2.run(PythonRDD.scala:62) > 14/07/09 01:46:15 INFO MemoryStore: Block rdd_2212_0 stored as values to > memory (estimated size 60.0 KB, free 38.7 MB) > Exception in thread "stdin writer for python" java.lang.OutOfMemoryError: > Java heap space > at java.io.BufferedOutputStream.<init>(Unknown Source) > at > org.apache.spark.api.python.PythonRDD$$anon$2.run(PythonRDD.scala:62) > 14/07/09 01:46:18 INFO BlockManagerMasterActor$BlockManagerInfo: Added > rdd_2212_0 in memory on shawn-PC:51451 (size: 60. > 0 KB, free: 38.7 MB) > PySpark worker failed with exception: > Traceback (most recent call last): > File "F:\spark-0.9.1\spark-0.9.1\bin\..\/python/pyspark/worker.py", line > 50, in main > split_index = read_int(infile) > File "F:\spark-0.9.1\spark-0.9.1\python\pyspark\serializers.py", line > 328, in read_int > raise EOFError > EOFError > > 14/07/09 01:46:25 INFO BlockManagerMaster: Updated info of block rdd_2212_0 > Exception in thread "stdin writer for python" java.lang.OutOfMemoryError: > Java heap space > PySpark worker failed with exception: > Traceback (most recent call last): > File "F:\spark-0.9.1\spark-0.9.1\bin\..\/python/pyspark/worker.py", line > 50, in main > split_index = read_int(infile) > File "F:\spark-0.9.1\spark-0.9.1\python\pyspark\serializers.py", line > 328, in read_int > raise EOFError > EOFError > > Exception in thread "Executor task launch worker-3" > java.lang.OutOfMemoryError: Java heap space > > Exception: java.lang.OutOfMemoryError thrown from the > UncaughtExceptionHandler in thread "spark-akka.actor.default-dispa > tcher-15" > Exception in thread "Executor task launch worker-1" Exception in thread > "Executor task launch worker-2" java.lang.OutOfM > emoryError: Java heap space > java.lang.OutOfMemoryError: Java heap space > Exception in thread "Executor task launch worker-0" Exception in thread > "Executor task launch worker-5" java.lang.OutOfM > emoryError: Java heap space > java.lang.OutOfMemoryError: Java heap space > 14/07/09 01:46:52 WARN BlockManagerMaster: Error sending message to > BlockManagerMaster in 1 attempts > akka.pattern.AskTimeoutException: > Recipient[Actor[akka://spark/user/BlockManagerMaster#920823400]] had > already been term > inated. > at > akka.pattern.AskableActorRef$.ask$extension(AskSupport.scala:134) > at > org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:161) > at > org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52) > at org.apache.spark.storage.BlockManager.org > $apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97) > > at > org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135) > at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80) > at > akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > 14/07/09 01:46:56 WARN BlockManagerMaster: Error sending message to > BlockManagerMaster in 2 attempts > akka.pattern.AskTimeoutException: > Recipient[Actor[akka://spark/user/BlockManagerMaster#920823400]] had > already been term > inated. > at > akka.pattern.AskableActorRef$.ask$extension(AskSupport.scala:134) > at > org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:161) > at > org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52) > at org.apache.spark.storage.BlockManager.org > $apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97) > > at > org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135) > at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80) > at > akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > 14/07/09 01:46:59 WARN BlockManagerMaster: Error sending message to > BlockManagerMaster in 3 attempts > akka.pattern.AskTimeoutException: > Recipient[Actor[akka://spark/user/BlockManagerMaster#920823400]] had > already been term > inated. > at > akka.pattern.AskableActorRef$.ask$extension(AskSupport.scala:134) > at > org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:161) > at > org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52) > at org.apache.spark.storage.BlockManager.org > $apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97) > > at > org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135) > at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80) > at > akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > Exception in thread "Connection manager future execution context-0" > java.lang.Error: org.apache.spark.SparkException: Er > ror sending message to BlockManagerMaster [message = > HeartBeat(BlockManagerId(<driver>, shawn-PC, 51451, 0))] > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > Caused by: org.apache.spark.SparkException: Error sending message to > BlockManagerMaster [message = HeartBeat(BlockManage > rId(<driver>, shawn-PC, 51451, 0))] > at > org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:176) > at > org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52) > at org.apache.spark.storage.BlockManager.org > $apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97) > > at > org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135) > at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80) > at > akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241) > ... 3 more > Caused by: akka.pattern.AskTimeoutException: > Recipient[Actor[akka://spark/user/BlockManagerMaster#920823400]] had alread > y been terminated. > at > akka.pattern.AskableActorRef$.ask$extension(AskSupport.scala:134) > at > org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:161) > ... 8 more > Traceback (most recent call last): > File ".\naive_bayes_analyser.py", line 110, in <module> > model = NaiveBayes.train(sc.parallelize(array(train_data))) > File > "F:\spark-0.9.1\spark-0.9.1\python\pyspark\mllib\classification.py", line > 122, in train > ans = sc._jvm.PythonMLLibAPI().trainNaiveBayes(dataBytes._jrdd, > lambda_) > File > "F:\spark-0.9.1\spark-0.9.1\bin\..\python\lib\py4j-0.8.1-src.zip\py4j\java_gateway.py", > line 535, in __call__ > File > "F:\spark-0.9.1\spark-0.9.1\bin\..\python\lib\py4j-0.8.1-src.zip\py4j\java_gateway.py", > line 363, in send_command > > File > "F:\spark-0.9.1\spark-0.9.1\bin\..\python\lib\py4j-0.8.1-src.zip\py4j\java_gateway.py", > line 472, in send_command > > File "C:\Python27\lib\socket.py", line 430, in readline > data = recv(1) > KeyboardInterrupt > ^CThe process tried to write to a nonexistent pipe. > ^CTerminate batch job (Y/N)? The process tried to write to a nonexistent > pipe. > > ################## > > Thanks > -- > Rahul K Bhojwani > 3rd Year B.Tech > Computer Science and Engineering > National Institute of Technology, Karnataka > -- Deepak