[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289633#comment-14289633 ]
Sean Owen commented on SPARK-5386: ---------------------------------- You are allocating 8G for executors? or just the workers? standalone mode? Someone who knows a little more might be able to confirm or deny, but I think you're hitting trouble allocating such a large chunk of memory at once. It may be that there is enough heap but not all in one place, and making a huge dense vector means allocating a huge array of doubles. Or it could simply be a really abrupt out-of-memory condition, because in fact it's holding several of these vectors in memory at once and running out. > Reduce fails with vectors of big length > --------------------------------------- > > Key: SPARK-5386 > URL: https://issues.apache.org/jira/browse/SPARK-5386 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.2.0 > Environment: Overall: > 6 machine cluster (Xeon 3.3GHz 4 cores, 16GB RAM, Ubuntu), each runs 2 Workers > Spark: > ./spark-shell --executor-memory 8G --driver-memory 8G > spark.driver.maxResultSize 0 > "java.io.tmpdir" and "spark.local.dir" set to a disk with a lot of free space > Reporter: Alexander Ulanov > Fix For: 1.3.0 > > > Code: > import org.apache.spark.mllib.rdd.RDDFunctions._ > import breeze.linalg._ > import org.apache.log4j._ > Logger.getRootLogger.setLevel(Level.OFF) > val n = 60000000 > val p = 12 > val vv = sc.parallelize(0 until p, p).map(i => DenseVector.rand[Double]( n )) > vv.reduce(_ + _) > When executing in shell it crashes after some period of time. One of the node > contain the following in stdout: > Java HotSpot(TM) 64-Bit Server VM warning: INFO: > os::commit_memory(0x0000000755500000, 2863661056, 0) failed; error='Cannot > allocate memory' (errno=12) > # > # There is insufficient memory for the Java Runtime Environment to continue. > # Native memory allocation (malloc) failed to allocate 2863661056 bytes for > committing reserved memory. > # An error report file with more information is saved as: > # /datac/spark/app-20150123091936-0000/89/hs_err_pid2247.log > During the execution there is a message: Job aborted due to stage failure: > Exception while getting task result: java.io.IOException: Connection from > server-12.net/10.10.10.10:54701 closed -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org