Hi Jay,
Please try increasing executor memory (if the available memory is more
than 2GB) and reduce numBlocks in ALS. The current implementation
stores all subproblems in memory and hence the memory requirement is
significant when k is large. You can also try reducing k and see
whether the
I am not sure this can help you. I have 57 million rating,about 4million user
and 4k items. I used 7-14 total-executor-cores,executal-memory 13g,cluster
have 4 nodes,each have 4cores,max memory 16g.
I found set as follows may help avoid this problem:
Hi,How many clients and how many products do you have?CheersGen
jaykatukuri wrote
Hi all,I am running into an out of memory error while running ALS using
MLLIB on a reasonably small data set consisting of around 6 Million
ratings.The stack trace is below:java.lang.OutOfMemoryError: Java heap
How many working nodes do these 100 executors locate at?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/MLLib-ALS-java-lang-OutOfMemoryError-Java-heap-space-tp20584p20610.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
