Re: StackOverflow Error when run ALS with 100 iterations

Just wanted to add a comment to the Jira ticket but I don't think I have permission to do so, so answering here instead. I am encountering the same issue with a stackOverflow Exception. I would like to point out that there is a localCheckpoint

Re: StackOverflow Error when run ALS with 100 iterations

ALS.setCheckpointInterval was added in Spark 1.3.1. You need to upgrade Spark to use this feature. -Xiangrui On Wed, Apr 22, 2015 at 9:03 PM, amghost wrote: > Hi, would you please how to checkpoint the training set rdd since all things > are done in ALS.train method. > > > > -- > View this messag

Re: StackOverflow Error when run ALS with 100 iterations

Hi, would you please how to checkpoint the training set rdd since all things are done in ALS.train method. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/StackOverflow-Error-when-run-ALS-with-100-iterations-tp4296p22619.html Sent from the Apache Spark User

Re: StackOverflow Error when run ALS with 100 iterations

I'd also say that running for 100 iterations is a waste of resources, as ALS will typically converge pretty quickly, as in within 10-20 iterations. On Wed, Apr 16, 2014 at 3:54 AM, Xiaoli Li wrote: > Thanks a lot for your information. It really helps me. > > > On Tue, Apr 15, 2014 at 7:57 PM, C

Re: StackOverflow Error when run ALS with 100 iterations

Thanks a lot for your information. It really helps me. On Tue, Apr 15, 2014 at 7:57 PM, Cheng Lian wrote: > Probably this JIRA > issuesolves your > problem. When running with large iteration number, the lineage > DAG of ALS becomes very d

Re: StackOverflow Error when run ALS with 100 iterations

Probably this JIRA issuesolves your problem. When running with large iteration number, the lineage DAG of ALS becomes very deep, both DAGScheduler and Java serializer may overflow because they are implemented in a recursive way. You may resort