Re: StackOverflow when computing MatrixFactorizationModel.recommendProductsForUsers

2016-01-22 Thread Ram VISWANADHA
Any help? Not sure what I am doing wrong. Best Regards, Ram From: Ram VISWANADHA <ram.viswana...@dailymotion.com<mailto:ram.viswana...@dailymotion.com>> Date: Friday, January 22, 2016 at 10:25 AM To: user <user@spark.apache.org<mailto:user@spark.apache.org>> Sub

StackOverflow when computing MatrixFactorizationModel.recommendProductsForUsers

2016-01-22 Thread Ram VISWANADHA
Hi, I am getting this StackOverflowError when fetching recommendations from ALS. Any help is much appreciated int features = 100; double alpha = 0.1; double lambda = 0.001; boolean implicit = true; int iterations = 10; ALS als = new ALS()

Re: Improve saveAsTextFile performance

2015-12-05 Thread Ram VISWANADHA
0 8 3.9 MB / 95334 Best Regards, Ram From: Akhil Das <ak...@sigmoidanalytics.com<mailto:ak...@sigmoidanalytics.com>> Date: Saturday, December 5, 2015 at 1:32 AM To: Ram VISWANADHA <ram.viswana...@dailymotion.com<mailto:ram.viswana...@dailymotion.com>> C

Re: Improve saveAsTextFile performance

2015-12-05 Thread Ram VISWANADHA
Regards, Ram From: Akhil Das <ak...@sigmoidanalytics.com<mailto:ak...@sigmoidanalytics.com>> Date: Saturday, December 5, 2015 at 1:32 AM To: Ram VISWANADHA <ram.viswana...@dailymotion.com<mailto:ram.viswana...@dailymotion.com>> Cc: user <user@spark.apache.org<mailto

Re: Improve saveAsTextFile performance

2015-12-04 Thread Ram VISWANADHA
at 10:18 PM To: Ram VISWANADHA <ram.viswana...@dailymotion.com<mailto:ram.viswana...@dailymotion.com>> Cc: Ted Yu <yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>>, user <user@spark.apache.org<mailto:user@spark.apache.org>> Subject: Re: Improve saveAsTextFile

Re: Improve saveAsTextFile performance

2015-12-02 Thread Ram VISWANADHA
Yes. That did not help. Best Regards, Ram From: Ted Yu <yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>> Date: Wednesday, December 2, 2015 at 3:25 PM To: Ram VISWANADHA <ram.viswana...@dailymotion.com<mailto:ram.viswana...@dailymotion.com>> Cc: user <user@s

Improve saveAsTextFile performance

2015-12-02 Thread Ram VISWANADHA
JavaRDD.saveAsTextFile is taking a long time to succeed. There are 10 tasks, the first 9 complete in a reasonable time but the last task is taking a long time to complete. The last task contains the maximum number of records like 90% of the total number of records. Is there any way to

Save to paquet files failed

2015-10-22 Thread Ram VISWANADHA
Hi , I am trying to load 931MB file into an RDD, then create a DataFrame and store the data in a Parquet file. The save method of Parquet file is hanging. I have set the timeout to 1800 but still the system fails to respond and hangs. I can’t spot any errors in my code. Can someone help me?