Thanks for your reply.
Yes, Its netflix dataset. And when I get no space on device, my ‘/mnt’
directory gets filled up. I checked.
/usr/lib/spark/bin/spark-submit --deploy-mode cluster --master yarn --class
/usr/lib/spark/examples/jars/spark-examples_2.11-2.0.0.jar --rank 32
--numIterations 100 --kryo s3://dataset_netflix
When I run above command, I get following error
Job aborted due to stage failure: Task 221 in stage 53.0 failed 4 times, most
recent failure: Lost task 221.3 in stage 53.0 (TID 9817, ):
(No space left on device)
I think I should not need to increase the space on device, as data is not that
big. So, is there any way, I can setup parameters so that it does not use much
disk space. I don’t know much about tuning parameters.
It will be great if anyone can help me with this.
> On Sep 16, 2016, at 9:18 AM, Sean Owen <so...@cloudera.com> wrote: