Re: Debugging Spark AWS S3

2014-05-16 Thread Ian Ferreira
Did you check the executor stderr logs? On 5/16/14, 2:37 PM, "Robert James" wrote: >I have Spark code which runs beautifully when MASTER=local. When I >run it with MASTER set to a spark ec2 cluster, the workers seem to >run, but the results, which are supposed to be put to AWS S3, don't >appear

Debugging Spark AWS S3

2014-05-16 Thread Robert James
I have Spark code which runs beautifully when MASTER=local. When I run it with MASTER set to a spark ec2 cluster, the workers seem to run, but the results, which are supposed to be put to AWS S3, don't appear on S3. I'm at a loss for how to debug this. I don't see any S3 exceptions anywhere. Ca