Re: Problem in persisting file in S3 using Spark: xxx file does not exist Exception

Hi Sorted ..I just replaced s3 with s3aI think I recall similar issues in the past with aws libraries. Thx anyway for getting back Kr On Wed, May 2, 2018, 4:57 PM Paul Tremblay wrote: > I would like to see the full error. However, S3 can give misleading > messages

Re: Problem in persisting file in S3 using Spark: xxx file does not exist Exception

I would like to see the full error. However, S3 can give misleading messages if you don't have the correct permissions. On Tue, Apr 24, 2018, 2:28 PM Marco Mistroni wrote: > HI all > i am using the following code for persisting data into S3 (aws keys are > already stored

Problem in persisting file in S3 using Spark: xxx file does not exist Exception

HI all i am using the following code for persisting data into S3 (aws keys are already stored in the environment variables) dataFrame.coalesce(1).write.format("com.databricks.spark.csv").save(fileName) However, i keep on receiving an exception that the file does not exist here's what comes