Yong Zhang wrote:
> Can't you just catch that exception and return an empty dataframe?
>
>
> Yong
>
>
> --
> *From:* Sumona Routh
> *Sent:* Wednesday, July 12, 2017 4:36 PM
> *To:* user
> *Subject:* DataFrameReader read from S3
>
Can't you just catch that exception and return an empty dataframe?
Yong
From: Sumona Routh
Sent: Wednesday, July 12, 2017 4:36 PM
To: user
Subject: DataFrameReader read from S3 org.apache.spark.sql.AnalysisException:
Path does not exist
Hi there,
I
Hi there,
I'm trying to read a list of paths from S3 into a dataframe for a window of
time using the following:
sparkSession.read.parquet(listOfPaths:_*)
In some cases, the path may not be there because there is no data, which is
an acceptable scenario.
However, Spark throws an AnalysisException: