Hi All,
When I try to read a list parquet files from S3, my application errors out
if even one of the files are absent. When I searched for solutions most of
them suggested filtering the list of files (on presence) before calling
read.
Shouldn't this be handled by Spark by providing an option for c
You can call `SparkContext#addSparkListener` with a listener that
implements `onApplicationEnd`.
On Tue, May 14, 2019 at 1:51 AM Nasrulla Khan Haris
wrote:
> HI All,
>
>
>
> Is there a API for sparkContext where we can add our custom code before
> stopping sparkcontext ?
>
> Appreciate your help