Spark ShutdownHook through python jobs.

2020-08-13 Thread Shriraj Bhardwaj
to shutdown hook in java/scala, can somebody help how to do it ? I couldn't fine the shutdown hook interface in pyspark ? -- Regards, Shriraj Bhardwaj *कर्मण्येवाधिकारस्ते मा फलेषु कदाचन। मा कर्मफलहेतुर्भूर्मा ते सङ्गोऽस्त्वकर्मणि॥ *

Re: Spark Job Fails with Unknown Error writing to S3 from AWS EMR

2020-07-22 Thread Shriraj Bhardwaj
We faced this similar situation with jre 8u262 try reverting back... On Thu, Jul 23, 2020, 5:18 AM koti reddy wrote: > Hi, > > Can someone help to resolve this issue? > Thank you in advance. > > Error logs : > > java.io.EOFException: Unexpected EOF while trying to read response from server >