How to check the liveness of a SparkSession

2023-01-19 Thread Yeachan Park
Hi all, We have a long running PySpark session running on client mode that occasionally dies. We'd like to check whether the session is still alive. One solution we came up with was checking whether the UI is still up, but we were wondering if there's maybe an easier way then that. Maybe

[Spark Standalone Mode] How to read from kerberised HDFS in spark standalone mode

2023-01-19 Thread Bansal, Jaimita
Hi Spark Team, We are facing an issue when trying to read from HDFS via spark running in standalone cluster. The issue comes from the executor node not able to authenticate. It is using auth:SIMPLE when actually we have setup auth as Kerberos. Could you please help in resolving this? Caused

unsubscribe

2023-01-19 Thread 김병찬
제목unsubscribe