Gao
, dev , user
Subject: Re: [External Sender] Re: Spark 2.4.1 on Kubernetes - DNS resolution
of driver fails
Hi!
I am assuming you’re running it in cluster mode,
Service should be created by the submit binary, in this file:
org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala
uot;
Date: Tuesday 18 June 2019 at 16:15
To: Jose Luis Pedrosa
Cc: Olivier Girardot , Li Gao
, dev , user
Subject: Re: [External Sender] Re: Spark 2.4.1 on Kubernetes - DNS resolution
of driver fails
Thanks for the response Oliver.
I am facing this issue intermittently, once in a while i
/apache/spark/pull/24702
2. Disable negative DNS caching at JVM level, on the entrypoint.sh
JL
From: Olivier Girardot
Date: Tuesday 18 June 2019 at 10:06
To: "Prudhvi Chennuru (CONT)"
Cc: Li Gao , dev , user
Subject: Re: [External Sender] Re: Spark 2.4.1 on Kubernetes - DNS reso
Hi Prudhvi,
not really but we took a drastic approach mitigating this, modifying the
bundled launch script to be more resilient.
In the kubernetes/dockerfiles/spark/entrypoint.sh in the executor case we
added something like that :
executor)
DRIVER_HOST=$(echo $SPARK_DRIVER_URL | cut -d "@"
