Mich, Can you please share with the list where exactly you are citing these configs from?
As far as I can tell, these two configs don’t exist and have never existed in the Spark codebase: spark.executor.decommission.enabled=true spark.executor.decommission.gracefulShutdown=true Where exactly are you getting this information from (and then posting it to the list as advice)? Please be clear and provide specific references. Nick > On Oct 9, 2024, at 1:20 PM, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > > Before responding, what configuration parameters are you using to make this > work? > > spark.executor.decommission.enabled=true > spark.executor.decommission.gracefulShutdown=true > spark.executor.decommission.forceKillTimeout=100s > > HTH > > Mich Talebzadeh, > > Architect | Data Engineer | Data Science | Financial Crime > PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial College > London <https://en.wikipedia.org/wiki/Imperial_College_London> > London, United Kingdom > > view my Linkedin profile > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > https://en.everybodywiki.com/Mich_Talebzadeh > > > Disclaimer: The information provided is correct to the best of my knowledge > but of course cannot be guaranteed . It is essential to note that, as with > any advice, quote "one test result is worth one-thousand expert opinions > (Werner <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun > <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". > > > On Wed, 9 Oct 2024 at 11:05, Jay Han <tunyu...@gmail.com > <mailto:tunyu...@gmail.com>> wrote: >> Hi spark community, >> I have such a question: Why driver doesn't shutdown executors >> gracefully on k8s. For instance, >> kubernetesClient.pods().withGracePeriod(100).delete(). >> >> >> -- >> Best, >> Jay
