spark.executor.decommission.gracefulShutdown=true WHAT?
ons. 9. okt. 2024 kl. 12:30 skrev Mich Talebzadeh <mich.talebza...@gmail.com >: > Before responding, what configuration parameters are you using to make > this work? > > spark.executor.decommission.enabled=true > spark.executor.decommission.gracefulShutdown=true > spark.executor.decommission.forceKillTimeout=100s > > HTH > > Mich Talebzadeh, > > Architect | Data Engineer | Data Science | Financial Crime > PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial College > London <https://en.wikipedia.org/wiki/Imperial_College_London> > London, United Kingdom > > > view my Linkedin profile > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > https://en.everybodywiki.com/Mich_Talebzadeh > > > > *Disclaimer:* The information provided is correct to the best of my > knowledge but of course cannot be guaranteed . It is essential to note > that, as with any advice, quote "one test result is worth one-thousand > expert opinions (Werner <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von > Braun <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". > > > On Wed, 9 Oct 2024 at 11:05, Jay Han <tunyu...@gmail.com> wrote: > >> Hi spark community, >> I have such a question: Why driver doesn't shutdown executors >> gracefully on k8s. For instance, >> kubernetesClient.pods().withGracePeriod(100).delete(). >> >> >> -- >> Best, >> Jay >> > -- Bjørn Jørgensen Vestre Aspehaug 4, 6010 Ålesund Norge +47 480 94 297
