Hi BR, thanks for your reply. I want to mimic the issue and kill tasks at a certain stage. Killing executor is also an option for me. I'm curious how do core spark contributors test spark fault tolerance?
вс, 10 февр. 2019 г. в 16:57, Gabor Somogyi <gabor.g.somo...@gmail.com>: > Hi Serega, > > If I understand your problem correctly you would like to kill one executor > only and the rest of the app has to be untouched. > If that's true yarn -kill is not what you want because it stops the whole > application. > > I've done similar thing when tested/testing Spark's HA features. > - jps -vlm | grep > "org.apache.spark.executor.CoarseGrainedExecutorBackend.*applicationid" > - kill -9 pidofoneexecutor > > Be aware if it's a multi-node cluster check whether at least one process > runs on a specific node(it's not required). > Happy killing... > > BR, > G > > > On Sun, Feb 10, 2019 at 4:19 PM Jörn Franke <jornfra...@gmail.com> wrote: > >> yarn application -kill applicationid ? >> >> > Am 10.02.2019 um 13:30 schrieb Serega Sheypak <serega.shey...@gmail.com >> >: >> > >> > Hi there! >> > I have weird issue that appears only when tasks fail at specific stage. >> I would like to imitate failure on my own. >> > The plan is to run problematic app and then kill entire executor or >> some tasks when execution reaches certain stage. >> > >> > Is it do-able? >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >>
