For your information, the SparkSubmit runs at the host you executed the spark-submit shell script (which in turns invoke the SparkSubmit program). Since you are running in yarn-cluster mode, the SparkSubmit program just reported the status of the job submitted to Yarn. So when you killed the ApplicationMaster, the SparkSubmit will return "yarnAppState: KILLED" and then terminated itself. This is what happens to me using cdh 5.0.2
Which distribution of hadoop you are using? On Tue, Jul 15, 2014 at 10:42 AM, Jerry Lam <chiling...@gmail.com> wrote: > when I use yarn application -kill, both SparkSubmit and ApplicationMaster > are killed. I also checked jps at the machine that has SparkSubmit running, > it is terminated as well. Sorry, I cannot reproduce it. > > > On Mon, Jul 14, 2014 at 7:36 PM, hsy...@gmail.com <hsy...@gmail.com> > wrote: > >> Before "yarn application -kill" If you do jps You'll have a list >> of SparkSubmit and ApplicationMaster >> >> After you use yarn applicaton -kill you only kill the SparkSubmit >> >> >> >> On Mon, Jul 14, 2014 at 4:29 PM, Jerry Lam <chiling...@gmail.com> wrote: >> >>> Then yarn application -kill appid should work. This is what I did 2 >>> hours ago. >>> >>> Sorry I cannot provide more help. >>> >>> >>> Sent from my iPhone >>> >>> On 14 Jul, 2014, at 6:05 pm, "hsy...@gmail.com" <hsy...@gmail.com> >>> wrote: >>> >>> yarn-cluster >>> >>> >>> On Mon, Jul 14, 2014 at 2:44 PM, Jerry Lam <chiling...@gmail.com> wrote: >>> >>>> Hi Siyuan, >>>> >>>> I wonder if you --master yarn-cluster or yarn-client? >>>> >>>> Best Regards, >>>> >>>> Jerry >>>> >>>> >>>> On Mon, Jul 14, 2014 at 5:08 PM, hsy...@gmail.com <hsy...@gmail.com> >>>> wrote: >>>> >>>>> Hi all, >>>>> >>>>> A newbie question, I start a spark yarn application through >>>>> spark-submit .... >>>>> How do I kill this app. I can kill the yarn app by "yarn application >>>>> -kill appid" but the application master is still running. What's the >>>>> proper >>>>> way to shutdown the entire app? >>>>> >>>>> Best, >>>>> Siyuan >>>>> >>>> >>>> >>> >> >
