if you are deploying your spark application on YARN cluster,
1. ssh into master node
2. List the currently running application and retreive the application_id
yarn application --list
3. Kill the application using application_id of the form
application_xxxxx_xxxx from output of list command
yarn application --kill <application_id>
On Wed, Dec 5, 2018 at 1:42 PM kant kodali <[email protected]> wrote:
> Hi All,
>
> How to track batch jobs in spark? For example, is there some id or token i
> can get after I spawn a batch job and use it to track the progress or to
> kill the batch job itself?
>
> For Streaming, we have StreamingQuery.id()
>
> Thanks!
>
