Hi, Does the delegator map task of SparkLauncher need to stay alive until Spark job finishes ?
1) Currently, I have mapper tasks, which launches Spark job via SparkLauncer#startApplication() Does my map task need to wait until Spark job finishes ? Or is there any way, my map task finishes after launching Spark job, and I can still query and get status of Spark job outside of map task (or failure reason, if it has failed) ? (maybe by querying Spark job id ?) 2) I guess also if i want my Spark job to be killed, if corresponding delegator map task is killed, that means my map task needs to stay alive, so i still have SparkAppHandle reference ? Thanks.
