Hi, I’m trying to do some dynamic scheduling by an external application by looking at the jobs in a Spark framework.
I need the job description to know which kind of query I’m dealing with. The problem is that the job description (set with: sparkCtx.setJobDescription) but in case of a job with multiple stages, the job description is not seen in the UI, and more importantly in my case, is not retrievable in the API endpoint of `host:api/v1/application/x/jobs` Is there a reason why the job description is not shown for non-finished, multi-stage, jobs? Further info: Spark 1.5.1, the jobs for now are sent ad-hoc through the spark-shell. Thanks, Hans -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Job-description-only-visible-after-job-finish-tp16795.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
