Hi Kevin, I think it would be great if you added this. It never got added in the first place b/c the original PR was already pretty bloated, and just never got back to this. I agree with Reynold -- you shouldn't need to increase the version for just adding new endpoints (or even adding new fields to existing endpoints). See the guarantees we make here:
http://spark.apache.org/docs/latest/monitoring.html#rest-api (Though if you think we should make different guarantees around versions, that would be worth discussing as well.) Can you file a jira, and we move discussion there? Please cc me, and maybe also Josh Rosen (I'm not sure if he has cycles now but he's been very helpful on these issues in the past). thanks, Imran On Wed, Sep 16, 2015 at 9:10 PM, Kevin Chen <kc...@palantir.com> wrote: > Just wanted to bring this email up again in case there were any thoughts. > Having all the information from the web UI accessible through a supported > json API is very important to us; are there any objections to us adding a > v2 API to Spark? > > Thanks! > > From: Kevin Chen <kc...@palantir.com> > Date: Friday, September 11, 2015 at 11:30 AM > To: "dev@spark.apache.org" <dev@spark.apache.org> > Cc: Matt Cheah <mch...@palantir.com>, Mingyu Kim <m...@palantir.com> > Subject: New Spark json endpoints > > Hello Spark Devs, > > I noticed that [SPARK-3454], which introduces new json endpoints at > /api/v1/[path] for information previously only shown on the web UI, does > not expose several useful properties about Spark jobs that are exposed on > the web UI and on the unofficial /json endpoint. > > Specific examples include the maximum number of allotted cores per > application, amount of memory allotted to each slave, and number of cores > used by each worker. These are provided at ‘app.cores, app.memoryperslave, > and worker.coresused’ in the /json endpoint, and also all appear on the web > UI page. > > Is there any specific reason that these fields are not exposed in the > public API? If not, would it be reasonable to add them to the json blobs, > possibly in a future /api/v2 API? > > Thank you, > Kevin Chen > >
