Hello Spark Devs, I noticed that [SPARK-3454], which introduces new json endpoints at /api/v1/[path] for information previously only shown on the web UI, does not expose several useful properties about Spark jobs that are exposed on the web UI and on the unofficial /json endpoint.
Specific examples include the maximum number of allotted cores per application, amount of memory allotted to each slave, and number of cores used by each worker. These are provided at ‘app.cores, app.memoryperslave, and worker.coresused’ in the /json endpoint, and also all appear on the web UI page. Is there any specific reason that these fields are not exposed in the public API? If not, would it be reasonable to add them to the json blobs, possibly in a future /api/v2 API? Thank you, Kevin Chen
smime.p7s
Description: S/MIME cryptographic signature
