GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/4435
[SPARK-3454] [WIP] separate json endpoints for data in the UI
Exposes data available in the UI as json over http. There are some TODOs
and the code needs cleanup, but there is enough here to get feedback on the
overall direction I think. Key points:
* new set of public case classes for what is represented in the json. Lots
of boilerplate code to copy from other internal representations into these
ones, but I figure its best to keep the public versions complete separate.
* Though the code to serve the json no longer lives in the XyzPage classes,
the json is still tied closely to the UI. Its basically pulling data from the
various `UIData` classes and massaging it a little.
* json is all constructed by jackson, no manual creation of the json ASTs
via json4s.
* there are a bunch of new files in
core/src/test/resources/HistoryServerExpectations with the produced json --
both as examples of what is generated and tests to ensure it doesn't change
going forward.
* instead of making a new servlethandler per endpoint, as in the page
classes, its all just one servlet handler which has some simple logic to
dispatch the requests, see `JsonRequestHandler`
If this direction seems alright it should be straightforward to fill in the
missing bits.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark SPARK-3454
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4435.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4435
----
commit 4b398d0588e25b9861de09e0b31ac99aaca3346e
Author: Imran Rashid <[email protected]>
Date: 2015-02-06T20:53:21Z
expose UI data as json in new endpoints
commit d1a8c9253f53541bfc95d6d2df16ad8e176c131d
Author: Imran Rashid <[email protected]>
Date: 2015-02-06T20:53:50Z
add sbt-revolved plugin, to make it easier to start & stop http servers in
sbt
commit b252e7a3752e70de1e986faba159d0a0087480cd
Author: Imran Rashid <[email protected]>
Date: 2015-02-06T21:30:21Z
small cleanup of accidental changes
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
