Author: stevel
Date: Thu Dec 11 15:17:06 2014
New Revision: 1644671
URL: http://svn.apache.org/r1644671
Log:
SLIDER-708 Document existing (read-only) REST APIs
Added:
incubator/slider/site/trunk/content/docs/api/
incubator/slider/site/trunk/content/docs/api/slider_REST_apis.md
Modified:
incubator/slider/site/trunk/content/index.md
Added: incubator/slider/site/trunk/content/docs/api/slider_REST_apis.md
URL:
http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/api/slider_REST_apis.md?rev=1644671&view=auto
==============================================================================
--- incubator/slider/site/trunk/content/docs/api/slider_REST_apis.md (added)
+++ incubator/slider/site/trunk/content/docs/api/slider_REST_apis.md Thu Dec 11
15:17:06 2014
@@ -0,0 +1,206 @@
+# Slider 0.60 REST APIs
+
+## December 2014
+
+This document covers the external REST APIs of Slider
+
+ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
+ NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
+ "OPTIONAL" in this document are to be interpreted as described in
+ RFC 2119.
+
+# Introduction and Background
+
+Slider 0.60, the version shipping with HDP2.2 uses Hadoop IPC for
communications between the Slider client and the per-instance application
master. This uses protobuf-formatted payloads, with the Hadoop IPC layer
handling security: authorization, authentication and encryption.
+
+It provides read-only views of generated content and state through a set of
web service URLs. All these services are implemented by the Jersey JAX-RS
engine, operating in an embedded Jetty Web engine, with the YARN AmWebFilter
class redirecting all requests not coming from the RM Proxy IP address to that
RM Proxy via a 302 (redirect-as-GET)
+
+Excluding the internal-use-only the AM<->Agent channel operating on a private
HTTP port, there are a number of current services
+
+# Read only access to the YARN registry: /ws/v1/slider/registry
+
+This was done primarily to get a browseable view of what the YARN-913 registry
was publishing in the absence of any web view built in to that registry. The
unit tests against this service actually discovered some JSON marshalling
problems with the registry design and Jersey integration.
+
+The root path `/v1/slider/registry` lists the root path of the registry; all
entries below it follow the path in the registry.
+
+ /ws/v1/slider/registry: {"nodes":["services","users"]}
+
+
+
+ Every entry in the path serves an instance of
`org.apache.slider.server.appmaster.web.rest.registry.PathEntryResource`; a
list of child directory entries, and, if present, the Yarn registry service
record at that path (`org.apache.hadoop.registry.client.types.ServiceRecord`).
+
+
/ws/v1/slider/registry/users/stevel/services/org-apache-slider/apps-through-agent
:
+
+ {
+ "nodes": ["components"],
+ "service": {
+ "type": "JSONServiceRecord",
+ "description": "Slider Application Master",
+ "external": [
+ {
+ "api": "http://";,
+ "addressType": "uri",
+ "protocolType": "webui",
+ "addresses": [{"uri": "http://devix.cotham.uk:37283"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.management",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri":
"http://devix.cotham.uk:37283/ws/v1/slider/mgmt"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.publisher",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri":
"http://devix.cotham.uk:37283/ws/v1/slider/publisher"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.registry",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri":
"http://devix.cotham.uk:37283/ws/v1/slider/registry"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.publisher.configurations",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri":
"http://devix.cotham.uk:37283/ws/v1/slider/publisher/slider"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.publisher.exports",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri":
"http://devix.cotham.uk:37283/ws/v1/slider/publisher/exports"}]
+ }
+ ],
+ "internal": [
+ {
+ "api": "classpath:org.apache.slider.agents.secure",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri":
"https://devix.cotham.uk:44586/ws/v1/slider/agents"}]
+ },
+ {
+ "api": "classpath:org.apache.slider.agents.oneway",
+ "addressType": "uri",
+ "protocolType": "REST",
+ "addresses": [{"uri":
"https://devix.cotham.uk:36590/ws/v1/slider/agents"}]
+ }
+ ],
+ "yarn:persistence": "application",
+ "yarn:id": "application_1418130391694_0002"
+ }
+ }
+
+The slider code makes no direct use of this registry service except for a pair
of tests. The Agents MUST use the core (ZK-based) registry service to locate a
restarted AM. The slider client operations to enumerate and retrieve
configuration files could use it, but they currently also use the YARN registry
directly.
+
+As and when the YARN registry itself implements a REST view (possibly an R/W
one), then this view will become obsolete âand possibly disabled.
+
+# Generated configurations: `/ws/v1/slider/publisher`
+
+This service contains a one or more "published configuration sets"
+
+
http://devix.cotham.uk:8088/proxy/application_1418130391694_0002/ws/v1/slider/publisher
+
+ {
+ "uris": {
+ "slider":"http://devix.cotham.uk:37283/ws/v1/slider/publisher/slider";,
+ "exports":"http://devix.cotham.uk:37283/ws/v1/slider/publisher/exports";
+ }
+ }
+
+each one contains a list of available configuration documents , such as under
/ws/v1/slider/publisher/slider:
+
+ {
+ "configurations":{
+ "core-site":{
+ "description":"Core site settings",
+ "updated":0,
+ "entries":{},"
+ empty":true
+ }
+ }
+ }
+
+The final path is built from the published configuration *and* the desired
file format,the latter as a suffix. Thus the following links all server the
same content, merely in different formats
+
+ /ws/v1/slider/publisher/slider/core-site.xml
+ /ws/v1/slider/publisher/slider/core-site.properties
+ /ws/v1/slider/publisher/slider/core-site.json
+
+The path for these becomes:
+
+ /ws/v1/slider/publisher/${set-name}/{configuration}.{format}
+
+The value of individual properties can be retrieved as JSON entries under the
path
+
+ /ws/v1/slider/publisher/${set-name}/{configuration}/{key}
+
+Thus the response to a GET of
`/ws/v1/slider/publisher/slider/core-site/fs.defaultFS`
+
+is
+
+ {"fs.defaultFS":"hdfs://devix.cotham.uk:8020"}
+
+Slider uses this service to publish its configuration (`core-site.xml`), etc,
for diagnostics âand the configurations of the deployed applications on the
command line
+
+ slider registry --getconf hbase-site.xml --name hbase1 --destdir out
+
+This configuration publishing service is listed in the YARN Registry, and
could be implemented by other applications to publish their configurations
âincluding the standard YARN services.
+
+# Application state in slider v1 format
+
+The original "v1" formatted slider specification/state is provided, wrapped in
HTML, at:
+
+
http://devix.cotham.uk:8088/proxy/application_1418130391694_0002/slideram/spec
+
+This combines the declaration of what is desired with information about what
has actually been provided
+
+(this shows the merits of a single file model which can describe both the
desired state and the actual state; the same architecture of [the HPL Cells
cloud
infrastructure](http://www.hpl.hp.com/techreports/2012/HPL-2012-158R1.pdf))
+
+# Slider 0.70: codahale metrics servlets
+
+Slider 0.70 is adding metrics via the Codahale metrics library; as well as
(optionally) publishing to Ganglia, the metrics service export metrics,health
and liveness information, as well as a thread dump
+
+<table>
+ <tr>
+ <td>path</td>
+ <td>content</td>
+ </tr>
+ <tr>
+ <td>/system/metrics</td>
+ <td>Metrics information, available in JSON if &format=json
appended</td>
+ </tr>
+ <tr>
+ <td>/system/health</td>
+ <td>health information from registered "health" classes</td>
+ </tr>
+ <tr>
+ <td>/system/ping</td>
+ <td>minimal liveness probe</td>
+ </tr>
+ <tr>
+ <td>/system/threads</td>
+ <td>thread dump</td>
+ </tr>
+</table>
+
+
+ /system/metrics :
+
+ {
+ "version": "3.0.0",
+ "gauges": {},
+ "counters": {},
+ "histograms": {},
+ "meters": {},
+ "timers": {}
+ }
+
+ /system/health :
+
+ {"AM Health": {"healthy": true}}
+
+As metrics are added, this content will start to be meaningful.
+
Modified: incubator/slider/site/trunk/content/index.md
URL:
http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/index.md?rev=1644671&r1=1644670&r2=1644671&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/index.md (original)
+++ incubator/slider/site/trunk/content/index.md Thu Dec 11 15:17:06 2014
@@ -75,12 +75,13 @@ code changes and as of this writing, it
* [Application configuration model](/docs/configuration/index.html)
* [High Availability](/docs/high_availability.html)
* [Troubleshooting](/docs/troubleshooting.html)
+* [REST API](/docs/api/index.html)
## Developing
* [Building](/developing/building.html)
-* [Debugging](/docs/debugging.html)
+* [Debugging](/developing/debugging.html)
* [Testing](/developing/testing.html)
* [Functional Testing](/developing/functional_tests.html)
* [Manual Testing](/developing/manual_testing.html)
