Author: sumit
Date: Mon May 4 18:52:17 2015
New Revision: 1677654
URL: http://svn.apache.org/r1677654
Log:
KNOX-481 docs for Stacks and misc fixes
Modified:
knox/site/books/knox-0-6-0/dev-guide.html
knox/site/books/knox-0-6-0/user-guide.html
knox/site/index.html
knox/site/issue-tracking.html
knox/site/license.html
knox/site/mail-lists.html
knox/site/project-info.html
knox/site/team-list.html
knox/trunk/books/0.6.0/book.md
knox/trunk/books/0.6.0/book_service-details.md
knox/trunk/books/0.6.0/dev-guide/book.md
knox/trunk/books/0.6.0/service_storm.md
Modified: knox/site/books/knox-0-6-0/dev-guide.html
URL:
http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/dev-guide.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/books/knox-0-6-0/dev-guide.html (original)
+++ knox/site/books/knox-0-6-0/dev-guide.html Mon May 4 18:52:17 2015
@@ -144,6 +144,10 @@
<td>The implementation of the Access service POC. </td>
</tr>
<tr>
+ <td>gateway-service-definitions </td>
+ <td>The implementation of the Service definition and rewrite files. </td>
+ </tr>
+ <tr>
<td>gateway-service-hbase </td>
<td>The implementation of the HBase service. </td>
</tr>
@@ -160,10 +164,6 @@
<td>The implementation of the Ticket Granting service POC. </td>
</tr>
<tr>
- <td>gateway-service-webhcat </td>
- <td>The implementation of the WebHCat service. </td>
- </tr>
- <tr>
<td>gateway-service-webhdfs </td>
<td>The implementation of the WebHdfs service. </td>
</tr>
@@ -521,6 +521,94 @@ public interface ServiceDeploymentContri
</dependencies>
</project>
+</code></pre><h4><a id="Service+Definition+Files"></a>Service Definition
Files</h4><p>As of release 0.6.0, the gateway now also supports a declarative
way of plugging-in a new Service. A Service can be defined with a combination
of two files, these are:</p>
+<pre><code>service.xml
+rewrite.xml
+</code></pre><p>The rewrite.xml file contains the rewrite rules as defined in
other sections of this guide, and the service.xml file contains the various
routes (paths) to be provided by the Service and the rewrite rule bindings to
those paths. This will be described in further detail in this
section.</p><p>While the service.xml file is absolutely required, the
rewrite.xml file in theory is optional (though it is highly unlikely that no
rewrite rules are needed).</p><p>To add a new service, simply add a service.xml
and rewrite.xml file in an appropriate directory (see <a
href="#Service+Definition+Directory+Structure">Service Definition Directory
Structure</a>) in the module gateway-service-definitions to make the new
service part of the Knox build.</p><h5><a
id="service.xml"></a>service.xml</h5><p>Below is a sample of a very simple
service.xml file, taking the same weather api example.</p>
+<pre><code class="xml"><service role="WEATHER"
name="weather" version="0.1.0">
+ <routes>
+ <route path="/weather/**?**"/>
+ </routes>
+</service>
+
+</code></pre>
+<dl><dt><strong>service</strong></dt><dd>The root tag is ‘service’
that has the three required attributes: <em>role</em>, <em>name</em> and
<em>version</em>. These three values disambiguate this service definition from
others. To ensure the exact same service definition is being used in a topology
file, all values should be specified,</dd>
+</dl>
+<pre><code class="xml"><topology>
+ <gateway>
+ ....
+ </gateway>
+ <service>
+ <role>WEATHER</role>
+ <name>weather</name>
+ <version>0.1.0</version>
+ <url>http://api.openweathermap.org/data</url>
+ </service>
+ ....
+</topology>
+</code></pre><p>If only <em>role</em> is specified in the topology file (the
only required element other than <em>url</em>) then the first service
definition of that role found will be used with the highest version of that
role and name. Similarly if only the <em>version</em> is omitted from the
topology specification of the service, the service definition of the highest
version will be used. It is therefore important to specify a version for a
service if it is desired that a topology be locked down to a specific version
of a service.</p>
+<dl><dt><strong>routes</strong></dt><dd>Wrapper element for one or more
routes.</dd><dt><strong>route</strong></dt><dd>A route specifies the
<em>path</em> that the service is routing as well as any rewrite bindings or
policy bindings. Another child element that may be used here is a
<em>dispatch</em> element.</dd><dt><strong>rewrite</strong></dt><dd>A rewrite
rule or function that is to be applied to the path. A rewrite element contains
a <em>apply</em> attribute that references the rewrite function or rule by
name. Along with the <em>apply</em> attribute, a <em>to</em> attribute must be
used. The <em>to</em> specifies what part of the request or response to
rewrite. The valid values for the <em>to</em> attribute are:</dd>
+</dl>
+<ul>
+ <li>request.url</li>
+ <li>request.headers</li>
+ <li>request.cookies</li>
+ <li>request.body</li>
+ <li>response.headers</li>
+ <li>response.cookies</li>
+ <li>response.body</li>
+</ul><p>Below is an example of a snippet from the WebHDFS service
definition</p>
+<pre><code class="xml"> <route path="/webhdfs/v1/**?**">
+ <rewrite
apply="WEBHDFS/webhdfs/inbound/namenode/file"
to="request.url"/>
+ <rewrite
apply="WEBHDFS/webhdfs/outbound/namenode/headers"
to="response.headers"/>
+ </route>
+</code></pre>
+<dl><dt><strong>dispatch</strong></dt><dd>The dispatch element can be used to
plug-in a custom dispatch class. The interface for Dispatch can be found in the
module gateway-spi, org.apache.hadoop.gateway.dispatch.Dispatch.</dd>
+</dl><p>This element can be used at the service level (i.e. as a child of the
service tag) or at the route level. A dispatch specified at the route level
takes precedence over a dispatch specified at the service level. By default the
dispatch used is org.apache.hadoop.gateway.dispatch.DefaultDispatch.</p><p>The
dispatch tag has four attributes that can be
specified.</p><p><em>contributor-name</em> : This attribute can be used to
specify a deployment contributor to be invoked for a custom
dispatch.</p><p><em>classname</em> : This attribute can be used to specify a
custom dispatch class.</p><p><em>ha-contributor-name</em> : This attribute can
be used to specify a deployment contributor to be invoked for custom HA
dispatch functionality.</p><p><em>ha-classname</em> : This attribute can be
used to specify a custom dispatch class with HA functionality.</p><p>Only one
of contributor-name or classname should be specified and one of
ha-contributor-name or ha-classname should be specified.</
p><p>If providing a custom dispatch, either a jar should be provided, see <a
href="#Class+Path">Class Path</a> or a <a href="#Maven+Module">Maven Module</a>
should be created.</p>
+<dl><dt><strong>policies</strong></dt><dd>This is a wrapper tag for
<em>policy</em> elements and can be a child of the <em>service</em> tag or the
<em>route</em> tag. Once again, just like with dispatch, the route level
policies defined override the ones at the service level.</dd>
+</dl><p>This element can contain one or more <em>policy</em> elements. The
order of the <em>policy</em> elements is important as that will be the order of
execution.</p>
+<dl><dt><strong>policy</strong></dt><dd>At this time the policy element just
has two attributes, <em>role</em> and <em>name</em>. These are used to execute
a deployment contributor by that role and name. Therefore new policies must be
added by using the deployment contributor mechanism.</dd>
+</dl><p>For example,</p>
+<pre><code class="xml"><service role="FOO" name="foo"
version="1.0.0">
+ <policies>
+ <policy role="webappsec"/>
+ <policy role="authentication"/>
+ <policy role="rewrite"/>
+ <policy role="identity-assertion"/>
+ <policy role="authorization"/>
+ </policies>
+ <routes>
+ <route path="/foo/?**">
+ <rewrite apply="FOO/foo/inbound"
to="request.url"/>
+ <policies>
+ <policy role="webappsec"/>
+ <policy role="federation"/>
+ <policy role="identity-assertion"/>
+ <policy role="authorization"/>
+ <policy role="rewrite"/>
+ </policies>
+ <dispatch contributor-name="http-client" />
+ </route>
+ </routes>
+ <dispatch contributor-name="custom-client"
ha-contributor-name="ha-client"/>
+</service>
+</code></pre><h5><a id="rewrite.xml"></a>rewrite.xml</h5><p>The rewrite.xml
file that accompanies the service.xml file follows the same rules as described
in the section <a href="#Rewrite+Provider">Rewrite Provider</a>.</p><h4><a
id="Service+Definition+Directory+Structure"></a>Service Definition Directory
Structure</h4><p>On installation of the Knox gateway, the following directory
structure can be found under ${GATEWAY_HOME}/data. This is a mirror of the
directories and files under the module gateway-service-definitions.</p>
+<pre><code>services
+ |______ service name
+ |______ version
+ |______service.xml
+ |______rewrite.xml
+</code></pre><p>For example,</p>
+<pre><code>services
+ |______ webhdfs
+ |______ 2.4.0
+ |______service.xml
+ |______rewrite.xml
+</code></pre><p>To test out a new service, you can just add the appropriate
files (service.xml and rewrite.xml) in a directory under
${GATEWAY_HOME}/data/services. If you want to make the service contribution to
the Knox build, they files need to go in the gateway-service-definitions
module.</p><h4><a id="Service+Definition+Runtime+Behavior"></a>Service
Definition Runtime Behavior</h4><p>The runtime artifacts as well as the
behavior does not change whether the service is plugged in via the deployment
descriptors or through a service.xml file.</p><h4><a
id="Custom+Dispatch+Dependency+Injection"></a>Custom Dispatch Dependency
Injection</h4><p>When writing a custom dispatch class, one often needs
configuration or gateway services. A lightweight dependency injection system is
used that can inject instances of classes or primitives available in the filter
configuration’s init params or as a servlet context
attribute.</p><p>Details of this can be found in the module gateway-util-con
figinjector and also an example use of it is in the class
org.apache.hadoop.gateway.dispatch.DefaultDispatch. Look at the following
method for example:</p>
+<pre><code class="java"> @Configure
+ protected void setReplayBufferSize(@Default("8") int size) {
+ replayBufferSize = size;
+ }
</code></pre><h3><a id="Providers"></a>Providers</h3>
<pre><code class="java">public interface ProviderDeploymentContributor {
String getRole();
Modified: knox/site/books/knox-0-6-0/user-guide.html
URL:
http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/user-guide.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/books/knox-0-6-0/user-guide.html (original)
+++ knox/site/books/knox-0-6-0/user-guide.html Mon May 4 18:52:17 2015
@@ -50,6 +50,7 @@
<li><a href="#HBase">HBase</a></li>
<li><a href="#Hive">Hive</a></li>
<li><a href="#Yarn">Yarn</a></li>
+ <li><a href="#Storm">Storm</a></li>
</ul></li>
<li><a href="#Limitations">Limitations</a></li>
<li><a href="#Troubleshooting">Troubleshooting</a></li>
@@ -2007,6 +2008,7 @@ dep/commons-codec-1.7.jar
<li><a href="#HBase">HBase</a></li>
<li><a href="#Hive">Hive</a></li>
<li><a href="#Yarn">Yarn</a></li>
+ <li><a href="#Storm">Storm</a></li>
</ul><h3><a id="Assumptions"></a>Assumptions</h3><p>This document assumes a
few things about your environment in order to simplify the examples.</p>
<ul>
<li>The JVM is executable as simply java.</li>
@@ -3343,7 +3345,7 @@ The contents of the state-killed.json fi
curl -ikv -u guest:guest-password -H Content-Type:application/json -X PUT -T
state-killed.json
'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/state'
-</code></pre><h4><a id="Storm"></a>Storm</h4><p>Storm is a distributed
realtime computation system. Storm exposes REST APIs for UI functionality that
can be used for retrieving metrics data and configuration information as well
as management operations such as starting or stopping topologies.</p><p>The
docs for this can be found here</p><p><a
href="https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md";>https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md</a></p><p>To
enable this functionality, a topology file needs to have the following
configuration:</p>
+</code></pre><h3><a id="Storm"></a>Storm</h3><p>Storm is a distributed
realtime computation system. Storm exposes REST APIs for UI functionality that
can be used for retrieving metrics data and configuration information as well
as management operations such as starting or stopping topologies.</p><p>The
docs for this can be found here</p><p><a
href="https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md";>https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md</a></p><p>To
enable this functionality, a topology file needs to have the following
configuration:</p>
<pre><code><service>
<role>STORM</role>
<url>http://<hostname>:<port></url>
@@ -3377,7 +3379,7 @@ curl -ikv -u guest:guest-password -H Con
<td><code>http://{storm-logviewer-host}:{storm-logviewer-port}</code>
</td>
</tr>
</tbody>
-</table><h4><a id="Storm+Examples+via+cURL"></a>Storm Examples via
cURL</h4><p>Some of the various calls that can be made and examples using curl
are listed below.</p>
+</table><h4><a id="Storm+Examples"></a>Storm Examples</h4><p>Some of the
various calls that can be made and examples using curl are listed below.</p>
<pre><code># 0. Getting cluster configuration
curl -ikv -u guest:guest-password -X GET
'https://localhost:8443/gateway/sandbox/storm/api/v1/cluster/configuration'
Modified: knox/site/index.html
URL:
http://svn.apache.org/viewvc/knox/site/index.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-05-01</span>
+ | <span id="publishDate">Last Published:
2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/issue-tracking.html
URL:
http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-05-01</span>
+ | <span id="publishDate">Last Published:
2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/license.html
URL:
http://svn.apache.org/viewvc/knox/site/license.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-05-01</span>
+ | <span id="publishDate">Last Published:
2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/mail-lists.html
URL:
http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-05-01</span>
+ | <span id="publishDate">Last Published:
2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/project-info.html
URL:
http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-05-01</span>
+ | <span id="publishDate">Last Published:
2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/team-list.html
URL:
http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-05-01</span>
+ | <span id="publishDate">Last Published:
2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/trunk/books/0.6.0/book.md
URL:
http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/book.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/book.md (original)
+++ knox/trunk/books/0.6.0/book.md Mon May 4 18:52:17 2015
@@ -55,6 +55,7 @@
* #[HBase]
* #[Hive]
* #[Yarn]
+ * #[Storm]
* #[Limitations]
* #[Troubleshooting]
* #[Export Controls]
Modified: knox/trunk/books/0.6.0/book_service-details.md
URL:
http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/book_service-details.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/book_service-details.md (original)
+++ knox/trunk/books/0.6.0/book_service-details.md Mon May 4 18:52:17 2015
@@ -35,6 +35,7 @@ These are the current Hadoop services wi
* #[HBase]
* #[Hive]
* #[Yarn]
+* #[Storm]
### Assumptions
Modified: knox/trunk/books/0.6.0/dev-guide/book.md
URL:
http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/dev-guide/book.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/dev-guide/book.md (original)
+++ knox/trunk/books/0.6.0/dev-guide/book.md Mon May 4 18:52:17 2015
@@ -116,11 +116,11 @@ This is also true of the other -release
| gateway-provider-security-shiro | Shiro authentiation
integration. |
| gateway-provider-security-webappsec | Filters to prevent common
webapp security issues. |
| gateway-service-as | The implementation of the
Access service POC. |
+| gateway-service-definitions | The implementation of the
Service definition and rewrite files. |
| gateway-service-hbase | The implementation of the
HBase service. |
| gateway-service-hive | The implementation of the
Hive service. |
| gateway-service-oozie | The implementation of the
Oozie service. |
| gateway-service-tgs | The implementation of the
Ticket Granting service POC. |
-| gateway-service-webhcat | The implementation of the
WebHCat service. |
| gateway-service-webhdfs | The implementation of the
WebHdfs service. |
| gateway-server | The implementation of the
Knox gateway server. |
| gateway-shell | The implementation of the
Knox Groovy shell. |
@@ -805,6 +805,202 @@ allRules.addRules( newRules );
</project>
```
+#### Service Definition Files ####
+
+As of release 0.6.0, the gateway now also supports a declarative way of
plugging-in a new Service. A Service can be defined with a
+combination of two files, these are:
+
+ service.xml
+ rewrite.xml
+
+The rewrite.xml file contains the rewrite rules as defined in other sections
of this guide, and the service.xml file contains
+the various routes (paths) to be provided by the Service and the rewrite rule
bindings to those paths. This will be described in further detail
+in this section.
+
+While the service.xml file is absolutely required, the rewrite.xml file in
theory is optional (though it is highly unlikely that no rewrite rules
+are needed).
+
+To add a new service, simply add a service.xml and rewrite.xml file in an
appropriate directory (see #[Service Definition Directory Structure])
+in the module gateway-service-definitions to make the new service part of the
Knox build.
+
+##### service.xml #####
+
+Below is a sample of a very simple service.xml file, taking the same weather
api example.
+
+```xml
+<service role="WEATHER" name="weather" version="0.1.0">
+ <routes>
+ <route path="/weather/**?**"/>
+ </routes>
+</service>
+
+```
+
+**service**
+: The root tag is 'service' that has the three required attributes: *role*,
*name* and *version*. These three values disambiguate this
+service definition from others. To ensure the exact same service definition is
being used in a topology file, all values should be specified,
+
+```xml
+<topology>
+ <gateway>
+ ....
+ </gateway>
+ <service>
+ <role>WEATHER</role>
+ <name>weather</name>
+ <version>0.1.0</version>
+ <url>http://api.openweathermap.org/data</url>
+ </service>
+ ....
+</topology>
+```
+
+If only *role* is specified in the topology file (the only required element
other than *url*) then the first service definition of that role found
+will be used with the highest version of that role and name. Similarly if only
the *version* is omitted from the topology specification of the service,
+the service definition of the highest version will be used. It is therefore
important to specify a version for a service if it is desired that
+a topology be locked down to a specific version of a service.
+
+
+**routes**
+: Wrapper element for one or more routes.
+
+**route**
+: A route specifies the *path* that the service is routing as well as any
rewrite bindings or policy bindings. Another child element that may be used here
+is a *dispatch* element.
+
+**rewrite**
+: A rewrite rule or function that is to be applied to the path. A rewrite
element contains a *apply* attribute that references the rewrite function or
rule
+by name. Along with the *apply* attribute, a *to* attribute must be used. The
*to* specifies what part of the request or response to rewrite.
+The valid values for the *to* attribute are:
+
+- request.url
+- request.headers
+- request.cookies
+- request.body
+- response.headers
+- response.cookies
+- response.body
+
+Below is an example of a snippet from the WebHDFS service definition
+
+```xml
+ <route path="/webhdfs/v1/**?**">
+ <rewrite apply="WEBHDFS/webhdfs/inbound/namenode/file"
to="request.url"/>
+ <rewrite apply="WEBHDFS/webhdfs/outbound/namenode/headers"
to="response.headers"/>
+ </route>
+```
+
+**dispatch**
+: The dispatch element can be used to plug-in a custom dispatch class. The
interface for Dispatch can be found in the module
+gateway-spi, org.apache.hadoop.gateway.dispatch.Dispatch.
+
+This element can be used at the service level (i.e. as a child of the service
tag) or at the route level. A dispatch specified
+at the route level takes precedence over a dispatch specified at the service
level. By default the dispatch used is
+org.apache.hadoop.gateway.dispatch.DefaultDispatch.
+
+The dispatch tag has four attributes that can be specified.
+
+*contributor-name* : This attribute can be used to specify a deployment
contributor to be invoked for a custom dispatch.
+
+*classname* : This attribute can be used to specify a custom dispatch class.
+
+*ha-contributor-name* : This attribute can be used to specify a deployment
contributor to be invoked for custom HA dispatch functionality.
+
+*ha-classname* : This attribute can be used to specify a custom dispatch class
with HA functionality.
+
+Only one of contributor-name or classname should be specified and one of
ha-contributor-name or ha-classname should be specified.
+
+If providing a custom dispatch, either a jar should be provided, see #[Class
Path] or a #[Maven Module] should be created.
+
+**policies**
+: This is a wrapper tag for *policy* elements and can be a child of the
*service* tag or the *route* tag. Once again, just like with
+dispatch, the route level policies defined override the ones at the service
level.
+
+This element can contain one or more *policy* elements. The order of the
*policy* elements is important as that will be the
+order of execution.
+
+**policy**
+: At this time the policy element just has two attributes, *role* and *name*.
These are used to execute a deployment contributor
+by that role and name. Therefore new policies must be added by using the
deployment contributor mechanism.
+
+For example,
+
+```xml
+<service role="FOO" name="foo" version="1.0.0">
+ <policies>
+ <policy role="webappsec"/>
+ <policy role="authentication"/>
+ <policy role="rewrite"/>
+ <policy role="identity-assertion"/>
+ <policy role="authorization"/>
+ </policies>
+ <routes>
+ <route path="/foo/?**">
+ <rewrite apply="FOO/foo/inbound" to="request.url"/>
+ <policies>
+ <policy role="webappsec"/>
+ <policy role="federation"/>
+ <policy role="identity-assertion"/>
+ <policy role="authorization"/>
+ <policy role="rewrite"/>
+ </policies>
+ <dispatch contributor-name="http-client" />
+ </route>
+ </routes>
+ <dispatch contributor-name="custom-client"
ha-contributor-name="ha-client"/>
+</service>
+```
+
+##### rewrite.xml #####
+
+The rewrite.xml file that accompanies the service.xml file follows the same
rules as described in the section #[Rewrite Provider].
+
+#### Service Definition Directory Structure ####
+
+On installation of the Knox gateway, the following directory structure can be
found under ${GATEWAY_HOME}/data. This
+is a mirror of the directories and files under the module
gateway-service-definitions.
+
+ services
+ |______ service name
+ |______ version
+ |______service.xml
+ |______rewrite.xml
+
+
+For example,
+
+ services
+ |______ webhdfs
+ |______ 2.4.0
+ |______service.xml
+ |______rewrite.xml
+
+
+To test out a new service, you can just add the appropriate files (service.xml
and rewrite.xml) in a directory under
+${GATEWAY_HOME}/data/services. If you want to make the service contribution to
the Knox build, they files need to go
+in the gateway-service-definitions module.
+
+#### Service Definition Runtime Behavior ####
+
+The runtime artifacts as well as the behavior does not change whether the
service is plugged in via the deployment
+descriptors or through a service.xml file.
+
+#### Custom Dispatch Dependency Injection ####
+
+When writing a custom dispatch class, one often needs configuration or gateway
services. A lightweight dependency injection
+system is used that can inject instances of classes or primitives available in
the filter configuration's init params or
+as a servlet context attribute.
+
+Details of this can be found in the module gateway-util-configinjector and
also an example use of it is in the class
+org.apache.hadoop.gateway.dispatch.DefaultDispatch. Look at the following
method for example:
+
+```java
+ @Configure
+ protected void setReplayBufferSize(@Default("8") int size) {
+ replayBufferSize = size;
+ }
+```
+
### Providers ###
```java
Modified: knox/trunk/books/0.6.0/service_storm.md
URL:
http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/service_storm.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/service_storm.md (original)
+++ knox/trunk/books/0.6.0/service_storm.md Mon May 4 18:52:17 2015
@@ -15,7 +15,7 @@
limitations under the License.
--->
-#### Storm ###
+### Storm ###
Storm is a distributed realtime computation system. Storm exposes REST APIs
for UI functionality that can be used for
retrieving metrics data and configuration information as well as management
operations such as starting or stopping topologies.
@@ -60,7 +60,7 @@ For the log viewer the mapping is as fol
| Cluster | `http://{storm-logviewer-host}:{storm-logviewer-port}`
|
-#### Storm Examples via cURL
+#### Storm Examples
Some of the various calls that can be made and examples using curl are listed
below.
