Author: sumit
Date: Thu Apr 16 15:30:53 2015
New Revision: 1674094
URL: http://svn.apache.org/r1674094
Log:
KNOX-530 oozie docs for Webhdfs HA
Modified:
knox/site/books/knox-0-6-0/user-guide.html
knox/site/index.html
knox/site/issue-tracking.html
knox/site/license.html
knox/site/mail-lists.html
knox/site/project-info.html
knox/site/team-list.html
knox/trunk/books/0.6.0/config_ldap_authc_cache.md
knox/trunk/books/0.6.0/service_oozie.md
Modified: knox/site/books/knox-0-6-0/user-guide.html
URL:
http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/user-guide.html?rev=1674094&r1=1674093&r2=1674094&view=diff
==============================================================================
--- knox/site/books/knox-0-6-0/user-guide.html (original)
+++ knox/site/books/knox-0-6-0/user-guide.html Thu Apr 16 15:30:53 2015
@@ -941,7 +941,7 @@ ldapRealm.userDnTemplate=uid={0},ou=peop
<value>authcBasic</value>
</param>
</provider>
-</code></pre><h3><a id="Trying+out+caching"></a>Trying out caching</h3><p>Knox
bundles a template topology files that can be used to try out the caching
functionality. The template file located under {GATEWAY_HOME}/templates is
sandbox.knoxrealm.ehcache.xml.</p><p>To try this out</p><p>cd {GATEWAY_HOME} cp
templates/sandbox.knoxrealm.ehcache.xml deployments/sandbox.xml bin/ldap.sh
start bin/gateway.sh start</p><p>The following call to WebHDFS should report:
{“Path”:“/user/sam”}</p><p>curl -i -v -k -u
guest:guest-password -X GET <a
href="https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY";>https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY</a></p><p>In
order to see the cache working, LDAP can now be shutdown and the user will
still authenticate successfully.</p><p>bin/ldap.sh stop</p><p>and then the
following should still return successfully like it did earlier.</p><p>curl -i
-v -k -u guest:guest-password -X GET <a href="ht
tps://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY">https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY</a></p><h4><a
id="Advanced+Caching+Config"></a>Advanced Caching Config</h4><p>By default the
ehcache support in shiro contains a ehcache.xml in its classpath which is the
following</p>
+</code></pre><h3><a id="Trying+out+caching"></a>Trying out caching</h3><p>Knox
bundles a template topology files that can be used to try out the caching
functionality. The template file located under {GATEWAY_HOME}/templates is
sandbox.knoxrealm.ehcache.xml.</p><p>To try this out</p><p>cd {GATEWAY_HOME} cp
templates/sandbox.knoxrealm.ehcache.xml conf/topologies/sandbox.xml bin/ldap.sh
start bin/gateway.sh start</p><p>The following call to WebHDFS should report:
{“Path”:“/user/guest”}</p><p>curl -i -v -k -u
guest:guest-password -X GET <a
href="https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY";>https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY</a></p><p>In
order to see the cache working, LDAP can now be shutdown and the user will
still authenticate successfully.</p><p>bin/ldap.sh stop</p><p>and then the
following should still return successfully like it did earlier.</p><p>curl -i
-v -k -u guest:guest-password -X GET <a hr
ef="https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY";>https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY</a></p><h4><a
id="Advanced+Caching+Config"></a>Advanced Caching Config</h4><p>By default the
ehcache support in shiro contains a ehcache.xml in its classpath which is the
following</p>
<pre><code><ehcache>
<!-- Sets the path to the directory where cache .data files are created.
@@ -2337,7 +2337,12 @@ session.shutdown()
<name>dfs.namenode.rpc-address</name>
<value>sandbox.hortonworks.com:8020</value>
</property>
-</code></pre><p>The information above must be provided to the gateway via a
topology descriptor file. These topology descriptor files are placed in
<code>{GATEWAY_HOME}/deployments</code>. An example that is setup for the
default configuration of the Sandbox is {GATEWAY_HOME}/deployments/sandbox.xml.
These values will need to be changed for non-default Sandbox or other Hadoop
cluster configuration.</p>
+</code></pre><p>If HDFS has been configured to be in High Availability mode
(HA), then instead of the RPC address mentioned above for the Name Node, look
up and use the logical name of the service found via dfs.nameservices in
hdfs-site.xml. For example,</p>
+<pre><code><property>
+ <name>dfs.nameservices</name>
+ <value>ha-service</value>
+</property>
+</code></pre><p>Please note, only one of the URL’s, either the RPC
endpoint or the HA service name should be used as the NAMENODE hdfs URL in the
gateway topology file.</p><p>The information above must be provided to the
gateway via a topology descriptor file. These topology descriptor files are
placed in <code>{GATEWAY_HOME}/deployments</code>. An example that is setup for
the default configuration of the Sandbox is
{GATEWAY_HOME}/deployments/sandbox.xml. These values will need to be changed
for non-default Sandbox or other Hadoop cluster configuration.</p>
<pre><code><service>
<role>NAMENODE</role>
<url>hdfs://localhost:8020</url>
Modified: knox/site/index.html
URL:
http://svn.apache.org/viewvc/knox/site/index.html?rev=1674094&r1=1674093&r2=1674094&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Thu Apr 16 15:30:53 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150407" />
+ <meta name="Date-Revision-yyyymmdd" content="20150416" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-04-07</span>
+ | <span id="publishDate">Last Published:
2015-04-16</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/issue-tracking.html
URL:
http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1674094&r1=1674093&r2=1674094&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Thu Apr 16 15:30:53 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150407" />
+ <meta name="Date-Revision-yyyymmdd" content="20150416" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-04-07</span>
+ | <span id="publishDate">Last Published:
2015-04-16</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/license.html
URL:
http://svn.apache.org/viewvc/knox/site/license.html?rev=1674094&r1=1674093&r2=1674094&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Thu Apr 16 15:30:53 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150407" />
+ <meta name="Date-Revision-yyyymmdd" content="20150416" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-04-07</span>
+ | <span id="publishDate">Last Published:
2015-04-16</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/mail-lists.html
URL:
http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1674094&r1=1674093&r2=1674094&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Thu Apr 16 15:30:53 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150407" />
+ <meta name="Date-Revision-yyyymmdd" content="20150416" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-04-07</span>
+ | <span id="publishDate">Last Published:
2015-04-16</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/project-info.html
URL:
http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1674094&r1=1674093&r2=1674094&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Thu Apr 16 15:30:53 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150407" />
+ <meta name="Date-Revision-yyyymmdd" content="20150416" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-04-07</span>
+ | <span id="publishDate">Last Published:
2015-04-16</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/team-list.html
URL:
http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1674094&r1=1674093&r2=1674094&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Thu Apr 16 15:30:53 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 -->
<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css"
media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150407" />
+ <meta name="Date-Revision-yyyymmdd" content="20150416" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a
href="https://cwiki.apache.org/confluence/display/KNOX/Index";
class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published:
2015-04-07</span>
+ | <span id="publishDate">Last Published:
2015-04-16</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/trunk/books/0.6.0/config_ldap_authc_cache.md
URL:
http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/config_ldap_authc_cache.md?rev=1674094&r1=1674093&r2=1674094&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/config_ldap_authc_cache.md (original)
+++ knox/trunk/books/0.6.0/config_ldap_authc_cache.md Thu Apr 16 15:30:53 2015
@@ -98,11 +98,11 @@ The template file located under {GATEWAY
To try this out
cd {GATEWAY_HOME}
-cp templates/sandbox.knoxrealm.ehcache.xml deployments/sandbox.xml
+cp templates/sandbox.knoxrealm.ehcache.xml conf/topologies/sandbox.xml
bin/ldap.sh start
bin/gateway.sh start
-The following call to WebHDFS should report: {"Path":"/user/sam"}
+The following call to WebHDFS should report: {"Path":"/user/guest"}
curl -i -v -k -u guest:guest-password -X GET
https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY
Modified: knox/trunk/books/0.6.0/service_oozie.md
URL:
http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/service_oozie.md?rev=1674094&r1=1674093&r2=1674094&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/service_oozie.md (original)
+++ knox/trunk/books/0.6.0/service_oozie.md Thu Apr 16 15:30:53 2015
@@ -48,6 +48,16 @@ In a Sandbox installation this can typic
<value>sandbox.hortonworks.com:8020</value>
</property>
+If HDFS has been configured to be in High Availability mode (HA), then instead
of the RPC address mentioned above for the Name Node, look up
+and use the logical name of the service found via dfs.nameservices in
hdfs-site.xml. For example,
+
+ <property>
+ <name>dfs.nameservices</name>
+ <value>ha-service</value>
+ </property>
+
+Please note, only one of the URL's, either the RPC endpoint or the HA service
name should be used as the NAMENODE hdfs URL in the gateway topology file.
+
The information above must be provided to the gateway via a topology
descriptor file.
These topology descriptor files are placed in `{GATEWAY_HOME}/deployments`.
An example that is setup for the default configuration of the Sandbox is
{GATEWAY_HOME}/deployments/sandbox.xml.
