Author: sumit Date: Thu Apr 16 15:30:53 2015 New Revision: 1674094 URL: http://svn.apache.org/r1674094 Log: KNOX-530 oozie docs for Webhdfs HA
Modified: knox/site/books/knox-0-6-0/user-guide.html knox/site/index.html knox/site/issue-tracking.html knox/site/license.html knox/site/mail-lists.html knox/site/project-info.html knox/site/team-list.html knox/trunk/books/0.6.0/config_ldap_authc_cache.md knox/trunk/books/0.6.0/service_oozie.md Modified: knox/site/books/knox-0-6-0/user-guide.html URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/user-guide.html?rev=1674094&r1=1674093&r2=1674094&view=diff ============================================================================== --- knox/site/books/knox-0-6-0/user-guide.html (original) +++ knox/site/books/knox-0-6-0/user-guide.html Thu Apr 16 15:30:53 2015 @@ -941,7 +941,7 @@ ldapRealm.userDnTemplate=uid={0},ou=peop <value>authcBasic</value> </param> </provider> -</code></pre><h3><a id="Trying+out+caching"></a>Trying out caching</h3><p>Knox bundles a template topology files that can be used to try out the caching functionality. The template file located under {GATEWAY_HOME}/templates is sandbox.knoxrealm.ehcache.xml.</p><p>To try this out</p><p>cd {GATEWAY_HOME} cp templates/sandbox.knoxrealm.ehcache.xml deployments/sandbox.xml bin/ldap.sh start bin/gateway.sh start</p><p>The following call to WebHDFS should report: {“Path”:“/user/sam”}</p><p>curl -i -v -k -u guest:guest-password -X GET <a href="https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY">https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY</a></p><p>In order to see the cache working, LDAP can now be shutdown and the user will still authenticate successfully.</p><p>bin/ldap.sh stop</p><p>and then the following should still return successfully like it did earlier.</p><p>curl -i -v -k -u guest:guest-password -X GET <a href="ht tps://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY">https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY</a></p><h4><a id="Advanced+Caching+Config"></a>Advanced Caching Config</h4><p>By default the ehcache support in shiro contains a ehcache.xml in its classpath which is the following</p> +</code></pre><h3><a id="Trying+out+caching"></a>Trying out caching</h3><p>Knox bundles a template topology files that can be used to try out the caching functionality. The template file located under {GATEWAY_HOME}/templates is sandbox.knoxrealm.ehcache.xml.</p><p>To try this out</p><p>cd {GATEWAY_HOME} cp templates/sandbox.knoxrealm.ehcache.xml conf/topologies/sandbox.xml bin/ldap.sh start bin/gateway.sh start</p><p>The following call to WebHDFS should report: {“Path”:“/user/guest”}</p><p>curl -i -v -k -u guest:guest-password -X GET <a href="https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY">https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY</a></p><p>In order to see the cache working, LDAP can now be shutdown and the user will still authenticate successfully.</p><p>bin/ldap.sh stop</p><p>and then the following should still return successfully like it did earlier.</p><p>curl -i -v -k -u guest:guest-password -X GET <a hr ef="https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY">https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY</a></p><h4><a id="Advanced+Caching+Config"></a>Advanced Caching Config</h4><p>By default the ehcache support in shiro contains a ehcache.xml in its classpath which is the following</p> <pre><code><ehcache> <!-- Sets the path to the directory where cache .data files are created. @@ -2337,7 +2337,12 @@ session.shutdown() <name>dfs.namenode.rpc-address</name> <value>sandbox.hortonworks.com:8020</value> </property> -</code></pre><p>The information above must be provided to the gateway via a topology descriptor file. These topology descriptor files are placed in <code>{GATEWAY_HOME}/deployments</code>. An example that is setup for the default configuration of the Sandbox is {GATEWAY_HOME}/deployments/sandbox.xml. These values will need to be changed for non-default Sandbox or other Hadoop cluster configuration.</p> +</code></pre><p>If HDFS has been configured to be in High Availability mode (HA), then instead of the RPC address mentioned above for the Name Node, look up and use the logical name of the service found via dfs.nameservices in hdfs-site.xml. For example,</p> +<pre><code><property> + <name>dfs.nameservices</name> + <value>ha-service</value> +</property> +</code></pre><p>Please note, only one of the URL’s, either the RPC endpoint or the HA service name should be used as the NAMENODE hdfs URL in the gateway topology file.</p><p>The information above must be provided to the gateway via a topology descriptor file. These topology descriptor files are placed in <code>{GATEWAY_HOME}/deployments</code>. An example that is setup for the default configuration of the Sandbox is {GATEWAY_HOME}/deployments/sandbox.xml. These values will need to be changed for non-default Sandbox or other Hadoop cluster configuration.</p> <pre><code><service> <role>NAMENODE</role> <url>hdfs://localhost:8020</url> Modified: knox/site/index.html URL: http://svn.apache.org/viewvc/knox/site/index.html?rev=1674094&r1=1674093&r2=1674094&view=diff ============================================================================== --- knox/site/index.html (original) +++ knox/site/index.html Thu Apr 16 15:30:53 2015 @@ -1,5 +1,5 @@ <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 --> +<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 --> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> @@ -10,7 +10,7 @@ @import url("./css/site.css"); </style> <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" /> - <meta name="Date-Revision-yyyymmdd" content="20150407" /> + <meta name="Date-Revision-yyyymmdd" content="20150416" /> <meta http-equiv="Content-Language" content="en" /> <script type="text/javascript">var _gaq = _gaq || []; @@ -57,7 +57,7 @@ <a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a> - | <span id="publishDate">Last Published: 2015-04-07</span> + | <span id="publishDate">Last Published: 2015-04-16</span> | <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span> </div> <div class="clear"> Modified: knox/site/issue-tracking.html URL: http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1674094&r1=1674093&r2=1674094&view=diff ============================================================================== --- knox/site/issue-tracking.html (original) +++ knox/site/issue-tracking.html Thu Apr 16 15:30:53 2015 @@ -1,5 +1,5 @@ <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 --> +<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 --> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> @@ -10,7 +10,7 @@ @import url("./css/site.css"); </style> <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" /> - <meta name="Date-Revision-yyyymmdd" content="20150407" /> + <meta name="Date-Revision-yyyymmdd" content="20150416" /> <meta http-equiv="Content-Language" content="en" /> <script type="text/javascript">var _gaq = _gaq || []; @@ -57,7 +57,7 @@ <a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a> - | <span id="publishDate">Last Published: 2015-04-07</span> + | <span id="publishDate">Last Published: 2015-04-16</span> | <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span> </div> <div class="clear"> Modified: knox/site/license.html URL: http://svn.apache.org/viewvc/knox/site/license.html?rev=1674094&r1=1674093&r2=1674094&view=diff ============================================================================== --- knox/site/license.html (original) +++ knox/site/license.html Thu Apr 16 15:30:53 2015 @@ -1,5 +1,5 @@ <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 --> +<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 --> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> @@ -10,7 +10,7 @@ @import url("./css/site.css"); </style> <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" /> - <meta name="Date-Revision-yyyymmdd" content="20150407" /> + <meta name="Date-Revision-yyyymmdd" content="20150416" /> <meta http-equiv="Content-Language" content="en" /> <script type="text/javascript">var _gaq = _gaq || []; @@ -57,7 +57,7 @@ <a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a> - | <span id="publishDate">Last Published: 2015-04-07</span> + | <span id="publishDate">Last Published: 2015-04-16</span> | <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span> </div> <div class="clear"> Modified: knox/site/mail-lists.html URL: http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1674094&r1=1674093&r2=1674094&view=diff ============================================================================== --- knox/site/mail-lists.html (original) +++ knox/site/mail-lists.html Thu Apr 16 15:30:53 2015 @@ -1,5 +1,5 @@ <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 --> +<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 --> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> @@ -10,7 +10,7 @@ @import url("./css/site.css"); </style> <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" /> - <meta name="Date-Revision-yyyymmdd" content="20150407" /> + <meta name="Date-Revision-yyyymmdd" content="20150416" /> <meta http-equiv="Content-Language" content="en" /> <script type="text/javascript">var _gaq = _gaq || []; @@ -57,7 +57,7 @@ <a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a> - | <span id="publishDate">Last Published: 2015-04-07</span> + | <span id="publishDate">Last Published: 2015-04-16</span> | <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span> </div> <div class="clear"> Modified: knox/site/project-info.html URL: http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1674094&r1=1674093&r2=1674094&view=diff ============================================================================== --- knox/site/project-info.html (original) +++ knox/site/project-info.html Thu Apr 16 15:30:53 2015 @@ -1,5 +1,5 @@ <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 --> +<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 --> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> @@ -10,7 +10,7 @@ @import url("./css/site.css"); </style> <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" /> - <meta name="Date-Revision-yyyymmdd" content="20150407" /> + <meta name="Date-Revision-yyyymmdd" content="20150416" /> <meta http-equiv="Content-Language" content="en" /> <script type="text/javascript">var _gaq = _gaq || []; @@ -57,7 +57,7 @@ <a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a> - | <span id="publishDate">Last Published: 2015-04-07</span> + | <span id="publishDate">Last Published: 2015-04-16</span> | <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span> </div> <div class="clear"> Modified: knox/site/team-list.html URL: http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1674094&r1=1674093&r2=1674094&view=diff ============================================================================== --- knox/site/team-list.html (original) +++ knox/site/team-list.html Thu Apr 16 15:30:53 2015 @@ -1,5 +1,5 @@ <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-07 --> +<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-16 --> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> @@ -10,7 +10,7 @@ @import url("./css/site.css"); </style> <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" /> - <meta name="Date-Revision-yyyymmdd" content="20150407" /> + <meta name="Date-Revision-yyyymmdd" content="20150416" /> <meta http-equiv="Content-Language" content="en" /> <script type="text/javascript">var _gaq = _gaq || []; @@ -57,7 +57,7 @@ <a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a> - | <span id="publishDate">Last Published: 2015-04-07</span> + | <span id="publishDate">Last Published: 2015-04-16</span> | <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span> </div> <div class="clear"> Modified: knox/trunk/books/0.6.0/config_ldap_authc_cache.md URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/config_ldap_authc_cache.md?rev=1674094&r1=1674093&r2=1674094&view=diff ============================================================================== --- knox/trunk/books/0.6.0/config_ldap_authc_cache.md (original) +++ knox/trunk/books/0.6.0/config_ldap_authc_cache.md Thu Apr 16 15:30:53 2015 @@ -98,11 +98,11 @@ The template file located under {GATEWAY To try this out cd {GATEWAY_HOME} -cp templates/sandbox.knoxrealm.ehcache.xml deployments/sandbox.xml +cp templates/sandbox.knoxrealm.ehcache.xml conf/topologies/sandbox.xml bin/ldap.sh start bin/gateway.sh start -The following call to WebHDFS should report: {"Path":"/user/sam"} +The following call to WebHDFS should report: {"Path":"/user/guest"} curl -i -v -k -u guest:guest-password -X GET https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY Modified: knox/trunk/books/0.6.0/service_oozie.md URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/service_oozie.md?rev=1674094&r1=1674093&r2=1674094&view=diff ============================================================================== --- knox/trunk/books/0.6.0/service_oozie.md (original) +++ knox/trunk/books/0.6.0/service_oozie.md Thu Apr 16 15:30:53 2015 @@ -48,6 +48,16 @@ In a Sandbox installation this can typic <value>sandbox.hortonworks.com:8020</value> </property> +If HDFS has been configured to be in High Availability mode (HA), then instead of the RPC address mentioned above for the Name Node, look up +and use the logical name of the service found via dfs.nameservices in hdfs-site.xml. For example, + + <property> + <name>dfs.nameservices</name> + <value>ha-service</value> + </property> + +Please note, only one of the URL's, either the RPC endpoint or the HA service name should be used as the NAMENODE hdfs URL in the gateway topology file. + The information above must be provided to the gateway via a topology descriptor file. These topology descriptor files are placed in `{GATEWAY_HOME}/deployments`. An example that is setup for the default configuration of the Sandbox is {GATEWAY_HOME}/deployments/sandbox.xml.