http://git-wip-us.apache.org/repos/asf/hbase-site/blob/6b4bae59/book.html ---------------------------------------------------------------------- diff --git a/book.html b/book.html index 1e01ff8..fd60020 100644 --- a/book.html +++ b/book.html @@ -1404,15 +1404,13 @@ to be described later below.</p> <table id="java" class="tableblock frame-all grid-all spread"> <caption class="title">Table 2. Java</caption> <colgroup> -<col style="width: 14%;"> -<col style="width: 14%;"> -<col style="width: 14%;"> -<col style="width: 57%;"> +<col style="width: 16%;"> +<col style="width: 16%;"> +<col style="width: 66%;"> </colgroup> <thead> <tr> <th class="tableblock halign-left valign-top">HBase Version</th> -<th class="tableblock halign-left valign-top">JDK 6</th> <th class="tableblock halign-left valign-top">JDK 7</th> <th class="tableblock halign-left valign-top">JDK 8</th> </tr> @@ -1420,51 +1418,38 @@ to be described later below.</p> <tbody> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">2.0</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="http://search-hadoop.com/m/DHED4Zlz0R1";>Not Supported</a></p></td> <td class="tableblock halign-left valign-top"><p class="tableblock"><a href="http://search-hadoop.com/m/YGbbsPxZ723m3as";>Not Supported</a></p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">1.3</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="http://search-hadoop.com/m/DHED4Zlz0R1";>Not Supported</a></p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">1.2</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="http://search-hadoop.com/m/DHED4Zlz0R1";>Not Supported</a></p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">1.1</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="http://search-hadoop.com/m/DHED4Zlz0R1";>Not Supported</a></p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Running with JDK 8 will work but is not well tested.</p></td> </tr> +</tbody> +</table> +<div class="admonitionblock note"> +<table> <tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">1.0</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="http://search-hadoop.com/m/DHED4Zlz0R1";>Not Supported</a></p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Running with JDK 8 will work but is not well tested.</p></td> -</tr> -<tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">0.98</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Running with JDK 8 works but is not well tested. Building with JDK 8 would require removal of the -deprecated <code>remove()</code> method of the <code>PoolMap</code> class and is under consideration. See -<a href="https://issues.apache.org/jira/browse/HBASE-7608";>HBASE-7608</a> for more information about JDK 8 -support.</p></td> -</tr> -<tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">0.94</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">yes</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +HBase will neither build nor compile with Java 6. +</td> </tr> -</tbody> </table> +</div> <div class="admonitionblock note"> <table> <tr> @@ -1472,7 +1457,7 @@ support.</p></td> <i class="fa icon-note" title="Note"></i> </td> <td class="content"> -In HBase 0.98.5 and newer, you must set <code>JAVA_HOME</code> on each node of your cluster. <em>hbase-env.sh</em> provides a handy mechanism to do this. +You must set <code>JAVA_HOME</code> on each node of your cluster. <em>hbase-env.sh</em> provides a handy mechanism to do this. </td> </tr> </table> @@ -1581,8 +1566,8 @@ See <a href="http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Su <div class="title">Hadoop 2.x is recommended.</div> <div class="paragraph"> <p>Hadoop 2.x is faster and includes features, such as short-circuit reads, which will help improve your HBase random read profile. -Hadoop 2.x also includes important bug fixes that will improve your overall HBase experience. -HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBase 1.0 will not support Hadoop 1.x.</p> +Hadoop 2.x also includes important bug fixes that will improve your overall HBase experience. HBase does not support running with +earlier versions of Hadoop. See the table below for requirements specific to different HBase versions.</p> </div> <div class="paragraph"> <p>Hadoop 3.x is still in early access releases and has not yet been sufficiently tested by the HBase community for production use cases.</p> @@ -1610,21 +1595,15 @@ HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBas </div> <table class="tableblock frame-all grid-all spread"> <colgroup> -<col style="width: 12%;"> -<col style="width: 12%;"> -<col style="width: 12%;"> -<col style="width: 12%;"> -<col style="width: 12%;"> -<col style="width: 12%;"> -<col style="width: 12%;"> -<col style="width: 12%;"> +<col style="width: 20%;"> +<col style="width: 20%;"> +<col style="width: 20%;"> +<col style="width: 20%;"> +<col style="width: 20%;"> </colgroup> <thead> <tr> <th class="tableblock halign-left valign-top"></th> -<th class="tableblock halign-left valign-top">HBase-0.94.x</th> -<th class="tableblock halign-left valign-top">HBase-0.98.x (Support for Hadoop 1.1+ is deprecated.)</th> -<th class="tableblock halign-left valign-top">HBase-1.0.x (Hadoop 1.x is NOT supported)</th> <th class="tableblock halign-left valign-top">HBase-1.1.x</th> <th class="tableblock halign-left valign-top">HBase-1.2.x</th> <th class="tableblock halign-left valign-top">HBase-1.3.x</th> @@ -1633,40 +1612,7 @@ HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBas </thead> <tbody> <tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-1.0.x</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -</tr> -<tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-1.1.x</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -</tr> -<tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-0.23.x</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -</tr> -<tr> <td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-2.0.x-alpha</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> @@ -1674,9 +1620,6 @@ HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBas </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-2.1.0-beta</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> @@ -1685,9 +1628,6 @@ HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBas <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-2.2.0</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> @@ -1695,18 +1635,12 @@ HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBas <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-2.3.x</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-2.4.x</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> @@ -1714,9 +1648,6 @@ HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBas </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-2.5.x</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> @@ -1728,16 +1659,10 @@ HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBas <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-2.6.1+</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> @@ -1748,16 +1673,10 @@ HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBas <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-2.7.1+</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">S</p></td> @@ -1768,9 +1687,6 @@ HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBas <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">X</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">Hadoop-3.0.0-alphax</p></td> @@ -1778,9 +1694,6 @@ HBase 0.98 drops support for Hadoop 1.0, deprecates use of Hadoop 1.1+, and HBas <td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">NT</p></td> </tr> </tbody> </table> @@ -1871,100 +1784,7 @@ Hadoop version mismatch issues have various manifestations but often all looks l </table> </div> <div class="sect3"> -<h4 id="hadoop2.hbase_0.94"><a class="anchor" href="#hadoop2.hbase_0.94"></a>4.1.1. Apache HBase 0.94 with Hadoop 2</h4> -<div class="paragraph"> -<p>To get 0.94.x to run on Hadoop 2.2.0, you need to change the hadoop 2 and protobuf versions in the <em>pom.xml</em>: Here is a diff with pom.xml changes:</p> -</div> -<div class="listingblock"> -<div class="content"> -<pre class="CodeRay highlight"><code data-lang="java"><span class="error">$</span> svn diff pom.xml -Index: pom.xml -=================================================================== ---- pom.xml (revision <span class="integer">1545157</span>) -+++ pom.xml (working copy) -<span class="error">@</span><span class="error">@</span> -<span class="integer">1034</span>,<span class="integer">7</span> +<span class="integer">1034</span>,<span class="integer">7</span> <span class="error">@</span><span class="error">@</span> - <slf4j.version><span class="float">1.4</span><span class="float">.3</span></slf4j.version> - <log4j.version><span class="float">1.2</span><span class="float">.16</span></log4j.version> - <mockito-all.version><span class="float">1.8</span><span class="float">.5</span></mockito-all.version> -- <protobuf.version><span class="float">2.4</span><span class="float">.0</span>a</protobuf.version> -+ <protobuf.version><span class="float">2.5</span><span class="float">.0</span></protobuf.version> - <stax-api.version><span class="float">1.0</span><span class="float">.1</span></stax-api.version> - <thrift.version><span class="float">0.8</span><span class="float">.0</span></thrift.version> - <zookeeper.version><span class="float">3.4</span><span class="float">.5</span></zookeeper.version> -<span class="error">@</span><span class="error">@</span> -<span class="integer">2241</span>,<span class="integer">7</span> +<span class="integer">2241</span>,<span class="integer">7</span> <span class="error">@</span><span class="error">@</span> - </property> - </activation> - <properties> -- <hadoop.version><span class="float">2.0</span><span class="float">.0</span>-alpha</hadoop.version> -+ <hadoop.version><span class="float">2.2</span><span class="float">.0</span></hadoop.version> - <slf4j.version><span class="float">1.6</span><span class="float">.1</span></slf4j.version> - </properties> - <dependencies></code></pre> -</div> -</div> -<div class="paragraph"> -<p>The next step is to regenerate Protobuf files and assuming that the Protobuf has been installed:</p> -</div> -<div class="ulist"> -<ul> -<li> -<p>Go to the HBase root folder, using the command line;</p> -</li> -<li> -<p>Type the following commands:</p> -<div class="listingblock"> -<div class="content"> -<pre class="CodeRay highlight"><code data-lang="bourne">$ protoc -Isrc/main/protobuf --java_out=src/main/java src/main/protobuf/hbase.proto</code></pre> -</div> -</div> -<div class="listingblock"> -<div class="content"> -<pre class="CodeRay highlight"><code data-lang="bourne">$ protoc -Isrc/main/protobuf --java_out=src/main/java src/main/protobuf/ErrorHandling.proto</code></pre> -</div> -</div> -</li> -</ul> -</div> -<div class="paragraph"> -<p>Building against the hadoop 2 profile by running something like the following command:</p> -</div> -<div class="listingblock"> -<div class="content"> -<pre>$ mvn clean install assembly:single -Dhadoop.profile=2.0 -DskipTests</pre> -</div> -</div> -</div> -<div class="sect3"> -<h4 id="hadoop.hbase_0.94"><a class="anchor" href="#hadoop.hbase_0.94"></a>4.1.2. Apache HBase 0.92 and 0.94</h4> -<div class="paragraph"> -<p>HBase 0.92 and 0.94 versions can work with Hadoop versions, 0.20.205, 0.22.x, 1.0.x, and 1.1.x. -HBase-0.94 can additionally work with Hadoop-0.23.x and 2.x, but you may have to recompile the code using the specific maven profile (see top level pom.xml)</p> -</div> -</div> -<div class="sect3"> -<h4 id="hadoop.hbase_0.96"><a class="anchor" href="#hadoop.hbase_0.96"></a>4.1.3. Apache HBase 0.96</h4> -<div class="paragraph"> -<p>As of Apache HBase 0.96.x, Apache Hadoop 1.0.x at least is required. -Hadoop 2 is strongly encouraged (faster but also has fixes that help MTTR). We will no longer run properly on older Hadoops such as 0.20.205 or branch-0.20-append. -Do not move to Apache HBase 0.96.x if you cannot upgrade your Hadoop. See <a href="http://search-hadoop.com/m/7vFVx4EsUb2";>HBase, mail # dev - DISCUSS: - Have hbase require at least hadoop 1.0.0 in hbase 0.96.0?</a></p> -</div> -</div> -<div class="sect3"> -<h4 id="hadoop.older.versions"><a class="anchor" href="#hadoop.older.versions"></a>4.1.4. Hadoop versions 0.20.x - 1.x</h4> -<div class="paragraph"> -<p>DO NOT use Hadoop versions older than 2.2.0 for HBase versions greater than 1.0. Check release documentation if you are using an older version of HBase for Hadoop related information.</p> -</div> -</div> -<div class="sect3"> -<h4 id="hadoop.security"><a class="anchor" href="#hadoop.security"></a>4.1.5. Apache HBase on Secure Hadoop</h4> -<div class="paragraph"> -<p>Apache HBase will run on any Hadoop 0.20.x that incorporates Hadoop security features as long as you do as suggested above and replace the Hadoop jar that ships with HBase with the secure version. -If you want to read more about how to setup Secure HBase, see <a href="#hbase.secure.configuration">hbase.secure.configuration</a>.</p> -</div> -</div> -<div class="sect3"> -<h4 id="dfs.datanode.max.transfer.threads"><a class="anchor" href="#dfs.datanode.max.transfer.threads"></a>4.1.6. <code>dfs.datanode.max.transfer.threads</code> </h4> +<h4 id="dfs.datanode.max.transfer.threads"><a class="anchor" href="#dfs.datanode.max.transfer.threads"></a>4.1.1. <code>dfs.datanode.max.transfer.threads</code> </h4> <div class="paragraph"> <p>An HDFS DataNode has an upper bound on the number of files that it will serve at any one time. Before doing any loading, make sure you have configured Hadoop’s <em>conf/hdfs-site.xml</em>, setting the <code>dfs.datanode.max.transfer.threads</code> value to at least the following:</p> @@ -2000,8 +1820,8 @@ For example:</p> <div class="sect2"> <h3 id="zookeeper.requirements"><a class="anchor" href="#zookeeper.requirements"></a>4.2. ZooKeeper Requirements</h3> <div class="paragraph"> -<p>ZooKeeper 3.4.x is required as of HBase 1.0.0. -HBase makes use of the <code>multi</code> functionality that is only available since Zookeeper 3.4.0. The <code>hbase.zookeeper.useMulti</code> configuration property defaults to <code>true</code> in HBase 1.0.0. +<p>ZooKeeper 3.4.x is required. +HBase makes use of the <code>multi</code> functionality that is only available since Zookeeper 3.4.0. The <code>hbase.zookeeper.useMulti</code> configuration property defaults to <code>true</code>. Refer to <a href="https://issues.apache.org/jira/browse/HBASE-12241";>HBASE-12241 (The crash of regionServer when taking deadserver’s replication queue breaks replication)</a> and <a href="https://issues.apache.org/jira/browse/HBASE-6775";>HBASE-6775 (Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix)</a> for background. The property is deprecated and useMulti is always enabled in HBase 2.0.</p> </div> @@ -2241,9 +2061,6 @@ Check them out especially if HBase had trouble starting.</p> By default it’s deployed on the Master host at port 16010 (HBase RegionServers listen on port 16020 by default and put up an informational HTTP server at port 16030). If the Master is running on a host named <code>master.example.org</code> on the default port, point your browser at http://master.example.org:16010 to see the web interface.</p> </div> <div class="paragraph"> -<p>Prior to HBase 0.98 the master UI was deployed on port 60010, and the HBase RegionServers UI on port 60030.</p> -</div> -<div class="paragraph"> <p>Once HBase has started, see the <a href="#shell_exercises">shell exercises</a> section for how to create tables, add data, scan your insertions, and finally disable and drop your tables.</p> </div> <div class="paragraph"> @@ -5375,7 +5192,7 @@ example9</code></pre> <div class="sect3"> <h4 id="hbase_env"><a class="anchor" href="#hbase_env"></a>8.1.3. <em>hbase-env.sh</em></h4> <div class="paragraph"> -<p>The following lines in the <em>hbase-env.sh</em> file show how to set the <code>JAVA_HOME</code> environment variable (required for HBase 0.98.5 and newer) and set the heap to 4 GB (rather than the default value of 1 GB). If you copy and paste this example, be sure to adjust the <code>JAVA_HOME</code> to suit your environment.</p> +<p>The following lines in the <em>hbase-env.sh</em> file show how to set the <code>JAVA_HOME</code> environment variable (required for HBase) and set the heap to 4 GB (rather than the default value of 1 GB). If you copy and paste this example, be sure to adjust the <code>JAVA_HOME</code> to suit your environment.</p> </div> <div class="listingblock"> <div class="content"> @@ -6163,6 +5980,19 @@ for warning about incompatible changes). All effort will be made to provide a de </div> <div class="sect2"> <h3 id="hbase.versioning.pre10"><a class="anchor" href="#hbase.versioning.pre10"></a>11.2. Pre 1.0 versions</h3> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +<div class="title">HBase Pre-1.0 versions are all EOM</div> +For new installations, do not deploy 0.94.y, 0.96.y, or 0.98.y. Deploy our stable version. See <a href="https://issues.apache.org/jira/browse/HBASE-11642";>EOL 0.96</a>, <a href="https://issues.apache.org/jira/browse/HBASE-16215";>clean up of EOM releases</a>, and <a href="http://www.apache.org/dist/hbase/";>the header of our downloads</a>. +</td> +</tr> +</table> +</div> <div class="paragraph"> <p>Before the semantic versioning scheme pre-1.0, HBase tracked either Hadoop’s versions (0.2x) or 0.9x versions. If you are into the arcane, checkout our old wiki page on <a href="http://wiki.apache.org/hadoop/Hbase/HBaseVersions";>HBase Versioning</a> which tries to connect the HBase version dots. Below sections cover ONLY the releases before 1.0.</p> </div> @@ -6315,19 +6145,6 @@ for further discussion.</p> <h3 id="upgrade0.96"><a class="anchor" href="#upgrade0.96"></a>12.4. Upgrading from 0.94.x to 0.96.x</h3> <div class="sect3"> <h4 id="_the_singularity"><a class="anchor" href="#_the_singularity"></a>12.4.1. The "Singularity"</h4> -<div class="admonitionblock note"> -<table> -<tr> -<td class="icon"> -<i class="fa icon-note" title="Note"></i> -</td> -<td class="content"> -<div class="title">HBase 0.96.x was EOL’d, September 1st, 2014</div> -Do not deploy 0.96.x Deploy at least 0.98.x. See <a href="https://issues.apache.org/jira/browse/HBASE-11642";>EOL 0.96</a>. -</td> -</tr> -</table> -</div> <div class="paragraph"> <p>You will have to stop your old 0.94.x cluster completely to upgrade. If you are replicating between clusters, both clusters will have to go down to upgrade. Make sure it is a clean shutdown. The less WAL files around, the faster the upgrade will run (the upgrade will split any log files it finds in the filesystem as part of the upgrade process). All clients must be upgraded to 0.96 too.</p> </div> @@ -20483,7 +20300,7 @@ The effect is that the duplicate coprocessor is effectively ignored. </div> <div class="listingblock"> <div class="content"> -<pre class="CodeRay highlight"><code data-lang="java"><span class="directive">public</span> <span class="type">class</span> <span class="class">SumEndPoint</span> <span class="directive">extends</span> SumService <span class="directive">implements</span> Coprocessor, CoprocessorService { +<pre class="CodeRay highlight"><code data-lang="java"><span class="directive">public</span> <span class="type">class</span> <span class="class">SumEndPoint</span> <span class="directive">extends</span> Sum.SumService <span class="directive">implements</span> Coprocessor, CoprocessorService { <span class="directive">private</span> RegionCoprocessorEnvironment env; @@ -20503,31 +20320,33 @@ The effect is that the duplicate coprocessor is effectively ignored. <span class="annotation">@Override</span> <span class="directive">public</span> <span class="type">void</span> stop(CoprocessorEnvironment env) <span class="directive">throws</span> <span class="exception">IOException</span> { - <span class="comment">// do mothing</span> + <span class="comment">// do nothing</span> } <span class="annotation">@Override</span> - <span class="directive">public</span> <span class="type">void</span> getSum(RpcController controller, SumRequest request, RpcCallback done) { + <span class="directive">public</span> <span class="type">void</span> getSum(RpcController controller, Sum.SumRequest request, RpcCallback<Sum.SumResponse> done) { Scan scan = <span class="keyword">new</span> Scan(); scan.addFamily(Bytes.toBytes(request.getFamily())); scan.addColumn(Bytes.toBytes(request.getFamily()), Bytes.toBytes(request.getColumn())); - SumResponse response = <span class="predefined-constant">null</span>; + + Sum.SumResponse response = <span class="predefined-constant">null</span>; InternalScanner scanner = <span class="predefined-constant">null</span>; + <span class="keyword">try</span> { scanner = env.getRegion().getScanner(scan); - <span class="predefined-type">List</span> results = <span class="keyword">new</span> <span class="predefined-type">ArrayList</span>(); + <span class="predefined-type">List</span><Cell> results = <span class="keyword">new</span> <span class="predefined-type">ArrayList</span><>(); <span class="type">boolean</span> hasMore = <span class="predefined-constant">false</span>; - <span class="type">long</span> sum = <span class="integer">0L</span>; - <span class="keyword">do</span> { - hasMore = scanner.next(results); - <span class="keyword">for</span> (Cell cell : results) { - sum = sum + Bytes.toLong(CellUtil.cloneValue(cell)); - } - results.clear(); - } <span class="keyword">while</span> (hasMore); + <span class="type">long</span> sum = <span class="integer">0L</span>; - response = SumResponse.newBuilder().setSum(sum).build(); + <span class="keyword">do</span> { + hasMore = scanner.next(results); + <span class="keyword">for</span> (Cell cell : results) { + sum = sum + Bytes.toLong(CellUtil.cloneValue(cell)); + } + results.clear(); + } <span class="keyword">while</span> (hasMore); + response = Sum.SumResponse.newBuilder().setSum(sum).build(); } <span class="keyword">catch</span> (<span class="exception">IOException</span> ioe) { ResponseConverter.setControllerException(controller, ioe); } <span class="keyword">finally</span> { @@ -20537,6 +20356,7 @@ The effect is that the duplicate coprocessor is effectively ignored. } <span class="keyword">catch</span> (<span class="exception">IOException</span> ignored) {} } } + done.run(response); } }</code></pre> @@ -20545,33 +20365,33 @@ The effect is that the duplicate coprocessor is effectively ignored. <div class="listingblock"> <div class="content"> <pre class="CodeRay highlight"><code data-lang="java"><span class="predefined-type">Configuration</span> conf = HBaseConfiguration.create(); -<span class="comment">// Use below code for HBase version 1.x.x or above.</span> <span class="predefined-type">Connection</span> connection = ConnectionFactory.createConnection(conf); TableName tableName = TableName.valueOf(<span class="string"><span class="delimiter">"</span><span class="content">users</span><span class="delimiter">"</span></span>); Table table = connection.getTable(tableName); -<span class="comment">//Use below code HBase version 0.98.xx or below.</span> -<span class="comment">//HConnection connection = HConnectionManager.createConnection(conf);</span> -<span class="comment">//HTableInterface table = connection.getTable("users");</span> - -<span class="directive">final</span> SumRequest request = SumRequest.newBuilder().setFamily(<span class="string"><span class="delimiter">"</span><span class="content">salaryDet</span><span class="delimiter">"</span></span>).setColumn(<span class="string"><span class="delimiter">"</span><span class="content">gross</span><span class="delimiter">"</span></span>) - .build(); +<span class="directive">final</span> Sum.SumRequest request = Sum.SumRequest.newBuilder().setFamily(<span class="string"><span class="delimiter">"</span><span class="content">salaryDet</span><span class="delimiter">"</span></span>).setColumn(<span class="string"><span class="delimiter">"</span><span class="content">gross</span><span class="delimiter">"</span></span>).build(); <span class="keyword">try</span> { -<span class="predefined-type">Map</span><<span class="type">byte</span><span class="type">[]</span>, <span class="predefined-type">Long</span>> results = table.CoprocessorService (SumService.class, <span class="predefined-constant">null</span>, <span class="predefined-constant">null</span>, -<span class="keyword">new</span> Batch.Call<SumService, <span class="predefined-type">Long</span>>() { - <span class="annotation">@Override</span> - <span class="directive">public</span> <span class="predefined-type">Long</span> call(SumService aggregate) <span class="directive">throws</span> <span class="exception">IOException</span> { -BlockingRpcCallback rpcCallback = <span class="keyword">new</span> BlockingRpcCallback(); - aggregate.getSum(<span class="predefined-constant">null</span>, request, rpcCallback); - SumResponse response = rpcCallback.get(); - <span class="keyword">return</span> response.hasSum() ? response.getSum() : <span class="integer">0L</span>; + <span class="predefined-type">Map</span><<span class="type">byte</span><span class="type">[]</span>, <span class="predefined-type">Long</span>> results = table.coprocessorService( + Sum.SumService.class, + <span class="predefined-constant">null</span>, <span class="comment">/* start key */</span> + <span class="predefined-constant">null</span>, <span class="comment">/* end key */</span> + <span class="keyword">new</span> Batch.Call<Sum.SumService, <span class="predefined-type">Long</span>>() { + <span class="annotation">@Override</span> + <span class="directive">public</span> <span class="predefined-type">Long</span> call(Sum.SumService aggregate) <span class="directive">throws</span> <span class="exception">IOException</span> { + BlockingRpcCallback<Sum.SumResponse> rpcCallback = <span class="keyword">new</span> BlockingRpcCallback<>(); + aggregate.getSum(<span class="predefined-constant">null</span>, request, rpcCallback); + Sum.SumResponse response = rpcCallback.get(); + + <span class="keyword">return</span> response.hasSum() ? response.getSum() : <span class="integer">0L</span>; + } } - }); + ); + <span class="keyword">for</span> (<span class="predefined-type">Long</span> sum : results.values()) { <span class="predefined-type">System</span>.out.println(<span class="string"><span class="delimiter">"</span><span class="content">Sum = </span><span class="delimiter">"</span></span> + sum); } } <span class="keyword">catch</span> (ServiceException e) { -e.printStackTrace(); + e.printStackTrace(); } <span class="keyword">catch</span> (<span class="predefined-type">Throwable</span> e) { e.printStackTrace(); }</code></pre> @@ -20650,15 +20470,10 @@ set: <code>arg1=1,arg2=2</code>. These could have been added to <code>hbase-site <div class="listingblock"> <div class="content"> <pre class="CodeRay highlight"><code data-lang="java"><span class="predefined-type">Configuration</span> conf = HBaseConfiguration.create(); -<span class="comment">// Use below code for HBase version 1.x.x or above.</span> <span class="predefined-type">Connection</span> connection = ConnectionFactory.createConnection(conf); TableName tableName = TableName.valueOf(<span class="string"><span class="delimiter">"</span><span class="content">users</span><span class="delimiter">"</span></span>); Table table = connection.getTable(tableName); -<span class="comment">//Use below code HBase version 0.98.xx or below.</span> -<span class="comment">//HConnection connection = HConnectionManager.createConnection(conf);</span> -<span class="comment">//HTableInterface table = connection.getTable("users");</span> - Get get = <span class="keyword">new</span> Get(Bytes.toBytes(<span class="string"><span class="delimiter">"</span><span class="content">admin</span><span class="delimiter">"</span></span>)); <span class="predefined-type">Result</span> result = table.get(get); <span class="keyword">for</span> (Cell c : result.rawCells()) { @@ -27829,49 +27644,30 @@ To create the full installable HBase package takes a little bit more work, so re <div class="sect3"> <h4 id="maven.build.hadoop"><a class="anchor" href="#maven.build.hadoop"></a>146.1.3. Building against various hadoop versions.</h4> <div class="paragraph"> -<p>As of 0.96, Apache HBase supports building against Apache Hadoop versions: 1.0.3, 2.0.0-alpha and 3.0.0-SNAPSHOT. -By default, in 0.96 and earlier, we will build with Hadoop-1.0.x. -As of 0.98, Hadoop 1.x is deprecated and Hadoop 2.x is the default. -To change the version to build against, add a hadoop.profile property when you invoke mvn:</p> +<p>HBase supports building against Apache Hadoop versions: 2.y and 3.y (early release artifacts). By default we build against Hadoop 2.x.</p> +</div> +<div class="paragraph"> +<p>To build against a specific release from the Hadoop 2.y line, set e.g. <code>-Dhadoop-two.version=2.6.3</code>.</p> </div> <div class="listingblock"> <div class="content"> -<pre class="CodeRay highlight"><code data-lang="bourne">mvn -Dhadoop.profile=1.0 ...</code></pre> +<pre class="CodeRay highlight"><code data-lang="bourne">mvn -Dhadoop-two.version=2.6.3 ...</code></pre> </div> </div> <div class="paragraph"> -<p>The above will build against whatever explicit hadoop 1.x version we have in our <em>pom.xml</em> as our '1.0' version. -Tests may not all pass so you may need to pass <code>-DskipTests</code> unless you are inclined to fix the failing tests.</p> +<p>To change the major release line of Hadoop we build against, add a hadoop.profile property when you invoke mvn:</p> </div> -<div class="admonitionblock note"> -<table> -<tr> -<td class="icon"> -<i class="fa icon-note" title="Note"></i> -</td> -<td class="content"> -<div class="title">'dependencyManagement.dependencies.dependency.artifactId' fororg.apache.hbase:${compat.module}:test-jar with value '${compat.module}'does not match a valid id pattern</div> -<div class="paragraph"> -<p>You will see ERRORs like the above title if you pass the <em>default</em> profile; e.g. -if you pass hadoop.profile=1.1 when building 0.96 or hadoop.profile=2.0 when building hadoop 0.98; just drop the hadoop.profile stipulation in this case to get your build to run again. -This seems to be a maven peculiarity that is probably fixable but we’ve not spent the time trying to figure it.</p> +<div class="listingblock"> +<div class="content"> +<pre class="CodeRay highlight"><code data-lang="bourne">mvn -Dhadoop.profile=3.0 ...</code></pre> </div> -</td> -</tr> -</table> </div> <div class="paragraph"> -<p>Similarly, for 3.0, you would just replace the profile value. -Note that Hadoop-3.0.0-SNAPSHOT does not currently have a deployed maven artifact - you will need to build and install your own in your local maven repository if you want to run against this profile.</p> +<p>The above will build against whatever explicit hadoop 3.y version we have in our <em>pom.xml</em> as our '3.0' version. +Tests may not all pass so you may need to pass <code>-DskipTests</code> unless you are inclined to fix the failing tests.</p> </div> <div class="paragraph"> -<p>In earlier versions of Apache HBase, you can build against older versions of Apache Hadoop, notably, Hadoop 0.22.x and 0.23.x. -If you are running, for example HBase-0.94 and wanted to build against Hadoop 0.23.x, you would run with:</p> -</div> -<div class="listingblock"> -<div class="content"> -<pre class="CodeRay highlight"><code data-lang="bourne">mvn -Dhadoop.profile=22 ...</code></pre> -</div> +<p>To pick a particular Hadoop 3.y release, you’d set e.g. <code>-Dhadoop-three.version=3.0.0-alpha1</code>.</p> </div> </div> <div class="sect3"> @@ -27988,30 +27784,6 @@ See <a href="#java">java</a> for Java requirements per HBase release.</p> </tr> </table> </div> -<div class="sect2"> -<h3 id="_building_against_hbase_0_96_0_98"><a class="anchor" href="#_building_against_hbase_0_96_0_98"></a>147.1. Building against HBase 0.96-0.98</h3> -<div class="paragraph"> -<p>HBase 0.96.x will run on Hadoop 1.x or Hadoop 2.x. -HBase 0.98 still runs on both, but HBase 0.98 deprecates use of Hadoop 1. -HBase 1.x will <em>not</em> run on Hadoop 1. -In the following procedures, we make a distinction between HBase 1.x builds and the awkward process involved building HBase 0.96/0.98 for either Hadoop 1 or Hadoop 2 targets.</p> -</div> -<div class="paragraph"> -<p>You must choose which Hadoop to build against. -It is not possible to build a single HBase binary that runs against both Hadoop 1 and Hadoop 2. -Hadoop is included in the build, because it is needed to run HBase in standalone mode. -Therefore, the set of modules included in the tarball changes, depending on the build target. -To determine which HBase you have, look at the HBase version. -The Hadoop version is embedded within it.</p> -</div> -<div class="paragraph"> -<p>Maven, our build system, natively does not allow a single product to be built against different dependencies. -Also, Maven cannot change the set of included modules and write out the correct <em>pom.xml</em> files with appropriate dependencies, even using two build targets, one for Hadoop 1 and another for Hadoop 2. -A prerequisite step is required, which takes as input the current <em>pom.xml_s and generates Hadoop 1 or Hadoop 2 versions using a script in the _dev-tools/</em> directory, called <em>generate-hadoopX-poms.sh</em> where <em class="replaceable">X</em> is either <code>1</code> or <code>2</code>. -You then reference these generated poms when you build. -For now, just be aware of the difference between HBase 1.x builds and those of HBase 0.96-0.98. -This difference is important to the build instructions.</p> -</div> <div id="maven.settings.xml" class="exampleblock"> <div class="title">Example 59. Example <em>~/.m2/settings.xml</em> File</div> <div class="content"> @@ -28060,9 +27832,8 @@ For the build to sign them for you, you a properly configured <em>settings.xml</ </div> </div> </div> -</div> <div class="sect2"> -<h3 id="maven.release"><a class="anchor" href="#maven.release"></a>147.2. Making a Release Candidate</h3> +<h3 id="maven.release"><a class="anchor" href="#maven.release"></a>147.1. Making a Release Candidate</h3> <div class="admonitionblock note"> <table> <tr> @@ -28070,9 +27841,7 @@ For the build to sign them for you, you a properly configured <em>settings.xml</ <i class="fa icon-note" title="Note"></i> </td> <td class="content"> -These instructions are for building HBase 1.0.x. -For building earlier versions, e.g. 0.98.x, the process is different. -See this section under the respective release documentation folders. +These instructions are for building HBase 1.y.z </td> </tr> </table> @@ -28402,7 +28171,7 @@ $ svn commit ...</code></pre> </div> </div> <div class="sect2"> -<h3 id="maven.snapshot"><a class="anchor" href="#maven.snapshot"></a>147.3. Publishing a SNAPSHOT to maven</h3> +<h3 id="maven.snapshot"><a class="anchor" href="#maven.snapshot"></a>147.2. Publishing a SNAPSHOT to maven</h3> <div class="paragraph"> <p>Make sure your <em>settings.xml</em> is set up properly (see <a href="#maven.settings.xml">Example <em>~/.m2/settings.xml</em> File</a>). Make sure the hbase version includes <code>-SNAPSHOT</code> as a suffix. @@ -29286,9 +29055,7 @@ Private interfaces do not need an <code>@InterfaceStability</code> classificatio </dd> <dt class="hdlist1"><code>@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC)</code></dt> <dd> -<p>APIs for HBase coprocessor writers. -As of HBase 0.92/0.94/0.96/0.98 this api is still unstable. -No guarantees on compatibility with future versions.</p> +<p>APIs for HBase coprocessor writers.</p> </dd> <dt class="hdlist1">No <code>@InterfaceAudience</code> Classification</dt> <dd> @@ -31197,21 +30964,18 @@ If owners are absent — busy or otherwise — t <div id="hbase.fix.version.in.jira" class="paragraph"> <div class="title">How to set fix version in JIRA on issue resolve</div> <p>Here is how <a href="http://search-hadoop.com/m/azemIi5RCJ1";>we agreed</a> to set versions in JIRA when we resolve an issue. -If master is going to be 0.98.0 then:</p> +If master is going to be 2.0.0, and branch-1 1.4.0 then:</p> </div> <div class="ulist"> <ul> <li> -<p>Commit only to master: Mark with 0.98</p> -</li> -<li> -<p>Commit to 0.95 and master: Mark with 0.98, and 0.95.x</p> +<p>Commit only to master: Mark with 2.0.0</p> </li> <li> -<p>Commit to 0.94.x and 0.95, and master: Mark with 0.98, 0.95.x, and 0.94.x</p> +<p>Commit to branch-1 and master: Mark with 2.0.0, and 1.4.0</p> </li> <li> -<p>Commit to 89-fb: Mark with 89-fb.</p> +<p>Commit to branch-1.3, branch-1, and master: Mark with 2.0.0, 1.4.0, and 1.3.x</p> </li> <li> <p>Commit site fixes: no version</p>
http://git-wip-us.apache.org/repos/asf/hbase-site/blob/6b4bae59/bulk-loads.html ---------------------------------------------------------------------- diff --git a/bulk-loads.html b/bulk-loads.html index ffb0680..0e23717 100644 --- a/bulk-loads.html +++ b/bulk-loads.html @@ -7,7 +7,7 @@ <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20170418" /> + <meta name="Date-Revision-yyyymmdd" content="20170419" /> <meta http-equiv="Content-Language" content="en" /> <title>Apache HBase – Bulk Loads in Apache HBase (TM) @@ -207,18 +207,6 @@ </li> </ul> </li> - - <li class="dropdown-submenu"> - <a href="" title="0.94 Documentation">0.94 Documentation</a> - <ul class="dropdown-menu"> - <li> <a href="0.94/apidocs/index.html" target="_blank" title="API">API</a> -</li> - <li> <a href="0.94/xref/index.html" target="_blank" title="X-Ref">X-Ref</a> -</li> - <li> <a href="0.94/book.html" target="_blank" title="Ref Guide (single-page)">Ref Guide (single-page)</a> -</li> - </ul> - </li> </ul> </li> <li class="dropdown"> @@ -323,7 +311,7 @@ under the License. --> <a href="https://www.apache.org/";>The Apache Software Foundation</a>. All rights reserved. - <li id="publishDate" class="pull-right">Last Published: 2017-04-18</li> + <li id="publishDate" class="pull-right">Last Published: 2017-04-19</li> </p> </div>
