This is an automated email from the ASF dual-hosted git repository. mwalch pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/asf-site by this push: new e7c6322 Jekyll build from master:942bfcb e7c6322 is described below commit e7c6322dd754345be95381a7c1c11a2b7a5bfa0e Author: Mike Walch <mwa...@apache.org> AuthorDate: Tue Jan 15 11:08:14 2019 -0500 Jekyll build from master:942bfcb Updates links to M/R examples --- docs/2.x/administration/in-depth-install.html | 16 ++++++++-------- docs/2.x/development/high_speed_ingest.html | 5 ++--- docs/2.x/development/mapreduce.html | 12 +++++++++++- docs/2.x/getting-started/clients.html | 13 ------------- docs/2.x/getting-started/quickstart.html | 2 +- feed.xml | 4 ++-- search_data.json | 10 +++++----- 7 files changed, 29 insertions(+), 33 deletions(-) diff --git a/docs/2.x/administration/in-depth-install.html b/docs/2.x/administration/in-depth-install.html index e7b70ea..a56187a 100644 --- a/docs/2.x/administration/in-depth-install.html +++ b/docs/2.x/administration/in-depth-install.html @@ -593,8 +593,8 @@ and specify the following:</p> <p>Accumulo uses <code class="highlighter-rouge">HADOOP_HOME</code> and <code class="highlighter-rouge">ZOOKEEPER_HOME</code> to locate Hadoop and Zookeeper jars and add them the <code class="highlighter-rouge">CLASSPATH</code> variable. If you are running a vendor-specific release of Hadoop or Zookeeper, you may need to change how your <code class="highlighter-rouge">CLASSPATH</code> is built in <a href="/docs/2.x/configuration/files#accumulo-envsh">accumulo-env.sh</a>. If -Accumulo has problems later on finding jars, run <code class="highlighter-rouge">accumulo classpath -d</code> to debug and print -Accumulo’s classpath.</p> +Accumulo has problems later on finding jars, run <code class="highlighter-rouge">accumulo classpath</code> to print Accumulo’s +classpath.</p> <p>You may want to change the default memory settings for Accumulo’s TabletServer which are by set in the <code class="highlighter-rouge">JAVA_OPTS</code> settings for ‘tservers’ in <a href="/docs/2.x/configuration/files#accumulo-envsh">accumulo-env.sh</a>. Note the @@ -799,10 +799,12 @@ consideration. There is no enforcement of these warnings via the API.</p> <h3 id="configuring-the-classloader">Configuring the ClassLoader</h3> -<p>Accumulo builds its Java classpath in <a href="/docs/2.x/configuration/files#accumulo-envsh">accumulo-env.sh</a>. After an Accumulo application has started, it will load classes from the locations -specified in the deprecated <a href="/docs/2.x/configuration/server-properties#general_classpaths">general.classpaths</a> property. Additionally, Accumulo will load classes from the locations specified in the -<a href="/docs/2.x/configuration/server-properties#general_dynamic_classpaths">general.dynamic.classpaths</a> property and will monitor and reload them if they change. The reloading feature is useful during the development -and testing of iterators as new or modified iterator classes can be deployed to Accumulo without having to restart the database.</p> +<p>Accumulo builds its Java classpath in <a href="/docs/2.x/configuration/files#accumulo-envsh">accumulo-env.sh</a>. This classpath can be viewed by running <code class="highlighter-rouge">accumulo classpath</code>.</p> + +<p>After an Accumulo application has started, it will load classes from the locations specified in the deprecated <a href="/docs/2.x/configuration/server-properties#general_classpaths">general.classpaths</a> property. +Additionally, Accumulo will load classes from the locations specified in the <a href="/docs/2.x/configuration/server-properties#general_dynamic_classpaths">general.dynamic.classpaths</a> property and will monitor and reload +them if they change. The reloading feature is useful during the development and testing of iterators as new or modified iterator classes can be +deployed to Accumulo without having to restart the database.</p> <p>Accumulo also has an alternate configuration for the classloader which will allow it to load classes from remote locations. This mechanism uses Apache Commons VFS which enables locations such as http and hdfs to be used. This alternate configuration also uses the @@ -810,8 +812,6 @@ uses Apache Commons VFS which enables locations such as http and hdfs to be used <a href="/docs/2.x/configuration/server-properties#general_vfs_classpaths">general.vfs.classpaths</a> property instead of the <a href="/docs/2.x/configuration/server-properties#general_dynamic_classpaths">general.dynamic.classpaths</a> property. As in the default configuration, this alternate configuration will also monitor the vfs locations for changes and reload if necessary.</p> -<p>The Accumulo classpath can be viewed in human readable format by running <code class="highlighter-rouge">accumulo classpath -d</code>.</p> - <h5 id="classloader-contexts">ClassLoader Contexts</h5> <p>With the addition of the VFS based classloader, we introduced the notion of classloader contexts. A context is identified diff --git a/docs/2.x/development/high_speed_ingest.html b/docs/2.x/development/high_speed_ingest.html index 3210596..3f0159b 100644 --- a/docs/2.x/development/high_speed_ingest.html +++ b/docs/2.x/development/high_speed_ingest.html @@ -533,10 +533,9 @@ import file.</p> <h2 id="mapreduce-ingest">MapReduce Ingest</h2> <p>It is possible to efficiently write many mutations to Accumulo in parallel via a -MapReduce job. Typically, a MapReduce job will process data that lives in HDFS +MapReduce job. Typically, a MapReduce job will process data that lives in HDFS and write mutations to Accumulo using <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-hadoop-mapreduce/2.0.0-alpha-1/org/apache/accumulo/hadoop/mapreduce/AccumuloOutputFormat.html">AccumuloOutputFormat</a>. For more information -on how use to use MapReduce with Accumulo, see the <a href="/docs/2.x/development/mapreduce">MapReduce documentation</a> -and the <a href="https://github.com/apache/accumulo-examples/blob/master/docs/mapred.md">MapReduce example code</a>.</p> +on how use to use MapReduce with Accumulo, see the <a href="/docs/2.x/development/mapreduce">MapReduce documentation</a>.</p> diff --git a/docs/2.x/development/mapreduce.html b/docs/2.x/development/mapreduce.html index fc1e310..2e5a4bf 100644 --- a/docs/2.x/development/mapreduce.html +++ b/docs/2.x/development/mapreduce.html @@ -608,7 +608,17 @@ can then be bulk imported into Accumulo:</p> </li> </ol> -<p>The <a href="https://github.com/apache/accumulo-examples/blob/master/docs/mapred.md">MapReduce example</a> contains a complete example of using MapReduce with Accumulo.</p> +<h2 id="example-code">Example Code</h2> + +<p>The <a href="https://github.com/apache/accumulo-examples/">Accumulo Examples repo</a> has several MapReduce examples:</p> + +<ul> + <li><a href="https://github.com/apache/accumulo-examples/blob/master/docs/wordcount.md">wordcount</a> - Uses MapReduce and Accumulo to do a word count on text files</li> + <li><a href="https://github.com/apache/accumulo-examples/blob/master/docs/regex.md">regex</a> - Uses MapReduce and Accumulo to find data using regular expressions</li> + <li><a href="https://github.com/apache/accumulo-examples/blob/master/docs/rowhash.md">rowhash</a> - Uses MapReduce to read a table and write to a new column in the same table</li> + <li><a href="https://github.com/apache/accumulo-examples/blob/master/docs/tablettofile.md">tablettofile</a> - Uses MapReduce to read a table and write one of its columns to a file in HDFS</li> + <li><a href="https://github.com/apache/accumulo-examples/blob/master/docs/uniquecols.md">uniquecols</a> - Uses MapReduce to count unique columns in Accumulo</li> +</ul> diff --git a/docs/2.x/getting-started/clients.html b/docs/2.x/getting-started/clients.html index 535ab53..fbb7af2 100644 --- a/docs/2.x/getting-started/clients.html +++ b/docs/2.x/getting-started/clients.html @@ -760,7 +760,6 @@ of the different ways to execute client code.</p> <li>build and execute an uber jar</li> <li>add <code class="highlighter-rouge">accumulo classpath</code> to your Java classpath</li> <li>use the <code class="highlighter-rouge">accumulo</code> command</li> - <li>use the <code class="highlighter-rouge">accumulo-util hadoop-jar</code> command</li> </ul> <h3 id="build-and-execute-an-uber-jar">Build and execute an uber jar</h3> @@ -778,12 +777,6 @@ to include all of Accumulo’s dependencies on your classpath:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>java -classpath /path/to/my.jar:/path/to/dep.jar:$(accumulo classpath) com.my.Main arg1 arg2 </code></pre></div></div> -<p>If you would like to review which jars are included, the <code class="highlighter-rouge">accumulo classpath</code> command can -output a more human readable format using the <code class="highlighter-rouge">-d</code> option which enables debugging:</p> - -<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>accumulo classpath -d -</code></pre></div></div> - <h3 id="use-the-accumulo-command">Use the accumulo command</h3> <p>Another option for running your code is to use the Accumulo script which can execute a @@ -801,12 +794,6 @@ the accumulo command.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>export CLASSPATH=/path/to/my.jar:/path/to/dep.jar; accumulo com.foo.Client arg1 arg2 </code></pre></div></div> -<h3 id="use-the-accumulo-util-hadoop-jar-command">Use the ‘accumulo-util hadoop-jar’ command</h3> - -<p>If you are writing map reduce job that accesses Accumulo, then you can use -<code class="highlighter-rouge">accumulo-util hadoop-jar</code> to run those jobs. See the <a href="https://github.com/apache/accumulo-examples/blob/master/docs/mapred.md">MapReduce example</a> -for more information.</p> - <h2 id="additional-documentation">Additional Documentation</h2> <p>This page covers Accumulo client basics. Below are links to additional documentation that may be useful when creating Accumulo clients:</p> diff --git a/docs/2.x/getting-started/quickstart.html b/docs/2.x/getting-started/quickstart.html index e1d3765..6317f3c 100644 --- a/docs/2.x/getting-started/quickstart.html +++ b/docs/2.x/getting-started/quickstart.html @@ -503,7 +503,7 @@ installations. Accumulo will use these locations to find Hadoop and Zookeeper ja them to your <code class="highlighter-rouge">CLASSPATH</code> variable. If you you are running a vendor-specific release of Hadoop or Zookeeper, you may need to modify how the <code class="highlighter-rouge">CLASSPATH</code> variable is built in <a href="/docs/2.x/configuration/files#accumulo-envsh">accumulo-env.sh</a>. If Accumulo has problems loading classes when you start it, run -<code class="highlighter-rouge">accumulo classpath -d</code> to debug and print Accumulo’s classpath.</p> +<code class="highlighter-rouge">accumulo classpath</code> to print Accumulo’s classpath.</p> </li> <li> <p>Accumulo tablet servers are configured by default to use 1GB of memory (768MB is allocated to diff --git a/feed.xml b/feed.xml index a7bcc03..17d8955 100644 --- a/feed.xml +++ b/feed.xml @@ -6,8 +6,8 @@ </description> <link>https://accumulo.apache.org/</link> <atom:link href="https://accumulo.apache.org/feed.xml" rel="self" type="application/rss+xml"/> - <pubDate>Mon, 07 Jan 2019 14:26:50 -0500</pubDate> - <lastBuildDate>Mon, 07 Jan 2019 14:26:50 -0500</lastBuildDate> + <pubDate>Tue, 15 Jan 2019 11:08:06 -0500</pubDate> + <lastBuildDate>Tue, 15 Jan 2019 11:08:06 -0500</lastBuildDate> <generator>Jekyll v3.7.3</generator> diff --git a/search_data.json b/search_data.json index 90f31f2..2fd8ae1 100644 --- a/search_data.json +++ b/search_data.json @@ -16,7 +16,7 @@ "docs-2-x-administration-in-depth-install": { "title": "In-depth Installation", - "content" : "This document provides detailed instructions for installing Accumulo. For basicinstructions, see the quick start.HardwareBecause we are running essentially two or three systems simultaneously layeredacross the cluster: HDFS, Accumulo and MapReduce, it is typical for hardware toconsist of 4 to 8 cores, and 8 to 32 GB RAM. This is so each running process can haveat least one core and 2 - 4 GB each.One core running HDFS can typically keep 2 to 4 disks busy, so each machi [...] + "content" : "This document provides detailed instructions for installing Accumulo. For basicinstructions, see the quick start.HardwareBecause we are running essentially two or three systems simultaneously layeredacross the cluster: HDFS, Accumulo and MapReduce, it is typical for hardware toconsist of 4 to 8 cores, and 8 to 32 GB RAM. This is so each running process can haveat least one core and 2 - 4 GB each.One core running HDFS can typically keep 2 to 4 disks busy, so each machi [...] "url": " /docs/2.x/administration/in-depth-install", "categories": "administration" }, @@ -93,7 +93,7 @@ "docs-2-x-development-high-speed-ingest": { "title": "High-Speed Ingest", - "content" : "Accumulo is often used as part of a larger data processing and storage system. Tomaximize the performance of a parallel system involving Accumulo, the ingestionand query components should be designed to provide enough parallelism andconcurrency to avoid creating bottlenecks for users and other systems writing toand reading from Accumulo. There are several ways to achieve high ingestperformance.Pre-Splitting New TablesNew tables consist of a single tablet by default. A [...] + "content" : "Accumulo is often used as part of a larger data processing and storage system. Tomaximize the performance of a parallel system involving Accumulo, the ingestionand query components should be designed to provide enough parallelism andconcurrency to avoid creating bottlenecks for users and other systems writing toand reading from Accumulo. There are several ways to achieve high ingestperformance.Pre-Splitting New TablesNew tables consist of a single tablet by default. A [...] "url": " /docs/2.x/development/high_speed_ingest", "categories": "development" }, @@ -107,7 +107,7 @@ "docs-2-x-development-mapreduce": { "title": "MapReduce", - "content" : "Accumulo tables can be used as the source and destination of MapReduce jobs.General MapReduce configurationAdd Accumulo’s MapReduce API to your dependenciesIf you are using Maven, add the following dependency to your pom.xml to use Accumulo’s MapReduce API:&lt;dependency&gt; &lt;groupId&gt;org.apache.accumulo&lt;/groupId&gt; &lt;artifactId&gt;accumulo-hadoop-mapreduce&lt;/artifactId&gt; &lt;version&gt;2.0.0-alpha-1&am [...] + "content" : "Accumulo tables can be used as the source and destination of MapReduce jobs.General MapReduce configurationAdd Accumulo’s MapReduce API to your dependenciesIf you are using Maven, add the following dependency to your pom.xml to use Accumulo’s MapReduce API:&lt;dependency&gt; &lt;groupId&gt;org.apache.accumulo&lt;/groupId&gt; &lt;artifactId&gt;accumulo-hadoop-mapreduce&lt;/artifactId&gt; &lt;version&gt;2.0.0-alpha-1&am [...] "url": " /docs/2.x/development/mapreduce", "categories": "development" }, @@ -135,7 +135,7 @@ "docs-2-x-getting-started-clients": { "title": "Accumulo Clients", - "content" : "Creating Client CodeIf you are using Maven to create Accumulo client code, add the following dependency to your pom:&lt;dependency&gt; &lt;groupId&gt;org.apache.accumulo&lt;/groupId&gt; &lt;artifactId&gt;accumulo-core&lt;/artifactId&gt; &lt;version&gt;2.0.0-alpha-1&lt;/version&gt;&lt;/dependency&gt;When writing code that uses Accumulo, only use the Accumulo Public API.The accumulo-core artifact include [...] + "content" : "Creating Client CodeIf you are using Maven to create Accumulo client code, add the following dependency to your pom:&lt;dependency&gt; &lt;groupId&gt;org.apache.accumulo&lt;/groupId&gt; &lt;artifactId&gt;accumulo-core&lt;/artifactId&gt; &lt;version&gt;2.0.0-alpha-1&lt;/version&gt;&lt;/dependency&gt;When writing code that uses Accumulo, only use the Accumulo Public API.The accumulo-core artifact include [...] "url": " /docs/2.x/getting-started/clients", "categories": "getting-started" }, @@ -163,7 +163,7 @@ "docs-2-x-getting-started-quickstart": { "title": "Quick Start", - "content" : "This quick start provides basic instructions for installing and running Accumulo. For detailed instructions,see the in-depth installation guide.Consider using automated toolsIf you are setting up Accumulo for testing or development, consider using the following tools: Uno sets up Accumulo on a single machine for development Muchos sets up Accumulo on a cluster (optionally launched in Amazon EC2)If you are setting up Accumulo for a production environment, follow the [...] + "content" : "This quick start provides basic instructions for installing and running Accumulo. For detailed instructions,see the in-depth installation guide.Consider using automated toolsIf you are setting up Accumulo for testing or development, consider using the following tools: Uno sets up Accumulo on a single machine for development Muchos sets up Accumulo on a cluster (optionally launched in Amazon EC2)If you are setting up Accumulo for a production environment, follow the [...] "url": " /docs/2.x/getting-started/quickstart", "categories": "getting-started" },