http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/docs/fluo/1.0.0-incubating/install/index.html ---------------------------------------------------------------------- diff --git a/docs/fluo/1.0.0-incubating/install/index.html b/docs/fluo/1.0.0-incubating/install/index.html new file mode 100644 index 0000000..d958a24 --- /dev/null +++ b/docs/fluo/1.0.0-incubating/install/index.html @@ -0,0 +1,387 @@ +<!DOCTYPE html> +<html> + <head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <link rel="stylesheet" href="/stylesheets/fluo.css"> + <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400";> + <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather";> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css"; > + <link rel="canonical" href="http://fluo.io/docs/fluo/1.0.0-incubating/install/";> + <link rel="icon" type="image/png" href="/resources/favicon.png"> + <title>Fluo Install Instructions | Apache Fluo</title> + <!--[if lt IE 9]> + <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> + <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> + <![endif]--> + </head> + <body> + <div id="fluo-nav" class="navbar navbar-default navbar-static-top"> + <div class="container"> + <div class="navbar-header"> + <div class="navbar-toggle-wrapper visible-xs"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse"> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + </div> + <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a> + </div> + <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> + <ul class="navbar-nav nav"> + <li><a href="/release/">Releases</a></li> + <li><a href="/tour/">Tour</a></li> + <li><a href="/docs/">Docs</a></li> + <li><a href="/apidocs/">API</a></li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="/getinvolved/">Get Involved</a></li> + <li><a href="/blog/">Blog</a></li> + <li><a href="/people/">People</a></li> + <li><a href="/related-projects/">Related Projects</a></li> + <li><a href="/poweredby/">Powered By</a></li> + </ul> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="/how-to-contribute/">How To Contribute</a></li> + <li><a href="/release-process/">Release Process</a></li> + </ul> + </li> + </ul> + <ul class="navbar-nav nav navbar-right"> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="https://www.apache.org";>Apache Homepage</a></li> + <li><a href="https://www.apache.org/licenses/LICENSE-2.0";>License</a></li> + <li><a href="https://www.apache.org/foundation/sponsorship";>Sponsorship</i></a></li> + <li><a href="https://www.apache.org/security";>Security</a></li> + <li><a href="https://www.apache.org/foundation/thanks";>Thanks</a></li> + <li><a href="https://www.apache.org/foundation/policies/conduct";>Code of Conduct</a></li> + </ul> + </li> + </ul> + </div> + </div> + </div> + <div class="container"> + <div class="row"> + <div class="col-sm-12"> + <div class="fluo-doc"> + + + + + + <header class="post-header"> + <h2 class="post-title">Fluo Install Instructions</h2> + </header> + + <article id="page-content"> + <p>Instructions for installing Apache Fluo and starting a Fluo application in YARN on a cluster where +Accumulo, Hadoop & Zookeeper are running. If you need help setting up these dependencies, see the +<a href="https://fluo.apache.org/related-projects/";>related projects page</a> for external projects that may help.</p> + +<h2 id="requirements">Requirements</h2> + +<p>Before you install Fluo, the following software must be installed and running on your local machine +or cluster:</p> + +<table> + <thead> + <tr> + <th>Software</th> + <th>Recommended Version</th> + <th>Minimum Version</th> + </tr> + </thead> + <tbody> + <tr> + <td><a href="https://accumulo.apache.org/";>Accumulo</a></td> + <td>1.7.2</td> + <td>1.6.1</td> + </tr> + <tr> + <td><a href="http://hadoop.apache.org/";>Hadoop</a></td> + <td>2.7.2</td> + <td>2.6.0</td> + </tr> + <tr> + <td><a href="http://zookeeper.apache.org/";>Zookeeper</a></td> + <td>3.4.8</td> + <td>Â </td> + </tr> + <tr> + <td><a href="http://openjdk.java.net/";>Java</a></td> + <td>JDK 8</td> + <td>JDK 8</td> + </tr> + </tbody> +</table> + +<h2 id="obtain-a-distribution">Obtain a distribution</h2> + +<p>Before you can install Fluo, you will need to obtain a distribution tarball. It is recommended that +you download the <a href="https://fluo.apache.org/download/";>latest release</a>. You can also build a distribution from the master +branch by following these steps which create a tarball in <code class="highlighter-rouge">modules/distribution/target</code>:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>git clone https://github.com/apache/incubator-fluo.git +cd fluo/ +mvn package +</code></pre> +</div> + +<h2 id="install-fluo">Install Fluo</h2> + +<p>After you obtain a Fluo distribution tarball, follow these steps to install Fluo.</p> + +<ol> + <li> + <p>Choose a directory with plenty of space and untar the distribution:</p> + + <div class="highlighter-rouge"><pre class="highlight"><code>tar -xvzf fluo-1.0.0-incubating-bin.tar.gz +</code></pre> + </div> + </li> + <li> + <p>Copy the example configuration to the base of your configuration directory to create the default +configuration for your Fluo install:</p> + + <div class="highlighter-rouge"><pre class="highlight"><code>cp conf/examples/* conf/ +</code></pre> + </div> + + <p>The default configuration will be used as the base configuration for each new application.</p> + </li> + <li> + <p>Modify <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/config/fluo.properties";>fluo.properties</a> for your environment. However, you should not configure any +application settings (like observers).</p> + + <p>NOTE - All properties that have a default are set with it. Uncomment a property if you want +to use a value different than the default. Properties that are unset and uncommented must be +set by the user.</p> + </li> + <li> + <p>Fluo needs to build its classpath using jars from the versions of Hadoop, Accumulo, and +Zookeeper that you are using. Choose one of the two ways below to make these jars available +to Fluo:</p> + + <ul> + <li>Set <code class="highlighter-rouge">HADOOP_PREFIX</code>, <code class="highlighter-rouge">ACCUMULO_HOME</code>, and <code class="highlighter-rouge">ZOOKEEPER_HOME</code> in your environment or configure + these variables in <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/config/fluo-env.sh";>fluo-env.sh</a>. Fluo will look in these locations for jars.</li> + <li> + <p>Run <code class="highlighter-rouge">./lib/fetch.sh ahz</code> to download Hadoop, Accumulo, and Zookeeper jars to <code class="highlighter-rouge">lib/ahz</code> and + configure <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/config/fluo-env.sh";>fluo-env.sh</a> to look in this directory. By default, this command will download the + default versions set in <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/lib/ahz/pom.xml";>lib/ahz/pom.xml</a>. If you are not using the default versions, you can + override them:</p> + + <div class="highlighter-rouge"><pre class="highlight"><code> ./lib/fetch.sh ahz -Daccumulo.version=1.7.2 -Dhadoop.version=2.7.2 -Dzookeeper.version=3.4.8 +</code></pre> + </div> + </li> + </ul> + </li> + <li> + <p>Fluo needs more dependencies than what is available from Hadoop, Accumulo, and Zookeeper. These +extra dependencies need to be downloaded to <code class="highlighter-rouge">lib/</code> using the command below:</p> + + <div class="highlighter-rouge"><pre class="highlight"><code> ./lib/fetch.sh extra +</code></pre> + </div> + </li> +</ol> + +<p>You are now ready to use the Fluo command script.</p> + +<h2 id="fluo-command-script">Fluo command script</h2> + +<p>The Fluo command script is located at <code class="highlighter-rouge">bin/fluo</code> of your Fluo installation. All Fluo commands are +invoked by this script.</p> + +<p>Modify and add the following to your <code class="highlighter-rouge">~/.bashrc</code> if you want to be able to execute the fluo script +from any directory:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>export PATH=/path/to/fluo-1.0.0-incubating/bin:$PATH +</code></pre> +</div> + +<p>Source your <code class="highlighter-rouge">.bashrc</code> for the changes to take effect and test the script</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>source ~/.bashrc +fluo +</code></pre> +</div> + +<p>Running the script without any arguments prints a description of all commands.</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>./bin/fluo +</code></pre> +</div> + +<h2 id="configure-a-fluo-application">Configure a Fluo application</h2> + +<p>You are now ready to configure a Fluo application. Use the command below to create the +configuration necessary for a new application. Feel free to pick a different name (other than +<code class="highlighter-rouge">myapp</code>) for your application:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>fluo new myapp +</code></pre> +</div> + +<p>This command will create a directory for your application at <code class="highlighter-rouge">apps/myapp</code> of your Fluo install which +will contain a <code class="highlighter-rouge">conf</code> and <code class="highlighter-rouge">lib</code>.</p> + +<p>The <code class="highlighter-rouge">apps/myapp/conf</code> directory contains a copy of the <code class="highlighter-rouge">fluo.properties</code> from your default +configuration. This should be configured for your application:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>vim apps/myapp/fluo.properties +</code></pre> +</div> + +<p>When configuring the observer section in fluo.properties, you can configure your instance for the +<a href="https://github.com/fluo-io/phrasecount";>phrasecount</a> application if you have not created your own application. See the <a href="https://github.com/fluo-io/phrasecount";>phrasecount</a> +example for instructions. You can also choose not to configure any observers but your workers will +be idle when started.</p> + +<p>The <code class="highlighter-rouge">apps/myapp/lib</code> directory should contain any observer jars for your application. If you +configured <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/config/fluo.properties";>fluo.properties</a> for observers, copy any jars containing these observer classes to this +directory.</p> + +<h2 id="initialize-your-application">Initialize your application</h2> + +<p>After your application has been configured, use the command below to initialize it:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>fluo init myapp +</code></pre> +</div> + +<p>This only needs to be called once and stores configuration in Zookeeper.</p> + +<h2 id="start-your-application">Start your application</h2> + +<p>A Fluo application consists of one oracle process and multiple worker processes. Before starting +your application, you can configure the number of worker process in your <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/config/fluo.properties";>fluo.properties</a> file.</p> + +<p>When you are ready to start your Fluo application on your YARN cluster, run the command below:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>fluo start myapp +</code></pre> +</div> + +<p>The start command above will work for a single-node or a large cluster. By using YARN, you do not +need to deploy the Fluo binaries to every node on your cluster or start processes on every node.</p> + +<p>You can use the following command to check the status of your instance:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>fluo status myapp +</code></pre> +</div> + +<p>For more detailed information on the YARN containers running Fluo:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>fluo info myapp +</code></pre> +</div> + +<p>You can also use <code class="highlighter-rouge">yarn application -list</code> to check the status of your Fluo instance in YARN. Logs +are viewable within YARN.</p> + +<p>When you have data in your fluo instance, you can view it using the command <code class="highlighter-rouge">fluo scan</code>. Pipe the +output to <code class="highlighter-rouge">less</code> using the command <code class="highlighter-rouge">fluo scan | less</code> if you want to page through the data.</p> + +<h2 id="stop-your-fluo-application">Stop your Fluo application</h2> + +<p>Use the following command to stop your Fluo application:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>fluo stop myapp +</code></pre> +</div> + +<p>If stop fails, there is also a kill command.</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>fluo kill myapp +</code></pre> +</div> + +<h2 id="tuning-accumulo">Tuning Accumulo</h2> + +<p>Fluo will reread the same data frequently when it checks conditions on mutations. When Fluo +initializes a table it enables data caching to make this more efficient. However you may need to +increase the amount of memory available for caching in the tserver by increasing +<code class="highlighter-rouge">tserver.cache.data.size</code>. Increasing this may require increasing the maximum tserver java heap size +in <code class="highlighter-rouge">accumulo-env.sh</code>.</p> + +<p>Fluo will run many client threads, will want to ensure the tablet server has enough threads. Should +probably increase the <code class="highlighter-rouge">tserver.server.threads.minimum</code> Accumulo setting.</p> + +<p>Using at least Accumulo 1.6.1 is recommended because multiple performance bugs were fixed.</p> + +<h2 id="tuning-yarn">Tuning YARN</h2> + +<p>When running Fluo oracles and workers in YARN, the number of instances, max memory, and number of +cores for Fluo processes can be configured in <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/config/fluo.properties";>fluo.properties</a>. If YARN is killing processes +consider increasing <code class="highlighter-rouge">twill.java.reserved.memory.mb</code> (which defaults to 200 and is set in +yarn-site.xml). The <code class="highlighter-rouge">twill.java.reserved.memory.mb</code> config determines the gap between the YARN +memory limit set in <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/config/fluo.properties";>fluo.properties</a> and the java -Xmx setting. For example, if max memory is 1024 +and twill reserved memory is 200, the java -Xmx setting will be 1024-200 = 824 MB.</p> + +<h2 id="run-locally-without-yarn">Run locally without YARN</h2> + +<p>If you do not have YARN set up, you can start the oracle and worker as a local Fluo process using +the following commands:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>local-fluo start-oracle +local-fluo start-worker +</code></pre> +</div> + +<p>Use the following commands to stop a local Fluo process:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>local-fluo stop-worker +local-fluo stop-oracle +</code></pre> +</div> + +<p>In a distributed environment, you will need to deploy and configure a Fluo distribution on every +node in your cluster.</p> + + + </article> + +</div> + + </div> + </div> + <hr> + <div class="row footer"> + <div class="col-sm-12 text-center"> + <div class="center-block"> + <a href="https://apache.org";><img src="/resources/feather.png" alt="Apache"></a> + Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a> + </div> + </div> + </div> + </div> + <script src="/javascripts/jquery.min.js"></script> + <script src="/javascripts/bootstrap.min.js"></script> + <!-- Place your <script> tags here. --> + +<!-- Google Analytics --> +<script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); + + ga('create', 'UA-55360307-1', 'auto'); + ga('send', 'pageview'); + +</script> + + </body> +</html>
http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/docs/fluo/1.0.0-incubating/metrics/index.html ---------------------------------------------------------------------- diff --git a/docs/fluo/1.0.0-incubating/metrics/index.html b/docs/fluo/1.0.0-incubating/metrics/index.html new file mode 100644 index 0000000..2a30538 --- /dev/null +++ b/docs/fluo/1.0.0-incubating/metrics/index.html @@ -0,0 +1,271 @@ +<!DOCTYPE html> +<html> + <head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <link rel="stylesheet" href="/stylesheets/fluo.css"> + <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400";> + <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather";> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css"; > + <link rel="canonical" href="http://fluo.io/docs/fluo/1.0.0-incubating/metrics/";> + <link rel="icon" type="image/png" href="/resources/favicon.png"> + <title>Fluo Metrics | Apache Fluo</title> + <!--[if lt IE 9]> + <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> + <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> + <![endif]--> + </head> + <body> + <div id="fluo-nav" class="navbar navbar-default navbar-static-top"> + <div class="container"> + <div class="navbar-header"> + <div class="navbar-toggle-wrapper visible-xs"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse"> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + </div> + <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a> + </div> + <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> + <ul class="navbar-nav nav"> + <li><a href="/release/">Releases</a></li> + <li><a href="/tour/">Tour</a></li> + <li><a href="/docs/">Docs</a></li> + <li><a href="/apidocs/">API</a></li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="/getinvolved/">Get Involved</a></li> + <li><a href="/blog/">Blog</a></li> + <li><a href="/people/">People</a></li> + <li><a href="/related-projects/">Related Projects</a></li> + <li><a href="/poweredby/">Powered By</a></li> + </ul> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="/how-to-contribute/">How To Contribute</a></li> + <li><a href="/release-process/">Release Process</a></li> + </ul> + </li> + </ul> + <ul class="navbar-nav nav navbar-right"> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="https://www.apache.org";>Apache Homepage</a></li> + <li><a href="https://www.apache.org/licenses/LICENSE-2.0";>License</a></li> + <li><a href="https://www.apache.org/foundation/sponsorship";>Sponsorship</i></a></li> + <li><a href="https://www.apache.org/security";>Security</a></li> + <li><a href="https://www.apache.org/foundation/thanks";>Thanks</a></li> + <li><a href="https://www.apache.org/foundation/policies/conduct";>Code of Conduct</a></li> + </ul> + </li> + </ul> + </div> + </div> + </div> + <div class="container"> + <div class="row"> + <div class="col-sm-12"> + <div class="fluo-doc"> + + + + + + <header class="post-header"> + <h2 class="post-title">Fluo Metrics</h2> + </header> + + <article id="page-content"> + <p>A Fluo application can be configured (in <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/config/fluo.properties";>fluo.properties</a>) to report metrics. When metrics are +configured, Fluo will report some âdefaultâ metrics about an application that help users monitor its +performance. Users can also write code to report âapplication-specificâ metrics from their +applications. Both âapplication-specificâ and âdefaultâ metrics share the same reporter configured +by <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/config/fluo.properties";>fluo.properties</a> and are described in detail below.</p> + +<h2 id="configuring-reporters">Configuring reporters</h2> + +<p>Fluo metrics are not published by default. To publish metrics, configure a reporter in the âmetricsâ +section of <a href="https://github.com/apache/fluo/blob/rel/fluo-1.0.0-incubating/modules/distribution/src/main/config/fluo.properties";>fluo.properties</a>. There are several different reporter types (i.e Console, CSV, +Graphite, JMX, SLF4J) that are implemented using <a href="https://dropwizard.github.io/metrics/3.1.0/";>Dropwizard</a>. The choice of which reporter to use +depends on the visualization tool used. If you are not currently using a visualization tool, there +is <a href="/docs/fluo/1.0.0-incubating/grafana/">documentation</a> for reporting Fluo metrics to Grafana/InfluxDB.</p> + +<h2 id="metrics-names">Metrics names</h2> + +<p>When Fluo metrics are reported, they are published using a naming scheme that encodes additional +information. This additional information is represented using all caps variables (i.e <code class="highlighter-rouge">METRIC</code>) +below.</p> + +<p>Default metrics start with <code class="highlighter-rouge">fluo.class</code> or <code class="highlighter-rouge">fluo.system</code> and have following naming schemes:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code> fluo.class.APPLICATION.REPORTER_ID.METRIC.CLASS + fluo.system.APPLICATION.REPORTER_ID.METRIC +</code></pre> +</div> + +<p>Application metrics start with <code class="highlighter-rouge">fluo.app</code> and have following scheme:</p> + +<div class="highlighter-rouge"><pre class="highlight"><code> fluo.app.REPORTER_ID.METRIC +</code></pre> +</div> + +<p>The variables below describe the additional information that is encoded in metrics names.</p> + +<ol> + <li><code class="highlighter-rouge">APPLICATION</code> - Fluo application name</li> + <li><code class="highlighter-rouge">REPORTER_ID</code> - Unique ID of the Fluo oracle, worker, or client that is reporting the metric. + When running in YARN, this ID is of the format <code class="highlighter-rouge">worker-INSTANCE_ID</code> or <code class="highlighter-rouge">oracle-INSTANCE_ID</code> + where <code class="highlighter-rouge">INSTANCE_ID</code> corresponds to instance number. When not running in YARN, this ID consists + of a hostname and a base36 long that is unique across all fluo processes.</li> + <li><code class="highlighter-rouge">METRIC</code> - Name of the metric. For âdefaultâ metrics, this is set by Fluo. For âapplicationâ + metrics, this is set by user. Name should be unique and avoid using period â.â in name.</li> + <li><code class="highlighter-rouge">CLASS</code> - Name of Fluo observer or loader class that produced metric. This allows things like + transaction collisions to be tracked per class.</li> +</ol> + +<h2 id="application-specific-metrics">Application-specific metrics</h2> + +<p>Application metrics are implemented by retrieving a <a href="https://static.javadoc.io/org.apache.fluo/fluo-api/1.0.0-incubating/org/apache/fluo/api/metrics/MetricsReporter.html";>MetricsReporter</a> from an <a href="https://static.javadoc.io/org.apache.fluo/fluo-api/1.0.0-incubating/org/apache/fluo/api/observer/Observer.html";>Observer</a>, <a href="https://static.javadoc.io/org.apache.fluo/fluo-api/1.0.0-incubating/org/apache/fluo/api/client/Loader.html";>Loader</a>, +or <a href="https://static.javadoc.io/org.apache.fluo/fluo-api/1.0.0-incubating/org/apache/fluo/api/client/FluoClient.html";>FluoClient</a>. These metrics are named using the format <code class="highlighter-rouge">fluo.app.REPORTER_ID.METRIC</code>.</p> + +<h2 id="default-metrics">Default metrics</h2> + +<p>Default metrics report for a particular Observer/Loader class or system-wide.</p> + +<p>Below are metrics that are reported from each Observer/Loader class that is configured in a Fluo +application. These metrics are reported after each transaction and named using the format +<code class="highlighter-rouge">fluo.class.APPLICATION.REPORTER_ID.METRIC.CLASS</code>.</p> + +<ul> + <li>tx_lock_wait_time - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#timers";>Timer</a> + <ul> + <li>Time transaction spent waiting on locks held by other transactions.</li> + <li>Only updated for transactions that have non-zero lock time.</li> + </ul> + </li> + <li>tx_execution_time - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#timers";>Timer</a> + <ul> + <li>Time transaction took to execute.</li> + <li>Updated for failed and successful transactions.</li> + <li>This does not include commit time, only the time from start until commit is called.</li> + </ul> + </li> + <li>tx_with_collision - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#meters";>Meter</a> + <ul> + <li>Rate of transactions with collisions.</li> + </ul> + </li> + <li>tx_collisions - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#meters";>Meter</a> + <ul> + <li>Rate of collisions.</li> + </ul> + </li> + <li>tx_entries_set - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#meters";>Meter</a> + <ul> + <li>Rate of row/columns set by transaction</li> + </ul> + </li> + <li>tx_entries_read - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#meters";>Meter</a> + <ul> + <li>Rate of row/columns read by transaction that existed.</li> + <li>There is currently no count of all reads (including non-existent data)</li> + </ul> + </li> + <li>tx_locks_timedout - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#meters";>Meter</a> + <ul> + <li>Rate of timedout locks rolled back by transaction.</li> + <li>These are locks that are held for very long periods by another transaction that appears to be +alive based on zookeeper.</li> + </ul> + </li> + <li>tx_locks_dead - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#meters";>Meter</a> + <ul> + <li>Rate of dead locks rolled by a transaction.</li> + <li>These are locks held by a process that appears to be dead according to zookeeper.</li> + </ul> + </li> + <li>tx_status_<code class="highlighter-rouge"><STATUS></code> - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#meters";>Meter</a> + <ul> + <li>Rate of different ways (i.e <code class="highlighter-rouge"><STATUS></code>) a transaction can terminate</li> + </ul> + </li> +</ul> + +<p>Below are system-wide metrics that are reported for the entire Fluo application. These metrics are +named using the format <code class="highlighter-rouge">fluo.system.APPLICATION.REPORTER_ID.METRIC</code>.</p> + +<ul> + <li>oracle_response_time - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#timers";>Timer</a> + <ul> + <li>Time each RPC call to oracle for stamps took</li> + </ul> + </li> + <li>oracle_client_stamps - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#histograms";>Histogram</a> + <ul> + <li>Number of stamps requested for each request for stamps to the server</li> + </ul> + </li> + <li>oracle_server_stamps - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#histograms";>Histogram</a> + <ul> + <li>Number of stamps requested for each request for stamps from a client</li> + </ul> + </li> + <li>worker_notifications_queued - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#gauges";>Gauge</a> + <ul> + <li>The current number of notifications queued for processing.</li> + </ul> + </li> + <li>transactor_committing - <a href="https://dropwizard.github.io/metrics/3.1.0/getting-started/#gauges";>Gauge</a> + <ul> + <li>The current number of transactions that are working their way through the commit steps.</li> + </ul> + </li> +</ul> + +<p>Histograms and Timers have a counter. In the case of a histogram, the counter is the number of times +the metric was updated and not a sum of the updates. For example if a request for 5 timestamps was +made to the oracle followed by a request for 3 timestamps, then the count for <code class="highlighter-rouge">oracle_server_stamps</code> +would be 2 and the mean would be (5+3)/2.</p> + + + </article> + +</div> + + </div> + </div> + <hr> + <div class="row footer"> + <div class="col-sm-12 text-center"> + <div class="center-block"> + <a href="https://apache.org";><img src="/resources/feather.png" alt="Apache"></a> + Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a> + </div> + </div> + </div> + </div> + <script src="/javascripts/jquery.min.js"></script> + <script src="/javascripts/bootstrap.min.js"></script> + <!-- Place your <script> tags here. --> + +<!-- Google Analytics --> +<script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); + + ga('create', 'UA-55360307-1', 'auto'); + ga('send', 'pageview'); + +</script> + + </body> +</html> http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/docs/fluo/1.0.0-incubating/resources/fluo-architecture.png ---------------------------------------------------------------------- diff --git a/docs/fluo/1.0.0-incubating/resources/fluo-architecture.png b/docs/fluo/1.0.0-incubating/resources/fluo-architecture.png new file mode 100644 index 0000000..3ba96fd Binary files /dev/null and b/docs/fluo/1.0.0-incubating/resources/fluo-architecture.png differ http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/docs/fluo/index.html ---------------------------------------------------------------------- diff --git a/docs/fluo/index.html b/docs/fluo/index.html index 5992797..1aaba5f 100644 --- a/docs/fluo/index.html +++ b/docs/fluo/index.html @@ -1,122 +1,10 @@ <!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <meta http-equiv="X-UA-Compatible" content="IE=edge"> - <meta name="viewport" content="width=device-width, initial-scale=1"> - <link rel="stylesheet" href="/stylesheets/fluo.css"> - <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400";> - <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather";> - <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css"; > - <link rel="canonical" href="http://fluo.io/docs/fluo/";> - <link rel="icon" type="image/png" href="/resources/favicon.png"> - <title>Fluo Documentation Archive | Apache Fluo</title> - <!--[if lt IE 9]> - <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> - <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> - <![endif]--> - </head> - <body> - <div id="fluo-nav" class="navbar navbar-default navbar-static-top"> - <div class="container"> - <div class="navbar-header"> - <div class="navbar-toggle-wrapper visible-xs"> - <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse"> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - </button> - </div> - <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a> - </div> - <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> - <ul class="navbar-nav nav"> - <li><a href="/docs/">Documentation</a></li> - <li><a href="/download/">Download</a></li> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="/getinvolved/">Get Involved</a></li> - <li><a href="/blog/">Blog</a></li> - <li><a href="/people/">People</a></li> - <li><a href="/related-projects/">Related Projects</a></li> - <li><a href="/poweredby/">Powered By</a></li> - </ul> - </li> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="/how-to-contribute/">How To Contribute</a></li> - <li><a href="/release-process/">Release Process</a></li> - </ul> - </li> - </ul> - <ul class="navbar-nav nav navbar-right"> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="https://www.apache.org";>Apache Homepage</a></li> - <li><a href="https://www.apache.org/licenses/LICENSE-2.0";>License</a></li> - <li><a href="https://www.apache.org/foundation/sponsorship";>Sponsorship</i></a></li> - <li><a href="https://www.apache.org/security";>Security</a></li> - <li><a href="https://www.apache.org/foundation/thanks";>Thanks</a></li> - <li><a href="https://www.apache.org/foundation/policies/conduct";>Code of Conduct</a></li> - </ul> - </li> - </ul> - </div> - </div> - </div> - <div class="container"> - <div class="row"> - <div class="col-sm-12"> - <div class="page"> - - <header class="post-header"> - <h2 class="post-title">Fluo Documentation Archive</h2> - </header> - - <article id="page-content"> - <p>Fluo documentation before Apache incubation:</p> - -<ul> - <li><a href="/docs/fluo/1.0.0-beta-2/">1.0.0-beta-2</a> - January 12, 2016</li> - <li><a href="/docs/fluo/1.0.0-beta-1/">1.0.0-beta-1</a> - June 9, 2015</li> - <li><a href="/docs/fluo/1.0.0-alpha-1/">1.0.0-alpha-1</a> - October 2, 2014</li> -</ul> - - - </article> - -</div> - - </div> - </div> - <hr> - <div class="row footer"> - <div class="col-sm-12 text-center"> - <div class="center-block"> - <a href="https://apache.org";><img src="/resources/feather.png" alt="Apache"></a> - Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a> - </div> - </div> - </div> - </div> - <script src="/javascripts/jquery.min.js"></script> - <script src="/javascripts/bootstrap.min.js"></script> - <!-- Place your <script> tags here. --> - -<!-- Google Analytics --> -<script> - (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ - (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), - m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) - })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); - - ga('create', 'UA-55360307-1', 'auto'); - ga('send', 'pageview'); - -</script> - - </body> +<html lang="en-US"> +<meta charset="utf-8"> +<title>Redirectingâ¦</title> +<link rel="canonical" href="http://fluo.io/docs/";> +<meta http-equiv="refresh" content="0; url=http://fluo.io/docs/";> +<h1>Redirectingâ¦</h1> +<a href="http://fluo.io/docs/";>Click here if you are not redirected.</a> +<script>location="http://fluo.io/docs/";</script> </html> http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/docs/index.html ---------------------------------------------------------------------- diff --git a/docs/index.html b/docs/index.html index a2df0e8..dee080d 100644 --- a/docs/index.html +++ b/docs/index.html @@ -31,8 +31,10 @@ </div> <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> <ul class="navbar-nav nav"> - <li><a href="/docs/">Documentation</a></li> - <li><a href="/download/">Download</a></li> + <li><a href="/release/">Releases</a></li> + <li><a href="/tour/">Tour</a></li> + <li><a href="/docs/">Docs</a></li> + <li><a href="/apidocs/">API</a></li> <li class="dropdown"> <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> <ul class="dropdown-menu"> @@ -81,17 +83,19 @@ <p><a href="https://github.com/apache/fluo";>Apache Fluo</a> and <a href="https://github.com/apache/fluo-recipes";>Apache Fluo Recipes</a> have seperate documentation as they are different repositories with their own release cycle.</p> -<h2 id="apache-fluo">Apache Fluo</h2> +<h3 id="apache-fluo-documentation">Apache Fluo documentation</h3> -<p>The first release of Apache Fluo has not been made yet.</p> +<ul> + <li><a href="/docs/fluo/1.0.0-incubating/">1.0.0-incubating</a> - September 27, 2016</li> +</ul> -<h2 id="apache-fluo-recipes">Apache Fluo Recipes</h2> +<h3 id="apache-fluo-recipes-documentation">Apache Fluo Recipes documentation</h3> -<p>The first release of Apache Fluo Recipes has not been made yet.</p> +<ul> + <li>The first release of Apache Fluo Recipes has not been made yet.</li> +</ul> -<h2 id="pre-apache-documentation">Pre Apache Documentation</h2> - -<p>Documentation about releases made before Apache Fluo entered incubation at Apache has been moved <a href="pre-asf">here</a></p> +<p>Documentation for releases before joining Apache are available in the <a href="archive">Pre-ASF archive</a>.</p> </article> http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/docs/pre-asf/index.html ---------------------------------------------------------------------- diff --git a/docs/pre-asf/index.html b/docs/pre-asf/index.html deleted file mode 100644 index daee0ed..0000000 --- a/docs/pre-asf/index.html +++ /dev/null @@ -1,136 +0,0 @@ -<!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <meta http-equiv="X-UA-Compatible" content="IE=edge"> - <meta name="viewport" content="width=device-width, initial-scale=1"> - <link rel="stylesheet" href="/stylesheets/fluo.css"> - <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400";> - <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather";> - <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css"; > - <link rel="canonical" href="http://fluo.io/docs/pre-asf/";> - <link rel="icon" type="image/png" href="/resources/favicon.png"> - <title>Pre Apache Documentation | Apache Fluo</title> - <!--[if lt IE 9]> - <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> - <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> - <![endif]--> - </head> - <body> - <div id="fluo-nav" class="navbar navbar-default navbar-static-top"> - <div class="container"> - <div class="navbar-header"> - <div class="navbar-toggle-wrapper visible-xs"> - <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse"> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - </button> - </div> - <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a> - </div> - <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> - <ul class="navbar-nav nav"> - <li><a href="/docs/">Documentation</a></li> - <li><a href="/download/">Download</a></li> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="/getinvolved/">Get Involved</a></li> - <li><a href="/blog/">Blog</a></li> - <li><a href="/people/">People</a></li> - <li><a href="/related-projects/">Related Projects</a></li> - <li><a href="/poweredby/">Powered By</a></li> - </ul> - </li> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="/how-to-contribute/">How To Contribute</a></li> - <li><a href="/release-process/">Release Process</a></li> - </ul> - </li> - </ul> - <ul class="navbar-nav nav navbar-right"> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="https://www.apache.org";>Apache Homepage</a></li> - <li><a href="https://www.apache.org/licenses/LICENSE-2.0";>License</a></li> - <li><a href="https://www.apache.org/foundation/sponsorship";>Sponsorship</i></a></li> - <li><a href="https://www.apache.org/security";>Security</a></li> - <li><a href="https://www.apache.org/foundation/thanks";>Thanks</a></li> - <li><a href="https://www.apache.org/foundation/policies/conduct";>Code of Conduct</a></li> - </ul> - </li> - </ul> - </div> - </div> - </div> - <div class="container"> - <div class="row"> - <div class="col-sm-12"> - <div class="page"> - - <header class="post-header"> - <h2 class="post-title">Pre Apache Documentation</h2> - </header> - - <article id="page-content"> - <div class="alert alert-danger" role="alert">Please note - This is documentation related to non-ASF releases of Fluo. These releases were made before Apache Fluo became an ASF project and are not endorsed by the ASF.</div> - -<h2 id="fluo">Fluo</h2> - -<p>Last non-ASF release was <code class="highlighter-rouge">1.0.0-beta-2</code> made on January 12, 2016.</p> - -<ul> - <li>Documentation - <a href="/docs/fluo/1.0.0-beta-2/">Latest</a> | <a href="/docs/fluo/">Archive</a></li> - <li>API - <a href="/apidocs/fluo/1.0.0-beta-2/">Latest</a> | <a href="/apidocs/fluo/">Archive</a></li> - <li>Release summary - <a href="/release-summaries/1.0.0-beta-2/">Latest</a> | <a href="/release-summaries/">Archive</a></li> - <li>Release notes - <a href="/release-notes/1.0.0-beta-2/">Latest</a> | <a href="/release-notes/">Archive</a></li> -</ul> - -<h2 id="fluo-recipes">Fluo Recipes</h2> - -<p>Last non-ASF release was <code class="highlighter-rouge">1.0.0-beta-2</code> made on March 29, 2016.</p> - -<ul> - <li>Documentation - <a href="/docs/fluo-recipes/1.0.0-beta-2/">Latest</a> | <a href="/docs/fluo-recipes/">Archive</a></li> - <li>API - <a href="/apidocs/fluo-recipes/1.0.0-beta-2/">Latest</a> | <a href="/apidocs/fluo-recipes/">Archive</a></li> -</ul> - - - </article> - -</div> - - </div> - </div> - <hr> - <div class="row footer"> - <div class="col-sm-12 text-center"> - <div class="center-block"> - <a href="https://apache.org";><img src="/resources/feather.png" alt="Apache"></a> - Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a> - </div> - </div> - </div> - </div> - <script src="/javascripts/jquery.min.js"></script> - <script src="/javascripts/bootstrap.min.js"></script> - <!-- Place your <script> tags here. --> - -<!-- Google Analytics --> -<script> - (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ - (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), - m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) - })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); - - ga('create', 'UA-55360307-1', 'auto'); - ga('send', 'pageview'); - -</script> - - </body> -</html> http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/download/index.html ---------------------------------------------------------------------- diff --git a/download/index.html b/download/index.html index 375a6d4..fd16e80 100644 --- a/download/index.html +++ b/download/index.html @@ -31,8 +31,10 @@ </div> <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> <ul class="navbar-nav nav"> - <li><a href="/docs/">Documentation</a></li> - <li><a href="/download/">Download</a></li> + <li><a href="/release/">Releases</a></li> + <li><a href="/tour/">Tour</a></li> + <li><a href="/docs/">Docs</a></li> + <li><a href="/apidocs/">API</a></li> <li class="dropdown"> <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> <ul class="dropdown-menu">
