http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/release-notes/1.0.0-beta-1/index.html ---------------------------------------------------------------------- diff --git a/release-notes/1.0.0-beta-1/index.html b/release-notes/1.0.0-beta-1/index.html deleted file mode 100644 index 02e2d21..0000000 --- a/release-notes/1.0.0-beta-1/index.html +++ /dev/null @@ -1,243 +0,0 @@ -<!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <meta http-equiv="X-UA-Compatible" content="IE=edge"> - <meta name="viewport" content="width=device-width, initial-scale=1"> - <link rel="stylesheet" href="/stylesheets/fluo.css"> - <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400";> - <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather";> - <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css"; > - <link rel="canonical" href="http://fluo.io/release-notes/1.0.0-beta-1/";> - <link rel="icon" type="image/png" href="/resources/favicon.png"> - <title>Fluo 1.0.0-beta-1 Release Notes | Apache Fluo</title> - <!--[if lt IE 9]> - <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> - <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> - <![endif]--> - </head> - <body> - <div id="fluo-nav" class="navbar navbar-default navbar-static-top"> - <div class="container"> - <div class="navbar-header"> - <div class="navbar-toggle-wrapper visible-xs"> - <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse"> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - </button> - </div> - <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a> - </div> - <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> - <ul class="navbar-nav nav"> - <li><a href="/docs/">Documentation</a></li> - <li><a href="/download/">Download</a></li> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="/getinvolved/">Get Involved</a></li> - <li><a href="/blog/">Blog</a></li> - <li><a href="/people/">People</a></li> - <li><a href="/related-projects/">Related Projects</a></li> - <li><a href="/poweredby/">Powered By</a></li> - </ul> - </li> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="/how-to-contribute/">How To Contribute</a></li> - <li><a href="/release-process/">Release Process</a></li> - </ul> - </li> - </ul> - <ul class="navbar-nav nav navbar-right"> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="https://www.apache.org";>Apache Homepage</a></li> - <li><a href="https://www.apache.org/licenses/LICENSE-2.0";>License</a></li> - <li><a href="https://www.apache.org/foundation/sponsorship";>Sponsorship</i></a></li> - <li><a href="https://www.apache.org/security";>Security</a></li> - <li><a href="https://www.apache.org/foundation/thanks";>Thanks</a></li> - <li><a href="https://www.apache.org/foundation/policies/conduct";>Code of Conduct</a></li> - </ul> - </li> - </ul> - </div> - </div> - </div> - <div class="container"> - <div class="row"> - <div class="col-sm-12"> - <div class="post-header"> - <h2>Fluo 1.0.0-beta-1 Release Notes</h2> - <p class="text-muted">09 Jun 2015</p> -</div> - - - <div class="alert alert-danger" role="alert">These release notes are for a release of Fluo that made before Apache incubation.</div> - - -<div class="post-content"> - <p>Fluo 1.0.0-beta-1 is the second official release of Fluo. The release closed <a href="https://github.com/fluo-io/fluo/issues?q=milestone%3A1.0.0-beta-1+is%3Aclosed";>133 tickets</a>. This release is not recommended for production use.</p> - -<h2 id="significant-features">Significant features</h2> - -<p>This release contains many new features that makes it easier to run, develop, and monitor Fluo applications.</p> - -<h4 id="simplified-fluo-administration-on-a-local-machine-or-ec2-cluster">Simplified Fluo administration on a local machine or EC2 cluster</h4> - -<p>Developers can now run Fluo and its dependencies on their local machine (<a href="https://github.com/fluo-io/fluo/issues/92";>#92</a>) or an AWS EC2 cluster (<a href="https://github.com/fluo-io/fluo/issues/356";>#356</a>) using a few simple commands. -This was done by creating two administration tools called <a href="https://github.com/fluo-io/fluo-dev";>Fluo-dev</a> and <a href="https://github.com/fluo-io/fluo-deploy";>Fluo-deploy</a> whose scripts and configuration reside in repos -separate from the Fluo code base. These tools allow developers to collaborate and share configuration for running Fluo.</p> - -<h4 id="transaction-metrics-are-viewable-using-common-monitoring-tools">Transaction metrics are viewable using common monitoring tools</h4> - -<p>Fluo now publishes metrics (<a href="https://github.com/fluo-io/fluo/issues/20";>#20</a>) about transactions, collisions, and timestamps using <a href="https://dropwizard.github.io/metrics/3.1.0/";>Dropwizard metrics</a>. These metrics are by default published -using JMX and are viewable using JConsole or JVisualVM. Fluo can also be configured to publish metrics to Graphite or Ganglia. View the <a href="https://github.com/fluo-io/fluo/blob/1.0.0-beta-1/docs/metrics.md";>metrics documentation</a> -for more information.</p> - -<h4 id="improved-processing-of-notifications-in-fluo-workers">Improved processing of notifications in Fluo workers</h4> - -<p>Fluo workers were refactored to separate the code used for finding and executing work. Each worker uses a single thread for finding -work (<a href="https://github.com/fluo-io/fluo/issues/19";>#19</a>). A new method was introduced to partition work among workers using a hash+mod of notifications (<a href="https://github.com/fluo-io/fluo/issues/282";>#282</a>). -The commit message for <a href="https://github.com/fluo-io/fluo/commit/4100e236b8438350e30eda924a6360e2c722ae37";>4100e23</a> contains a good description of some of the benefits and drawback of the current -hashing approach.</p> - -<h4 id="improved-the-deletion-of-observer-notifications">Improved the deletion of observer notifications</h4> - -<p>When a cell is deleted in Accumulo, a delete marker is inserted. Delete markers stay around until -all files in a tablet are compacted. For Fluo this could cause a lot of notification delete markers -to build up over time. To avoid this buildup, the way Fluo deletes notifications was changed in -(<a href="https://github.com/fluo-io/fluo/issues/457";>#457</a>). Accumulo delete markers are no longer used, Fluo now uses a custom delete marker for -notifications. The custom deleter marker allows Fluo to do analysis when Accumulo flushes memory to -disk and avoid writing many delete markers to persistent storage.</p> - -<h4 id="easier-management-of-fluo-from-the-command-line">Easier management of Fluo from the command line</h4> - -<p>Fluo now provides different scripts (<strong>fluo</strong>, <strong>mini-fluo</strong>, & <strong>local-fluo</strong>) for managing Fluo using a YARN cluster, MiniFluo, or local processes. Several commands -were created for these scripts. A <strong>scan</strong> command allows users to print a snapshot of a Fluo table (<a href="https://github.com/fluo-io/fluo/issues/319";>#319</a>). A <strong>info</strong> command shows locations of containers -when running Fluo in YARN (<a href="https://github.com/fluo-io/fluo/issues/297";>#297</a>). A <strong>classpath</strong> command gives users a list of jars needed to execute Fluo client code (<a href="https://github.com/fluo-io/fluo/issues/436";>#436</a>). A <strong>wait</strong> command will -sleep until all notifications are processed (<a href="https://github.com/fluo-io/fluo/issues/434";>#434</a>).</p> - -<h4 id="support-for-running-multiple-fluo-applications-on-a-single-cluster">Support for running multiple Fluo applications on a single cluster</h4> - -<p>Users can now run multiple Fluo applications using a single cluster (<a href="https://github.com/fluo-io/fluo/issues/454";>#454</a>). This enables different Fluo users to share the same cluster. Fluo applications -can be started and stopped independently. Each application has its own configuration.</p> - -<h4 id="fluo-build-improvements">Fluo build improvements</h4> - -<p>On each build, all Java code is automatically formatted based on Google Java Style (<a href="https://github.com/fluo-io/fluo/issues/479";>#479</a>). Also, checkstyle and findbugs will fail the build if certain -standards are not reached (<a href="https://github.com/fluo-io/fluo/issues/185";>#185</a>). The POM is also sorted (<a href="https://github.com/fluo-io/fluo/issues/493";>#493</a>).</p> - -<h4 id="organized-fluo-code-base">Organized Fluo code base</h4> - -<p>The Fluo stress test was moved to its own <a href="https://github.com/fluo-io/fluo-stress";>repo</a> and is no longer a sub-module (<a href="https://github.com/fluo-io/fluo/issues/385";>#385</a>). MiniFluo was moved from fluo-core to the fluo-mini -module/jar (<a href="https://github.com/fluo-io/fluo/issues/439";>#439</a>). This reduced the number of dependencies in fluo-core. However, developers will now need to include the fluo-mini jar in their Maven -POM if they start MiniFluo.</p> - -<h4 id="fluo-testing-improvements">Fluo testing improvements</h4> - -<p>Integration tests can now be run from Eclipse (<a href="https://github.com/fluo-io/fluo/issues/322";>#322</a>). Several new <a href="https://github.com/fluo-io/fluo/issues?utf8=%E2%9C%93&q=milestone%3A1.0.0-beta-1+is%3Aclosed+%22unit+test%22";>unit tests</a> were created.</p> - -<h4 id="other-important-improvements-and-bug-fixes">Other important improvements and bug fixes</h4> - -<ul> - <li><a href="https://github.com/fluo-io/fluo/issues/470";>#470</a> - Replaced FluoFileOutputFormat with an Accumulo Key/Value generator</li> - <li><a href="https://github.com/fluo-io/fluo/issues/460";>#460</a> - Reduced Fluo API module dependencies</li> - <li><a href="https://github.com/fluo-io/fluo/issues/456";>#456</a> - Fixed bug with notifications being lost when processes died</li> - <li><a href="https://github.com/fluo-io/fluo/issues/446";>#446</a> - Simplified log configuration and configure rolling log files in YARN</li> - <li><a href="https://github.com/fluo-io/fluo/issues/442";>#442</a> - Reduced the number of curator clients in FluoAdmin</li> - <li><a href="https://github.com/fluo-io/fluo/issues/383";>#383</a> - Improved transaction logging to help users debug collisions. See <a href="https://github.com/fluo-io/fluo/blob/1.0.0-beta-1/docs/applications.md#debugging-applications";>debugging documentation</a>.</li> - <li><a href="https://github.com/fluo-io/fluo/issues/365";>#365</a> - Analyze Fluo code to see what non-public Accumulo APIs are used</li> - <li><a href="https://github.com/fluo-io/fluo/issues/362";>#362</a> - Made API data objects immutable</li> - <li><a href="https://github.com/fluo-io/fluo/issues/349";>#349</a> - Support application level configuration in fluo.properties</li> - <li><a href="https://github.com/fluo-io/fluo/issues/342";>#342</a> - Add a configurable retry timeout to Fluo clients</li> - <li><a href="https://github.com/fluo-io/fluo/issues/294";>#294</a> - Fluo now uses chroot suffix in its Zookeeper connection.</li> - <li><a href="https://github.com/fluo-io/fluo/issues/293";>#293</a> - Add argument checking to FluoConfiguration</li> - <li><a href="https://github.com/fluo-io/fluo/issues/244";>#244</a> - Make re-initialization easier for user</li> -</ul> - -<h2 id="testing">Testing</h2> - -<p>A successful long stress test run was conducted using Fluo built from commit -<a href="https://github.com/fluo-io/fluo/commit/fb647dd6a470e8015654f8ed99b9196f5f49582a";>fb647dd</a>. The test ran very well and never fell behind like a -<a href="/blog/2014/12/30/stress-test-long-run/">previous long run</a> of stress did. The test had the following -properties.</p> - -<ul> - <li>Initialized stress test using 1 billion random integers.</li> - <li>Ran 150 incremental loads of 100 thousand integers. Slept 3 minutes between loads.</li> - <li>Used 19 m3.xlarge nodes on EC2. 16 workers and 3 masters</li> - <li>Configuration for the test committed and tagged in git : <a href="https://github.com/keith-turner/fluo-deploy/tree/beta-long-test-1";>fluo-deploy tag</a> and <a href="https://github.com/keith-turner/fluo-stress/tree/beta-long-test-1";>fluo-stress tag</a></li> - <li>Opened two issues as a result of test <a href="https://github.com/fluo-io/fluo/issues/499";>#499</a> and <a href="https://github.com/fluo-io/fluo-stress/issues/30";>fluo-stress#30</a></li> -</ul> - -<p>Below is the trailing output from running the test.</p> - -<div class="highlighter-rouge"><pre class="highlight"><code>*****Generating and loading incremental data set 148***** -*****Generating and loading incremental data set 149***** -*****Generating and loading incremental data set 150***** -*****Calculating # of unique integers using MapReduce***** - UNIQUE=1014486419 -*****Wait for Fluo to finish processing***** -05:33:40.158 [main] INFO io.fluo.cluster.runner.AppRunner - The wait command will exit when all notifications are processed -05:33:40.417 [Thread-3] INFO io.fluo.core.oracle.OracleClient - Connected to oracle at worker4:9913 -05:33:41.308 [main] INFO io.fluo.cluster.runner.AppRunner - All processing has finished! -*****Printing # of unique integers calculated by Fluo***** -Total at root : 1014486419 -Nodes Scanned : 59605 -*****Verifying Fluo & MapReduce results match***** -Success! Fluo & MapReduce both calculated 1014486419 unique integers -</code></pre> -</div> - -<p>The test ran for a little over 12 hours. Below are two plots pulled from -graphite showing the number of notifications queued and transaction rate over -the entire test run. Load transactions are not included in the rate. The rate -is transactions per second. The accuracy of these plots is uncertain because -no graphite configuration changes were made. The plots do seem within the -ballpark.</p> - -<p><img src="/resources/release-notes/1.0.0-beta-1/queued.png" alt="Notifications Queued" title="Notifications Queued" /> <img src="/resources/release-notes/1.0.0-beta-1/rate.png" alt="Transaction rate" title="Transaction Rate" /></p> - - -</div> - -<hr> - -<div> - <p>All Fluo release notes can be viewed in the <a href="/release-notes/">archive</a></p> -</div> - - </div> - </div> - <hr> - <div class="row footer"> - <div class="col-sm-12 text-center"> - <div class="center-block"> - <a href="https://apache.org";><img src="/resources/feather.png" alt="Apache"></a> - Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a> - </div> - </div> - </div> - </div> - <script src="/javascripts/jquery.min.js"></script> - <script src="/javascripts/bootstrap.min.js"></script> - <!-- Place your <script> tags here. --> - -<!-- Google Analytics --> -<script> - (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ - (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), - m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) - })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); - - ga('create', 'UA-55360307-1', 'auto'); - ga('send', 'pageview'); - -</script> - - </body> -</html>
http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/release-notes/1.0.0-beta-2/index.html ---------------------------------------------------------------------- diff --git a/release-notes/1.0.0-beta-2/index.html b/release-notes/1.0.0-beta-2/index.html deleted file mode 100644 index 241442c..0000000 --- a/release-notes/1.0.0-beta-2/index.html +++ /dev/null @@ -1,224 +0,0 @@ -<!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <meta http-equiv="X-UA-Compatible" content="IE=edge"> - <meta name="viewport" content="width=device-width, initial-scale=1"> - <link rel="stylesheet" href="/stylesheets/fluo.css"> - <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400";> - <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather";> - <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css"; > - <link rel="canonical" href="http://fluo.io/release-notes/1.0.0-beta-2/";> - <link rel="icon" type="image/png" href="/resources/favicon.png"> - <title>Fluo 1.0.0-beta-2 Release Notes | Apache Fluo</title> - <!--[if lt IE 9]> - <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> - <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> - <![endif]--> - </head> - <body> - <div id="fluo-nav" class="navbar navbar-default navbar-static-top"> - <div class="container"> - <div class="navbar-header"> - <div class="navbar-toggle-wrapper visible-xs"> - <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse"> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - </button> - </div> - <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a> - </div> - <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> - <ul class="navbar-nav nav"> - <li><a href="/docs/">Documentation</a></li> - <li><a href="/download/">Download</a></li> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="/getinvolved/">Get Involved</a></li> - <li><a href="/blog/">Blog</a></li> - <li><a href="/people/">People</a></li> - <li><a href="/related-projects/">Related Projects</a></li> - <li><a href="/poweredby/">Powered By</a></li> - </ul> - </li> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="/how-to-contribute/">How To Contribute</a></li> - <li><a href="/release-process/">Release Process</a></li> - </ul> - </li> - </ul> - <ul class="navbar-nav nav navbar-right"> - <li class="dropdown"> - <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a> - <ul class="dropdown-menu"> - <li><a href="https://www.apache.org";>Apache Homepage</a></li> - <li><a href="https://www.apache.org/licenses/LICENSE-2.0";>License</a></li> - <li><a href="https://www.apache.org/foundation/sponsorship";>Sponsorship</i></a></li> - <li><a href="https://www.apache.org/security";>Security</a></li> - <li><a href="https://www.apache.org/foundation/thanks";>Thanks</a></li> - <li><a href="https://www.apache.org/foundation/policies/conduct";>Code of Conduct</a></li> - </ul> - </li> - </ul> - </div> - </div> - </div> - <div class="container"> - <div class="row"> - <div class="col-sm-12"> - <div class="post-header"> - <h2>Fluo 1.0.0-beta-2 Release Notes</h2> - <p class="text-muted">12 Jan 2016</p> -</div> - - - <div class="alert alert-danger" role="alert">These release notes are for a release of Fluo that made before Apache incubation.</div> - - -<div class="post-content"> - <p>Fluo 1.0.0-beta-2 is the third official release of Fluo. This release closed -<a href="https://github.com/fluo-io/fluo/issues?q=milestone%3A1.0.0-beta-2+is%3Aclosed";>48 tickets</a>. There is no upgrade path from 1.0.0-beta-1 to -1.0.0-beta-2.</p> - -<h2 id="significant-features">Significant features</h2> - -<p>This release contains many new features that makes it easier to run, develop, and monitor Fluo applications.</p> - -<h4 id="improved-fluo-metrics-that-can-be-sent-to-influxdb-and-viewed-in-grafana">Improved Fluo metrics that can be sent to InfluxDB and viewed in Grafana</h4> - -<p>In <a href="https://github.com/fluo-io/fluo/issues/569";>#569</a>, <a href="https://github.com/fluo-io/fluo/issues/570";>#570</a>, & <a href="https://github.com/fluo-io/fluo/issues/580";>#580</a>, Fluo metrics and monitoring were refactored to fix several -bugs and allow metrics to be sent to <a href="https://github.com/influxdata/influxdb";>InfluxDB</a> and viewed in <a href="http://grafana.org/";>Grafana</a>. Fluo metrics -are still instrumented using <a href="https://dropwizard.github.io/metrics/3.1.0/";>Dropwizard metrics</a> but in <a href="https://github.com/fluo-io/fluo/issues/574";>#574</a> metrics configuration was -moved from its own file and to now reside in <code class="highlighter-rouge">fluo.properties</code>. While Fluo metrics can still be sent to -many different tools (i.e Graphite, Ganglia), Fluo now ships with configuration that can be used to configure -a Fluo dashboard in Grafana that queries InfluxDB. To set up the sending of Fluo metrics to InfluxDB/Grafana, -view this <a href="/docs/fluo/1.0.0-beta-2/grafana/">documentation</a> or consider using <a href="https://github.com/fluo-io/fluo-dev";>Fluo-dev</a> or <a href="https://github.com/fluo-io/zetten";>Zetten</a> -to run Fluo as they can install InfluxDB+Grafana and setup metrics for you.</p> - -<h4 id="improved-fluo-administration">Improved Fluo administration</h4> - -<p>Several commands were added to the <code class="highlighter-rouge">fluo</code> script which is used to administer Fluo. A <code class="highlighter-rouge">fluo exec</code> command -(<a href="https://github.com/fluo-io/fluo/pull/581";>#581</a>) was created to provide an easy way to execute application code using Fluo classes and dependencies. -A <code class="highlighter-rouge">fluo list</code> command (<a href="https://github.com/fluo-io/fluo/issues/523";>#523</a>) was created to let users list all Fluo applications within a Fluo instance. -The <code class="highlighter-rouge">fluo scan</code> command now has a <code class="highlighter-rouge">--raw</code> option (<a href="https://github.com/fluo-io/fluo/issues/597";>#597</a>) that prints Fluo data as stored in Accumulo. This -was accomplished by moving the Fluo formatter from Accumulo shell to the scan command. The scan command can now -format non-ascii characters as hex (<a href="https://github.com/fluo-io/fluo/issues/568";>#568</a>). The <code class="highlighter-rouge">fluo new</code> command was improved to prevent users from -setting invalid Fluo application names (<a href="https://github.com/fluo-io/fluo/issues/510";>#510</a>). A bug was fixed in the <code class="highlighter-rouge">fluo start</code> command that was causing -time outs when starting applications (<a href="https://github.com/fluo-io/fluo/issues/562";>#562</a>). Finally, the <code class="highlighter-rouge">fluo</code> command can now be run without the <code class="highlighter-rouge">apps/</code> -directory configured for an application for most commands (<a href="https://github.com/fluo-io/fluo/issues/524";>#524</a>). Only the <code class="highlighter-rouge">init</code> and <code class="highlighter-rouge">start</code> commands need -this directory configured. All other commands only require the default properties file to be configured at -<code class="highlighter-rouge">conf/fluo.properties</code>.</p> - -<h4 id="made-fluo-work-better-with-spark">Made Fluo work better with Spark</h4> - -<p>Several changes were made to Fluo to allow it work better with Apache Spark. All Fluo data types now implement -<code class="highlighter-rouge">Serializable</code> and can be use in Spark RDDs (<a href="https://github.com/fluo-io/fluo/issues/539";>#539</a>). Fluo data types also now implement <code class="highlighter-rouge">Comparable</code> which -allows RDDs of Fluo data types to be sorted (<a href="https://github.com/fluo-io/fluo/issues/544";>#544</a>). Also, a no args constructor was created for the -MutableBytes data type to enable Kryo serialization in Spark (<a href="https://github.com/fluo-io/fluo/pull/549";>#549</a>). Finally, a new InputFormat called -<code class="highlighter-rouge">FluoEntryInputFormat</code> was created that returns key/value entries and the existing <code class="highlighter-rouge">FluoInputFormat</code> that returns -rows was renamed <code class="highlighter-rouge">FluoRowInputFormat</code> (<a href="https://github.com/fluo-io/fluo/issues/538";>#538</a>,<a href="https://github.com/fluo-io/fluo/issues/540";>#540</a>).</p> - -<h4 id="performance-improvements">Performance improvements</h4> - -<p>A good bit of time was spent analyzing Fluo while it was running to determine -where time is spent when executing transactions. Based on this analysis, it -was found that a good bit of time was spent committing transactions. Changes -were made in Fluo and Accumulo in order to decrease commit time. For Fluo, the -following changes were made :</p> - -<ul> - <li><a href="https://github.com/fluo-io/fluo/issues/591";>#591</a> - Shared batch writer increases transaction history</li> - <li><a href="https://github.com/fluo-io/fluo/pull/590";>#590</a> - Increased batch writer threads and made configurable</li> - <li><a href="https://github.com/fluo-io/fluo/pull/589";>#589</a> - Added 2nd conditional writer and logging of commit times</li> - <li><a href="https://github.com/fluo-io/fluo/pull/584";>#584</a> - Adjust number of conditional writer threads based on cluster size</li> -</ul> - -<p>For Accumulo, changes are being made in <a href="https://issues.apache.org/jira/browse/ACCUMULO-4066";>ACCUMULO-4066</a> to decrease the time it -takes to process conditional mutations. Conditional mutations are used when -Fluo commits a transaction.</p> - -<p>These changes resulted in nice improvements over beta-1 in testing. However there -is probably still room for improvement. More analysis is needed.</p> - -<h4 id="api-changes">API Changes</h4> - -<p>Once Fluo 1.0.0 is released, all releases after that will follow -<a href="http://semver.org/";>semver</a>. For now some small API changes are still being -made. The following API changes happened between beta-1 and beta-2.</p> - -<ul> - <li><a href="https://github.com/fluo-io/fluo/pull/566";>#566</a> - Added RowColumnValue and made Accumulo init code use it</li> - <li><a href="https://github.com/fluo-io/fluo/pull/551";>#551</a> - Added method to get start timestamp of transaction</li> - <li><a href="https://github.com/fluo-io/fluo/pull/550";>#550</a> - Changed setObservers() to addObservers()</li> -</ul> - -<h4 id="other-important-improvements-and-bug-fixes">Other important improvements and bug fixes</h4> - -<ul> - <li><a href="https://github.com/fluo-io/fluo/pull/598";>#598</a> - Upgraded Hadoop to 2.6.3 and Accumulo to 1.6.4</li> - <li><a href="https://github.com/fluo-io/fluo/pull/587";>#587</a> - Specified datasource for all graphs in fluoâs Grafana dashboard</li> - <li><a href="https://github.com/fluo-io/fluo/pull/586";>#586</a> - Added efficient and easy way to build Bytes objects</li> - <li><a href="https://github.com/fluo-io/fluo/pull/578";>#578</a> - Plot nothing in Grafana when no data exists</li> - <li><a href="https://github.com/fluo-io/fluo/pull/573";>#573</a> - Fixed issues building against Accumulo 1.8.0-SNAPSHOT</li> - <li><a href="https://github.com/fluo-io/fluo/pull/561";>#561</a> - Stopped checkstyle mvn plugin from running at validate</li> - <li><a href="https://github.com/fluo-io/fluo/issues/559";>#559</a> - Eventually drop deleted data</li> - <li><a href="https://github.com/fluo-io/fluo/pull/558";>#558</a> - Added arguments to deploy command to skip findbugs, checkstyle, and auto-formatting</li> - <li><a href="https://github.com/fluo-io/fluo/issues/556";>#556</a> - Make TravisCI deploy snapshot jars after successful builds</li> - <li><a href="https://github.com/fluo-io/fluo/pull/552";>#552</a> - Made eclipse stop complaining about unknown plugins</li> - <li><a href="https://github.com/fluo-io/fluo/issues/547";>#547</a> - Provide better documentation for LoaderExecutor</li> - <li><a href="https://github.com/fluo-io/fluo/pull/535";>#535</a> - Upgraded Twill to 0.6.0-incubating</li> - <li><a href="https://github.com/fluo-io/fluo/issues/520";>#520</a> - Consolidate all implementation properties into FluoConfigurationImpl</li> - <li><a href="https://github.com/fluo-io/fluo/issues/518";>#518</a> - Make Oracle run on a random port</li> - <li><a href="https://github.com/fluo-io/fluo/issues/513";>#513</a> - Unable to pass spaces to scan command</li> - <li><a href="https://github.com/fluo-io/fluo/issues/495";>#495</a> - Add support for notifications to Fluo formatter</li> -</ul> - -<h2 id="testing">Testing</h2> - -<p>For this release, a long run of the <a href="https://github.com/fluo-io/webindex";>Webindex</a> application was performed and is documented in -<a href="/blog/2016/01/11/webindex-long-run/">a blog post</a>. A long run of <a href="https://github.com/fluo-io/fluo-stress";>Fluo stress</a> was run and documented in -<a href="/blog/2015/12/22/beta-2-pre-release-stress-test/">another blog post</a>.</p> - - -</div> - -<hr> - -<div> - <p>All Fluo release notes can be viewed in the <a href="/release-notes/">archive</a></p> -</div> - - </div> - </div> - <hr> - <div class="row footer"> - <div class="col-sm-12 text-center"> - <div class="center-block"> - <a href="https://apache.org";><img src="/resources/feather.png" alt="Apache"></a> - Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a> - </div> - </div> - </div> - </div> - <script src="/javascripts/jquery.min.js"></script> - <script src="/javascripts/bootstrap.min.js"></script> - <!-- Place your <script> tags here. --> - -<!-- Google Analytics --> -<script> - (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ - (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), - m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) - })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); - - ga('create', 'UA-55360307-1', 'auto'); - ga('send', 'pageview'); - -</script> - - </body> -</html> http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/release-notes/fluo-1.0.0-alpha-1/index.html ---------------------------------------------------------------------- diff --git a/release-notes/fluo-1.0.0-alpha-1/index.html b/release-notes/fluo-1.0.0-alpha-1/index.html new file mode 100644 index 0000000..f77a2a9 --- /dev/null +++ b/release-notes/fluo-1.0.0-alpha-1/index.html @@ -0,0 +1,308 @@ +<!DOCTYPE html> +<html> + <head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <link rel="stylesheet" href="/stylesheets/fluo.css"> + <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400";> + <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather";> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css"; > + <link rel="canonical" href="http://fluo.io/release-notes/fluo-1.0.0-alpha-1/";> + <link rel="icon" type="image/png" href="/resources/favicon.png"> + <title>Fluo 1.0.0-alpha-1 Release Notes | Apache Fluo</title> + <!--[if lt IE 9]> + <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> + <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> + <![endif]--> + </head> + <body> + <div id="fluo-nav" class="navbar navbar-default navbar-static-top"> + <div class="container"> + <div class="navbar-header"> + <div class="navbar-toggle-wrapper visible-xs"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse"> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + </div> + <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a> + </div> + <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> + <ul class="navbar-nav nav"> + <li><a href="/release/">Releases</a></li> + <li><a href="/tour/">Tour</a></li> + <li><a href="/docs/">Docs</a></li> + <li><a href="/apidocs/">API</a></li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="/getinvolved/">Get Involved</a></li> + <li><a href="/blog/">Blog</a></li> + <li><a href="/people/">People</a></li> + <li><a href="/related-projects/">Related Projects</a></li> + <li><a href="/poweredby/">Powered By</a></li> + </ul> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="/how-to-contribute/">How To Contribute</a></li> + <li><a href="/release-process/">Release Process</a></li> + </ul> + </li> + </ul> + <ul class="navbar-nav nav navbar-right"> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="https://www.apache.org";>Apache Homepage</a></li> + <li><a href="https://www.apache.org/licenses/LICENSE-2.0";>License</a></li> + <li><a href="https://www.apache.org/foundation/sponsorship";>Sponsorship</i></a></li> + <li><a href="https://www.apache.org/security";>Security</a></li> + <li><a href="https://www.apache.org/foundation/thanks";>Thanks</a></li> + <li><a href="https://www.apache.org/foundation/policies/conduct";>Code of Conduct</a></li> + </ul> + </li> + </ul> + </div> + </div> + </div> + <div class="container"> + <div class="row"> + <div class="col-sm-12"> + <div class="post-header"> + <h2>Fluo 1.0.0-alpha-1 Release Notes</h2> + <p class="text-muted">02 Oct 2014</p> +</div> + + + <div class="alert alert-danger" role="alert">These release notes are for a release of Fluo that made before Apache incubation.</div> + + +<div class="post-content"> + <p>Fluo 1.0.0-alpha-1 is the first official release of Fluo. It contains 83 initial tickets marked for this first milestone. The roadmap for this release included a working initial implementation, completing an initial pass at the client API (which will be finalized in 1.0.0), and writing a stress test application. This release has not been tested much at scale. More scale and peformance testing will be done for the beta release.</p> + +<h2 id="significant-features">Significant features</h2> + +<p>This alpha release contained a lot of initial features to help establish the basic implementation and client API. All of these features will be expected to become more robust over time.</p> + +<h4 id="minifluo">MiniFluo</h4> + +<p>MiniAccumulo and MiniFluo make it easy to start a local Fluo instance for integration testing. MiniFluo also makes it easy for developers to experiment with Fluo locally without installing it on an Accumulo cluster.</p> + +<h4 id="garbage-collection">Garbage Collection</h4> + +<p>Without garbage collection, the columnns in a Fluo table would grow without bound as updates are applied over time. Because of this, itâs important to clean up old fully-processed (and no longer being read) transaction data. The initial implementation of garbage collection kept N versions of columns. <a href="https://github.com/fluo-io/fluo/issues/8";>Fluo-8</a> added the ability to garbage collect all column versions that are no longer being read by transactions anywhere else in the cluster.</p> + +<h4 id="mapreduce-inputformat">MapReduce InputFormat</h4> + +<p>The FluoInputFormat allows a specific snapshot to be read into a mapreduce job. It was established in <a href="https://github.com/fluo-io/fluo/issues/7";>Fluo-7</a>.</p> + +<h4 id="fluofileoutputformat-and-fluooutputformat">FluoFileOutputFormat and FluoOutputFormat</h4> + +<p>The FluoFileOuputFormat enables the bulk ingest of a Fluo table using mapreduce by creating the Accumulo r-files in HDFS. The FluoOutputFormat pushes keys directly into the Accumulo tablet servers through the client API. <a href="https://github.com/fluo-io/fluo/issues/35";>Fluo-35</a> added this feature.</p> + +<h4 id="fluo-workers-and-oracle-running-in-yarn">Fluo Workers and Oracle running in Yarn</h4> + +<p>Fluo can use Yarn to manage fault tolerance and guarantee multiple worker processes and some number of Oracle processes are kept running across the cluster. <a href="http://twill.incubator.apache.org/";>Apache Twill</a> was used for the Yarn integration. <a href="https://github.com/fluo-io/fluo/issues/11";>Fluo-11</a> added an application that will deploy Fluo to a Yarn cluster.</p> + +<h4 id="fluo-api">Fluo API</h4> + +<p>Fluo provides a fluent-builder API for creating transactions to read and mutate data of various different data types in the Fluo table. This API was finalized in <a href="https://github.com/fluo-io/fluo/issues/23";>Fluo-23</a> and <a href="https://github.com/fluo-io/fluo/issues/98";>Fluo-98</a> when it was moved into its own module. It includes a type layer (finished in <a href="https://github.com/fluo-io/fluo/issues/153";>Fluo-153</a>) that makes it easier to work with different types, default values, null return types, and row/column maps.</p> + +<h4 id="stress-test">Stress test</h4> + +<p>A <a href="https://github.com/fluo-io/fluo/blob/1.0.0-alpha-1/modules/stress/README.md";>stress test</a> was created that uses a tree to force collisions on transactions trying to mutate multiple columns at the same time. This was established in <a href="https://github.com/fluo-io/fluo/issues/24";>Fluo-24</a>.</p> + +<p>Other important improvements and features:</p> + +<ul> + <li><a href="https://github.com/fluo-io/fluo/issues/1";>Fluo-1</a> - Create scripts for starting/stopping services</li> + <li><a href="https://github.com/fluo-io/fluo/issues/3";>Fluo-3</a> - use Curator for leader election and leader discovery</li> + <li><a href="https://github.com/fluo-io/fluo/issues/13";>Fluo-13</a> - Register clients that perform transactions in Zookeeper</li> + <li><a href="https://github.com/fluo-io/fluo/issues/15";>Fluo-15</a> - Leader election for the Oracle Server</li> + <li><a href="https://github.com/fluo-io/fluo/issues/16";>Fluo-16</a> - Allow observer notification to be configured in AbstractObserver class</li> + <li><a href="https://github.com/fluo-io/fluo/issues/89";>Fluo-89</a> - OracleApp should allow configurable number of instances</li> + <li><a href="https://github.com/fluo-io/fluo/issues/98";>Fluo-98</a> - Create a Range class for Fluo</li> + <li><a href="https://github.com/fluo-io/fluo/issues/123";>Fluo-123</a> - Hide implementation of Bytes in the API</li> + <li><a href="https://github.com/fluo-io/fluo/issues/130";>Fluo-130</a> - Expose MiniFluo in public API</li> + <li><a href="https://github.com/fluo-io/fluo/issues/131";>Fluo-131</a> - Make LoaderExector implement closeable</li> + <li><a href="https://github.com/fluo-io/fluo/issues/147";>Fluo-147</a> - Create fluo-accumulo module for Accumulo iterators and formatter</li> + <li><a href="https://github.com/fluo-io/fluo/issues/175";>Fluo-175</a> - Add newTransaction() method to FluoClient</li> + <li><a href="https://github.com/fluo-io/fluo/issues/186";>Fluo-186</a> - Add Exceptions to API</li> +</ul> + +<h2 id="testing">Testing</h2> + +<p>All unit and integration tests passed. Two <a href="https://github.com/fluo-io/fluo/blob/1.0.0-alpha-1/modules/stress/README.md";>stress tests</a> were run successfully on a 20-node cluster with one test generating 2000 integers and another generating 5000 integers. The 5000 integer test took about 10 minutes to run after the MapReduce load of integers was complete.</p> + +<h2 id="current-status-of-percolator-implementation">Current Status of Percolator Implementation</h2> + +<p>The following tables identify the features necessary for a full implementation of Percolator and the current state of Fluo with regards to the implementation. For alpha, a test to excercise some of these features at scale was created, however, it was not run at scale for long periods. One of the goals for beta is to have at least a 24hr run of the scale test, on at least 20 nodes, succeed.</p> + +<h3 id="necessary-features">Necessary features:</h3> + +<table> + <thead> + <tr> + <th>Percolator Feature</th> + <th>Details</th> + <th>Testing</th> + </tr> + </thead> + <tbody> + <tr> + <td>ACID-compliant cross-row transactions</td> + <td>Provides distributed transactions with ACID guarantees that work across Accumulo rows. Rows are distributed across tablet servers.</td> + <td>IT,CST,<a href="https://github.com/fluo-io/fluo/issues/240";>#240</a><sup>1</sup></td> + </tr> + <tr> + <td>Observers</td> + <td>Perform transactions by sending read/write RPCs to Accumulo tablet servers, which in turn send read/write RPCs to HDFS.</td> + <td>IT,CST</td> + </tr> + <tr> + <td>Worker Process</td> + <td>Scans Accumulo for changed columns and invokes necessary observers as function calls on the worker.</td> + <td>IT,CST</td> + </tr> + <tr> + <td>Timestamp Oracle</td> + <td>Provides strictly increasing timestamps; a property required for correct operation of the snapshot isolation protocol.</td> + <td>IT,CST</td> + </tr> + <tr> + <td>Oracle Failover</td> + <td>When an active Oracle process dies, timestamps should be served from a different Oracle. They should never go backwards.</td> + <td>IT,<a href="https://github.com/fluo-io/fluo/issues/37";>#37</a>,<a href="https://github.com/fluo-io/fluo/issues/241";>#241</a><sup>1</sup></td> + </tr> + <tr> + <td>Lightweight Lock Service</td> + <td>Makes searches for dirty notifications more efficient. Zookeeper is used in this case. Should be low-latency, replicated, balanced, and persist to a backing store.</td> + <td>IT,CST</td> + </tr> + <tr> + <td>Multithreaded Transasctions</td> + <td>Allows highly parallel execution of transactions and synchronization only during get() and commit() upon which transactions actually collide.</td> + <td>IT,CST</td> + </tr> + <tr> + <td>Snapshot Isolation</td> + <td>Non-serializable isolation mostly implemented through MVCC provided through Accumuloâs timestamp portion of the key.</td> + <td>IT,CST,<a href="https://github.com/fluo-io/fluo/issues/240";>#240</a><sup>1</sup></td> + </tr> + <tr> + <td>Roll-back / Roll-forward semantics</td> + <td>Failures during the first phase of commit cause transactions to be rolled back while failures during the second phase can cause them to be rolled forward.</td> + <td>IT,<a href="https://github.com/fluo-io/fluo/issues/240";>#240</a><sup>1</sup></td> + </tr> + </tbody> +</table> + +<p><sup>1</sup> This feature has been at least partially tested.</p> + +<h3 id="performance-enhancements">Performance enhancements:</h3> + +<table> + <thead> + <tr> + <th>Enhancement</th> + <th>Details</th> + <th>Testing</th> + </tr> + </thead> + <tbody> + <tr> + <td>Locality Group for NOTIFY column</td> + <td>Add an extra index in the underlying r-files for the notify column where it can be scanned faster by workers.</td> + <td>IT,CST</td> + </tr> + <tr> + <td>Worker scan conflict resolution</td> + <td>Workers register random tablets they choose to scan. When two workers generate the same tablet, one of them re-generates a new random tablet.</td> + <td><a href="https://github.com/fluo-io/fluo/issues/5";>#5</a></td> + </tr> + <tr> + <td>Worker advertisement</td> + <td>Workers advertise through the lock service so that the system can know if they failed and any transactions upon which they are working can be rolled back or rolled forward</td> + <td>IT</td> + </tr> + <tr> + <td>Oracle batching</td> + <td>Timestamps get allocated in batch to workers. Each worker should share one connection to Oracle to limit RPCs across threads</td> + <td>IT,CST</td> + </tr> + <tr> + <td>Weak Notifications</td> + <td>Allows notification of a cell without the possibility of transactional conflict through a special type of column that can be notified but not written to</td> + <td>IT</td> + </tr> + <tr> + <td>Conditional Mutations</td> + <td>Allows READ-MODIFY-WRITE of a mutation in a single RPC.</td> + <td>IT,CST</td> + </tr> + <tr> + <td>Conditional Mutation batching</td> + <td>Delay sending of conditional mutations destined for same tablet server so they can be batched. Lowers number of RPCs, adds latency but provides better parallelism.</td> + <td>IT,CST</td> + </tr> + <tr> + <td>Read batching</td> + <td>Delaying of read operations on tablet server for batching</td> + <td><a href="https://github.com/fluo-io/fluo/issues/138";>#138</a></td> + </tr> + <tr> + <td>Prefetching & read-ahead caching</td> + <td>Prediction made each time a column is read, which other columns will be read in the same transaction based on previous behavior.</td> + <td><a href="https://github.com/fluo-io/fluo/issues/12";>#12</a></td> + </tr> + </tbody> +</table> + +<ul> + <li>IT = Integration tests, CST = Cluster stress tests</li> + <li>Testing columns providing a ticket number have not yet been implemented.</li> +</ul> + + +</div> + +<hr> + +<div> + <p>All Fluo release notes can be viewed in the <a href="/release-notes/">archive</a></p> +</div> + + </div> + </div> + <hr> + <div class="row footer"> + <div class="col-sm-12 text-center"> + <div class="center-block"> + <a href="https://apache.org";><img src="/resources/feather.png" alt="Apache"></a> + Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a> + </div> + </div> + </div> + </div> + <script src="/javascripts/jquery.min.js"></script> + <script src="/javascripts/bootstrap.min.js"></script> + <!-- Place your <script> tags here. --> + +<!-- Google Analytics --> +<script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); + + ga('create', 'UA-55360307-1', 'auto'); + ga('send', 'pageview'); + +</script> + + </body> +</html> http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/release-notes/fluo-1.0.0-beta-1/index.html ---------------------------------------------------------------------- diff --git a/release-notes/fluo-1.0.0-beta-1/index.html b/release-notes/fluo-1.0.0-beta-1/index.html new file mode 100644 index 0000000..c337b0d --- /dev/null +++ b/release-notes/fluo-1.0.0-beta-1/index.html @@ -0,0 +1,245 @@ +<!DOCTYPE html> +<html> + <head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <link rel="stylesheet" href="/stylesheets/fluo.css"> + <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400";> + <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather";> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css"; > + <link rel="canonical" href="http://fluo.io/release-notes/fluo-1.0.0-beta-1/";> + <link rel="icon" type="image/png" href="/resources/favicon.png"> + <title>Fluo 1.0.0-beta-1 Release Notes | Apache Fluo</title> + <!--[if lt IE 9]> + <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> + <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> + <![endif]--> + </head> + <body> + <div id="fluo-nav" class="navbar navbar-default navbar-static-top"> + <div class="container"> + <div class="navbar-header"> + <div class="navbar-toggle-wrapper visible-xs"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse"> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + </div> + <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a> + </div> + <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> + <ul class="navbar-nav nav"> + <li><a href="/release/">Releases</a></li> + <li><a href="/tour/">Tour</a></li> + <li><a href="/docs/">Docs</a></li> + <li><a href="/apidocs/">API</a></li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="/getinvolved/">Get Involved</a></li> + <li><a href="/blog/">Blog</a></li> + <li><a href="/people/">People</a></li> + <li><a href="/related-projects/">Related Projects</a></li> + <li><a href="/poweredby/">Powered By</a></li> + </ul> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="/how-to-contribute/">How To Contribute</a></li> + <li><a href="/release-process/">Release Process</a></li> + </ul> + </li> + </ul> + <ul class="navbar-nav nav navbar-right"> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="https://www.apache.org";>Apache Homepage</a></li> + <li><a href="https://www.apache.org/licenses/LICENSE-2.0";>License</a></li> + <li><a href="https://www.apache.org/foundation/sponsorship";>Sponsorship</i></a></li> + <li><a href="https://www.apache.org/security";>Security</a></li> + <li><a href="https://www.apache.org/foundation/thanks";>Thanks</a></li> + <li><a href="https://www.apache.org/foundation/policies/conduct";>Code of Conduct</a></li> + </ul> + </li> + </ul> + </div> + </div> + </div> + <div class="container"> + <div class="row"> + <div class="col-sm-12"> + <div class="post-header"> + <h2>Fluo 1.0.0-beta-1 Release Notes</h2> + <p class="text-muted">09 Jun 2015</p> +</div> + + + <div class="alert alert-danger" role="alert">These release notes are for a release of Fluo that made before Apache incubation.</div> + + +<div class="post-content"> + <p>Fluo 1.0.0-beta-1 is the second official release of Fluo. The release closed <a href="https://github.com/fluo-io/fluo/issues?q=milestone%3A1.0.0-beta-1+is%3Aclosed";>133 tickets</a>. This release is not recommended for production use.</p> + +<h2 id="significant-features">Significant features</h2> + +<p>This release contains many new features that makes it easier to run, develop, and monitor Fluo applications.</p> + +<h4 id="simplified-fluo-administration-on-a-local-machine-or-ec2-cluster">Simplified Fluo administration on a local machine or EC2 cluster</h4> + +<p>Developers can now run Fluo and its dependencies on their local machine (<a href="https://github.com/fluo-io/fluo/issues/92";>#92</a>) or an AWS EC2 cluster (<a href="https://github.com/fluo-io/fluo/issues/356";>#356</a>) using a few simple commands. +This was done by creating two administration tools called <a href="https://github.com/fluo-io/fluo-dev";>Fluo-dev</a> and <a href="https://github.com/fluo-io/fluo-deploy";>Fluo-deploy</a> whose scripts and configuration reside in repos +separate from the Fluo code base. These tools allow developers to collaborate and share configuration for running Fluo.</p> + +<h4 id="transaction-metrics-are-viewable-using-common-monitoring-tools">Transaction metrics are viewable using common monitoring tools</h4> + +<p>Fluo now publishes metrics (<a href="https://github.com/fluo-io/fluo/issues/20";>#20</a>) about transactions, collisions, and timestamps using <a href="https://dropwizard.github.io/metrics/3.1.0/";>Dropwizard metrics</a>. These metrics are by default published +using JMX and are viewable using JConsole or JVisualVM. Fluo can also be configured to publish metrics to Graphite or Ganglia. View the <a href="https://github.com/fluo-io/fluo/blob/1.0.0-beta-1/docs/metrics.md";>metrics documentation</a> +for more information.</p> + +<h4 id="improved-processing-of-notifications-in-fluo-workers">Improved processing of notifications in Fluo workers</h4> + +<p>Fluo workers were refactored to separate the code used for finding and executing work. Each worker uses a single thread for finding +work (<a href="https://github.com/fluo-io/fluo/issues/19";>#19</a>). A new method was introduced to partition work among workers using a hash+mod of notifications (<a href="https://github.com/fluo-io/fluo/issues/282";>#282</a>). +The commit message for <a href="https://github.com/fluo-io/fluo/commit/4100e236b8438350e30eda924a6360e2c722ae37";>4100e23</a> contains a good description of some of the benefits and drawback of the current +hashing approach.</p> + +<h4 id="improved-the-deletion-of-observer-notifications">Improved the deletion of observer notifications</h4> + +<p>When a cell is deleted in Accumulo, a delete marker is inserted. Delete markers stay around until +all files in a tablet are compacted. For Fluo this could cause a lot of notification delete markers +to build up over time. To avoid this buildup, the way Fluo deletes notifications was changed in +(<a href="https://github.com/fluo-io/fluo/issues/457";>#457</a>). Accumulo delete markers are no longer used, Fluo now uses a custom delete marker for +notifications. The custom deleter marker allows Fluo to do analysis when Accumulo flushes memory to +disk and avoid writing many delete markers to persistent storage.</p> + +<h4 id="easier-management-of-fluo-from-the-command-line">Easier management of Fluo from the command line</h4> + +<p>Fluo now provides different scripts (<strong>fluo</strong>, <strong>mini-fluo</strong>, & <strong>local-fluo</strong>) for managing Fluo using a YARN cluster, MiniFluo, or local processes. Several commands +were created for these scripts. A <strong>scan</strong> command allows users to print a snapshot of a Fluo table (<a href="https://github.com/fluo-io/fluo/issues/319";>#319</a>). A <strong>info</strong> command shows locations of containers +when running Fluo in YARN (<a href="https://github.com/fluo-io/fluo/issues/297";>#297</a>). A <strong>classpath</strong> command gives users a list of jars needed to execute Fluo client code (<a href="https://github.com/fluo-io/fluo/issues/436";>#436</a>). A <strong>wait</strong> command will +sleep until all notifications are processed (<a href="https://github.com/fluo-io/fluo/issues/434";>#434</a>).</p> + +<h4 id="support-for-running-multiple-fluo-applications-on-a-single-cluster">Support for running multiple Fluo applications on a single cluster</h4> + +<p>Users can now run multiple Fluo applications using a single cluster (<a href="https://github.com/fluo-io/fluo/issues/454";>#454</a>). This enables different Fluo users to share the same cluster. Fluo applications +can be started and stopped independently. Each application has its own configuration.</p> + +<h4 id="fluo-build-improvements">Fluo build improvements</h4> + +<p>On each build, all Java code is automatically formatted based on Google Java Style (<a href="https://github.com/fluo-io/fluo/issues/479";>#479</a>). Also, checkstyle and findbugs will fail the build if certain +standards are not reached (<a href="https://github.com/fluo-io/fluo/issues/185";>#185</a>). The POM is also sorted (<a href="https://github.com/fluo-io/fluo/issues/493";>#493</a>).</p> + +<h4 id="organized-fluo-code-base">Organized Fluo code base</h4> + +<p>The Fluo stress test was moved to its own <a href="https://github.com/fluo-io/fluo-stress";>repo</a> and is no longer a sub-module (<a href="https://github.com/fluo-io/fluo/issues/385";>#385</a>). MiniFluo was moved from fluo-core to the fluo-mini +module/jar (<a href="https://github.com/fluo-io/fluo/issues/439";>#439</a>). This reduced the number of dependencies in fluo-core. However, developers will now need to include the fluo-mini jar in their Maven +POM if they start MiniFluo.</p> + +<h4 id="fluo-testing-improvements">Fluo testing improvements</h4> + +<p>Integration tests can now be run from Eclipse (<a href="https://github.com/fluo-io/fluo/issues/322";>#322</a>). Several new <a href="https://github.com/fluo-io/fluo/issues?utf8=%E2%9C%93&q=milestone%3A1.0.0-beta-1+is%3Aclosed+%22unit+test%22";>unit tests</a> were created.</p> + +<h4 id="other-important-improvements-and-bug-fixes">Other important improvements and bug fixes</h4> + +<ul> + <li><a href="https://github.com/fluo-io/fluo/issues/470";>#470</a> - Replaced FluoFileOutputFormat with an Accumulo Key/Value generator</li> + <li><a href="https://github.com/fluo-io/fluo/issues/460";>#460</a> - Reduced Fluo API module dependencies</li> + <li><a href="https://github.com/fluo-io/fluo/issues/456";>#456</a> - Fixed bug with notifications being lost when processes died</li> + <li><a href="https://github.com/fluo-io/fluo/issues/446";>#446</a> - Simplified log configuration and configure rolling log files in YARN</li> + <li><a href="https://github.com/fluo-io/fluo/issues/442";>#442</a> - Reduced the number of curator clients in FluoAdmin</li> + <li><a href="https://github.com/fluo-io/fluo/issues/383";>#383</a> - Improved transaction logging to help users debug collisions. See <a href="https://github.com/fluo-io/fluo/blob/1.0.0-beta-1/docs/applications.md#debugging-applications";>debugging documentation</a>.</li> + <li><a href="https://github.com/fluo-io/fluo/issues/365";>#365</a> - Analyze Fluo code to see what non-public Accumulo APIs are used</li> + <li><a href="https://github.com/fluo-io/fluo/issues/362";>#362</a> - Made API data objects immutable</li> + <li><a href="https://github.com/fluo-io/fluo/issues/349";>#349</a> - Support application level configuration in fluo.properties</li> + <li><a href="https://github.com/fluo-io/fluo/issues/342";>#342</a> - Add a configurable retry timeout to Fluo clients</li> + <li><a href="https://github.com/fluo-io/fluo/issues/294";>#294</a> - Fluo now uses chroot suffix in its Zookeeper connection.</li> + <li><a href="https://github.com/fluo-io/fluo/issues/293";>#293</a> - Add argument checking to FluoConfiguration</li> + <li><a href="https://github.com/fluo-io/fluo/issues/244";>#244</a> - Make re-initialization easier for user</li> +</ul> + +<h2 id="testing">Testing</h2> + +<p>A successful long stress test run was conducted using Fluo built from commit +<a href="https://github.com/fluo-io/fluo/commit/fb647dd6a470e8015654f8ed99b9196f5f49582a";>fb647dd</a>. The test ran very well and never fell behind like a +<a href="/blog/2014/12/30/stress-test-long-run/">previous long run</a> of stress did. The test had the following +properties.</p> + +<ul> + <li>Initialized stress test using 1 billion random integers.</li> + <li>Ran 150 incremental loads of 100 thousand integers. Slept 3 minutes between loads.</li> + <li>Used 19 m3.xlarge nodes on EC2. 16 workers and 3 masters</li> + <li>Configuration for the test committed and tagged in git : <a href="https://github.com/keith-turner/fluo-deploy/tree/beta-long-test-1";>fluo-deploy tag</a> and <a href="https://github.com/keith-turner/fluo-stress/tree/beta-long-test-1";>fluo-stress tag</a></li> + <li>Opened two issues as a result of test <a href="https://github.com/fluo-io/fluo/issues/499";>#499</a> and <a href="https://github.com/fluo-io/fluo-stress/issues/30";>fluo-stress#30</a></li> +</ul> + +<p>Below is the trailing output from running the test.</p> + +<div class="highlighter-rouge"><pre class="highlight"><code>*****Generating and loading incremental data set 148***** +*****Generating and loading incremental data set 149***** +*****Generating and loading incremental data set 150***** +*****Calculating # of unique integers using MapReduce***** + UNIQUE=1014486419 +*****Wait for Fluo to finish processing***** +05:33:40.158 [main] INFO io.fluo.cluster.runner.AppRunner - The wait command will exit when all notifications are processed +05:33:40.417 [Thread-3] INFO io.fluo.core.oracle.OracleClient - Connected to oracle at worker4:9913 +05:33:41.308 [main] INFO io.fluo.cluster.runner.AppRunner - All processing has finished! +*****Printing # of unique integers calculated by Fluo***** +Total at root : 1014486419 +Nodes Scanned : 59605 +*****Verifying Fluo & MapReduce results match***** +Success! Fluo & MapReduce both calculated 1014486419 unique integers +</code></pre> +</div> + +<p>The test ran for a little over 12 hours. Below are two plots pulled from +graphite showing the number of notifications queued and transaction rate over +the entire test run. Load transactions are not included in the rate. The rate +is transactions per second. The accuracy of these plots is uncertain because +no graphite configuration changes were made. The plots do seem within the +ballpark.</p> + +<p><img src="/resources/release-notes/1.0.0-beta-1/queued.png" alt="Notifications Queued" title="Notifications Queued" /> <img src="/resources/release-notes/1.0.0-beta-1/rate.png" alt="Transaction rate" title="Transaction Rate" /></p> + + +</div> + +<hr> + +<div> + <p>All Fluo release notes can be viewed in the <a href="/release-notes/">archive</a></p> +</div> + + </div> + </div> + <hr> + <div class="row footer"> + <div class="col-sm-12 text-center"> + <div class="center-block"> + <a href="https://apache.org";><img src="/resources/feather.png" alt="Apache"></a> + Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a> + </div> + </div> + </div> + </div> + <script src="/javascripts/jquery.min.js"></script> + <script src="/javascripts/bootstrap.min.js"></script> + <!-- Place your <script> tags here. --> + +<!-- Google Analytics --> +<script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); + + ga('create', 'UA-55360307-1', 'auto'); + ga('send', 'pageview'); + +</script> + + </body> +</html> http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/ebd1f593/release-notes/fluo-1.0.0-beta-2/index.html ---------------------------------------------------------------------- diff --git a/release-notes/fluo-1.0.0-beta-2/index.html b/release-notes/fluo-1.0.0-beta-2/index.html new file mode 100644 index 0000000..725ae3c --- /dev/null +++ b/release-notes/fluo-1.0.0-beta-2/index.html @@ -0,0 +1,226 @@ +<!DOCTYPE html> +<html> + <head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <link rel="stylesheet" href="/stylesheets/fluo.css"> + <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400";> + <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather";> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css"; > + <link rel="canonical" href="http://fluo.io/release-notes/fluo-1.0.0-beta-2/";> + <link rel="icon" type="image/png" href="/resources/favicon.png"> + <title>Fluo 1.0.0-beta-2 Release Notes | Apache Fluo</title> + <!--[if lt IE 9]> + <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script> + <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script> + <![endif]--> + </head> + <body> + <div id="fluo-nav" class="navbar navbar-default navbar-static-top"> + <div class="container"> + <div class="navbar-header"> + <div class="navbar-toggle-wrapper visible-xs"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse"> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + </div> + <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a> + </div> + <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px"> + <ul class="navbar-nav nav"> + <li><a href="/release/">Releases</a></li> + <li><a href="/tour/">Tour</a></li> + <li><a href="/docs/">Docs</a></li> + <li><a href="/apidocs/">API</a></li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="/getinvolved/">Get Involved</a></li> + <li><a href="/blog/">Blog</a></li> + <li><a href="/people/">People</a></li> + <li><a href="/related-projects/">Related Projects</a></li> + <li><a href="/poweredby/">Powered By</a></li> + </ul> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="/how-to-contribute/">How To Contribute</a></li> + <li><a href="/release-process/">Release Process</a></li> + </ul> + </li> + </ul> + <ul class="navbar-nav nav navbar-right"> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="https://www.apache.org";>Apache Homepage</a></li> + <li><a href="https://www.apache.org/licenses/LICENSE-2.0";>License</a></li> + <li><a href="https://www.apache.org/foundation/sponsorship";>Sponsorship</i></a></li> + <li><a href="https://www.apache.org/security";>Security</a></li> + <li><a href="https://www.apache.org/foundation/thanks";>Thanks</a></li> + <li><a href="https://www.apache.org/foundation/policies/conduct";>Code of Conduct</a></li> + </ul> + </li> + </ul> + </div> + </div> + </div> + <div class="container"> + <div class="row"> + <div class="col-sm-12"> + <div class="post-header"> + <h2>Fluo 1.0.0-beta-2 Release Notes</h2> + <p class="text-muted">12 Jan 2016</p> +</div> + + + <div class="alert alert-danger" role="alert">These release notes are for a release of Fluo that made before Apache incubation.</div> + + +<div class="post-content"> + <p>Fluo 1.0.0-beta-2 is the third official release of Fluo. This release closed +<a href="https://github.com/fluo-io/fluo/issues?q=milestone%3A1.0.0-beta-2+is%3Aclosed";>48 tickets</a>. There is no upgrade path from 1.0.0-beta-1 to +1.0.0-beta-2.</p> + +<h2 id="significant-features">Significant features</h2> + +<p>This release contains many new features that makes it easier to run, develop, and monitor Fluo applications.</p> + +<h4 id="improved-fluo-metrics-that-can-be-sent-to-influxdb-and-viewed-in-grafana">Improved Fluo metrics that can be sent to InfluxDB and viewed in Grafana</h4> + +<p>In <a href="https://github.com/fluo-io/fluo/issues/569";>#569</a>, <a href="https://github.com/fluo-io/fluo/issues/570";>#570</a>, & <a href="https://github.com/fluo-io/fluo/issues/580";>#580</a>, Fluo metrics and monitoring were refactored to fix several +bugs and allow metrics to be sent to <a href="https://github.com/influxdata/influxdb";>InfluxDB</a> and viewed in <a href="http://grafana.org/";>Grafana</a>. Fluo metrics +are still instrumented using <a href="https://dropwizard.github.io/metrics/3.1.0/";>Dropwizard metrics</a> but in <a href="https://github.com/fluo-io/fluo/issues/574";>#574</a> metrics configuration was +moved from its own file and to now reside in <code class="highlighter-rouge">fluo.properties</code>. While Fluo metrics can still be sent to +many different tools (i.e Graphite, Ganglia), Fluo now ships with configuration that can be used to configure +a Fluo dashboard in Grafana that queries InfluxDB. To set up the sending of Fluo metrics to InfluxDB/Grafana, +view this <a href="/docs/fluo/1.0.0-beta-2/grafana/">documentation</a> or consider using <a href="https://github.com/fluo-io/fluo-dev";>Fluo-dev</a> or <a href="https://github.com/fluo-io/zetten";>Zetten</a> +to run Fluo as they can install InfluxDB+Grafana and setup metrics for you.</p> + +<h4 id="improved-fluo-administration">Improved Fluo administration</h4> + +<p>Several commands were added to the <code class="highlighter-rouge">fluo</code> script which is used to administer Fluo. A <code class="highlighter-rouge">fluo exec</code> command +(<a href="https://github.com/fluo-io/fluo/pull/581";>#581</a>) was created to provide an easy way to execute application code using Fluo classes and dependencies. +A <code class="highlighter-rouge">fluo list</code> command (<a href="https://github.com/fluo-io/fluo/issues/523";>#523</a>) was created to let users list all Fluo applications within a Fluo instance. +The <code class="highlighter-rouge">fluo scan</code> command now has a <code class="highlighter-rouge">--raw</code> option (<a href="https://github.com/fluo-io/fluo/issues/597";>#597</a>) that prints Fluo data as stored in Accumulo. This +was accomplished by moving the Fluo formatter from Accumulo shell to the scan command. The scan command can now +format non-ascii characters as hex (<a href="https://github.com/fluo-io/fluo/issues/568";>#568</a>). The <code class="highlighter-rouge">fluo new</code> command was improved to prevent users from +setting invalid Fluo application names (<a href="https://github.com/fluo-io/fluo/issues/510";>#510</a>). A bug was fixed in the <code class="highlighter-rouge">fluo start</code> command that was causing +time outs when starting applications (<a href="https://github.com/fluo-io/fluo/issues/562";>#562</a>). Finally, the <code class="highlighter-rouge">fluo</code> command can now be run without the <code class="highlighter-rouge">apps/</code> +directory configured for an application for most commands (<a href="https://github.com/fluo-io/fluo/issues/524";>#524</a>). Only the <code class="highlighter-rouge">init</code> and <code class="highlighter-rouge">start</code> commands need +this directory configured. All other commands only require the default properties file to be configured at +<code class="highlighter-rouge">conf/fluo.properties</code>.</p> + +<h4 id="made-fluo-work-better-with-spark">Made Fluo work better with Spark</h4> + +<p>Several changes were made to Fluo to allow it work better with Apache Spark. All Fluo data types now implement +<code class="highlighter-rouge">Serializable</code> and can be use in Spark RDDs (<a href="https://github.com/fluo-io/fluo/issues/539";>#539</a>). Fluo data types also now implement <code class="highlighter-rouge">Comparable</code> which +allows RDDs of Fluo data types to be sorted (<a href="https://github.com/fluo-io/fluo/issues/544";>#544</a>). Also, a no args constructor was created for the +MutableBytes data type to enable Kryo serialization in Spark (<a href="https://github.com/fluo-io/fluo/pull/549";>#549</a>). Finally, a new InputFormat called +<code class="highlighter-rouge">FluoEntryInputFormat</code> was created that returns key/value entries and the existing <code class="highlighter-rouge">FluoInputFormat</code> that returns +rows was renamed <code class="highlighter-rouge">FluoRowInputFormat</code> (<a href="https://github.com/fluo-io/fluo/issues/538";>#538</a>,<a href="https://github.com/fluo-io/fluo/issues/540";>#540</a>).</p> + +<h4 id="performance-improvements">Performance improvements</h4> + +<p>A good bit of time was spent analyzing Fluo while it was running to determine +where time is spent when executing transactions. Based on this analysis, it +was found that a good bit of time was spent committing transactions. Changes +were made in Fluo and Accumulo in order to decrease commit time. For Fluo, the +following changes were made :</p> + +<ul> + <li><a href="https://github.com/fluo-io/fluo/issues/591";>#591</a> - Shared batch writer increases transaction history</li> + <li><a href="https://github.com/fluo-io/fluo/pull/590";>#590</a> - Increased batch writer threads and made configurable</li> + <li><a href="https://github.com/fluo-io/fluo/pull/589";>#589</a> - Added 2nd conditional writer and logging of commit times</li> + <li><a href="https://github.com/fluo-io/fluo/pull/584";>#584</a> - Adjust number of conditional writer threads based on cluster size</li> +</ul> + +<p>For Accumulo, changes are being made in <a href="https://issues.apache.org/jira/browse/ACCUMULO-4066";>ACCUMULO-4066</a> to decrease the time it +takes to process conditional mutations. Conditional mutations are used when +Fluo commits a transaction.</p> + +<p>These changes resulted in nice improvements over beta-1 in testing. However there +is probably still room for improvement. More analysis is needed.</p> + +<h4 id="api-changes">API Changes</h4> + +<p>Once Fluo 1.0.0 is released, all releases after that will follow +<a href="http://semver.org/";>semver</a>. For now some small API changes are still being +made. The following API changes happened between beta-1 and beta-2.</p> + +<ul> + <li><a href="https://github.com/fluo-io/fluo/pull/566";>#566</a> - Added RowColumnValue and made Accumulo init code use it</li> + <li><a href="https://github.com/fluo-io/fluo/pull/551";>#551</a> - Added method to get start timestamp of transaction</li> + <li><a href="https://github.com/fluo-io/fluo/pull/550";>#550</a> - Changed setObservers() to addObservers()</li> +</ul> + +<h4 id="other-important-improvements-and-bug-fixes">Other important improvements and bug fixes</h4> + +<ul> + <li><a href="https://github.com/fluo-io/fluo/pull/598";>#598</a> - Upgraded Hadoop to 2.6.3 and Accumulo to 1.6.4</li> + <li><a href="https://github.com/fluo-io/fluo/pull/587";>#587</a> - Specified datasource for all graphs in fluoâs Grafana dashboard</li> + <li><a href="https://github.com/fluo-io/fluo/pull/586";>#586</a> - Added efficient and easy way to build Bytes objects</li> + <li><a href="https://github.com/fluo-io/fluo/pull/578";>#578</a> - Plot nothing in Grafana when no data exists</li> + <li><a href="https://github.com/fluo-io/fluo/pull/573";>#573</a> - Fixed issues building against Accumulo 1.8.0-SNAPSHOT</li> + <li><a href="https://github.com/fluo-io/fluo/pull/561";>#561</a> - Stopped checkstyle mvn plugin from running at validate</li> + <li><a href="https://github.com/fluo-io/fluo/issues/559";>#559</a> - Eventually drop deleted data</li> + <li><a href="https://github.com/fluo-io/fluo/pull/558";>#558</a> - Added arguments to deploy command to skip findbugs, checkstyle, and auto-formatting</li> + <li><a href="https://github.com/fluo-io/fluo/issues/556";>#556</a> - Make TravisCI deploy snapshot jars after successful builds</li> + <li><a href="https://github.com/fluo-io/fluo/pull/552";>#552</a> - Made eclipse stop complaining about unknown plugins</li> + <li><a href="https://github.com/fluo-io/fluo/issues/547";>#547</a> - Provide better documentation for LoaderExecutor</li> + <li><a href="https://github.com/fluo-io/fluo/pull/535";>#535</a> - Upgraded Twill to 0.6.0-incubating</li> + <li><a href="https://github.com/fluo-io/fluo/issues/520";>#520</a> - Consolidate all implementation properties into FluoConfigurationImpl</li> + <li><a href="https://github.com/fluo-io/fluo/issues/518";>#518</a> - Make Oracle run on a random port</li> + <li><a href="https://github.com/fluo-io/fluo/issues/513";>#513</a> - Unable to pass spaces to scan command</li> + <li><a href="https://github.com/fluo-io/fluo/issues/495";>#495</a> - Add support for notifications to Fluo formatter</li> +</ul> + +<h2 id="testing">Testing</h2> + +<p>For this release, a long run of the <a href="https://github.com/fluo-io/webindex";>Webindex</a> application was performed and is documented in +<a href="/blog/2016/01/11/webindex-long-run/">a blog post</a>. A long run of <a href="https://github.com/fluo-io/fluo-stress";>Fluo stress</a> was run and documented in +<a href="/blog/2015/12/22/beta-2-pre-release-stress-test/">another blog post</a>.</p> + + +</div> + +<hr> + +<div> + <p>All Fluo release notes can be viewed in the <a href="/release-notes/">archive</a></p> +</div> + + </div> + </div> + <hr> + <div class="row footer"> + <div class="col-sm-12 text-center"> + <div class="center-block"> + <a href="https://apache.org";><img src="/resources/feather.png" alt="Apache"></a> + Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a> + </div> + </div> + </div> + </div> + <script src="/javascripts/jquery.min.js"></script> + <script src="/javascripts/bootstrap.min.js"></script> + <!-- Place your <script> tags here. --> + +<!-- Google Analytics --> +<script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); + + ga('create', 'UA-55360307-1', 'auto'); + ga('send', 'pageview'); + +</script> + + </body> +</html>
