http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2014-03-06-accumulo-1.5.1.md ---------------------------------------------------------------------- diff --git a/_posts/release/2014-03-06-accumulo-1.5.1.md b/_posts/release/2014-03-06-accumulo-1.5.1.md new file mode 100644 index 0000000..71ca1a7 --- /dev/null +++ b/_posts/release/2014-03-06-accumulo-1.5.1.md @@ -0,0 +1,205 @@ +--- +title: Apache Accumulo 1.5.1 +redirect_from: /release_notes/1.5.1.html +--- + +Apache Accumulo 1.5.1 is a maintenance release on the 1.5 version branch. +This release contains changes from over 200 issues, comprised of bug fixes +(client side and server side), new test cases, and updated Hadoop support +contributed by over 30 different contributors and committers. +As this is a maintenance release, Apache Accumulo 1.5.1 has no client API +incompatibilities over Apache Accumulo 1.5.0 and requires no manual upgrade +process. Users of 1.5.0 are strongly encouraged to update as soon as possible +to benefit from the improvements. + + +## Notable Improvements + +While new features are typically not added in a bug-fix release as 1.5.1, the +community does create a variety of improvements that are API compatible. Contained +here are some of the more notable improvements. + +### PermGen Leak from Client API + +Accumulo's client code creates background threads that users presently cannot +stop through the API. This is quick to cause problems when invoking the Accumulo +API in application containers such as Apache Tomcat or JBoss and repeatedly +redeploying an application. [ACCUMULO-2128][3] introduces a static utility, +org.apache.accumulo.core.util.CleanUp, that users can invoke as part of a +teardown hook in their container that will stop these threads and avoid +the eventual OutOfMemoryError "PermGen space". + +### Prefer IPv4 when starting Accumulo processes + +While Hadoop [does not support IPv6 networks][28], attempting to run on a +system that does not have IPv6 completely disabled can cause strange failures. +[ACCUMULO-2262][4] invokes the JVM-provided configuration parameter at process +startup to prefer IPv4 over IPv6. + +### Memory units in configuration + +In previous versions, units of memory had to be provided as upper-case (e.g. '2G', not '2g'). +Additionally, a non-intuitive error was printed when a lower-case unit was provided. +[ACCUMULO-1933][7] allows lower-case memory units in all Accumulo configurations. + +### Apache Thrift maximum frame size + +Apache Thrift is used as the internal RPC service. [ACCUMULO-2360][14] allows +users to configure the maximum frame size an Accumulo server will read. This +prevents non Accumulo client from connecting and causing memory exhaustion. + +### MultiTableBatchWriter concurrency + +The MultiTableBatchWriter is a class which allows multiple tables to be written to +from a single object that maintains a single buffer for caching Mutations across all tables. This is desirable +as it greatly simplifies the JVM heap usage from caching Mutations across +many tables. Sadly, in Apache Accumulo 1.5.0, concurrent access to a single MultiTableBatchWriter +heavily suffered from synchronization issues. [ACCUMULO-1833][35] introduces a fix +which alleviates the blocking and idle-wait that previously occurred when multiple threads accessed +a single MultiTableBatchWriter instance concurrently. + +### Hadoop Versions + +Since Apache Accumulo 1.5.0 was released, Apache Hadoop 2.2.0 was also released +as the first generally available (GA) Hadoop 2 release. This was a very exciting release +for a number of reasons, but this also caused additional effort on Accumulo's part to +ensure that Apache Accumulo continues to work across multiple Hadoop versions. Apache Accumulo 1.5.1 +should function with any recent Hadoop 1 or Hadoop 2 without any special steps, tricks or instructions +required. + + +## Notable Bug Fixes + +As with any Apache Accumulo release, we have numerous bug fixes that have been fixed. Most +are very subtle and won't affect the common user; however, some notable bugs were resolved +as a part of 1.5.1 that are rather common. + +### Failure of ZooKeeper server in quorum kills connected Accumulo services + +Apache ZooKeeper provides a number of wonderful features that Accumulo uses to accomplish +a variety of tasks, most notably a distributed locking service. Typically, multiple ZooKeeper +servers are run to provide resilience against a certain number of node failures. [ACCUMULO-1572][13] +resolves an issue where Accumulo processes would kill themselves when the ZooKeeper server they +were communicating with died instead of failing over to another ZooKeeper server in the quorum. + +### Monitor table state isn't updated + +The Accumulo Monitor contains a column for the state of each table in the Accumulo instance. +The previous resolution was to restart the Monitor process when it got in this state. +[ACCUMULO-1920][25] resolves an issue where the Monitor would not see updates from ZooKeeper. + +### Two locations for the same extent + +The !METADATA table is the brains behind the data storage for each table, tracking information +like which files comprise a Tablet, and which TabletServers are hosting which Tablets. [ACCUMULO-2057][9] +fixes an issue where the !METADATA table contained multiple locations (hosting server) for +a single Tablet. + +### Deadlock on !METADATA tablet unload + +Tablets are unloaded, typically, when a shutdown request is issued. [ACCUMULO-1143][27] resolves +a potential deadlock issue when a merging-minor compaction is issued to flush in-memory data +to disk before unloading a Tablet. + +### Other notable fixes + + * [ACCUMULO-1800][5] Fixed deletes made via the Proxy. + * [ACCUMULO-1994][6] Fixed ranges in the Proxy. + * [ACCUMULO-2234][8] Fixed offline map reduce over non default HDFS location. + * [ACCUMULO-1615][15] Fixed `service accumulo-tserver stop`. + * [ACCUMULO-1876][16] Fixed issues depending on Accumulo using Apache Ivy. + * [ACCUMULO-2261][10] Duplicate locations for a Tablet. + * [ACCUMULO-2037][11] Tablets assigned to previous location. + * [ACCUMULO-1821][12] Avoid recovery on recovering Tablets. + * [ACCUMULO-2078][20] Incorrectly computed ACCUMULO_LOG_HOST in example configurations. + * [ACCUMULO-1985][21] Configuration to bind Monitor on all network interfaces. + * [ACCUMULO-1999][22] Allow '0' to signify random port for the Master. + * [ACCUMULO-1630][24] Fixed GC to interpret any IP/hostname. + + +## Known Issues + +When using Accumulo 1.5 and Hadoop 2, Accumulo will call hsync() on HDFS. +Calling hsync improves durability by ensuring data is on disk (where other older +Hadoop versions might lose data in the face of power failure); however, calling +hsync frequently does noticably slow writes. A simple work around is to increase +the value of the tserver.mutation.queue.max configuration parameter via accumulo-site.xml. + +A value of "4M" is a better recommendation, and memory consumption will increase by +the number of concurrent writers to that TabletServer. For example, a value of 4M with +50 concurrent writers would equate to approximately 200M of Java heap being used for +mutation queues. + +For more information, see [ACCUMULO-1950][2] and [this comment][1]. + +## Documentation + +The following documentation updates were made: + + * [ACCUMULO-1956][18] + * [ACCUMULO-1428][19] + * [ACCUMULO-1687][29] + * [ACCUMULO-2141][30] + * [ACCUMULO-1946][31] + * [ACCUMULO-2223][32] + * [ACCUMULO-2226][33] + * [ACCUMULO-1470][34] + +## Testing + +Below is a list of all platforms that 1.5.1 was tested against by developers. Each Apache Accumulo release +has a set of tests that must be run before the candidate is capable of becoming an official release. That list includes the following: + + 1. Successfully run all unit tests + 2. Successfully run all functional test (test/system/auto) + 3. Successfully complete two 24-hour RandomWalk tests (LongClean module), with and without "agitation" + 4. Successfully complete two 24-hour Continuous Ingest tests, with and without "agitation", with data verification + 5. Successfully complete two 72-hour Continuous Ingest tests, with and without "agitation" + +Each unit and functional test only runs on a single node, while the RandomWalk and Continuous Ingest tests run +on any number of nodes. *Agitation* refers to randomly restarting Accumulo processes and Hadoop Datanode processes, +and, in HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS | Hadoop | Nodes | ZooKeeper | HDFS High-Availability | Tests | +|------------|----------------------------|-------|----------------------------|------------------------|---------------------------------------------------| +| CentOS 6.5 | HDP 2.0 (Apache 2.2.0) | 6 | HDP 2.0 (Apache 3.4.5) | Yes (QJM) | All required tests | +| CentOS 6.4 | CDH 4.5.0 (2.0.0+cdh4.5.0) | 7 | CDH 4.5.0 (3.4.5+cdh4.5.0) | Yes (QJM) | Unit, functional and 24hr Randomwalk w/ agitation | +| CentOS 6.4 | CDH 4.5.0 (2.0.0+cdh4.5.0) | 7 | CDH 4.5.0 (3.4.5+cdh4.5.0) | Yes (QJM) | 2x 24/hr continuous ingest w/ verification | +| CentOS 6.3 | Apache 1.0.4 | 1 | Apache 3.3.5 | No | Local testing, unit and functional tests | +| RHEL 6.4 | Apache 2.2.0 | 10 | Apache 3.4.5 | No | Functional tests | + +[1]: https://issues.apache.org/jira/browse/ACCUMULO-1905?focusedCommentId=13915208&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13915208 +[2]: https://issues.apache.org/jira/browse/ACCUMULO-1950 +[3]: https://issues.apache.org/jira/browse/ACCUMULO-2128 +[4]: https://issues.apache.org/jira/browse/ACCUMULO-2262 +[5]: https://issues.apache.org/jira/browse/ACCUMULO-1800 +[6]: https://issues.apache.org/jira/browse/ACCUMULO-1994 +[7]: https://issues.apache.org/jira/browse/ACCUMULO-1933 +[8]: https://issues.apache.org/jira/browse/ACCUMULO-2234 +[9]: https://issues.apache.org/jira/browse/ACCUMULO-2057 +[10]: https://issues.apache.org/jira/browse/ACCUMULO-2261 +[11]: https://issues.apache.org/jira/browse/ACCUMULO-2037 +[12]: https://issues.apache.org/jira/browse/ACCUMULO-1821 +[13]: https://issues.apache.org/jira/browse/ACCUMULO-1572 +[14]: https://issues.apache.org/jira/browse/ACCUMULO-2360 +[15]: https://issues.apache.org/jira/browse/ACCUMULO-1615 +[16]: https://issues.apache.org/jira/browse/ACCUMULO-1876 +[17]: https://issues.apache.org/jira/browse/ACCUMULO-2127 +[18]: https://issues.apache.org/jira/browse/ACCUMULO-1956 +[19]: https://issues.apache.org/jira/browse/ACCUMULO-1428 +[20]: https://issues.apache.org/jira/browse/ACCUMULO-2078 +[21]: https://issues.apache.org/jira/browse/ACCUMULO-1985 +[22]: https://issues.apache.org/jira/browse/ACCUMULO-1999 +[23]: https://issues.apache.org/jira/browse/ACCUMULO-2065 +[24]: https://issues.apache.org/jira/browse/ACCUMULO-1630 +[25]: https://issues.apache.org/jira/browse/ACCUMULO-1920 +[27]: https://issues.apache.org/jira/browse/ACCUMULO-1143 +[28]: https://wiki.apache.org/hadoop/HadoopIPv6 +[29]: https://issues.apache.org/jira/browse/ACCUMULO-1687 +[30]: https://issues.apache.org/jira/browse/ACCUMULO-2141 +[31]: https://issues.apache.org/jira/browse/ACCUMULO-1946 +[32]: https://issues.apache.org/jira/browse/ACCUMULO-2223 +[33]: https://issues.apache.org/jira/browse/ACCUMULO-2226 +[34]: https://issues.apache.org/jira/browse/ACCUMULO-1470 +[35]: https://issues.apache.org/jira/browse/ACCUMULO-1833
http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2014-05-02-accumulo-1.6.0.md ---------------------------------------------------------------------- diff --git a/_posts/release/2014-05-02-accumulo-1.6.0.md b/_posts/release/2014-05-02-accumulo-1.6.0.md new file mode 100644 index 0000000..b6529a8 --- /dev/null +++ b/_posts/release/2014-05-02-accumulo-1.6.0.md @@ -0,0 +1,350 @@ +--- +title: Apache Accumulo 1.6.0 +redirect_from: /release_notes/1.6.0.html +--- + +Apache Accumulo 1.6.0 adds some major new features and fixes many bugs. This release contains changes from 609 issues contributed by 36 contributors and committers. + +Accumulo 1.6.0 runs on Hadoop 1, however Hadoop 2 with HA namenode is recommended for production systems. In addition to HA, Hadoop 2 also offers better data durability guarantees, in the case when nodes lose power, than Hadoop 1. + +## Notable Improvements + +### Multiple volume support + +[BigTable's][1] design allows for its internal metadata to automatically spread across multiple nodes. Accumulo has followed this design and scales very well as a result. There is one impediment to scaling though, and this is the HDFS namenode. There are two problems with the namenode when it comes to scaling. First, the namenode stores all of its filesystem metadata in memory on a single machine. This introduces an upper bound on the number of files Accumulo can have. Second, there is an upper bound on the number of file operations per second that a single namenode can support. For example, a namenode can only support a few thousand delete or create file request per second. + +To overcome this bottleneck, support for multiple namenodes was added under [ACCUMULO-118][ACCUMULO-118]. This change allows Accumulo to store its files across multiple namenodes. To use this feature, place comma separated list of namenode URIs in the new *instance.volumes* configuration property in accumulo-site.xml. When upgrading to 1.6.0 and multiple namenode support is desired, modify this setting **only** after a successful upgrade. + +### Table namespaces + +Administering an Accumulo instance with many tables is cumbersome. To ease this, [ACCUMULO-802][ACCUMULO-802] introduced table namespaces which allow tables to be grouped into logical collections. This allows configuration and permission changes to made to a namespace, which will apply to all of its tables. + +### Conditional Mutations + +Accumulo now offers a way to make atomic read,modify,write row changes from the client side. Atomic test and set row operations make this possible. [ACCUMULO-1000][ACCUMULO-1000] added conditional mutations and a conditional writer. A conditional mutation has tests on columns that must pass before any changes are made. These test are executed in server processes while a row lock is held. Below is a simple example of making atomic row changes using conditional mutations. + + 1. Read columns X,Y,SEQ into a,b,s from row R1 using an isolated scanner. + 2. For row R1 write conditional mutation X=f(a),Y=g(b),SEQ=s+1 if SEQ==s. + 3. If conditional mutation failed, then goto step 1. + +The only built in test that conditional mutations support are equality and isNull. However, iterators can be configured on a conditional mutation to run before these test. This makes it possible to implement any number of test such as less than, greater than, contains, etc. + +### Encryption + +Encryption is still an experimental feature, but much progress has been made since 1.5.0. Support for encrypting rfiles and write ahead logs were added in [ACCUMULO-958][ACCUMULO-958] and [ACCUMULO-980][ACCUMULO-980]. Support for encrypting data over the wire using SSL was added in [ACCUMULO-1009][ACCUMULO-1009]. + +When a tablet server fails, its write ahead logs are sorted and stored in HDFS. In 1.6.0, encrypting these sorted write ahead logs is not supported. [ACCUMULO-981][ACCUMULO-981] is open to address this issue. + +### Pluggable compaction strategies + +One of the key elements of the [BigTable][1] design is use of the [Log Structured Merge Tree][2]. This entails sorting data in memory, writing out sorted files, and then later merging multiple sorted files into a single file. These automatic merges happen in the background and Accumulo decides when to merge files based comparing relative sizes of files to a compaction ratio. Before 1.6.0 adjusting the compaction ratio was the only way a user could control this process. [ACCUMULO-1451][ACCUMULO-1451] introduces pluggable compaction strategies which allow users to choose when and what files to compact. [ACCUMULO-1808][ACCUMULO-1808] adds a compaction strategy that prevents compaction of files over a configurable size. + +### Lexicoders + +Accumulo only sorts data lexicographically. Getting something like a pair of (*String*,*Integer*) to sort correctly in Accumulo is tricky. It's tricky because you only want to compare the integers if the strings are equal. It's possible to make this sort properly in Accumulo if the data is encoded properly, but can be difficult. To make this easier [ACCUMULO-1336][ACCUMULO-1336] added Lexicoders to the Accumulo API. Lexicoders provide an easy way to serialize data so that it sorts properly lexicographically. Below is a simple example. + + PairLexicoder plex = new PairLexicoder(new StringLexicoder(), new IntegerLexicoder()); + byte[] ba1 = plex.encode(new ComparablePair<String, Integer>("b",1)); + byte[] ba2 = plex.encode(new ComparablePair<String, Integer>("aa",1)); + byte[] ba3 = plex.encode(new ComparablePair<String, Integer>("a",2)); + byte[] ba4 = plex.encode(new ComparablePair<String, Integer>("a",1)); + byte[] ba5 = plex.encode(new ComparablePair<String, Integer>("aa",-3)); + + //sorting ba1,ba2,ba3,ba4, and ba5 lexicographically will result in the same order as sorting the ComparablePairs + +### Locality groups in memory + +In cases where a very small amount of data is stored in a locality group one would expect fast scans over that locality group. However this was not always the case because recently written data stored in memory was not partitioned by locality group. Therefore if a table had 100GB of data in memory and 1MB of that was in locality group A, then scanning A would have required reading all 100GB. [ACCUMULO-112][ACCUMULO-112] changes this and partitions data by locality group as its written. + +### Service IP addresses + +Previous versions of Accumulo always used IP addresses internally. This could be problematic in virtual machine environments where IP addresses change. In [ACCUMULO-1585][ACCUMULO-1585] this was changed, now Accumulo uses the exact hostnames from its config files for internal addressing. + +All Accumulo processes running on a cluster are locatable via zookeeper. Therefore using well known ports is not really required. [ACCUMULO-1664][ACCUMULO-1664] makes it possible to for all Accumulo processes to use random ports. This makes it easier to run multiple Accumulo instances on a single node. + +While Hadoop [does not support IPv6 networks][3], attempting to run on a system that does not have IPv6 completely disabled can cause strange failures. [ACCUMULO-2262][ACCUMULO-2262] invokes the JVM-provided configuration parameter at process startup to prefer IPv4 over IPv6. + +### ViewFS + +Multiple bug-fixes were made to support running Accumulo over multiple HDFS instances using ViewFS. [ACCUMULO-2047][ACCUMULO-2047] is the parent +ticket that contains numerous fixes to enable this support. + +### Maven Plugin + +This version of Accumulo is accompanied by a new maven plugin for testing client apps ([ACCUMULO-1030][ACCUMULO-1030]). You can execute the accumulo-maven-plugin inside your project by adding the following to your pom.xml's build plugins section: + + <plugin> + <groupId>org.apache.accumulo</groupId> + <artifactId>accumulo-maven-plugin</artifactId> + <version>1.6.0</version> + <configuration> + <instanceName>plugin-it-instance</instanceName> + <rootPassword>ITSecret</rootPassword> + </configuration> + <executions> + <execution> + <id>run-plugin</id> + <goals> + <goal>start</goal> + <goal>stop</goal> + </goals> + </execution> + </executions> + </plugin> + +This plugin is designed to work in conjunction with the maven-failsafe-plugin. A small test instance of Accumulo will run during the pre-integration-test phase of the Maven build lifecycle, and will be stopped in the post-integration-test phase. Your integration tests, executed by maven-failsafe-plugin can access this instance with a MiniAccumuloInstance connector (the plugin uses MiniAccumuloInstance, internally), as in the following example: + + private static Connector conn; + + @BeforeClass + public static void setUp() throws Exception { + String instanceName = "plugin-it-instance"; + Instance instance = new MiniAccumuloInstance(instanceName, new File("target/accumulo-maven-plugin/" + instanceName)); + conn = instance.getConnector("root", new PasswordToken("ITSecret")); + } + +This plugin is quite limited, currently only supporting an instance name and a root user password as configuration parameters. Improvements are expected in future releases, so feedback is welcome and appreciated (file bugs/requests under the "maven-plugin" component in the Accumulo JIRA). + +### Packaging + +One notable change that was made to the binary tarball is the purposeful omission of a pre-built copy of the Accumulo "native map" library. +This shared library is used at ingest time to implement an off-JVM-heap sorted map that greatly increases ingest throughput while side-stepping +issues such as JVM garbage collection pauses. In earlier releases, a pre-built copy of this shared library was included in the binary tarball; however, the decision was made to omit this due to the potential variance in toolchains on the target system. + +It is recommended that users invoke the provided build\_native\_library.sh before running Accumulo: + + $ACCUMULO_HOME/bin/build_native_library.sh + +Be aware that you will need a C++ compiler/toolchain installed to build this library. Check your GNU/Linux distribution documentation for the package manager command. + +### Size-Based Constraint on New Tables + +A Constraint is an interface that can determine if a Mutation should be applied or rejected server-side. After [ACCUMULO-466][ACCUMULO-466], new tables that are created in 1.6.0 will automatically have the `DefaultKeySizeConstraint` set. +As performance can suffer when large Keys are inserted into a table, this Constraint will reject any Key that is larger than 1MB. If this constraint is undesired, it can be removed using the `constraint` shell +command. See the help message on the command for more information. + +### Other notable changes + + * [ACCUMULO-842][ACCUMULO-842] Added FATE administration to shell + * [ACCUMULO-1042][ACCUMULO-1042] CTRL-C no longer kills shell + * [ACCUMULO-1345][ACCUMULO-1345] Stuck compactions now log a warning with a stack trace, tablet id, and filename. + * [ACCUMULO-1442][ACCUMULO-1442] JLine2 support was added to the shell. This adds features like history search and other nice things GNU Readline has. + * [ACCUMULO-1481][ACCUMULO-1481] The root tablet is now the root table. + * [ACCUMULO-1537][ACCUMULO-1537] Python functional test were converted to maven Integration test that use MAC + * [ACCUMULO-1566][ACCUMULO-1566] When read-ahead starts in the scanner is now configurable. + * [ACCUMULO-1650][ACCUMULO-1650] Made common admin commands easier to run, try `bin/accumulo admin --help` + * [ACCUMULO-1667][ACCUMULO-1667] Added a synchronous version of online and offline table + * [ACCUMULO-1706][ACCUMULO-1706] Admin utilities now respect EPIPE + * [ACCUMULO-1833][ACCUMULO-1833] Multitable batch writer is faster now when used by multiple threads + * [ACCUMULO-1933][ACCUMULO-1933] Lower case can be given for memory units now. + * [ACCUMULO-1985][ACCUMULO-1985] Configuration to bind Monitor on all network interfaces. + * [ACCUMULO-2128][ACCUMULO-2128] Provide resource cleanup via static utility + * [ACCUMULO-2360][ACCUMULO-2360] Allow configuration of the maximum thrift message size a server will read. + +## Notable Bug Fixes + + * [ACCUMULO-324][ACCUMULO-324] System/site constraints and iterators should NOT affect the METADATA table + * [ACCUMULO-335][ACCUMULO-335] Can't batchscan over the !METADATA table + * [ACCUMULO-391][ACCUMULO-391] Added support for reading from multiple tables in a Map Reduce job. + * [ACCUMULO-1018][ACCUMULO-1018] Client does not give informative message when user can not read table + * [ACCUMULO-1492][ACCUMULO-1492] bin/accumulo should follow symbolic links + * [ACCUMULO-1572][ACCUMULO-1572] Single node zookeeper failure kills connected Accumulo servers + * [ACCUMULO-1661][ACCUMULO-1661] AccumuloInputFormat cannot fetch empty column family + * [ACCUMULO-1696][ACCUMULO-1696] Deep copy in the compaction scope iterators can throw off the stats + * [ACCUMULO-1698][ACCUMULO-1698] stop-here doesn't consider system hostname + * [ACCUMULO-1901][ACCUMULO-1901] start-here.sh starts only one GC process even if more are defined + * [ACCUMULO-1920][ACCUMULO-1920] Monitor was not seeing zookeeper updates for tables + * [ACCUMULO-1994][ACCUMULO-1994] Proxy does not handle Key timestamps correctly + * [ACCUMULO-2037][ACCUMULO-2037] Tablets are now assigned to the last location + * [ACCUMULO-2174][ACCUMULO-2174] VFS Classloader has potential to collide localized resources + * [ACCUMULO-2225][ACCUMULO-2225] Need to better handle DNS failure propagation from Hadoop + * [ACCUMULO-2234][ACCUMULO-2234] Cannot run offline mapreduce over non-default instance.dfs.dir value + * [ACCUMULO-2261][ACCUMULO-2261] Duplicate locations for a Tablet. + * [ACCUMULO-2334][ACCUMULO-2334] Lacking fallback when ACCUMULO_LOG_HOST isn't set + * [ACCUMULO-2408][ACCUMULO-2408] metadata table not assigned after root table is loaded + * [ACCUMULO-2519][ACCUMULO-2519] FATE operation failed across upgrade + +## Known Issues + +### Slower writes than previous Accumulo versions + +When using Accumulo 1.6 and Hadoop 2, Accumulo will call hsync() on HDFS. +Calling hsync improves durability by ensuring data is on disk (where other older +Hadoop versions might lose data in the face of power failure); however, calling +hsync frequently does noticeably slow writes. A simple work around is to increase +the value of the tserver.mutation.queue.max configuration parameter via accumulo-site.xml. + +A value of "4M" is a better recommendation, and memory consumption will increase by +the number of concurrent writers to that TabletServer. For example, a value of 4M with +50 concurrent writers would equate to approximately 200M of Java heap being used for +mutation queues. + +For more information, see [ACCUMULO-1950][ACCUMULO-1950] and [this comment][ACCUMULO-1905-comment]. + +Another possible cause of slower writes is the change in write ahead log replication +between 1.4 and 1.5. Accumulo 1.4. defaulted to two loggers servers. Accumulo 1.5 and 1.6 store +write ahead logs in HDFS and default to using three datanodes. + +### BatchWriter hold time error + +If a `BatchWriter` fails with `MutationsRejectedException` and the message contains +`"# server errors 1"` then it may be [ACCUMULO-2388][ACCUMULO-2388]. To confirm this look in the tablet server logs +for `org.apache.accumulo.tserver.HoldTimeoutException` around the time the `BatchWriter` failed. +If this is happening often a possible work around is to set `general.rpc.timeout` to `240s`. + +### Other known issues + + * [ACCUMULO-981][ACCUMULO-981] Sorted write ahead logs are not encrypted. + * [ACCUMULO-1507][ACCUMULO-1507] Dynamic Classloader still can't keep proper track of jars + * [ACCUMULO-1588][ACCUMULO-1588] Monitor XML and JSON differ + * [ACCUMULO-1628][ACCUMULO-1628] NPE on deep copied dumped memory iterator + * [ACCUMULO-1708][ACCUMULO-1708] [ACCUMULO-2495][ACCUMULO-2495] Out of memory errors do not always kill tservers leading to unexpected behavior + * [ACCUMULO-2008][ACCUMULO-2008] Block cache reserves section for in-memory blocks + * [ACCUMULO-2059][ACCUMULO-2059] Namespace constraints easily get clobbered by table constraints + * [ACCUMULO-2677][ACCUMULO-2677] Tserver failure during map reduce reading from table can cause sub-optimal performance + +## Documentation updates + + * [ACCUMULO-1218][ACCUMULO-1218] document the recovery from a failed zookeeper + * [ACCUMULO-1375][ACCUMULO-1375] Update README files in proxy module. + * [ACCUMULO-1407][ACCUMULO-1407] Fix documentation for deleterows + * [ACCUMULO-1428][ACCUMULO-1428] Document native maps + * [ACCUMULO-1946][ACCUMULO-1946] Include dfs.datanode.synconclose in hdfs configuration documentation + * [ACCUMULO-1956][ACCUMULO-1956] Add section on decomissioning or adding nodes to an Accumulo cluster + * [ACCUMULO-2441][ACCUMULO-2441] Document internal state stored in RFile names + * [ACCUMULO-2590][ACCUMULO-2590] Update public API in readme to clarify what's included + +## API Changes + +The following deprecated methods were removed in [ACCUMULO-1533][ACCUMULO-1533] + + * Many map reduce methods deprecated in [ACCUMULO-769][ACCUMULO-769] were removed + * `SecurityErrorCode o.a.a.core.client.AccumuloSecurityException.getErrorCode()` *deprecated in [ACCUMULO-970][ACCUMULO-970]* + * `Connector o.a.a.core.client.Instance.getConnector(AuthInfo)` *deprecated in [ACCUMULO-1024][ACCUMULO-1024]* + * `Connector o.a.a.core.client.ZooKeeperInstance.getConnector(AuthInfo)` *deprecated in [ACCUMULO-1024][ACCUMULO-1024]* + * `static String o.a.a.core.client.ZooKeeperInstance.getInstanceIDFromHdfs(Path)` *deprecated in [ACCUMULO-1][ACCUMULO-1]* + * `static String ZooKeeperInstance.lookupInstanceName (ZooCache,UUID)` *deprecated in [ACCUMULO-765][ACCUMULO-765]* + * `void o.a.a.core.client.ColumnUpdate.setSystemTimestamp(long)` *deprecated in [ACCUMULO-786][ACCUMULO-786]* + +## Testing + +Below is a list of all platforms that 1.6.0 was tested against by developers. Each Apache Accumulo release +has a set of tests that must be run before the candidate is capable of becoming an official release. That list includes the following: + + 1. Successfully run all unit tests + 2. Successfully run all functional test (test/system/auto) + 3. Successfully complete two 24-hour RandomWalk tests (LongClean module), with and without "agitation" + 4. Successfully complete two 24-hour Continuous Ingest tests, with and without "agitation", with data verification + 5. Successfully complete two 72-hour Continuous Ingest tests, with and without "agitation" + +Each unit and functional test only runs on a single node, while the RandomWalk and Continuous Ingest tests run +on any number of nodes. *Agitation* refers to randomly restarting Accumulo processes and Hadoop Datanode processes, +and, in HDFS High-Availability instances, forcing NameNode failover. + +The following acronyms are used in the test testing table. + + * CI : Continuous Ingest + * HA : High-Availability + * IT : Integration test, run w/ `mvn verify` + * RW : Random Walk + +{: #release_notes_testing .table } +| OS | Java | Hadoop | Nodes | ZooKeeper | HDFS HA | Version/Commit hash | Tests | +|------------|----------------------------|-----------------------------------|--------------|--------------|---------|----------------------------------|--------------------------------------------------------------------| +| CentOS 6.5 | CentOS OpenJDK 1.7 | Apache 2.2.0 | 20 EC2 nodes | Apache 3.4.5 | No | 1.6.0 RC1 + ACCUMULO\_2668 patch | 24-hour CI w/o agitation. Verified. | +| CentOS 6.5 | CentOS OpenJDK 1.7 | Apache 2.2.0 | 20 EC2 nodes | Apache 3.4.5 | No | 1.6.0 RC2 | 24-hour RW (Conditional.xml module) w/o agitation | +| CentOS 6.5 | CentOS OpenJDK 1.7 | Apache 2.2.0 | 20 EC2 nodes | Apache 3.4.5 | No | 1.6.0 RC5 | 24-hour CI w/ agitation. Verified. | +| CentOS 6.5 | CentOS OpenJDK 1.6 and 1.7 | Apache 1.2.1, 2.2.0 | Single | Apache 3.3.6 | No | 1.6.0 RC5 | All unit and ITs w/ `-Dhadoop.profile=2` and `-Dhadoop.profile=1` | +| Gentoo | Sun JDK 1.6.0\_45 | Apache 1.2.1, 2.2.0, 2.3.0, 2.4.0 | Single | Apache 3.4.5 | No | 1.6.0 RC5 | All unit and ITs. 2B entries ingested/verified with CI | +| CentOS 6.4 | Sun JDK 1.6.0\_31 | CDH 4.5.0 | 7 | CDH 4.5.0 | Yes | 1.6.0 RC4 and RC5 | 24-hour RW (LongClean) with and without agitation | +| CentOS 6.4 | Sun JDK 1.6.0\_31 | CDH 4.5.0 | 7 | CDH 4.5.0 | Yes | 3a1b38 | 72-hour CI with and without agitation. Verified. | +| CentOS 6.4 | Sun JDK 1.6.0\_31 | CDH 4.5.0 | 7 | CDH 4.5.0 | Yes | 1.6.0 RC2 | 24-hour CI without agitation. Verified. | +| CentOS 6.4 | Sun JDK 1.6.0\_31 | CDH 4.5.0 | 7 | CDH 4.5.0 | Yes | 1.6.0 RC3 | 24-hour CI with agitation. Verified. | + +[ACCUMULO-1]: https://issues.apache.org/jira/browse/ACCUMULO-1 +[ACCUMULO-112]: https://issues.apache.org/jira/browse/ACCUMULO-112 "Partition data in memory by locality group" +[ACCUMULO-118]: https://issues.apache.org/jira/browse/ACCUMULO-118 "Multiple namenode support" +[ACCUMULO-324]: https://issues.apache.org/jira/browse/ACCUMULO-324 "System/site constraints and iterators should NOT affect the METADATA table" +[ACCUMULO-335]: https://issues.apache.org/jira/browse/ACCUMULO-335 "Batch scanning over the !METADATA table can cause issues" +[ACCUMULO-391]: https://issues.apache.org/jira/browse/ACCUMULO-391 "Multi-table input format" +[ACCUMULO-466]: https://issues.apache.org/jira/browse/ACCUMULO-466 +[ACCUMULO-765]: https://issues.apache.org/jira/browse/ACCUMULO-765 +[ACCUMULO-769]: https://issues.apache.org/jira/browse/ACCUMULO-769 +[ACCUMULO-786]: https://issues.apache.org/jira/browse/ACCUMULO-786 +[ACCUMULO-802]: https://issues.apache.org/jira/browse/ACCUMULO-802 "Table namespaces" +[ACCUMULO-842]: https://issues.apache.org/jira/browse/ACCUMULO-842 "Add FATE administration to shell" +[ACCUMULO-958]: https://issues.apache.org/jira/browse/ACCUMULO-958 "Support pluggable encryption in walogs" +[ACCUMULO-970]: https://issues.apache.org/jira/browse/ACCUMULO-970 +[ACCUMULO-980]: https://issues.apache.org/jira/browse/ACCUMULO-980 "Support pluggable codecs for RFile" +[ACCUMULO-981]: https://issues.apache.org/jira/browse/ACCUMULO-981 "support pluggable encryption when recovering write-ahead logs" +[ACCUMULO-1000]: https://issues.apache.org/jira/browse/ACCUMULO-1000 "Conditional Mutations" +[ACCUMULO-1009]: https://issues.apache.org/jira/browse/ACCUMULO-1009 "Support encryption over the wire" +[ACCUMULO-1018]: https://issues.apache.org/jira/browse/ACCUMULO-1018 "Client does not give informative message when user can not read table" +[ACCUMULO-1024]: https://issues.apache.org/jira/browse/ACCUMULO-1024 +[ACCUMULO-1030]: https://issues.apache.org/jira/browse/ACCUMULO-1030 "Create a Maven plugin to run MiniAccumuloCluster for integration testing" +[ACCUMULO-1042]: https://issues.apache.org/jira/browse/ACCUMULO-1042 "Ctrl-C in shell terminates the process" +[ACCUMULO-1218]: https://issues.apache.org/jira/browse/ACCUMULO-1218 "document the recovery from a failed zookeeper" +[ACCUMULO-1336]: https://issues.apache.org/jira/browse/ACCUMULO-1336 "Add lexicoders from Typo to Accumulo" +[ACCUMULO-1345]: https://issues.apache.org/jira/browse/ACCUMULO-1345 "Provide feedback that a compaction is 'stuck'" +[ACCUMULO-1375]: https://issues.apache.org/jira/browse/ACCUMULO-1375 "Update README files in proxy module." +[ACCUMULO-1407]: https://issues.apache.org/jira/browse/ACCUMULO-1407 "Fix documentation for deleterows" +[ACCUMULO-1428]: https://issues.apache.org/jira/browse/ACCUMULO-1428 "Document native maps" +[ACCUMULO-1442]: https://issues.apache.org/jira/browse/ACCUMULO-1442 "Replace JLine with JLine2" +[ACCUMULO-1451]: https://issues.apache.org/jira/browse/ACCUMULO-1451 "Make Compaction triggers extensible" +[ACCUMULO-1481]: https://issues.apache.org/jira/browse/ACCUMULO-1481 "Root tablet in its own table" +[ACCUMULO-1492]: https://issues.apache.org/jira/browse/ACCUMULO-1492 "bin/accumulo should follow symbolic links" +[ACCUMULO-1507]: https://issues.apache.org/jira/browse/ACCUMULO-1507 "Dynamic Classloader still can't keep proper track of jars" +[ACCUMULO-1533]: https://issues.apache.org/jira/browse/ACCUMULO-1533 +[ACCUMULO-1537]: https://issues.apache.org/jira/browse/ACCUMULO-1537 "convert auto tests to integration tests, where possible for continuous integration" +[ACCUMULO-1585]: https://issues.apache.org/jira/browse/ACCUMULO-1585 "Use node addresses from config files verbatim" +[ACCUMULO-1562]: https://issues.apache.org/jira/browse/ACCUMULO-1562 "add a troubleshooting section to the user guide" +[ACCUMULO-1566]: https://issues.apache.org/jira/browse/ACCUMULO-1566 "Add ability for client to start Scanner readahead immediately" +[ACCUMULO-1572]: https://issues.apache.org/jira/browse/ACCUMULO-1572 "Single node zookeeper failure kills connected accumulo servers" +[ACCUMULO-1585]: https://issues.apache.org/jira/browse/ACCUMULO-1585 "Use FQDN/verbatim data from config files" +[ACCUMULO-1588]: https://issues.apache.org/jira/browse/ACCUMULO-1588 "Monitor XML and JSON differ" +[ACCUMULO-1628]: https://issues.apache.org/jira/browse/ACCUMULO-1628 "NPE on deep copied dumped memory iterator" +[ACCUMULO-1650]: https://issues.apache.org/jira/browse/ACCUMULO-1650 "Make it easier to find and run admin commands" +[ACCUMULO-1661]: https://issues.apache.org/jira/browse/ACCUMULO-1661 "AccumuloInputFormat cannot fetch empty column family" +[ACCUMULO-1664]: https://issues.apache.org/jira/browse/ACCUMULO-1664 "Make all processes able to use random ports" +[ACCUMULO-1667]: https://issues.apache.org/jira/browse/ACCUMULO-1667 "Allow On/Offline Command To Execute Synchronously" +[ACCUMULO-1696]: https://issues.apache.org/jira/browse/ACCUMULO-1696 "Deep copy in the compaction scope iterators can throw off the stats" +[ACCUMULO-1698]: https://issues.apache.org/jira/browse/ACCUMULO-1698 "stop-here doesn't consider system hostname" +[ACCUMULO-1704]: https://issues.apache.org/jira/browse/ACCUMULO-1704 "IteratorSetting missing (int,String,Class,Map) constructor" +[ACCUMULO-1706]: https://issues.apache.org/jira/browse/ACCUMULO-1706 "Admin Utilities Should Respect EPIPE" +[ACCUMULO-1708]: https://issues.apache.org/jira/browse/ACCUMULO-1708 "Error during minor compaction left tserver in bad state" +[ACCUMULO-1808]: https://issues.apache.org/jira/browse/ACCUMULO-1808 "Create compaction strategy that has size limit" +[ACCUMULO-1833]: https://issues.apache.org/jira/browse/ACCUMULO-1833 "MultiTableBatchWriterImpl.getBatchWriter() is not performant for multiple threads" +[ACCUMULO-1901]: https://issues.apache.org/jira/browse/ACCUMULO-1901 "start-here.sh starts only one GC process even if more are defined" +[ACCUMULO-1905-comment]: https://issues.apache.org/jira/browse/ACCUMULO-1905?focusedCommentId=13915208&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13915208 +[ACCUMULO-1920]: https://issues.apache.org/jira/browse/ACCUMULO-1920 "monitor not seeing zookeeper updates" +[ACCUMULO-1933]: https://issues.apache.org/jira/browse/ACCUMULO-1933 "Make unit on memory parameters case-insensitive" +[ACCUMULO-1946]: https://issues.apache.org/jira/browse/ACCUMULO-1946 "Include dfs.datanode.synconclose in hdfs configuration documentation" +[ACCUMULO-1950]: https://issues.apache.org/jira/browse/ACCUMULO-1950 "Reduce the number of calls to hsync" +[ACCUMULO-1956]: https://issues.apache.org/jira/browse/ACCUMULO-1956 "Add section on decomissioning or adding nodes to an Accumulo cluster" +[ACCUMULO-1958]: https://issues.apache.org/jira/browse/ACCUMULO-1958 "Range constructor lacks key checks, should be non-public" +[ACCUMULO-1985]: https://issues.apache.org/jira/browse/ACCUMULO-1985 "Cannot bind monitor on remote host to all interfaces" +[ACCUMULO-1994]: https://issues.apache.org/jira/browse/ACCUMULO-1994 "Proxy does not handle Key timestamps correctly" +[ACCUMULO-2008]: https://issues.apache.org/jira/browse/ACCUMULO-2008 "Block cache reserves section for in-memory blocks" +[ACCUMULO-2037]: https://issues.apache.org/jira/browse/ACCUMULO-2037 "Tablets not assigned to last location" +[ACCUMULO-2047]: https://issues.apache.org/jira/browse/ACCUMULO-2047 "Failures using viewfs with multiple namenodes" +[ACCUMULO-2059]: https://issues.apache.org/jira/browse/ACCUMULO-2059 "Namespace constraints easily get clobbered by table constraints" +[ACCUMULO-2128]: https://issues.apache.org/jira/browse/ACCUMULO-2128 "Provide resource cleanup via static utility rather than Instance.close" +[ACCUMULO-2174]: https://issues.apache.org/jira/browse/ACCUMULO-2174 "VFS Classloader has potential to collide localized resources" +[ACCUMULO-2225]: https://issues.apache.org/jira/browse/ACCUMULO-2225 "Need to better handle DNS failure propagation from Hadoop" +[ACCUMULO-2234]: https://issues.apache.org/jira/browse/ACCUMULO-2234 "Cannot run offline mapreduce over non-default instance.dfs.dir value" +[ACCUMULO-2261]: https://issues.apache.org/jira/browse/ACCUMULO-2261 "duplicate locations" +[ACCUMULO-2262]: https://issues.apache.org/jira/browse/ACCUMULO-2262 "Include java.net.preferIPv4Stack=true in process startup" +[ACCUMULO-2334]: https://issues.apache.org/jira/browse/ACCUMULO-2334 "Lacking fallback when ACCUMULO_LOG_HOST isn't set" +[ACCUMULO-2360]: https://issues.apache.org/jira/browse/ACCUMULO-2360 "Need a way to configure TNonblockingServer.maxReadBufferBytes to prevent OOMs" +[ACCUMULO-2388]: https://issues.apache.org/jira/browse/ACCUMULO-2388 +[ACCUMULO-2408]: https://issues.apache.org/jira/browse/ACCUMULO-2408 "metadata table not assigned after root table is loaded" +[ACCUMULO-2441]: https://issues.apache.org/jira/browse/ACCUMULO-2441 "Document internal state stored in RFile names" +[ACCUMULO-2495]: https://issues.apache.org/jira/browse/ACCUMULO-2495 "OOM exception didn't bring down tserver" +[ACCUMULO-2519]: https://issues.apache.org/jira/browse/ACCUMULO-2519 "FATE operation failed across upgrade" +[ACCUMULO-2590]: https://issues.apache.org/jira/browse/ACCUMULO-2590 "Update public API in readme to clarify what's included" +[ACCUMULO-2659]: https://issues.apache.org/jira/browse/ACCUMULO-2659 +[ACCUMULO-2677]: https://issues.apache.org/jira/browse/ACCUMULO-2677 "Single node bottle neck during map reduce" + +[1]: https://research.google.com/archive/bigtable.html +[2]: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.44.2782&rep=rep1&type=pdf +[3]: https://wiki.apache.org/hadoop/HadoopIPv6 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2014-09-25-accumulo-1.6.1.md ---------------------------------------------------------------------- diff --git a/_posts/release/2014-09-25-accumulo-1.6.1.md b/_posts/release/2014-09-25-accumulo-1.6.1.md new file mode 100644 index 0000000..45ead5e --- /dev/null +++ b/_posts/release/2014-09-25-accumulo-1.6.1.md @@ -0,0 +1,189 @@ +--- +title: Apache Accumulo 1.6.1 +redirect_from: /release_notes/1.6.1.html +--- + +Apache Accumulo 1.6.1 is a maintenance release on the 1.6 version branch. +This release contains changes from over 175 issues, comprised of bug-fixes, performance +improvements and better test cases. As this is a maintenance release, Apache Accumulo +1.6.1 has no client API incompatibilities over Apache Accumulo 1.6.0. Users of 1.6.0 +are strongly encouraged to update as soon as possible to benefit from the improvements. + +New users are encouraged to use this release over 1.6.0 or any other older releases. For +information about improvements since Accumulo 1.5, see the [1.6.0 release notes][32]. + +## Performance Improvements + +Apache Accumulo 1.6.1 includes a number of performance-related fixes over previous versions. +Many of these improvements were also included in the recently released Apache Accumulo 1.5.2. + + +### Write-Ahead Log sync performance + +The Write-Ahead Log (WAL) files are used to ensure durability of updates made to Accumulo. +A sync is called on the file in HDFS to make sure that the changes to the WAL are persisted +to disk, which allows Accumulo to recover in the case of failure. [ACCUMULO-2766][9] fixed +an issue where an operation against a WAL would unnecessarily wait for multiple syncs, slowing +down the ingest on the system. + +### Minor-Compactions not aggressive enough + +On a system with ample memory provided to Accumulo, long hold-times were observed which +blocks the ingest of new updates. Trying to free more server-side memory by running minor +compactions more frequently increased the overall throughput on the node. These changes +were made in [ACCUMULO-2905][10]. + +### HeapIterator optimization + +Iterators, a notable feature of Accumulo, are provided to users as a server-side programming +construct, but are also used internally for numerous server operations. One of these system iterator +is the HeapIterator which implements a PriorityQueue of other Iterators. One way this iterator is +used is to merge multiple files in HDFS to present a single, sorted stream of Key-Value pairs. [ACCUMULO-2827][11] +introduces a performance optimization to the HeapIterator which can improve the speed of the +HeapIterator in common cases. + +### Write-Ahead log sync implementation + +In Hadoop-2, two implementations of sync are provided: hflush and hsync. Both of these +methods provide a way to request that the datanodes write the data to the underlying +medium and not just hold it in memory (the *fsync* syscall). While both of these methods +inform the Datanodes to sync the relevant block(s), *hflush* does not wait for acknowledgement +from the Datanodes that the sync finished, where *hsync* does. To provide the most reliable system +"out of the box", Accumulo defaults to *hsync* so that your data is as secure as possible in +a variety of situations (notably, unexpected power outages). + +The downside is that performance tends to suffer because waiting for a sync to disk is a very +expensive operation. [ACCUMULO-2842][12] introduces a new system property, tserver.wal.sync.method, +that lets users to change the HDFS sync implementation from *hsync* to *hflush*. Using *hflush* instead +of *hsync* may result in about a 30% increase in ingest performance. + +For users upgrading from Hadoop-1 or Hadoop-0.20 releases, *hflush* is the equivalent of how +sync was implemented in these older versions of Hadoop and should give comparable performance. + +## Other improvements + +### Use of Hadoop CredentialProviders + +Apache Hadoop 2.6.0 introduced a new API aimed at providing ways to separate sensitive values +from being stored in plaintext as a part of [HADOOP-10607][28]. Accumulo has had two sensitive +configuration properties stored in *accumulo-site.xml* for every standard installation: instance.secret +and trace.token.property.password. If either of these properties are compromised, it could lead to +unwanted access of Accumulo. [ACCUMULO-2464][29] modifies Accumulo so that it can stored any sensitive +configuration properties in a Hadoop CredentialProvider. With sensitive values removed from accumulo-site.xml, +it can be shared without concern and security can be focused solely on the CredentialProvider. + +## Notable Bug Fixes + +### Add configurable maximum frame size to Apache Thrift proxy + +The Thrift proxy server was subject to memory exhaustion, typically +due to bad input, where the server would attempt to allocate a very large +buffer and die in the process. [ACCUMULO-2658][2] introduces a configuration +parameter, like [ACCUMULO-2360][3], to prevent this error. + +### Offline tables can prevent tablet balancing + +Before 1.6.1, when a table with many tablets was created, ingested into, and +taken offline, tablet balancing may have stoppped. This would happen if there +were tablet migrations for the table, because the migrations couldn't occur. +The balancer will not run when there are outstanding migrations; therefore, a +system could become unbalanced. [ACCUMULO-2694][4] introduces a fix to ensure +that offline tables do not block balancing and improves the server-side +logging. + +### MiniAccumuloCluster process management + +MiniAccumuloCluster had a few issues which could cause deadlock or a method that +never returns. Most of these are related to management of the Accumulo processes +([ACCUMULO-2764][5], [ACCUMULO-2985][6], and [ACCUMULO-3055][7]). + +### IteratorSettings not correctly serialized in RangeInputSplit + +The Writable interface methods on the RangeInputSplit class accidentally omitted +calls to serialize the IteratorSettings configured for the Job. [ACCUMULO-2962][8] +fixes the serialization and adds some additional tests. + +### Constraint violation causes hung scans + +A failed bulk import transaction had the ability to create an infinitely retrying +loop due to a constraint violation. This directly prevents scans from completing, +but will also hang compactions. [ACCUMULO-3096][14] fixes the issue so that the +constraint no longer hangs the entire system. + +### Unable to upgrade cleanly from 1.5 + +When upgrading a table from 1.5.1 to 1.6.0, a user experienced an error where the table +never came online. [ACCUMULO-2974][27] fixes an issue from the change of file references +stored as absolute paths instead of relative paths in the Accumulo metadata table. + +### Guava dependency changed + +[ACCUMULO-3100][30] lowered the dependency on Guava from 15.0 to 14.0.1. This dependency +now matches what Hadoop is depending on for the 2.x.y version line. Depending on a newer +version of Guava introduces many issues stemming from deprecated classes in use by Hadoop +which have been removed. While installations of Accumulo will likely work as expected with +newer versions of Guava on the classpath (because the Hadoop processes will have their own +classpath), use of MiniDfsClusters with the new Guava version will result in errors. + +Users can attempt to use a newer version of Guava on the Accumulo server classpath; however, +the success is dependent on Hadoop client libraries not using (missing) Guava methods internally. + +### Scanners eat InterruptedException + +Scanners previously consumed InterruptedExceptions and did not exit after. In multi-threaded +environments, this is very problematic as there is no means to stop the Scanner from reading data. +[ACCUMULO-3030][31] fixes the Scanner so that interrupts are observed and the Scanner exits as expected. + +## Documentation + +The following documentation updates were made: + + * [ACCUMULO-2767][15] + * [ACCUMULO-2796][16] + * [ACCUMULO-2919][17] + * [ACCUMULO-3008][18] + * [ACCUMULO-2874][19] + * [ACCUMULO-2821][20] + * [ACCUMULO-3097][21] + * [ACCUMULO-3097][22] + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk and Continuous Ingest tests run +on any number of nodes. *Agitation* refers to randomly restarting Accumulo processes and Hadoop Datanode processes, +and, in HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS | Hadoop | Nodes | ZooKeeper | HDFS HA | Tests | +|------------|-----------------------|-------|--------------|---------|-------------------------------------------------------------------------------------------------------------| +| Gentoo | Apache 2.6.0-SNAPSHOT | 2 | Apache 3.4.5 | No | Unit and Functional Tests, ContinuousIngest w/ verification (2B entries) | +| CentOS 6 | Apache 2.3.0 | 20 | Apache 3.4.5 | No | 24/hr RandomWalk, ContinuousIngest w/ verification w/ and w/o agitation (17B entries), 24hr Randomwalk test | + +[1]: https://issues.apache.org/jira/browse/ACCUMULO-2586 +[2]: https://issues.apache.org/jira/browse/ACCUMULO-2658 +[3]: https://issues.apache.org/jira/browse/ACCUMULO-2360 +[4]: https://issues.apache.org/jira/browse/ACCUMULO-2694 +[5]: https://issues.apache.org/jira/browse/ACCUMULO-2764 +[6]: https://issues.apache.org/jira/browse/ACCUMULO-2985 +[7]: https://issues.apache.org/jira/browse/ACCUMULO-3055 +[8]: https://issues.apache.org/jira/browse/ACCUMULO-2962 +[9]: https://issues.apache.org/jira/browse/ACCUMULO-2766 +[10]: https://issues.apache.org/jira/browse/ACCUMULO-2905 +[11]: https://issues.apache.org/jira/browse/ACCUMULO-2827 +[12]: https://issues.apache.org/jira/browse/ACCUMULO-2842 +[13]: https://issues.apache.org/jira/browse/ACCUMULO-3018 +[14]: https://issues.apache.org/jira/browse/ACCUMULO-3096 +[15]: https://issues.apache.org/jira/browse/ACCUMULO-2767 +[16]: https://issues.apache.org/jira/browse/ACCUMULO-2796 +[17]: https://issues.apache.org/jira/browse/ACCUMULO-2919 +[18]: https://issues.apache.org/jira/browse/ACCUMULO-3008 +[19]: https://issues.apache.org/jira/browse/ACCUMULO-2874 +[20]: https://issues.apache.org/jira/browse/ACCUMULO-2821 +[21]: https://issues.apache.org/jira/browse/ACCUMULO-3097 +[22]: https://issues.apache.org/jira/browse/ACCUMULO-3097 +[27]: https://issues.apache.org/jira/browse/ACCUMULO-2974 +[28]: https://issues.apache.org/jira/browse/HADOOP-10607 +[29]: https://issues.apache.org/jira/browse/ACCUMULO-2464 +[30]: https://issues.apache.org/jira/browse/ACCUMULO-3100 +[31]: https://issues.apache.org/jira/browse/ACCUMULO-3030 +[32]: {{ site.baseurl }}/release_notes/1.6.0 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2015-02-16-accumulo-1.6.2.md ---------------------------------------------------------------------- diff --git a/_posts/release/2015-02-16-accumulo-1.6.2.md b/_posts/release/2015-02-16-accumulo-1.6.2.md new file mode 100644 index 0000000..57b07b2 --- /dev/null +++ b/_posts/release/2015-02-16-accumulo-1.6.2.md @@ -0,0 +1,172 @@ +--- +title: Apache Accumulo 1.6.2 +redirect_from: /release_notes/1.6.2.html +--- + +Apache Accumulo 1.6.2 is a maintenance release on the 1.6 version branch. +This release contains changes from over 150 issues, comprised of bug-fixes, performance +improvements and better test cases. Apache Accumulo 1.6.2 is the first release since the +community has adopted [Semantic Versioning][1] which means that all changes to the [public API][2] +are guaranteed to be made without adding to or removing from the public API. This ensures +that client code that runs against 1.6.1 is guaranteed to run against 1.6.2 and vice versa. + +Users of 1.6.0 or 1.6.1 are strongly encouraged to update as soon as possible to benefit from +the improvements with very little concern in change of underlying functionality. Users of 1.4 or 1.6 +are seeking to upgrade to 1.6 should consider 1.6.2 the starting point over 1.6.0 or 1.6.1. For +information about improvements since Accumulo 1.5, see the [1.6.0][3] and [1.6.1][4] release notes. + +## Notable Bug Fixes + +### Only first ZooKeeper server is used + +In constructing a `ZooKeeperInstance`, the user provides a comma-separated list of addresses for ZooKeeper +servers. 1.6.0 and 1.6.1 incorrectly truncated the provided list of ZooKeeper servers used to the first. This +would cause clients to fail when the first ZooKeeper server in the list became unavailable and not properly +load balance requests to all available servers in the quorum. [ACCUMULO-3218][5] fixes the parsing of +the ZooKeeper quorum list to use all servers, not just the first. + +### Incorrectly handled ZooKeeper exception + +Use of ZooKeeper's API requires very careful exception handling as some thrown exceptions from the ZooKeeper +API are considered "normal" and must be retried by the client. In 1.6.1, Accumulo improved its handling of +these "expected failures" to better insulate calls to ZooKeeper; however, the wrapper which sets data to a ZNode +incorrectly handled all cases. [ACCUMULO-3448][6] fixed the implementation of `ZooUtil.putData(...)` to handle +the expected error conditions correctly. + +### `scanId` is not set in `ActiveScan` + +The `ActiveScan` class is the returned object by `InstanceOperations.listScans`. This class represents a +"scan" running on Accumulo servers, either from a `Scanner` or `BatchScanner`. The `ActiveScan` class +is meant to represent all of the information that represents the scan and can be useful to administrators +or DevOps-types to observe and act on scans which are running for excessive periods of time. [ACCUMULO-2641][7] +fixes `ActiveScan` to ensure that the internal identifier `scanId` is properly set. + +### Table state change doesn't wait when requested + +An Accumulo table has two states: `ONLINE` and `OFFLINE`. An offline table in Accumulo consumes no TabletServer +resources, only HDFS resources, which makes it useful to save infrequently used data. The Accumulo methods provided +to transition a state from `ONLINE` to `OFFLINE` and vice versa did not respect the `wait=true` parameter +when set. [ACCUMULO-3301][8] fixes the underlying implementation to ensure that when `wait=true` is provided, +the method will not return until the table's state transition has fully completed. + +### KeyValue doesn't implement `hashCode()` or `equals()` + +The `KeyValue` class is an implementation of `Entry<Key,Value>` which is returned by the classes like +`Scanner` and `BatchScanner`. [ACCUMULO-3217][9] adds these methods which ensure that the returned `Entry<Key,Value>` +operates as expected with `HashMaps` and `HashSets`. + +### Potential deadlock in TabletServer + +Internal to the TabletServer, there are methods to construct instances of configuration objects for tables +and namespaces. The locking on these methods was not correctly implemented which created the possibility to +have concurrent requests to a TabletServer to deadlock. [ACCUMULO-3372][10] found this problem while performing +bulk imports of RFiles into Accumulo. Additional synchronization was added server-side to prevent this deadlock +from happening in the future. + +### The `DateLexicoder` incorrectly serialized `Dates` prior 1970 + +The `DateLexicode`, a part of the `Lexicoders` classes which implement methods to convert common type primitives +into lexicographically sorting Strings/bytes, incorrectly converted `Date` objects for dates prior to 1970. +[ACCUMULO-3385][11] fixed the `DateLexicoder` to correctly (de)serialize data `Date` objects. For users with +data stored in Accumulo using the broken implementation, the following can be performed to read the old data. + + Lexicoder lex = new ULongLexicoder(); + for (Entry<Key, Value> e : scanner) { + Date d = new Date(lex.decode(TextUtil.getBytes(e.getKey().getRow()))); + // ... + } + +### Reduce MiniAccumuloCluster failures due to random port allocations + +`MiniAccumuloCluster` has had issues where it fails to properly start due to the way it attempts to choose +a random, unbound port on the local machine to start the ZooKeeper and Accumulo processes. Improvements have +been made, including retry logic, to withstand a few failed port choices. The changes made by [ACCUMULO-3233][12] +and the related issues should eliminate sporadic failures users of `MiniAccumuloCluster` might have observed. + +### Tracer doesn't handle trace table state transition + +The Tracer is an optional Accumulo server process that serializes Spans, elements of a distributed trace, +to the trace table for later inspection and correlation with other Spans. By default, the Tracer writes +to a "trace" table. In earlier versions of Accumulo, if this table was put offline, the Tracer would fail +to write new Spans to the table when it came back online. [ACCUMULO-3351][13] ensures that the Tracer process +will resume writing Spans to the trace table when it transitions to online after being offline. + +### Tablet not major compacting + +It was noticed that a system performing many bulk imports, there was a tablet with hundreds of files which +was not major compacting nor was scheduled to be major compacted. [ACCUMULO-3462][14] identified as fix +server-side which would prevent this from happening in the future. + +### YARN job submission fails with Hadoop-2.6.0 + +Hadoop 2.6.0 introduced a new component, the TimelineServer, which is a centralized metrics service designed +for other Hadoop components to leverage. MapReduce jobs submitted via `accumulo` and `tool.sh` failed to +run the job because it attempted to contact the TimelineServer and Accumulo was missing a dependency on +the classpath to communicate with the TimelineServer. [ACCUMULO-3230][15] updates the classpath in the example +configuration files to include the necessary dependencies for the TimelineServer to ensure that YARN job +submission operates as previously. + +## Performance Improvements + +### User scans can block root and metadata table scans + +The TabletServer provides a feature to limit the number of open files as a resource management configuration. +To perform a scan against a normal table, the metadata and root table, when not cached, need to be consulted +first. With a sufficient number of concurrent scans against normal tables, adding to the open file count, +scans against the metadata and root tables could be blocked from running because no files can be opened. +This prevents other system operations from happening as expected. [ACCUMULO-3297][16] fixes the internal semaphore +used to implement this resource management to ensure that root and metadata table scans can proceed. + + +## Other improvements + +### Limit available ciphers for SSL/TLS + +Since Apache Accumulo 1.5.2 and 1.6.1, the [POODLE][17] man-in-the-middle attack was found which exploits a client's +ability to fallback to the SSLv3.0 protocol. The main mitigation strategy was to prevent the use of old ciphers/protocols +when using SSL connectors. In Accumulo, both the Apache Thrift RPC servers and Jetty server for the Accumulo +monitor have the ability to enable SSL. [ACCUMULO-3316][18] is the parent issue which provides new configuration +properties in accumulo-site.xml which can limit the accepted ciphers/protocols. By default, insecure or out-dated +protocols have been removed from the default set in order to protect users by default. + + +## Documentation + +Documentation was added to the Administration chapter for moving from a Non-HA Namenode setup to an HA Namenode setup. +New chapters were added for the configuration of SSL and for summaries of Implementation Details (initially describing +FATE operations). A section was added to the Configuration chapter for describing how to arrive at optimal settings +for configuring an instance with native maps. + + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk and Continuous Ingest tests run +on any number of nodes. *Agitation* refers to randomly restarting Accumulo processes and Hadoop Datanode processes, +and, in HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS | Hadoop | Nodes | ZooKeeper | HDFS HA | Tests | +|-----------|--------|-------|-----------|---------|-------------------------------------------------------------------------------------------| +| Gentoo | N/A | 1 | N/A | No | Unit and Integration Tests | +| Mac OSX | N/A | 1 | N/A | No | Unit and Integration Tests | +| Fedora 21 | N/A | 1 | N/A | No | Unit and Integration Tests | +| CentOS 6 | 2.6 | 20 | 3.4.5 | No | ContinuousIngest w/ verification w/ and w/o agitation (31B and 21B entries, respectively) | + +[1]: https://semver.org +[2]: https://github.com/apache/accumulo#api +[3]: {{ site.baseurl }}/release_notes/1.6.0 +[4]: {{ site.baseurl }}/release_notes/1.6.1 +[5]: https://issues.apache.org/jira/browse/ACCUMULO-3218 +[6]: https://issues.apache.org/jira/browse/ACCUMULO-3448 +[7]: https://issues.apache.org/jira/browse/ACCUMULO-2641 +[8]: https://issues.apache.org/jira/browse/ACCUMULO-3301 +[9]: https://issues.apache.org/jira/browse/ACCUMULO-3217 +[10]: https://issues.apache.org/jira/browse/ACCUMULO-3372 +[11]: https://issues.apache.org/jira/browse/ACCUMULO-3385 +[12]: https://issues.apache.org/jira/browse/ACCUMULO-3233 +[13]: https://issues.apache.org/jira/browse/ACCUMULO-3351 +[14]: https://issues.apache.org/jira/browse/ACCUMULO-3462 +[15]: https://issues.apache.org/jira/browse/ACCUMULO-3230 +[16]: https://issues.apache.org/jira/browse/ACCUMULO-3297 +[17]: https://en.wikipedia.org/wiki/POODLE +[18]: https://issues.apache.org/jira/browse/ACCUMULO-3316 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2015-05-18-accumulo-1.7.0.md ---------------------------------------------------------------------- diff --git a/_posts/release/2015-05-18-accumulo-1.7.0.md b/_posts/release/2015-05-18-accumulo-1.7.0.md new file mode 100644 index 0000000..040ac37 --- /dev/null +++ b/_posts/release/2015-05-18-accumulo-1.7.0.md @@ -0,0 +1,399 @@ +--- +title: Apache Accumulo 1.7.0 +redirect_from: /release_notes/1.7.0.html +--- + +Apache Accumulo 1.7.0 is a significant release that includes many important +milestone features which expand the functionality of Accumulo. These include +features related to security, availability, and extensibility. Nearly 700 JIRA +issues were resolved in this version. Approximately two-thirds were bugs and +one-third were improvements. + +In the context of Accumulo's [Semantic Versioning][semver] [guidelines][api], +this is a "minor version". This means that new APIs have been created, some +deprecations may have been added, but no deprecated APIs have been removed. +Code written against 1.6.x should work against 1.7.0, likely binary-compatible +but definitely source-compatible. As always, the Accumulo developers take API compatibility +very seriously and have invested much time to ensure that we meet the promises set forth to our users. + +# Major Changes # + +## Updated Minimum Requirements ## + +Apache Accumulo 1.7.0 comes with an updated set of minimum requirements. + + * Java7 is required. Java6 support is dropped. + * Hadoop 2.2.0 or greater is required. Hadoop 1.x support is dropped. + * ZooKeeper 3.4.x or greater is required. + +## Client Authentication with Kerberos ## + +Kerberos is the de-facto means to provide strong authentication across Hadoop +and other related components. Kerberos requires a centralized key distribution +center to authentication users who have credentials provided by an +administrator. When Hadoop is configured for use with Kerberos, all users must +provide Kerberos credentials to interact with the filesystem, launch YARN +jobs, or even view certain web pages. + +While Accumulo has long supported operating on Kerberos-enabled HDFS, it still +required Accumulo users to use password-based authentication to authenticate +with Accumulo. [ACCUMULO-2815][ACCUMULO-2815] added support for allowing +Accumulo clients to use the same Kerberos credentials to authenticate to +Accumulo that they would use to authenticate to other Hadoop components, +instead of a separate user name and password just for Accumulo. + +This authentication leverages [Simple Authentication and Security Layer +(SASL)][SASL] and [GSSAPI][GSSAPI] to support Kerberos authentication over the +existing Apache Thrift-based RPC infrastructure that Accumulo employs. + +These additions represent a significant forward step for Accumulo, bringing +its client-authentication up to speed with the rest of the Hadoop ecosystem. +This results in a much more cohesive authentication story for Accumulo that +resonates with the battle-tested cell-level security and authorization model +already familiar to Accumulo users. + +More information on configuration, administration, and application of Kerberos +client authentication can be found in the [Kerberos chapter][kerberos] of the +Accumulo User Manual. + +## Data-Center Replication ## + +In previous releases, Accumulo only operated within the constraints of a +single installation. Because single instances of Accumulo often consist of +many nodes and Accumulo's design scales (near) linearly across many nodes, it +is typical that one Accumulo is run per physical installation or data-center. +[ACCUMULO-378][ACCUMULO-378] introduces support in Accumulo to automatically +copy data from one Accumulo instance to another. + +This data-center replication feature is primarily applicable to users wishing +to implement a disaster recovery strategy. Data can be automatically copied +from a primary instance to one or more other Accumulo instances. In contrast +to normal Accumulo operation, in which ingest and query are strongly +consistent, data-center replication is a lazy, eventually consistent +operation. This is desirable for replication, as it prevents additional +latency for ingest operations on the primary instance. Additionally, the +implementation of this feature can sustain prolonged outages between the +primary instance and replicas without any administrative overhead. + +The Accumulo User Manual contains a [new chapter on replication][replication] +which details the design and implementation of the feature, explains how users +can configure replication, and describes special cases to consider when +choosing to integrate the feature into a user application. + +## User-Initiated Compaction Strategies ## + +Per-table compaction strategies were added in 1.6.0 to provide custom logic to +decide which files are involved in a major compaction. In 1.7.0, the ability +to specify a compaction strategy for a user-initiated compaction was added in +[ACCUMULO-1798][ACCUMULO-1798]. This allows surgical compactions on a subset +of tablet files. Previously, a user-initiated compaction would compact all +files in a tablet. + +In the Java API, this new feature can be accessed in the following way: + + Connection conn = ... + CompactionStrategyConfig csConfig = new CompactionStrategyConfig(strategyClassName).setOptions(strategyOpts); + CompactionConfig compactionConfig = new CompactionConfig().setCompactionStrategy(csConfig); + connector.tableOperations().compact(tableName, compactionConfig) + +In [ACCUMULO-3134][ACCUMULO-3134], the shell's `compact` command was modified +to enable selecting which files to compact based on size, name, and path. +Options were also added to the shell's compaction command to allow setting +RFile options for the compaction output. Setting the output options could be +useful for testing. For example, one tablet to be compacted using snappy +compression. + +The following is an example shell command that compacts all files less than +10MB, if the tablet has at least two files that meet this criteria. If a +tablet had a 100MB, 50MB, 7MB, and 5MB file then the 7MB and 5MB files would +be compacted. If a tablet had a 100MB and 5MB file, then nothing would be done +because there are not at least two files meeting the selection criteria. + + compact -t foo --min-files 2 --sf-lt-esize 10M + +The following is an example shell command that compacts all bulk imported +files in a table. + + compact -t foo --sf-ename I.* + +These provided convenience options to select files execute using a specialized +compaction strategy. Options were also added to the shell to specify an +arbitrary compaction strategy. The option to specify an arbitrry compaction +strategy is mutually exclusive with the file selection and file creation +options, since those options are unique to the specialized compaction strategy +provided. See `compact --help` in the shell for the available options. + +## API Clarification ## + +The declared API in 1.6.x was incomplete. Some important classes like +ColumnVisibility were not declared as Accumulo API. Significant work was done +under [ACCUMULO-3657][ACCUMULO-3657] to correct the API statement and clean up +the API to be representative of all classes which users are intended to +interact with. The expanded and simplified API statement is in the +[README][api]. + +In some places in the API, non-API types were used. Ideally, public API +members would only use public API types. A tool called [APILyzer][apilyzer] +was created to find all API members that used non-API types. Many of the +violations found by this tool were deprecated to clearly communicate that a +non-API type was used. One example is a public API method that returned a +class called `KeyExtent`. `KeyExtent` was never intended to be in the public +API because it contains code related to Accumulo internals. `KeyExtent` and +the API methods returning it have since been deprecated. These were replaced +with a new class for identifying tablets that does not expose internals. +Deprecating a type like this from the API makes the API more stable while also +making it easier for contributors to change Accumulo internals without +impacting the API. + +The changes in [ACCUMULO-3657][ACCUMULO-3657] also included an Accumulo API +regular expression for use with checkstyle. Starting with 1.7.0, projects +building on Accumulo can use this checkstyle rule to ensure they are only +using Accumulo's public API. The regular expression can be found in the +[README][api]. + +# Performance Improvements # + +## Configurable Threadpool Size for Assignments ## + +During start-up, the Master quickly assigns tablets to Tablet Servers. However, +Tablet Servers load those assigned tablets one at a time. In 1.7, the servers +will be more aggressive, and will load tablets in parallel, so long as they do +not have mutations that need to be recovered. + +[ACCUMULO-1085] allows the size of the threadpool used in the Tablet Servers +for assignment processing to be configurable. + +## Group-Commit Threshold as a Factor of Data Size ## + +When ingesting data into Accumulo, the majority of time is spent in the +write-ahead log. As such, this is a common place that optimizations are added. +One optimization is known as "group-commit". When multiple clients are +writing data to the same Accumulo tablet, it is not efficient for each of them +to synchronize the WAL, flush their updates to disk for durability, and then +release the lock. The idea of group-commit is that multiple writers can queue +the write for their mutations to the WAL and then wait for a sync that will +satisfy the durability constraints of their batch of updates. This has a +drastic improvement on performance, since many threads writing batches +concurrently can "share" the same `fsync`. + +In previous versions, Accumulo controlled the frequency in which this +group-commit sync was performed as a factor of the number of clients writing +to Accumulo. This was both confusing to correctly configure and also +encouraged sub-par performance with few write threads. +[ACCUMULO-1950][ACCUMULO-1950] introduced a new configuration property +`tserver.total.mutation.queue.max` which defines the amount of data that is +queued before a group-commit is performed in such a way that is agnostic of +the number of writers. This new configuration property is much easier to +reason about than the previous (now deprecated) `tserver.mutation.queue.max`. +Users who have set `tserver.mutation.queue.max` in the past are encouraged +to start using the new `tserver.total.mutation.queue.max` property. + +# Other improvements # + +## Balancing Groups of Tablets ## + +By default, Accumulo evenly spreads each table's tablets across a cluster. In +some situations, it is advantageous for query or ingest to evenly spreads +groups of tablets within a table. For [ACCUMULO-3439][ACCUMULO-3439], a new +balancer was added to evenly spread groups of tablets to optimize performance. +This [blog post][group_balancer] provides more details about when and why +users may desire to leverage this feature.. + +## User-specified Durability ## + +Accumulo constantly tries to balance durability with performance. Guaranteeing +durability of every write to Accumulo is very difficult in a +massively-concurrent environment that requires high throughput. One common +area of focus is the write-ahead log, since it must eventually call `fsync` on +the local filesystem to guarantee that data written is durable in the face of +unexpected power failures. In some cases where durability can be sacrificed, +either due to the nature of the data itself or redundant power supplies, +ingest performance improvements can be attained. + +Prior to 1.7, a user could only configure the level of durability for +individual tables. With the implementation of [ACCUMULO-1957][ACCUMULO-1957], +the durability can be specified by the user when creating a `BatchWriter`, +giving users control over durability at the level of the individual writes. +Every `Mutation` written using that `BatchWriter` will be written with the +provided durability. This can result in substantially faster ingest rates when +the durability can be relaxed. + +## waitForBalance API ## + +When creating a new Accumulo table, the next step is typically adding splits +to that table before starting ingest. This can be extremely important since a +table without any splits will only be hosted on a single tablet server and +create a ingest bottleneck until the table begins to naturally split. Adding +many splits before ingesting will ensure that a table is distributed across +many servers and result in high throughput when ingest first starts. + +Adding splits to a table has long been a synchronous operation, but the +assignment of those splits was asynchronous. A large number of splits could be +processed, but it was not guaranteed that they would be evenly distributed +resulting in the same problem as having an insufficient number of splits. +[ACCUMULO-2998][ACCUMULO-2998] adds a new method to `InstanceOperations` which +allows users to wait for all tablets to be balanced. This method lets users +wait until tablets are appropriately distributed so that ingest can be run at +full-bore immediately. + +## Hadoop Metrics2 Support ## + +Accumulo has long had its own metrics system implemented using Java MBeans. +This enabled metrics to be reported by Accumulo services, but consumption by +other systems often required use of an additional tool like jmxtrans to read +the metrics from the MBeans and send them to some other system. + +[ACCUMULO-1817][ACCUMULO-1817] replaces this custom metrics system Accumulo +with Hadoop Metrics2. Metrics2 has a number of benefits, the most common of +which is invalidating the need for an additional process to send metrics to +common metrics storage and visualization tools. With Metrics2 support, +Accumulo can send its metrics to common tools like Ganglia and Graphite. + +For more information on enabling Hadoop Metrics2, see the [Metrics +Chapter][metrics] in the Accumulo User Manual. + +## Distributed Tracing with HTrace ## + +HTrace has recently started gaining traction as a standalone project, +especially with its adoption in HDFS. Accumulo has long had distributed +tracing support via its own "Cloudtrace" library, but this wasn't intended for +use outside of Accumulo. + +[ACCUMULO-898][ACCUMULO-898] replaces Accumulo's Cloudtrace code with HTrace. +This has the benefit of adding timings (spans) from HDFS into Accumulo spans +automatically. + +Users who inspect traces via the Accumulo Monitor (or another system) will begin +to see timings from HDFS during operations like Major and Minor compactions when +running with at least Apache Hadoop 2.6.0. + +## VERSIONS file present in binary distribution ## + +In the pre-built binary distribution or distributions built by users from the +official source release, users will now see a `VERSIONS` file present in the +`lib/` directory alongside the Accumulo server-side jars. Because the created +tarball strips off versions from the jar file names, it can require extra work +to actually find what the version of each dependent jar (typically inspecting +the jar's manifest). + +[ACCUMULO-2863][ACCUMULO-2863] adds a `VERSIONS` file to the `lib/` directory +which contains the Maven groupId, artifactId, and verison (GAV) information for +each jar file included in the distribution. + +## Per-Table Volume Chooser ## + +The `VolumeChooser` interface is a server-side extension point that allows user +tables to provide custom logic in choosing where its files are written when +multiple HDFS instances are available. By default, a randomized volume chooser +implementation is used to evenly balance files across all HDFS instances. + +Previously, this VolumeChooser logic was instance-wide which meant that it would +affect all tables. This is potentially undesirable as it might unintentionally +impact other users in a multi-tenant system. [ACCUMULO-3177][ACCUMULO-3177] +introduces a new per-table property which supports configuration of a +`VolumeChooser`. This ensures that the implementation to choose how HDFS +utilization happens when multiple are available is limited to the expected +subset of all tables. + +## Table and namespace custom properties ## + +In order to avoid errors caused by mis-typed configuration properties, Accumulo was strict about which configuration properties +could be set. However, this prevented users from setting arbitrary properties that could be used by custom balancers, compaction +strategies, volume choosers, and iterators. Under [ACCUMULO-2841][ACCUMULO-2841], the ability to set arbitrary table and +namespace properties was added. The properties need to be prefixed with `table.custom.`. The changes made in +[ACCUMULO-3177][ACCUMULO-3177] and [ACCUMULO-3439][ACCUMULO-3439] leverage this new feature. + +# Notable Bug Fixes # + +## SourceSwitchingIterator Deadlock ## + +An instance of SourceSwitchingIterator, the Accumulo iterator which +transparently manages whether data for a tablet read from memory (the +in-memory map) or disk (HDFS after a minor compaction), was found deadlocked +in a production system. + +This deadlock prevented the scan and the minor compaction from ever +successfully completing without restarting the tablet server. +[ACCUMULO-3745][ACCUMULO-3745] fixes the inconsistent synchronization inside +of the SourceSwitchingIterator to prevent this deadlock from happening in the +future. + +The only mitigation of this bug was to restart the tablet server that is +deadlocked. + +## Table flush blocked indefinitely ## + +While running the Accumulo RandomWalk distributed test, it was observed that +all activity in Accumulo had stopped and there was an offline Accumulo +metadata table tablet. The system first tried to flush a user tablet, but the +metadata table was not online (likely due to the agitation process which stops +and starts Accumulo processes during the test). After this call, a call to +load the metadata tablet was queued but could not complete until the previous +flush call. Thus, a deadlock occurred. + +This deadlock happened because the synchronous flush call could not complete +before the load tablet call completed, but the load tablet call couldn't run +because of connection caching we perform in Accumulo's RPC layer to reduce the +quantity of sockets we need to create to send data. +[ACCUMULO-3597][ACCUMULO-3597] prevents this deadlock by forcing the use of a +non-cached connection for the RPC message requesting a metadata tablet to be +loaded. + +While this feature does result in additional network resources to be used, the +concern is minimal because the number of metadata tablets is typically very +small with respect to the total number of tablets in the system. + +The only mitigation of this bug was to restart the tablet server that is hung. + +# Testing # + +Each unit and functional test only runs on a single node, while the RandomWalk +and Continuous Ingest tests run on any number of nodes. *Agitation* refers to +randomly restarting Accumulo processes and Hadoop DataNode processes, and, in +HDFS High-Availability instances, forcing NameNode fail-over. + +During testing, multiple Accumulo developers noticed some stability issues +with HDFS using Apache Hadoop 2.6.0 when restarting Accumulo processes and +HDFS datanodes. The developers investigated these issues as a part of the +normal release testing procedures, but were unable to find a definitive cause +of these failures. Users are encouraged to follow +[ACCUMULO-2388][ACCUMULO-2388] if they wish to follow any future developments. +One possible workaround is to increase the `general.rpc.timeout` in the +Accumulo configuration from `120s` to `240s`. + +{: #release_notes_testing .table } +| OS | Hadoop | Nodes | ZooKeeper | HDFS HA | Tests | +|--------------|--------|----------------|-----------|---------|---------------------------------------------------------------------------| +| Gentoo | N/A | 1 | N/A | No | Unit and Integration Tests | +| Gentoo | 2.6.0 | 1 (2 TServers) | 3.4.5 | No | 24hr CI w/ agitation and verification, 24hr RW w/o agitation. | +| Centos 6.6 | 2.6.0 | 3 | 3.4.6 | No | 24hr RW w/ agitation, 24hr CI w/o agitation, 72hr CI w/ and w/o agitation | +| Amazon Linux | 2.6.0 | 20 m1large | 3.4.6 | No | 24hr CI w/o agitation | + +[ACCUMULO-378]: https://issues.apache.org/jira/browse/ACCUMULO-378 +[ACCUMULO-898]: https://issues.apache.org/jira/browse/ACCUMULO-898 +[ACCUMULO-1085]: https://issues.apache.org/jira/browse/ACCUMULO-1085 +[ACCUMULO-1798]: https://issues.apache.org/jira/browse/ACCUMULO-1798 +[ACCUMULO-1817]: https://issues.apache.org/jira/browse/ACCUMULO-1817 +[ACCUMULO-1950]: https://issues.apache.org/jira/browse/ACCUMULO-1950 +[ACCUMULO-1957]: https://issues.apache.org/jira/browse/ACCUMULO-1957 +[ACCUMULO-2388]: https://issues.apache.org/jira/browse/ACCUMULO-2388 +[ACCUMULO-2815]: https://issues.apache.org/jira/browse/ACCUMULO-2815 +[ACCUMULO-2841]: https://issues.apache.org/jira/browse/ACCUMULO-2841 +[ACCUMULO-2863]: https://issues.apache.org/jira/browse/ACCUMULO-2863 +[ACCUMULO-2998]: https://issues.apache.org/jira/browse/ACCUMULO-2998 +[ACCUMULO-3134]: https://issues.apache.org/jira/browse/ACCUMULO-3134 +[ACCUMULO-3177]: https://issues.apache.org/jira/browse/ACCUMULO-3177 +[ACCUMULO-3439]: https://issues.apache.org/jira/browse/ACCUMULO-3439 +[ACCUMULO-3597]: https://issues.apache.org/jira/browse/ACCUMULO-3597 +[ACCUMULO-3657]: https://issues.apache.org/jira/browse/ACCUMULO-3657 +[ACCUMULO-3745]: https://issues.apache.org/jira/browse/ACCUMULO-3745 +[GSSAPI]: https://en.wikipedia.org/wiki/Generic_Security_Services_Application_Program_Interface +[SASL]: https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer +[api]: https://github.com/apache/accumulo/blob/1.7.0/README.md#api +[apilyzer]: http://code.revelc.net/apilyzer-maven-plugin +[group_balancer]: https://blogs.apache.org/accumulo/entry/balancing_groups_of_tablets +[kerberos]: {{ site.baseurl }}/1.7/accumulo_user_manual#_kerberos +[metrics]: {{ site.baseurl }}/1.7/accumulo_user_manual#_metrics +[readme]: https://github.com/apache/accumulo/blob/1.7.0/README.md +[replication]: {{ site.baseurl }}/1.7/accumulo_user_manual#_replication +[semver]: http://semver.org
