http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2015-06-25-accumulo-1.5.3.md ---------------------------------------------------------------------- diff --git a/_posts/release/2015-06-25-accumulo-1.5.3.md b/_posts/release/2015-06-25-accumulo-1.5.3.md new file mode 100644 index 0000000..ab12740 --- /dev/null +++ b/_posts/release/2015-06-25-accumulo-1.5.3.md @@ -0,0 +1,113 @@ +--- +title: Apache Accumulo 1.5.3 +redirect_from: /release_notes/1.5.3.html +--- + +Apache Accumulo 1.5.3 is a bug-fix release for the 1.5 series. It is likely to be the last +1.5 release, with development shifting towards newer release lines. We recommend upgrading +to a newer version to continue to get bug fixes and new features. + +In the context of Accumulo's [Semantic Versioning][semver] [guidelines][api], +this is a "patch version". This means that there should be no public API changes. Any +changes which were made were done in a backwards-compatible manner. Code that +runs against 1.5.2 should run against 1.5.3. + +We'd like to thank all of the committers and contributors which had a part in +making this release, from code contributions to testing. Everyone's efforts are +greatly appreciated. + +## Security Changes + +### [SSLv3 disabled (POODLE)][ACCUMULO-3316] + +Many Accumulo services were capable of enabling wire encryption using +SSL connectors. To be safe, [ACCUMULO-3316] disables the problematic SSLv3 version by default which was +potentially susceptible to the man-in-the-middle attack. [ACCUMULO-3317] also disables SSLv3 in the monitor, +so it will not accept SSLv3 client connections, when running it with https. + +## Notable Bug Fixes + +### [SourceSwitchingIterator Deadlock][ACCUMULO-3745] + +An instance of SourceSwitchingIterator, the Accumulo iterator which transparently manages +whether data for a tablet read from memory (the in-memory map) or disk (HDFS after a minor +compaction), was found deadlocked in a production system. + +This deadlock prevented the scan and the minor compaction from ever successfully completing +without restarting the tablet server. [ACCUMULO-3745] fixes the inconsistent synchronization +inside of the SourceSwitchingIterator to prevent this deadlock from happening in the future. + +The only mitigation of this bug was to restart the tablet server that is deadlocked. + +### [Table flush blocked indefinitely][ACCUMULO-3597] + +While running the Accumulo RandomWalk distributed test, it was observed that all activity in +Accumulo had stopped and there was an offline Accumulo metadata table tablet. The system first +tried to flush a user tablet, but the metadata table was not online (likely due to the agitation +process which stops and starts Accumulo processes during the test). After this call, a call to +load the metadata tablet was queued but could not complete until the previous flush call. Thus, +a deadlock occurred. + +This deadlock happened because the synchronous flush call could not complete before the load +tablet call completed, but the load tablet call couldn't run because of connection caching we +perform in Accumulo's RPC layer to reduce the quantity of sockets we need to create to send data. +[ACCUMULO-3597] prevents this deadlock by forcing the use of a non-cached connection for the RPC +message requesting a metadata tablet to be loaded. + +While this feature does result in additional network resources to be used, the concern is minimal +because the number of metadata tablets is typically very small with respect to the total number of +tablets in the system. + +The only mitigation of this bug was to restart the tablet server that is hung. + +### [RPC Connections not cached][ACCUMULO-3574] + +It was observed that the underlying connection for invoking RPCs were not actually being cached, +despite it being requested that they should be cached. While this did not result in a noticed +performance impact, it was deficiency. [ACCUMULO-3574] ensures that connections are cached when +it is requested that they are. + +### [Deletes on Apache Thrift Proxy API ignored][ACCUMULO-3474] + +A user noted that when trying to specify a delete using the Accumulo Thrift Proxy, the delete +was treated as an update. [ACCUMULO-3474] fixes the Proxy server such that deletes are properly +respected as specified by the client. + +## Other Changes + +Other changes for this version can be found [in JIRA][CHANGES]. + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk +and Continuous Ingest tests run on any number of nodes. *Agitation* refers to +randomly restarting Accumulo processes and Hadoop DataNode processes, and, in +HDFS High-Availability instances, forcing NameNode fail-over. + +During testing, multiple Accumulo developers noticed some stability issues +with HDFS using Apache Hadoop 2.6.0 when restarting Accumulo processes and +HDFS datanodes. The developers investigated these issues as a part of the +normal release testing procedures, but were unable to find a definitive cause +of these failures. Users are encouraged to follow +[ACCUMULO-2388][ACCUMULO-2388] if they wish to follow any future developments. +One possible workaround is to increase the `general.rpc.timeout` in the +Accumulo configuration from `120s` to `240s`. + +{: #release_notes_testing .table } +| OS | Hadoop | Nodes | ZooKeeper | HDFS High-Availability | Tests | +|------------|--------|-------|-----------|------------------------|------------------------------| +| Gentoo | 2.6.0 | 1 | 3.4.5 | No | Unit and Integration Tests | +| Centos 6.5 | 2.7.1 | 6 | 3.4.5 | No | Continuous Ingest and Verify | + +[ACCUMULO-3316]: https://issues.apache.org/jira/browse/ACCUMULO-3316 +[ACCUMULO-3317]: https://issues.apache.org/jira/browse/ACCUMULO-3317 +[ACCUMULO-2388]: https://issues.apache.org/jira/browse/ACCUMULO-2388 +[ACCUMULO-3474]: https://issues.apache.org/jira/browse/ACCUMULO-3474 +[ACCUMULO-3574]: https://issues.apache.org/jira/browse/ACCUMULO-3574 +[ACCUMULO-3597]: https://issues.apache.org/jira/browse/ACCUMULO-3597 +[ACCUMULO-3745]: https://issues.apache.org/jira/browse/ACCUMULO-3745 +[api]: https://github.com/apache/accumulo/blob/1.7.0/README.md#api +[readme]: https://github.com/apache/accumulo/blob/1.5.3/README.md +[semver]: http://semver.org +[CHANGES]: https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12328662 +[REL_152]: {{ site.baseurl }}/release_notes/1.5.2
http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2015-07-04-accumulo-1.6.3.md ---------------------------------------------------------------------- diff --git a/_posts/release/2015-07-04-accumulo-1.6.3.md b/_posts/release/2015-07-04-accumulo-1.6.3.md new file mode 100644 index 0000000..2de1679 --- /dev/null +++ b/_posts/release/2015-07-04-accumulo-1.6.3.md @@ -0,0 +1,113 @@ +--- +title: Apache Accumulo 1.6.3 +redirect_from: /release_notes/1.6.3.html +--- + +Apache Accumulo 1.6.3 is a maintenance release on the 1.6 version branch. +This release contains changes from over 63 issues, comprised of bug-fixes, +performance improvements and better test cases. See [JIRA][JIRA_163] for a +complete list. + +Users of 1.6.0, 1.6.1, and 1.6.2 are strongly encouraged to update as soon as +possible to benefit from the improvements with very little concern in change +of underlying functionality. Users of 1.4 or 1.5 that are seeking to upgrade +to 1.6 should consider 1.6.3 as a starting point. For information about +improvements since Accumulo 1.5, see the [1.6.0][3], [1.6.1][4], and +[1.6.2][5] release notes. + +## Fixed BatchWriter hold time error + +In previous releases, a `BatchWriter` could fail with a +`MutationsRejectedException` with server errors. If inspection of the tserver +logs showed `HoldTimeoutException` was the cause, the workaround was to +increase the value of `general.rpc.timeout`. Changing this setting is no +longer necessary as this bug was fixed by [ACCUMULO-2388][ACCUMULO-2388]. + +## Severe bug fixes + + * [ACCUMULO-3597][ACCUMULO-3597] Fixed a deadlock where a table flush and + metadata tablet load were waiting on each other. This was a rare bug. If it + occurred it could impact the availability of Accumulo as most Accumulo + operations depend on metadata tablets. + * [ACCUMULO-3709][ACCUMULO-3709] Fixed a potential data loss bug where + AccumuloOutputFormat close did not rethrow exception. + * [ACCUMULO-3745][ACCUMULO-3745] Fixed a deadlock in SourceSwitchingIterator + that occurred when using custom iterators that called `deepCopy`. This bug + would cause scans to hang indefinitely until the offending tserver was killed. + * [ACCUMULO-3859][ACCUMULO-3859] Fixed a race condition that could prevent table + constraints from ever loading for a Tablet. It is likely to only affect users + when the constraint is first added to a table. + +## Notable bug fixes + + * [ACCUMULO-3589][ACCUMULO-3589] `du` in Shell does not check table existence. + * [ACCUMULO-3692][ACCUMULO-3692] Offline'ing a table disabled subsequent balancing. + * [ACCUMULO-3696][ACCUMULO-3696] Tracing could queue too many traces + * [ACCUMULO-3718][ACCUMULO-3718] Fixed a bug that prevented a Mutation from + being created in Scala. + * [ACCUMULO-3747][ACCUMULO-3747] Thrashing tablet servers would be removed from the Monitor's Dead Server list. + * [ACCUMULO-3750][ACCUMULO-3750] Fixed an issue where the Master would perpetually + fail when there was a bad `instance.secret` setting. + * [ACCUMULO-3784][ACCUMULO-3784] Fixed a bug in `getauths` Shell command where it + treated visibilities that differed only in case as the same. + * [ACCUMULO-3796][ACCUMULO-3796] Added documentation about turning off zone + reclaim. + * [ACCUMULO-3880][ACCUMULO-3880] Fixed an issue where malformed configuration caused + TabletServers to shutdown. + * [ACCUMULO-3890][ACCUMULO-3890] Fixed a performance issue with CredentialProvider. Information + stored in the CredentialProvider was not cached which resulted in repeatedly reading the + file from HDFS which can degrade HDFS performance. + +## Known Issues + +During testing [HDFS-8406][1] was encountered which is summarized by write-ahead log recovery +that was never completed due to an inability to recover the HDFS lease on the WAL. To work around +this issue, the following steps can be done: + + 1. Locate block for walog whose lease can not be recovered. + 2. Copy block into HDFS as temp file TMP_WALOG + 3. Delete the walog whose lease can not be recovered. + 4. Move TMP_WALOG to the filename of the walog deleted in the previous step. + +Using the `fetchColumns()` method on a scanner in conjunction with custom iterators that +add column families in their `seek()` method can lead to unexpected behavior. See +[ACCUMULO-3905][ACCUMULO-3905] for more details. In that issue javadoc updates were made, +but the updates did not make it into 1.6.3. + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk +and Continuous Ingest tests run on any number of nodes. *Agitation* refers to +randomly restarting Accumulo processes and Hadoop Datanode processes, and, in +HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS | Hadoop | Nodes | ZooKeeper | HDFS HA | Tests | +|----------------------|--------|-------|-----------|---------|-----------------------------------------------------------------| +| Amazon Linux 2014.09 | 2.6.0 | 20 | 3.4.5 | No | 24hr ContinuousIngest w/ verification w/ and w/o agitation | +| Amazon Linux 2014.09 | 2.6.0 | 20 | 3.4.5 | No | 24hr Randomwalk w/o agitation | +| Centos 6.5 | 2.7.1 | 6 | 3.4.5 | No | Continuous Ingest and Verify (6B entries) | +| Centos 6.6 | 2.2.0 | 6 | 3.4.5 | No | All integration test passed. Some needed to be run a 2nd time. | + +[1]: https://issues.apache.org/jira/browse/HDFS-8406 +[3]: {{ site.baseurl }}/release_notes/1.6.0 +[4]: {{ site.baseurl }}/release_notes/1.6.1 +[5]: {{ site.baseurl }}/release_notes/1.6.2 + +[ACCUMULO-2388]: https://issues.apache.org/jira/browse/ACCUMULO-2388 +[ACCUMULO-3589]: https://issues.apache.org/jira/browse/ACCUMULO-3589 +[ACCUMULO-3597]: https://issues.apache.org/jira/browse/ACCUMULO-3597 +[ACCUMULO-3692]: https://issues.apache.org/jira/browse/ACCUMULO-3692 +[ACCUMULO-3696]: https://issues.apache.org/jira/browse/ACCUMULO-3696 +[ACCUMULO-3709]: https://issues.apache.org/jira/browse/ACCUMULO-3709 +[ACCUMULO-3718]: https://issues.apache.org/jira/browse/ACCUMULO-3718 +[ACCUMULO-3745]: https://issues.apache.org/jira/browse/ACCUMULO-3745 +[ACCUMULO-3747]: https://issues.apache.org/jira/browse/ACCUMULO-3747 +[ACCUMULO-3750]: https://issues.apache.org/jira/browse/ACCUMULO-3750 +[ACCUMULO-3784]: https://issues.apache.org/jira/browse/ACCUMULO-3784 +[ACCUMULO-3796]: https://issues.apache.org/jira/browse/ACCUMULO-3796 +[ACCUMULO-3859]: https://issues.apache.org/jira/browse/ACCUMULO-3859 +[ACCUMULO-3880]: https://issues.apache.org/jira/browse/ACCUMULO-3880 +[ACCUMULO-3890]: https://issues.apache.org/jira/browse/ACCUMULO-3890 +[ACCUMULO-3905]: https://issues.apache.org/jira/browse/ACCUMULO-3905 +[JIRA_163]: https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12329154 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2015-09-19-accumulo-1.5.2.md ---------------------------------------------------------------------- diff --git a/_posts/release/2015-09-19-accumulo-1.5.2.md b/_posts/release/2015-09-19-accumulo-1.5.2.md new file mode 100644 index 0000000..d876840 --- /dev/null +++ b/_posts/release/2015-09-19-accumulo-1.5.2.md @@ -0,0 +1,179 @@ +--- +title: Apache Accumulo 1.5.2 +redirect_from: /release_notes/1.5.2.html +--- + +Apache Accumulo 1.5.2 is a maintenance release on the 1.5 version branch. +This release contains changes from over 100 issues, comprised of bug fixes +(client side and server side), new test cases, and updated Hadoop support +contributed by over 30 different contributors and committers. +As this is a maintenance release, Apache Accumulo 1.5.2 has no client API +incompatibilities over Apache Accumulo 1.5.0 and 1.5.1 and requires no manual upgrade +process. Users of 1.5.0 or 1.5.1 are strongly encouraged to update as soon as possible +to benefit from the improvements. + +Users who are new to Accumulo are encouraged to use a 1.6 release as opposed +to the 1.5 line as development has already shifted towards the 1.6 line. For those +who cannot or do not want to upgrade to 1.6, 1.5.2 is still an excellent choice +over earlier versions in the 1.5 line. + + +## Performance Improvements + +Apache Accumulo 1.5.2 includes a number of performance-related fixes over previous versions. + + +### Write-Ahead Log sync performance + +The Write-Ahead Log (WAL) files are used to ensure durability of updates made to Accumulo. +A sync is called on the file in HDFS to make sure that the changes to the WAL are persisted +to disk, which allows Accumulo to recover in the case of failure. [ACCUMULO-2766][9] fixed +an issue where an operation against a WAL would unnecessarily wait for multiple syncs, slowing +down the ingest on the system. + +### Minor-Compactions not aggressive enough + +On a system with ample memory provided to Accumulo, long hold-times were observed which +blocks the ingest of new updates. Trying to free more server-side memory by running minor +compactions more frequently increased the overall throughput on the node. These changes +were made in [ACCUMULO-2905][10]. + +### HeapIterator optimization + +Iterators, a notable feature of Accumulo, are provided to users as a server-side programming +construct, but are also used internally for numerous server operations. One of these system iterator +is the HeapIterator which implements a PriorityQueue of other Iterators. One way this iterator is +used is to merge multiple files in HDFS to present a single, sorted stream of Key-Value pairs. [ACCUMULO-2827][11] +introduces a performance optimization to the HeapIterator which can improve the speed of the +HeapIterator in common cases. + +### Write-Ahead log sync implementation + +In Hadoop-2, two implementations of sync are provided: hflush and hsync. Both of these +methods provide a way to request that the datanodes write the data to the underlying +medium and not just hold it in memory (the *fsync* syscall). While both of these methods +inform the Datanodes to sync the relevant block(s), *hflush* does not wait for acknowledgement +from the Datanodes that the sync finished, where *hsync* does. To provide the most reliable system +"out of the box", Accumulo defaults to *hsync* so that your data is as secure as possible in +a variety of situations (notably, unexpected power outages). + +The downside is that performance tends to suffer because waiting for a sync to disk is a very +expensive operation. [ACCUMULO-2842][12] introduces a new system property, tserver.wal.sync.method, +that lets users to change the HDFS sync implementation from *hsync* to *hflush*. Using *hflush* instead +of *hsync* may result in about a 30% increase in ingest performance. + +For users upgrading from Hadoop-1 or Hadoop-0.20 releases, *hflush* is the equivalent of how +sync was implemented in these older versions of Hadoop and should give comparable performance. + +### Server-side mutation queue size + +When users desire writes to be as durable as possible, using *hsync*, the ingest performance +of the system can be improved by increasing the tserver.mutation.queue.max property. The cost +of this change is that it will cause TabletServers to use additional memory per writer. In 1.5.1, +the value of this parameter defaulted to a conservative 256K, which resulted in sub-par ingest +performance. + +1.5.2 and [ACCUMULO-3018][13] increases this buffer to 1M which has a noticeable positive impact on +ingest performance with a minimal increase in TabletServer memory usage. + +## Notable Bug Fixes + +### Fixes MapReduce package name change + +1.5.1 inadvertently included a change to RangeInputSplit which created an incompatibility +with 1.5.0. The original class has been restored to ensure that users accessing +the RangeInputSplit class do not have to alter their client code. See [ACCUMULO-2586][1] for +more information + +### Add configurable maximum frame size to Apache Thrift proxy + +The Thrift proxy server was subject to memory exhaustion, typically +due to bad input, where the server would attempt to allocate a very large +buffer and die in the process. [ACCUMULO-2658][2] introduces a configuration +parameter, like [ACCUMULO-2360][3], to prevent this error. + +### Offline tables can prevent tablet balancing + +Before 1.5.2, when a table with many tablets was created, ingested into, and +taken offline, tablet balancing may have stoppped. This would happen if there +were tablet migrations for the table, because the migrations couldn't occur. +The balancer will not run when there are outstanding migrations; therefore, a +system could become unbalanced. [ACCUMULO-2694][4] introduces a fix to ensure +that offline tables do not block balancing and improves the server-side +logging. + +### MiniAccumuloCluster process management + +MiniAccumuloCluster had a few issues which could cause deadlock or a method that +never returns. Most of these are related to management of the Accumulo processes +([ACCUMULO-2764][5], [ACCUMULO-2985][6], and [ACCUMULO-3055][7]). + +### IteratorSettings not correctly serialized in RangeInputSplit + +The Writable interface methods on the RangeInputSplit class accidentally omitted +calls to serialize the IteratorSettings configured for the Job. [ACCUMULO-2962][8] +fixes the serialization and adds some additional tests. + +### Constraint violation causes hung scans + +A failed bulk import transaction had the ability to create an infinitely retrying +loop due to a constraint violation. This directly prevents scans from completing, +but will also hang compactions. [ACCUMULO-3096][14] fixes the issue so that the +constraint no longer hangs the entire system. + +## Documentation + +The following documentation updates were made: + + * [ACCUMULO-2540][15] + * [ACCUMULO-2767][16] + * [ACCUMULO-2796][17] + * [ACCUMULO-2443][18] + * [ACCUMULO-3008][19] + * [ACCUMULO-2919][20] + * [ACCUMULO-2874][21] + * [ACCUMULO-2653][22] + * [ACCUMULO-2437][23] + * [ACCUMULO-3097][24] + * [ACCUMULO-2499][25] + * [ACCUMULO-1669][26] + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk and Continuous Ingest tests run +on any number of nodes. *Agitation* refers to randomly restarting Accumulo processes and Hadoop Datanode processes, +and, in HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS | Hadoop | Nodes | ZooKeeper | HDFS High-Availability | Tests | +|----------|-----------------------|-------|--------------|------------------------|-----------------------------------------------------------------------------------------------------| +| Gentoo | Apache 2.6.0-SNAPSHOT | 1 | Apache 3.4.5 | No | Unit and Functional Tests, ContinuousIngest w/ verification (1B entries) | +| CentOS 6 | Apache 2.3.0 | 20 | Apache 3.4.5 | No | 24/hr RandomWalk, 24/hr ContinuousIngest w/ verification w/ and w/o agitation (30B and 23B entries) | + + +[1]: https://issues.apache.org/jira/browse/ACCUMULO-2586 +[2]: https://issues.apache.org/jira/browse/ACCUMULO-2658 +[3]: https://issues.apache.org/jira/browse/ACCUMULO-2360 +[4]: https://issues.apache.org/jira/browse/ACCUMULO-2694 +[5]: https://issues.apache.org/jira/browse/ACCUMULO-2764 +[6]: https://issues.apache.org/jira/browse/ACCUMULO-2985 +[7]: https://issues.apache.org/jira/browse/ACCUMULO-3055 +[8]: https://issues.apache.org/jira/browse/ACCUMULO-2962 +[9]: https://issues.apache.org/jira/browse/ACCUMULO-2766 +[10]: https://issues.apache.org/jira/browse/ACCUMULO-2905 +[11]: https://issues.apache.org/jira/browse/ACCUMULO-2827 +[12]: https://issues.apache.org/jira/browse/ACCUMULO-2842 +[13]: https://issues.apache.org/jira/browse/ACCUMULO-3018 +[14]: https://issues.apache.org/jira/browse/ACCUMULO-3096 +[15]: https://issues.apache.org/jira/browse/ACCUMULO-2540 +[16]: https://issues.apache.org/jira/browse/ACCUMULO-2767 +[17]: https://issues.apache.org/jira/browse/ACCUMULO-2796 +[18]: https://issues.apache.org/jira/browse/ACCUMULO-2443 +[19]: https://issues.apache.org/jira/browse/ACCUMULO-3008 +[20]: https://issues.apache.org/jira/browse/ACCUMULO-2919 +[21]: https://issues.apache.org/jira/browse/ACCUMULO-2874 +[22]: https://issues.apache.org/jira/browse/ACCUMULO-2653 +[23]: https://issues.apache.org/jira/browse/ACCUMULO-2437 +[24]: https://issues.apache.org/jira/browse/ACCUMULO-3097 +[25]: https://issues.apache.org/jira/browse/ACCUMULO-2499 +[26]: https://issues.apache.org/jira/browse/ACCUMULO-1669 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2015-09-21-accumulo-1.5.4.md ---------------------------------------------------------------------- diff --git a/_posts/release/2015-09-21-accumulo-1.5.4.md b/_posts/release/2015-09-21-accumulo-1.5.4.md new file mode 100644 index 0000000..336ca06 --- /dev/null +++ b/_posts/release/2015-09-21-accumulo-1.5.4.md @@ -0,0 +1,68 @@ +--- +title: Apache Accumulo 1.5.4 +redirect_from: /release_notes/1.5.4.html +--- + +Apache Accumulo 1.5.4 is one more bug-fix release for the 1.5 series. Like 1.5.3 before it, this release contains a +very small changeset when considering the normal size of changes in a release. + +This release contains no changes to the [public API][api]. As such, there are no concerns +for the compatibility of user code running against 1.5.3. All users are encourage to upgrade +immediately without concern of stability and compatibility. + +A full list of changes is available via [CHANGES][CHANGES]. + +We'd like to thank all of the committers and contributors which had a part in +making this release, from code contributions to testing. Everyone's efforts are +greatly appreciated. + +## Correctness Bugs + +### Silent data-loss via bulk imported files + +A user recently reported that a simple bulk-import application would occasionally lose some records. Through investigation, +it was found that when bulk imports into a table failed the initial assignment, the logic that automatically retries +the imports was incorrectly choosing the tablets to import the files into. [ACCUMULO-3967][ACCUMULO-3967] contains +more information on the cause and identification of the bug. The data-loss condition would only affect entire files. +If records from a file exist in Accumulo, it is still guaranteed that all records within that imported file were +successful. + +As such, users who have bulk import applications using previous versions of Accumulo should verify that all of their +data was correctly ingested into Accumulo and immediately update to Accumulo 1.5.4. + +Thanks to Edward Seidl for reporting this bug to us! + +## Server-side auditing changes + +Thanks to James Mello for reporting and providing the fixes to the following server-side auditing issues. + +### Incorrect audit initialization + +It was observed that the implementation used to audit user API requests on Accumulo server processes +was not being correctly initialized which caused audit messages to never be generated. This was rectified +in [ACCUMULO-3939][ACCUMULO-3939]. + +### Missing audit implementations + +It was also observed that some server-side API implementations did not include audit messages which resulted +in an incomplete historical picture on what operations a user might have invoked. The missing audits (and those +that were added) are described in [ACCUMULO-3946][ACCUMULO-3946]. + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk +and Continuous Ingest tests run on any number of nodes. *Agitation* refers to +randomly restarting Accumulo processes and Hadoop DataNode processes, and, in +HDFS High-Availability instances, forcing NameNode fail-over. + +{: #release_notes_testing .table } +| OS | Hadoop | Nodes | ZooKeeper | HDFS High-Availability | Tests | +|------------|--------|-------|-----------|------------------------|----------------------------------------------------------------| +| OSX | 2.6.0 | 1 | 3.4.5 | No | Unit and Functional Tests | +| Centos 6.5 | 2.7.1 | 6 | 3.4.5 | No | Continuous Ingest and Verify (10B entries), Randomwalk (24hrs) | + +[ACCUMULO-3967]: https://issues.apache.org/jira/browse/ACCUMULO-3967 +[ACCUMULO-3939]: https://issues.apache.org/jira/browse/ACCUMULO-3939 +[ACCUMULO-3946]: https://issues.apache.org/jira/browse/ACCUMULO-3946 +[api]: https://github.com/apache/accumulo/blob/1.7.0/README.md#api +[CHANGES]: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12312121&version=12333106 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2015-10-03-accumulo-1.6.4.md ---------------------------------------------------------------------- diff --git a/_posts/release/2015-10-03-accumulo-1.6.4.md b/_posts/release/2015-10-03-accumulo-1.6.4.md new file mode 100644 index 0000000..40dfe3d --- /dev/null +++ b/_posts/release/2015-10-03-accumulo-1.6.4.md @@ -0,0 +1,69 @@ +--- +title: Apache Accumulo 1.6.4 +redirect_from: /release_notes/1.6.4.html +--- + +Apache Accumulo 1.6.4 is a maintenance release on the 1.6 version branch. +This release contains changes from 21 issues, comprised of bug-fixes, +performance improvements and better test cases. See [JIRA][JIRA_164] for a +complete list. + +Users of any previous 1.6.x release are strongly encouraged to update as soon as +possible to benefit from the improvements with very little concern in change +of underlying functionality. Users of 1.4 or 1.5 that are seeking to upgrade +to 1.6 should consider 1.6.4 as a starting point. + +## Silent data-loss via bulk imported files + +A user recently reported that a simple bulk-import application would occasionally +lose some records. Through investigation, it was found that when bulk imports into +a table failed the initial assignment, the logic that automatically retries the +imports was incorrectly choosing the tablets to import the files into. [ACCUMULO-3967][ACCUMULO-3967] +contains more information on the cause and identification of the bug. The data-loss +condition would only affect entire files. If records from a file exist in Accumulo, +it is still guaranteed that all records within that imported file were successful. + +As such, users who have bulk import applications using previous versions of Accumulo +should verify that all of their data was correctly ingested into Accumulo and +immediately update to Accumulo 1.6.4. + +## Other bug fixes + + * [ACCUMULO-3979][ACCUMULO-3979] Fixed an issue where the BulkImporter failed + with an error message "QUERY_METADATA already started". + * [ACCUMULO-3965][ACCUMULO-3965] The `listscans` shell command did not contain + the `scanId` attribute for currently running scans. + * [ACCUMULO-3946][ACCUMULO-3946] Verified that all user-facing operations contained + appropriate audit messages. + * [ACCUMULO-3977][ACCUMULO-3977] Isolated scans with Iterators in use incorrectly + fail around invocation of `deepCopy`. + * [ACCUMULO-3905][ACCUMULO-3905] RowDeletingIterator functions incorrectly when + columns are provided by the client. This restores intended functionality without + the need for a [workaround][3905-workaround]. + * [ACCUMULO-3959][ACCUMULO-3959] [ACCUMULO-3934][ACCUMULO-3934] Multiple documentation + improvements to `BatchScanner`. + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk +and Continuous Ingest tests run on any number of nodes. *Agitation* refers to +randomly restarting Accumulo processes and Hadoop Datanode processes, and, in +HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS | Hadoop | Nodes | ZooKeeper | HDFS HA | Tests | +|----------------------|--------|-------|-----------|---------|---------------------------------------------------------------------| +| Amazon Linux 2014.09 | 2.6.0 | 20 | 3.4.5 | No | ContinuousIngest w/ verification w/ and w/o agitation (37B entries) | + +[ACCUMULO-3979]: https://issues.apache.org/jira/browse/ACCUMULO-3979 +[ACCUMULO-3965]: https://issues.apache.org/jira/browse/ACCUMULO-3965 +[ACCUMULO-3946]: https://issues.apache.org/jira/browse/ACCUMULO-3946 +[ACCUMULO-3977]: https://issues.apache.org/jira/browse/ACCUMULO-3977 +[ACCUMULO-3905]: https://issues.apache.org/jira/browse/ACCUMULO-3905 +[3905-workaround]: https://issues.apache.org/jira/browse/ACCUMULO-1801?focusedCommentId=13970204&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13970204 +[ACCUMULO-3959]: https://issues.apache.org/jira/browse/ACCUMULO-3959 +[ACCUMULO-3934]: https://issues.apache.org/jira/browse/ACCUMULO-3934 +[ACCUMULO-3967]: https://issues.apache.org/jira/browse/ACCUMULO-3967 + + +[JIRA_164]: https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12332840 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2016-02-17-accumulo-1.6.5.md ---------------------------------------------------------------------- diff --git a/_posts/release/2016-02-17-accumulo-1.6.5.md b/_posts/release/2016-02-17-accumulo-1.6.5.md new file mode 100644 index 0000000..efc4e98 --- /dev/null +++ b/_posts/release/2016-02-17-accumulo-1.6.5.md @@ -0,0 +1,110 @@ +--- +title: Apache Accumulo 1.6.5 +redirect_from: /release_notes/1.6.5.html +--- + +Apache Accumulo 1.6.5 is a maintenance release on the 1.6 version branch. This +release contains changes from 55 issues, comprised of bug-fixes, performance +improvements, build quality improvements, and more. See [JIRA][JIRA_165] for a +complete list. + +Users of any previous 1.6.x release are strongly encouraged to update as soon as +possible to benefit from the improvements with very little concern in change of +underlying functionality. Users of 1.4 or 1.5 that are seeking to upgrade to 1.6 +should consider 1.6.5 as a starting point. + +## Outstanding Known Issues + +Be aware that a small documentation bug exists with the compact command in the +shell ([ACCUMULO-4138][ACCUMULO-4138]). The documentation for the begin row and +end row should be described as exclusive and inclusive, respectively, rather +than the incorrect description of both being inclusive. + +## Highlights + +### Queued Compactions Not Running + +Found and fixed a bug ([ACCUMULO-4016][ACCUMULO-4016]) in which some queued +compactions would never run if the number of files changed while the tablet was +queued. + +### Faster Processing of Conditional Mutations + +Improved ConditionalMutation processing time by a factor of 3. +([ACCUMULO-4066][ACCUMULO-4066]) + +### Slow GC While Bulk Importing + +Found and worked around an issue where lots of bulk imports creating many new +files would significantly impair the Accumulo GC service, and possibly prevent +it from running to completion entirely. ([ACCUMULO-4021][ACCUMULO-4021]) + +### Improvements in Locating Client Configuration File + +Fixed some unexpected error messages related to setting +ACCUMULO_CLIENT_CONF_PATH, and improved the detection of the client.conf file if +ACCUMULO_CLIENT_CONF_PATH was set to a directory containing client.conf. +([ACCUMULO-4026][ACCUMULO-4026],[ACCUMULO-4027][ACCUMULO-4027]) + +### Transient ZooKeeper disconnect causes FATE threads to exit + +ZooKeeper clients are expected to handle the situation where they become +disconnected from the ZooKeeper server and must wait to be reconnected +before continuing ZooKeeper operations. + +The dedicated threads running inside the Accumulo Master process for FATE +actions had the potential unexpectedly exit in this disconnected state. +This caused a scenario where all future FATE-based operations would +be blocked until the Accumulo Master process was restarted. ([ACCUMULO-4060][ACCUMULO-4060]) + +### Incorrect management of certain Apache Thrift RPCs + +Accumulo relies on Apache Thrift to implement remote procedure calls between +Accumulo services. Accumulo's use of Thrift uncovered an unfortunate situation +where a special RPC (a "oneway" call) would leave unwanted data on the underlying +Thrift connection. After this extra data was left on connection, all subsequent RPCs +re-using that connection would fail with "out of sequence response" error messages. +Accumulo would be left in a bad state until the mishandled connections were released +or Accumulo services were restarted. ([ACCUMULO-4065][ACCUMULO-4065]) + +## Other Notable Changes + + * [ACCUMULO-3509][ACCUMULO-3509] Fixed some lock contention in TabletServer, preventing resource cleanup + * [ACCUMULO-3734][ACCUMULO-3734] Fixed quote-escaping bug in VisibilityConstraint + * [ACCUMULO-4025][ACCUMULO-4025] Fixed cleanup of bulk load fate transactions + * [ACCUMULO-4070][ACCUMULO-4070] Fixed Kerberos ticket renewal for all Accumulo services + * [ACCUMULO-4098][ACCUMULO-4098],[ACCUMULO-4113][ACCUMULO-4113] Fixed widespread misuse of ByteBuffer + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk +and Continuous Ingest tests run on any number of nodes. *Agitation* refers to +randomly restarting Accumulo processes and Hadoop Datanode processes, and, in +HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS | Hadoop | Nodes | ZooKeeper | HDFS HA | Tests | +|--------------------------|-----------------|-------|-----------|---------|-----------------------------------------------------------------------------------------------------------------| +| CentOS 7.1 | 2.6.3 | 9 | 3.4.6 | No | Random walk (All.xml) 18-hour run (2 failures, both conflicting operations on same table in Concurrent test) | +| CentOS 7.1 | 2.6.3 | 6 | 3.4.6 | No | Continuous ingest with agitation (2B entries) | +| CentOS 6.7 | 2.2.0 and 1.2.1 | 1 | 3.3.6 | No | All unit and integration tests | +| CentOS 7.1 (Oracle JDK8) | 2.6.3 | 9 | 3.4.6 | No | Continuous ingest with agitation (24hrs, 32B entries verified) on EC2 (1 m3.xlarge leader; 8 d2.xlarge workers) | + + +[JIRA_165]: https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12333674 + +[ACCUMULO-3509]: https://issues.apache.org/jira/browse/ACCUMULO-3509 +[ACCUMULO-3734]: https://issues.apache.org/jira/browse/ACCUMULO-3734 +[ACCUMULO-4016]: https://issues.apache.org/jira/browse/ACCUMULO-4016 +[ACCUMULO-4021]: https://issues.apache.org/jira/browse/ACCUMULO-4021 +[ACCUMULO-4025]: https://issues.apache.org/jira/browse/ACCUMULO-4025 +[ACCUMULO-4026]: https://issues.apache.org/jira/browse/ACCUMULO-4026 +[ACCUMULO-4027]: https://issues.apache.org/jira/browse/ACCUMULO-4027 +[ACCUMULO-4060]: https://issues.apache.org/jira/browse/ACCUMULO-4060 +[ACCUMULO-4065]: https://issues.apache.org/jira/browse/ACCUMULO-4065 +[ACCUMULO-4066]: https://issues.apache.org/jira/browse/ACCUMULO-4066 +[ACCUMULO-4070]: https://issues.apache.org/jira/browse/ACCUMULO-4070 +[ACCUMULO-4098]: https://issues.apache.org/jira/browse/ACCUMULO-4098 +[ACCUMULO-4113]: https://issues.apache.org/jira/browse/ACCUMULO-4113 +[ACCUMULO-4138]: https://issues.apache.org/jira/browse/ACCUMULO-4138 + http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2016-02-26-accumulo-1.7.1.md ---------------------------------------------------------------------- diff --git a/_posts/release/2016-02-26-accumulo-1.7.1.md b/_posts/release/2016-02-26-accumulo-1.7.1.md new file mode 100644 index 0000000..ee64607 --- /dev/null +++ b/_posts/release/2016-02-26-accumulo-1.7.1.md @@ -0,0 +1,150 @@ +--- +title: Apache Accumulo 1.7.1 +redirect_from: /release_notes/1.7.1.html +--- + +Apache Accumulo 1.7.1 is a maintenance release on the 1.7 version branch. This +release contains changes from more than 150 issues, comprised of bug-fixes, +performance improvements, build quality improvements, and more. See +[JIRA][JIRA_171] for a complete list. + +Users of any previous 1.7.x release are strongly encouraged to update as soon +as possible to benefit from the improvements with very little concern in change +of underlying functionality. Users of 1.6 or earlier that are seeking to +upgrade to 1.7 should consider 1.7.1 as a starting point. + +## Highlights + +### Silent data-loss via bulk imported files + +A user recently reported that a simple bulk-import application would +occasionally lose some records. Through investigation, it was found that when +bulk imports into a table failed the initial assignment, the logic that +automatically retries the imports was incorrectly choosing the tablets to +import the files into. [ACCUMULO-3967][ACCUMULO-3967] contains more information +on the cause and identification of the bug. The data-loss condition would only +affect entire files. If records from a file exist in Accumulo, it is still +guaranteed that all records within that imported file were successful. + +As such, users who have bulk import applications using previous versions of +Accumulo should verify that all of their data was correctly ingested into +Accumulo and immediately update to Accumulo 1.7.1 (This is the same bug that +was fixed in 1.6.4, so you won't be affected if you're running 1.6.4 or newer). + +### Queued Compactions Not Running + +Found and fixed a bug ([ACCUMULO-4016][ACCUMULO-4016]) in which some queued +compactions would never run if the number of files changed while the tablet was +queued. + +### Kerberos Ticket Renewals + +A bug was fixed which caused Accumulo clients and services to fail to check and +(if necessary) renew their Kerberos credentials. This would eventually lead to +these components failing to properly authenticate until they were restarted. +([ACCUMULO-4069][ACCUMULO-4069]) + +### Updated commons-collection + +The bundled commons-collection library was updated from version 3.2.1 to 3.2.2 +because of a reported vulnerability in that library. +([ACCUMULO-4056][ACCUMULO-4056]) + +### Faster Processing of Conditional Mutations + +Improved ConditionalMutation processing time by a factor of 3. +([ACCUMULO-4066][ACCUMULO-4066]) + +### Slow GC While Bulk Importing + +Found and worked around an issue where lots of bulk imports creating many new +files would significantly impair the Accumulo GC service, and possibly prevent +it from running to completion entirely. ([ACCUMULO-4021][ACCUMULO-4021]) + +### Unnoticed Per-table Configuration Updates + +Fixed a bug which caused tablet servers to not notice changes to the per-table +constraints, under some circumstances. ([ACCUMULO-3859][ACCUMULO-3859]) + +### TabletServers kill themselves on CentOS7 + +Reduced the aggressiveness with which Accumulo Tablet Servers preemptively +killed themselves when a local filesystem switched to read-only (indicating a +possible failure). To reduce false positives, such as those which can occur +with systemd's extra cgroup mounts in CentOS7, an additional check was added to +ensure that tablet servers would only kill themselves if an ext- or +xfs-formatted disk switched to read-only. ([ACCUMULO-4080][ACCUMULO-4080]) + +### Improvements in Locating Client Configuration File + +Fixed some unexpected error messages related to setting +ACCUMULO_CLIENT_CONF_PATH, and improved the detection of the client.conf file if +ACCUMULO_CLIENT_CONF_PATH was set to a directory containing client.conf. +([ACCUMULO-4026][ACCUMULO-4026],[ACCUMULO-4027][ACCUMULO-4027]) + +### Transient ZooKeeper disconnect causes FATE threads to exit + +ZooKeeper clients are expected to handle the situation where they become +disconnected from the ZooKeeper server and must wait to be reconnected +before continuing ZooKeeper operations. + +The dedicated threads running inside the Accumulo Master process for FATE +actions had the potential unexpectedly exit in this disconnected state. +This caused a scenario where all future FATE-based operations would +be blocked until the Accumulo Master process was restarted. ([ACCUMULO-4060][ACCUMULO-4060]) + +### Incorrect management of certain Apache Thrift RPCs + +Accumulo relies on Apache Thrift to implement remote procedure calls between +Accumulo services. Accumulo's use of Thrift uncovered an unfortunate situation +where a special RPC (a "oneway" call) would leave unwanted data on the underlying +Thrift connection. After this extra data was left on connection, all subsequent RPCs +re-using that connection would fail with "out of sequence response" error messages. +Accumulo would be left in a bad state until the mishandled connections were released +or Accumulo services were restarted. ([ACCUMULO-4065][ACCUMULO-4065]) + +## Other Notable Changes + + * [ACCUMULO-3509][ACCUMULO-3509] Fixed some lock contention in TabletServer, preventing resource cleanup + * [ACCUMULO-3734][ACCUMULO-3734] Fixed quote-escaping bug in VisibilityConstraint + * [ACCUMULO-4025][ACCUMULO-4025] Fixed cleanup of bulk load fate transactions + * [ACCUMULO-4098][ACCUMULO-4098],[ACCUMULO-4113][ACCUMULO-4113] Fixed widespread misuse of ByteBuffer + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk +and Continuous Ingest tests run on any number of nodes. *Agitation* refers to +randomly restarting Accumulo processes and Hadoop Datanode processes, and, in +HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS/Environment | Hadoop | Nodes | ZooKeeper | HDFS HA | Tests | +|---------------------------------------------------------------------------|--------|-------|-----------|---------|--------------------------------------------------------------------------------------------------------------------------------------| +| CentOS 7.1 w/Oracle JDK8 on EC2 (1 m3.xlarge, 8 d2.xlarge) | 2.6.3 | 9 | 3.4.6 | No | Random walk (All.xml) 24-hour run, saw [ACCUMULO-3794][ACCUMULO-3794] and [ACCUMULO-4151][ACCUMULO-4151]. | +| CentOS 7.1 w/Oracle JDK8 on EC2 (1 m3.xlarge, 8 d2.xlarge) | 2.6.3 | 9 | 3.4.6 | No | 21 hr run of CI w/ agitation, 23.1B entries verified. | +| CentOS 7.1 w/Oracle JDK8 on EC2 (1 m3.xlarge, 8 d2.xlarge) | 2.6.3 | 9 | 3.4.6 | No | 24 hr run of CI w/o agitation, 23.0B entries verified; saw performance issues outlined in comment on [ACCUMULO-4146][ACCUMULO-4146]. | +| CentOS 6.7 (OpenJDK 7), Fedora 23 (OpenJDK 8), and CentOS 7.2 (OpenJDK 7) | 2.6.1 | 1 | 3.4.6 | No | All unit tests and ITs pass with -Dhadoop.version=2.6.1; Kerberos ITs had a problem with earlier versions of Hadoop | + +[JIRA_171]: https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12329940 + +[ACCUMULO-3509]: https://issues.apache.org/jira/browse/ACCUMULO-3509 +[ACCUMULO-3734]: https://issues.apache.org/jira/browse/ACCUMULO-3734 +[ACCUMULO-3794]: https://issues.apache.org/jira/browse/ACCUMULO-3794 +[ACCUMULO-3859]: https://issues.apache.org/jira/browse/ACCUMULO-3859 +[ACCUMULO-3967]: https://issues.apache.org/jira/browse/ACCUMULO-3967 +[ACCUMULO-4016]: https://issues.apache.org/jira/browse/ACCUMULO-4016 +[ACCUMULO-4021]: https://issues.apache.org/jira/browse/ACCUMULO-4021 +[ACCUMULO-4025]: https://issues.apache.org/jira/browse/ACCUMULO-4025 +[ACCUMULO-4026]: https://issues.apache.org/jira/browse/ACCUMULO-4026 +[ACCUMULO-4027]: https://issues.apache.org/jira/browse/ACCUMULO-4027 +[ACCUMULO-4056]: https://issues.apache.org/jira/browse/ACCUMULO-4056 +[ACCUMULO-4060]: https://issues.apache.org/jira/browse/ACCUMULO-4060 +[ACCUMULO-4065]: https://issues.apache.org/jira/browse/ACCUMULO-4065 +[ACCUMULO-4066]: https://issues.apache.org/jira/browse/ACCUMULO-4066 +[ACCUMULO-4069]: https://issues.apache.org/jira/browse/ACCUMULO-4069 +[ACCUMULO-4080]: https://issues.apache.org/jira/browse/ACCUMULO-4080 +[ACCUMULO-4098]: https://issues.apache.org/jira/browse/ACCUMULO-4098 +[ACCUMULO-4113]: https://issues.apache.org/jira/browse/ACCUMULO-4113 +[ACCUMULO-4146]: https://issues.apache.org/jira/browse/ACCUMULO-4146 +[ACCUMULO-4151]: https://issues.apache.org/jira/browse/ACCUMULO-4151 + http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2016-06-22-accumulo-1.7.2.md ---------------------------------------------------------------------- diff --git a/_posts/release/2016-06-22-accumulo-1.7.2.md b/_posts/release/2016-06-22-accumulo-1.7.2.md new file mode 100644 index 0000000..8d156ff --- /dev/null +++ b/_posts/release/2016-06-22-accumulo-1.7.2.md @@ -0,0 +1,94 @@ +--- +title: Apache Accumulo 1.7.2 +redirect_from: /release_notes/1.7.2.html +--- + +Apache Accumulo 1.7.2 is a maintenance release on the 1.7 version branch. This +release contains changes from more than 150 issues, comprised of bug-fixes, +performance improvements, build quality improvements, and more. See +[JIRA][JIRA_172] for a complete list. + +Below are resources for this release: + +* [User Manual](/1.7/accumulo_user_manual.html) +* [Javadocs](/1.7/apidocs) +* [Examples](/1.7/examples) + +Users of any previous 1.7.x release are strongly encouraged to update as soon +as possible to benefit from the improvements with very little concern in change +of underlying functionality. Users of 1.6 or earlier that are seeking to +upgrade to 1.7 should consider 1.7.2 as a starting point. + +## Highlights + +### Write-Ahead Logs can be prematurely deleted + +There were cases where the Accumulo Garbage Collector may inadvertently delete a WAL for a tablet server that it has erroneously determined to be down, causing data loss. This has been corrected. See [ACCUMULO-4157][ACCUMULO-4157] for additional detail. + +### Upgrade to Commons-VFS 2.1 + +Upgrading to Apache Commons VFS 2.1 fixes several issues with classloading out of HDFS. For further detail see [ACCUMULO-4146][ACCUMULO-4146]. Additional fixes to a potential HDFS class loading deadlock situation were made in [ACCUMULO-4341][ACCUMULO-4341]. + +### Native Map failed to increment mutation count properly + +There was a bug ([ACCUMULO-4148][ACCUMULO-4148]) where multiple put calls with identical keys and no timestamp would exhibit different behaviour depending on whether native maps were enabled or not. This behaviour would result in hidden mutations with native maps, and has been corrected. + +### Open WAL files could prevent DataNode decomission + +An improvement was introduced to allow a max age before WAL files would be automatically rolled. Without a max age, they could stay open for writing indefinitely, blocking the Hadoop DataNode decomissioning process. For more information, see [ACCUMULO-4004][ACCUMULO-4004]. + +### Remove unnecessary copy of cached RFile index blocks + +Accumulo maintains an cache for file blocks in-memory as a performance optimization. This can be done safely because Accumulo RFiles are immutable, thus their blocks are also immutable. There are two types of these blocks: index and data blocks. Index blocks refer to the b-tree style index inside of each Accumulo RFile, while data blocks contain the sorted Key-Value pairs. In previous versions, when Accumulo extracted an Index block from the in-memory cache, it would copy the data. [ACCUMULO-4164][ACCUMULO-4164] removes this unnecessary copy as the contents are immutable and can be passed by reference. Ensuring that the Index blocks are not copied when accessed from the cache is a big performance gain at the file-access level. + +### Analyze Key-length to avoid choosing large Keys for RFile Index blocks + +Accumulo's RFile index blocks are made up of a Key which exists in the file and points to that specific location in the corresponding RFile data block. Thus, the size of the RFile index blocks is largely dominated by the size of the Keys which are used by the index. [ACCUMULO-4314][ACCUMULO-4314] is an improvement which uses statistics on the length of the Keys in the Rfile to avoid choosing Keys for the index whose length is greater than three standard deviations for the RFile. By choosing smaller Keys for the index, Accumulo can access the RFile index faster and keep more Index blocks cached in memory. Initial tests showed that with this change, the RFile index size was nearly cut in half. + +### Minor performance improvements. + +Tablet servers would previously always hsync at the start of a minor compaction, causing delays in the write pipeline. These additional syncs were determined to provide no additional durability guarantees and have been removed. See [ACCUMULO-4112][ACCUMULO-4112] for additional detail. + +A performance issue was identified and corrected ([ACCUMULO-1755][ACCUMULO-1755]) where the BatchWriter would block calls to addMutation while looking up destination tablet server metadata. The writer has been fixed to allow both operations in parallel. + + +## Other Notable Changes + + * [ACCUMULO-3923][ACCUMULO-3923] bootstrap_hdfs.sh script would copy incorrect jars to hdfs. + * [ACCUMULO-4146][ACCUMULO-4146] Avoid copy of RFile Index Blocks when already in cache. + * [ACCUMULO-4155][ACCUMULO-4155] No longer publish javadoc for non-public API to website. (Still available in javadoc jars in maven) + * [ACCUMULO-4173][ACCUMULO-4173] Provide balancer to balance table within subset of hosts. + * [ACCUMULO-4334][ACCUMULO-4334] Ingest rates reported through JMX did not match rates reported by Monitor. + * [ACCUMULO-4335][ACCUMULO-4335] Error conditions that result in a Halt should ensure non-zero process exit code. + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk +and Continuous Ingest tests run on any number of nodes. *Agitation* refers to +randomly restarting Accumulo processes and Hadoop Datanode processes, and, in +HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS/Environment | Hadoop | Nodes | ZooKeeper | HDFS HA | Tests | +|--------------------------------------------|--------|-------|-----------|---------|------------------------------------------------------| +| CentOS 7; EC2 m3.xlarge, d2.xlarge workers | 2.6.3 | 9 | 3.4.8 | No | 24 HR Continuous Ingest with and without Agitation. | +| CentOS 6: EC2 m3.2xlarge | 2.6.1 | 1 | 3.4.5 | No | Unit tests and Integration Tests | + +[JIRA_172]: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12312121&version=12333776 + +[ACCUMULO-4157]: https://issues.apache.org/jira/browse/ACCUMULO-4157 +[ACCUMULO-4146]: https://issues.apache.org/jira/browse/ACCUMULO-4146 +[ACCUMULO-4341]: https://issues.apache.org/jira/browse/ACCUMULO-4341 +[ACCUMULO-4148]: https://issues.apache.org/jira/browse/ACCUMULO-4148 +[ACCUMULO-4004]: https://issues.apache.org/jira/browse/ACCUMULO-4004 +[ACCUMULO-4112]: https://issues.apache.org/jira/browse/ACCUMULO-4112 +[ACCUMULO-1755]: https://issues.apache.org/jira/browse/ACCUMULO-1755 +[ACCUMULO-4146]: https://issues.apache.org/jira/browse/ACCUMULO-4146 +[ACCUMULO-4335]: https://issues.apache.org/jira/browse/ACCUMULO-4335 +[ACCUMULO-4334]: https://issues.apache.org/jira/browse/ACCUMULO-4334 +[ACCUMULO-4314]: https://issues.apache.org/jira/browse/ACCUMULO-4314 +[ACCUMULO-3923]: https://issues.apache.org/jira/browse/ACCUMULO-3923 +[ACCUMULO-4155]: https://issues.apache.org/jira/browse/ACCUMULO-4155 +[ACCUMULO-4173]: https://issues.apache.org/jira/browse/ACCUMULO-4173 +[ACCUMULO-4151]: https://issues.apache.org/jira/browse/ACCUMULO-4151 +[ACCUMULO-4164]: https://issues.apache.org/jira/browse/ACCUMULO-4164 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2016-09-06-accumulo-1.8.0.md ---------------------------------------------------------------------- diff --git a/_posts/release/2016-09-06-accumulo-1.8.0.md b/_posts/release/2016-09-06-accumulo-1.8.0.md new file mode 100644 index 0000000..cade1c4 --- /dev/null +++ b/_posts/release/2016-09-06-accumulo-1.8.0.md @@ -0,0 +1,196 @@ +--- +title: Apache Accumulo 1.8.0 +redirect_from: /release_notes/1.8.0.html +--- + +Apache Accumulo 1.8.0 is a significant release that includes many important +milestone features which expand the functionality of Accumulo. These include +features related to security, availability, and extensibility. Over +350 JIRA issues were resolved in this version. This includes over +200 bug fixes and 71 improvements and 4 new features. See [JIRA][JIRA_180] +for the complete list. + +Below are resources for this release: + +* [User Manual](/1.8/accumulo_user_manual.html) +* [Javadocs](/1.8/apidocs) +* [Examples](/1.8/examples) + +In the context of Accumulo's [Semantic Versioning][semver] [guidelines][api], +this is a "minor version". This means that new APIs have been created, some +deprecations may have been added, but no deprecated APIs have been removed. +Code written against 1.7.x should work against 1.8.0 -- binary compatibility +has been preserved with one exception of an already-deprecated Mock Accumulo +utility class. As always, the Accumulo developers take API compatibility +very seriously and have invested much time to ensure that we meet the promises set forth to our users. + +## Major Changes + +### Speed up WAL roll overs + +Performance of writing mutations is improved by refactoring the +bookeeping required for Write-Ahead Log (WAL) files and by creating a +standby WAL for faster switching when the log is full. This was a +substantial refactor in the way WALs worked, but smoothes overall +ingest performance in addition to provides a increase in write speed +as shown by the simple test below. The top entry is before +[ACCUMULO-3423][ACCUMULO-3423] and the bottom graph is after the +refactor. + +![Graph of WAL speed up after ACCUMULO-3423][IMG-3423] + +### User level API for RFile + +Previously the only public API available to write RFiles was via the AccumuloFileOutputFormat. There was no way to read RFiles in the public +API. [ACCUMULO-4165][ACCUMULO-4165] exposes a brand new public [API][RFILE_API] for reading and writing RFiles as well as cleans up some of the internal APIs. + +### Suspend Tablet assignment for rolling restarts + +When a tablet server dies, Accumulo attempted to reassign the tablets as quickly as possible to maintain availability. +A new configuration property `table.suspend.duration` (with a default of zero seconds) now controls how long to wait before reassigning +a tablet from a dead tserver. The property is configurable via the +Accumulo shell, so you can set it, do a rolling restart, and then +set it back to 0. A new state as introduced, TableState.SUSPENDED to support this feature. By default, metadata tablet +reassignment is not suspended, but that can also be changed with the `master.metadata.suspendable` property that is false by +default. Root tablet assignment can not be suspended. See [ACCUMULO-4353] for more info. + +### Run multiple Tablet Servers on one node + +[ACCUMULO-4328] introduces the capability of running multiple tservers on a single node. This is intended for nodes with a large +amounts of memory and/or disk. This feature is disabled by default. There are several related tickets: [ACCUMULO-4072], [ACCUMULO-4331] +and [ACCUMULO-4406]. Note that when this is enabled, the names of the log files change. Previous log file names were defined in the +generic_logger.xml as `${org.apache.accumulo.core.application}_{org.apache.accumulo.core.ip.localhost.hostname}.log`. +The files will now include the instance id after the application with +`${org.apache.accumulo.core.application}_${instance}_${org.apache.accumulo.core.ip.localhost.hostname}.log`. + +For example: tserver_host.domain.com.log will become tserver_1_host.domain.log when multiple TabletServers +are run per host. The same change also applies to the debug logs provided in the example configurations. The log +names do not change if this feature is not used. + +### Rate limiting Major Compactions + +Major Compactions can significantly increase the amount of load on +TabletServers. [ACCUMULO-4187] restricts the rate at which data is +read and written when performing major compactions. This has a direct +effect on the IO load caused by major compactions with a similar +effect on the CPU utilization. This behavior is controlled by a new +property `tserver.compaction.major.throughput` with a defaults of 0B +which disables the rate limiting. + +### Table Sampling + +Queryable sample data was added by [ACCUMULO-3913]. This allows users to configure a pluggable +function to generate sample data. At scan time, the sample data can optionally be scanned. +Iterators also have access to sample data. Iterators can access all data and sample data, this +allows an iterator to use sample data for query optimizations. The new user level RFile API +supports writing RFiles with sample data for bulk import. + +A simple configurable sampler function is included with Accumulo. This sampler uses hashing and +can be configured to use a subset of Key fields. For example if it was desired to have entire rows +in the sample, then this sampler would be configured to hash+mod the row. Then when a row is +selected for the sample, all of its columns and all of its updates will be in the sample data. +Another scenario is one in which a document id is in the column qualifier. In this scenario, one +would either want all data related to a document in the sample data or none. To achieve this, the +sample could be configured to hash+mod on the column qualifier. See the sample [Readme +example][sample] and javadocs on the new APIs for more information. + +For sampling to work, all tablets scanned must have pre-generated sample data that was generated in +the same way. If this is not the case then scans will fail. For existing tables, samples can be +generated by configuring sampling on the table and compacting the table. + +### Upgrade to Apache Thrift 0.9.3 + +Accumulo relies on Apache Thrift to implement remote procedure calls +between Accumulo services. Ticket [ACCUMULO-4077][ACCUMULO-4077] +updates our dependency to 0.9.3. See the +[Apache Thrift 0.9.3 Release Notes][THRIFT-0.9.3-RN] for details on +the changes to Thrift. **NOTE:** The Thrift 0.9.3 Java library is not +compatible other versions of Thrift. Applications running against Accumulo +1.8 must use Thrift 0.9.3. Different versions of Thrift on the classpath +will not work. + +### Iterator Test Harness + +Users often write a new iterator without fully understanding its limits and lifetime. Previously, Accumulo did +not provide any means in which a user could test iterators to catch common issues that only become apparent +in multi-node production deployments. [ACCUMULO-626] provides a framework and a collection of initial tests +which can be used to simulate common issues with Iterators that only appear in production deployments. This test +harness can be used directly by users as a supplemental tool to unit tests and integration tests with MiniAccumuloCluster. + +Please see the [Accumulo User Manual chapter on Iterator Testing][ITER_TEST] for more information + +### Default port for Monitor changed to 9995 + +Previously, the default port for the monitor was 50095. You will need to update your links to point to port 9995. The default +port for the GC process was also changed from 50091 to 9998, although this an RPC port used internally and automatically discovered. +These default ports were changed because the previous defaults fell in the Linux Ephemeral port range. This means that the operating +system, when a port in this range was unusued, would allocate this port for dynamic network communication. This has the side-effect of +temporal bind issues when trying to start these services (as the operating +system might have allocated them elsewhere). By moving these +defaults out of the ephemeral range, we can guarantee that the Monitor and GC +will reliably start. These values are still configurable by setting +`monitor.port.client`and `gc.port.client` in the accumulo-site.xml. + + +## Other Notable Changes + + * [ACCUMULO-1055] Configurable maximum file size for merging minor compactions + * [ACCUMULO-1124] Optimization of RFile index + * [ACCUMULO-2883] API to fetch current tablet assignments + * [ACCUMULO-3871] Support for running integration tests in MapReduce + * [ACCUMULO-3920] Deprecate the MockAccumulo class and remove usage in our tests + * [ACCUMULO-4339] Make hadoop-minicluster optional dependency of acccumulo-minicluster + * [ACCUMULO-4318] BatchWriter, ConditionalWriter, and ScannerBase now extend AutoCloseable + * [ACCUMULO-4326] Value constructor now accepts Strings (and Charsequences) + * [ACCUMULO-4354] Bump dependency versions to include gson, jetty, and sl4j + * [ACCUMULO-3735] Bulk Import status page on the monitor + * [ACCUMULO-4066] Reduced time to processes conditional mutations. + * [ACCUMULO-4164] Reduced seek time for cached data. + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk +and Continuous Ingest tests run on any number of nodes. *Agitation* refers to +randomly restarting Accumulo processes and Hadoop Datanode processes, and, in +HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS/Environment | Hadoop | Nodes | ZooKeeper | HDFS HA | Tests | +|----------------------------------------------------------------------------|----------------------|-------|------------------|---------|----------------------------------------------| +| CentOS7/openJDK7/EC2; 3 m3.xlarge leaders, 8 d2.xlarge workers | 2.6.4 | 11 | 3.4.8 | No | 24 HR Continuous Ingest without Agitation. | +| CentOS7/openJDK7/EC2; 3 m3.xlarge leaders, 8 d2.xlarge workers | 2.6.4 | 11 | 3.4.8 | No | 16 HR Continuous Ingest with Agitation. | +| CentOS7/openJDK7/OpenStack VMs (16G RAM 2cores 2disk3; 1 leader, 5 workers | HDP 2.5 (Hadoop 2.7) | 7 | HDP 2.5 (ZK 3.4) | No | 24 HR Continuous Ingest without Agitation. | +| CentOS7/openJDK7/OpenStack VMs (16G RAM 2cores 2disk3; 1 leader, 5 workers | HDP 2.5 (Hadoop 2.7) | 7 | HDP 2.5 (ZK 3.4) | No | 24 HR Continuous Ingest with Agitation. | + +[ACCUMULO-1055]: https://issues.apache.org/jira/browse/ACCUMULO-1055 +[ACCUMULO-1124]: https://issues.apache.org/jira/browse/ACCUMULO-1124 +[ACCUMULO-2883]: https://issues.apache.org/jira/browse/ACCUMULO-2883 +[ACCUMULO-3409]: https://issues.apache.org/jira/browse/ACCUMULO-3409 +[ACCUMULO-3423]: https://issues.apache.org/jira/browse/ACCUMULO-3423 +[ACCUMULO-3735]: https://issues.apache.org/jira/browse/ACCUMULO-3735 +[ACCUMULO-3871]: https://issues.apache.org/jira/browse/ACCUMULO-3871 +[ACCUMULO-3913]: https://issues.apache.org/jira/browse/ACCUMULO-3913 +[ACCUMULO-3920]: https://issues.apache.org/jira/browse/ACCUMULO-3920 +[ACCUMULO-4072]: https://issues.apache.org/jira/browse/ACCUMULO-4072 +[ACCUMULO-4077]: https://issues.apache.org/jira/browse/ACCUMULO-4077 +[ACCUMULO-4066]: https://issues.apache.org/jira/browse/ACCUMULO-4066 +[ACCUMULO-4164]: https://issues.apache.org/jira/browse/ACCUMULO-4164 +[ACCUMULO-4165]: https://issues.apache.org/jira/browse/ACCUMULO-4165 +[ACCUMULO-4187]: https://issues.apache.org/jira/browse/ACCUMULO-4187 +[ACCUMULO-4318]: https://issues.apache.org/jira/browse/ACCUMULO-4318 +[ACCUMULO-4326]: https://issues.apache.org/jira/browse/ACCUMULO-4326 +[ACCUMULO-4328]: https://issues.apache.org/jira/browse/ACCUMULO-4328 +[ACCUMULO-4331]: https://issues.apache.org/jira/browse/ACCUMULO-4331 +[ACCUMULO-4339]: https://issues.apache.org/jira/browse/ACCUMULO-4339 +[ACCUMULO-4353]: https://issues.apache.org/jira/browse/ACCUMULO-4353 +[ACCUMULO-4354]: https://issues.apache.org/jira/browse/ACCUMULO-4354 +[ACCUMULO-4406]: https://issues.apache.org/jira/browse/ACCUMULO-4406 +[ACCUMULO-626]: https://issues.apache.org/jira/browse/ACCUMULO-626 +[IMG-3423]: https://issues.apache.org/jira/secure/attachment/12705402/WAL-slowdown-graphs.jpg "Graph of WAL speed up after ACCUMULO-3423" +[JIRA_180]: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12312121&version=12329879 +[THRIFT-0.9.3-RN]: https://github.com/apache/thrift/blob/0.9.3/CHANGES +[api]: https://github.com/apache/accumulo/blob/1.8/README.md#api +[semver]: http://semver.org +[sample]: ../1.8/examples/sample +[ITER_TEST]: ../1.8/accumulo_user_manual.html#_iterator_testing +[RFILE_API]: ../1.8/apidocs/org/apache/accumulo/core/client/rfile/RFile.html http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/_posts/release/2016-09-18-accumulo-1.6.6.md ---------------------------------------------------------------------- diff --git a/_posts/release/2016-09-18-accumulo-1.6.6.md b/_posts/release/2016-09-18-accumulo-1.6.6.md new file mode 100644 index 0000000..bba330b --- /dev/null +++ b/_posts/release/2016-09-18-accumulo-1.6.6.md @@ -0,0 +1,136 @@ +--- +title: Apache Accumulo 1.6.6 +redirect_from: /release_notes/1.6.6.html +--- + +Apache Accumulo 1.6.6 is a maintenance release on the 1.6 version branch. This +release contains changes from more than 40 issues, comprised of bug-fixes, +performance improvements, build quality improvements, and more. See +[JIRA][JIRA_166] for a complete list. + +Below are resources for this release: + +* [User Manual](/1.6/accumulo_user_manual.html) +* [Javadocs](/1.6/apidocs) +* [Examples](/1.6/examples) + +Users of any previous 1.6.x release are strongly encouraged to update as soon +as possible to benefit from the improvements with very little concern in change +of underlying functionality. + +As of this release, active development has ceased for the 1.6 release line, so +users should consider upgrading to a newer, actively maintained version when +they can. While the developers may release another 1.6 version to address a +severe issue, there's a strong possibility that this will be the last 1.6 +release. That would also mean that this will be the last Accumulo version to +support Java 6 and Hadoop 1. + +## Highlights + +### Write-Ahead Logs can be prematurely deleted + +There were cases where the Accumulo Garbage Collector may inadvertently delete +a WAL for a tablet server that it has erroneously determined to be down, +causing data loss. This has been corrected. See [ACCUMULO-4157][ACCUMULO-4157] +for additional detail. + +### Upgrade to Commons-VFS 2.1 + +Upgrading to Apache Commons VFS 2.1 fixes several issues with classloading out +of HDFS. For further detail see [ACCUMULO-4146][ACCUMULO-4146]. Additional +fixes to a potential HDFS class loading deadlock situation were made in +[ACCUMULO-4341][ACCUMULO-4341]. + +### Native Map failed to increment mutation count properly + +There was a bug ([ACCUMULO-4148][ACCUMULO-4148]) where multiple put calls with +identical keys and no timestamp would exhibit different behaviour depending on +whether native maps were enabled or not. This behaviour would result in hidden +mutations with native maps, and has been corrected. + +### Open WAL files could prevent DataNode decomission + +An improvement was introduced to allow a max age before WAL files would be +automatically rolled. Without a max age, they could stay open for writing +indefinitely, blocking the Hadoop DataNode decomissioning process. For more +information, see [ACCUMULO-4004][ACCUMULO-4004]. + +### Remove unnecessary copy of cached RFile index blocks + +Accumulo maintains an cache for file blocks in-memory as a performance +optimization. This can be done safely because Accumulo RFiles are immutable, +thus their blocks are also immutable. There are two types of these blocks: +index and data blocks. Index blocks refer to the b-tree style index inside of +each Accumulo RFile, while data blocks contain the sorted Key-Value pairs. In +previous versions, when Accumulo extracted an Index block from the in-memory +cache, it would copy the data. [ACCUMULO-4164][ACCUMULO-4164] removes this +unnecessary copy as the contents are immutable and can be passed by reference. +Ensuring that the Index blocks are not copied when accessed from the cache is a +big performance gain at the file-access level. + +### Analyze Key-length to avoid choosing large Keys for RFile Index blocks + +Accumulo's RFile index blocks are made up of a Key which exists in the file and +points to that specific location in the corresponding RFile data block. Thus, +the size of the RFile index blocks is largely dominated by the size of the Keys +which are used by the index. [ACCUMULO-4314][ACCUMULO-4314] is an improvement +which uses statistics on the length of the Keys in the Rfile to avoid choosing +Keys for the index whose length is greater than three standard deviations for +the RFile. By choosing smaller Keys for the index, Accumulo can access the +RFile index faster and keep more Index blocks cached in memory. Initial tests +showed that with this change, the RFile index size was nearly cut in half. + +### Gson version bump + +Due to an [upstream bug with Gson 2.2.2][GSONBUG], we've bumped our bundled +dependency ([ACCUMULO-4345][ACCUMULO-4345]) to version 2.2.4. Please take note +of this when you upgrade, if you were using the version shipped with Accumulo, +and were relying on the buggy behavior in the previous version in your own +code. + +### Minor performance improvements. + +A performance issue was identified and corrected +([ACCUMULO-1755][ACCUMULO-1755]) where the BatchWriter would block calls to +addMutation while looking up destination tablet server metadata. The writer has +been fixed to allow both operations in parallel. + + +## Other Notable Changes + + * [ACCUMULO-4155][ACCUMULO-4155] No longer publish javadoc for non-public API + to website. (Still available in javadoc jars in maven) + * [ACCUMULO-4334][ACCUMULO-4334] Ingest rates reported through JMX did not + match rates reported by Monitor. + * [ACCUMULO-4335][ACCUMULO-4335] Error conditions that result in a Halt should + ensure non-zero process exit code. + +## Testing + +Each unit and functional test only runs on a single node, while the RandomWalk +and Continuous Ingest tests run on any number of nodes. *Agitation* refers to +randomly restarting Accumulo processes and Hadoop Datanode processes, and, in +HDFS High-Availability instances, forcing NameNode failover. + +{: #release_notes_testing .table } +| OS/Environment | Hadoop | Nodes | ZooKeeper | HDFS HA | Tests | +|----------------|--------|-------|-----------|---------|----------------------------------| +| CentOS 7 | 1.2.1 | 1 | 3.3.6 | No | Unit tests and Integration Tests | +| CentOS 7 | 2.2.0 | 1 | 3.3.6 | No | Unit tests and Integration Tests | + +[JIRA_166]: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12312121&version=12334846 + +[GSONBUG]: https://github.com/google/gson/issues/362 + +[ACCUMULO-1755]: https://issues.apache.org/jira/browse/ACCUMULO-1755 +[ACCUMULO-4004]: https://issues.apache.org/jira/browse/ACCUMULO-4004 +[ACCUMULO-4146]: https://issues.apache.org/jira/browse/ACCUMULO-4146 +[ACCUMULO-4148]: https://issues.apache.org/jira/browse/ACCUMULO-4148 +[ACCUMULO-4155]: https://issues.apache.org/jira/browse/ACCUMULO-4155 +[ACCUMULO-4157]: https://issues.apache.org/jira/browse/ACCUMULO-4157 +[ACCUMULO-4164]: https://issues.apache.org/jira/browse/ACCUMULO-4164 +[ACCUMULO-4314]: https://issues.apache.org/jira/browse/ACCUMULO-4314 +[ACCUMULO-4334]: https://issues.apache.org/jira/browse/ACCUMULO-4334 +[ACCUMULO-4335]: https://issues.apache.org/jira/browse/ACCUMULO-4335 +[ACCUMULO-4341]: https://issues.apache.org/jira/browse/ACCUMULO-4341 +[ACCUMULO-4345]: https://issues.apache.org/jira/browse/ACCUMULO-4345 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/downloads/index.md ---------------------------------------------------------------------- diff --git a/downloads/index.md b/downloads/index.md index d99f2e1..cf2c9c3 100644 --- a/downloads/index.md +++ b/downloads/index.md @@ -193,9 +193,9 @@ Older releases can be found in the [archives][ARCHIVES]. [CHANGES_17]: https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12333776 "1.7.2 CHANGES" [CHANGES_18]: https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12329879 "1.8.0 CHANGES" -[REL_NOTES_16]: {{ site.baseurl }}/release_notes/1.6.6 "1.6.6 Release Notes" -[REL_NOTES_17]: {{ site.baseurl }}/release_notes/1.7.2 "1.7.2 Release Notes" -[REL_NOTES_18]: {{ site.baseurl }}/release_notes/1.8.0 "1.8.0 Release Notes" +[REL_NOTES_16]: {{ site.baseurl }}/release/accumulo-1.6.6/ "1.6.6 Release Notes" +[REL_NOTES_17]: {{ site.baseurl }}/release/accumulo-1.7.2/ "1.7.2 Release Notes" +[REL_NOTES_18]: {{ site.baseurl }}/release/accumulo-1.8.0/ "1.8.0 Release Notes" [MD5SUM_16]: https://www.apache.org/dist/accumulo/1.6.6/MD5SUM "1.6.6 MD5 file hashes" [MD5SUM_17]: https://www.apache.org/dist/accumulo/1.7.2/MD5SUM "1.7.2 MD5 file hashes" http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/index.md ---------------------------------------------------------------------- diff --git a/index.md b/index.md index 4465abf..5e7c5f3 100644 --- a/index.md +++ b/index.md @@ -19,9 +19,9 @@ legal_notice: Apache Accumulo, Accumulo, Apache Hadoop, Apache Thrift, Apache, t <p style="font-size: 24px; margin-bottom: 0px;">Latest News</p> {% for post in site.posts limit:site.num_home_posts %} <div class="row latest-news-item"> - <div class="col-sm-12"> + <div class="col-sm-12" style="margin-bottom: 5px"> + <span style="font-size: 12px; margin-right: 5px;">{{ post.date | date: "%b %Y" }}</span> <a href="{{ site.baseurl }}{{ post.url }}">{{ post.title }}</a> - <span style="font-size: 12px">{{ post.date | date: "%b %Y" }}</span> </div> </div> {% endfor %} http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/news.md ---------------------------------------------------------------------- diff --git a/news.md b/news.md index 26edf6d..cc6aa35 100644 --- a/news.md +++ b/news.md @@ -3,13 +3,17 @@ title: News Archive redirect_from: /blog/ --- -<table> +<div> {% for post in site.posts %} {% assign post_year = post.date | date: "%Y" %} {% assign newer_post_year = post.next.date | date: "%Y" %} {% if post_year != newer_post_year %} - <tr><th colspan="2">{{ post_year }}</th></tr> + <hr> + <h3>{{ post_year }}</h3> {% endif %} - <tr><td style="padding: 10px"><small>{{ post.date | date: "%b %d" }}</small></td><td><a href="{{ site.baseurl }}{{ post.url }}" class="post-title-archive">{{ post.title }}</a></td></tr> + <div class="row" style="margin-top: 15px"> + <div class="col-md-1">{{ post.date | date: "%b %d" }}</div> + <div class="col-md-10"><a href="{{ site.baseurl }}{{ post.url }}">{{ post.title }}</a></div> + </div> {% endfor %} -</table> +</div> http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/notable_features.md ---------------------------------------------------------------------- diff --git a/notable_features.md b/notable_features.md index e52047b..7be8da7 100644 --- a/notable_features.md +++ b/notable_features.md @@ -432,7 +432,7 @@ that fall completely within a range are simply dropped. Tablets overlapping the beginning and end of the range are split, compacted, and then merged. [FATE]: #fate -[M-A-P]: {{ site.baseurl }}/release_notes/1.6.0#maven-plugin +[M-A-P]: {{ site.baseurl }}/release/accumulo-1.6.0/#maven-plugin [CMUT]: {{ site.baseurl }}/1.6/apidocs/org/apache/accumulo/core/data/ConditionalMutation [CWRI]: {{ site.baseurl }}/1.6/apidocs/org/apache/accumulo/core/client/ConditionalWriter [4]: {{ site.baseurl }}/1.5/accumulo_user_manual#_writing_accumulo_clients http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/old_documentation.md ---------------------------------------------------------------------- diff --git a/old_documentation.md b/old_documentation.md deleted file mode 100644 index 2b0f1f1..0000000 --- a/old_documentation.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: Old Documentation -nav: old_documentation ---- - -This page contains pointers to the documentation for major versions of Accumulo that are no longer under active development. - -#### 1.5 Documentation - -* [README][README_15] -* [PDF manual][MANUAL_PDF_15] -* [html manual][MANUAL_HTML_15] -* [examples][EXAMPLES_15] -* [Javadoc][JAVADOC_15] - -#### 1.4 Documentation - -* [README][README_14] -* [PDF manual][MANUAL_PDF_14] -* [html manual][MANUAL_HTML_14] -* [examples][EXAMPLES_14] -* [Javadoc][JAVADOC_14] - -#### 1.3 Documentation -* [README][README_13] -* [html manual][MANUAL_HTML_13] -* [examples][EXAMPLES_13] - -[README_15]: https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=blob_plain;f=README;hb=1.5.4 -{: onClick="javascript: _gaq.push(['_trackPageview', '/1.5/README']);" } -[MANUAL_PDF_15]: {{ site.baseurl }}/1.5/accumulo_user_manual.pdf -{: onClick="javascript: _gaq.push(['_trackPageview', '/1.5/accumulo_user_manual.pdf']);" } -[MANUAL_HTML_15]: {{ site.baseurl }}/1.5/accumulo_user_manual "1.5 user manual" -[EXAMPLES_15]: {{ site.baseurl }}/1.5/examples "1.5 examples" -[JAVADOC_15]: {{ site.baseurl }}/1.5/apidocs/ -{: onClick="javascript: _gaq.push(['_trackPageview', '/1.5/apidocs/']);" } - -[README_14]: https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=blob_plain;f=README;hb=f7d87b6e407de6597b6c0ca60ca1b6a321faf237 -{: onClick="javascript: _gaq.push(['_trackPageview', '/1.4/README']);" } -[MANUAL_PDF_14]: {{ site.baseurl }}/1.4/accumulo_user_manual.pdf -{: onClick="javascript: _gaq.push(['_trackPageview', '/1.4/accumulo_user_manual.pdf']);" } -[MANUAL_HTML_14]: {{ site.baseurl }}/1.4/user_manual "1.4 user manual" -[EXAMPLES_14]: {{ site.baseurl }}/1.4/examples "1.4 examples" -[JAVADOC_14]: {{ site.baseurl }}/1.4/apidocs/ -{: onClick="javascript: _gaq.push(['_trackPageview', '/1.4/apidocs/']);" } - -[README_13]: https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=blob_plain;f=README;h=86713d9b6add9038d5130b4a23ba4a79b72d0f15;hb=3b4ffc158945c1f834fc6f257f21484c61691d0f -{: onClick="javascript: _gaq.push(['_trackPageview', '/1.3/README']);" } -[MANUAL_HTML_13]: {{ site.baseurl }}/user_manual_1.3-incubating/ "1.3 user manual" -[EXAMPLES_13]: {{ site.baseurl }}/user_manual_1.3-incubating/examples/ "1.3 examples" http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/pages/examples.md ---------------------------------------------------------------------- diff --git a/pages/examples.md b/pages/examples.md new file mode 100644 index 0000000..474f00a --- /dev/null +++ b/pages/examples.md @@ -0,0 +1,10 @@ +--- +title: Accumulo Examples Archive +permalink: /examples/ +--- + +* [Apache Accumulo 1.8 Examples](/1.8/examples) +* [Apache Accumulo 1.7 Examples](/1.7/examples) +* [Apache Accumulo 1.6 Examples](/1.6/examples) +* [Apache Accumulo 1.5 Examples](/1.5/examples) +* [Apache Accumulo 1.4 Examples](/1.4/examples) http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/pages/javadocs.md ---------------------------------------------------------------------- diff --git a/pages/javadocs.md b/pages/javadocs.md new file mode 100644 index 0000000..21be96c --- /dev/null +++ b/pages/javadocs.md @@ -0,0 +1,10 @@ +--- +title: Javadocs Archive +permalink: /javadocs/ +--- + +* [Apache Accumulo 1.8 Javadocs](/1.8/apidocs) +* [Apache Accumulo 1.7 Javadocs](/1.7/apidocs) +* [Apache Accumulo 1.6 Javadocs](/1.6/apidocs) +* [Apache Accumulo 1.5 Javadocs](/1.5/apidocs) +* [Apache Accumulo 1.4 Javadocs](/1.4/apidocs) http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/pages/old_archive.md ---------------------------------------------------------------------- diff --git a/pages/old_archive.md b/pages/old_archive.md new file mode 100644 index 0000000..2b3f2aa --- /dev/null +++ b/pages/old_archive.md @@ -0,0 +1,57 @@ +--- +title: Release Notes +--- + +Apache Accumulo generates release notes for the benefit of users which summarize the important and notable changes that +are contained in that release. Users can leverage these release notes to make make decisions about the benefits (or +concerns) in updating to a new version. + +Each release notes document aim to provide content broken down into the following categories of interest. + +* New Features +* Performance Improvements +* Bug Fixes +* Reference to special upgrade instructions (if any) +* Security Concerns +* API changes outside of standard guarantees +* Community-driven release testing + +## Archives of all release notes + +### 1.7 releases + +* [Apache Accumulo 1.7.2][REL_172] +* [Apache Accumulo 1.7.1][REL_171] +* [Apache Accumulo 1.7.0][REL_170] + +### 1.6 releases + +* [Apache Accumulo 1.6.6][REL_166] +* [Apache Accumulo 1.6.5][REL_165] +* [Apache Accumulo 1.6.4][REL_164] +* [Apache Accumulo 1.6.3][REL_163] +* [Apache Accumulo 1.6.2][REL_162] +* [Apache Accumulo 1.6.1][REL_161] +* [Apache Accumulo 1.6.0][REL_160] + +### 1.5 releases + +* [Apache Accumulo 1.5.4][REL_154] +* [Apache Accumulo 1.5.3][REL_153] +* [Apache Accumulo 1.5.2][REL_152] +* [Apache Accumulo 1.5.1][REL_151] + +[REL_151]: 1.5.1 +[REL_152]: 1.5.2 +[REL_153]: 1.5.3 +[REL_154]: 1.5.4 +[REL_160]: 1.6.0 +[REL_161]: 1.6.1 +[REL_162]: 1.6.2 +[REL_163]: 1.6.3 +[REL_164]: 1.6.4 +[REL_165]: 1.6.5 +[REL_166]: 1.6.6 +[REL_170]: 1.7.0 +[REL_171]: 1.7.1 +[REL_172]: 1.7.2 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/pages/release.md ---------------------------------------------------------------------- diff --git a/pages/release.md b/pages/release.md new file mode 100644 index 0000000..f8c92ae --- /dev/null +++ b/pages/release.md @@ -0,0 +1,24 @@ +--- +title: Release Archive +permalink: "/release/" +redirect_from: + - "/release_notes/" + - "/release_notes.html" +--- + +<div> +<hr> +<h3>{{ site.categories.release[0].date | date: "%Y" }}</h3> +{% for release in site.categories.release %} + {% assign release_year = release.date | date: "%Y" %} + {% assign newer_release_year = release.next.date | date: "%Y" %} + {% if release_year != newer_release_year %} + <hr> + <h3>{{ release_year }}</h3> + {% endif %} + <div class="row" style="margin-top: 15px"> + <div class="col-md-1">{{ release.date | date: "%b %d" }}</div> + <div class="col-md-10"><a href="{{ site.baseurl }}{{ release.url }}">{{ release.title }}</a></div> + </div> +{% endfor %} +</div> http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/9a50bd13/pages/user-manual.md ---------------------------------------------------------------------- diff --git a/pages/user-manual.md b/pages/user-manual.md new file mode 100644 index 0000000..8d2ead5 --- /dev/null +++ b/pages/user-manual.md @@ -0,0 +1,11 @@ +--- +title: User Manual Archive +permalink: /user-manual/ +--- + +* [Apache Accumulo 1.8 User Manual](/1.8/accumulo_user_manual.html) +* [Apache Accumulo 1.7 User Manual](/1.7/accumulo_user_manual.html) +* [Apache Accumulo 1.6 User Manual](/1.6/accumulo_user_manual.html) +* [Apache Accumulo 1.5 User Manual](/1.5/accumulo_user_manual.html) +* [Apache Accumulo 1.4 User Manual](/1.4/accumulo_user_manual.pdf) +* [Apache Accumulo 1.3 User Manual](/1.3/user_manual/)
