This is an automated email from the ASF dual-hosted git repository. kturner pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/master by this push: new 65a48e6 Improved 2.0 release notes and added issue liquid tags 65a48e6 is described below commit 65a48e69a7644e5b6ab54ed732785a98f22907dd Author: Keith Turner <ktur...@apache.org> AuthorDate: Sun Oct 14 15:21:35 2018 -0400 Improved 2.0 release notes and added issue liquid tags --- README.md | 2 + _plugins/links.rb | 31 +++++++++++++++ _posts/release/2017-09-05-accumulo-2.0.0.md | 62 ++++++++++++++++++++++++++--- 3 files changed, 89 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 11f139d..cca86c5 100644 --- a/README.md +++ b/README.md @@ -48,6 +48,8 @@ The source for these tags is at [_plugins/links.rb](_plugins/links.rb). | purl | Creates Property URL | Default is server property. Use `-c` to link to client properties. Accepts server property prefixes (i.e `table.*`) | `{% purl instance.volumes %}` | | dlink | Creates Documentation link | None | `{% dlink getting-stared/clients %}` | | durl | Creates Documentation URL | None | `{% durl troubleshooting/performance %}` | +| ghi | GitHub issue link | None | `{% ghi 100 %}` | +| jira | Jira issue link | None | `{% jira ACCUMULO-1000 %}` | ## Updating property documentation diff --git a/_plugins/links.rb b/_plugins/links.rb index c6e3866..1043c96 100755 --- a/_plugins/links.rb +++ b/_plugins/links.rb @@ -175,9 +175,40 @@ class DocUrlTag < Liquid::Tag end end +class GitHubIssueTag < Liquid::Tag + def initialize(tag_name, text, tokens) + super + @text = text + end + + def render(context) + # TODO sanity check text + # TODO support optional pound char like #123 + url = "https://github.com/apache/accumulo/issues/#{@text}" + return "[##{@text}](#{url})" + end +end + +class JiraTag < Liquid::Tag + def initialize(tag_name, text, tokens) + super + @text = text + end + + def render(context) + # TODO sanity check text + # TODO accept number without ACCUMULO- prefix + url = "https://issues.apache.org/jira/browse/#{@text}" + return "[#{@text}](#{url})" + end +end + + Liquid::Template.register_tag('jlink', JavadocLinkTag) Liquid::Template.register_tag('jurl', JavadocUrlTag) Liquid::Template.register_tag('plink', PropertyLinkTag) Liquid::Template.register_tag('purl', PropertyUrlTag) Liquid::Template.register_tag('dlink', DocLinkTag) Liquid::Template.register_tag('durl', DocUrlTag) +Liquid::Template.register_tag('ghi', GitHubIssueTag) +Liquid::Template.register_tag('jira', JiraTag) diff --git a/_posts/release/2017-09-05-accumulo-2.0.0.md b/_posts/release/2017-09-05-accumulo-2.0.0.md index 590b69f..c9268c4 100644 --- a/_posts/release/2017-09-05-accumulo-2.0.0.md +++ b/_posts/release/2017-09-05-accumulo-2.0.0.md @@ -3,7 +3,10 @@ title: Apache Accumulo 2.0.0 draft: true --- -Apache Accumulo 2.0.0 is a significant release. +Apache Accumulo 2.0.0 is a significant release. These release notes are still a work in +progress. The notes are fairly complete in terms of features added to 2.0, but some +features may still be missing. Also, the notes will be updated in the future to point +to documentation for new features in addition to issues. ## Notable Changes @@ -14,6 +17,13 @@ The new API deprecates `ClientConfiguration` and introduces its own properties file called `accumulo-client.properties` that ships with the Accumulo tarball. See the [client documentation][clients] for more information on how to use the new API. +### Hadoop 3 and Java 8. + +Accumulo 2.x expects at least Java 8 and Hadoop 3. It is built against Java 8 +and Hadoop 3 and the binary tarball is targeted to work with a Java 8 and +Hadoop 3 system. See {% jira ACCUMULO-4826 %}, {% ghi 531 %}, and {% jira +ACCUMULO-4299 %}. + ### Simplified Accumulo scripts and configuration files Accumulo's scripts and configuration were refactored in [ACCUMULO-4490] to make Accumulo @@ -27,17 +37,45 @@ has been reduced from 20 scripts to the four scripts below: Read [this blog post][script-post] for more information on this change. +### New Bulk Import API + +A new bulk import API was added in 2.0 that has very different implementation. This new API supports the following new functionality. + + * Bulk import to an offline table. + * Load plans that specify where files go in a table which avoids opening the + files for inspection. + * Inspection of file on the client side. Inspection of all files is done + before the FATE operation starts. This results in less namenode operations + and fail-fast for bad files (no longer need a fail directory). + * A new improved algorithm to load files into tablets. This new algorithm + scans the metadata table and makes asynchronous load calls to all tablets. + This queues load operations on all tablets at around the same time. The + async RPC calls and beforehand inspection make the bulk load FATE operation + much shorter. + +The shell command for doing bulk load supports the old and new API. To use the +new API from the shell simply omit the failure directory argument. +TODO link to javadoc for new API. See {% ghi 436 %}, {% ghi 472 %}, and {% ghi 570 %}. + +### Summaries + +[Summaries]({% durl development/summaries %}) enables continually generating +statistics about a table with user defined functions. This feature can inform +a user about what is in their table and be used by compaction strategies to +make decisions. For example, using this feature it would be possible to compact +all tablets where deletes are more than 25% of the data. + ### Scan Executors -[Scan executors]({% durl administration/scan-executors %}) were added to -support prioritizing and dedicating scan resources. Each executor has a -configurable number of threads and an optional custom prioritizer. Tables can -be configured in a flexible way to dispatch scans to different executors. +[Scan executors]({% durl administration/scan-executors %}) support prioritizing +and dedicating scan resources. Each executor has a configurable number of +threads and an optional custom prioritizer. Tables can be configured in a +flexible way to dispatch scans to different executors. ### Official Accumulo docker image was created An [official Accumulo docker images][accumulo-docker] was created in [ACCUMULO-4706] to make -it easier for users to run Accumulo in Docker. To support running in Docker, a few changes were +it easier for users to run Accumulo in Docker. To support running in Docker, a few changes were made to Accumulo: * The `--upload-accumulo-site` option was added to `accumulo init` to set properties in accumulo-site.xml @@ -106,6 +144,18 @@ default to the value for crypto.cipher.suite. * [ACCUMULO-4449] - Removed 'slave' terminology and replaced with 'tserver' in most cases. The former 'slaves' config file is now named 'tservers'. Added checks to scripts to fail if 'slaves' file is present. +* {% jira ACCUMULO-4808 %} - Can now create table with splits and offline. Specifying splits +at table creation time can be much faster than adding splits after creation. +* {% jira ACCUMULO-4463 %} - Caching is now pluggable. +* {% jira ACCUMULO-4177 %} - New built in cache implementation based on TinyLFU. +* {% jira ACCUMULO-4376 %} {% jira ACCUMULO-4746 %} - Mutation and Key Fluent APIs allow easy mixing of types. For example a family of type `String` and qualifier of type `byte[]` is much easier to write using this new API. +* {% jira ACCUMULO-4771 %} - The Accumulo monitor was completely rewritten. +* {% jira ACCUMULO-4732 %} - Specify iterators and locality groups at table creation time. +* {% jira ACCUMULO-4612 %} - Use percentages for memory related configuration. +* {% jira ACCUMULO-1787 %} - Two tier compaction strategy. Support compacting small files with snappy and large files with gzip. +* {% ghi 536 %} - Removed mock Accumulo. +* {% ghi 438 %} - Added support for ZStandard compression +* {% ghi 404 %} - Added basic Grafana dashboard example. TODO better link? ## Upgrading