[accumulo-testing] branch master updated: Fix error in Dockerfile. Closes #109
This is an automated email from the ASF dual-hosted git repository. mmiller pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/accumulo-testing.git The following commit(s) were added to refs/heads/master by this push: new 9ece86d Fix error in Dockerfile. Closes #109 9ece86d is described below commit 9ece86d7449a8dde14096451869464a81e8dbc10 Author: Mike Miller AuthorDate: Wed Sep 11 15:49:09 2019 -0400 Fix error in Dockerfile. Closes #109 --- Dockerfile | 1 + 1 file changed, 1 insertion(+) diff --git a/Dockerfile b/Dockerfile index 8c86058..b544020 100644 --- a/Dockerfile +++ b/Dockerfile @@ -39,6 +39,7 @@ COPY ./conf/accumulo-testing.properties /opt/at/conf/ COPY ./conf/log4j.properties* /opt/at/conf/ RUN touch /opt/at/conf/env.sh +COPY ./bin/build /opt/at/bin COPY ./bin/cingest /opt/at/bin COPY ./bin/rwalk /opt/at/bin COPY ./bin/gcs /opt/at/bin
[accumulo-website] branch asf-site updated: Jekyll build from master:f4882d0
This is an automated email from the ASF dual-hosted git repository.
kturner pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new b5dc820 Jekyll build from master:f4882d0
b5dc820 is described below
commit b5dc8204cb31b3c81438645c78d3108755cfffbd
Author: Keith Turner
AuthorDate: Wed Sep 11 15:34:48 2019 -0400
Jekyll build from master:f4882d0
Improved information about new table creation options in 2.0 blog (#193)
---
blog/2019/08/12/why-upgrade.html | 16 +---
feed.xml | 20 +++-
search_data.json | 2 +-
3 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/blog/2019/08/12/why-upgrade.html b/blog/2019/08/12/why-upgrade.html
index 3728ae1..e35c1e0 100644
--- a/blog/2019/08/12/why-upgrade.html
+++ b/blog/2019/08/12/why-upgrade.html
@@ -230,10 +230,20 @@ See the https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.0
More examples for https://github.com/apache/accumulo/blob/master/core/src/test/java/org/apache/accumulo/core/data/KeyBuilderTest.java;>Key
and https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.0.0/org/apache/accumulo/core/data/Mutation.html#at()">Mutation.
-Offline creation
+Table creation options
-Tables can now be created with splits offline. This frees up online
resources to perform other critical operations.
-See https://github.com/apache/accumulo/issues/573;>#573 .
+Tables can now be created with splits, which is much faster than creating a
+table and then adding splits. Tables can also be created in an offline state
+now. The new bulk import API supports offline tables. This enables the
+following method of getting a lot of data into a new table very quickly.
+
+
+ Create offline table with splits
+ Bulk import into new offline table
+ Bring table online
+
+
+See the javadoc for https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.0.0/org/apache/accumulo/core/client/admin/NewTableConfiguration.html;>NewTableConfiguration
and search for methods introduced in 2.0.0 for more information.
Search Documentation
diff --git a/feed.xml b/feed.xml
index 79343de..3a37d3e 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
https://accumulo.apache.org/
https://accumulo.apache.org/feed.xml; rel="self"
type="application/rss+xml"/>
-Wed, 11 Sep 2019 14:28:48 -0400
-Wed, 11 Sep 2019 14:28:48 -0400
+Wed, 11 Sep 2019 15:34:41 -0400
+Wed, 11 Sep 2019 15:34:41 -0400
Jekyll v3.8.6
@@ -239,10 +239,20 @@ See the a
href=https://static.javadoc.io/org.apache.accumulo/accumulo-
pMore examples for a
href=https://github.com/apache/accumulo/blob/master/core/src/test/java/org/apache/accumulo/core/data/KeyBuilderTest.javaKey/a;
and a
href=https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.0.0/org/apache/accumulo/core/data/Mutation.html#at()Mutation/a./p
-h2 id=offline-creationOffline creation/h2
+h2 id=table-creation-optionsTable creation
options/h2
-pTables can now be created with splits offline. This frees up online
resources to perform other critical operations.
-See a
href=https://github.com/apache/accumulo/issues/573#573
/a./p
+pTables can now be created with splits, which is much faster than
creating a
+table and then adding splits. Tables can also be created in an offline state
+now. The new bulk import API supports offline tables. This enables the
+following method of getting a lot of data into a new table very
quickly./p
+
+ul
+ liCreate offline table with splits/li
+ liBulk import into new offline table/li
+ liBring table online/li
+/ul
+
+pSee the javadoc for a
href=https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.0.0/org/apache/accumulo/core/client/admin/NewTableConfiguration.htmlNewTableConfiguration/a;
and search for methods introduced in 2.0.0 for more information./p
h2 id=search-documentationSearch Documentation/h2
diff --git a/search_data.json b/search_data.json
index a5c99df..7a183d4 100644
--- a/search_data.json
+++ b/search_data.json
@@ -305,7 +305,7 @@
"blog-2019-08-12-why-upgrade-html": {
"title": "Top 10 Reasons to Upgrade",
- "content" : "Accumulo 2.0 has been in development for quite some
time now and is packed with new features, bugfixes, performance improvements
and redesigned components. All of these changes bring challengeswhen upgrading
your production cluster so you may be wondering… why should I upgrade?My top 10
reasons to upgrade. For all changes see the release notes Summaries New Bulk
Import Simplified Scripts and Config New Monitor New APIs Offline creation
Search Documentation On [...]
+ "content" : "Accumulo 2.0 has been in development for quite some
time now and is packed with new features, bugfixes, performance improvements
and redesigned
[accumulo-website] branch master updated: Improved information about new table creation options in 2.0 blog (#193)
This is an automated email from the ASF dual-hosted git repository.
kturner pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/master by this push:
new f4882d0 Improved information about new table creation options in 2.0
blog (#193)
f4882d0 is described below
commit f4882d00406812d2910185d8d20a0a18e028a49c
Author: Keith Turner
AuthorDate: Wed Sep 11 15:34:04 2019 -0400
Improved information about new table creation options in 2.0 blog (#193)
---
_posts/blog/2019-08-12-why-upgrade.md | 14 +++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/_posts/blog/2019-08-12-why-upgrade.md
b/_posts/blog/2019-08-12-why-upgrade.md
index cb1023c..d8b4b2e 100644
--- a/_posts/blog/2019-08-12-why-upgrade.md
+++ b/_posts/blog/2019-08-12-why-upgrade.md
@@ -87,10 +87,18 @@ m.at().family("002").qualifier(new byte[] {0,1}).delete();
More examples for [Key] and [Mutation].
-## Offline creation
+## Table creation options
-Tables can now be created with splits offline. This frees up online resources
to perform other critical operations.
-See {% ghi 573 %}.
+Tables can now be created with splits, which is much faster than creating a
+table and then adding splits. Tables can also be created in an offline state
+now. The new bulk import API supports offline tables. This enables the
+following method of getting a lot of data into a new table very quickly.
+
+ * Create offline table with splits
+ * Bulk import into new offline table
+ * Bring table online
+
+See the javadoc for {% jlink
org.apache.accumulo.core.client.admin.NewTableConfiguration %} and search for
methods introduced in 2.0.0 for more information.
## Search Documentation
[accumulo-website] branch asf-site updated: Jekyll build from master:0b8d301
This is an automated email from the ASF dual-hosted git repository. kturner pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/accumulo-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 0ccf9a5 Jekyll build from master:0b8d301 0ccf9a5 is described below commit 0ccf9a5c555262878609cf362cf143034ff8ad14 Author: Keith Turner AuthorDate: Wed Sep 11 14:28:56 2019 -0400 Jekyll build from master:0b8d301 fix links in why upgrade to 2.0 blog --- blog/2019/08/12/why-upgrade.html | 12 ++-- feed.xml | 16 search_data.json | 2 +- 3 files changed, 15 insertions(+), 15 deletions(-) diff --git a/blog/2019/08/12/why-upgrade.html b/blog/2019/08/12/why-upgrade.html index 76d5e16..3728ae1 100644 --- a/blog/2019/08/12/why-upgrade.html +++ b/blog/2019/08/12/why-upgrade.html @@ -178,7 +178,7 @@ part of your Tables, so they won’t impact ingest or query performance of your An admin could analyze R-File summaries for better performance tuning of a cluster -For more info check out the summary docs for 2.0 +For more info check out the summary docs for 2.0 New Bulk Import @@ -233,24 +233,24 @@ See the https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.0 Offline creation Tables can now be created with splits offline. This frees up online resources to perform other critical operations. -See the https://github.com/apache/accumulo/issues/573 )">GitHub issue. +See https://github.com/apache/accumulo/issues/573;>#573 . Search Documentation New ability to quickly search documentation on the website. The user manual was completely redone -for 2.0. Check it out here. Users can now quickly https://accumulo.apache.org/search/;>search the website across all 2.x documentation. +for 2.0. Check it out here. Users can now quickly https://accumulo.apache.org/search/;>search the website across all 2.x documentation. New Crypto On disk encryption was redone to be more secure and flexible. For an in depth description of how Accumulo -does on disk encryption, see the user manual. NOTE: This is currently an experimental feature. +does on disk encryption, see the user manual. NOTE: This is currently an experimental feature. An experimental feature is considered a work in progress or incomplete and could change. Zstandard compression Support for Zstandard compression was added in 2.0. It has been measured to perform better than gzip (better compression ratio and speed) and snappy (better compression ratio). Checkout Facebook’s https://facebook.github.io/zstd/;>github for Zstandard and -the table.file.compress.type property for configuring Accumulo. +the table.file.compress.type property for configuring Accumulo. New Scan Executors @@ -262,7 +262,7 @@ better cluster resource utilization. a majority of the server resources causing the one really fast scan to be delayed. Scan executors allow an admin to configure the cluster in a way that allows the one fast scan to be prioritized and not have to wait. -Checkout some examples in the user guide. +Checkout some examples in the user guide. diff --git a/feed.xml b/feed.xml index fe51dbd..79343de 100644 --- a/feed.xml +++ b/feed.xml @@ -6,8 +6,8 @@ https://accumulo.apache.org/ https://accumulo.apache.org/feed.xml; rel="self" type="application/rss+xml"/> -Wed, 11 Sep 2019 09:10:18 -0400 -Wed, 11 Sep 2019 09:10:18 -0400 +Wed, 11 Sep 2019 14:28:48 -0400 +Wed, 11 Sep 2019 14:28:48 -0400 Jekyll v3.8.6 @@ -187,7 +187,7 @@ part of your Tables, so they won’t impact ingest or query performance of your liAn admin could analyze R-File summaries for better performance tuning of a cluster/li /ul -pFor more info check out the a href=[summaries ](/docs/2.x//development/summaries )summary docs for 2.0/a/p +pFor more info check out the a href=/docs/2.x//development/summariessummary docs for 2.0/a/p h3 id=new-bulk-importNew Bulk Import/h3 @@ -242,24 +242,24 @@ See the a href=https://static.javadoc.io/org.apache.accumulo/accumulo- h2 id=offline-creationOffline creation/h2 pTables can now be created with splits offline. This frees up online resources to perform other critical operations. -See the a href=[#573 ](https://github.com/apache/accumulo/issues/573 )GitHub issue/a./p +See a href=https://github.com/apache/accumulo/issues/573#573 /a./p h2 id=search-documentationSearch Documentation/h2 pNew ability to quickly search documentation on the website. The user manual was completely redone -for 2.0. Check it out a href=[quickstart ](/docs/2.x//getting-started/quickstart )here/a. Users can now quickly a href=https://accumulo.apache.org/search/search/a; the website across all 2.x documentation./p +for 2.0. Check it out a href=/docs/2.x//getting-started/quickstarthere/a. Users can now quickly a
[accumulo-website] branch master updated: fix links in why upgrade to 2.0 blog
This is an automated email from the ASF dual-hosted git repository.
kturner pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/master by this push:
new 0b8d301 fix links in why upgrade to 2.0 blog
0b8d301 is described below
commit 0b8d30159823d6f758dcebf045a5431f54d625e3
Author: Keith Turner
AuthorDate: Wed Sep 11 14:26:18 2019 -0400
fix links in why upgrade to 2.0 blog
---
_posts/blog/2019-08-12-why-upgrade.md | 15 +++
1 file changed, 7 insertions(+), 8 deletions(-)
diff --git a/_posts/blog/2019-08-12-why-upgrade.md
b/_posts/blog/2019-08-12-why-upgrade.md
index 12c30f1..cb1023c 100644
--- a/_posts/blog/2019-08-12-why-upgrade.md
+++ b/_posts/blog/2019-08-12-why-upgrade.md
@@ -90,7 +90,7 @@ More examples for [Key] and [Mutation].
## Offline creation
Tables can now be created with splits offline. This frees up online resources
to perform other critical operations.
-See the [GitHub issue][offline].
+See {% ghi 573 %}.
## Search Documentation
@@ -121,18 +121,17 @@ to configure the cluster in a way that allows the one
fast scan to be prioritize
Checkout some examples in the [user guide][scans].
-[FATE]: {% dlink /administration/fate %}
+[FATE]: {% durl /administration/fate %}
[new-bulk]:
https://accumulo.apache.org/release/accumulo-2.0.0/#new-bulk-import-api
[scripts]:
https://accumulo.apache.org/blog/2016/11/16/simpler-scripts-and-config.html
-[summary]: {% dlink /development/summaries %}
+[summary]: {% durl /development/summaries %}
[client]: {% jurl org.apache.accumulo.core.client.Accumulo %}
[Key]:
https://github.com/apache/accumulo/blob/master/core/src/test/java/org/apache/accumulo/core/data/KeyBuilderTest.java
[Mutation]:
https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.0.0/org/apache/accumulo/core/data/Mutation.html#at()
-[offline]: {% ghi 573 %}
-[manual]: {% dlink /getting-started/quickstart %}
+[manual]: {% durl /getting-started/quickstart %}
[search]: https://accumulo.apache.org/search/
-[crypto]: {% dlink /security/on-disk-encryption %}
+[crypto]: {% durl /security/on-disk-encryption %}
[rel]: https://accumulo.apache.org/release/accumulo-2.0.0/
[zstd]: https://facebook.github.io/zstd/
-[z-config]: {% dlink /configuration/server-properties %}
-[scans]: {% dlink /administration/scan-executors %}
+[z-config]: {% durl /configuration/server-properties %}
+[scans]: {% durl /administration/scan-executors %}
[accumulo] branch master updated: Use concurrent map instead of cache for bulkImported map (#1356)
This is an automated email from the ASF dual-hosted git repository.
kturner pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/master by this push:
new efc9311 Use concurrent map instead of cache for bulkImported map
(#1356)
efc9311 is described below
commit efc931102180562d1f17f9493bb7d4cd55f3f4a1
Author: Keith Turner
AuthorDate: Wed Sep 11 13:08:27 2019 -0400
Use concurrent map instead of cache for bulkImported map (#1356)
While analyzing some tablet code I noticed a Guava cahce was being used
like a map. When I first saw the code I was concerned the cache may
evict entries (which would be a bug). However the cache was created
with default settings, which according to the javadoc does not
automatically evict. This commit replaces the cache with a concurrent
map so no one has to do this analysis again.
---
.../tserver/tablet/BulkImportCacheCleaner.java | 2 +-
.../org/apache/accumulo/tserver/tablet/Tablet.java | 34 --
2 files changed, 19 insertions(+), 17 deletions(-)
diff --git
a/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/BulkImportCacheCleaner.java
b/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/BulkImportCacheCleaner.java
index c908840..03f33e2 100644
---
a/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/BulkImportCacheCleaner.java
+++
b/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/BulkImportCacheCleaner.java
@@ -40,7 +40,7 @@ public class BulkImportCacheCleaner implements Runnable {
// gather the list of transactions the tablets have cached
final Set tids = new HashSet<>();
for (Tablet tablet : server.getOnlineTablets().values()) {
- tids.addAll(tablet.getBulkIngestedFiles().keySet());
+ tids.addAll(tablet.getBulkIngestedTxIds());
}
try {
// get the current transactions from ZooKeeper
diff --git
a/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/Tablet.java
b/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/Tablet.java
index bbc70a4..1e89624 100644
---
a/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/Tablet.java
+++
b/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/Tablet.java
@@ -38,7 +38,7 @@ import java.util.PriorityQueue;
import java.util.Set;
import java.util.SortedMap;
import java.util.TreeMap;
-import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
@@ -152,7 +152,6 @@ import org.slf4j.LoggerFactory;
import com.google.common.base.Preconditions;
import com.google.common.cache.Cache;
-import com.google.common.cache.CacheBuilder;
import com.google.common.collect.ImmutableSet;
import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
@@ -254,8 +253,13 @@ public class Tablet {
// tablet lock.
private final Set bulkImporting = new HashSet<>();
- // Files that were successfully bulk imported.
- private final Cache> bulkImported =
CacheBuilder.newBuilder().build();
+ // Files that were successfully bulk imported. Using a concurrent map
supports non-locking
+ // operations on the key set which is useful for the periodic task that
cleans up completed bulk
+ // imports for all tablets. However the values of this map are ArrayList
which do not support
+ // concurrency. This is ok because all operations on the values are done
while the tablet lock is
+ // held.
+ private final ConcurrentHashMap> bulkImported =
+ new ConcurrentHashMap>();
private final int logId;
@@ -344,7 +348,7 @@ public class Tablet {
this.location = locationPath;
this.tabletDirectory = tabletPaths.dir;
for (Entry> entry : data.getBulkImported().entrySet()) {
- this.bulkImported.put(entry.getKey(), new
CopyOnWriteArrayList<>(entry.getValue()));
+ this.bulkImported.put(entry.getKey(), new ArrayList<>(entry.getValue()));
}
final List logEntries = tabletPaths.logEntries;
@@ -2246,17 +2250,17 @@ public class Tablet {
MetadataTableUtil.splitTablet(high, extent.getPrevEndRow(), splitRatio,
getTabletServer().getContext(), getTabletServer().getLock());
MasterMetadataUtil.addNewTablet(getTabletServer().getContext(), low,
lowDirectory,
- getTabletServer().getTabletSession(), lowDatafileSizes,
getBulkIngestedFiles(), time,
- lastFlushID, lastCompactID, getTabletServer().getLock());
+ getTabletServer().getTabletSession(), lowDatafileSizes,
bulkImported, time, lastFlushID,
+ lastCompactID, getTabletServer().getLock());
MetadataTableUtil.finishSplit(high, highDatafileSizes,
highDatafilesToRemove,
getTabletServer().getContext(), getTabletServer().getLock());
[accumulo] branch master updated: format MasterTime #1354
This is an automated email from the ASF dual-hosted git repository.
kturner pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/master by this push:
new b978cbd format MasterTime #1354
b978cbd is described below
commit b978cbd57374632d0a7631c112c7bb810bc74934
Author: Keith Turner
AuthorDate: Wed Sep 11 13:06:19 2019 -0400
format MasterTime #1354
---
.../master/src/main/java/org/apache/accumulo/master/MasterTime.java | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git
a/server/master/src/main/java/org/apache/accumulo/master/MasterTime.java
b/server/master/src/main/java/org/apache/accumulo/master/MasterTime.java
index 6c88a4d..fcd910b 100644
--- a/server/master/src/main/java/org/apache/accumulo/master/MasterTime.java
+++ b/server/master/src/main/java/org/apache/accumulo/master/MasterTime.java
@@ -57,8 +57,7 @@ public class MasterTime extends TimerTask {
try {
zk.putPersistentData(zPath, "0".getBytes(UTF_8), NodeExistsPolicy.SKIP);
skewAmount = new AtomicLong(
- Long.parseLong(new String(zk.getData(zPath, null), UTF_8))
- - System.nanoTime());
+ Long.parseLong(new String(zk.getData(zPath, null), UTF_8)) -
System.nanoTime());
} catch (Exception ex) {
throw new IOException("Error updating master time", ex);
}
@@ -105,7 +104,8 @@ public class MasterTime extends TimerTask {
case UNLOAD_METADATA_TABLETS:
case UNLOAD_ROOT_TABLET:
try {
- zk.putPersistentData(zPath, Long.toString(System.nanoTime() +
skewAmount.get()).getBytes(UTF_8),
+ zk.putPersistentData(zPath,
+ Long.toString(System.nanoTime() +
skewAmount.get()).getBytes(UTF_8),
NodeExistsPolicy.OVERWRITE);
} catch (Exception ex) {
if (log.isDebugEnabled()) {
[accumulo] branch master updated (a4c7dad -> 3be238c)
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/accumulo.git. from a4c7dad Merge branch '2.0' add 3be238c Remove synchronization and use AtomicLong (#1354) No new revisions were added by this update. Summary of changes: .../java/org/apache/accumulo/master/MasterTime.java | 17 + 1 file changed, 9 insertions(+), 8 deletions(-)
[accumulo-website] branch asf-site updated: Jekyll build from master:e2c23d8
This is an automated email from the ASF dual-hosted git repository.
kturner pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new e29fede Jekyll build from master:e2c23d8
e29fede is described below
commit e29fede55a49559d45048d63dcae4695dbe3253c
Author: Keith Turner
AuthorDate: Wed Sep 11 09:10:26 2019 -0400
Jekyll build from master:e2c23d8
Add blog post about storing Accumulo data in S3 (#192)
---
blog/2019/09/10/accumulo-S3-notes.html | 302 +
feed.xml | 210 +++
index.html | 14 +-
news/index.html| 7 +
redirects.json | 2 +-
search_data.json | 8 +
6 files changed, 468 insertions(+), 75 deletions(-)
diff --git a/blog/2019/09/10/accumulo-S3-notes.html
b/blog/2019/09/10/accumulo-S3-notes.html
new file mode 100644
index 000..cb536ed
--- /dev/null
+++ b/blog/2019/09/10/accumulo-S3-notes.html
@@ -0,0 +1,302 @@
+
+
+
+
+
+
+
+https://maxcdn.bootstrapcdn.com/bootswatch/3.3.7/paper/bootstrap.min.css;
rel="stylesheet"
integrity="sha384-awusxf8AUojygHf2+joICySzB780jVvQaVCAt1clU3QsyAitLGul28Qxb2r1e5g+"
crossorigin="anonymous">
+
+https://cdn.datatables.net/v/bs/jq-2.2.3/dt-1.10.12/datatables.min.css;>
+
+
+Using S3 as a data store for Accumulo
+
+https://cdnjs.cloudflare.com/ajax/libs/jquery/2.2.4/jquery.min.js";
integrity="sha256-BbhdlvQf/xTY9gja0Dq3HiwQF8LaCRTXxZKRutelT44="
crossorigin="anonymous">
+https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js";
integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa"
crossorigin="anonymous">
+https://cdn.datatables.net/v/bs/jq-2.2.3/dt-1.10.12/datatables.min.js";>
+
+ // show location of canonical site if not currently on the canonical site
+ $(function() {
+var host = window.location.host;
+if (typeof host !== 'undefined' && host !== 'accumulo.apache.org') {
+ $('#non-canonical').show();
+}
+ });
+
+ $(function() {
+// decorate section headers with anchors
+return $("h2, h3, h4, h5, h6").each(function(i, el) {
+ var $el, icon, id;
+ $el = $(el);
+ id = $el.attr('id');
+ icon = '';
+ if (id) {
+return $el.append($("").addClass("header-link").attr("href", "#"
+ id).html(icon));
+ }
+});
+ });
+
+ // fix sidebar width in documentation
+ $(function() {
+var $affixElement = $('div[data-spy="affix"]');
+$affixElement.width($affixElement.parent().width());
+ });
+
+
+
+
+
+
+
+
+
+Toggle navigation
+
+
+
+
+
+
+
+
+Download
+Tour
+
+ Releases
+
+2.0.0 (Latest)
+1.9.3
+Archive
+
+
+
+ Documentation
+
+User Manual (2.x)
+Quickstart (1.x)
+Accumulo Maven
Plugin
+User Manual
(1.9)
+Javadocs (1.9)
+External Docs
+Archive
+
+
+
+ Community
+
+Contact Us
+How To Contribute
+People
+Related Projects
+
+
+Search
+
+
+
+ https://www.apache.org/foundation/press/kit/feather.svg; width="15"/>
+
+https://www.apache.org;>Apache Homepage
+https://www.apache.org/licenses/;>License
+https://www.apache.org/foundation/sponsorship;>Sponsorship
+https://www.apache.org/security;>Security
+https://www.apache.org/foundation/thanks;>Thanks
+https://www.apache.org/foundation/policies/conduct;>Code of Conduct
+https://www.apache.org/events/current-event.html;>Current Event
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Visit the official site at: https://accumulo.apache.org;>https://accumulo.apache.org
+
+
+
+ Using S3 as a data store for Accumulo
+
+
+Author: Keith Turner
+Date: 10 Sep 2019
+
+
+
+Accumulo can store its files in S3, however S3 does not support the needs of
+write ahead logs and the Accumulo metadata table. One way to solve this problem
+is to store the metadata table and write ahead logs in HDFS and everything else
+in S3. This post shows how to do that using Accumulo 2.0 and Hadoop 3.2.0.
+Running on S3 requires a new feature in Accumulo 2.0, that volume choosers are
+aware of write ahead logs.
+
+Hadoop setup
+
+At least the following settings should be added to Hadoop’s core-site.xml file on each node in the
cluster.
+
+property
+
[accumulo-website] branch master updated (bc6bd30 -> e2c23d8)
This is an automated email from the ASF dual-hosted git repository. kturner pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/accumulo-website.git. from bc6bd30 Fix #190 Update PMC member add 79dce71 Add blog post about storing Accumulo data in S3 add 4424656 code review update new e2c23d8 Add blog post about storing Accumulo data in S3 (#192) The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: _posts/blog/2019-09-10-accumulo-S3-notes.md | 145 1 file changed, 145 insertions(+) create mode 100644 _posts/blog/2019-09-10-accumulo-S3-notes.md
[accumulo-website] 01/01: Add blog post about storing Accumulo data in S3 (#192)
This is an automated email from the ASF dual-hosted git repository. kturner pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/accumulo-website.git commit e2c23d83216c0a47cac8d7a1531eafe5443e7085 Merge: bc6bd30 4424656 Author: Keith Turner AuthorDate: Wed Sep 11 09:05:10 2019 -0400 Add blog post about storing Accumulo data in S3 (#192) _posts/blog/2019-09-10-accumulo-S3-notes.md | 145 1 file changed, 145 insertions(+)
