This is an automated email from the ASF dual-hosted git repository.
mwalch pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 4aa1892 Jekyll build from master:ca3ff8a
4aa1892 is described below
commit 4aa189249eb145d2493a521960630f5deeda6c14
Author: Mike Walch <[email protected]>
AuthorDate: Wed Sep 6 15:41:44 2017 -0400
Jekyll build from master:ca3ff8a
Removed 2.0.0 notes from feed.xml & latest news (#24)
---
feed.xml | 197 ++++++++++++++++++++++++++++++++++----
index.html | 13 +--
release/accumulo-2.0.0/index.html | 1 +
3 files changed, 185 insertions(+), 26 deletions(-)
diff --git a/feed.xml b/feed.xml
index 3e5b5c2..0504dc4 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,28 +6,10 @@
</description>
<link>https://accumulo.apache.org/</link>
<atom:link href="https://accumulo.apache.org/feed.xml"; rel="self"
type="application/rss+xml"/>
- <pubDate>Tue, 05 Sep 2017 15:29:06 -0400</pubDate>
- <lastBuildDate>Tue, 05 Sep 2017 15:29:06 -0400</lastBuildDate>
+ <pubDate>Wed, 06 Sep 2017 15:41:36 -0400</pubDate>
+ <lastBuildDate>Wed, 06 Sep 2017 15:41:36 -0400</lastBuildDate>
<generator>Jekyll v3.3.1</generator>
- <item>
- <title>Apache Accumulo 2.0.0</title>
- <description><h2 id="major-changes">Major
Changes</h2>
-
-<h2 id="other-notable-changes">Other Notable Changes</h2>
-
-<h2 id="upgrading">Upgrading</h2>
-
-<h2 id="testing">Testing</h2>
-</description>
- <pubDate>Tue, 05 Sep 2017 00:00:00 -0400</pubDate>
- <link>https://accumulo.apache.org/release/accumulo-2.0.0/</link>
- <guid
isPermaLink="true">https://accumulo.apache.org/release/accumulo-2.0.0/</guid>
-
-
- <category>release</category>
-
- </item>
<item>
<title>Accumulo Summit is on October 16th!</title>
@@ -1368,5 +1350,180 @@ Commands:
</item>
+ <item>
+ <title>Durability Performance Implications</title>
+ <description><h2 id="overview">Overview</h2>
+
+<p>Accumulo stores recently written data in a sorted in memory map.
Before data is
+added to this map, it’s written to an unsorted write ahead log(WAL). In the
+case when a tablet server dies, the recently written data is recovered from the
+WAL.</p>
+
+<p>When data is written to Accumulo the following happens :</p>
+
+<ul>
+ <li>Client sends a batch of mutations to a tablet server</li>
+ <li>Tablet server does the following :
+ <ul>
+ <li>Writes mutation to tablet servers’ WAL</li>
+ <li>Sync or flush tablet servers’ WAL</li>
+ <li>Adds mutations to sorted in memory map of each
tablet.</li>
+ <li>Reports success back to client.</li>
+ </ul>
+ </li>
+</ul>
+
+<p>The sync/flush step above moves data written to the WAL from memory
to disk.
+Write ahead logs are stored in HDFS. HDFS supports two ways of forcing data to
+disk for an open file : <code
class="highlighter-rouge">hsync</code> and <code
class="highlighter-rouge">hflush</code>.</p>
+
+<h2 id="hdfs-syncflush-details">HDFS Sync/Flush
Details</h2>
+
+<p>When <code
class="highlighter-rouge">hflush</code> is called on a WAL,
it does not guarantee data is on disk. It
+only guarantees that data is in OS buffers on each datanode and on its way to
disk.
+As a result calls to <code
class="highlighter-rouge">hflush</code> are very fast. If a
WAL is replicated to 3 data
+nodes then data may be lost if all three machines reboot or die. If the
datanode
+process dies, then data loss will not happen because the data was in OS buffers
+waiting to be written to disk. The machines have to reboot or die for data
loss to
+occur.</p>
+
+<p>In order to avoid data loss in the event of reboot, <code
class="highlighter-rouge">hsync</code> can be called. This
+will ensure data is written to disk on all datanodes before returning. When
+using <code class="highlighter-rouge">hsync</code> for
the WAL, if Accumulo reports success to a user it means the
+data is on disk. However <code
class="highlighter-rouge">hsync</code> is much slower than
<code class="highlighter-rouge">hflush</code> and the way
it’s
+implemented exacerbates the problem. For example <code
class="highlighter-rouge">hflush</code> make take 1ms and
+<code class="highlighter-rouge">hsync</code> may take
50ms. This difference will impact writes to Accumulo and can
+be mitigated in some situations with larger buffers in Accumulo.</p>
+
+<p>HDFS keeps checksum data internally by default. Datanodes store
checksum data
+in a separate file in the local filesystem. This means when <code
class="highlighter-rouge">hsync</code> is called
+on a WAL, two files must be synced on each datanode. Syncing two files doubles
+the time. To make matters even worse, when the two files are synced the local
+filesystem metadata is also synced. Depending on the local filesystem and its
+configuration, syncing the metadata may or may not take time. In the worst
+case, we need to wait for four sync operations at the local filesystem level on
+each datanode. One thing I am not sure about, is if these sync operations occur
+in parallel on the replicas on different datanodes. If anyone can answer this
+question, please let us know on the <a
href="/mailing_list">dev list</a>. The following pointers
show
+where sync occurs in the datanode code.</p>
+
+<ul>
+ <li><a
href="https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java#L358">BlockReceiver.flushOrSync()</a>
calls <a
href="https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/ReplicaOutputStreams.java#L78">ReplicaOutputStreams.syncDataOut()</a>
a [...]
+ <li>The methods in ReplicaOutputStreams call <a
href="https://docs.oracle.com/javase/8/docs/api/java/nio/channels/FileChannel.html#force-boolean-">FileChannel.force(true)</a>
which
+synchronously flushes data and filesystem metadata.</li>
+</ul>
+
+<p>If files were preallocated (this would avoid syncing local filesystem
metadata)
+and checksums were stored in-line, then 1 sync could be done instead of
4.</p>
+
+<h2 id="configuring-wal-flushsync-in-accumulo-16">Configuring
WAL flush/sync in Accumulo 1.6</h2>
+
+<p>Accumulo 1.6.0 only supported <code
class="highlighter-rouge">hsync</code> and this caused <a
href="/release/accumulo-1.6.0#slower-writes-than-previous-accumulo-versions">performance
+problems</a>. In order to offer better performance, the option to
+configure <code class="highlighter-rouge">hflush</code>
was <a
href="/release/accumulo-1.6.1#write-ahead-log-sync-implementation">added
in 1.6.1</a>. The
+<a
href="/1.6/accumulo_user_manual#_tserver_wal_sync_method">tserver.wal.sync.method</a>
configuration option was added to support
+this feature. This was a tablet server wide option that applied to everything
+written to any table.</p>
+
+<h2 id="group-commit">Group Commit</h2>
+
+<p>Each Accumulo tablet server has a single WAL. When multiple clients
send
+mutations to a tablet server at around the same time, the tablet sever may
group
+all of this into a single WAL operation. It will do this instead of writing
and
+syncing or flushing each client’s mutations to the WAL separately. Doing this
+increase throughput and lowers average latency for clients.</p>
+
+<h2 id="configuring-wal-flushsync-in-accumulo-17">Configuring
WAL flush/sync in Accumulo 1.7+</h2>
+
+<p>Accumulo 1.7.0 introduced <a
href="/1.7/accumulo_user_manual#_table_durability">table.durability</a>,
a new per table property
+for configuring durability. It also stopped using the <code
class="highlighter-rouge">tserver.wal.sync.method</code>
+property. The <code
class="highlighter-rouge">table.durability</code> property
has the following four legal values.
+This property defaults to the most durable option which is <code
class="highlighter-rouge">sync</code>.</p>
+
+<ul>
+ <li><strong>none</strong> : Do not write to WAL</li>
+ <li><strong>log</strong> : Write to WAL, but do not
sync</li>
+ <li><strong>flush</strong> : Write to WAL and call
<code class="highlighter-rouge">hflush</code></li>
+ <li><strong>sync</strong> : Write to WAL and call <code
class="highlighter-rouge">hsync</code></li>
+</ul>
+
+<p>If multiple writes arrive at around the same time with different
durability
+settings, then the group commit code will choose the most durable. This can
+cause one tables settings to slow down writes to another table. Basically, one
+table that is set to <code
class="highlighter-rouge">sync</code> can impact the entire
system.</p>
+
+<p>In Accumulo 1.6, it was easy to make all writes use <code
class="highlighter-rouge">hflush</code> because there was
+only one tserver setting. Getting everything to use <code
class="highlighter-rouge">flush</code> in 1.7 and later
+can be a little tricky because by default the Accumulo metadata table is set to
+use <code class="highlighter-rouge">sync</code>. The
following shell commands show this. The first command sets
+<code
class="highlighter-rouge">table.durability=flush</code> as a
system wide default for all tables. However, the
+metadata table is still set to <code
class="highlighter-rouge">sync</code>, because it has a per
table override for
+that setting. This override is set when Accumulo is initialized. To get this
+table to use <code
class="highlighter-rouge">flush</code>, the per table
override must be deleted. After deleting
+those properties, the metadata tables will inherit the system wide
setting.</p>
+
+<div class="highlighter-rouge"><pre
class="highlight"><code>root@uno&gt; config -s
table.durability=flush
+root@uno&gt; createtable foo
+root@uno foo&gt; config -t foo -f table.durability
+-----------+---------------------+----------------------------------------------
+SCOPE | NAME | VALUE
+-----------+---------------------+----------------------------------------------
+default | table.durability .. | sync
+system | @override ...... | flush
+-----------+---------------------+----------------------------------------------
+root@uno&gt; config -t accumulo.metadata -f table.durability
+-----------+---------------------+----------------------------------------------
+SCOPE | NAME | VALUE
+-----------+---------------------+----------------------------------------------
+default | table.durability .. | sync
+system | @override ...... | flush
+table | @override ...... | sync
+-----------+---------------------+----------------------------------------------
+root@uno&gt; config -t accumulo.metadata -d table.durability
+root@uno&gt; config -t accumulo.metadata -f table.durability
+-----------+---------------------+----------------------------------------------
+SCOPE | NAME | VALUE
+-----------+---------------------+----------------------------------------------
+default | table.durability .. | sync
+system | @override ...... | flush
+-----------+---------------------+----------------------------------------------
+</code></pre>
+</div>
+
+<p>In short, executing the following commands will make all writes use
<code class="highlighter-rouge">flush</code>
+(assuming no other tables or namespaces have been specifically set to <code
class="highlighter-rouge">sync</code>).</p>
+
+<div class="highlighter-rouge"><pre
class="highlight"><code>config -s table.durability=flush
+config -t accumulo.metadata -d table.durability
+config -t accumulo.root -d table.durability
+</code></pre>
+</div>
+
+<p>Even with these settings adjusted, minor compactions could still
force <code class="highlighter-rouge">hsync</code>
+to be called in 1.7.0 and 1.7.1. This was fixed in 1.7.2 and 1.8.0. See the
+<a
href="/release/accumulo-1.7.2#minor-performance-improvements">1.7.2
release notes</a> and <a
href="https://issues.apache.org/jira/browse/ACCUMULO-4112">ACCUMULO-4112</a>
for more details.</p>
+
+<p>In addition to the per table durability setting, a per batch writer
durability
+setting was also added in 1.7.0. See
+<a
href="/1.8/apidocs/org/apache/accumulo/core/client/BatchWriterConfig.html#setDurability(org.apache.accumulo.core.client.Durability)">BatchWriterConfig.setDurability(…)</a>.
This means any client could
+potentially cause a <code
class="highlighter-rouge">hsync</code> operation to occur,
even if the system is
+configured to use <code
class="highlighter-rouge">hflush</code>.</p>
+
+<h2 id="improving-the-situation">Improving the
situation</h2>
+
+<p>The more granular durability settings introduced in 1.7.0 can cause
some
+unexpected problems. <a
href="https://issues.apache.org/jira/browse/ACCUMULO-4146">ACCUMULO-4146</a>
suggests one possible way to solve these
+problems with Per-durability write ahead logs.</p>
+
+</description>
+ <pubDate>Wed, 02 Nov 2016 13:00:00 -0400</pubDate>
+
<link>https://accumulo.apache.org/blog/2016/11/02/durability-performance.html</link>
+ <guid
isPermaLink="true">https://accumulo.apache.org/blog/2016/11/02/durability-performance.html</guid>
+
+
+ <category>blog</category>
+
+ </item>
+
</channel>
</rss>
diff --git a/index.html b/index.html
index 25b68f8..4d4b705 100644
--- a/index.html
+++ b/index.html
@@ -187,12 +187,6 @@
<div class="col-sm-12 panel panel-default">
<p style="font-size: 24px; margin-bottom: 0px;">Latest News</p>
- <div class="row latest-news-item">
- <div class="col-sm-12" style="margin-bottom: 5px">
- <span style="font-size: 12px; margin-right: 5px;">Sep 2017</span>
- <a href="/release/accumulo-2.0.0/">Apache Accumulo 2.0.0</a>
- </div>
- </div>
<div class="row latest-news-item">
<div class="col-sm-12" style="margin-bottom: 5px">
@@ -222,6 +216,13 @@
</div>
</div>
+ <div class="row latest-news-item">
+ <div class="col-sm-12" style="margin-bottom: 5px">
+ <span style="font-size: 12px; margin-right: 5px;">Mar 2017</span>
+ <a href="/blog/2017/03/21/happy-anniversary-accumulo.html">Happy
Anniversary Accumulo</a>
+ </div>
+ </div>
+
<div id="news-archive-link">
<p>View all posts in the <a href="/news">news archive</a></p>
</div>
diff --git a/release/accumulo-2.0.0/index.html
b/release/accumulo-2.0.0/index.html
index 988d554..0c5502b 100644
--- a/release/accumulo-2.0.0/index.html
+++ b/release/accumulo-2.0.0/index.html
@@ -157,6 +157,7 @@
<h2 id="major-changes">Major Changes</h2>
<h2 id="other-notable-changes">Other Notable Changes</h2>
+<p>ACCUMULO-3652 - Replaced string concatenation in log statements with slf4j
where applicable. Removed tserver TLevel logging class.</p>
<h2 id="upgrading">Upgrading</h2>
--
To stop receiving notification emails like this one, please contact
['"[email protected]" <[email protected]>'].
