Christopher Tubbs created ACCUMULO-4146:
-------------------------------------------
Summary: Per-durability write-ahead logs
Key: ACCUMULO-4146
URL: https://issues.apache.org/jira/browse/ACCUMULO-4146
Project: Accumulo
Issue Type: Improvement
Reporter: Christopher Tubbs
Fix For: 1.8.0
[~kturner] showed me a scenario with Continuous Ingest, where the system was
configured to use "flush" for it's durability, but the Metadata table was
configured to use "sync" (default for metadata). When the system grew large
enough to have many tablets, lots of tablets were constantly writing to the
metadata table on a relatively few number of tablet servers. This caused all
ingest to drop to 0 very frequently as the tables waiting on a "flush" had to
wait for the WAL to "sync".
This problem is primarily caused by the fact that we have only one WAL per
tablet server, and so tablet writes waiting on a flush had to wait on a sync if
any concurrent write to the tablet server required a sync.
We could possibly alleviate this problem if we permitted tablet servers to have
multiple WALs open. One potentially good way to manage these multiple WALs is
to group them by durability, so there'd be one WAL for sync, and another for
flush. That way, writes requiring a flush would not wait on sync's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
