Try increasing tserver.mutation.queue.max and see if that helps.
See ACCUMULO-1905 and ACCUMULO-1950
On Wed, Dec 4, 2013 at 5:14 AM, Peter Tillotson slatem...@yahoo.co.ukwrote:
I've been trying to get the most out of streaming data into Accumulo 1.5
(Hadoop Cloudera CDH4). Having tried a
One thing you can do is reduce the replication factor for the WAL. We have
found that makes a pretty significant different in write performance. That
can be modified with the tserver.wal.replication property. Setting it to 2
instead of the default (probably 3) should give you some performance
Peter --
I don't know if this was made entirely clear.
The reason that things are much slower when you have the WAL turned on
is that you're suddenly writing N extra copies of your data to disk.
When you don't have the WAL turned on, you're simply writing to
Accumulo's in-memory data
How many concurrent writers do you have? I made some other comments below
inline.
On Wed, Dec 4, 2013 at 10:53 AM, Peter Tillotson slatem...@yahoo.co.ukwrote:
Keith
I tried tserver.mutation.queue.max=4M and it improved but by no where near
a significant difference. I my app records get
I've 3 tables, each with a BatchWriter splitting 16M buffers across 8 threads.
So up to 24 peak concurrent write threads, normally of order 10 actually
concurrent. I'm not too worried for the moment increasing mutation.queue.max
feels like an unsustainable workaround so I'm better off dumping
