Re: BatchWriter performance on 1.4

2013-09-20 Thread John Vines
for it there? ** ** Thanks, David ** ** *From:* Keith Turner [mailto:ke...@deenlo.com] *Sent:* Thursday, September 19, 2013 7:01 PM *To:* user@accumulo.apache.org *Subject:* Re: BatchWriter performance on 1.4 ** ** On Thu, Sep 19, 2013 at 5:08 PM, Slater, David M. david.sla...@jhuapl.edu wrote

Re: BatchWriter performance on 1.4

2013-09-19 Thread Adam Fuchs
The addMutations method blocks when the client-side buffer fills up, so you may see a lot of time spent in that method due to a bottleneck downstream. There are a number of things you could try to speed that up. Here are a few: 1. Increase the BatchWriter's buffer size. This can smooth out the

RE: BatchWriter performance on 1.4

2013-09-19 Thread Slater, David M.
be nice to have a different thread handle the ingest for each BatchWriter, so I might try that out. From: David Medinets [mailto:david.medin...@gmail.com] Sent: Wednesday, September 18, 2013 10:41 PM To: accumulo-user Subject: Re: BatchWriter performance on 1.4 Have you looked at generating

Re: BatchWriter performance on 1.4

2013-09-19 Thread Keith Turner
Are you aware of the multi table batch writer? I am not sure if it would be useful, but wanted to make sure you knew about it. It will use the same thread pool to process mutations for multiple tables. Also it will batch mutations for multiple tablets into the same rpc calls. On Wed, Sep 18,

Re: BatchWriter performance on 1.4

2013-09-19 Thread Keith Turner
for you? Keith ** ** *From:* Keith Turner [mailto:ke...@deenlo.com] *Sent:* Thursday, September 19, 2013 12:39 PM *To:* user@accumulo.apache.org *Subject:* Re: BatchWriter performance on 1.4 ** ** Are you aware of the multi table batch writer? I am not sure if it would be useful