[
https://issues.apache.org/jira/browse/ACCUMULO-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169342#comment-15169342
]
ASF GitHub Bot commented on ACCUMULO-1755:
------------------------------------------
Github user keith-turner commented on the pull request:
https://github.com/apache/accumulo/pull/75#issuecomment-189376744
I think this change will result in a RPC per thread using the batch writer.
If 3 threads add mutations for the same tablet server, then it will no
longer send all of those in one RPC. If 100 threads add mutations for 100
tablets server, the batch writer will make ~10,000 RPCs instead of ~100.
I have some idea on how to improve the batch writer based on what I learned
while writing the conditional writer, I will try to write those up. It may or
may not be useful.
> BatchWriter blocks all addMutation calls while binning mutations
> ----------------------------------------------------------------
>
> Key: ACCUMULO-1755
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1755
> Project: Accumulo
> Issue Type: Improvement
> Components: client
> Reporter: Adam Fuchs
> Assignee: Dave Marion
> Fix For: 1.8.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Through code inspection, we found that the BatchWriter bins mutations inside
> of a synchronized block that covers calls to addMutation. Binning potentially
> involves lookups of tablet metadata and processes a fair amount of
> information. We will get better parallelism if we can either unlock the lock
> while binning, dedicate another thread to do the binning, or use one of the
> send threads to do the binning.
> This has not been verified empirically yet, so there is not yet any profiling
> info to indicate the level of improvement that we should expect. Profiling
> and repeatable demonstration of this performance bottleneck should be the
> first step on this ticket.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
