[
https://issues.apache.org/jira/browse/ACCUMULO-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169390#comment-15169390
]
ASF GitHub Bot commented on ACCUMULO-1755:
------------------------------------------
Github user keith-turner commented on the pull request:
https://github.com/apache/accumulo/pull/75#issuecomment-189385925
> The set of mutations is still global,
OIC. Your right, its not as bad as I thought. Although there is still
problem. With the per thread mutation writers, mutations bound for the same
tserver may never be merged in some cases. This situations are less likely to
occur though. I am thinking of something like the following.
* Thread 1 bins 1000 mutations for 20 tservers
* 3 tserver queues are processed by send threads
* Thread 2 bins 1000 mutations for 20 tserver (at this point 17 tservers
from the 1st binning are not processed by send threads, those mutations are not
merged together).
This makes me wonder, why have a mutation writer per thread? Why not just
add to the mutation writer outside of the sync?
> BatchWriter blocks all addMutation calls while binning mutations
> ----------------------------------------------------------------
>
> Key: ACCUMULO-1755
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1755
> Project: Accumulo
> Issue Type: Improvement
> Components: client
> Reporter: Adam Fuchs
> Assignee: Dave Marion
> Fix For: 1.8.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Through code inspection, we found that the BatchWriter bins mutations inside
> of a synchronized block that covers calls to addMutation. Binning potentially
> involves lookups of tablet metadata and processes a fair amount of
> information. We will get better parallelism if we can either unlock the lock
> while binning, dedicate another thread to do the binning, or use one of the
> send threads to do the binning.
> This has not been verified empirically yet, so there is not yet any profiling
> info to indicate the level of improvement that we should expect. Profiling
> and repeatable demonstration of this performance bottleneck should be the
> first step on this ticket.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
