[
https://issues.apache.org/jira/browse/ACCUMULO-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169613#comment-15169613
]
ASF GitHub Bot commented on ACCUMULO-1755:
------------------------------------------
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/75#discussion_r54289506
--- Diff:
core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchWriter.java
---
@@ -699,7 +701,16 @@ else if (Tables.getTableState(context.getInstance(),
table) == TableState.OFFLIN
}
- void addMutations(MutationSet mutationsToSend) {
+ void queueMutations(final MutationSet mutationsToSend) {
+ queueThreadPool.submit(new Runnable() {
--- End diff --
I think it be simpler to use a producer/consumer model for the
`MutationSet`'s with a single consumer thread instead of constantly submitting
new `Runnable`'s. The consumer thread can essentially do the same thing: poll
for a `MutationSet`, and then run `addMutations` with that object. It
eliminates the need for another ExecutorService and repeatedly creating new
Runnables. We might also be able to implement some back-pressure on the client
via that Queue as well.
> BatchWriter blocks all addMutation calls while binning mutations
> ----------------------------------------------------------------
>
> Key: ACCUMULO-1755
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1755
> Project: Accumulo
> Issue Type: Improvement
> Components: client
> Reporter: Adam Fuchs
> Assignee: Dave Marion
> Fix For: 1.8.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Through code inspection, we found that the BatchWriter bins mutations inside
> of a synchronized block that covers calls to addMutation. Binning potentially
> involves lookups of tablet metadata and processes a fair amount of
> information. We will get better parallelism if we can either unlock the lock
> while binning, dedicate another thread to do the binning, or use one of the
> send threads to do the binning.
> This has not been verified empirically yet, so there is not yet any profiling
> info to indicate the level of improvement that we should expect. Profiling
> and repeatable demonstration of this performance bottleneck should be the
> first step on this ticket.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
