[ https://issues.apache.org/jira/browse/ACCUMULO-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170196#comment-15170196 ]
ASF GitHub Bot commented on ACCUMULO-1755: ------------------------------------------ Github user dlmarion commented on a diff in the pull request: https://github.com/apache/accumulo/pull/75#discussion_r54321264 --- Diff: core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchWriter.java --- @@ -638,6 +648,21 @@ public MutationWriter(int numSendThreads) { queued = new HashSet<String>(); sendThreadPool = new SimpleThreadPool(numSendThreads, this.getClass().getName()); locators = new HashMap<String,TabletLocator>(); + binTimer.schedule(new TimerTask() { + @Override + public void run() { + MutationSet m = queue.poll(); + while (null != m) { + try { + addMutations(m); + } catch (Exception e) { + updateUnknownErrors("Error processing mutation set", e); + } + m = queue.poll(); + } + } + }, 0, 500); --- End diff -- Well, in the previous revision I was adding a MutationSet to an ExecutorService that had an unbounded queue and 1 thread. That's not immediate. Can MutationSets be binned concurrently? If so, then I can change the timer from a fixed delay to fixed rate. Thoughts? > BatchWriter blocks all addMutation calls while binning mutations > ---------------------------------------------------------------- > > Key: ACCUMULO-1755 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1755 > Project: Accumulo > Issue Type: Improvement > Components: client > Reporter: Adam Fuchs > Assignee: Dave Marion > Fix For: 1.8.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Through code inspection, we found that the BatchWriter bins mutations inside > of a synchronized block that covers calls to addMutation. Binning potentially > involves lookups of tablet metadata and processes a fair amount of > information. We will get better parallelism if we can either unlock the lock > while binning, dedicate another thread to do the binning, or use one of the > send threads to do the binning. > This has not been verified empirically yet, so there is not yet any profiling > info to indicate the level of improvement that we should expect. Profiling > and repeatable demonstration of this performance bottleneck should be the > first step on this ticket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)