[
https://issues.apache.org/jira/browse/ACCUMULO-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170202#comment-15170202
]
ASF GitHub Bot commented on ACCUMULO-1755:
------------------------------------------
Github user keith-turner commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/75#discussion_r54321821
--- Diff:
core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchWriter.java
---
@@ -638,6 +648,21 @@ public MutationWriter(int numSendThreads) {
queued = new HashSet<String>();
sendThreadPool = new SimpleThreadPool(numSendThreads,
this.getClass().getName());
locators = new HashMap<String,TabletLocator>();
+ binTimer.schedule(new TimerTask() {
+ @Override
+ public void run() {
+ MutationSet m = queue.poll();
+ while (null != m) {
+ try {
+ addMutations(m);
+ } catch (Exception e) {
+ updateUnknownErrors("Error processing mutation set", e);
+ }
+ m = queue.poll();
+ }
+ }
+ }, 0, 500);
--- End diff --
> Well, in the previous revision I was adding a MutationSet to an
ExecutorService that had an unbounded queue and 1 thread. That's not immediate
Yeah, its certainly not immediate. However, it probably much less than
1ms between adding to thread pool and thread calling run() on Runnable (when
thread is idle). Relative to ~250ms, its like kinda immediate.
> Can MutationSets be binned concurrently?
I have no idea. Could not say yes w/o much more review of the code than I
want to do late on a Fri afternoon :). I suspect something would blow up.
> BatchWriter blocks all addMutation calls while binning mutations
> ----------------------------------------------------------------
>
> Key: ACCUMULO-1755
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1755
> Project: Accumulo
> Issue Type: Improvement
> Components: client
> Reporter: Adam Fuchs
> Assignee: Dave Marion
> Fix For: 1.8.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Through code inspection, we found that the BatchWriter bins mutations inside
> of a synchronized block that covers calls to addMutation. Binning potentially
> involves lookups of tablet metadata and processes a fair amount of
> information. We will get better parallelism if we can either unlock the lock
> while binning, dedicate another thread to do the binning, or use one of the
> send threads to do the binning.
> This has not been verified empirically yet, so there is not yet any profiling
> info to indicate the level of improvement that we should expect. Profiling
> and repeatable demonstration of this performance bottleneck should be the
> first step on this ticket.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
