[
https://issues.apache.org/jira/browse/ACCUMULO-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170162#comment-15170162
]
ASF GitHub Bot commented on ACCUMULO-1755:
------------------------------------------
Github user keith-turner commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/75#discussion_r54319650
--- Diff:
core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchWriter.java
---
@@ -638,6 +648,21 @@ public MutationWriter(int numSendThreads) {
queued = new HashSet<String>();
sendThreadPool = new SimpleThreadPool(numSendThreads,
this.getClass().getName());
locators = new HashMap<String,TabletLocator>();
+ binTimer.schedule(new TimerTask() {
+ @Override
+ public void run() {
+ MutationSet m = queue.poll();
+ while (null != m) {
+ try {
+ addMutations(m);
+ } catch (Exception e) {
+ updateUnknownErrors("Error processing mutation set", e);
+ }
+ m = queue.poll();
+ }
+ }
+ }, 0, 500);
--- End diff --
Does this mean every 500ms check for something on the queue to bin? Whats
the rational for this?
> BatchWriter blocks all addMutation calls while binning mutations
> ----------------------------------------------------------------
>
> Key: ACCUMULO-1755
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1755
> Project: Accumulo
> Issue Type: Improvement
> Components: client
> Reporter: Adam Fuchs
> Assignee: Dave Marion
> Fix For: 1.8.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Through code inspection, we found that the BatchWriter bins mutations inside
> of a synchronized block that covers calls to addMutation. Binning potentially
> involves lookups of tablet metadata and processes a fair amount of
> information. We will get better parallelism if we can either unlock the lock
> while binning, dedicate another thread to do the binning, or use one of the
> send threads to do the binning.
> This has not been verified empirically yet, so there is not yet any profiling
> info to indicate the level of improvement that we should expect. Profiling
> and repeatable demonstration of this performance bottleneck should be the
> first step on this ticket.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
