[
https://issues.apache.org/jira/browse/ACCUMULO-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110880#comment-15110880
]
Eric Newton commented on ACCUMULO-4112:
---------------------------------------
I ran another set of experiments, trying to determine the basic costs of SYNC
over LOG Durabilities for lots of MinCs.
The main determining factor for speed was using a "reasonable" IMM. Using 100k
IMM for 10 tablets (plus the metadata tablets), leaves less than 10k per
tablet. By simply increasing the IMM to 200k, performance is improved (and,
probably fewer MinCs occurred).
So, I've been unable to reproduce the issue that [~kturner] experienced with
Fluo in a small test. But I have learned that there is a bare minimum IMM size
required for a given number of tablets. And that's 10-20K for each hosted
tablet. These are easily affordable on any modern computer even with 1000
tablets.
I've attached an image with the numbers (sorry, but that's easier than trying
to format the wiki markup).
> MinC start/stop updates are always hsync'd
> ------------------------------------------
>
> Key: ACCUMULO-4112
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4112
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Environment: Fluo testing on a 20-node cluster
> Reporter: Eric Newton
> Assignee: Eric Newton
> Priority: Minor
> Attachments: HSyncOverheadExperiment.png, MinCFlushPerfTest.java,
> Sync-Flush-Log-Performance.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> [~kturner] writes:
> {quote}
> I was running a Fluo test with 1.8.0-SNAP on my workstation. My Fluo table
> had a ton of tablets. I was seeing terrible performance. I started
> looking at the tserver and noticed it was always calling hsync. I tracked
> down the problem to the fact that when minc start and stop events are written
> to the log they are always written w/ sync level. My poor little tserver
> was constantly minor compacting (probably had around 600 tablets that were
> all being written to).
> I changed the test config to create like 15 tablets and the performance was
> much better. All cores were 100% utilized, which was not the case when hsync
> was always called.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
