[
https://issues.apache.org/jira/browse/ACCUMULO-931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13545949#comment-13545949
]
Jeremy Kepner commented on ACCUMULO-931:
----------------------------------------
So the phenomena appears to be that mutations can return control to the
ingestor before the minor compactions they cause have completed. Thus a single
ingestor can cause multiple simultaneous compactions on the same tablet. If
the table only has one tablet this cause the ingest process back up (see file
1ingestor_1table_1tablet.pdf). The solution is to presplit the tablet. The
files 1ingestor_1table_2tablet.pdf and 1ingestor_1table_4tablet.pdf shows that
these splits solves the problem. The file 2,3,4ingestor_1table_4tablet.pdf
show the performance of three separate runs using 2, 3, and 4 ingestors into a
table with 4 tablets.. Likewise, the file
10,r8,r10,r12ingestor_1table_12tablet.pdf shows four separate runs using 10
local, 8 remote, 10 remote, and 12 remote ingestors. In all cases, the
splitting resolves the performance issue.
If there are no objections, this issue can be closed.
> Oscillations in Accumulo Ingest Performance
> -------------------------------------------
>
> Key: ACCUMULO-931
> URL: https://issues.apache.org/jira/browse/ACCUMULO-931
> Project: Accumulo
> Issue Type: Bug
> Affects Versions: 1.4.2
> Environment:
> Reporter: Jeremy Kepner
> Attachments: 1ingestor_1table_1tablet.pdf, 1ingestor.pdf,
> 2ingestor.pdf, 3ingestor.pdf, 4ingestor.pdf, ingest_performance_explained.pdf
>
>
> Linux 2.6.32, single node, 32 cores, 96 GB RAM, 3x3TB SATA drives, RAID5
> Ingest performance into accumulo varies by 2.5x. Depending upon the number
> of ingestors.
> Performance tests were carried out using Graph500 benchmark (see
> d4m_api/examples/3Scaling/2ParallelDatabase/pDB10_EdgeInsertTEST.m from
> http://www.mit.edu/~kepner/D4M/).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
