[
https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14000816#comment-14000816
]
Pavel Yaskevich commented on CASSANDRA-4718:
--------------------------------------------
I think you have plotted numbers from May16. I'm not sure what do you mean by
"often" the problem with numbers is that they apparently cut off for both
branches :( We have to redo the test I think, [~jasobrown] is there a way to
guarantee that both branches are going to do the same number of runs with
stress? I disagree with #2 because sep shows a sudden drop at the end as do
runs 1 and 3 so we don't really know what is going to happen with sep on the
high stress concurrency in those runs.
bq. This work is clearly disk bound as the same hardware was pushing 250k/s
with similar record sizes when exclusively in memory - we're seeing only 5% of
that now. Unless possibly in-memory index scans are occupying all of the time
(but according to Jason CPU utilisation was around 30% from a random
non-scientific poll).
I'm not sure if we can count 250K/s as a disk bound workload, which is only 3
buffer reads per second.
> More-efficient ExecutorService for improved throughput
> ------------------------------------------------------
>
> Key: CASSANDRA-4718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4718
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Jonathan Ellis
> Assignee: Benedict
> Priority: Minor
> Labels: performance
> Fix For: 2.1.0
>
> Attachments: 4718-v1.patch, PerThreadQueue.java,
> austin_diskbound_read.svg, aws.svg, aws_read.svg,
> backpressure-stress.out.txt, baq vs trunk.png,
> belliotsmith_branches-stress.out.txt, jason_read.svg, jason_read_latency.svg,
> jason_run1.svg, jason_run2.svg, jason_run3.svg, jason_write.svg, op costs of
> various queues.ods, stress op rate with various queues.ods,
> stress_2014May15.txt, stress_2014May16.txt, v1-stress.out
>
>
> Currently all our execution stages dequeue tasks one at a time. This can
> result in contention between producers and consumers (although we do our best
> to minimize this by using LinkedBlockingQueue).
> One approach to mitigating this would be to make consumer threads do more
> work in "bulk" instead of just one task per dequeue. (Producer threads tend
> to be single-task oriented by nature, so I don't see an equivalent
> opportunity there.)
> BlockingQueue has a drainTo(collection, int) method that would be perfect for
> this. However, no ExecutorService in the jdk supports using drainTo, nor
> could I google one.
> What I would like to do here is create just such a beast and wire it into (at
> least) the write and read stages. (Other possible candidates for such an
> optimization, such as the CommitLog and OutboundTCPConnection, are not
> ExecutorService-based and will need to be one-offs.)
> AbstractExecutorService may be useful. The implementations of
> ICommitLogExecutorService may also be useful. (Despite the name these are not
> actual ExecutorServices, although they share the most important properties of
> one.)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
