[
https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13788501#comment-13788501
]
Benedict commented on CASSANDRA-4718:
-------------------------------------
Not necessarily. I still think that was most likely variance:
- I have BAQ at same speed as LBQ in application
- a 2x slow down of LBQ -> 0.01x slow down of application
- a 10x slow down of LBQ -> 0.05x slow down of application
=> the queue speed is currently only ~1% of application cost. It's possible the
faster queue is causing greater contention at a sync point, but this wouldn't
work in the opposite direction if the contention at the sync point is low.
Either way, if this were true we'd see the artificially slow queues also
improve stress performance.
Ryan also ran some of my tests and found no difference. I wouldn't absolutely
rule out the possibility his test was valid, though, as I did not swap out the
queues in OutboundTcpConnection for these tests as, at the time, I was
concerned about the calls to size() which are expensive for my test queues, and
I wanted the queue swap to be on equal terms across the board. I realise now
these are only called via JMX, so shouldn't stop me swapping them in.
I've just tried a quick test of directly (in process) stressing through the
MessagingService and found no measureable difference to putting BAQ in the
OutboundTcpConnection, though if I swap out across the board it is about 25%
slower, which itself is interesting as this is close to a full stress, minus
thrift.
> More-efficient ExecutorService for improved throughput
> ------------------------------------------------------
>
> Key: CASSANDRA-4718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4718
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Jonathan Ellis
> Assignee: Jason Brown
> Priority: Minor
> Labels: performance
> Attachments: baq vs trunk.png, op costs of various queues.ods,
> PerThreadQueue.java, stress op rate with various queues.ods
>
>
> Currently all our execution stages dequeue tasks one at a time. This can
> result in contention between producers and consumers (although we do our best
> to minimize this by using LinkedBlockingQueue).
> One approach to mitigating this would be to make consumer threads do more
> work in "bulk" instead of just one task per dequeue. (Producer threads tend
> to be single-task oriented by nature, so I don't see an equivalent
> opportunity there.)
> BlockingQueue has a drainTo(collection, int) method that would be perfect for
> this. However, no ExecutorService in the jdk supports using drainTo, nor
> could I google one.
> What I would like to do here is create just such a beast and wire it into (at
> least) the write and read stages. (Other possible candidates for such an
> optimization, such as the CommitLog and OutboundTCPConnection, are not
> ExecutorService-based and will need to be one-offs.)
> AbstractExecutorService may be useful. The implementations of
> ICommitLogExecutorService may also be useful. (Despite the name these are not
> actual ExecutorServices, although they share the most important properties of
> one.)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
