[
https://issues.apache.org/jira/browse/CASSANDRA-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969827#action_12969827
]
Peter Schuller commented on CASSANDRA-1470:
-------------------------------------------
Note that at minimum the amount written per direct write has to be high enough
that the seek overhead of constituent disks becomes irrelevant (and keeping in
mind that there may be multiple spindles under a RAID controller). In addition
on a loaded system there will be additional latency resulting from the I/O
requests queueing, which increases the demands for the size of individual
writes. I would expect a sensible "chunk" size to be several megabyte at
minimum for writes.
The other trade-off for reads (and writes if not using battery-backed write
caching in a RAID controller) is that too large "chunk:s" will be detrimental
to latency of live traffic. I exclude writes on battery backed controllers
under the assumption that the chunk size is well below cache size in the
controller. However, on the other hand there is a throttling issue here so
unless rate limiting is applied you would see detrimental affects with a RAID
controller too since it is going to eat your writes until it becomes full, at
which point submission of a huge write will probably again have the detrimental
effects on latency.
> use direct io for compaction
> ----------------------------
>
> Key: CASSANDRA-1470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1470
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Pavel Yaskevich
> Fix For: 0.7.1
>
> Attachments: 1470-v2.txt, 1470.txt, CASSANDRA-1470-for-0.6.patch,
> CASSANDRA-1470-v10-for-0.7.patch, CASSANDRA-1470-v11-for-0.7.patch,
> CASSANDRA-1470-v12-0.7.patch, CASSANDRA-1470-v2.patch,
> CASSANDRA-1470-v3-0.7-with-LastErrorException-support.patch,
> CASSANDRA-1470-v4-for-0.7.patch, CASSANDRA-1470-v5-for-0.7.patch,
> CASSANDRA-1470-v6-for-0.7.patch, CASSANDRA-1470-v7-for-0.7.patch,
> CASSANDRA-1470-v8-for-0.7.patch, CASSANDRA-1470-v9-for-0.7.patch,
> CASSANDRA-1470.patch,
> use.DirectIORandomAccessFile.for.commitlog.against.1022235.patch
>
>
> When compaction scans through a group of sstables, it forces the data in the
> os buffer cache being used for hot reads, which can have a dramatic negative
> effect on performance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
