[
https://issues.apache.org/jira/browse/CASSANDRA-8894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14726544#comment-14726544
]
Albert P Tobey commented on CASSANDRA-8894:
-------------------------------------------
Sorry I'm late to the thread, but the recently added patch seems very
overcomplicated for little benefit.
I noticed the massive over-read in 2.1 and tracked it down independently by
profiling with disk_access_mode: standard. I then ran a build with the
DEFAULT_BUFFER_SIZE at 4K and saw an instant 4x increase in TXN/s on a simple
-stress test with a 60% reduction in wasted disk IO. This over-read is causing
performance problems on every Cassandra 2.1 cluster that isn't 100% writes. It
doesn't always show up because of the massive amount of RAM a lot of folks are
running, but under low memory situations it is killing even very fast SSDs.
Patch for 2.1: https://gist.github.com/tobert/10c307cf3709a585a7cf
In reading through the history, I think this is being overthought. If anything,
the readahead and buffering in the read path should be *removed* and instead
issue precise reads wherever it's possible. For now, the change to a 4K buffer
size should be added to 2.1 in order to significantly speed up read workloads.
> Our default buffer size for (uncompressed) buffered reads should be smaller,
> and based on the expected record size
> ------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-8894
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8894
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Benedict
> Assignee: Stefania
> Labels: benedict-to-commit
> Fix For: 3.0 alpha 1
>
> Attachments: 8894_25pct.yaml, 8894_5pct.yaml, 8894_tiny.yaml
>
>
> A large contributor to slower buffered reads than mmapped is likely that we
> read a full 64Kb at once, when average record sizes may be as low as 140
> bytes on our stress tests. The TLB has only 128 entries on a modern core, and
> each read will touch 32 of these, meaning we are unlikely to almost ever be
> hitting the TLB, and will be incurring at least 30 unnecessary misses each
> time (as well as the other costs of larger than necessary accesses). When
> working with an SSD there is little to no benefit reading more than 4Kb at
> once, and in either case reading more data than we need is wasteful. So, I
> propose selecting a buffer size that is the next larger power of 2 than our
> average record size (with a minimum of 4Kb), so that we expect to read in one
> operation. I also propose that we create a pool of these buffers up-front,
> and that we ensure they are all exactly aligned to a virtual page, so that
> the source and target operations each touch exactly one virtual page per 4Kb
> of expected record size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
