[
https://issues.apache.org/jira/browse/CASSANDRA-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605763#comment-14605763
]
Joshua McKenzie edited comment on CASSANDRA-9658 at 6/29/15 3:44 PM:
---------------------------------------------------------------------
[~iamaleksey]: My initial assumption is that getting buffered close to parity
w/mmap on Windows is going to be both much more programmer-hour intensive and
much more invasive than getting mmap stabilized on Windows in time for 2.2.x
stabilizing. I agree on the long-term goal of standardizing on a single read
path; I'll do some stress-testing today to get an initial read on how much pain
enabling mmap'ed I/O on Windows might cause us.
[~stefania_alborghetti]: I don't think 7066 will actually be necessary for us
(edit: specifically in this context of enabling mmap on Windows w/out access
violations, not saying 7066 isn't necessary :)) after CASSANDRA-8535 and then
CASSANDRA-8984, however I'll need to stress test the paths today to get a
better feel for it post 8984. Let's sit tight on these test results w/mmap on
Windows before taking any other steps to try and get buffered reads closer to
parity right now on account of this ticket.
was (Author: joshuamckenzie):
[~iamaleksey]: My initial assumption is that getting buffered close to parity
w/mmap on Windows is going to be both much more programmer-hour intensive and
much more invasive than getting mmap stabilized on Windows in time for 2.2.x
stabilizing. I agree on the long-term goal of standardizing on a single read
path; I'll do some stress-testing today to get an initial read on how much pain
enabling mmap'ed I/O on Windows might cause us.
[~stefania_alborghetti]: I don't think 7066 will actually be necessary for us
after CASSANDRA-8535 and then CASSANDRA-8984, however I'll need to stress test
the paths today to get a better feel for it post 8984. Let's sit tight on these
test results w/mmap on Windows before taking any other steps to try and get
buffered reads closer to parity right now on account of this ticket.
> Re-enable memory-mapped index file reads on Windows
> ---------------------------------------------------
>
> Key: CASSANDRA-9658
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9658
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Joshua McKenzie
> Assignee: Joshua McKenzie
> Labels: Windows, performance
> Fix For: 2.2.x
>
>
> It appears that the impact of buffered vs. memory-mapped index file reads has
> changed dramatically since last I tested. [Here's some results on various
> platforms we pulled together yesterday
> w/2.2-HEAD|https://docs.google.com/spreadsheets/d/1JaO2x7NsK4SSg_ZBqlfH0AwspGgIgFZ9wZ12fC4VZb0/edit#gid=0].
> TL;DR: On linux we see a 40% hit in performance from 108k ops/sec on reads to
> 64.8k ops/sec. While surprising in itself, the really unexpected result (to
> me) is on Windows - with standard access we're getting 16.8k ops/second on
> our bare-metal perf boxes vs. 184.7k ops/sec with memory-mapped index files,
> an over 10-fold increase in throughput. While testing w/standard access,
> CPU's on the stress machine and C* node are both sitting < 4%, network
> doesn't appear bottlenecked, resource monitor doesn't show anything
> interesting, and performance counters in the kernel show very little. Changes
> in thread count simply serve to increase median latency w/out impacting any
> other visible metric that we're measuring, so I'm at a loss as to why the
> disparity is so huge on the platform.
> The combination of my changes to get the 2.1 branch to behave on Windows
> along with [~benedict] and [~Stefania]'s changes in lifecycle and cleanup
> patterns on 2.2 should hopefully have us in a state where transitioning back
> to using memory-mapped I/O on Windows will only cause trouble on snapshot
> deletion. Fairly simple runs of stress w/compaction aren't popping up any
> obvious errors on file access or renaming - I'm going to do some much heavier
> testing (ccm multi-node clusters, long stress w/repair and compaction, etc)
> and see if there's any outstanding issues that need to be stamped out to call
> mmap'ed index files on Windows safe. The one thing we'll never be able to
> support is deletion of snapshots while a node is running and sstables are
> mapped, but for a > 10x throughput increase I think users would be willing to
> make that sacrifice.
> The combination of the powercfg profile change, the kernel timer resolution,
> and memory-mapped index files are giving some pretty interesting performance
> numbers on EC2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
