For one thing, what does ReadStage's pending look like if you repeatedly run "nodetool tpstats" on these nodes? If you're simply bottlenecking on I/O on reads, that is the most easy and direct way to observe this empirically. If you're saturated, you'll see active close to maximum at all times, and pending racking up consistently. If you're just close, you'll likely see spikes sometimes.
-- / Peter Schuller (@scode, http://worldmodscode.wordpress.com)
