Hello Leonardo, In Cassandra, updates are written to the memory and they are flushed to disks time to time. As a result, most of the time writes happen in the order milliseconds. However, the read latency is highly dependent on the I/O subsystem. If you have enough disk bandwidth (at least two disks to saturate a single core) then achieving a perfectly tuned system is possible after putting a little effort.
Using jconsole to monitor the server is a good way to understand its internals. Regarding the system-level tuning, I would first scale down the dataset size to something that can fit in to your memory. Ideally you should be able to saturate your cores once the data is cached in your memory. Then you can gradually increase your dataset size and monitor the system behavior (e.g., I/O, network). Also here are some of the common knowledge we have gained so far: 1- OS file cache is very important for Cassandra. So reserve at least the half of your memory for the OS file cache. 2- Use a separate disk for commit logs. 3- Increase the number of read threads to overlap the I/O latency (will increase the latency and throughput at the same time). 4- Refrain overloading the machine. Utilizing the server more than 70-80%, triggers major GC frequently and server becomes unresponsive . 5- If the compaction kicks in all the time, you can disable it by changing the attribute specified in ColumnFamily. Compaction eats a lot of I/O bandwidth and real world deployments of Cassandra (e.g., Netflix) do it offline. BTW, You can also check the YCSB paper for the typical latency requirements (http://research.yahoo.com/node/3202). Please let me know if you need any further help, Onur ________________________________________ From: Leonardo Piga [[email protected]] Sent: Monday, July 02, 2012 6:09 PM To: [email protected] Subject: Hints for tuning Data Serving Hi guys, Do you have any hints for tuning Data Serving benchmark? What is the typical latency requirements? How do you run the benchmark to saturate the machine? Many thanks, Leonardo
