i've noticed the longer i let my test clients run the higher the read latency
becomes. if i kill the clients, the latency drops back down to a reasonable
value. write latency isn't affected. here are two cfstats listings from the
same machine, the first before and the second after i killed the clients. i do
not touch the servers. this seems odd. if the clients are doing some bad, it
seems that the server's latency wouldn't be affected. is this a thrift issue?
before
---------
Keyspace: uds
Read Count: 2003
Read Latency: 59.807 ms.
Write Count: 65145
Write Latency: 0.047 ms.
Pending Tasks: 0
Column Family: bucket
Memtable Columns Count: 168751
Memtable Data Size: 351695390
Memtable Switch Count: 94
Read Count: 2003
Read Latency: 59.807 ms.
Write Count: 65176
Write Latency: 0.047 ms.
Pending Tasks: 0
after
-------
Keyspace: uds
Read Count: 53249
Read Latency: 3.653 ms.
Write Count: 52888
Write Latency: 0.035 ms.
Pending Tasks: 0
Column Family: bucket
Memtable Columns Count: 383297
Memtable Data Size: 798864312
Memtable Switch Count: 94
Read Count: 53271
Read Latency: 3.649 ms.
Write Count: 52937
Write Latency: 0.035 ms.
Pending Tasks: 0
