I found the issue. Stupid mistake.
Although I have 1000000 customers in the cluster, I do a get_range_slices to get the first 10000 customers. I use these customers as a seed for my test program. I random select customers from that set and fire requests at the cluster. Those 10000 customers all reside on the first node (actually the first three nodes because of the replication factor of three). Thanks, Rene 2013/6/28 Rene Kochen <rene.koc...@schange.com> > Hi All, > > I have the following situation: > > - Cassandra 1.0.11 > - A 6 node cluster > - Random partitioner > - Tokens are balanced (according to node-tool) > - Data-load is balanced (according to node-tool) > > I have a customers column-family with 1000000 customers. I also have a > test client which requests random customers from the cluster in a > round-robin fashion, i.e. each request is targeted to a different machine > and each request contains a random customer-id (a number in the range of 0 > - 1000000). > > What I see is that one node (node 4) is only proxying. Node 4 does receive > requests. I can see that in the storage-proxy counters. However, I see in > the Customers column-family a zero in the ReadCount counter (local reads). > Node 4 should also serve reads because I have 1000000 customers in the > cluster, the cluster is balanced and I do requests with random > customer-ids. In fact, two of the four nodes have this behavior. The result > is that the processing load within the cluster is not balanced. > > I have never witnessed this behavior before. > > Am I missing something? > > Thanks, > > Rene >
