Re: get_slice slow

2010-08-24 Thread Artie Copeland
Have you tried using a super column, it seems that having a row with over 100K columns and growing would be alot for cassandra to deserialize? what is iostat and jmeter telling you? it would be interesting to see that data. also what are you using for you key or row caching? do you need to use a

script to manually generate tokens

2010-08-22 Thread Artie Copeland
i created a simple python script to ask for a cluster size and then generate tokens for each node http://github.com/yestech/yessandbox/blob/master/cassandra-gen-tokens.py it is derived from ben black's cassandra talk: http://www.slideshare.net/benjaminblack/cassandra-summit-2010-operations-troub

cache sizes using percentages

2010-08-17 Thread Artie Copeland
if i set a key cache size of 100% the way i understand how that works is: - the cache is not write through, but read through - a key gets added to the cache on the first read if not already available - the size of the cache will always increase for ever item read. so if you have 100mil items your

move data between clusters

2010-08-17 Thread Artie Copeland
what is the best way to move data between clusters. we currently have a 4 node prod cluster with 80G of data and want to move it to a dev env with 3 nodes. we have plenty of disk were looking into nodetool snapshot, but it look like that wont work because of the system tables. sstabletojson does

Re: backport of pre cache load

2010-08-09 Thread Artie Copeland
being commodity. Our data set is currently over 100GB. On Fri, Aug 6, 2010 at 5:54 PM, Jonathan Ellis wrote: > are you caching 100% of the CF? > > if not this is not super useful. > > On Fri, Aug 6, 2010 at 7:10 PM, Artie Copeland > wrote: > > would it be possible to

Re: row cache during bootstrap

2010-08-09 Thread Artie Copeland
Are you adding nodes often? > Currently not that often. The main issue is we have very stringent latency requirements and anything that would affect those we have to understand the worst case cost to see if we can avoid them. > > Aaron > > On 7 Aug 2010, at 11:17, Artie Cope

row cache during bootstrap

2010-08-06 Thread Artie Copeland
the way i understand how row caches work is that each node has an independent cache, in that they do not push there cache contents with other nodes. if that the case is it also true that when a new node is added to the cluster it has to build up its own cache. if thats the case i see that as a po

backport of pre cache load

2010-08-06 Thread Artie Copeland
would it be possible to backport the 0.7 feature, the ability to safe and preload row caches after a restart. i think that is a very nice and important feature that would help users with very large caches, that take a long time to get the proper hot set. for example we can get pretty good cache r

Re: when should new nodes be added to a cluster

2010-08-03 Thread Artie Copeland
t; bandwidth, regardless. > > > b > > On Mon, Aug 2, 2010 at 11:47 AM, Artie Copeland > wrote: > > i have a question on what are the signs from cassandra that new nodes > should > > be added to the cluster. We are currently seeing long read times from > the &g

Re: when should new nodes be added to a cluster

2010-08-02 Thread Artie Copeland
and 1TB (sda) for commit log Anything else i can provide that might help diagnose. Thanx Artie > > Aaron > > > On 03 Aug, 2010,at 06:47 AM, Artie Copeland > wrote: > > i have a question on what are the signs from cassandra that new nodes > should be added to the cluster

when should new nodes be added to a cluster

2010-08-02 Thread Artie Copeland
i have a question on what are the signs from cassandra that new nodes should be added to the cluster. We are currently seeing long read times from the one node that has about 70GB of data with 60GB in one column family. we are using a replication factor of 3. I have tracked down the slow to occu

Re: live nodes list in ring

2010-07-13 Thread Artie Copeland
Benjamin, Yes i have seen this when adding a new node into the cluster. the new node doesnt see the complete ring through nodetool, but the strange part is that looking at the ring through jconsole shows the complete ring. it as if there is a big in nodetool publishing the actual ring. has anyo