Re: Routine nodetool repair

2011-12-21 Thread Peter Schuller
> Could the lack of routine repair be why nodetool ring reports: node(1) Load > -> 78.24 MB and node(2) Load -> 67.21 MB? The load span between the two > nodes has been increasing ever so slowly... No. Generally there will be a variation in load depending on what state compaction happens to be in

Monitoring move progress

2011-12-21 Thread Ethan Rowe
I've got some nodes in a "moving" state in a cluster (the nodes to which they stream shouldn't overlap), and I'm finding it difficult to determine if they're actually doing anything related to the move at this point, or if they're stuck in the state and not actually doing anything. In each case, I

Re: Routine nodetool repair

2011-12-21 Thread Blake Starkenburg
Output from nodetool ring: Address DC RackStatus State Load OwnsToken 85070591730234615865843651857942052864 110.82.155.2 datacenter1 rack1 Up Normal 78.23 MB 50.00% 0 110.82.155.4 datacenter1 rack1 Up Normal 67.21 MB 50.00% 8507059173023

Re: Handling topology changes

2011-12-21 Thread Pierre-Yves Ritschard
A couple of nodes per month, but with peaks. I will test the nodetool move based scenario then. Cheers, - pyr On Wed, Dec 21, 2011 at 10:10 PM, aaron morton wrote: > How often is "relatively often" ? > > * having a fixed amount of nodes with initial tokens and letting new >  ones auto bootstr

Re: Counter read requests spread across replicas ?

2011-12-21 Thread Philippe
along the same line of the last experimient I did (cluster is only being updated by a single threaded batching processing.) All nodes are the same hardware & configuration. Why on earth would one node require disk IO and not the 2 replicas ? Primary replica show some disk activity (iostat shows ab

Re: Counter read requests spread across replicas ?

2011-12-21 Thread Philippe
Hi Aaron, >How many rows are you asking for in the multget_slice and what thread pools are showing pending tasks ? I am querying in batches of 256 keys max. Each batch may slice between 1 and 5 explicit super columns (I need all the columns in each super column, there are at the very most a couple

Re: Routine nodetool repair

2011-12-21 Thread aaron morton
Post the output from nodetool ring and take a look at http://wiki.apache.org/cassandra/Operations#Token_selection Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 22/12/2011, at 5:21 AM, Blake Starkenburg wrote: > Thank You! > > Could the

Re: Doubts related to composite type column names/values

2011-12-21 Thread aaron morton
Keys are sorted by their token, when using the RandomPartitioner this is a MD5 hash. So they are essentially randomly sorted. I would use CompositeTypes as keys if they make sense for your app. e.g. you are storing time series data and the row key is the time stamp and the length of the time

Re: Handling topology changes

2011-12-21 Thread aaron morton
How often is "relatively often" ? > * having a fixed amount of nodes with initial tokens and letting new > ones auto bootstrap themselves Generally a bad idea, you should make sure nodes re given sensible tokens that evenly distribute the data. Cheers - Aaron Morton Freelan

Re: Choosing a Partitioner Type for Random java.util.UUID Row Keys

2011-12-21 Thread aaron morton
AFAIK there are no plans kill the BOP, but I would still try to make your life easier by using the RP. . My understanding of the problem is at certain times you snapshot the files in a dir; and the main query you want to handle is "At what points between time t0 and time t1 did files x,y and z

Re: Counter read requests spread across replicas ?

2011-12-21 Thread aaron morton
How many rows are you asking for in the multget_slice and what thread pools are showing pending tasks ? Also, what happens when you reduce the number of rows in the request? Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 21/12/2011, at 11

Re: Creating column families per client

2011-12-21 Thread Nick Bailey
The overhead for column families was greatly reduced in 0.8 and 1.0. It should now be possible to have hundreds or thousands of column families. The setting 'memtable_total_space_in_mb' was introduced that allows for a global memtable threshold, and cassandra will handle flushing on its own. See

Re: Creating column families per client

2011-12-21 Thread Ryan Lowe
What we have done to avoid creating multiple column families is to sort of namespace the row key. So if we have a column family of Users and accounts: "AccountA" and "AccountB", we do the following: Column Family User: "AccountA/ryan" : { first: Ryan, last: Lowe } "AccountB/ryan" : { first:

Re: questions on datastax opscenter

2011-12-21 Thread Jonathan Ellis
On Tue, Dec 20, 2011 at 1:49 PM, Feng Qu wrote: > I have two questions for community version of opscenter > > 1) does it work with multiple cassandra cluster? Tyler answered the second part. As for managing multiple clusters from a single opscenter: not yet, but it's on our radar. -- Jonathan E

Re: Routine nodetool repair

2011-12-21 Thread Blake Starkenburg
Thank You! Could the lack of routine repair be why nodetool ring reports: node(1) Load -> 78.24 MB and node(2) Load -> 67.21 MB? The load span between the two nodes has been increasing ever so slowly... On Wed, Dec 21, 2011 at 1:00 AM, aaron morton wrote: > Here you go > http://wiki.apache.org/c

Re: Creating column families per client

2011-12-21 Thread Flavio Baronti
Hi, based on my experience with Cassandra 0.7.4, i strongly discourage you to do that: we tried dynamical creation of column families, and it was a nightmare. First of all, the operation can not be done concurrently, therefore you must find a way to avoid parallel creation (over all the cluster

Re: Creating column families per client

2011-12-21 Thread Philippe
Every node? I hadn't realized that. Is there a place where i can compute how much memory is being 'wasted' ? Le 21 déc. 2011 15:09, "Alain RODRIGUEZ" a écrit : > Hi, I don't know if this will be technically possible, but I just want to > warn you about creating a lot of column families. When you

Re: Creating column families per client

2011-12-21 Thread Alain RODRIGUEZ
Hi, I don't know if this will be technically possible, but I just want to warn you about creating a lot of column families. When you will have a lot clients, you will have a lot of column families and, if I'm right, each column family uses memory on every node. You will run out of memory very fast,

Creating column families per client

2011-12-21 Thread Rafael Almeida
Hello, I am evaluating the usage of cassandra for my system. I will have several clients who won't share data with each other. My idea is to create one column family per client. When a new client comes in and adds data to the system, I'd like to create a column family dynamically. Is that relia

Re: Doubts related to composite type column names/values

2011-12-21 Thread R. Verlangen
Is it true that you can also just get the same results as when you pick a UTF8 key with this content: keyA:keyB Of should you really use the composite keys? If so, what is the big advantage of composite over combined utf-8 keys? Robin 2011/12/21 Sylvain Lebresne > On Tue, Dec 20, 2011 at 9:33

Handling topology changes

2011-12-21 Thread pyr
Hi, I wonder about the best strategy when a cluster sees changes in its topology relatively often. My main concern is how to handle the initial token for new nodes. If a cluster is first created with 7 nodes for which the initial token is calculated with the formula here: http://wiki.apache.org/c

Re: Composite Column Question

2011-12-21 Thread Sylvain Lebresne
On Wed, Dec 21, 2011 at 7:04 AM, Martin Arrowsmith wrote: > Dear Cassandra Experts, > > Are the number of composite attributes fixed for each column family ? > > I have been doing : "create column family MyCF with comparator = > 'CompositeType(IntegerType, UTF8Type)' > > And this creates a composi

Re: Doubts related to composite type column names/values

2011-12-21 Thread Sylvain Lebresne
On Tue, Dec 20, 2011 at 9:33 PM, Maxim Potekhin wrote: > Thank you Aaron! As long as I have plain strings, would you say that I would > do almost as well with catenation? Not without a concatenation aware comparator. The padding aaron is talking of is not a mixed type problem only. What I mean he

Re: Can I slice on composite indexes?

2011-12-21 Thread aaron morton
You can slice the "key1" row to get the columns that have "xyz" as the value for the first component in the column name. Check the docs in your client for how to do that. Hope that helps. - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 21/12/2

Re: Routine nodetool repair

2011-12-21 Thread aaron morton
Here you go http://wiki.apache.org/cassandra/Operations#Dealing_with_the_consequences_of_nodetool_repair_not_running_within_GCGraceSeconds Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 21/12/2011, at 2:44 PM, Blake Starkenburg wrote: > I