Re: random partitioner and key scan

2012-07-21 Thread Sylvain Lebresne
I don't know much about Cassandra internals, but from a user point of view, a scan for a range of tokens is not a common use-case. All of boostrap/move/decommission/repair rely heavily on being able to scan efficiently a range of token. Otherwise, a boostrap/move/decommission/repair of a node

Re: random partitioner and key scan

2012-07-21 Thread prasenjit mukherjee
On Sat, Jul 21, 2012 at 3:37 PM, Sylvain Lebresne sylv...@datastax.com wrote: I don't know much about Cassandra internals, but from a user point of view, a scan for a range of tokens is not a common use-case. All of boostrap/move/decommission/repair rely heavily on being able to scan

Re: random partitioner and key scan

2012-07-20 Thread aaron morton
Repair and token moves work on ranges of Tokens, not row keys. These operations need to scan through all the rows in the token range. Ordering the rows by row key locally would mean that every row on the node would have to be scanned to find the ones whose token was in the required token

Re: random partitioner and key scan

2012-07-20 Thread prasenjit mukherjee
Agree to aaron's point. But still I think there are ways to overcome this problem, as I believe that the row-scan use-case is very important. One simple ( but expensive ) approach could be to duplicate the rows with RP tokens on primary node, which will be used only in case of repairs. Any

Re: random partitioner and key scan

2012-07-20 Thread Patrik Modesto
On Fri, Jul 20, 2012 at 11:17 AM, aaron morton aa...@thelastpickle.com wrote: Ordering the rows by row key locally would mean that every row on the node would have to be scanned to find the ones whose token was in the required token range. I don't know much about Cassandra internals, but from

random partitioner and key scan

2012-07-19 Thread Patrik Modesto
Hi, I know that RandomPartitioner does MD5 of a key and the MD5 is then used for key distribution AND key ordering. I was just wondering if it's possible to have RandomPartitioner just for key distribution and OrderedPartitioner just for per-node key ordering. That would solve the often requested

Re: random partitioner and key scan

2012-07-19 Thread prasenjit mukherjee
I agree. The probem could be while redistributing the tokens. In that case the hashes has to be recalculated on each fo the candidate node. -Thanks, Prasenjit On Thu, Jul 19, 2012 at 12:19 PM, Patrik Modesto patrik.mode...@gmail.com wrote: Hi, I know that RandomPartitioner does MD5 of a key

Re: random partitioner and key scan

2012-07-19 Thread Patrik Modesto
Hi Prasenjit, I don't see the need to recalculate anything. One key has a one MD5 hash, it doesn't change. Just use the hash to select a node, than just the plain key. Can you elaborate on the redistribution please? Regards, P. On Thu, Jul 19, 2012 at 9:09 AM, prasenjit mukherjee

Re: random partitioner and key scan

2012-07-19 Thread prasenjit mukherjee
When a request for token change is issued ( via nodetool ) then on what basis a node will move some of its rows to other node, as there will be no way to scan rows based on MD5 hash in a given node ( if the keys are not prefixed with MD5 hash ) On Thu, Jul 19, 2012 at 1:43 PM, Patrik Modesto