Re: Cassandra backup queston regarding commitlogs

2012-04-30 Thread Roshan
Hi Aaron Thanks for the comments. Yes for the durability will keep them in a safe place. But such crash situation, how can I restore the data (because those are not in a SSTable and only in commit log). Do I need to replay only that commit log when server starts after crash? Will it override

timezone time series data model

2012-04-30 Thread samal
Hello List, I need suggestion/ recommendation on time series data. I have requirement where users belongs to different timezone and they can subscribe to global group. When users at specific timezone send update to group it is available to every user in different timezone. I am using

Composite metadata

2012-04-30 Thread Juan Ezquerro
Hi, I have a columnfamily like that: CREATE COLUMN FAMILY Clients WITH key_validation_class = 'CompositeType(LexicalUUIDType,UTF8Type)' AND comparator = LexicalUUIDType AND column_metadata = [ {column_name: name, validation_class: UTF8Type} ]; My metadata definition

RE: Crash by truncate with cassandra 1.1

2012-04-30 Thread Pierre Chalamet
Hi Aaron, No, I just avoid truncating for the moment L (I'm adding CQL prepared statement to Cassandra-Sharp). It's a bit strange anyway, JVM options are: set JAVA_OPTS=-ea^ -javaagent:%CASSANDRA_HOME%\lib\jamm-0.2.5.jar^ -Xms1G^ -Xmx1G^ -XX:+HeapDumpOnOutOfMemoryError^

Re: Data model question, storing Queue Message

2012-04-30 Thread Morgan Segalis
Hi Aaron, Thank you for your answer, I was beginning to think that my question would never be answered ;-) Actually, this is what I was going for, except one thing, instead of partitioning row per month, I though about partitioning per day, like that everyday I launch the cleaning tool, and

Re: Data model question, storing Queue Message

2012-04-30 Thread samal
On Mon, Apr 30, 2012 at 4:25 PM, Morgan Segalis msega...@gmail.com wrote: Hi Aaron, Thank you for your answer, I was beginning to think that my question would never be answered ;-) Actually, this is what I was going for, except one thing, instead of partitioning row per month, I though

Re: Data model question, storing Queue Message

2012-04-30 Thread Morgan Segalis
Hi Samal, Thanks for the TTL feature, I wasn't aware of it's existence. Day's partitioning will be less wider than month partitionning (about 30 times less give or take ;-) ) Per day it should have something like 100 000 messages stored, most of it would be retrieved so deleted before the TTL

Re: Data model question, storing Queue Message

2012-04-30 Thread samal
On Mon, Apr 30, 2012 at 5:52 PM, Morgan Segalis msega...@gmail.com wrote: Hi Samal, Thanks for the TTL feature, I wasn't aware of it's existence. Day's partitioning will be less wider than month partitionning (about 30 times less give or take ;-) ) Per day it should have something like 100

Re: timezone time series data model

2012-04-30 Thread Tyler Hobbs
Don't use dates or datestamps as the buckets for your row keys, use a unix timestamp modulo whatever size you want your bucket to be instead. Timestamps don't involve time zones or any of that nonsense. So, instead of having keys like user1uuid_30042012, the second half would be replaced the

strange gossip messages after node reboot with different ip

2012-04-30 Thread Piavlo
Hi, We have a cassandra cluster in ec2. If i stop a node and start it - as a result the node ip changes. The node is recognised as NEW node and is declared as replacing the previous node with same token.(But this is the same node of course) In this specific case the node ip before

Re: Data model question, storing Queue Message

2012-04-30 Thread Morgan Segalis
Isn't kafka too young for production using purpose ? Clearly that would fit much better my needs but I can't afford early stage project not ready for production. Is it ? Le 30 avr. 2012 à 14:28, samal samalgo...@gmail.com a écrit : On Mon, Apr 30, 2012 at 5:52 PM, Morgan Segalis

Re: timezone time series data model

2012-04-30 Thread samal
thanks tyler for reply. are you saying user1uuid_*{ts%86400}* would lead to unique day bucket which will be timezone {NZ to US} independent? I will try. On Mon, Apr 30, 2012 at 8:25 PM, Tyler Hobbs ty...@datastax.com wrote: Don't use dates or datestamps as the buckets for your row keys, use a

Re: timezone time series data model

2012-04-30 Thread Tyler Hobbs
Correct, that's exactly what I'm saying. On Mon, Apr 30, 2012 at 10:37 AM, samal samalgo...@gmail.com wrote: thanks tyler for reply. are you saying user1uuid_*{ts%86400}* would lead to unique day bucket which will be timezone {NZ to US} independent? I will try. On Mon, Apr 30, 2012 at

Re: timezone time series data model

2012-04-30 Thread samal
I did it with node.js but it is changing after some interval. code setInterval(function(){ var d =new Date().getTime(); console.log(== ); console.log(unix = ,d); i=parseInt(d) console.log(Divid i/86400= ,i/86400); console.log(Modulo i%86400= ,i%86400);

Re: timezone time series data model

2012-04-30 Thread Tyler Hobbs
getTime() returns the number of milliseconds since the epoch, not the number of seconds: http://www.w3schools.com/jsref/jsref_gettime.asp If you divide that number by 1000, it should work. On Mon, Apr 30, 2012 at 11:28 AM, samal samalgo...@gmail.com wrote: I did it with node.js but it is

Re: timezone time series data model

2012-04-30 Thread samal
thanks I didn't noticed. run script for 5 minutes = divide seems to produce result ,modulo is still changing. If divide is ok will do the trick. I will run this script on Singapore, East coast server, and New delhi server whole night today. == unix =

Re: timezone time series data model

2012-04-30 Thread Tyler Hobbs
Err, sorry, I should have said ts - (ts % 86400). Integer division does something similar. On Mon, Apr 30, 2012 at 12:39 PM, samal samalgo...@gmail.com wrote: thanks I didn't noticed. run script for 5 minutes = divide seems to produce result ,modulo is still changing. If divide is ok will

Re: Composite metadata

2012-04-30 Thread Tyler Hobbs
When you set comparator = LexicalUUIDType, you're saying that column names are UUIDs. In your column metadata, you need to use a UUID for column_name if that's what you want. I suspect that you either don't want LexicalUUIDType for your column names or you're looking for

Re: timezone time series data model

2012-04-30 Thread samal
hhmm. I will try both. thanks On Mon, Apr 30, 2012 at 11:29 PM, Tyler Hobbs ty...@datastax.com wrote: Err, sorry, I should have said ts - (ts % 86400). Integer division does something similar. On Mon, Apr 30, 2012 at 12:39 PM, samal samalgo...@gmail.com wrote: thanks I didn't noticed.

Re: AssertionError: originally calculated column size ...

2012-04-30 Thread Patrik Modesto
Hi Aaron, it's still an issue. However it's difficult to reproduce because I'd need to insert data to Cassandra for day or more which is not possible. I think the problem is somehow connected to an IntegerType secondary index. I had a different problem with CF with two secondary indexes, the

Re: Cassandra backup queston regarding commitlogs

2012-04-30 Thread aaron morton
When the server starts it reads the SSTables then applies the Commit Logs. There is nothing you need to do other than leave the commit logs where they are. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 30/04/2012, at 6:02 PM, Roshan

Re: Crash by truncate with cassandra 1.1

2012-04-30 Thread aaron morton
Try upgrading to the newest JVM first. Can you also say what version you were using ? If you still get the problem create a ticket on https://issues.apache.org/jira/browse/CASSANDRA with the steps to reproduce. Please include the OS version, RAM size, JVM information, commit_log_segment_size

execute_prepared_cql_query and variable range filter parameter

2012-04-30 Thread Pierre Chalamet
Hi all, Is there a support in Cassandra 1.1 for variable range filter parameter (sorry I can't find a right name for that): select * from TestCF where key in (?) using execute_prepared_cql_query ? In the query above, it seems I can only bind one value to '?'. I mean, if several

RE: Crash by truncate with cassandra 1.1

2012-04-30 Thread Pierre Chalamet
JVM is the latest available after all. Filled CASSANDRA-4201. Thanks, - Pierre From: aaron morton [mailto:aa...@thelastpickle.com] Sent: lundi 30 avril 2012 22:36 To: user@cassandra.apache.org Subject: Re: Crash by truncate with cassandra 1.1 Try upgrading to the newest JVM first.

Re: execute_prepared_cql_query and variable range filter parameter

2012-04-30 Thread paul cannon
Hi Pierre- Yes, each ? can only represent one value at a time (although it can take on a different value for each actual execution of the prepared query). This is certainly normal for SQL binding libraries. Not sure why you feel that defeats statement preparation. p On Mon, Apr 30, 2012 at 3:45

RE: execute_prepared_cql_query and variable range filter parameter

2012-04-30 Thread Pierre Chalamet
If I prepare “select * from Town where key in (?)” I will be able to bind ? to 'Paris' for example [è select * from Town where key in ('Paris')]. If I want to query for 'Paris' and 'London', the query should be restated to “select * from Town where key in (?, ?)” [è select * from Town where

Re: Cassandra backup queston regarding commitlogs

2012-04-30 Thread Roshan
Many Thanks Aaron. According to the datastax restore documentation, they ask to remove the commitlogs before restoring (Clear all files the /var/lib/cassandra/commitlog (by default)). In that case better not to follow this step in a server rash situation. Thanks /Roshan -- View this

Re: CassandraFS in 1.0?

2012-04-30 Thread Michael
David Strauss david at davidstrauss.net writes: It's not, currently, but I'm happy to answer questions about its architecture. David, Has your cassandraFS / WebDav software made it into the light of open source? Does yo integrate into other systems for user info, and simply store data

JNA + Cassandra security

2012-04-30 Thread Cord MacLeod
Hello group, I'm a new Cassandra and Java user so I'm still trying to get my head around a few things. If you've disabled swap on a machine what is the reason to use JNA? A second question is doesn't JNA break the Java inherent security mechanisms by allowing access to direct system calls

Re: strange gossip messages after node reboot with different ip

2012-04-30 Thread aaron morton
Gossip information about a node can stay in the cluster for up to 3 days. How long has this been going on for ? I'm unsure if this is expected behaviour. But it sounds like Gossip is kicking out the phantom node correctly. Can you use nodetool gossipinfo on the nodes to capture some artefacts

Re: Data model question, storing Queue Message

2012-04-30 Thread aaron morton
Isn't kafka too young for production using purpose ? The best way to advance the project is to use it and contribute your experience and time. btw, checking out kafka is a great idea. There are people around having Fun Times with Kafka in production Cheers - Aaron Morton

Re: Cassandra backup queston regarding commitlogs

2012-04-30 Thread aaron morton
Can you provide a link to that page ? Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 1/05/2012, at 10:12 AM, Roshan wrote: Many Thanks Aaron. According to the datastax restore documentation, they ask to remove the commitlogs before

Re: JNA + Cassandra security

2012-04-30 Thread aaron morton
If you've disabled swap on a machine what is the reason to use JNA? JNA will still be used to efficiently make hard links for snapshots. It's not necessary to lock the JVM memory when swap is disabled. A second question is doesn't JNA break the Java inherent security mechanisms by allowing

Re: AssertionError: originally calculated column size ...

2012-04-30 Thread Jonathan Ellis
On Mon, Apr 30, 2012 at 2:11 PM, Patrik Modesto patrik.mode...@gmail.com wrote: I think the problem is somehow connected to an IntegerType secondary index. Could be, but my money is on the supercolumns in the HH data model. Can you create a jira ticket? -- Jonathan Ellis Project Chair,

Re: JNA + Cassandra security

2012-04-30 Thread Jonathan Ellis
On Mon, Apr 30, 2012 at 7:49 PM, Cord MacLeod cordmacl...@gmail.com wrote: Hello group, I'm a new Cassandra and Java user so I'm still trying to get my head around a few things.  If you've disabled swap on a machine what is the reason to use JNA? Faster snapshots, giving hints to the page

Re: incremental_backups

2012-04-30 Thread Jonathan Ellis
Incremental snapshots contain only new data, so they are *much* smaller. On Mon, Apr 30, 2012 at 12:39 AM, Tamar Fraenkel ta...@tok-media.comwrote: Hi! I wonder what are the advantages of doing incremental snapshot over non incremental? Are the snapshots smaller is size? Are there any other

Re: incremental_backups

2012-04-30 Thread Ben Coverston
The documentation in the cassandra.yaml file covers this pretty well. In summary, a snapshot will create a hard link for each file in the data directory (if JNA is installed and on the classpath). Turning on incremental backup will create a hard link to every new SSTable that is flushed out to

Re: Cql 3 wide rows filter expressions in where clause

2012-04-30 Thread Jonathan Ellis
That should work. I don't see anything obviously wrong with your query, other than the trivial (ascii values need to be quoted). Assuming that's not the problem, please file a ticket if you have a failing test case. On Fri, Apr 20, 2012 at 11:59 PM, Nagaraj J nagaraj.pe...@gmail.com wrote: Hi

Re: Cql 3 wide rows filter expressions in where clause

2012-04-30 Thread paul cannon
CQL doesn't currently let you filter on columns which aren't part of a primary key or indexed, so the and x=2 and z=2 part of your query is not valid. p On Mon, Apr 30, 2012 at 8:54 PM, Jonathan Ellis jbel...@gmail.com wrote: That should work. I don't see anything obviously wrong with your

Re: execute_prepared_cql_query and variable range filter parameter

2012-04-30 Thread paul cannon
No, there isn't right now. But note that there shouldn't be a whole lot of performance difference between select * from Town where key in ('Paris', 'London'); and select * from Town where key = 'Paris'; select * from Town where key = 'London'; ..other than round-trip times. Or, I

Re: Taking a Cluster Wide Snapshot

2012-04-30 Thread Tamar Fraenkel
Thanks for posting the script. I see that the snapshot is always a full one, and if I understand correctly, it replaces the old snapshot on S3. Am I right? *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54