Cassandra Upgrade from 0.8.1

2012-06-05 Thread Adeel Akbar
Dear Guys, Thank you so much for your reply. Currently I have two Cassandra nodes running in ring. I have installed Cassandra on following location; /root/apache-cassandra-0.8.1 Now my questions are; 1. How we upgrade (Step by Step version like 0.8.1 to 0.8.5, then

RE: Cassandra Upgrade from 0.8.1

2012-06-05 Thread Harshvardhan Ojha
You can follow these steps for your version also . http://www.datastax.com/docs/1.0/install/upgrading If you will keep the data directory same in Cassandra.yaml, data will be picked in new node. Regards Harsh From: Adeel Akbar [mailto:adeel.ak...@panasiangroup.com] Sent: Tuesday, June 05, 2012

RE: nodetool repair -pr enough in this scenario?

2012-06-05 Thread Viktor Jevdokimov
Understand simple mechanics first, decide how to act later. Without -PR there's no difference from which host to run repair, it runs for the whole 100% range, from start to end, the whole cluster, all nodes, at once. With -PR it runs only for a primary range of a node you are running a repair.

Re: nodetool repair -pr enough in this scenario?

2012-06-05 Thread R. Verlangen
In your case -pr would be just fine (see Viktor's explanation). 2012/6/5 Viktor Jevdokimov viktor.jevdoki...@adform.com Understand simple mechanics first, decide how to act later. ** ** Without –PR there’s no difference from which host to run repair, it runs for the whole 100% range,

Re: about multitenant datamodel

2012-06-05 Thread R. Verlangen
Every CF has a certain amount of overhead in memory. It's just not how Cassandra is designed to be used. Maybe you could think of a way to smash data down to indices and entities. With an abstraction layer you can store practically anything in Cassandra. 2012/6/5 Toru Inoko in...@ms.scsk.jp

Re: nodetool repair -pr enough in this scenario?

2012-06-05 Thread Sylvain Lebresne
On Tue, Jun 5, 2012 at 8:44 AM, Viktor Jevdokimov viktor.jevdoki...@adform.com wrote: Understand simple mechanics first, decide how to act later. ** ** Without –PR there’s no difference from which host to run repair, it runs for the whole 100% range, from start to end, the whole

RE: nodetool repair -pr enough in this scenario?

2012-06-05 Thread Viktor Jevdokimov
But in any case, repair is a two way process? I mean that repair without -PR on node N1 will repair N1 and N2 and N3, because N2 is a replica of N1 range and N1 is a replica of N3 range? And if there're more ranges, that not belongs to N1, that ranges and nodes will not be repaired? Am I

How to configure the Cassandra connection using Hector

2012-06-05 Thread Prakrati Agrawal
Dear all, I am writing a java code using Hector but when I am configuring the already created key space using HFactory.createKeySpace its giving me the error Null Point Exception. What am I doing wrong. Please help me. Thanks and Regards Prakrati Agrawal | Developer - Big Data(ID)| 9731648376

RE: How to configure the Cassandra connection using Hector

2012-06-05 Thread Harshvardhan Ojha
Check if you have rightly provided your cluster name. Regards Harsh From: Prakrati Agrawal [mailto:prakrati.agra...@mu-sigma.com] Sent: Tuesday, June 05, 2012 2:55 PM To: user@cassandra.apache.org Subject: How to configure the Cassandra connection using Hector Dear all, I am writing a java

RE: How to configure the Cassandra connection using Hector

2012-06-05 Thread Prakrati Agrawal
cluster = HFactory.getCluster(Test Cluster); keySpace = HFactory.createKeyspace(mstream, cluster); These are the lines that I am executing and getting an error Prakrati Agrawal | Developer - Big Data(ID)| 9731648376 | www.mu-sigma.com From: Harshvardhan Ojha

RE: How to configure the Cassandra connection using Hector

2012-06-05 Thread MOHD ARSHAD SALEEM
In the following line give cluster = HFactory.getCluster(Test Cluster,ip:9160); ip =ip address of your machine and respective port no. Regards Arshad From: Prakrati Agrawal [prakrati.agra...@mu-sigma.com] Sent: Tuesday, June 05, 2012 3:00 PM To:

RE: How to configure the Cassandra connection using Hector

2012-06-05 Thread Prakrati Agrawal
Thank you all for the quick response :) Prakrati Agrawal | Developer - Big Data(ID)| 9731648376 | www.mu-sigma.com From: MOHD ARSHAD SALEEM [mailto:marshadsal...@tataelxsi.co.in] Sent: Tuesday, June 05, 2012 3:07 PM To: user@cassandra.apache.org Subject: RE: How to configure the Cassandra

Recreate LevelCompacted SSTables with a new size

2012-06-05 Thread Samuel CARRIERE
Hi, We are using a 9 nodes cassandra cluster, and each node is currently handling about 40GB of data. Our CFs were created using LevedCompactionStrategy, with the default options (size of SSTables = 5MB). So the number of SSTables was going quite large, which led to problems described in

How to include two nodes in Java code using Hector

2012-06-05 Thread Prakrati Agrawal
Dear all I am using a two node Cassandra cluster. How do I code in Java using Hector to get data from both the nodes. Please help Thanks and Regards Prakrati Agrawal | Developer - Big Data(ID)| 9731648376 | www.mu-sigma.com This email message may contain

RE: How to include two nodes in Java code using Hector

2012-06-05 Thread Harshvardhan Ojha
Use Consistency Level =2. Regards Harsh From: Prakrati Agrawal [mailto:prakrati.agra...@mu-sigma.com] Sent: Tuesday, June 05, 2012 4:08 PM To: user@cassandra.apache.org Subject: How to include two nodes in Java code using Hector Dear all I am using a two node Cassandra cluster. How do I code

MeteredFlusher in system.log entries

2012-06-05 Thread rohit bhatia
I am trying to understand the variance in flushes frequency in a 8 node Cassandra cluster. All the flushes are of the same type and initiated by MeteredFlusher.java = INFO [OptionalTasks:1] 2012-06-05 06:32:05,873 MeteredFlusher.java (line 62) flushing high-traffic column family

Performance impact of static vs dynamic columns and mixing the two in the same CF

2012-06-05 Thread Edward Sargisson
Hi all, A question has come up in our team about the performance impact of static vs dynamic columns. We'd like to ask two questions: Quick background: We are using a custom app to write to Cassandra using Hector. Production is Solaris and pre-prod is generally Centos. We're currently on 0.7

Re: [phpcassa] multi_get and composite, cassandra crash my mind

2012-06-05 Thread Tyler Hobbs
The Cassandra users mailing list is a better place for this question, so I'm moving it there. Some comments inline: On Tue, Jun 5, 2012 at 6:47 AM, Juan Ezquerro LLanes arr...@gmail.comwrote: I have a columnfamily like: CREATE COLUMN FAMILY Watchdog WITH key_validation_class =

Re: 1.1 not removing commit log files?

2012-06-05 Thread aaron morton
In theory the the logging configurator is watching the log4j-server.properties file and check for changes every 10 seconds. I've nevery had much luck with it, but assumed it was me getting something wrong. Or you can modify the values on the fly using the setLog4jLevel() on the StorageService

Re: about multitenant datamodel

2012-06-05 Thread aaron morton
With an abstraction layer you can store practically anything in Cassandra. See virtual keyspaces in Hector. why do you think so? I'll let users create ristricted CFs, and limit a number of CFs which users create. is it still a bad one? Depends what your limits are, but in general still yes.

Re: about multitenant datamodel

2012-06-05 Thread samal
why do you think so? I'll let users create ristricted CFs, and limit a number of CFs which users create. is it still a bad one? Ok, get it, you want to limit the cf user can create (assume) 2, what about 10k shared users creating 2 cf each= 20k CF ~~20GB memory used with no data in it. Do you

Re: nodetool repair -pr enough in this scenario?

2012-06-05 Thread aaron morton
-pr is a new feature added in 1.0. It was added for efficiency, not functionality. With -pr repair does 1/RF the work it does without it. Am I understood correctly, that “repair” with or without –PR is not a “repair selected node” process, but “synchronize data range(s) between replicas”

Re: Recreate LevelCompacted SSTables with a new size

2012-06-05 Thread aaron morton
Not that I am aware of. AFAIK the files should eventually get to that size. Anyone else ? Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 5/06/2012, at 9:44 PM, Samuel CARRIERE wrote: Hi, We are using a 9 nodes cassandra cluster, and

Nodes not picking up data on repair, disk loaded unevenly

2012-06-05 Thread Luke Hospadaruk
I have a 4-node cluster with one keyspace (aside from the system keyspace) with the replication factor set to 4. The disk usage between the nodes is pretty wildly different and I'm wondering why. It's becoming a problem because one node is getting to the point where it sometimes fails to compact

Re: MeteredFlusher in system.log entries

2012-06-05 Thread aaron morton
See the section on memtable_total_space_in_mb here http://thelastpickle.com/2011/05/04/How-are-Memtables-measured/ Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 6/06/2012, at 2:27 AM, rohit bhatia wrote: I am trying to understand the

RE: memory issue on 1.1.0

2012-06-05 Thread Poziombka, Wade L
Alas, upgrading to 1.1.1 did not solve my issue. -Original Message- From: Brandon Williams [mailto:dri...@gmail.com] Sent: Monday, June 04, 2012 11:24 PM To: user@cassandra.apache.org Subject: Re: memory issue on 1.1.0 Perhaps the deletes:

Re: Performance impact of static vs dynamic columns and mixing the two in the same CF

2012-06-05 Thread aaron morton
(I'm assuming you are talking about column values here) 1. Does specifying the type of a column affect performance other than the cost of validating data as it is stored? e.g. does it help compaction, etc? No. Validation is normally pretty light weight. From my reading of the docs the

Re: nodetool repair -pr enough in this scenario?

2012-06-05 Thread David Daeschler
Thank you for all the replies. It has been enlightening to read. I think I now have a better idea of repair, ranges, replicas and how the data is distributed. It also seems that using -pr would be the best way to go in my scenario with 1.x+ Thank you for all the feedback. Glad to see such an

Re: memory issue on 1.1.0

2012-06-05 Thread Mina Naguib
Hi Wade I don't know if your scenario matches mine, but I've been struggling with memory pressure in 1.x as well. I made the jump from 0.7.9 to 1.1.0, along with enabling compression and levelled compactions, so I don't know which specifically is the main culprit. Specifically, all my nodes

RE Nodes not picking up data on repair, disk loaded unevenly

2012-06-05 Thread Samuel CARRIERE
Hi, To verify that the repair was successful, you can look for this kind of messages in the log : INFO [AntiEntropyStage:1] 2012-05-19 00:57:52,351 AntiEntropyService.java (line 762) [repair #e46a0a90-a13c-11e1--596f3d333ab7] UsersCF is fully synced (3 remaining column family to sync for

RE: memory issue on 1.1.0

2012-06-05 Thread Poziombka, Wade L
Thank you. I do have some of the same observations. Do you do deletes? My observation is that without deletes (or column updates I guess) I can run forever happy. but when I run (what for me is a batch process) operations that delete and modify column values I run into this. Reading bug

Re: Secondary Indexes, Quorum and Cluster Availability

2012-06-05 Thread Jim Ancona
On Mon, Jun 4, 2012 at 2:34 PM, aaron morton aa...@thelastpickle.comwrote: IIRC index slices work a little differently with consistency, they need to have CL level nodes available for all token ranges. If you drop it to CL ONE the read is local only for a particular token range. Yes, this is

Re: Node join streaming stuck at 100%

2012-06-05 Thread koji Lin
There is no error in the log about the streaming. And thanks for the information, we will try 1.1 when we start upgrade. koji 2012/6/5 aaron morton aa...@thelastpickle.com Are their any errors in the logs about failed streaming ? If you are getting time outs 1.0.8 added a streaming socket

RE: memory issue on 1.1.0

2012-06-05 Thread Poziombka, Wade L
Ok, so I have completely refactored to remove deletes and it still fails. So it is completely unrelated to deletes. I guess I need to go back to 1.0.10? When I originally evaluated I ran 1.0.8... perhaps I went a bridge too far with 1.1. I don't think I am doing anything exotic here. Here is

RE: How to include two nodes in Java code using Hector

2012-06-05 Thread Prakrati Agrawal
But the data is distributed on the nodes ( meaning 50% of data is on one node and 50% of data is on another node) so I need to specify the node ip address somewhere in the code. But where do I specify that is what I am clueless about. Please help me Prakrati Agrawal | Developer - Big Data(ID)|

my devious QA - how to recover with power lost situation

2012-06-05 Thread Poziombka, Wade L
They 1) setup a two node cluster and loaded 500K rows or something 2) add a third node, run nodetool move 3) while moving they pull the plug on the node Cassandra won't start with the exception below. Now, this is obviously a very exceptional situation but the question is

Re: How to include two nodes in Java code using Hector

2012-06-05 Thread samal
I don't use hector, don't know much about internals, this may help * Cluster cluster = HFactory.getOrCreateCluster( TestCluster,host1:9160,host2:9160,host3:9160)* If you have 2 node cluster with RF=2, your data will be present in both node. And if consistency level 2 is used both node must be

Re: MeteredFlusher in system.log entries

2012-06-05 Thread rohit bhatia
Hi Aaron Thanks for the link, I have gone through it. But this doesn't justify nodes of exactly same config/specs differing in their flushing frequency. The traffic on all node is same as we are using RandomPartitioner Thanks Rohit On Wed, Jun 6, 2012 at 12:24 AM, aaron morton

Removing a node in cluster

2012-06-05 Thread Prakrati Agrawal
Dear all I am trying to check the performance of Cassandra on adding or removing nodes. I want to know what happens to my existing data if I remove a node ? Please help me Thanks and Regards Prakrati Agrawal | Developer - Big Data(ID)| 9731648376 | www.mu-sigma.com