Re: Commitlog Disk Full

2011-05-13 Thread Sanjeev Kulkarni
Hi Peter, Thanks for the response. I haven't explictly set a value for the memtable_flush_after_mins parameter. Looks like the default is 60minutes. I will try to play around this value to see if that fixes things. Thanks again! On Thu, May 12, 2011 at 11:41 AM, Peter Schuller

Cassandra node is not blanced Rf=2 Random Partitioner

2011-05-13 Thread Ali Ahsan
My cluster is unbalanced.One have 99 GB Data and other have 87 GB can any one explain why this is happening. [root@cassandra2 conf]# /root/cassandra/bin/nodetool -h 10.0.0.4 ring Address Status Load Range Ring

Re: Cassandra node is not blanced Rf=2 Random Partitioner

2011-05-13 Thread Ali Ahsan
On 05/13/2011 11:46 AM, Ali Ahsan wrote: My cluster is unbalanced.One have 99 GB Data and other have 87 GB can any one explain why this is happening. [root@cassandra2 conf]# /root/cassandra/bin/nodetool -h 10.0.0.4 ring Address Status Load Range

Re: Commitlog Disk Full

2011-05-13 Thread Peter Schuller
I haven't explictly set a value for the memtable_flush_after_mins parameter. Looks like the default is 60minutes. I will try to play around this value to see if that fixes things. Is the amount of data in the commit log consistent with what you might have been writing during 60 minutes?

Re: Monitoring bytes read per cf

2011-05-13 Thread Peter Schuller
It seems to work fine but maybe someone here knows that this is not a good idea It depends. If you're trying to figure out what generates actual iops to disk, it won't really tell you since you're counting all I/O including that which is cached. Since whatever goes down to disk tends to

Inter node communication over UDP

2011-05-13 Thread pankajsoni0126
we are deploying cassandra over a huge cluster size and we think that inter node communication can become overload on nodes as the protocol used is TCP. Similarly gossip was earlier using udp but has been changed to use Tcp. Is there added advantage for using tcp over udp in cassandra both in

Re: Monitoring bytes read per cf

2011-05-13 Thread Daniel Doubleday
Thanks - yes I agree. Didn't want to judge solely based on this figure. It should just add to the picture. But since we know access patterns and other stats like key and row cache hit ratios we hope to be able to make a more educated guess whats going on. On May 13, 2011, at 9:08 AM, Peter

Re: Import/Export of Schema Migrations

2011-05-13 Thread aaron morton
What sort of schema changes are you making? can you manage them as a CLI script under source control ? You may also be interested in CASSANDRA-2221. Cheers Aaron - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 12 May 2011, at 20:45,

Feedback on ACM SOCC paper about elasticity and scalability

2011-05-13 Thread Thibault Dory
Hello, I have written with a few other people a paper for the ACM Symposium On Cloud Computing. This paper describes the methodology, infrastructure and configuration used as well as the results obtained for elasticity and scalability of three noSQL databases, of wich Cassandra. The paper can be

Re: Excessive allocation during hinted handoff

2011-05-13 Thread aaron morton
The number of Completed HH tasks is interesting. AFAIK a task is started when the node detects another in the cluster has returned. Were you doing some other restarts around the cluster ? I don't want to divert from the GC issue, just wondering if something else is going on as well. Like the

Re: nodetool move in cassandra

2011-05-13 Thread aaron morton
nodetool move decommissions the node then bootstraps it back . I've not checked the code but decommission will only send the data it needs to the other nodes, and I believe bootstrap can do that same but I am not sure. http://wiki.apache.org/cassandra/Operations#Moving_nodes Aaron

Re: Cassandra node is not blanced Rf=2 Random Partitioner

2011-05-13 Thread Eric tamme
On Fri, May 13, 2011 at 2:46 AM, Ali Ahsan ali.ah...@panasiangroup.com wrote: My cluster is unbalanced.One have 99 GB Data and other have 87 GB can any one explain why this is happening. They are pretty close ... since a row key is pinned to a node - it is possible that you have a really large

Re: assertion error in cassandra when doing nodetool move

2011-05-13 Thread aaron morton
I think you may have a data centre defined in the strategy_options for the keyspace that does not have any nodes in it. Check the dc names in the create keyspace statement match the names in your cassandra-topology.propeties file (assuming you are using the PropertyFileSnitch), or that you have

Re: Cassandra node is not blanced Rf=2 Random Partitioner

2011-05-13 Thread aaron morton
You need to manually assign the tokens to the nodes http://wiki.apache.org/cassandra/Operations#Load_balancing One should be 0 and the other 85070591730234615865843651857942052864 Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On

Re: Hinted Handoff

2011-05-13 Thread Maki Watanabe
HH will be stored into one of live replica node. It is just a hint, rather than data to be replicated. maki 2011/5/12 Anurag Gujral anurag.guj...@gmail.com: Hi All,            I have two questions: a) Is there  a way to turn on and off hinted handoff per keyspace rather than for multiple

Re: Import/Export of Schema Migrations

2011-05-13 Thread David Boxenhorn
Actually, I want a way to propagate *any* changes from development to staging to production, but schema changes are the most important. Could I use 2221 to propagate schema changes by deleting the schema in the target cluster, doing show schema in the source cluster, redirecting to a file, and

Re: Inter node communication over UDP

2011-05-13 Thread Jonathan Ellis
UDP is a bad fit for Cassandra; you'd have to implement your own ordering and robustness on top of it. Better to just use TCP. On Fri, May 13, 2011 at 2:55 AM, pankajsoni0126 pankajsoni0...@gmail.com wrote: we are deploying cassandra over a huge cluster size and we think that inter node

Re: How to invoke getNaturalEndpoints with jconsole?

2011-05-13 Thread Maki Watanabe
I wrote a small JMX client to invoke getNaturalEndpoints. It works fine at my test environment, but throws NPE for keyspace we will use for our application (both 0.7.5). Does anyone know quick resolution of that before I setting up cassandra on eclipse to inspect what happens :) thanks Exception

[howto measure disk usage]

2011-05-13 Thread Alexis Rodríguez
cassandra-people, I'm trying to measure disk usage by cassandra after inserting some columns in order to plan disk sizes and configurations for future deploys. My approach is very straightforward: clean_data (stop_cassandra rm -rf /var/lib/cassandra/{dara,commitlog,saved_caches}/*)

Re: assertion error in cassandra when doing nodetool move

2011-05-13 Thread Anurag Gujral
I checked file cassandra-topology.properties everything looks good to me. Please Advise on next steps I can do. Thanks Anurag On Fri, May 13, 2011 at 3:52 AM, aaron morton aa...@thelastpickle.comwrote: I think you may have a data centre defined in the strategy_options for the keyspace that

Re: How to invoke getNaturalEndpoints with jconsole?

2011-05-13 Thread Alex Araujo
On 5/13/11 10:08 AM, Maki Watanabe wrote: I wrote a small JMX client to invoke getNaturalEndpoints. It works fine at my test environment, but throws NPE for keyspace we will use for our application (both 0.7.5). Does anyone know quick resolution of that before I setting up cassandra on eclipse

Re: Excessive allocation during hinted handoff

2011-05-13 Thread Gabriel Tataranu
The number of Completed HH tasks is interesting. AFAIK a task is started when the node detects another in the cluster has returned. Were you doing some other restarts around the cluster ? Not at all. The restarts seem to happen as normal operation. I don't want to divert from the GC

Re: Commitlog Disk Full

2011-05-13 Thread mcasandra
Is there a way to look at the actual size of memtable? Would that help? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Commitlog-Disk-Full-tp6356797p6360001.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at

Re: Crash when uploading large data sets

2011-05-13 Thread James Cipar
It is a 64 bit VM. I didn't notice the hs_err_pid.log files since I'm staring over ssh, so they're in my home directory instead of my working directory. I've attached one of those below. I don't know much about Java, so I'm not sure how to interpret this file. # # An unexpected error

Converting separate clusters in mutliple dc to one cluster across multiple dcs

2011-05-13 Thread Anurag Gujral
Hi All, I have 3 separate cassandra clusters running in multiple data centers which I want to convert to one cassandra cluster across multiple data centers Does anyone tried this? If so what are the steps: I think I need to do the following: a)Change cluster name: What is the

Ghost token

2011-05-13 Thread Scott McPheeters
Has anyone seen this and know if it is causing an issue or how to fix it? Anytime I run nodetool ring (on any node) I get this extra token show up on the top of the list. Address Status State LoadOwnsToken 113427455640312821154458202477256070484 10.1.2.171 Up

Re: Ghost token

2011-05-13 Thread Ryan King
That's the same as the last one. The token space is a circle so the last one at the list is repeated at the top. -ryan On Fri, May 13, 2011 at 9:59 AM, Scott McPheeters smcpheet...@healthx.com wrote: Has anyone seen this and know if it is causing an issue or how to fix it?  Anytime I run

RE: Ghost token

2011-05-13 Thread Scott McPheeters
Ahh, thanks. I got questioned on that as soon as our test environment came up and I had no idea. Thanks for the reply. Scott -Original Message- From: Ryan King [mailto:r...@twitter.com] Sent: Friday, May 13, 2011 1:03 PM To: user@cassandra.apache.org Subject: Re: Ghost token

Re: assertion error in cassandra when doing nodetool move

2011-05-13 Thread Anurag Gujral
I am using network Topology strategy for one of my keyspaces. Thanks Anurag On Thu, May 12, 2011 at 6:45 PM, Anurag Gujral anurag.guj...@gmail.comwrote: Hi All, I run following command on one of my nodes to move the token from 0 to 2. /usr/cassandra/cassandra/bin/nodetool -h

Re: How to invoke getNaturalEndpoints with jconsole?

2011-05-13 Thread Jonathan Ellis
sounds like https://issues.apache.org/jira/browse/CASSANDRA-2353 On Fri, May 13, 2011 at 10:08 AM, Maki Watanabe watanabe.m...@gmail.com wrote: I wrote a small JMX client to invoke getNaturalEndpoints. It works fine at my test environment, but throws NPE for keyspace we will use for our

Inconsistent data issues when running nodetool move.

2011-05-13 Thread Ryan Hadley
Hi, I'm running Cassandra (0.7.4) on a 4 node ring. It was a 3 node ring, but we ended up expanding it to 4... So then I followed the many suggestions to rebalance the ring. I found a script that suggested I use: # ~/nodes_calc.py How many nodes are in your cluster? 4 node 0: 0 node 1:

Re: How to invoke getNaturalEndpoints with jconsole?

2011-05-13 Thread Maki Watanabe
I did not drop the keyspace, but your comment lead me to resolution. I found cassandra-cli is not case sensitive on keyspace. I used keyspace name FooBar on cassandra-cli, but it was Foobar in correct. cassandra-cli didn't complain on my mistake, but the JMX interface is less tolerance. If I use

problems using property snitch file

2011-05-13 Thread Anurag Gujral
Hi All, I have built a cassandra cluster which is using property snitch file to specify my network topology to cassandra.The keyspace I have use network topology strategy. When I tried to add new node to this cluster,I get the following error: RROR [main] 2011-05-13 23:45:44,152

Re: Crash when uploading large data sets

2011-05-13 Thread Jonathan Ellis
You should upgrade to the latest Sun JVM. OpenJDK is almost a year behind in bug fixes. On Fri, May 13, 2011 at 11:40 AM, James Cipar jci...@cmu.edu wrote: It is a 64 bit VM.  I didn't notice the hs_err_pid.log files since I'm staring over ssh, so they're in my home directory instead of my

Re: Commitlog Disk Full

2011-05-13 Thread Sanjeev Kulkarni
our write happen in bursts. So often times, clients write data as fast as they can. Conceivably one can write 5G in one hour. The other setting that we have is that our replication factor is 3 and we write using QUORUM. Not sure if that will affect things. On Fri, May 13, 2011 at 12:04 AM, Peter

Re: Commitlog Disk Full

2011-05-13 Thread mcasandra
5G in one hour is actually very low. Something else is wrong. Peter pointed to something related to memtable size could be causing this problem, can you turn down memtable_throughput and see if that helps. -- View this message in context: