Re: Some commit logs not deleting

2011-07-25 Thread Sylvain Lebresne
This is likely due to https://issues.apache.org/jira/browse/CASSANDRA-2829. Basically, if you have a column family on which you stop to write suddenly forever (which will be the case if you drop a cf), one commit log could get retained forever (forever meaning until next restart of the node in

cassandra server disk full

2011-07-25 Thread Donna Li
All: Could anyone help me? Best Regards Donna li -邮件原件- 发件人: Donna Li [mailto:donna...@utstar.com] 发送时间: 2011年7月22日 11:23 收件人: user@cassandra.apache.org 主题: cassandra server disk full All: Is there an easy way to fix the bug by change server's code? Best Regards Donna li

Re: host clocks

2011-07-25 Thread Paul Loy
And I guess there's the next problem, if you have more than one client... On Mon, Jul 25, 2011 at 5:25 AM, Edward Capriolo edlinuxg...@gmail.comwrote: You should always sync your host clocks. Clients provide timestamp but for the server gc_grace and ttl columns can have issues if server clocks

Re: host clocks

2011-07-25 Thread Paul Loy
As I understand it, this will not quarantee that they are millisecond accurate which is what you need for Cassandra to use the correct commit. We've seen problems in production and had to rearchitect parts of the system due to this even though all servers are NTP synched.

Re: do I need to add more nodes? minor compaction eat all IO

2011-07-25 Thread Yan Chunlu
I am using normal SATA disk, actually I was worrying about whether it is okay if every time cassandra using all the io resources? further more when is the good time to add more nodes when I was just using normal SATA disk and with 100r/s it could reach 100 %util how large the data size it

Re: host clocks

2011-07-25 Thread zGreenfelder
On Mon, Jul 25, 2011 at 8:51 AM, Paul Loy ketera...@gmail.com wrote: As I understand it, this will not quarantee that they are millisecond accurate which is what you need for Cassandra to use the correct commit. We've seen problems in production and had to rearchitect parts of the system due

Re: host clocks

2011-07-25 Thread Paul Loy
we don't have those guarantees on EC2. Networks can fluctuate wildly. On Mon, Jul 25, 2011 at 2:00 PM, zGreenfelder zgreenfel...@gmail.comwrote: On Mon, Jul 25, 2011 at 8:51 AM, Paul Loy ketera...@gmail.com wrote: As I understand it, this will not quarantee that they are millisecond

Re: host clocks

2011-07-25 Thread Peter Schuller
As I understand it, this will not quarantee that they are millisecond accurate which is what you need for Cassandra to use the correct commit. We've seen problems in production and had to rearchitect parts of the system due to this even though all servers are NTP synched. It does not

Re: do I need to add more nodes? minor compaction eat all IO

2011-07-25 Thread Yan Chunlu
as the wiki suggested: http://wiki.apache.org/cassandra/LargeDataSetConsiderations Adding nodes is a slow process if each node is responsible for a large amount of data. Plan for this; do not try to throw additional hardware at a cluster at the last minute. I really would like to know what's the

Re: After column deletion cassandra won't insert more data to a specific key

2011-07-25 Thread Guillermo Winkler
Hi, thanks both for the answers. The problem was indeed with the timestamps. What was happening also was that in a mutation involving 1 deletion and various insertions for the same key, all were using the same timestamp, so beside looking at the code doing this remove key insert key, col, val

Re: cassandra server disk full

2011-07-25 Thread Ryan King
We have a patch somewhere that will kill the node on IOErrors, since those tend to be of the class that are unrecoverable. -ryan On Thu, Jul 7, 2011 at 8:02 PM, Jonathan Ellis jbel...@gmail.com wrote: Yeah, ideally it should probably die or drop into read-only mode if it runs out of space.

Re: Counter consistency - are counters idempotent?

2011-07-25 Thread Aaron Turner
On Sun, Jul 24, 2011 at 3:36 PM, aaron morton aa...@thelastpickle.com wrote: What's your use case ? There are people out there having good times with counters, see http://www.slideshare.net/kevinweil/rainbird-realtime-analytics-at-twitter-strata-2011

Re: cassandra server disk full

2011-07-25 Thread Ryan King
Actually I was wrong– our patch will disable gosisp and thrift but leave the process running: https://issues.apache.org/jira/browse/CASSANDRA-2118 If people are interested in that I can make sure its up to date with our latest version. -ryan On Mon, Jul 25, 2011 at 10:07 AM, Ryan King

Re: Counter consistency - are counters idempotent?

2011-07-25 Thread Sylvain Lebresne
On Mon, Jul 25, 2011 at 7:35 PM, Aaron Turner synfina...@gmail.com wrote: On Sun, Jul 24, 2011 at 3:36 PM, aaron morton aa...@thelastpickle.com wrote: What's your use case ? There are people out there having good times with counters, see

Re: Counter consistency - are counters idempotent?

2011-07-25 Thread Aaron Turner
On Mon, Jul 25, 2011 at 11:24 AM, Sylvain Lebresne sylv...@datastax.com wrote: On Mon, Jul 25, 2011 at 7:35 PM, Aaron Turner synfina...@gmail.com wrote: On Sun, Jul 24, 2011 at 3:36 PM, aaron morton aa...@thelastpickle.com wrote: What's your use case ? There are people out there having good

sstableloader throws storage_port error

2011-07-25 Thread John Conwell
I'm trying to figure out how to use the sstableloader tool. For my test I have a single node cassandra instance running on my local machine. I have cassandra running, and validate this by connecting to it with cassandra-cli. I run sstableloader using the following command: bin/sstableloader

Re: cassandra server disk full

2011-07-25 Thread aaron morton
If the commit log or data disk is full it's not possible for the server to process any writes, the best it could do is perform reads. But reads may result in a write due to read repair and will also need to do some app logging, so IMHO it's really down / dead. You should free space and

Re: Cassandra 0.7.8 and 0.8.1 fail when major compaction on 37GB database

2011-07-25 Thread aaron morton
How much memory you need depends on a few things such as how many CF's you have, what your data is like, and what the usage patterns are like. There is no exact formula. Generally… * i would say 4GB of JVM heap is a good start * key and row caches are set when the CF is created, see help

Re: do I need to add more nodes? minor compaction eat all IO

2011-07-25 Thread aaron morton
There are no hard and fast rules to add new nodes, but here are two guidelines: 1) Single node load is getting too high, rule of thumb is 300GB is probably too high. 2) There are times when the cluster cannot keep up with throughout, for example the client is getting TimedOutExceptions or

Re: After column deletion cassandra won't insert more data to a specific key

2011-07-25 Thread aaron morton
It's just not possible to control time, as many super villains and Peter Schuller have shown us http://www.mail-archive.com/user@cassandra.apache.org/msg15636.html Often it's not necessary, you can design around simultaneous updates the same key, use a coordination layer such as zoo keeper or

Re: After column deletion cassandra won't insert more data to a specific key

2011-07-25 Thread Guillermo Winkler
I guess the problem it's not whether you can control time in a distributed system or not, but in this case at least, it's if you consider a timestamp set by a client outside the cluster as *safe*. When the timestamp gets hidden behind a client/wrapper library implementation default, realizing

Re: Interpreting the output of cfhistograms

2011-07-25 Thread Maki Watanabe
Offset represent different units for each columns. On SSTables columns, you can see following histgrams: 20 4291637 24 28680590 29 3876198 It means your 4291637 read operations required 20 SStables to read, 28680590 ops required 24, so on. In Write/Read latency columns, Offset

Re: Repair fails with java.io.IOError: java.io.EOFException

2011-07-25 Thread Sameer Farooqui
Looks like the repair finished successfully the second time. However, the cluster is still severely unbalanced. I was hoping the repair would balance the nodes. We're using random partitioner. One node has 900GB and others have 128GB, 191GB, 129GB, 257 GB, etc. The 900GB and the 646GB are just

Re: Repair fails with java.io.IOError: java.io.EOFException

2011-07-25 Thread aaron morton
Background: http://wiki.apache.org/cassandra/Operations Use node tool ring to check if the tokens are evenly distributed. If not then check the Load Balancing and Moving Nodes sections in the page above. If they are and repair has completed use node tool cleanup to remove the data the node is

Re: sstableloader throws storage_port error

2011-07-25 Thread Jonathan Ellis
sstableloader uses gossip to discover the Cassandra ring, so you'll need to run it on a different IP (127.0.0.2 is fine). On Mon, Jul 25, 2011 at 2:41 PM, John Conwell j...@iamjohn.me wrote: I'm trying to figure out how to use the sstableloader tool.  For my test I have a single node cassandra

Re: Cassandra 0.7.8 and 0.8.1 fail when major compaction on 37GB database

2011-07-25 Thread lebron james
I have only 4GB on server, so i give jvm 3 GB of heap, but this dont help, cassandra still fall when i launch major compaction on 37 GB database.