Re: commit logs are not deleted

2009-09-25 Thread Igor Katkov
I checked out and built 0.4 branch. It's all the same, files stays. I also noticed a side effect - as number of commit log segments is growing, server response time is also growing. I assume this is because Cassandra now has to read through some these files on reach read/write request On Fri, Sep

Re: read repair keeps occurring on every quorum read

2009-09-25 Thread Jonathan Ellis
No, you're mixing two related concepts. When you do a quorum read it will fetch the actual data from one replica and do digest reads from the others. If the data from the one does not match the hash from the others, then you have the digestmismatchexception Edmond is seeing and read repair is per

Re: read repair keeps occurring on every quorum read

2009-09-25 Thread Sandeep Tata
This is a known issue, and we should perhaps open a JIRA on it. The original Dynamo approach was to have 3 mechanisms -- HintedHandoff, read-repair, and Merk trees to guarantee convergence (eventual consistency). Cassandra only has the first two. There are some corner cases where hinted-handoff alo

read repair keeps occurring on every quorum read

2009-09-25 Thread Edmond Lau
I have a 3 node cluster with a replication factor of 2, running on 0.4 RC1. I've set both my read and write consistency levels to use a quorum. I'm observing that quorum reads keep invoking read repair and log DigestMismatchExceptions from the StorageProxy. Obviously, this significantly reduces

Re: commit logs are not deleted

2009-09-25 Thread Jonathan Ellis
This is fixed on the 0.4 branch (but not in trunk, yet) On Fri, Sep 25, 2009 at 1:57 PM, Jonathan Ellis wrote: > https://issues.apache.org/jira/browse/CASSANDRA-455 will address > FlushPeriod not working. > > On Fri, Sep 25, 2009 at 1:33 PM, Igor Katkov wrote: >> I tried latest stable version 0.

Re: differences between keyspaces and tables

2009-09-25 Thread Michael Greene
0.4 should be released very soon, pending approval by our Apache Incubator. 0.5 is the version currently under active development. Michael Side note: you may have been using an older version of that blog post. Evan seems to be updating it to incorporate changes, and I see on that page now. On F

Re: differences between keyspaces and tables

2009-09-25 Thread Jonathan Ellis
0.4 RC2 is better than anything using :) -final should be out soon. On Fri, Sep 25, 2009 at 3:27 PM, Joe Van Dyk wrote: > Ah, ok.  I was using > http://blog.evanweaver.com/articles/2009/07/06/up-and-running-with-cassandra/ > and didn't realize that it wasn't using the stable version of > Cassan

Re: differences between keyspaces and tables

2009-09-25 Thread Joe Van Dyk
Ah, ok. I was using http://blog.evanweaver.com/articles/2009/07/06/up-and-running-with-cassandra/ and didn't realize that it wasn't using the stable version of Cassandra. If starting a new project, is it recommended to use the latest beta version? Joe On Fri, Sep 25, 2009 at 1:00 PM, Michael Gr

Re: differences between keyspaces and tables

2009-09-25 Thread Michael Greene
They were renamed between 0.3 and 0.4. They are the same thing.See https://issues.apache.org/jira/browse/CASSANDRA-271 Michael On Fri, Sep 25, 2009 at 2:58 PM, Joe Van Dyk wrote: > Hi, > > In some storage.conf's, I see and in others I see . > > Are they the same thing? > > -- > Joe Van Dyk > h

differences between keyspaces and tables

2009-09-25 Thread Joe Van Dyk
Hi, In some storage.conf's, I see and in others I see . Are they the same thing? -- Joe Van Dyk http://fixieconsulting.com

Re: commit logs are not deleted

2009-09-25 Thread Jonathan Ellis
https://issues.apache.org/jira/browse/CASSANDRA-455 will address FlushPeriod not working. On Fri, Sep 25, 2009 at 1:33 PM, Igor Katkov wrote: > I tried latest stable version 0.3 and commit logs segments are in fact > deleted. > Tried it again on 0.4 set  periodic flush to 1min > (FlushPeriodInMi

Re: commit logs are not deleted

2009-09-25 Thread Igor Katkov
I tried latest stable version 0.3 and commit logs segments are in fact deleted. Tried it again on 0.4 set periodic flush to 1min (FlushPeriodInMinutes="1") => it's all the same, files remains there forever. I also noticed that there are other implicit CFs, can these prevent logs from being delete

Re: triggers in cassandra

2009-09-25 Thread Hector Yuen
Michael I will check that out Thanks On Fri, Sep 25, 2009 at 10:35 AM, Michael Greene wrote: > Additionally, Cassandra client connections and the Thrift API are not > designed to be stateful. For what it's worth, Apache Zookeeper offers such > change notifications, but has a very different arc

Re: triggers in cassandra

2009-09-25 Thread Michael Greene
Additionally, Cassandra client connections and the Thrift API are not designed to be stateful. For what it's worth, Apache Zookeeper offers such change notifications, but has a very different architecture. Michael On Fri, Sep 25, 2009 at 12:25 PM, Jonathan Ellis wrote: > That's a bad fit for Ca

Re: triggers in cassandra

2009-09-25 Thread Jonathan Ellis
That's a bad fit for Cassandra, since the machine any given client talks to is unlikely to be the machine that data got sent to. So you'd have to send the notifications from Client1 -> cassandra1 -> cassandra data node -> cassandra2 -> client 2 It would complicate things significantly. -Jonatha

Re: triggers in cassandra

2009-09-25 Thread Hector Yuen
Michael, What I want to do is to use a column as a place for keeping state between different clients. I would register callbacks for a column, so when there is any other guy changing the data I can get notified. That way I don't need to do continuous polling. Is there any functionality right now

Re: perfomance issue

2009-09-25 Thread Kirill A . Korinskiy
At Fri, 25 Sep 2009 19:23:59 +0400, Kirill A. Korinskiy wrote: my erlang code for test as attached. I use: cassandra from git: dd1688cf859b281a32ad28e19ebf01919d89e2b6 thrift: version 20080411-exported from freebsd ports Erlang R13B01 test_cassandra.erl Description: Binary data cassandra_cl

Re: perfomance issue

2009-09-25 Thread Kirill A . Korinskiy
At Fri, 25 Sep 2009 10:34:06 -0500, Michael Greene wrote: > > Thanks for the results.  Perhaps you could shed further light: > Is this a single node system? Ooops. I'm missing. Nope. It's a cluster. I use a 3 node and ReplicationFactor set to 2. > Is the log level changed from DEBUG to INFO?

Re: perfomance issue

2009-09-25 Thread Jonathan Ellis
I still have no idea what your test looks like or what your numbers mean. So how can I guess if it's expected? As a very rough rule of thumb you can expect about 1000 ops per second on a low-end-ish system. So if you are getting that much out of cassandra at least you are in the right ball park.

Re: perfomance issue

2009-09-25 Thread Kirill A . Korinskiy
At Fri, 25 Sep 2009 10:32:55 -0500, Jonathan Ellis wrote: > > Step zero in benchmarking cassandra is turning the log level to > INFO. sure, i'm switching off all log messages > Step one is testing on a machine where you can put the commitlog > directory on its own disk. you mean a dedicated di

Re: perfomance issue

2009-09-25 Thread Michael Greene
Thanks for the results. Perhaps you could shed further light:Is this a single node system? Is the log level changed from DEBUG to INFO? Are the commit log and data directories on the same drive? Are the sets/gets being processed interleaved in parallel, or one then the other? Note that writes are

Re: perfomance issue

2009-09-25 Thread Jonathan Ellis
It's impossible to say given what you have told us. Step zero in benchmarking cassandra is turning the log level to INFO. Step one is testing on a machine where you can put the commitlog directory on its own disk. It's true that frequently cassandra will be slower than custom code writing to loca

perfomance issue

2009-09-25 Thread Kirill A . Korinskiy
Hello. I'm write a small erlang benchmark: cassandra vs simple fs storage and I have a results: fs_storage: 10: 1732/228633/2873/1128 992/233261/2870/1026 25: 4531/35290/6084/632 1786/35292/5969/312 50: 4707/311825/13694/4644 1366/382913/14470/5823 cassandra: 10: 69014/424051/145871/28066 9553/4

Re: triggers in cassandra

2009-09-25 Thread Michael Greene
Hector, Can you describe explicitly what you'd want to see from callouts/triggers? One of the reasons I advocated for removal is that no one had a need for it or was working on it in the open source project. What's your scenario? Michael On Fri, Sep 25, 2009 at 9:40 AM, Eric Evans wrote: > On

Re: triggers in cassandra

2009-09-25 Thread Eric Evans
On Fri, 2009-09-25 at 02:17 -0700, Hector Yuen wrote: > I am just starting to use Cassandra, and I was wondering if it is > possible > to use triggers/ callbacks on columns so that they get notified when > there > is a change. > > Is there such functionality? No, there isn't. Someone was working

Re: Error message

2009-09-25 Thread Teodor Sigaev
If you can come up with a minimal script to reproduce that would be awesome. Script is in attachment (sorry, perl version). In short - it inserts a row and immediately removes it. I've run it three in parallel. Casssandra version is yesterday's cassandra-0.4.0-final. INFO - Saved Token not foun

triggers in cassandra

2009-09-25 Thread Hector Yuen
Hi all, I am just starting to use Cassandra, and I was wondering if it is possible to use triggers/ callbacks on columns so that they get notified when there is a change. Is there such functionality? Thanks -- -h