Re: maven repository

2011-04-05 Thread Stephen Connolly
maven central - Stephen --- Sent from my Android phone, so random spelling mistakes, random nonsense words and other nonsense are a direct result of using swype to type on the screen On 5 Apr 2011 06:47, Mikael Wikblom mikael.wikb...@sitevision.se wrote: Hi, is there a maven repository where I

Re: Abnormal memory consumption

2011-04-05 Thread Peter Schuller
Okay, I see. But isn't there a big issue for scaling here ? Imagine that I am the developper of a certain very successful website : At year 1 I need 20 CF. I might need to have 8Gb of RAM. Year 2 I need 50 CF because I added functionalities to my wonderful webiste will I need 20 Gb of RAM ?

Re: Re: nodetool cleanup - results in more disk use?

2011-04-05 Thread jonathan . colby
I think the key thing to remember is that compaction is performed on *similar* sized sstables. so it makes sense that over time this will have a cascading effect. I think by default it starts out with compacting 4 flushed sstables, then the cycle begins. On Apr 4, 2011 3:42pm, shimi

Renaming keyspace

2011-04-05 Thread Jean-Yves LEBLEU
Hi all, We are running .6.12, is there any particular precaution to rename the keyspace, is it enough to shutdown cassandra, update storag-conf.xml, rename data directory and start cassandra again. Thanks for your help. Jean-Yves

batch_mutate failed: out of sequence response

2011-04-05 Thread Héctor Izquierdo Seliva
Hi everyone. I'm having trouble while inserting big amounts of data into cassandra. I'm getting this exception: batch_mutate failed: out of sequence response I'm gessing is due to very big mutates. I have made the batch mutates smaller and it seems to be behaving. Can somebody shed some light?

Re: Renaming keyspace

2011-04-05 Thread Jonathan Ellis
That should work, but nodetool drain before shutdown to avoid confusing the commitlog. On Tue, Apr 5, 2011 at 3:56 AM, Jean-Yves LEBLEU jleb...@gmail.com wrote: Hi all, We are running .6.12, is there any particular precaution to rename the keyspace, is it enough to shutdown cassandra, update

RE: batch_mutate failed: out of sequence response

2011-04-05 Thread Dan Hendry
I too have seen the out of sequence response problem. My solution has just been to retry and it seems to work. None of my mutations are THAT large ( 200 columns). The only related information I could find points to a thrift/ubuntu bug of some kind

Re: too many open files - maybe a fd leak in indexslicequeries

2011-04-05 Thread Jonathan Ellis
sounds like they haven't been munmapped yet. try forcing a GC. On Sat, Apr 2, 2011 at 5:38 AM, Roland Gude roland.g...@yoochoose.com wrote: Hi, The open file limit is 1024 Sstable count is somewhere around 20 or so thread count is in the same order of magnitude I guess But lsof shows

RE: batch_mutate failed: out of sequence response

2011-04-05 Thread Héctor Izquierdo Seliva
El mar, 05-04-2011 a las 09:35 -0400, Dan Hendry escribió: I too have seen the out of sequence response problem. My solution has just been to retry and it seems to work. None of my mutations are THAT large ( 200 columns). The only related information I could find points to a thrift/ubuntu

Re: Error messages after rolling updating cassandra from 0.7.0 to 0.7.2

2011-04-05 Thread Jonathan Ellis
Oops, I saw EOFException and jumped to scrub. But your EOF is coming from TCP. Something (almost certainly a non-cassandra process) is connecting to the internal Cassandra communication port (the one that defaults to 7000) and disconnecting. On Mon, Apr 4, 2011 at 4:14 AM, Kazuo YAGI

Re: index file contains a different key or row size

2011-04-05 Thread Jonathan Ellis
You're right, the advice there is not helpful. On Mon, Apr 4, 2011 at 10:13 AM, shimi shim...@gmail.com wrote: It make sense to me that compaction should solved this as well since compaction creates new index files. Am I missing something here? WARN [CompactionExecutor:1] 2011-04-04

Re: IndexInterval Tuning

2011-04-05 Thread Jonathan Ellis
Adjusting indexinterval is unlikely to be useful on very narrow rows. (Its purpose is to make random access to _large_ rows doable.) On Mon, Apr 4, 2011 at 8:38 PM, Chris Burroughs chris.burrou...@gmail.com wrote: I have a case with very narrow rows.  As such I have a large row cache that does

Re: Error messages after rolling updating cassandra from 0.7.0 to 0.7.2

2011-04-05 Thread Sasha Dolgy
I've been seeing this EOF in my system.log file occasionally as well. Doesn't seem to be causing harm: ERROR [Thread-22] 2011-04-05 20:37:22,562 AbstractCassandraDaemon.java (line 112) Fatal exception in thread Thread[Thread-22,5,main] java.io.IOError: java.io.EOFException at

RE: mmap segment underflow

2011-04-05 Thread Or Yanay
I am considering treating the node as a dead node, delete it's data files and bootstrap from scratch. Is that a reasonable approach? Is there a way for me to identify the bad file/s and remove it/them? From: Or Yanay [mailto:o...@peer39.com] Sent: Monday, April 04, 2011 7:19 PM To:

Re: IndexInterval Tuning

2011-04-05 Thread Jonathan Ellis
On Tue, Apr 5, 2011 at 8:54 AM, Jonathan Ellis jbel...@gmail.com wrote: Adjusting indexinterval is unlikely to be useful on very narrow rows. (Its purpose is to make random access to _large_ rows doable.) Whoops, that's column_index_size_in_kb. I'd play w/ keycache before index_interval

Re: AW: Strange nodetool repair behaviour

2011-04-05 Thread Jonas Borgström
On 04/05/2011 03:49 PM, Jonathan Ellis wrote: Sounds like https://issues.apache.org/jira/browse/CASSANDRA-2324 Yes, that sounds like the issue I'm having. Any chance for a fix for this being backported to 0.7.x? Anyway, I guess I might as well share the test case I've used to reproduce this

if nodetool operations abort with timeout, did the operation continue?

2011-04-05 Thread Jonathan Colby
when doing a nodetool move , after about 15 minutes I got the below exception. The cassandra log seems to indicate that the move is still ongoing. Is this anything to worry about? Exception in thread main java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is:

Re: IndexInterval Tuning

2011-04-05 Thread Chris Burroughs
On 04/05/2011 09:57 AM, Jonathan Ellis wrote: On Tue, Apr 5, 2011 at 8:54 AM, Jonathan Ellis jbel...@gmail.com wrote: Adjusting indexinterval is unlikely to be useful on very narrow rows. (Its purpose is to make random access to _large_ rows doable.) Whoops, that's column_index_size_in_kb.

RE: batch_mutate failed: out of sequence response

2011-04-05 Thread Héctor Izquierdo Seliva
I'm still running into problems. Now I don't write more than 100 columns at a time, and I'm having lots of Stop-the-world gc pauses. I'm writing into three column families, with memtable_operations = 0.3 and memtable_throughput = 64. Is any of this wrong? -Original Message- From:

RE: batch_mutate failed: out of sequence response

2011-04-05 Thread Héctor Izquierdo Seliva
Update with more info: I'm still running into problems. Now I don't write more than 100 columns at a time, and I'm having lots of Stop-the-world gc pauses. I'm writing into three column families, with memtable_operations = 0.3 and memtable_throughput = 64. There is now swapping, and full GCs

Re: nodetool repair compact

2011-04-05 Thread Sylvain Lebresne
On Tue, Apr 5, 2011 at 12:01 AM, Maki Watanabe watanabe.m...@gmail.com wrote: Hello, On reading O'Reilly's Cassandra book and wiki, I'm a bit confusing on nodetool repair and compact. I believe we need to run nodetool repair regularly, and it synchronize all replica nodes at the end.

Re: LB scenario

2011-04-05 Thread Dave Viner
AJ, One issue that I found in using load balancer in front of cassandra nodes is that a single node might become bogged down by compaction, or other actions unrelated to the client. If the load balancer does not pick this up in time, it might route client requests to the node that is temporarily

Re: LB scenario

2011-04-05 Thread Peter Schuller
Can someone comment on this ? Or is the question too vague ? Honestly yeah I couldn't figure out what you were asking ;) What specifically about the diagram are you trying to convey? -- / Peter Schuller

Re: batch_mutate failed: out of sequence response

2011-04-05 Thread Jonathan Ellis
Step 1: disable swap. 2011/4/5 Héctor Izquierdo Seliva izquie...@strands.com: Update with more info: I'm still running into problems. Now I don't write more than 100 columns at a time, and I'm having lots of Stop-the-world gc pauses. I'm writing into three column families, with

Re: if nodetool operations abort with timeout, did the operation continue?

2011-04-05 Thread Jonathan Ellis
Other than you will have to watch log to know when it is done, no. On Tue, Apr 5, 2011 at 9:54 AM, Jonathan Colby jonathan.co...@gmail.com wrote: when doing a nodetool move , after about 15 minutes I got the below exception.   The cassandra log seems to indicate that the move is still

Re: OldNetworkTopology and Quorum

2011-04-05 Thread Jonathan Ellis
On Tue, Apr 5, 2011 at 9:50 AM, Jonathan Colby jonathan.co...@gmail.com wrote: Hi experts, We have a cluster of 4 nodes across 2 data centers and we are using OldNetworkTopology to balance the replicas (RF=3). Question:    Can we afford to lose 1 node in a data center and still have

Minor Follow-up: reduced cached mem; resident set size growth

2011-04-05 Thread Chris Burroughs
This is a minor followup to this thread which includes required context: http://www.mail-archive.com/user@cassandra.apache.org/msg09279.html I haven't solved the problem, but since negative results can also be useful I thought I would share them. Things I tried unsuccessfully (on individual

Disable Swap? batch_mutate failed: out of sequence response

2011-04-05 Thread Jonathan Colby
Hi Jonathan - Would you recommend to disable system swap as a rule? I'm running on Debian 64bit and am seeing light swapping: total used free sharedbuffers cached Mem: 8003 7969 33 0 0 4254 -/+ buffers/cache:

Re: Disable Swap? batch_mutate failed: out of sequence response

2011-04-05 Thread Peter Schuller
Would you recommend to disable system swap as a rule?   I'm running on Debian 64bit and am seeing light swapping: I'm not Jonathan, but *yes*. I would go so far as to say that disabling swap is a good rule of thumb for *most* production systems that serve latency sensitive traffic. For a

extreme memory consumption

2011-04-05 Thread Jonathan Colby
I've seen the other posts about memory consumption, but I'm seeing some weird behavior with 0.7.4 with 5 GB heap size (64 bit system with 8 GB ram total)... note the virtual mem used 20.6 GB ?! and Shared 8.4 GB ?! PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND

Re: extreme memory consumption

2011-04-05 Thread Peter Schuller
note the virtual mem used 20.6 GB ?!   and Shared 8.4 GB ?! http://wiki.apache.org/cassandra/FAQ#mmap -- / Peter Schuller

running compaction from a machine outside the cluster

2011-04-05 Thread Anurag Gujral
Hi All, Is there a way I can run compaction on the cassandra cluster from a machine where cassandra is not installed.I have a cluster of 6 machines but I want to run compaction on them from a different machine which does not have cassandra installed. Thanks Anurag

Re: Minor Follow-up: reduced cached mem; resident set size growth

2011-04-05 Thread Peter Schuller
- Different collectors: -XX:+UseParallelGC -XX:+UseParallelOldGC Unless you also removed the -XX:+UseConcMarkSweepGC I *think* it takes precedence, so that the above options would have no effect. I didn't test. In either case, did you definitely confirm CMS was no longer being used? (Should be

Re: running compaction from a machine outside the cluster

2011-04-05 Thread Peter Schuller
Is there a way I can run compaction on the cassandra cluster from a machine where cassandra is not installed.I have a cluster of 6 machines but I want to run compaction on them from a different machine which does not have cassandra installed. nodetool -h your-remote-host compact,

nothing happening in the cluster after a nodetool move

2011-04-05 Thread Jonathan Colby
I added a node to the cluster and I am having a difficult time reassigning the new tokens. It seems after a while nothing shows up in the new node's logs and it just stays in status Leaving. nodetool netstats on all nodes shows Nothing streaming to/from. There is no activity in the other

Update: Re: nothing happening in the cluster after a nodetool move

2011-04-05 Thread Jonathan Colby
Well, since my last post, about 10 minutes later, the node goes into bootstrap mode. It's kind of worrying that a lot of time goes by where it seems like nothing is happening, then all of a sudden things get going again. 22,584 keys. Time: 20,276ms. INFO [HintedHandoff:1] 2011-04-05

Re: running compaction from a machine outside the cluster

2011-04-05 Thread sridhar basam
If you can reach your jmx ip/port, you can use any jmx client to start a compaction. Use jconsole to connect to your jmx ip/port and then navigate to mbeans-org.apache.cassandra.db-columnfamilies- -operations Underneath there you can invoke a bunch of methods including compaction. Sridhar

Thrift version

2011-04-05 Thread Jeremiah Jordan
Anyone know if 0.7.4 will work with thirft 0.6? Or do I have to keep thrift 0.5 around to use it? Thanks! Jeremiah Jordan Application Developer Morningstar, Inc. Morningstar. Illuminating investing worldwide. +1 312 696-6128 voice

Location-aware replication based on objects' access pattern

2011-04-05 Thread Yudong Gao
Hi, I am thinking about using Cassandra for our research project, and we are thinking about one interesting feature. Our setup has multiple datacenters located in different geography locations. Data is accessed with predictable patterns. Think of something like Craigslist, data objects

Re: Thrift version

2011-04-05 Thread Jonathan Ellis
0.6 should be wire-compatible with 0.5 -- i.e., go ahead and build your clients with it. there's a few changes needed though if you want to build the server against 0.6: https://issues.apache.org/jira/browse/CASSANDRA-2412 On Tue, Apr 5, 2011 at 4:28 PM, Jeremiah Jordan

Re: Location-aware replication based on objects' access pattern

2011-04-05 Thread Jonathan Ellis
You'd really want https://issues.apache.org/jira/browse/CASSANDRA-2369 to control per-row. Let me know if you'd like to help tackle that. On Tue, Apr 5, 2011 at 5:05 PM, Yudong Gao st...@umich.edu wrote: Hi, I am thinking about using Cassandra for our research project, and we are thinking

Re: RE: batch_mutate failed: out of sequence response

2011-04-05 Thread Dan Washusen
Pelops raises a RuntimeException? Can you provide more info please? -- Dan Washusen Make big files fly visit digitalpigeon.com On Tuesday, 5 April 2011 at 11:43 PM, Héctor Izquierdo Seliva wrote: El mar, 05-04-2011 a las 09:35 -0400, Dan Hendry escribió: I too have seen the out of sequence

old JMX code is not working with new cassandra version

2011-04-05 Thread Anurag Gujral
Hi All, I had written code for cassandra 0.6.3 using JMX to call compaction,when I try to use that code to connect to 0.7.3 I get the following error Exception in thread main java.rmi.ConnectException: Connection refused to host: 127.0.0.1; nested exception is:

Re: Location-aware replication based on objects' access pattern

2011-04-05 Thread Yudong Gao
Thanks for the reply, Jonathan! This per-row control is exactly what I need. I will be happy to help tackle it in the long term. Is there some further information or plan for this issues? One thing I am worrying about is how to maintain the location information for each row. The current

Re: Location-aware replication based on objects' access pattern

2011-04-05 Thread Jonathan Ellis
On Tue, Apr 5, 2011 at 8:37 PM, Yudong Gao st...@umich.edu wrote: One thing I am worrying about is how to maintain the location information for each row. The current partitioner maps a key to MD5 hash, and it is almost impossible to control the hashed token by manipulating the value of the

Re: nodetool repair compact

2011-04-05 Thread Maki Watanabe
Thanks Sylvain, it's very clear. But should I still need to force major compaction regularly to clear tombstones? I know that minor compaction clear the tombstones after 0.7, but maximumCompactionThreshold limits the maximum number of sstable which will be merged at once, so to GC all tombstones

Re: Location-aware replication based on objects' access pattern

2011-04-05 Thread Yudong Gao
On Tue, Apr 5, 2011 at 9:59 PM, Jonathan Ellis jbel...@gmail.com wrote: On Tue, Apr 5, 2011 at 8:37 PM, Yudong Gao st...@umich.edu wrote: One thing I am worrying about is how to maintain the location information for each row. The current partitioner maps a key to MD5 hash, and it is almost

Re: Location-aware replication based on objects' access pattern

2011-04-05 Thread Jonathan Ellis
On Tue, Apr 5, 2011 at 10:45 PM, Yudong Gao st...@umich.edu wrote: A better solution would be to just push the DecoratedKey into the ReplicationStrategy so it can make its decision before information is thrown away. I agree. So in this case, I guess the hashed based token ring is still

Re: Questions on combining custom with built-in secondary indexes

2011-04-05 Thread Miroslav Madecki
Hi Aaron, I failed to mention that my queries have very high frequency so this is not a data analytics problem. That's the reason I wanted to reduce transferring a lot of data to client or using standard map/reduce solutions that can do the job but add too much latency. Pig or hive are

Re: old JMX code is not working with new cassandra version

2011-04-05 Thread Narendra Sharma
I think you need to specify the port in the JMXServiceURL. The exception indicates there is no service listening on given host and port. Also, I guess, based on 127.0.0.1, you are running the client on same m/c as Cassandra. If that is not the case then fix the host as well. You might want to look

Compaction threshold does not save with nodetool

2011-04-05 Thread Sheng Chen
Cassandra 0.7.4 # nodetool -h localhost getcompactionthreshold Keyspace1 Standard1 min=4 max=32 # nodetool -h localhost setcompactionthreshold Keyspace1 Standard1 0 0 # nodetool -h localhost getcompactionthreshold Keyspace1 Standard1 min=0 max=0 Now the thresholds have changed on the JMX pannel,