RE: How know node is fully up?

2009-12-23 Thread Brian Burruss
, Brian Burruss bburr...@real.com wrote: I never heard from anyone about this. I think it is important for bringing nodes out of service during upgrades so no data loss occurs. Also when introducing a new node you need to know when it is fully populated. Tux! Brian Burruss bburr

RE: MultiThread Client problem with thrift

2009-12-22 Thread Brian Burruss
i don't close the connection to a server unless i get exceptions. and when i close the connection i try a new server in the cluster just to keep the connections spread across the cluster. should i be closing them? if the connection is closed by client or server i'll just reconnect.

Re: How know node is fully up?

2009-12-22 Thread Brian Burruss
I never heard from anyone about this. I think it is important for bringing nodes out of service during upgrades so no data loss occurs. Also when introducing a new node you need to know when it is fully populated. Tux! Brian Burruss bburr...@real.com wrote: How can i tell that a node

How know node is fully up?

2009-12-18 Thread Brian Burruss
How can i tell that a node is completely up and taking reads and writes? - at startup? - after new bootstrap? - after a node has been unavailable for some time and rejoins the cluster? i see the INFO [main] [CassandraDaemon.java:141] Cassandra starting up... message in the log, but it seems to

another OOM

2009-12-18 Thread Brian Burruss
this time i simulated node 1 crashing, waited a few minutes, then restarted it. after a while node 2 OOM'ed. same 2 node cluster with RF=2, W=1, R=1. i up'ed the RAM to 6G this time. cluster contains ~126,281,657 data elements containing about 298G on one node's disk thx! system-errors.log

RE: hard disk size

2009-12-18 Thread Brian Burruss
Ellis [jbel...@gmail.com] Sent: Friday, December 18, 2009 2:04 PM To: cassandra-user@incubator.apache.org Subject: Re: hard disk size Does nodeprobe compact to help w/ read latency? On Fri, Dec 18, 2009 at 12:38 PM, Brian Burruss bburr...@real.com wrote: what are other folks' database size

RE: another OOM

2009-12-18 Thread Brian Burruss
://wiki.apache.org/cassandra/Operations for how to track metrics that will tell you how much is too much. Telling us more about your workload would be useful in sanity checking that hypothesis. :) -Jonathan On Fri, Dec 18, 2009 at 4:34 PM, Brian Burruss bburr...@real.com wrote: this time i

RE: OOM Exception

2009-12-16 Thread Brian Burruss
beta2 fixes the using-more-memory-on-startup problem. On Wed, Dec 16, 2009 at 5:16 PM, Brian Burruss bburr...@real.com wrote: i'll put my question first: - how can i determine how much RAM is required by cassandra? (for normal operation and restarting server) *** i've attached my storage

RE: OOM Exception

2009-12-16 Thread Brian Burruss
To: cassandra-user@incubator.apache.org Subject: Re: OOM Exception sorry, i meant the system.log the 2nd time (clear it out before replaying so it's not confused w/ other info, pls) On Wed, Dec 16, 2009 at 5:39 PM, Brian Burruss bburr...@real.com wrote: is this what you want? they are big - i'd rather

create only - no update

2009-12-15 Thread Brian Burruss
can the cassandra client (java specifically) specify that a particular put should be create only, do not update? If the value already exists in the database, i want the put to fail. for instance, two users want the exact same username, so they both do a get to determine if the username

RE: read latency creaping up

2009-12-14 Thread Brian Burruss
investigating. thx! From: Eric Evans [eev...@rackspace.com] Sent: Monday, December 14, 2009 8:23 AM To: cassandra-user@incubator.apache.org Subject: RE: read latency creaping up On Sun, 2009-12-13 at 13:18 -0800, Brian Burruss wrote: if this isn't a known issue

Re: read latency creaping up

2009-12-14 Thread Brian Burruss
Well not sure how that would affect he latency as reported by the Cassandra server using nodeprobe cfstats Jonathan Ellis jbel...@gmail.com wrote: possibly the clients are running into memory pressure? On Mon, Dec 14, 2009 at 4:27 PM, Brian Burruss bburr...@real.com wrote: thx, i'm actually

RE: read latency creaping up

2009-12-14 Thread Brian Burruss
Burruss bburr...@real.com wrote: Well not sure how that would affect he latency as reported by the Cassandra server using nodeprobe cfstats Jonathan Ellis jbel...@gmail.com wrote: possibly the clients are running into memory pressure? On Mon, Dec 14, 2009 at 4:27 PM, Brian Burruss bburr

Re: read latency creaping up

2009-12-14 Thread Brian Burruss
15, 2009, at 10:16 AM, Brian Burruss wrote: i agree. i don't know anything about thrift, and i don't know how it keeps connections open or manages resources from a client or server perspective, but this situation suggests that maybe killing the clients is forcing the server to free

RE: OOM Exception

2009-12-13 Thread Brian Burruss
) From: Brian Burruss Sent: Saturday, December 12, 2009 7:45 AM To: cassandra-user@incubator.apache.org Subject: OOM Exception this happened after cassandra was running for a couple of days. I have -Xmx3G on JVM. is there any other info you need so this makes

RE: OOM Exception

2009-12-13 Thread Brian Burruss
i don't have a problem increasing memory, just need a rough formula for how much to expect to use. anyone recommend a good heap walker? i can then tell you what is taking the memory. thx! Node 1 -- Pool NameActive Pending Completed FILEUTILS-DELETE-POOL

RE: read latency creaping up

2009-12-13 Thread Brian Burruss
. From: Brian Burruss Sent: Sunday, December 13, 2009 1:14 PM To: cassandra-user@incubator.apache.org Subject: read latency creaping up i've noticed the longer i let my test clients run the higher the read latency becomes. if i kill the clients, the latency drops back down to a reasonable

OOM Exception

2009-12-12 Thread Brian Burruss
this happened after cassandra was running for a couple of days. I have -Xmx3G on JVM. is there any other info you need so this makes sense? thx! 2009-12-11 21:38:37,216 ERROR [HINTED-HANDOFF-POOL:1] [DebuggableThreadPoolExecutor.java:157] Error in ThreadPoolExecutor

RE: exception during startup

2009-12-08 Thread Brian Burruss
to finish the replay, that is the easiest workaround and you can set it back the way it was once replay completes successfully. On Mon, Dec 7, 2009 at 6:59 PM, Brian Burruss bburr...@real.com wrote: wanted to pass this along ... i have 2G RAM allocated to cassandra. should it need more? what

exception during startup

2009-12-07 Thread Brian Burruss
wanted to pass this along ... i have 2G RAM allocated to cassandra. should it need more? what are the factors that determine the amount of memory required? thx! 2009-12-07 16:56:30,787 ERROR [main] [CassandraDaemon.java:184] Exception encountered during startup. java.lang.OutOfMemoryError: