Re: On 0.6.6 to 0.7.3 migration, DC-aware traffic and minimising data transfer

2011-03-14 Thread Jedd Rashbrooke
Jonathon, thank you for your answers here. To explain this bit ... On 11 March 2011 20:46, Jonathan Ellis jbel...@gmail.com wrote: On Thu, Mar 10, 2011 at 6:06 AM, Jedd Rashbrooke j...@visualdna.com wrote:  Copying a cluster between AWS DC's:  We have ~ 150-250GB per node

On 0.6.6 to 0.7.3 migration, DC-aware traffic and minimising data transfer

2011-03-10 Thread Jedd Rashbrooke
Howdi, Assortment of questions relating to an upgrade combined with a possible migration between Data Centers (or perhaps a multi-DC redesign). Apologies if some of these have been asked before - I have kept half an eye on the list in recent times but haven't seen anything covering these

Re: Memory leak with Sun Java 1.6 ?

2010-12-16 Thread Jedd Rashbrooke
Hi Peter, I've read through the Very high memory utilization (not caused by mmap on sstables) and the impact sounds similar .. alas they're using openJDK and I'm using Sun's. Of course, it's not impossible that the same bug is present in both JVM's. j.

Re: Memory leak with Sun Java 1.6 ?

2010-12-15 Thread Jedd Rashbrooke
Hey Peter, On 14 December 2010 20:19, Peter Schuller peter.schul...@infidyne.com wrote: So, now that I get that we have two different cases ;) Yup. My problem is java / environment based, occurring after several weeks, using 0.6.6 instances. Our thread hijacking friend / learned

Re: Memory leak with Sun Java 1.6 ?

2010-12-14 Thread Jedd Rashbrooke
Peter, Jonathon - thank you for your replies. I should probably have repeated myself in the body, but as I mentioned in the subject line, we're running Sun Java 1.6. On 10 December 2010 18:37, Peter Schuller peter.schul...@infidyne.com wrote: Memory-mapped files will account for both

KeysCached - clarification and recommendations on settings

2010-10-18 Thread Jedd Rashbrooke
Greetings, I would like to check my understanding is accurate on how KeysCached is understood by Cassandra (0.6.5), and then get some suggestions on settings / OS FS cache interplay. First - my initial understanding was that if you set KeysCached to 100%, Cassandra would do a best effort

Re: Dazed and confused with Cassandra on EC2 ...

2010-10-04 Thread Jedd Rashbrooke
Hi Peter, Thanks again for your time and thoughts on this problem. We think we've got a bit ahead of the problem by just scaling back (quite savagely) on the rate that we try to hit the cluster. Previously, with a surplus of optimism, we were throwing very big Hadoop jobs at Cassandra,

Re: Hardware change of a node in the cluster

2010-10-04 Thread Jedd Rashbrooke
On 4 October 2010 10:58, Utku Can Topçu u...@topcu.gen.tr wrote: Recently I've tried to upgrade (hw upgrade) one of the nodes in my cassandra cluster from ec2-small to ec2-large. Something that bit me on this (I've done it with both Cassandra and Hadoop boxes, and some problems might be more

Re: Dazed and confused with Cassandra on EC2 ...

2010-09-28 Thread Jedd Rashbrooke
Peter - my apologies for the slow response - we had to divert down a 'Plan B' approach last week involving MySQL, memcache, redis and various other uglies. On 20 September 2010 23:11, Peter Schuller peter.schul...@infidyne.com wrote: Are you running an old JVM by any chance? (Just grasping

Dazed and confused with Cassandra on EC2 ...

2010-09-17 Thread Jedd Rashbrooke
Howdi, I've just landed in an experiment to get Cassandra going, and fed by PHP via Thrift via Hadoop, all running on EC2. I've been lurking a bit on the list for a couple of weeks, mostly reading any threads with the word 'performance' in them. Few people have anything polite to say

Re: Dazed and confused with Cassandra on EC2 ...

2010-09-17 Thread Jedd Rashbrooke
Hi Dave, Thank you for your response. I can clarify a couple of things here: 2. You grew from 2 nodes to 4, but the original 2 nodes have 200GB and the 2 new ones have 40 GB.  What's the recommended practice for rebalancing (i.e., when should you do it), what's the actual procedure, and

Re: Dazed and confused with Cassandra on EC2 ...

2010-09-17 Thread Jedd Rashbrooke
Hi Rob, Thanks for your suggestions. I should have been a bit more verbose in my platform description -- I'm using 64-bit instances, which I think in a Ben Black video I saw led to a sensible default usage of mmap when left at auto. Should I look at forcing this setting? You don't