Offline migration: Random-Murmur

2013-12-23 Thread horschi
Hi list, has anyone ever tried to migrate a cluster from Random to Murmur? We would like to do so, to have a more standardized setup. I wrote a small (yet untested) utility, which should be able to read SSTable files from disk and write them into a cassandra cluster using Hector. This migration

Re: Offline migration: Random-Murmur

2013-12-23 Thread horschi
Interesting you even dare to do a live migration :-) Do you do all Murmur-writes with the timestamp from the Random-data? So that all migrated data is written with timestamps from the past. On Mon, Dec 23, 2013 at 3:59 PM, Rahul Menon ra...@apigee.com wrote: Christian, I have been planning

Re: Commitlog replay makes dropped and recreated keyspace and column family rows reappear

2013-12-23 Thread Aaron Morton
mmm, my bad there. First schema changes are always flushed to disk, so the commit log is not really an issue. Second when the commit log replays it just processes the mutations, the Drop keyspace” message comes from MigrationManager.announceKeyspaceDrop() and is not called. If you can

Re: Writes during schema migration

2013-12-23 Thread Aaron Morton
It depends a little on the nature of the change, but you need some coordination between the schema change and your code. e.g. add new column, change code to write to it or add new column, change code to use new column and not old column, remove old column. Cheers - Aaron

Re: How to tune cassandra to avoid OOM

2013-12-23 Thread Aaron Morton
Cassandra version is : apache-cassandra-1.2.4 The latest 1.2 version is 1.2.13, you really should be on that. commitlog_total_space_in_mb: 16 commitlog_segment_size_in_mb: 16 Reducing the total commit log size to 16 MB is a very bad idea, you should return it to 4096 and the segment size to

Re: Best way to measure write throughput...

2013-12-23 Thread Aaron Morton
nodetool proxyhistograms shows the throughput for the node, nodetool cfhistograms shows it for a single node. If you want to get an overview install something like Ops Centre http://www.datastax.com/what-we-offer/products-services/datastax-opscenter Cheers - Aaron Morton New

Re: Cassandra pytho pagination

2013-12-23 Thread Aaron Morton
First approach: Sounds good. Second approach ( I used in production ): If the row gets big enough this will have bad performance. A - Aaron Morton New Zealand @aaronmorton Co-Founder Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com On

Re: Cassandra pytho pagination

2013-12-23 Thread Aaron Morton
Is there something wrong with it? Here 1234555665_53323232 and 2344555665_53323232 are super columns. Also, If I have to represent this data with new composite comparator, How will I accomplish that? Composite types via pycassa

Re: WriteTimeoutException instead of UnavailableException

2013-12-23 Thread Aaron Morton
But in some cases, from one certain node, I get an WriteTimeoutException for a few minutes until an UnavailableException. It's like the coordinator don't know the status of the cluster. Any clue why is this happening? Depending on how the node goes down there can be a delay in other nodes

Re: Writes during schema migration

2013-12-23 Thread Ben Hood
Hey Aaron, Thanks for following up on this one. I guess you're right and in any case, you're unlikely to want two different versions of the code running at the same time. Cheers, Ben On Mon, Dec 23, 2013 at 5:42 PM, Aaron Morton aa...@thelastpickle.com wrote: It depends a little on the

Re: WriteTimeoutException on Lightweight transactions

2013-12-23 Thread Aaron Morton
Some background…. http://www.datastax.com/dev/blog/lightweight-transactions-in-cassandra-2-0 You can also get a timeout during the prepare phase, well anytime you are waiting on other node really. The WriteTimeoutException returned from the server includes a writeType

Re: Issue upgrading from 1.2 to 2.0.3

2013-12-23 Thread Aaron Morton
If this is still a concern can you post the output from nodetool gossipinfo ? It will give the details of the nodes think of the other ones. A - Aaron Morton New Zealand @aaronmorton Co-Founder Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com On

Broken pipe with Thrift

2013-12-23 Thread Vivek Mishra
Hi, I have a 6 node, 2DC cluster setup. I have configured consistency level to QUORUM. But very often i am getting Broken pipe com.impetus.client.cassandra.CassandraClientBase (CassandraClientBase.java:1926) - Error while executing native CQL query Caused by: .

Re: Broken pipe with Thrift

2013-12-23 Thread Vivek Mishra
Also to add. It works absolutely fine on single node. -Vivek On Tue, Dec 24, 2013 at 12:15 PM, Vivek Mishra mishra.v...@gmail.comwrote: Hi, I have a 6 node, 2DC cluster setup. I have configured consistency level to QUORUM. But very often i am getting Broken pipe

Re: Broken pipe with Thrift

2013-12-23 Thread Steven A Robenalt
Hi Vivek, Which release are you using? We had an issue with 2.0.2 that was solved by a fix in 2.0.3. On Mon, Dec 23, 2013 at 10:47 PM, Vivek Mishra mishra.v...@gmail.comwrote: Also to add. It works absolutely fine on single node. -Vivek On Tue, Dec 24, 2013 at 12:15 PM, Vivek Mishra

Re: Broken pipe with Thrift

2013-12-23 Thread Vivek Mishra
Hi Steven, Thanks for your reply. We are using version 1.2.9. -Vivek On Tue, Dec 24, 2013 at 12:27 PM, Steven A Robenalt srobe...@stanford.eduwrote: Hi Vivek, Which release are you using? We had an issue with 2.0.2 that was solved by a fix in 2.0.3. On Mon, Dec 23, 2013 at 10:47 PM,

Re: Broken pipe with Thrift

2013-12-23 Thread Vivek Mishra
Hi Steven, One question, which is confusing , it's a server side issue or client side? -Vivek On Tue, Dec 24, 2013 at 12:30 PM, Vivek Mishra mishra.v...@gmail.comwrote: Hi Steven, Thanks for your reply. We are using version 1.2.9. -Vivek On Tue, Dec 24, 2013 at 12:27 PM, Steven A