Re: Cannot start cassandra node anymore

2012-01-31 Thread huyle
Created https://issues.apache.org/jira/browse/CASSANDRA-3819. Thanks! Huy The schema change was that we created a new key space with composite type CFs, but later we had to change some definition/CF names, so we dropped the key space and recreated with new definition. sounds like a bug,

Re: Cannot start cassandra node anymore

2012-01-26 Thread huyle
This is on single node environment, and there is a lot of data in commit logs needs to be flushed to sstables. So remove commit logs will cause data lost. Fortunately, this issue happens on a new key space that have CFs that use composite type, and we can sacrifice loosing data on this new key

Re: Tried to restart a node, and now it won't come up

2011-12-13 Thread huyle
Looks like this is related to bug in https://issues.apache.org/jira/browse/CASSANDRA-3558. Show schema shows sstable compression chunk_length_kb on the node that the schame was applied is 65536, thought the schema update statement specified chunk_length_kb=64, and on other nodes on the

Re: Tried to restart a node, and now it won't come up

2011-12-13 Thread huyle
We got the server to start up by applying the patch from the mentioned JIRA, and modified the configuration parameters reader code to set chunk_lenghth_size to null on start so that cassandra will use the default value of 64k. We then deployed the code to the rest of the servers in the cluster,

Tried to restart a node, and now it won't come up

2011-12-12 Thread huyle
Hi, We are running 1.0.3. I tried to restart a node, but it won't come up. Below is the error logged: INFO [SSTableBatchOpen:1] 2011-12-13 04:14:52,095 SSTableReader.java (line 134) Opening /u01/cassandra/data/system/HintsColumnFamily-hb-114 (66 bytes) INFO [main] 2011-12-13 04:14:52,145

Re: decommissioned not show being gossipped

2011-12-02 Thread huyle
to apply the patch and compile new binary. We currently running 1.0.3. Thanks! Huy unsafeAssassinateEndpoint Brandon Williams wrote On Thu, Dec 1, 2011 at 1:10 PM, huyle lt;huyle@gt; wrote: The clocks are very sync'ed between the nodes as they have ntp running hitting our time servers

decommissioned not show being gossipped

2011-12-01 Thread huyle
Hi, We have 2 nodes have been decommissioned from the cluster running 1.0.3. However, the live nodes still making references to the decommissioned nodes 3 days after the nodes were decommissioned. Nodetool does not show the decommissioned noes. Here are sample log entries: INFO

Re: decommissioned not show being gossipped

2011-12-01 Thread huyle
The clocks are very sync'ed between the nodes as they have ntp running hitting our time servers. Huy -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/decommissioned-nodes-still-being-gossipped-tp7051526p7051701.html Sent from the

Re: is there a no disk storage mode ?

2011-12-01 Thread huyle
I am not sure of no disk option, but as for fast unit testing, you can try RAM disk for storage. Huy -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/is-there-a-no-disk-storage-mode-tp7051192p7051728.html Sent from the

Re: Decommission a node causing high IO usage on 2 other nodes

2011-11-29 Thread huyle
Yes, we do have RF=3. That explains it all. Thanks Peter! Huy Peter Schuller wrote What may be causing high IO wait on 2.new and 3.new? Do you have RF=3? The problem with 'nodetool ring' in terms of interpreting those '0%' is that it does not take RF and replication strategy into

Unable to instantiate cache provider org.apache.cassandra.cache.SerializingCacheProvider

2011-11-24 Thread huyle
Hi, Whenever I start up cassandra 1.0.3, I notice the following warning: WARN [main] 2011-11-24 07:57:13,129 CFMetaData.java (line 402) Unable to instantiate cache provider org.apache.cassandra.cache.SerializingCacheProvider; using default

Re: Added column does not sort as the last column

2011-11-22 Thread huyle
We don't use subcomparator . We doubled checked comparator, and it look fine. There was typo in my original post. I mean to say upgrade to 1.0.2 from 0.6.13. This issue is now resolved after we upgrade cass to 1.0.3. Thanks! Huy -- View this message in context:

Added column does not sort as the last column

2011-11-19 Thread huyle
Hi, We got Added column does not sort as the last column error in the logs after upgrading to cass 1.0.3 from 0.6.13. After running scrub, we still getting the error. Here is stack trace: java.lang.AssertionError: Added column does not sort as the last column at