Re: Which of these VPS configurations would perform better for Cassandra ?

2013-08-07 Thread Janne Jalkanen
Well, Amazon is expensive. Hetzner will sell you dedicated SSD RAID-1 servers with 32GB RAM and 4 cores with HT for €59/mth. However, if pricing is an issue, you could start with: 1 server : read at ONE, write at ONE, RF=1. You will have consistency, but not high availability. This is the

cassandra disk access

2013-08-07 Thread Nikolay Mihaylov
Hi I am researching various hash-tables and b-trees on disk. while I researched, I has a thoughts about cassandra sstables that I want to verify it here. 1. cassandra sstable uses sequential disk I/O when created. e.g. disk head write it from the beginning to the end. Assuming the disk is not

Re: cassandra disk access

2013-08-07 Thread Michał Michalski
2. when cassandra lookups a key in sstable (assuming bloom-filter and other stuff failed, also assuming the key is located in this single sstable), cassandra DO NOT USE sequential I/O. She probably will read the hash-table slot or similar structure, then cassandra will do another disk seek in

Re: cassandra disk access

2013-08-07 Thread Nikolay Mihaylov
thanks It will use the Index Sample (RAM) first, then it will use full Index (disk) and finally it will read data from SSTable (disk). There's no such thing like collision in this case. so it still have 2 seeks :) where I can see the internal structure of the sstable i tried to find it

Re: cassandra disk access

2013-08-07 Thread Michał Michalski
I'm not sure how accurate it is (it's from 2011, one of its sources is from 2010), but I'm pretty sure it's more or less OK: http://blog.csdn.net/firecoder/article/details/7019435 M. W dniu 07.08.2013 10:34, Nikolay Mihaylov pisze: thanks It will use the Index Sample (RAM) first, then it

System hints compaction stuck

2013-08-07 Thread David McNelis
Morning folks, For the last couple of days all of my nodes (17, all running 1.2.8) have been stuck at various percentages of completion for compacting system.hints. I've tried restarting the nodes (including a full rolling restart of the cluster) to no avail. When I turn on Debugging I am

Re: Can't perform repair on a 1.1.5 cassandra node -SSTable corrupted

2013-08-07 Thread Ondřej Černoš
Hello, please see these issues: https://issues.apache.org/jira/browse/CASSANDRA-5686 and https://issues.apache.org/jira/browse/CASSANDRA-5391 if you hit any of them. regards, ondrej cernos On Wed, Aug 7, 2013 at 5:00 PM, Madalina Matei madalinaima...@gmail.comwrote: Hi, I have a 5 nodes

lots of small nodes vs fewer big nodes

2013-08-07 Thread Paul Ingalls
Quick question about systems architecture. Would it be better to run 5 nodes with 7GB RAM and 4CPU's or 10 nodes with 3.5GB RAM and 2CPUS? I'm currently running the former, but am considering the latter. My goal would be to improve overall performance by spreading the IO across more disks.

Re: lots of small nodes vs fewer big nodes

2013-08-07 Thread Andrey Ilinykh
You still have the same amount of RAM, so you cache the same amount of data. I don't think you gain much here. On the other side, maintenance procedures (compaction, repair) may hit your 2CPU box. I wouldn't do it. Thank you, Andrey On Wed, Aug 7, 2013 at 10:24 AM, Paul Ingalls

Re: Is there update-in-place on maps?

2013-08-07 Thread Aaron Morton
As for the atomic increment, I take the answer is 'no, there is no atomic increment, I have to pull the value to the client and send an update with the new value'. Saying atomic increment is probably confusing. You cannot have Counters, the thing most people would think about when you say

Re: Any good GUI based tool to manage data in Casandra?

2013-08-07 Thread Aaron Morton
I think on of the versions of ops centre has the feature http://www.datastax.com/what-we-offer/products-services/datastax-opscenter otherwise people use the cassandra-cli or cqlsh. Cheers - Aaron Morton Cassandra Consultant New Zealand @aaronmorton

Re: System hints compaction stuck

2013-08-07 Thread Nate McCall
Thrift and ClientState are both unrelated to hints. What do you see in the logs after Started hinted handoff for host:... from HintedHandoffManager? It should either have an error message or something along the lines of Finished hinted handoff of:... Where there any schema updates that preceded

Re: Unable to bootstrap node

2013-08-07 Thread Aaron Morton
Thanks for the update :) A - Aaron Morton Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 7/08/2013, at 7:03 AM, sankalp kohli kohlisank...@gmail.com wrote: @Aaron This problem happens when you drop and recreate a keyspace with the same name

Re: System hints compaction stuck

2013-08-07 Thread David McNelis
Nate, We had a node that was flaking on us last week and had a lot of handoffs fail to that node. We ended up decommissioning that node entirely. I can't find the actual error we were getting at the time (logs have been rotated out), but currently we're not seeing any errors there. We haven't

Re: Any good GUI based tool to manage data in Casandra?

2013-08-07 Thread Nick Bailey
OpsCenter allows CRUD of column families themselves (although not CQL3 column families). It only allows viewing the data inside column families though, no support for writing or updating. On Wed, Aug 7, 2013 at 12:54 PM, Aaron Morton aa...@thelastpickle.comwrote: I think on of the versions of

Re: Any good GUI based tool to manage data in Casandra?

2013-08-07 Thread Tony Anecito
Thanks Nick for your reply. Good to know that. I knew OpsCenter was mainly schema management. Best Regards, -Tony From: Nick Bailey n...@datastax.com To: user@cassandra.apache.org Sent: Wednesday, August 7, 2013 12:04 PM Subject: Re: Any good GUI based tool

Re: System hints compaction stuck

2013-08-07 Thread Nate McCall
Is there anything else on the network that could be attempting to connect to 9160? That is the exact error you would get when someone initiates a connection and sends a null byte. You can reproduce it thusly: echo -n 'm' | nc localhost 9160 On Wed, Aug 7, 2013 at 11:11 AM, David McNelis

Re: System hints compaction stuck

2013-08-07 Thread David McNelis
Fwiw, similar to another issue of stuck compaction that was on the list several days ago, if I cleared out the hints, either by removing files while node was down, or running a scrub on system.hints during node startup, I was able to get these compactions cleared, an the nodes are starting to get

Re: Large number of pending gossip stage tasks in nodetool tpstats

2013-08-07 Thread Aaron Morton
When looking at nodetool gossipinfo, I notice that this node has updated to the latest schema hash, but that it thinks other nodes in the cluster are on the older version. What does describe cluster in cassandra-cli say ? It will let you know if there are multiple schema versions in the

Re: cassandra disk access

2013-08-07 Thread Aaron Morton
Some background on the read and write paths, some of the extra details are a little out of date but mostly correct in 1.2 http://www.slideshare.net/aaronmorton/cassandra-community-webinar-introduction-to-apache-cassandra-12-20353118/40

Re: Large number of pending gossip stage tasks in nodetool tpstats

2013-08-07 Thread Faraaz Sareshwala
Thanks Aaron. The node that was behaving this way was a production node so I had to take some drastic measures to get it back to doing the right thing. It's no longer behaving this way after wiping the system tables and having cassandra resync the schema from other nodes. In hindsight, maybe I

Re: Large number of pending gossip stage tasks in nodetool tpstats

2013-08-07 Thread Faraaz Sareshwala
And by that last statement, I mean are there any further things I should look for given the information in my response? I'll definitely look at implementing your suggestions and see what I can find. On Aug 7, 2013, at 7:31 PM, Faraaz Sareshwala fsareshw...@quantcast.com wrote: Thanks Aaron.

Re: Is there update-in-place on maps?

2013-08-07 Thread Alex Popescu
On Wed, Aug 7, 2013 at 10:47 AM, Aaron Morton aa...@thelastpickle.comwrote: As for the atomic increment, I take the answer is 'no, there is no atomic increment, I have to pull the value to the client and send an update with the new value'. Saying atomic increment is probably confusing. You