Problems with node rejoining cluster

2013-06-25 Thread Arindam Barua
We need to do a rolling upgrade of our Cassandra cluster in production, since we are upgrading Cassandra on solaris to Cassandra on CentOS. (We went with solaris initially since most of our other hosts in production are solaris, but were running into some lockup issues during perf tests, and

Re: How to do a CAS UPDATE on single column CF?

2013-06-25 Thread Blair Zajac
On 06/24/2013 08:35 PM, Arthur Zubarev wrote: On 06/24/2013 11:23 PM, Blair Zajac wrote: CAS UPDATE Since when C* has IF NOT EXISTS in DML part of CQL? It's new in 2.0. https://issues.apache.org/jira/browse/CASSANDRA-5062

Cassandra as storage for cache data

2013-06-25 Thread Dmitry Olshansky
Hello, we are using Cassandra as a data storage for our caching system. Our application generates about 20 put and get requests per second. An average size of one cache item is about 500 Kb. Cache items are placed into one column family with TTL set to 20 - 60 minutes. Keys and values are

Re: Cassandra as storage for cache data

2013-06-25 Thread Jeremy Hanna
If you have rapidly expiring data, then tombstones are probably filling your disk and your heap (depending on how you order the data on disk). To check to see if your queries are affected by tombstones, you might try using the query tracing that's built-in to 1.2. See:

Re: NREL has released open source Databus on github for time series data

2013-06-25 Thread Hiller, Dean
When you say aggregates, do you mean converting 1 minute data to 15 minute data or do you mean summing different streams such that you have the total energy from energy streams A, B, C, etc. Ps. We are working on supporting both….there is a clusterable cron job thing in place right now that

Is nexted selects supported by Cassandra JDBC??

2013-06-25 Thread Tony Anecito
Hi All, Is nested select supported by Cassandra JDBC driver? So for a simple example to get a list of user details from a users column family: Select * from user_details where user_id in (Select user_id from users) Thanks! -Tony

Re: Is nexted selects supported by Cassandra JDBC??

2013-06-25 Thread Sylvain Lebresne
No. CQL3 doesn't support nested selects. -- Sylvain On Tue, Jun 25, 2013 at 5:02 PM, Tony Anecito adanec...@yahoo.com wrote: Hi All, Is nested select supported by Cassandra JDBC driver? So for a simple example to get a list of user details from a users column family: Select * from

cassandra-unit 1.2.0.1 is released : CQL3 and Spring

2013-06-25 Thread Jérémy SEVELLEC
Hi all, Just to let you know that a new release of cassandra-unit is available with CQL3 dataset support and Spring integration. More here : http://www.unchticafe.fr/2013/06/cassandra-unit-1201-is-out-cql3-script.html Regards, -- Jérémy

Re: Is nexted selects supported by Cassandra JDBC??

2013-06-25 Thread Tony Anecito
Ok. So if I have a composite key table instead of a nested select I will have to run 2 queries else denormalize? Unless there is something provided by CQL 3 to do the same thing? Thanks, -Tony From: Sylvain Lebresne sylv...@datastax.com To:

Re: Is nexted selects supported by Cassandra JDBC??

2013-06-25 Thread Sylvain Lebresne
Yes, denormalization is usually the answer to the absence of sub-queries (and joins for that matter) in Cassandra (though sometimes, simply doing 2 queries is fine, depends on your use case and performance requirements). On Tue, Jun 25, 2013 at 6:46 PM, Tony Anecito adanec...@yahoo.com wrote:

Re: [Cassandra] Replacing a cassandra node with one of the same IP

2013-06-25 Thread Robert Coli
On Mon, Jun 24, 2013 at 8:53 PM, aaron morton aa...@thelastpickle.com wrote: so I am just wondering if this means the hinted handoffs are also updated to reflect the new Cassandra node uuid. Without checking the code I would guess not. Because it would involve a potentially large read / write

Re: Problems with node rejoining cluster

2013-06-25 Thread Robert Coli
On Mon, Jun 24, 2013 at 11:19 PM, Arindam Barua aba...@247-inc.com wrote: - We do not specify any tokens in cassandra.yaml relying on bootstrap assigning the tokens automatically. As cassandra.yaml comments state, you should never ever do this in a real cluster. I don't know what is

Re: Counter value becomes incorrect after several dozen reads writes

2013-06-25 Thread Robert Coli
On Mon, Jun 24, 2013 at 6:42 PM, Josh Dzielak j...@keen.io wrote: There is only 1 thread running this sequence, and consistency levels are set to ALL. The behavior is fairly repeatable - the unexpectation mutation will happen at least 10% of the time I run this program, but at different points.

Re: copy data between clusters

2013-06-25 Thread Robert Coli
On Mon, Jun 24, 2013 at 8:35 PM, S C as...@outlook.com wrote: I have a scenario here. I have a cluster A and cluster B running on cassandra 1.1. I need to copy data from Cluster A to Cluster B. Cluster A has few keyspaces that I need to copy over to Cluster B. What are my options?

Re: Cassandra terminates with OutOfMemory (OOM) error

2013-06-25 Thread sankalp kohli
Your young gen is 1/4 of 1.8G which is 450MB. Also in slice queries, the co-ordinator will get the results from replicas as per consistency level used and merge the results before returning to the client. What is the replication in your keyspace and what consistency you are reading with. Also 55MB

Re: Cassandra as storage for cache data

2013-06-25 Thread sankalp kohli
Apart from what Jeremy said, you can try these 1) Use replication = 1. It is cache data and you dont need persistence. 2) Try playing with memtable size. 3) Use netflix client library as it will reduce one hop. It will chose the node with data as the co ordinator. 4) Work on your schema. You might

Re: Counter value becomes incorrect after several dozen reads writes

2013-06-25 Thread Andrew Bialecki
If you can reproduce the invalid behavior 10+% of the time with steps to repro that take 5-10s/iteration, that sounds extremely interesting for getting to the bottom of the invalid shard issue (if that's what the root cause ends up being). Would be very interested in the set up to see if the

Re: Custom 1.2 Authentication plugin will not work unless user is in system_auth.users column family

2013-06-25 Thread Bao Le
Sorry for not following up on this one in time. I filed a JIRA (5651) and it seems user lookup is here to stay. https://issues.apache.org/jira/browse/CASSANDRA-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel On a related note, that column family is, by default, set up to

RE: copy data between clusters

2013-06-25 Thread S C
Bob and Arthur - thanks for your inputs. I tried sstableloader but ran into below issue. Anything to do with the configuration to run sstableloader? sstableloader -d 10.225.64.2,10.225.64.3 service/context INFO 14:43:49,937 Opening service/context/service-context-hf-50 (164863 bytes)DEBUG

what happen if coordinator node fails during write

2013-06-25 Thread Jiaan Zeng
Hi there, I am writing data to Cassandra by thrift client (not hector) and wonder what happen if the coordinator node fails. The same question applies for bulk loader which uses gossip protocol instead of thrift protocol. In my understanding, the HintedHandoff only takes care of the replica node

Re: copy data between clusters

2013-06-25 Thread Arthur Zubarev
Hello SC, whilst most of the sstableloader errors stem from incorrect setups I suspect this time you merely have a connectivity issue e.g. a firewall blocking traffic. From: S C Sent: Tuesday, June 25, 2013 5:28 PM To: user@cassandra.apache.org Subject: RE: copy data between clusters Bob and

Re: what happen if coordinator node fails during write

2013-06-25 Thread Andrey Ilinykh
It depends on cassandra version. As far as I know in 1.2 coordinator logs request before it updates replicas. If it fails it will replay log on startup. In 1.1 you may have inconsistant state, because only part of your request is propagated to replicas. Thank you, Andrey On Tue, Jun 25, 2013

Re: Date range queries

2013-06-25 Thread Colin Blower
You could just separate the history data from the current data. Then when the user's result is updated, just write into two tables. CREATE TABLE all_answers ( user_id uuid, created timeuuid, result text, question_id varint, PRIMARY KEY (user_id, created) ) CREATE TABLE current_answers

Re: what happen if coordinator node fails during write

2013-06-25 Thread sankalp kohli
Read this http://www.datastax.com/dev/blog/atomic-batches-in-cassandra-1-2 On Tue, Jun 25, 2013 at 8:45 PM, Andrey Ilinykh ailin...@gmail.com wrote: It depends on cassandra version. As far as I know in 1.2 coordinator logs request before it updates replicas. If it fails it will replay log on

RE: copy data between clusters

2013-06-25 Thread S C
Is there any configuration reference that help me? Thanks,SC From: arthur.zuba...@aol.com To: user@cassandra.apache.org Subject: Re: copy data between clusters Date: Tue, 25 Jun 2013 20:30:23 -0400 Hello SC, whilst most of the sstableloader errors stem from incorrect setups I suspect

Re: copy data between clusters

2013-06-25 Thread Arthur Zubarev
This is the best reference I have seen so far http://www.datastax.com/dev/blog/bulk-loading But I must tell it is not updated to match the most recent changes in C*. I suggest you read thru comments, too. From: S C Sent: Tuesday, June 25, 2013 10:23 PM To: user@cassandra.apache.org Subject:

Re: Cassandra terminates with OutOfMemory (OOM) error

2013-06-25 Thread Mohammed Guller
Replication is 3 and read consistency level is one. One of the non-cordinator mode is crashing, so the OOM is happening before aggregation of the data to be returned. Thanks for the info about the space allocated to young generation heap. That is helpful. Mohammed On Jun 25, 2013, at 1:28

Re: Heap is not released and streaming hangs at 0%

2013-06-25 Thread aaron morton
bloom_filter_fp_chance value that was changed from default to 0.1, looked at the filters and they are about 2.5G on disk and I have around 8G of heap. I will try increasing the value to 0.7 and report my results. You need to re-write the sstables on disk using nodetool upgradesstables.

Re: Cassandra 1.0.9 Performance

2013-06-25 Thread aaron morton
serving a load of approximately 600GB is that 600GB in the cluster or 600GB per node ? In pre 1.2 days we recommend around 300GB to 500GB per node with spinning disks and 1Gbe networking. It's a soft rule of thumb not a hard rule. Above that size repair and replacing a failed node can take a

Re: about FlushWriter All time blocked

2013-06-25 Thread aaron morton
FlushWriter 0 0191 0 12 This means there were 12 times the code wanted to put an memtable in the queue to be flushed to disk but the queue was full. The length of this queue is controlled by the memtable_flush_queue_size