Re: Issue with Cassandra consistency in results

2017-03-16 Thread srinivasarao daruna
Replication strategy is SimpleReplicationStrategy. Smith is : EC2 snitch. As we deployed cluster on EC2 instances. I was worried that CL=ALL have more read latency and read failures. But won't rule out trying it. Should I switch select count (*) to select partition_key column? Would that be of

Re: Issue with Cassandra consistency in results

2017-03-16 Thread srinivasarao daruna
Would switching to select partition_key instead of select count(*) help me any way ? I know that, Logically they both are same.. but just asking out of desperation. Is it worth a shot? On Mar 16, 2017 1:09 PM, "Ryan Svihla" wrote: Replication factor is 3, and write

Re: Issue with Cassandra consistency in results

2017-03-16 Thread Ryan Svihla
Depends actually, restore just restores what's there, so if only one node had a copy of the data then only one node had a copy of the data meaning quorum will still be wrong sometimes. On Thu, Mar 16, 2017 at 1:53 PM, Arvydas Jonusonis < arvydas.jonuso...@gmail.com> wrote: > If the data was

Re: Issue with Cassandra consistency in results

2017-03-16 Thread srinivasarao daruna
Want to make read and write QUORUM as well. On Mar 16, 2017 1:09 PM, "Ryan Svihla" wrote: Replication factor is 3, and write consistency is ONE and read consistency is QUORUM. That combination is not gonna work well: *Write succeeds to NODE A but fails on node B,C*

Re: Cassandra + NUMA integration

2017-03-16 Thread nokia ceph
Hello, In addition to my previous comment. Regarding the NUMA integration with Cassadra, we configured NODE-1 only utilize the resources. However after enabling this functionality, we noticed on NODE-1 Private allocated to 4G . What we did << #tail -n1 /etc/cassandra/conf/cassandra-env.sh

Re: Issue with Cassandra consistency in results

2017-03-16 Thread daemeon reiydelle
The discard due to oom is causing the zero returned. I would guess a cache miss problem of some sort, but not sure. Are you using row, index, etc. caches? Are you seeing the failed prep statement on random nodes (duh, nodes that have the relevant data ranges)? *...* *Daemeon C.M.

Re: Issue with Cassandra consistency in results

2017-03-16 Thread Ryan Svihla
Replication factor is 3, and write consistency is ONE and read consistency is QUORUM. That combination is not gonna work well: *Write succeeds to NODE A but fails on node B,C* *Read goes to NODE B, C* If you can tolerate some temporary inaccuracy you can use QUORUM but may still have

Re: Issue with Cassandra consistency in results

2017-03-16 Thread Arvydas Jonusonis
If the data was written at ONE, consistency is not guaranteed. ..but considering you just restored the cluster, there's a good chance something else is off. On Thu, Mar 16, 2017 at 18:19 srinivasarao daruna wrote: > Want to make read and write QUORUM as well. > > > On

JMX metric collection for cassandra.

2017-03-16 Thread Pranay akula
We are collecting JMX metrics through Jolokia. I am trying to find out active connections to the host, but i couldn't find in Jolokia. i am running jolokia list couldn't find it either. */usr/local/bin/jmx4perl http://localhost:6809/mon/jolokia list*

Re: Change the IP of a live node

2017-03-16 Thread Patrick McFadin
It also looks like you changed your topology by altering rack from r1 to RAC1. Shouldn't effect this issue but something to get consistent. On Thu, Mar 16, 2017 at 4:49 AM, George Sigletos wrote: > This was a network problem at our side after all which we fixed.

Re: Issue with Cassandra consistency in results

2017-03-16 Thread Arvydas Jonusonis
What are your replication strategy and snitch settings? Have you tried doing a read at CL=ALL? If it's an actual inconsistency issue (missing data), this should cause the correct results to be returned. You'll need to run a repair to fix the inconsistencies. If all the data is actually there,

Issue with Cassandra consistency in results

2017-03-16 Thread srinivasarao daruna
Hi Team, We are struggling with a problem related to cassandra counts, after backup and restore of the cluster. Aaron Morton has suggested to send this to user list, so some one of the list will be able to help me. We are have a rest api to talk to cassandra and one of our query which fetches

Re: Slow repair

2017-03-16 Thread siddharth verma
Hi, We did a similar thing when a new DC was added and had to populate it according to altered replication of keyspace. For repair we used Tickler approach rather than actual nodetool repair. (using the blocking read repair feature in cassandra) You can see 1. Ticker by ckalantzis :

Re: Issue with Cassandra consistency in results

2017-03-16 Thread srinivasarao daruna
Hi reiydelle, I cannot confirm the range as the volume of data is huge and the query frequency is also high. If the cache is the cause of issue, can we increase cache size or is there solution to avoid dropped prep statements.? Thank You, Regards, Srini On Thu, Mar 16, 2017 at 2:13 PM,

Cassandra + NUMA integration

2017-03-16 Thread nokia ceph
Hello, Recently we tried top integrate cassandra with NUMA == commented out the numactl --interleave in bin/cassandra and add -XX:+UseNUMA to cassandra-env.sh. == But after inplementing, we saw HEAP and STACK values were changed to null. #/usr/bin/numactl --cpunodebind=1 -- /usr/sbin/cassandra

Re: Change the IP of a live node

2017-03-16 Thread George Sigletos
This was a network problem at our side after all which we fixed. Cassandra was blocking connections between 192.168.xxx <-> 10.179.xxx on port 7000 On Wed, Mar 15, 2017 at 2:47 PM, Ryan Svihla wrote: > I've actually changed the ip address quite a bit (gossip complains on >

AW: Issue with Cassandra consistency in results

2017-03-16 Thread j.kesten
Hi, doing a quick scan over the thread two things that came into my mind: Frist, did the restore copy the sstables to the right machines back? Node As data to node A and so on? Second, did you run full repairs on every node? Not just incremental ones which now is the default? Also a look