Re : List users getting stuck and not returning results

2015-10-02 Thread sai krishnam raju potturi
We have 2 clusters running DSE. On one of the clusters we recently added additional nodes to a datacenter. On the cluster where we added nodes, we are getting authentication issues from client. We are also unable to "list users" on system_auth keyspace. It's getting stuck.

RE: Consistency Issues

2015-10-02 Thread Walsh, Stephen
Sorry for the late reply, I ran the nodetool resetlocalschema on all nodes but in the end it just removed all the schemas and crashed the applications. I need to reset and try again. I’ll try get you the gc stats today ☺ From: Jonathan Haddad [mailto:j...@jonhaddad.com] Sent: 01 October 2015

Re: Repair corrupt SSTable from power outage?

2015-10-02 Thread Tyler Hobbs
What version of Cassandra are you running? It sounds like the disk failure policy is incorrectly being applied to scrub, which kind of defeats the purpose of it. I'd recommend opening a ticket ( https://issues.apache.org/jira/browse/CASSANDRA) with the information you posted here. On Fri, Oct

Re: JSON Order By

2015-10-02 Thread Tyler Hobbs
On Thu, Oct 1, 2015 at 9:11 AM, Ashish Soni wrote: > I have a below structure stored in cassandra and i would like to get the > internal array sorted by a property when i select it , Please let me know > if there is way to do that . > > I need to sort the rules Array by

Re: Changing schema on multiple nodes while they are isolated

2015-10-02 Thread Stephen Baynes
Hi Jacques-Henri You are right - serious trouble. I managed some more testing and it does not repair or share any data. In the logs I see lots of: WARN [MessagingService-Incoming-/10.50.16.214] 2015-10-02 16:52:36,810 IncomingTcpConnection.java:100 - UnknownColumnFamilyException reading from

RE: Consistency Issues

2015-10-02 Thread Walsh, Stephen
Using the following cmd - sudo su cassandra -c "jstat -gccause 4162” Gave this (not sure if it will present correctly on the webpage) But during load we only see data move between the survivor spaces in Eden and the old gen never really grows S0 S1E O M

RE: Changing schema on multiple nodes while they are isolated

2015-10-02 Thread Jacques-Henri Berthemet
Hi Stephen, If you manage to create tables on each node while node A and B are separated, you’ll get into troubles when they will reconnect again. I had the case previously and Cassandra complained that tables with same names but different ids were present in the keyspace. I don’t know if

Changing schema on multiple nodes while they are isolated

2015-10-02 Thread Stephen Baynes
Is it safe to make schema changes ( e.g. create keyspace and tables ) on multiple separate nodes of a cluster while they are out of communication with other nodes in the cluster? For example create on node A while node B is down, create on node B while A is down, then bring both up together. We

RE: Changing schema on multiple nodes while they are isolated

2015-10-02 Thread Jacques-Henri Berthemet
Why don’t you simply let the node join the cluster? It will pull new tables and the data automatically. -- Jacques-Henri Berthemet From: Stephen Baynes [mailto:stephen.bay...@smoothwall.net] Sent: vendredi 2 octobre 2015 18:08 To: user@cassandra.apache.org Subject: Re: Changing schema on

RE: Cassandra certification

2015-10-02 Thread Fernandez Gomara, Ruben (CCI-Atlanta)
Thank you all so much for your emails. The link has very useful information. Thanks again Ruben From: sean_r_dur...@homedepot.com [mailto:sean_r_dur...@homedepot.com] Sent: Thursday, October 01, 2015 4:09 PM To: user@cassandra.apache.org Subject: RE: Cassandra certification I took (and passed)

Re: Re : List users getting stuck and not returning results

2015-10-02 Thread Nate McCall
Set the replication factor for the system_auth keyspace equal to the number of nodes, then issue a repair. On Fri, Oct 2, 2015 at 6:51 AM, sai krishnam raju potturi < pskraj...@gmail.com> wrote: > We have 2 clusters running DSE. On one of the clusters we recently added > additional nodes to a

Re: Changing schema on multiple nodes while they are isolated

2015-10-02 Thread Robert Coli
On Fri, Oct 2, 2015 at 9:08 AM, Stephen Baynes < stephen.bay...@smoothwall.net> wrote: > Will need to do some thinking about this. I wonder about shiping a backup > of a good system keyspace and restore it on each node before it starts for > the first time - but will that end up with each node

Re: GossipingPropertyFileSnitch and nodetool info

2015-10-02 Thread Robert Coli
On Thu, Oct 1, 2015 at 1:07 PM, Eric Plowe wrote: > I am using C* 2.1.9 and GossipingPropertyFileSnitch. I noticed that when I > run nodetool info I am seeing the data center and rack as > ... > Is this just a bug with nodetool? > Yep, fixed in 2.1.10.

Off heap memory leak?

2015-10-02 Thread Parag Patel
We have a 12node Cassandra cluster running on 1.2.12. Each node is using 1.1TB out of 2TB. Each node has a min+max heap of 24Gb and the physical server has 48Gb. Our nodes do not restart during the week, only on the weekend, and we're observing that the off heap memory that is consumed ramps

Re: Consistency Issues

2015-10-02 Thread Robert Coli
On Fri, Oct 2, 2015 at 1:32 AM, Walsh, Stephen wrote: > Sorry for the late reply, I ran the nodetool resetlocalschema on all > nodes but in the end it just removed all the schemas and crashed the > applications. > > I need to reset and try again. I’ll try get you the gc

Re: Off heap memory leak?

2015-10-02 Thread Robert Coli
On Fri, Oct 2, 2015 at 12:54 PM, Parag Patel wrote: > We have a 12node Cassandra cluster running on 1.2.12. Each node is using > 1.1TB out of 2TB. Each node has a min+max heap of 24Gb and the physical > server has 48Gb. Our nodes do not restart during the week, only

Duplicate records returned

2015-10-02 Thread Robert Wille
When I run the query "SELECT image FROM roll WHERE roll = :roll“ against this table CREATE TABLE roll ( roll INT, image BIGINT, data VARCHAR static, mid VARCHAR, imp_st VARCHAR, PRIMARY KEY ((roll), image) ) WITH gc_grace_seconds = 3456000 AND compaction = { 'class' :

Re: GossipingPropertyFileSnitch and nodetool info

2015-10-02 Thread Eric Plowe
Ah. My bad for not checking the jira first. Thanks! On Friday, October 2, 2015, Robert Coli wrote: > On Thu, Oct 1, 2015 at 1:07 PM, Eric Plowe > wrote: > >> I am using C* 2.1.9 and

Re: Changing schema on multiple nodes while they are isolated

2015-10-02 Thread Clint Martin
You could use a two key space method. At startup, wait some time for the node to join the cluster. the first time the app starts, you can be in one of three states: The happiest state is that you succeed in joining a cluster. in this case you will get replicated the cluster's keyspace and can

Re: Repair corrupt SSTable from power outage?

2015-10-02 Thread George Sigletos
I'm also facing problems regarding corrupt sstables and also couldn't run sstablescrub successfully. I restarted my nodes with disk failure policy "best_effort", then I run the "nodetool scrub " Once done I removed the corrupt tables manually and started repair On Thu, Oct 1, 2015 at 7:27 PM,

Re: Cassandra certification

2015-10-02 Thread Renato Perini
What credibility can have a certification with a non disclosure agreement? Il 02/10/2015 19:58, Fernandez Gomara, Ruben (CCI-Atlanta) ha scritto: Thank you all so much for your emails. The link has very useful information. Thanks again Ruben *From:*sean_r_dur...@homedepot.com