Re: Too many open files Cassandra 2.1.11.872

2015-11-06 Thread Jason Lewis
cat /proc/5980/limits Limit Soft Limit Hard Limit Units Max cpu time unlimitedunlimitedseconds Max file size unlimitedunlimitedbytes Max data size unlimited

Re: Re: Too many open files Cassandra 2.1.11.872

2015-11-06 Thread 郝加来
many connection ? 郝加来 From: Jason Lewis Date: 2015-11-07 10:38 To: user@cassandra.apache.org Subject: Re: Too many open files Cassandra 2.1.11.872 cat /proc/5980/limits Limit Soft Limit Hard Limit Units Max cpu time unlimited

Do I have to use the cql in the datastax java driver?

2015-11-06 Thread Dikang Gu
Hi there, In the datastax java driver, do I have to use the cql to talk to cassandra cluster? Can I still use thrift interface to talk to cassandra? Any reason that we should not use thrift anymore? Thanks. -- Dikang

Insertion Delay Cassandra 2.1.9

2015-11-06 Thread Greg Traub
Cassandra users, I have a 4 node Cassandra cluster set up. All nodes are in a single rack and distribution center. I have a loader program which loads 40 million rows into a table in a keyspace with a replication factor of 3. Immediately after inserting the rows (after the loader program

Re: Too many open files Cassandra 2.1.11.872

2015-11-06 Thread Branton Davis
We recently went down the rabbit hole of trying to understand the output of lsof. lsof -n has a lot of duplicates (files opened by multiple threads). Use 'lsof -p $PID' or 'lsof -u cassandra' instead. On Fri, Nov 6, 2015 at 12:49 PM, Bryan Cheng wrote: > Is your

Re: Insertion Delay Cassandra 2.1.9

2015-11-06 Thread Greg Traub
Vidur, Forgive me if I'm getting this wrong as I'm exceptionally new to Cassandra. By consistency, if you mean the USING CONSISTENCY clause, then I'm not specifying it which, per the CQL documentation, means a default of ONE. On Fri, Nov 6, 2015 at 1:49 PM, Vidur Malik

Re: Insertion Delay Cassandra 2.1.9

2015-11-06 Thread Vidur Malik
Ah, I thought you may have been using a higher consistency, which would explain your error since the data may not have been replicated across all 3 nodes when you made the query. Anyway, it seems to be happening because of replication. What version of Cassandra are you using? There may be a issue

Re: Too many open files Cassandra 2.1.11.872

2015-11-06 Thread Bryan Cheng
Is your compaction progressing as expected? If not, this may cause an excessive number of tiny db files. Had a node refuse to start recently because of this, had to temporarily remove limits on that process. On Fri, Nov 6, 2015 at 10:09 AM, Jason Lewis wrote: > I'm

Re: Insertion Delay Cassandra 2.1.9

2015-11-06 Thread Vidur Malik
What is your query consistency? On Fri, Nov 6, 2015 at 1:47 PM, Greg Traub wrote: > Cassandra users, > > I have a 4 node Cassandra cluster set up. All nodes are in a single rack > and distribution center. I have a loader program which loads 40 million > rows into a

Re: Insertion Delay Cassandra 2.1.9

2015-11-06 Thread Bryan Cheng
Your experience, then, is expected (although 20m delay seems excessive, and is a sign you may be overloading your cluster, which may be expected with an unthrottled bulk load like that). When you insert with consistency ONE on RF > 1, that means your query returns after one node confirms the

Re: store avro to cassandra

2015-11-06 Thread Jack Krupansky
Use a Cassandra map column - the keys of the map can be arbitrary. But if there are some standard set of columns that are always or commonly present, make them explicit Cassandra columns. And if the avro values have different types, you may want to have several Cassandra map columns, one for each

Too many open files Cassandra 2.1.11.872

2015-11-06 Thread Jason Lewis
I'm getting too many open files errors and I'm wondering what the cause may be. lsof -n | grep java show 1.4M files ~90k are inodes ~70k are pipes ~500k are cassandra services in /usr ~700K are the data files. What might be causing so many files to be open? jas

Re: Too many open files Cassandra 2.1.11.872

2015-11-06 Thread Sebastian Estevez
You probably need to configure ulimits correctly . What does this give you? /proc//limits All the best, [image: datastax_logo.png] Sebastián Estévez Solutions Architect |