data distribution along column family partitions
Hello, I am designing a model to store alerts users receive over time. I will want to store probably the last two years of alerts for each user. The first thought I had was having a column family partitioned by user + timebucket, where time bucket could be something like year + month. For instance: *partition key:* user-id = f47ac10b-58cc-*4*372-*a*567-0e02b2c3d479 time-bucket = 201502 *rest of primary key:* timestamp = column of tipy timestamp alert id = f47ac10b-58cc-*4*372-*a*567-0e02b2c3d480 Question, would this make it easier to delete old data? Supposing I am not using TTL and I want to remove alerts older than 2 years, what would be better, just deleting the entire time-bucket for each user-id (through a map/reduce process) or having just user-id as partition key and deleting, for each user, where X timestamp Y? Is it the same for Cassandra, internally? Another question is: would data be distributed enough if I just choose to partition by user-id? I will have some users with a large number of alerts, but in average I could consider alerts would have a good distribution along user ids. The problem is I don't fell confident having few partitions with A LOT of alerts would not be a problem at read time. What happens at read time when I try to read data from a big partition? Like, I want to read alerts for a user where X timestamp Y, but it would return 1 million alerts. As it's all in a single partition, this read will occur in the same node, thus allocating a lot of memory for this single operation, right? What if the memory needed for this operation is bigger than it fits in java heap? Would this be a problem to Cassandra? Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
index partition key
Hi,
This question is just for curiosity purposes, I don't need this in my
solution, but it's something I was asking myself...
Is there a way of indexing the partition key values in Cassandra? Does
anyone needed this and found a solution of any kind?
For example, I know the sample bellow doesn't work, but would it be
possible somehow?
I know select * from testcf where token(key1) token('Lucas'); works, but
the question is ordering by the values, even using another CF or index of
any kind...
CREATE KEYSPACE IF NOT EXISTS testks
WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor': '1' };
CREATE TABLE IF NOT EXISTS testcf (
key1 varchar,
value varchar,
PRIMARY KEY ((key1)));
CREATE INDEX testidx on testks.testcf (key1);
insert into testcf (key1, value) values ('Lucas', 'value1');
insert into testcf (key1, value) values ('Luiz', 'value2');
insert into testcf (key1, value) values ('Edu', 'value3');
insert into testcf (key1, value) values ('Marcelo', 'value4');
select * from testcf order by key1;
select * from testcf where key1 'Lucas';
Best regards,
Marcelo.
Re: index partition key
Yes, indeed using the evil ordered partitioner it would work, but it should
be avoided at all costs.
What I am trying to figure is: Can I tell Cassandra is not good for cases
where you need range queries over all the cluster?
It's just a kind of architectural rule I am trying to use to decide
whether when to use a kind of technology or another.
Solr, for instance, allows you to query the way you want over all the
cluster, so data stax enterprise could solve this problem, but I wonder
if it's the only way.
[]s
2014-09-18 13:35 GMT-03:00 DuyHai Doan doanduy...@gmail.com:
Your query select * from testcf where key1 'Lucas'; does work if you
choose the old OrderPartioner but since it's considered absolute evil (for
good reason, load distribution is horrendous), pratically it is not
possible to do your query.
And it's the same thing with secondary index. Query with inequality on
secondary index translates into full cluster scan...
Regards
Duy Hai DOAN
On Thu, Sep 18, 2014 at 5:44 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
Hi,
This question is just for curiosity purposes, I don't need this in my
solution, but it's something I was asking myself...
Is there a way of indexing the partition key values in Cassandra? Does
anyone needed this and found a solution of any kind?
For example, I know the sample bellow doesn't work, but would it be
possible somehow?
I know select * from testcf where token(key1) token('Lucas'); works,
but the question is ordering by the values, even using another CF or index
of any kind...
CREATE KEYSPACE IF NOT EXISTS testks
WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor': '1' };
CREATE TABLE IF NOT EXISTS testcf (
key1 varchar,
value varchar,
PRIMARY KEY ((key1)));
CREATE INDEX testidx on testks.testcf (key1);
insert into testcf (key1, value) values ('Lucas', 'value1');
insert into testcf (key1, value) values ('Luiz', 'value2');
insert into testcf (key1, value) values ('Edu', 'value3');
insert into testcf (key1, value) values ('Marcelo', 'value4');
select * from testcf order by key1;
select * from testcf where key1 'Lucas';
Best regards,
Marcelo.
Re: too many open files
IMHO, I think the drivers are fine. It was a dumb mistake of mine to use sessions as connections and not as connection pools. What was harder to figure, in my opinion, was that too many connections from the client would increase the amount of file descriptors used by the server. I didn't know Linux open a FD for each connection received and honestly I still don't know much about the details of this. When I got a too many open files error it took a good while to think about checking the connections. I think the documentation could point this fact, it would help other people with the same problem. There could be something talking about it here: http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootTooManyFiles_r.html []s 2014-08-09 12:55 GMT-03:00 Jack Krupansky j...@basetechnology.com: Maybe the drivers should have two modes: few sessions, and lots of sessions. The former would give you a developer-friendly driver error if you leave more than say a dozen or two dozen sessions open (or whatever is considered a best practice for parallel threads in a client), on the theory that you probably used the anti-pattern of failing to reuse sessions. The latter would be more for expert apps that have some good reason for having hundreds or thousands of simultaneous sessions open. Whether the latter also has some (configurable) limit that is simply a lot higher than the former or is unlimited, is probably not so important. Or, maybe, simply have a single limit, without the modes and default it to 10 or 25 or some other relatively low number for “normal” apps. This would be more developer-friendly, for both new and “normal” developers... I think. -- Jack Krupansky *From:* Marcelo Elias Del Valle marc...@s1mbi0se.com.br *Sent:* Saturday, August 9, 2014 12:41 AM *To:* user@cassandra.apache.org *Subject:* Re: too many open files Indeed, that was my mistake, that was exactly what we were doing in the code. []s 2014-08-09 0:56 GMT-03:00 Brian Zhang yikebo...@gmail.com: For cassandra driver,session is just like database connection pool,it maybe contains many tcp connections,if you create a new session every time,more and more tcp connections will be created,till surpass the max file description limit of os. You should create one session,use it repeatedly ,session can manage connections automatically,create new connection or close old connection for your requests. 在 2014年8月9日，6:52，Redmumba redmu...@gmail.com 写道： Just to chime in, I also ran into this issue when I was migrating to the Datastax client. Instead of reusing the session, I was opening a new session each time. For some reason, even though I was still closing the session on the client side, I was getting the same error. Plus, the only way I could recover was by restarting Cassandra. I did not really see the connections timeout over a period of a few minutes. Andrew On Aug 8, 2014 3:19 PM, Andrey Ilinykh ailin...@gmail.com wrote: You may have this problem if your client doesn't reuse the connection but opens new every type. So, run netstat and check the number of established connections. This number should not be big. Thank you, Andrey On Fri, Aug 8, 2014 at 12:35 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi, I am using Cassandra 2.0.9 running on Debian Wheezy, and I am having too many open files exceptions when I try to perform a large number of operations in my 10 node cluster. I saw the documentation http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootTooManyFiles_r.html and I have set everything to the recommended settings, but I keep getting the errors. In the documentation it says: Another, much less likely possibility, is a file descriptor leak in Cassandra. Run lsof -n | grep java to check that the number of file descriptors opened by Java is reasonable and reports the error if the number is greater than a few thousand. I guess it's not the case, or else a lot of people would be complaining about it, but I am not sure what I could do to solve the problem. Any hint about how to solve it? My client is written in python and uses Cassandra Python Driver. Here are the exceptions I am having in the client: [s1log] 2014-08-08 12:16:09,631 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.151, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,632 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,633 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.143, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,634 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds
too many open files
Hi, I am using Cassandra 2.0.9 running on Debian Wheezy, and I am having too many open files exceptions when I try to perform a large number of operations in my 10 node cluster. I saw the documentation http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootTooManyFiles_r.html and I have set everything to the recommended settings, but I keep getting the errors. In the documentation it says: Another, much less likely possibility, is a file descriptor leak in Cassandra. Run lsof -n | grep java to check that the number of file descriptors opened by Java is reasonable and reports the error if the number is greater than a few thousand. I guess it's not the case, or else a lot of people would be complaining about it, but I am not sure what I could do to solve the problem. Any hint about how to solve it? My client is written in python and uses Cassandra Python Driver. Here are the exceptions I am having in the client: [s1log] 2014-08-08 12:16:09,631 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.151, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,632 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,633 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.143, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,634 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,634 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.145, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,635 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.144, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,635 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.148, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,732 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.146, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,733 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.77, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,734 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.76, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,734 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.75, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,735 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,736 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.185, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,942 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.144, scheduling retry in 512.0 seconds: Timed out connecting to 200.200.200.144 [s1log] 2014-08-08 12:16:09,998 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.77, scheduling retry in 512.0 seconds: Timed out connecting to 200.200.200.77 And here is the exception I am having in the server: WARN [Native-Transport-Requests:163] 2014-08-08 14:27:30,499 BatchStatement.java (line 223) Batch of prepared statements for [identification.entity_lookup, identification.entity] is of size 25216, exceeding specified threshold of 5120 by 20096. ERROR [Native-Transport-Requests:150] 2014-08-08 14:27:31,611 ErrorMessage.java (line 222) Unexpected exception during request java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:192) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at
Re: too many open files
I am using datastax community, the packaged version for Debian. I am also using last version of opscenter and datastax-agent However, I just listed open files here: sudo lsof -n | grep java | wc -l 1096599 It seems it has exceed. Should I just increase? Or is it possible to be a memory leak? Best regards, Marcelo. 2014-08-08 17:06 GMT-03:00 Shane Hansen shanemhan...@gmail.com: Are you using apache or Datastax cassandra? The datastax distribution ups the file handle limit to 10. That number's hard to exceed. On Fri, Aug 8, 2014 at 1:35 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi, I am using Cassandra 2.0.9 running on Debian Wheezy, and I am having too many open files exceptions when I try to perform a large number of operations in my 10 node cluster. I saw the documentation http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootTooManyFiles_r.html and I have set everything to the recommended settings, but I keep getting the errors. In the documentation it says: Another, much less likely possibility, is a file descriptor leak in Cassandra. Run lsof -n | grep java to check that the number of file descriptors opened by Java is reasonable and reports the error if the number is greater than a few thousand. I guess it's not the case, or else a lot of people would be complaining about it, but I am not sure what I could do to solve the problem. Any hint about how to solve it? My client is written in python and uses Cassandra Python Driver. Here are the exceptions I am having in the client: [s1log] 2014-08-08 12:16:09,631 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.151, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,632 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,633 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.143, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,634 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,634 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.145, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,635 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.144, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,635 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.148, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,732 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.146, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,733 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.77, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,734 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.76, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,734 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.75, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,735 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,736 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.185, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,942 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.144, scheduling retry in 512.0 seconds: Timed out connecting to 200.200.200.144 [s1log] 2014-08-08 12:16:09,998 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.77, scheduling retry in 512.0 seconds: Timed out connecting to 200.200.200.77 And here is the exception I am having in the server: WARN [Native-Transport-Requests:163] 2014-08-08 14:27:30,499 BatchStatement.java (line 223) Batch of prepared statements for [identification.entity_lookup, identification.entity] is of size 25216, exceeding specified threshold of 5120 by 20096. ERROR [Native-Transport-Requests:150] 2014-08-08 14:27:31,611 ErrorMessage.java (line 222) Unexpected exception during request java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223
Re: too many open files
I just solved the issue, it was Cassandra process which was opening too many fds, indeed, but the problem was the amount of client connections being opened. It was opening more connection than needed in the client' side. Thanks for the help. []s 2014-08-08 17:17 GMT-03:00 Kevin Burton bur...@spinn3r.com: You may want to look at the the actual filenames. You might have an app leaving them open. Also, remember, sockets use FDs so they are in the list too. On Fri, Aug 8, 2014 at 1:13 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: I am using datastax community, the packaged version for Debian. I am also using last version of opscenter and datastax-agent However, I just listed open files here: sudo lsof -n | grep java | wc -l 1096599 It seems it has exceed. Should I just increase? Or is it possible to be a memory leak? Best regards, Marcelo. 2014-08-08 17:06 GMT-03:00 Shane Hansen shanemhan...@gmail.com: Are you using apache or Datastax cassandra? The datastax distribution ups the file handle limit to 10. That number's hard to exceed. On Fri, Aug 8, 2014 at 1:35 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi, I am using Cassandra 2.0.9 running on Debian Wheezy, and I am having too many open files exceptions when I try to perform a large number of operations in my 10 node cluster. I saw the documentation http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootTooManyFiles_r.html and I have set everything to the recommended settings, but I keep getting the errors. In the documentation it says: Another, much less likely possibility, is a file descriptor leak in Cassandra. Run lsof -n | grep java to check that the number of file descriptors opened by Java is reasonable and reports the error if the number is greater than a few thousand. I guess it's not the case, or else a lot of people would be complaining about it, but I am not sure what I could do to solve the problem. Any hint about how to solve it? My client is written in python and uses Cassandra Python Driver. Here are the exceptions I am having in the client: [s1log] 2014-08-08 12:16:09,631 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.151, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,632 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,633 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.143, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,634 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,634 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.145, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,635 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.144, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,635 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.148, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,732 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.146, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,733 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.77, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,734 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.76, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,734 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.75, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,735 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,736 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.185, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,942 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.144, scheduling retry in 512.0 seconds: Timed out connecting to 200.200.200.144 [s1log] 2014-08-08 12:16:09,998 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.77, scheduling retry in 512.0 seconds: Timed out connecting to 200.200.200.77 And here is the exception I am having in the server: WARN [Native-Transport-Requests:163] 2014-08-08 14:27
Re: too many open files
Indeed, that was my mistake, that was exactly what we were doing in the code. []s 2014-08-09 0:56 GMT-03:00 Brian Zhang yikebo...@gmail.com: For cassandra driver,session is just like database connection pool,it maybe contains many tcp connections,if you create a new session every time,more and more tcp connections will be created,till surpass the max file description limit of os. You should create one session,use it repeatedly ,session can manage connections automatically,create new connection or close old connection for your requests. 在 2014年8月9日，6:52，Redmumba redmu...@gmail.com 写道： Just to chime in, I also ran into this issue when I was migrating to the Datastax client. Instead of reusing the session, I was opening a new session each time. For some reason, even though I was still closing the session on the client side, I was getting the same error. Plus, the only way I could recover was by restarting Cassandra. I did not really see the connections timeout over a period of a few minutes. Andrew On Aug 8, 2014 3:19 PM, Andrey Ilinykh ailin...@gmail.com wrote: You may have this problem if your client doesn't reuse the connection but opens new every type. So, run netstat and check the number of established connections. This number should not be big. Thank you, Andrey On Fri, Aug 8, 2014 at 12:35 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi, I am using Cassandra 2.0.9 running on Debian Wheezy, and I am having too many open files exceptions when I try to perform a large number of operations in my 10 node cluster. I saw the documentation http://www.datastax.com/documentation/cassandra/2.0/cassandra/troubleshooting/trblshootTooManyFiles_r.html and I have set everything to the recommended settings, but I keep getting the errors. In the documentation it says: Another, much less likely possibility, is a file descriptor leak in Cassandra. Run lsof -n | grep java to check that the number of file descriptors opened by Java is reasonable and reports the error if the number is greater than a few thousand. I guess it's not the case, or else a lot of people would be complaining about it, but I am not sure what I could do to solve the problem. Any hint about how to solve it? My client is written in python and uses Cassandra Python Driver. Here are the exceptions I am having in the client: [s1log] 2014-08-08 12:16:09,631 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.151, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,632 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,633 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.143, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,634 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,634 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.145, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,635 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.144, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,635 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.148, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,732 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.146, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,733 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.77, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,734 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.76, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,734 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.75, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,735 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.142, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,736 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.185, scheduling retry in 600.0 seconds: [Errno 24] Too many open files [s1log] 2014-08-08 12:16:09,942 - cassandra.pool - WARNING - Error attempting to reconnect to 200.200.200.144, scheduling retry in 512.0 seconds: Timed out connecting to 200.200.200.144 [s1log] 2014-08-08 12:16:09,998 - cassandra.pool - WARNING - Error attempting
map reduce for Cassandra
Hi, I have the need to executing a map/reduce job to identity data stored in Cassandra before indexing this data to Elastic Search. I have already used ColumnFamilyInputFormat (before start using CQL) to write hadoop jobs to do that, but I use to have a lot of troubles to perform tunning, as hadoop depends on how map tasks are split in order to successfull execute things in parallel, for IO/bound processes. First question is: Am I the only one having problems with that? Is anyone else using hadoop jobs that reads from Cassandra in production? Second question is about the alternatives. I saw new version spark will have Cassandra support, but using CqlPagingInputFormat, from hadoop. I tried to use HIVE with Cassandra community, but it seems it only works with Cassandra Enterprise and doesn't do more than FB presto (http://prestodb.io/), which we have been using reading from Cassandra and so far it has been great for SQL-like queries. For custom map reduce jobs, however, it is not enough. Does anyone know some other tool that performs MR on Cassandra? My impression is most tools were created to work on top of HDFS and reading from a nosql db is some kind of workaround. Third question is about how these tools work. Most of them writtes mapped data on a intermediate storage, then data is shuffled and sorted, then it is reduced. Even when using CqlPagingInputFormat, if you are using hadoop it will write files to HDFS after the mapping phase, shuffle and sort this data, and then reduce it. I wonder if a tool supporting Cassandra out of the box wouldn't be smarter. Is it faster to write all your data to a file and then sorting it, or batch inserting data and already indexing it, as it happens when you store data in a Cassandra CF? I didn't do the calculations to check the complexity of each one, what should consider no index in Cassandra would be really large, as the maximum index size will always depend on the maximum capacity of a single host, but my guess is that a map / reduce tool written specifically to Cassandra, from the beggining, could perform much better than a tool written to HDFS and adapted. I hear people saying Map/Reduce on Cassandra/HBase is usually 30% slower than M/R in HDFS. Does it really make sense? Should we expect a result like this? Final question: Do you think writting a new M/R tool like described would be reinventing the wheel? Or it makes sense? Thanks in advance. Any opinions about this subject will be very appreciated. Best regards, Marcelo Valle.
Re: map reduce for Cassandra
Hi Jonathan, Do you know if this RDD can be used with Python? AFAIK, python + Cassandra will be supported just in the next version, but I would like to be wrong... Best regards, Marcelo Valle. 2014-07-21 13:06 GMT-03:00 Jonathan Haddad j...@jonhaddad.com: Hey Marcelo, You should check out spark. It intelligently deals with a lot of the issues you're mentioning. Al Tobey did a walkthrough of how to set up the OSS side of things here: http://tobert.github.io/post/2014-07-15-installing-cassandra-spark-stack.html It'll be less work than writing a M/R framework from scratch :) Jon On Mon, Jul 21, 2014 at 8:24 AM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi, I have the need to executing a map/reduce job to identity data stored in Cassandra before indexing this data to Elastic Search. I have already used ColumnFamilyInputFormat (before start using CQL) to write hadoop jobs to do that, but I use to have a lot of troubles to perform tunning, as hadoop depends on how map tasks are split in order to successfull execute things in parallel, for IO/bound processes. First question is: Am I the only one having problems with that? Is anyone else using hadoop jobs that reads from Cassandra in production? Second question is about the alternatives. I saw new version spark will have Cassandra support, but using CqlPagingInputFormat, from hadoop. I tried to use HIVE with Cassandra community, but it seems it only works with Cassandra Enterprise and doesn't do more than FB presto (http://prestodb.io/), which we have been using reading from Cassandra and so far it has been great for SQL-like queries. For custom map reduce jobs, however, it is not enough. Does anyone know some other tool that performs MR on Cassandra? My impression is most tools were created to work on top of HDFS and reading from a nosql db is some kind of workaround. Third question is about how these tools work. Most of them writtes mapped data on a intermediate storage, then data is shuffled and sorted, then it is reduced. Even when using CqlPagingInputFormat, if you are using hadoop it will write files to HDFS after the mapping phase, shuffle and sort this data, and then reduce it. I wonder if a tool supporting Cassandra out of the box wouldn't be smarter. Is it faster to write all your data to a file and then sorting it, or batch inserting data and already indexing it, as it happens when you store data in a Cassandra CF? I didn't do the calculations to check the complexity of each one, what should consider no index in Cassandra would be really large, as the maximum index size will always depend on the maximum capacity of a single host, but my guess is that a map / reduce tool written specifically to Cassandra, from the beggining, could perform much better than a tool written to HDFS and adapted. I hear people saying Map/Reduce on Cassandra/HBase is usually 30% slower than M/R in HDFS. Does it really make sense? Should we expect a result like this? Final question: Do you think writting a new M/R tool like described would be reinventing the wheel? Or it makes sense? Thanks in advance. Any opinions about this subject will be very appreciated. Best regards, Marcelo Valle. -- Jon Haddad http://www.rustyrazorblade.com skype: rustyrazorblade
Re: map reduce for Cassandra
Jonathan, By what I have read in the docs, Python API has some limitations yet, not being possible to use any hadoop binary input format. The python example for Cassandra is only in the master branch: https://github.com/apache/spark/blob/master/examples/src/main/python/cassandra_inputformat.py I may be lacking knowledge of Spark, but if I understood it correctly, the access to Cassandra data is still made by the CqlPagingInputFormat, from hadoop integration. Here is where I ask: even if Spark supports Cassandra, will it be fast enough? My understanding (please some correct me if I am wrong) is that when you insert N items in a Cassandra CF, you are executing N binary searches to insert the item already indexed by a key. When you read the data, it's already sorted. So you take O(N * log(N)) (binary search complexity to insert all data already sorted. However, by using a fast sort algorithm, you also take O(N * log(N)) to sort the data after ir was inserted, but then using more IO. If I write a job in Spark / Java with Cassandra, how will the mapped data be stored and sorted? Will it be stored in Cassandra too? Will spark run sort after the mapping? Best regards, Marcelo. 2014-07-21 14:06 GMT-03:00 Jonathan Haddad j...@jonhaddad.com: I haven't tried pyspark yet, but it's part of the distribution. My main language is Python too, so I intend on getting deep into it. On Mon, Jul 21, 2014 at 9:38 AM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi Jonathan, Do you know if this RDD can be used with Python? AFAIK, python + Cassandra will be supported just in the next version, but I would like to be wrong... Best regards, Marcelo Valle. 2014-07-21 13:06 GMT-03:00 Jonathan Haddad j...@jonhaddad.com: Hey Marcelo, You should check out spark. It intelligently deals with a lot of the issues you're mentioning. Al Tobey did a walkthrough of how to set up the OSS side of things here: http://tobert.github.io/post/2014-07-15-installing-cassandra-spark-stack.html It'll be less work than writing a M/R framework from scratch :) Jon On Mon, Jul 21, 2014 at 8:24 AM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi, I have the need to executing a map/reduce job to identity data stored in Cassandra before indexing this data to Elastic Search. I have already used ColumnFamilyInputFormat (before start using CQL) to write hadoop jobs to do that, but I use to have a lot of troubles to perform tunning, as hadoop depends on how map tasks are split in order to successfull execute things in parallel, for IO/bound processes. First question is: Am I the only one having problems with that? Is anyone else using hadoop jobs that reads from Cassandra in production? Second question is about the alternatives. I saw new version spark will have Cassandra support, but using CqlPagingInputFormat, from hadoop. I tried to use HIVE with Cassandra community, but it seems it only works with Cassandra Enterprise and doesn't do more than FB presto (http://prestodb.io/), which we have been using reading from Cassandra and so far it has been great for SQL-like queries. For custom map reduce jobs, however, it is not enough. Does anyone know some other tool that performs MR on Cassandra? My impression is most tools were created to work on top of HDFS and reading from a nosql db is some kind of workaround. Third question is about how these tools work. Most of them writtes mapped data on a intermediate storage, then data is shuffled and sorted, then it is reduced. Even when using CqlPagingInputFormat, if you are using hadoop it will write files to HDFS after the mapping phase, shuffle and sort this data, and then reduce it. I wonder if a tool supporting Cassandra out of the box wouldn't be smarter. Is it faster to write all your data to a file and then sorting it, or batch inserting data and already indexing it, as it happens when you store data in a Cassandra CF? I didn't do the calculations to check the complexity of each one, what should consider no index in Cassandra would be really large, as the maximum index size will always depend on the maximum capacity of a single host, but my guess is that a map / reduce tool written specifically to Cassandra, from the beggining, could perform much better than a tool written to HDFS and adapted. I hear people saying Map/Reduce on Cassandra/HBase is usually 30% slower than M/R in HDFS. Does it really make sense? Should we expect a result like this? Final question: Do you think writting a new M/R tool like described would be reinventing the wheel? Or it makes sense? Thanks in advance. Any opinions about this subject will be very appreciated. Best regards, Marcelo Valle. -- Jon Haddad http
Re: map reduce for Cassandra
Hi Robert, First of all, thanks for answering. 2014-07-21 20:18 GMT-03:00 Robert Coli rc...@eventbrite.com: You're wrong, unless you're talking about insertion into a memtable, which you probably aren't and which probably doesn't actually work that way enough to be meaningful. On disk, Cassandra has immutable datafiles, from which row fragments are merged into a row at read time. I'm pretty sure the rest of the stuff you said doesn't make any sense in light of this? Although several sstables (disk fragments) may have the same row key, inside a single sstable row keys and column keys are indexed, right? Otherwise, doing a GET in Cassandra would take some time. From the M/R perspective, I was reffering to the mem table, as I am trying to compare the time to insert in Cassandra against the time of sorting in hadoop. To make it more clear: hadoop has it's own partitioner, which is used after the map phase. The map output is written locally on each hadoop node, then it's shuffled from one node to the other (see slide 17 in this presentation: http://pt.slideshare.net/j_singh/nosql-and-mapreduce). In other words, you may read Cassandra data on hadoop, but the intermediate results are still stored in HDFS. Instead of using hadoop partitioner, I would like to store the intermediate results in a Cassandra CF, so the map output would go directly to an intermediate column family via batch inserts, instead of being written to a local disk first, then shuffled to the right node. Therefore, the mapper would write it's output the same way all data enters in Cassandra: first on a memtable, then being flush to a sstable, then read during the reduce phase. Shouldn't it be faster than storing intermediate results in HDFS? Best regards, Marcelo.
Re: map reduce for Cassandra
Hi, But if you are only relying on memtables to sort writes, that seems like a pretty heavyweight reason to use Cassandra? Actually, it's not a reason to use Cassandra. I already use Cassandra and I need to map reduce data from it. I am trying to see a reason to use the conventional M/R tools or to build a tool specific to Cassandra. but Cassandra, as a datastore with immutable data files, is not typically a good choice for short lived intermediate result sets... Indeed, but so far I am seeing it as the best option. I storing this intermediate files in HDFS is better, then I agree there is no reason to consider Cassandra to do it. are you planning to use DSE? Our company will probably hire DSE support when it reaches some size, but DSE as a product doesn't seem interesting to our case so far. The only tool that would help be at this moment would be HIVE, but honestly I didn't like the way DSE supports hive and I don't want to use a solution not available to DSC (see http://stackoverflow.com/questions/23959169/problems-using-hive-cassandra-community for details). []s 2014-07-21 22:09 GMT-03:00 Robert Coli rc...@eventbrite.com: On Mon, Jul 21, 2014 at 5:45 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Although several sstables (disk fragments) may have the same row key, inside a single sstable row keys and column keys are indexed, right? Otherwise, doing a GET in Cassandra would take some time. From the M/R perspective, I was reffering to the mem table, as I am trying to compare the time to insert in Cassandra against the time of sorting in hadoop. I was confused, because unless you are using new in-memory columnfamilies, which I believe are only available in DSE, there is no way to ensure that any given row stays in a memtable. Very rarely is there a view of the function of a memtable that only cares about its properties and not the closely related properties of SSTables. However yours is one of them, I see now why your question makes sense, you only care about the memtable for how quickly it sorts. But if you are only relying on memtables to sort writes, that seems like a pretty heavyweight reason to use Cassandra? I'm certainly not an expert in this area of Cassandra... but Cassandra, as a datastore with immutable data files, is not typically a good choice for short lived intermediate result sets... are you planning to use DSE? =Rob
Re: Batch of prepared statements exceeding specified threshold
Hi, I think it's a bit late for this reply, but anyway... We hired support from http://thelastpickle.com/ to solve our problem and thanks to them we were able to solve our issue as well. What was causing this behavior was a large query being executed by mistake in our code. It was needed to open the java heap with Cassandra source and find the query taking the large amount of memory. Lesson learned to me: Cassandra support is something really needed and useful, it's not like Oracle's, IBM, etc... Best regards, Marcelo Valle. 2014-06-20 18:15 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: Ok, in my case it was straightforward. It is just warning, which however says that batches with large data size (above 5Kb) can sometimes lead to node instability (why?). This limit seems to be hard-coded, I didn't find anyway to configure it externally. Anyway, removing batch and giving up atomicity, resolved the issue for me. http://mail-archives.apache.org/mod_mbox/cassandra-commits/201404.mbox/%3ceee5dd5bc4794ef0b5c5153fdb583...@git.apache.org%3E On Fri, Jun 20, 2014 at 3:55 PM, Pavel Kogan pavel.ko...@cortica.com wrote: Logged batch. On Fri, Jun 20, 2014 at 2:13 PM, DuyHai Doan doanduy...@gmail.com wrote: I think some figures from nodetool tpstats and nodetool compactionstats may help seeing clearer And Pavel, when you said batch, did you mean LOGGED batch or UNLOGGED batch ? On Fri, Jun 20, 2014 at 8:02 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: If you have 32 Gb RAM, the heap is probably 8Gb. 200 writes of 100 kb / s would be 20MB / s in the worst case, supposing all writes of a replica goes to a single node. I really don't see any reason why it should be filling up the heap. Anyone else? But did you check the logs for the GCInspector? In my case, nodes are falling because of the heap, in your case, maybe it's something else. Do you see increased times when looking for GCInspector in the logs? []s 2014-06-20 14:51 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: Hi Marcelo, No pending write tasks, I am writing a lot, about 100-200 writes each up to 100Kb every 15[s]. It is running on decent cluster of 5 identical nodes, quad cores i7 with 32Gb RAM and 480Gb SSD. Regards, Pavel On Fri, Jun 20, 2014 at 12:31 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Pavel, In my case, the heap was filling up faster than it was draining. I am still looking for the cause of it, as I could drain really fast with SSD. However, in your case you could check (AFAIK) nodetool tpstats and see if there are too many pending write tasks, for instance. Maybe you really are writting more than the nodes are able to flush to disk. How many writes per second are you achieving? Also, I would look for GCInspector in the log: cat system.log* | grep GCInspector | wc -l tail -1000 system.log | grep GCInspector Do you see it running a lot? Is it taking much more time to run each time it runs? I am no Cassandra expert, but I would try these things first and post the results here. Maybe other people in the list have more ideas. Best regards, Marcelo. 2014-06-20 8:50 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: The cluster is new, so no updates were done. Version 2.0.8. It happened when I did many writes (no reads). Writes are done in small batches of 2 inserts (writing to 2 column families). The values are big blobs (up to 100Kb). Any clues? Pavel On Thu, Jun 19, 2014 at 8:07 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Pavel, Out of curiosity, did it start to happen before some update? Which version of Cassandra are you using? []s 2014-06-19 16:10 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: What a coincidence! Today happened in my cluster of 7 nodes as well. Regards, Pavel On Wed, Jun 18, 2014 at 11:13 AM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: I have a 10 node cluster with cassandra 2.0.8. I am taking this exceptions in the log when I run my code. What my code does is just reading data from a CF and in some cases it writes new data. WARN [Native-Transport-Requests:553] 2014-06-18 11:04:51,391 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 6165, exceeding specified threshold of 5120 by 1045. WARN [Native-Transport-Requests:583] 2014-06-18 11:05:01,152 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 21266, exceeding specified threshold of 5120 by 16146. WARN [Native-Transport-Requests:581] 2014-06-18 11:05:20,229 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 22978, exceeding specified threshold of 5120 by 17858. INFO [MemoryMeter:1] 2014-06-18 11:05:32,682 Memtable.java (line
Re: Best way to do a multi_get using CQL
Yes, I am using the CQL datastax drivers.
It was a good advice, thanks a lot Janathan.
[]s
2014-06-20 0:28 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
The only case in which it might be better to use an IN clause is if
the entire query can be satisfied from that machine. Otherwise, go
async.
The native driver reuses connections and intelligently manages the
pool for you. It can also multiplex queries over a single connection.
I am assuming you're using one of the datastax drivers for CQL, btw.
Jon
On Thu, Jun 19, 2014 at 7:37 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
This is interesting, I didn't know that!
It might make sense then to use select = + async + token aware, I will
try
to change my code.
But would it be a recomended solution for these cases? Any other
options?
I still would if this is the right use case for Cassandra, to look for
random keys in a huge cluster. After all, the amount of connections to
Cassandra will still be huge, right... Wouldn't it be a problem?
Or when you use async the driver reuses the connection?
[]s
2014-06-19 22:16 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
If you use async and your driver is token aware, it will go to the
proper node, rather than requiring the coordinator to do so.
Realistically you're going to have a connection open to every server
anyways. It's the difference between you querying for the data
directly and using a coordinator as a proxy. It's faster to just ask
the node with the data.
On Thu, Jun 19, 2014 at 6:11 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
But using async queries wouldn't be even worse than using SELECT IN?
The justification in the docs is I could query many nodes, but I would
still
do it.
Today, I use both async queries AND SELECT IN:
SELECT_ENTITY_LOOKUP = SELECT entity_id FROM + ENTITY_LOOKUP +
WHERE
name=%s and value in(%s)
for name, values in identifiers.items():
query = self.SELECT_ENTITY_LOOKUP % ('%s',
','.join(['%s']*len(values)))
args = [name] + values
query_msg = query % tuple(args)
futures.append((query_msg, self.session.execute_async(query,
args)))
for query_msg, future in futures:
try:
rows = future.result(timeout=10)
for row in rows:
entity_ids.add(row.entity_id)
except:
logging.error(Query '%s' returned ERROR % (query_msg))
raise
Using async just with select = would mean instead of 1 async query
(example:
in (0, 1, 2)), I would do several, one for each value of values
array
above.
In my head, this would mean more connections to Cassandra and the same
amount of work, right? What would be the advantage?
[]s
2014-06-19 22:01 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
Your other option is to fire off async queries. It's pretty
straightforward w/ the java or python drivers.
On Thu, Jun 19, 2014 at 5:56 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
I was taking a look at Cassandra anti-patterns list:
http://www.datastax.com/documentation/cassandra/2.0/cassandra/architecture/architecturePlanningAntiPatterns_c.html
Among then is
SELECT ... IN or index lookups¶
SELECT ... IN and index lookups (formerly secondary indexes) should
be
avoided except for specific scenarios. See When not to use IN in
SELECT
and
When not to use an index in Indexing in
CQL for Cassandra 2.0
And Looking at the SELECT doc, I saw:
When not to use IN¶
The recommendations about when not to use an index apply to using
IN
in
the
WHERE clause. Under most conditions, using IN in the WHERE clause
is
not
recommended. Using IN can degrade performance because usually many
nodes
must be queried. For example, in a single, local data center
cluster
having
30 nodes, a replication factor of 3, and a consistency level of
LOCAL_QUORUM, a single key query goes out to two nodes, but if the
query
uses the IN condition, the number of nodes being queried are most
likely
even higher, up to 20 nodes depending on where the keys fall in the
token
range.
In my system, I have a column family called entity_lookup:
CREATE KEYSPACE IF NOT EXISTS Identification1
WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy',
'DC1' : 3 };
USE Identification1;
CREATE TABLE IF NOT EXISTS entity_lookup (
name varchar,
value varchar,
entity_id uuid,
PRIMARY KEY ((name, value), entity_id));
And I use the following select to query it:
SELECT entity_id FROM entity_lookup WHERE name=%s and value in(%s)
Is this an anti-pattern?
If not using SELECT IN, which other way would you recomend for
lookups
like
that? I have
Re: Batch of prepared statements exceeding specified threshold
Pavel, In my case, the heap was filling up faster than it was draining. I am still looking for the cause of it, as I could drain really fast with SSD. However, in your case you could check (AFAIK) nodetool tpstats and see if there are too many pending write tasks, for instance. Maybe you really are writting more than the nodes are able to flush to disk. How many writes per second are you achieving? Also, I would look for GCInspector in the log: cat system.log* | grep GCInspector | wc -l tail -1000 system.log | grep GCInspector Do you see it running a lot? Is it taking much more time to run each time it runs? I am no Cassandra expert, but I would try these things first and post the results here. Maybe other people in the list have more ideas. Best regards, Marcelo. 2014-06-20 8:50 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: The cluster is new, so no updates were done. Version 2.0.8. It happened when I did many writes (no reads). Writes are done in small batches of 2 inserts (writing to 2 column families). The values are big blobs (up to 100Kb). Any clues? Pavel On Thu, Jun 19, 2014 at 8:07 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Pavel, Out of curiosity, did it start to happen before some update? Which version of Cassandra are you using? []s 2014-06-19 16:10 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: What a coincidence! Today happened in my cluster of 7 nodes as well. Regards, Pavel On Wed, Jun 18, 2014 at 11:13 AM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: I have a 10 node cluster with cassandra 2.0.8. I am taking this exceptions in the log when I run my code. What my code does is just reading data from a CF and in some cases it writes new data. WARN [Native-Transport-Requests:553] 2014-06-18 11:04:51,391 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 6165, exceeding specified threshold of 5120 by 1045. WARN [Native-Transport-Requests:583] 2014-06-18 11:05:01,152 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 21266, exceeding specified threshold of 5120 by 16146. WARN [Native-Transport-Requests:581] 2014-06-18 11:05:20,229 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 22978, exceeding specified threshold of 5120 by 17858. INFO [MemoryMeter:1] 2014-06-18 11:05:32,682 Memtable.java (line 481) CFS(Keyspace='OpsCenter', ColumnFamily='rollups300') liveRatio is 14.249755859375 (just-counted was 9.85302734375). calculation took 3ms for 1024 cells After some time, one node of the cluster goes down. Then it goes back after some seconds and another node goes down. It keeps happening and there is always a node down in the cluster, when it goes back another one falls. The only exceptions I see in the log is connected reset by the peer, which seems to be relative to gossip protocol, when a node goes down. Any hint of what could I do to investigate this problem further? Best regards, Marcelo Valle.
Re: Best way to do a multi_get using CQL
A question, not sure if you guys know the answer:
Supose I async query 1000 rows using token aware and suppose I have 10
nodes. Suppose also each node would receive 100 row queries each.
How does async work in this case? Would it send each row query to each node
in a different connection? Different message?
I guess if there was a way to use batch with async, once you commit the
batch for the 1000 queries, it would create 1 connection to each host and
query 100 rows in a single message to each host.
This would decrease resource usage, am I wrong?
[]s
2014-06-20 12:12 GMT-03:00 Jeremy Jongsma jer...@barchart.com:
I've found that if you have any amount of latency between your client and
nodes, and you are executing a large batch of queries, you'll usually want
to send them together to one node unless execution time is of no concern.
The tradeoff is resource usage on the connected node vs. time to complete
all the queries, because you'll need fewer client - node network round
trips.
With large numbers of queries you will still want to make sure you split
them into manageable batches before sending them, to control memory usage
on the executing node. I've been limiting queries to batches of 100 keys in
scenarios like this.
On Fri, Jun 20, 2014 at 5:59 AM, Laing, Michael michael.la...@nytimes.com
wrote:
However my extensive benchmarking this week of the python driver from
master shows a performance *decrease* when using 'token_aware'.
This is on 12-node, 2-datacenter, RF-3 cluster in AWS.
Also why do the work the coordinator will do for you: send all the
queries, wait for everything to come back in whatever order, and sort the
result.
I would rather keep my app code simple.
But the real point is that you should benchmark in your own environment.
ml
On Fri, Jun 20, 2014 at 3:29 AM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
Yes, I am using the CQL datastax drivers.
It was a good advice, thanks a lot Janathan.
[]s
2014-06-20 0:28 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
The only case in which it might be better to use an IN clause is if
the entire query can be satisfied from that machine. Otherwise, go
async.
The native driver reuses connections and intelligently manages the
pool for you. It can also multiplex queries over a single connection.
I am assuming you're using one of the datastax drivers for CQL, btw.
Jon
On Thu, Jun 19, 2014 at 7:37 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
This is interesting, I didn't know that!
It might make sense then to use select = + async + token aware, I
will try
to change my code.
But would it be a recomended solution for these cases? Any other
options?
I still would if this is the right use case for Cassandra, to look for
random keys in a huge cluster. After all, the amount of connections to
Cassandra will still be huge, right... Wouldn't it be a problem?
Or when you use async the driver reuses the connection?
[]s
2014-06-19 22:16 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
If you use async and your driver is token aware, it will go to the
proper node, rather than requiring the coordinator to do so.
Realistically you're going to have a connection open to every server
anyways. It's the difference between you querying for the data
directly and using a coordinator as a proxy. It's faster to just ask
the node with the data.
On Thu, Jun 19, 2014 at 6:11 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
But using async queries wouldn't be even worse than using SELECT
IN?
The justification in the docs is I could query many nodes, but I
would
still
do it.
Today, I use both async queries AND SELECT IN:
SELECT_ENTITY_LOOKUP = SELECT entity_id FROM + ENTITY_LOOKUP +
WHERE
name=%s and value in(%s)
for name, values in identifiers.items():
query = self.SELECT_ENTITY_LOOKUP % ('%s',
','.join(['%s']*len(values)))
args = [name] + values
query_msg = query % tuple(args)
futures.append((query_msg, self.session.execute_async(query,
args)))
for query_msg, future in futures:
try:
rows = future.result(timeout=10)
for row in rows:
entity_ids.add(row.entity_id)
except:
logging.error(Query '%s' returned ERROR % (query_msg))
raise
Using async just with select = would mean instead of 1 async query
(example:
in (0, 1, 2)), I would do several, one for each value of values
array
above.
In my head, this would mean more connections to Cassandra and the
same
amount of work, right? What would be the advantage?
[]s
2014-06-19 22:01 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
Your other option is to fire off async queries. It's pretty
straightforward w/ the java or python drivers.
On Thu, Jun 19, 2014 at 5:56 PM, Marcelo Elias Del Valle
marc
Re: Batch of prepared statements exceeding specified threshold
If you have 32 Gb RAM, the heap is probably 8Gb. 200 writes of 100 kb / s would be 20MB / s in the worst case, supposing all writes of a replica goes to a single node. I really don't see any reason why it should be filling up the heap. Anyone else? But did you check the logs for the GCInspector? In my case, nodes are falling because of the heap, in your case, maybe it's something else. Do you see increased times when looking for GCInspector in the logs? []s 2014-06-20 14:51 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: Hi Marcelo, No pending write tasks, I am writing a lot, about 100-200 writes each up to 100Kb every 15[s]. It is running on decent cluster of 5 identical nodes, quad cores i7 with 32Gb RAM and 480Gb SSD. Regards, Pavel On Fri, Jun 20, 2014 at 12:31 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Pavel, In my case, the heap was filling up faster than it was draining. I am still looking for the cause of it, as I could drain really fast with SSD. However, in your case you could check (AFAIK) nodetool tpstats and see if there are too many pending write tasks, for instance. Maybe you really are writting more than the nodes are able to flush to disk. How many writes per second are you achieving? Also, I would look for GCInspector in the log: cat system.log* | grep GCInspector | wc -l tail -1000 system.log | grep GCInspector Do you see it running a lot? Is it taking much more time to run each time it runs? I am no Cassandra expert, but I would try these things first and post the results here. Maybe other people in the list have more ideas. Best regards, Marcelo. 2014-06-20 8:50 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: The cluster is new, so no updates were done. Version 2.0.8. It happened when I did many writes (no reads). Writes are done in small batches of 2 inserts (writing to 2 column families). The values are big blobs (up to 100Kb). Any clues? Pavel On Thu, Jun 19, 2014 at 8:07 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Pavel, Out of curiosity, did it start to happen before some update? Which version of Cassandra are you using? []s 2014-06-19 16:10 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: What a coincidence! Today happened in my cluster of 7 nodes as well. Regards, Pavel On Wed, Jun 18, 2014 at 11:13 AM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: I have a 10 node cluster with cassandra 2.0.8. I am taking this exceptions in the log when I run my code. What my code does is just reading data from a CF and in some cases it writes new data. WARN [Native-Transport-Requests:553] 2014-06-18 11:04:51,391 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 6165, exceeding specified threshold of 5120 by 1045. WARN [Native-Transport-Requests:583] 2014-06-18 11:05:01,152 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 21266, exceeding specified threshold of 5120 by 16146. WARN [Native-Transport-Requests:581] 2014-06-18 11:05:20,229 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 22978, exceeding specified threshold of 5120 by 17858. INFO [MemoryMeter:1] 2014-06-18 11:05:32,682 Memtable.java (line 481) CFS(Keyspace='OpsCenter', ColumnFamily='rollups300') liveRatio is 14.249755859375 (just-counted was 9.85302734375). calculation took 3ms for 1024 cells After some time, one node of the cluster goes down. Then it goes back after some seconds and another node goes down. It keeps happening and there is always a node down in the cluster, when it goes back another one falls. The only exceptions I see in the log is connected reset by the peer, which seems to be relative to gossip protocol, when a node goes down. Any hint of what could I do to investigate this problem further? Best regards, Marcelo Valle.
Re: Custom snitch classpath?
This is nice! I was looking for something like this to implement a multi DC cluster between OVh and Amazon. Thanks for sharing! []s 2014-06-20 15:35 GMT-03:00 Jeremy Jongsma jer...@barchart.com: Sharing in case anyone else wants to use this: https://github.com/barchart/cassandra-plugins/blob/master/src/main/java/com/barchart/cassandra/plugins/snitch/GossipingPropertyFileWithEC2FallbackSnitch.java Basically it is a proxy that attempts to use GossipingPropertyFileSnitch, and it that fails to initialize due to missing rack or datacenter values, it falls back to Ec2MultiRegionSnitch. We are using it for hybrid cloud deployments between AWS and our private datacenter. On Fri, Jun 20, 2014 at 1:04 PM, Tyler Hobbs ty...@datastax.com wrote: The lib directory (where all the other jars are). bin/cassandra.in.sh does this: for jar in $CASSANDRA_HOME/lib/*.jar; do CLASSPATH=$CLASSPATH:$jar done On Fri, Jun 20, 2014 at 12:58 PM, Jeremy Jongsma jer...@barchart.com wrote: Where do I add my custom snitch JAR to the Cassandra classpath so I can use it? -- Tyler Hobbs DataStax http://datastax.com/
Re: Best way to do a multi_get using CQL
I am using python + CQL Driver.
I wonder how they do...
These things seems little important, but they are fundamental to get a good
performance in Cassandra...
I wish there was a simpler way to query in batches. Opening a large amount
of connections and sending 1 message at a time seems bad to me, as
sometimes you want to work with small rows.
It's no surprise Cassandra performs better when we use average row sizes.
But honestly I disagree with this part of Cassandra/Driver's design.
[]s
2014-06-20 14:37 GMT-03:00 Jeremy Jongsma jer...@barchart.com:
That depends on the connection pooling implementation in your driver.
Astyanax will keep N connections open to each node (configurable) and route
each query in a separate message over an existing connection, waiting until
one becomes available if all are in use.
On Fri, Jun 20, 2014 at 12:32 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
A question, not sure if you guys know the answer:
Supose I async query 1000 rows using token aware and suppose I have 10
nodes. Suppose also each node would receive 100 row queries each.
How does async work in this case? Would it send each row query to each
node in a different connection? Different message?
I guess if there was a way to use batch with async, once you commit the
batch for the 1000 queries, it would create 1 connection to each host and
query 100 rows in a single message to each host.
This would decrease resource usage, am I wrong?
[]s
2014-06-20 12:12 GMT-03:00 Jeremy Jongsma jer...@barchart.com:
I've found that if you have any amount of latency between your client and
nodes, and you are executing a large batch of queries, you'll usually want
to send them together to one node unless execution time is of no concern.
The tradeoff is resource usage on the connected node vs. time to complete
all the queries, because you'll need fewer client - node network round
trips.
With large numbers of queries you will still want to make sure you split
them into manageable batches before sending them, to control memory usage
on the executing node. I've been limiting queries to batches of 100 keys in
scenarios like this.
On Fri, Jun 20, 2014 at 5:59 AM, Laing, Michael
michael.la...@nytimes.com wrote:
However my extensive benchmarking this week of the python driver from
master shows a performance *decrease* when using 'token_aware'.
This is on 12-node, 2-datacenter, RF-3 cluster in AWS.
Also why do the work the coordinator will do for you: send all the
queries, wait for everything to come back in whatever order, and sort the
result.
I would rather keep my app code simple.
But the real point is that you should benchmark in your own environment.
ml
On Fri, Jun 20, 2014 at 3:29 AM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
Yes, I am using the CQL datastax drivers.
It was a good advice, thanks a lot Janathan.
[]s
2014-06-20 0:28 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
The only case in which it might be better to use an IN clause is if
the entire query can be satisfied from that machine. Otherwise, go
async.
The native driver reuses connections and intelligently manages the
pool for you. It can also multiplex queries over a single connection.
I am assuming you're using one of the datastax drivers for CQL, btw.
Jon
On Thu, Jun 19, 2014 at 7:37 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
This is interesting, I didn't know that!
It might make sense then to use select = + async + token aware, I
will try
to change my code.
But would it be a recomended solution for these cases? Any other
options?
I still would if this is the right use case for Cassandra, to look
for
random keys in a huge cluster. After all, the amount of connections
to
Cassandra will still be huge, right... Wouldn't it be a problem?
Or when you use async the driver reuses the connection?
[]s
2014-06-19 22:16 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
If you use async and your driver is token aware, it will go to the
proper node, rather than requiring the coordinator to do so.
Realistically you're going to have a connection open to every
server
anyways. It's the difference between you querying for the data
directly and using a coordinator as a proxy. It's faster to just
ask
the node with the data.
On Thu, Jun 19, 2014 at 6:11 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
But using async queries wouldn't be even worse than using SELECT
IN?
The justification in the docs is I could query many nodes, but I
would
still
do it.
Today, I use both async queries AND SELECT IN:
SELECT_ENTITY_LOOKUP = SELECT entity_id FROM + ENTITY_LOOKUP
+
WHERE
name=%s and value in(%s)
for name, values in identifiers.items():
query = self.SELECT_ENTITY_LOOKUP % ('%s',
','.join(['%s']*len(values)))
args = [name] + values
query_msg
Re: Batch of prepared statements exceeding specified threshold
-06-19 19:45:59,412 GCInspector.java (line 116) GC for ConcurrentMarkSweep: 36616 ms for 2 collections, 8494231408 used; max is 8506048512 INFO [ScheduledTasks:1] 2014-06-19 19:47:27,072 GCInspector.java (line 116) GC for ConcurrentMarkSweep: 87408 ms for 5 collections, 8497634840 used; max is 8506048512 INFO [ScheduledTasks:1] 2014-06-19 19:48:03,811 GCInspector.java (line 116) GC for ConcurrentMarkSweep: 36654 ms for 2 collections, 8499869936 used; max is 8506048512 INFO [ScheduledTasks:1] 2014-06-19 19:50:15,419 GCInspector.java (line 116) GC for ConcurrentMarkSweep: 131381 ms for 7 collections, 8504981328 used; max is 8506048512 INFO [ScheduledTasks:1] 2014-06-19 19:52:05,398 GCInspector.java (line 116) GC for ConcurrentMarkSweep: 109789 ms for 6 collections, 8505124200 used; max is 8506048512 INFO [ScheduledTasks:1] 2014-06-19 19:54:25,825 GCInspector.java (line 116) GC for ConcurrentMarkSweep: 140241 ms for 8 collections, 8503231888 used; max is 8506048512 INFO [ScheduledTasks:1] 2014-06-19 19:56:13,206 GCInspector.java (line 116) GC for ConcurrentMarkSweep: 106587 ms for 7 collections, 784762904 used; max is 8506048512 Best regards, Marcelo Valle. 2014-06-19 16:10 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: What a coincidence! Today happened in my cluster of 7 nodes as well. Regards, Pavel On Wed, Jun 18, 2014 at 11:13 AM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: I have a 10 node cluster with cassandra 2.0.8. I am taking this exceptions in the log when I run my code. What my code does is just reading data from a CF and in some cases it writes new data. WARN [Native-Transport-Requests:553] 2014-06-18 11:04:51,391 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 6165, exceeding specified threshold of 5120 by 1045. WARN [Native-Transport-Requests:583] 2014-06-18 11:05:01,152 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 21266, exceeding specified threshold of 5120 by 16146. WARN [Native-Transport-Requests:581] 2014-06-18 11:05:20,229 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 22978, exceeding specified threshold of 5120 by 17858. INFO [MemoryMeter:1] 2014-06-18 11:05:32,682 Memtable.java (line 481) CFS(Keyspace='OpsCenter', ColumnFamily='rollups300') liveRatio is 14.249755859375 (just-counted was 9.85302734375). calculation took 3ms for 1024 cells After some time, one node of the cluster goes down. Then it goes back after some seconds and another node goes down. It keeps happening and there is always a node down in the cluster, when it goes back another one falls. The only exceptions I see in the log is connected reset by the peer, which seems to be relative to gossip protocol, when a node goes down. Any hint of what could I do to investigate this problem further? Best regards, Marcelo Valle.
Re: Batch of prepared statements exceeding specified threshold
Pavel, Out of curiosity, did it start to happen before some update? Which version of Cassandra are you using? []s 2014-06-19 16:10 GMT-03:00 Pavel Kogan pavel.ko...@cortica.com: What a coincidence! Today happened in my cluster of 7 nodes as well. Regards, Pavel On Wed, Jun 18, 2014 at 11:13 AM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: I have a 10 node cluster with cassandra 2.0.8. I am taking this exceptions in the log when I run my code. What my code does is just reading data from a CF and in some cases it writes new data. WARN [Native-Transport-Requests:553] 2014-06-18 11:04:51,391 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 6165, exceeding specified threshold of 5120 by 1045. WARN [Native-Transport-Requests:583] 2014-06-18 11:05:01,152 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 21266, exceeding specified threshold of 5120 by 16146. WARN [Native-Transport-Requests:581] 2014-06-18 11:05:20,229 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 22978, exceeding specified threshold of 5120 by 17858. INFO [MemoryMeter:1] 2014-06-18 11:05:32,682 Memtable.java (line 481) CFS(Keyspace='OpsCenter', ColumnFamily='rollups300') liveRatio is 14.249755859375 (just-counted was 9.85302734375). calculation took 3ms for 1024 cells After some time, one node of the cluster goes down. Then it goes back after some seconds and another node goes down. It keeps happening and there is always a node down in the cluster, when it goes back another one falls. The only exceptions I see in the log is connected reset by the peer, which seems to be relative to gossip protocol, when a node goes down. Any hint of what could I do to investigate this problem further? Best regards, Marcelo Valle.
Best way to do a multi_get using CQL
I was taking a look at Cassandra anti-patterns list:
http://www.datastax.com/documentation/cassandra/2.0/cassandra/architecture/architecturePlanningAntiPatterns_c.html
Among then is
SELECT ... IN or index lookups¶
http://www.datastax.com/documentation/cassandra/2.0/cassandra/architecture/architecturePlanningAntiPatterns_c.html?scroll=archPlanAntiPattern__AntiPatMultiGet
SELECT ... IN and index lookups (formerly secondary indexes) should be
avoided except for specific scenarios. See *When not to use IN* in SELECT
http://www.datastax.com/documentation/cql/3.1/cql/cql_reference/select_r.html
and *When not to use an index* in Indexing
http://www.datastax.com/documentation/cql/3.1/cql/ddl/ddl_primary_index_c.html
in
*CQL for Cassandra 2.0*
And Looking at the SELECT doc, I saw:
When *not* to use IN¶
http://www.datastax.com/documentation/cql/3.1/cql/cql_reference/select_r.html?scroll=reference_ds_d35_v2q_xj__selectInNot
The recommendations about when not to use an index
http://www.datastax.com/documentation/cql/3.1/cql/ddl/ddl_when_use_index_c.html
apply to using IN in the WHERE clause. Under most conditions, using IN in
the WHERE clause is not recommended. Using IN can degrade performance
because usually many nodes must be queried. For example, in a single, local
data center cluster having 30 nodes, a replication factor of 3, and a
consistency level of LOCAL_QUORUM, a single key query goes out to two
nodes, but if the query uses the IN condition, the number of nodes being
queried are most likely even higher, up to 20 nodes depending on where the
keys fall in the token range.
In my system, I have a column family called entity_lookup:
CREATE KEYSPACE IF NOT EXISTS Identification1
WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy',
'DC1' : 3 };
USE Identification1;
CREATE TABLE IF NOT EXISTS entity_lookup (
name varchar,
value varchar,
entity_id uuid,
PRIMARY KEY ((name, value), entity_id));
And I use the following select to query it:
SELECT entity_id FROM entity_lookup WHERE name=%s and value in(%s)
Is this an anti-pattern?
If not using SELECT IN, which other way would you recomend for lookups like
that? I have several values I would like to search in cassandra and they
might not be in the same particion, as above.
Is Cassandra the wrong tool for lookups like that?
Best regards,
Marcelo Valle.
Re: Best way to do a multi_get using CQL
But using async queries wouldn't be even worse than using SELECT IN?
The justification in the docs is I could query many nodes, but I would
still do it.
Today, I use both async queries AND SELECT IN:
SELECT_ENTITY_LOOKUP = SELECT entity_id FROM + ENTITY_LOOKUP + WHERE
name=%s and value in(%s)
for name, values in identifiers.items():
query = self.SELECT_ENTITY_LOOKUP % ('%s', ','.join(['%s']*len(values)))
args = [name] + values
query_msg = query % tuple(args)
futures.append((query_msg, self.session.execute_async(query, args)))
for query_msg, future in futures:
try:
rows = future.result(timeout=10)
for row in rows:
entity_ids.add(row.entity_id)
except:
logging.error(Query '%s' returned ERROR % (query_msg))
raise
Using async just with select = would mean instead of 1 async query
(example: in (0, 1, 2)), I would do several, one for each value of values
array above.
In my head, this would mean more connections to Cassandra and the same
amount of work, right? What would be the advantage?
[]s
2014-06-19 22:01 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
Your other option is to fire off async queries. It's pretty
straightforward w/ the java or python drivers.
On Thu, Jun 19, 2014 at 5:56 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
I was taking a look at Cassandra anti-patterns list:
http://www.datastax.com/documentation/cassandra/2.0/cassandra/architecture/architecturePlanningAntiPatterns_c.html
Among then is
SELECT ... IN or index lookups¶
SELECT ... IN and index lookups (formerly secondary indexes) should be
avoided except for specific scenarios. See When not to use IN in SELECT
and
When not to use an index in Indexing in
CQL for Cassandra 2.0
And Looking at the SELECT doc, I saw:
When not to use IN¶
The recommendations about when not to use an index apply to using IN in
the
WHERE clause. Under most conditions, using IN in the WHERE clause is not
recommended. Using IN can degrade performance because usually many nodes
must be queried. For example, in a single, local data center cluster
having
30 nodes, a replication factor of 3, and a consistency level of
LOCAL_QUORUM, a single key query goes out to two nodes, but if the query
uses the IN condition, the number of nodes being queried are most likely
even higher, up to 20 nodes depending on where the keys fall in the token
range.
In my system, I have a column family called entity_lookup:
CREATE KEYSPACE IF NOT EXISTS Identification1
WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy',
'DC1' : 3 };
USE Identification1;
CREATE TABLE IF NOT EXISTS entity_lookup (
name varchar,
value varchar,
entity_id uuid,
PRIMARY KEY ((name, value), entity_id));
And I use the following select to query it:
SELECT entity_id FROM entity_lookup WHERE name=%s and value in(%s)
Is this an anti-pattern?
If not using SELECT IN, which other way would you recomend for lookups
like
that? I have several values I would like to search in cassandra and they
might not be in the same particion, as above.
Is Cassandra the wrong tool for lookups like that?
Best regards,
Marcelo Valle.
--
Jon Haddad
http://www.rustyrazorblade.com
skype: rustyrazorblade
Re: Best way to do a multi_get using CQL
This is interesting, I didn't know that!
It might make sense then to use select = + async + token aware, I will try
to change my code.
But would it be a recomended solution for these cases? Any other options?
I still would if this is the right use case for Cassandra, to look for
random keys in a huge cluster. After all, the amount of connections to
Cassandra will still be huge, right... Wouldn't it be a problem?
Or when you use async the driver reuses the connection?
[]s
2014-06-19 22:16 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
If you use async and your driver is token aware, it will go to the
proper node, rather than requiring the coordinator to do so.
Realistically you're going to have a connection open to every server
anyways. It's the difference between you querying for the data
directly and using a coordinator as a proxy. It's faster to just ask
the node with the data.
On Thu, Jun 19, 2014 at 6:11 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
But using async queries wouldn't be even worse than using SELECT IN?
The justification in the docs is I could query many nodes, but I would
still
do it.
Today, I use both async queries AND SELECT IN:
SELECT_ENTITY_LOOKUP = SELECT entity_id FROM + ENTITY_LOOKUP + WHERE
name=%s and value in(%s)
for name, values in identifiers.items():
query = self.SELECT_ENTITY_LOOKUP % ('%s',
','.join(['%s']*len(values)))
args = [name] + values
query_msg = query % tuple(args)
futures.append((query_msg, self.session.execute_async(query, args)))
for query_msg, future in futures:
try:
rows = future.result(timeout=10)
for row in rows:
entity_ids.add(row.entity_id)
except:
logging.error(Query '%s' returned ERROR % (query_msg))
raise
Using async just with select = would mean instead of 1 async query
(example:
in (0, 1, 2)), I would do several, one for each value of values array
above.
In my head, this would mean more connections to Cassandra and the same
amount of work, right? What would be the advantage?
[]s
2014-06-19 22:01 GMT-03:00 Jonathan Haddad j...@jonhaddad.com:
Your other option is to fire off async queries. It's pretty
straightforward w/ the java or python drivers.
On Thu, Jun 19, 2014 at 5:56 PM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
I was taking a look at Cassandra anti-patterns list:
http://www.datastax.com/documentation/cassandra/2.0/cassandra/architecture/architecturePlanningAntiPatterns_c.html
Among then is
SELECT ... IN or index lookups¶
SELECT ... IN and index lookups (formerly secondary indexes) should be
avoided except for specific scenarios. See When not to use IN in
SELECT
and
When not to use an index in Indexing in
CQL for Cassandra 2.0
And Looking at the SELECT doc, I saw:
When not to use IN¶
The recommendations about when not to use an index apply to using IN
in
the
WHERE clause. Under most conditions, using IN in the WHERE clause is
not
recommended. Using IN can degrade performance because usually many
nodes
must be queried. For example, in a single, local data center cluster
having
30 nodes, a replication factor of 3, and a consistency level of
LOCAL_QUORUM, a single key query goes out to two nodes, but if the
query
uses the IN condition, the number of nodes being queried are most
likely
even higher, up to 20 nodes depending on where the keys fall in the
token
range.
In my system, I have a column family called entity_lookup:
CREATE KEYSPACE IF NOT EXISTS Identification1
WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy',
'DC1' : 3 };
USE Identification1;
CREATE TABLE IF NOT EXISTS entity_lookup (
name varchar,
value varchar,
entity_id uuid,
PRIMARY KEY ((name, value), entity_id));
And I use the following select to query it:
SELECT entity_id FROM entity_lookup WHERE name=%s and value in(%s)
Is this an anti-pattern?
If not using SELECT IN, which other way would you recomend for lookups
like
that? I have several values I would like to search in cassandra and
they
might not be in the same particion, as above.
Is Cassandra the wrong tool for lookups like that?
Best regards,
Marcelo Valle.
--
Jon Haddad
http://www.rustyrazorblade.com
skype: rustyrazorblade
--
Jon Haddad
http://www.rustyrazorblade.com
skype: rustyrazorblade
Re: incremental backups
Wouldn't be better to use nodetool clearsnapshot? []s 2014-06-14 17:38 GMT-03:00 S C as...@outlook.com: I am thinking of rm file.db once the backup is complete. Any special cases to be careful about? -Kumar -- Date: Sat, 14 Jun 2014 13:13:10 -0700 Subject: Re: incremental backups From: psanf...@retailnext.net To: user@cassandra.apache.org You should delete the backup files once you have copied them off. Otherwise they will start to use disk space as the live SSTables diverge from the snapshots/incrementals. -psanford On Sat, Jun 14, 2014 at 10:17 AM, S C as...@outlook.com wrote: Is it ok to delete files from backups directory (hardlinks) once I have it copied over remotely? Any caution to take? Thanks, Kumar
Batch of prepared statements exceeding specified threshold
I have a 10 node cluster with cassandra 2.0.8. I am taking this exceptions in the log when I run my code. What my code does is just reading data from a CF and in some cases it writes new data. WARN [Native-Transport-Requests:553] 2014-06-18 11:04:51,391 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 6165, exceeding specified threshold of 5120 by 1045. WARN [Native-Transport-Requests:583] 2014-06-18 11:05:01,152 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 21266, exceeding specified threshold of 5120 by 16146. WARN [Native-Transport-Requests:581] 2014-06-18 11:05:20,229 BatchStatement.java (line 228) Batch of prepared statements for [identification1.entity, identification1.entity_lookup] is of size 22978, exceeding specified threshold of 5120 by 17858. INFO [MemoryMeter:1] 2014-06-18 11:05:32,682 Memtable.java (line 481) CFS(Keyspace='OpsCenter', ColumnFamily='rollups300') liveRatio is 14.249755859375 (just-counted was 9.85302734375). calculation took 3ms for 1024 cells After some time, one node of the cluster goes down. Then it goes back after some seconds and another node goes down. It keeps happening and there is always a node down in the cluster, when it goes back another one falls. The only exceptions I see in the log is connected reset by the peer, which seems to be relative to gossip protocol, when a node goes down. Any hint of what could I do to investigate this problem further? Best regards, Marcelo Valle.
Re: running out of diskspace during maintenance tasks
AFAIK, when you run a repair a snapshot is created. After the repair, I run nodetool clearsnapshot to save disk space. Not sure it's you case or not. []s 2014-06-18 13:10 GMT-03:00 Brian Tarbox tar...@cabotresearch.com: We do a repair -pr on each node once a week on a rolling basis. Should we be running cleanup as well? My understanding that was only used after adding/removing nodes? We'd like to avoid adding nodes if possible (which might not be). Still curious if we can get C* to do the maintenance task on a separate volume. Thanks. On Wed, Jun 18, 2014 at 12:03 PM, Jeremy Jongsma jer...@barchart.com wrote: One option is to add new nodes, and do a node repair/cleanup on everything. That will at least reduce your per-node data size. On Wed, Jun 18, 2014 at 11:01 AM, Brian Tarbox tar...@cabotresearch.com wrote: I'm running on AWS m2.2xlarge instances using the ~800 gig ephemeral/attached disk for my data directory. My data size per node is nearing 400 gig. Sometimes during maintenance operations (repairs mostly I think) I run out of disk space as my understanding is that some of these operations require double the space of one's data. Since I can't change the size of attached storage for my instance type my question is can I somehow get these maintenance operations to use other volumes? Failing that, what are my options? Thanks. Brian Tarbox
Re: error creating keyspace in cqlsh
Is replication_factor your DC name?
Here is what I would using:
CREATE KEYSPACE IF NOT EXISTS animals
WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy',
'DC1' : 3 };
But in my case, I am using GossipPropertyFileSnitch and DC1 is
configured there, so Cassandra knows which nodes are in DC1 (data
center 1). Which snitch are you using?
http://www.datastax.com/documentation/cassandra/2.0/cassandra/architecture/architectureSnitchesAbout_c.html
Best regards,
Marcelo.
2014-06-18 23:54 GMT-03:00 Tim Dunphy bluethu...@gmail.com:
hey all,
I know that something pretty basic must be wrong here. But what is the
mistake I'm making in creating this keyspace?
cqlsh create keyspace animals with replication = { 'class':
'NetworkTopologyStrategy', 'replication_factor' : 3};
Bad Request: Error constructing replication strategy class
Thanks
Tim
--
GPG me!!
gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
Re: migration to a new model
Michael, I will try to test it up to tomorrow and I will let you know all the results. Thanks a lot! Best regards, Marcelo. 2014-06-04 22:28 GMT-03:00 Laing, Michael michael.la...@nytimes.com: BTW you might want to put a LIMIT clause on your SELECT for testing. -ml On Wed, Jun 4, 2014 at 6:04 PM, Laing, Michael michael.la...@nytimes.com wrote: Marcelo, Here is a link to the preview of the python fast copy program: https://gist.github.com/michaelplaing/37d89c8f5f09ae779e47 It will copy a table from one cluster to another with some transformation- they can be the same cluster. It has 3 main throttles to experiment with: 1. fetch_size: size of source pages in rows 2. worker_count: number of worker subprocesses 3. concurrency: number of async callback chains per worker subprocess It is easy to overrun Cassandra and the python driver, so I recommend starting with the defaults: fetch_size: 1000; worker_count: 2; concurrency: 10. Additionally there are switches to set 'policies' by source and destination: retry (downgrade consistency), dc_aware, and token_aware. retry is useful if you are getting timeouts. For the others YMMV. To use it you need to define the SELECT and UPDATE cql statements as well as the 'map_fields' method. The worker subprocesses divide up the token range among themselves and proceed quasi-independently. Each worker opens a connection to each cluster and the driver sets up connection pools to the nodes in the cluster. Anyway there are a lot of processes, threads, callbacks going at once so it is fun to watch. On my regional cluster of small nodes in AWS I got about 3000 rows per second transferred after things warmed up a bit - each row about 6kb. ml On Wed, Jun 4, 2014 at 11:49 AM, Laing, Michael michael.la...@nytimes.com wrote: OK Marcelo, I'll work on it today. -ml On Tue, Jun 3, 2014 at 8:24 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi Michael, For sure I would be interested in this program! I am new both to python and for cql. I started creating this copier, but was having problems with timeouts. Alex solved my problem here on the list, but I think I will still have a lot of trouble making the copy to work fine. I open sourced my version here: https://github.com/s1mbi0se/cql_record_processor Just in case it's useful for anything. However, I saw CQL has support for concurrency itself and having something made by someone who knows Python CQL Driver better would be very helpful. My two servers today are at OVH (ovh.com), we have servers at AWS but but several cases we prefer other hosts. Both servers have SDD and 64 Gb RAM, I could use the script as a benchmark for you if you want. Besides, we have some bigger clusters, I could run on the just to test the speed if this is going to help. Regards Marcelo. 2014-06-03 11:40 GMT-03:00 Laing, Michael michael.la...@nytimes.com: Hi Marcelo, I could create a fast copy program by repurposing some python apps that I am using for benchmarking the python driver - do you still need this? With high levels of concurrency and multiple subprocess workers, based on my current actual benchmarks, I think I can get well over 1,000 rows/second on my mac and significantly more in AWS. I'm using variable size rows averaging 5kb. This would be the initial version of a piece of the benchmark suite we will release as part of our nyt⨍aбrik project on 21 June for my Cassandra Day NYC talk re the python driver. ml On Mon, Jun 2, 2014 at 2:15 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi Jens, Thanks for trying to help. Indeed, I know I can't do it using just CQL. But what would you use to migrate data manually? I tried to create a python program using auto paging, but I am getting timeouts. I also tried Hive, but no success. I only have two nodes and less than 200Gb in this cluster, any simple way to extract the data quickly would be good enough for me. Best regards, Marcelo. 2014-06-02 15:08 GMT-03:00 Jens Rantil jens.ran...@tink.se: Hi Marcelo, Looks like you can't do this without migrating your data manually: https://stackoverflow.com/questions/18421668/alter-cassandra-column-family-primary-key-using-cassandra-cli-or-cql Cheers, Jens On Mon, Jun 2, 2014 at 7:48 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi, I have some cql CFs in a 2 node Cassandra 2.0.8 cluster. I realized I created my column family with the wrong partition. Instead of: CREATE TABLE IF NOT EXISTS entity_lookup ( name varchar, value varchar, entity_id uuid, PRIMARY KEY ((name, value), entity_id)) WITH caching=all; I used: CREATE TABLE IF NOT EXISTS entitylookup ( name varchar, value varchar, entity_id uuid, PRIMARY KEY (name, value, entity_id)) WITH caching=all; Now I need to migrate the data from the second CF to the first one. I am using Data Stax
Re: Cassandra 2.0 unbalanced ring with vnodes after adding new node
Actually, I have the same doubt. The same happens to me, but I guess it's because of lack of knowledge in Cassandra vnodes, somehow... I just added 3 nodes to my old 2 nodes cluster, now I have a 5 nodes cluster. As rows should be in a node calculated by HASH / number of nodes, adding a new node should move data from all other nodes to the new ones, right? Considering I have an enough number of different row keys. I noticed that: 1. Even reading data with read consistency = ALL, I get the wrong results while the repair is not complete. Should this happen? 2. I have run nodetool repair in each new node and nodetool cleanup in the 2 old nodes. There is some streaming happening, but it's really slow, considering my bandwith and use of SSDs. What should I do make the data stream from the old nodes to the new ones faster? And everytime I add new nodes to the cluster I will have to stop my processes that reads data from cassandra until the move is complete? Isn't there any other way? Best regards, Marcelo. 2014-06-04 13:52 GMT-03:00 Владимир Рудев vladimir.ru...@gmail.com: Hello to everyone! Please, can someone explain where we made a mistake? We have cluster with 4 nodes which uses vnodes(256 per node, default settings), snitch is default on every node: SimpleSnitch. These four nodes was from beginning of a cluster. In this cluster we have keyspace with this options: Keyspace: K: Replication Strategy: org.apache.cassandra.locator.SimpleStrategy Durable Writes: true Options: [replication_factor:3] All was normal and nodetool status K shows that each node owns 75% of all key range. All 4 nodes are located in same datacenter and have same first two bytes in IP address(others are different). Then we buy new server on different datacenter and add it to the cluster with same settings as in previous four nodes(difference only in listen_address), assuming that the effective own of each node for this keyspace will be 300/5=60% or near. But after 3-5 minutes after start nodetool status K show this: nodetool status K; Datacenter: datacenter1 === Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- AddressLoad Tokens Owns (effective) Host ID Rack UN N1 6,06 GB256 50.0% 62f295b3-0da6-4854-a53a-f03d6b424b03 rack1 UN N2 5,89 GB256 50.0% af4e4a23-2610-44dd-9061-09c7a6512a54 rack1 UN N3 6,02 GB256 50.0% 0f0e4e78-6fb2-479f-ad76-477006f76795 rack1 UN N4 5,8 GB 256 50.0% 670344c0-9856-48cf-9ec9-1a98f9a89460 rack1 UN N5 7,51 GB256 100.0% 82473d14-9e36-4ae7-86d2-a3e526efb53f rack1 N5 is newly added node nodetool repair -pr on N5 doesn't change anything nodetool describering K shows that new node N5 participate in EACH range. This is not we want at all. It looks like cassandra add new node to each range because it located in different datacenter, but all settings and output are exactly prevent this. Also interesting point is that while in all config files snitch is defined as SimpleSnitch the output of the command nodetool describecluster is: Cluster Information: Name: Some Cluster Name Snitch: org.apache.cassandra.locator.*DynamicEndpointSnitch* Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Schema versions: 26b8fa37-e666-31ed-aa3b-85be75f2aa1a: [N1, N2, N3, N4, N5] We use Cassandra 2.0.6 Questions we have at this moment: 1. How to rebalance ring so all nodes will own 60% of range? 1a. Removing node from cluster and adding it again is a solution? 2. Where we possibly make a mistake when adding new node? 3. If we add new 6th node to ring it will take 50% from N5 or some portion from each node? Thanks in advance! -- С уважением, Владимир Рудев (With regards, Vladimir Rudev) vladimir.ru...@gmail.com
Re: python cql driver - cassandra.ReadTimeout - “Operation timed out - received only 1 responses.”
Indeed Alex, the problem was in the rpc timeouts on the server...
Thanks a lot, it's simple but I was losing time thinking my client config
was wrong!
[]s
2014-06-02 18:15 GMT-03:00 Alex Popescu al...@datastax.com:
If I'm reading this correctly, what you are seeing is the read_timeout on
Cassandra side and not the client side timeout. Even if you set the client
side timeouts, the C* read write timeouts are still respected on that
side.
On Mon, Jun 2, 2014 at 10:55 AM, Marcelo Elias Del Valle
marc...@s1mbi0se.com.br wrote:
I am using Cassandra 2.0 with python CQL.
I have created a column family as follows:
CREATE KEYSPACE IF NOT EXISTS Identification
WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy',
'DC1' : 1 };
USE Identification;
CREATE TABLE IF NOT EXISTS entitylookup (
name varchar,
value varchar,
entity_id uuid,
PRIMARY KEY ((name, value), entity_id))
WITH
caching=all;
I then try to count the number of records in this CF as follows:
#!/usr/bin/env pythonimport argparseimport sysimport tracebackfrom cassandra
import ConsistencyLevelfrom cassandra.cluster import Clusterfrom
cassandra.query import SimpleStatement
def count(host, cf):
keyspace = identification
cluster = Cluster([host], port=9042,
control_connection_timeout=6)
session = cluster.connect(keyspace)
session.default_timeout=6
st = SimpleStatement(SELECT count(*) FROM %s % cf,
consistency_level=ConsistencyLevel.ALL)
for row in session.execute(st, timeout=6):
print count for cf %s = %s % (cf, str(row))
dump_pool.close()
dump_pool.join()
if __name__ == __main__:
parser = argparse.ArgumentParser()
parser.add_argument(-cf, --column-family, default=entitylookup,
help=Column Family to query)
parser.add_argument(-H, --host, default=localhost, help=Cassandra
host)
args = parser.parse_args()
count(args.host, args.column_family)
print fim
The count is not that useful to me, it's just a test with an operation
that takes long to complete.
Although I have defined timeout as 6 seconds, after less than 30
seconds I get the following error:
./count_entity_lookup.py -H localhost -cf entitylookup
Traceback (most recent call last):
File ./count_entity_lookup.py, line 27, in module
count(args.host, args.column_family)
File ./count_entity_lookup.py, line 16, in count
for row in session.execute(st, timeout=None):
File
/home/mvalle/pyenv0/local/lib/python2.7/site-packages/cassandra/cluster.py,
line 1026, in execute
result = future.result(timeout)
File
/home/mvalle/pyenv0/local/lib/python2.7/site-packages/cassandra/cluster.py,
line 2300, in result
raise self._final_exception
cassandra.ReadTimeout: code=1200 [Timeout during read request]
message=Operation timed out - received only 1 responses.
info={'received_responses': 1, 'data_retrieved': True, 'required_responses':
2, 'consistency': 5}
It seems the answer was found in just a replica, but this really doesn't
make sense to me. Should't cassandra be able to query it anyway?
These tests are running in a two node cluster, with RF = 2, write and
read consistency = ALL (but same results using QUORUM).
Thanks in advance.
Best regards,
Marcelo.
--
:- a)
Alex Popescu
Sen. Product Manager @ DataStax
@al3xandru
migration to a new model
Hi, I have some cql CFs in a 2 node Cassandra 2.0.8 cluster. I realized I created my column family with the wrong partition. Instead of: CREATE TABLE IF NOT EXISTS entity_lookup ( name varchar, value varchar, entity_id uuid, PRIMARY KEY ((name, value), entity_id)) WITH caching=all; I used: CREATE TABLE IF NOT EXISTS entitylookup ( name varchar, value varchar, entity_id uuid, PRIMARY KEY (name, value, entity_id)) WITH caching=all; Now I need to migrate the data from the second CF to the first one. I am using Data Stax Community Edition. What would be the best way to convert data from one CF to the other? Best regards, Marcelo.
Problems using Hive + Cassandra community
Hi,
Has anyone used HIVE + Cassandra Community successfully? I am having
problems mapping the keyspace, but I started wondering if only DSE has
support for it.
I am trying to use HIVE 0.13 to access cassandra 2.0.8 column families
created with CQL3.
Here is how I created my column families:
CREATE KEYSPACE IF NOT EXISTS Identification
WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy',
'DC1' : 2 };
USE Identification;
CREATE TABLE IF NOT EXISTS entitylookup (
name varchar,
value varchar,
entity_id uuid,
PRIMARY KEY ((name, value), entity_id))
WITH
caching=all
;
I followed the instructions from the README of this project:
https://github.com/tuplejump/cash/tree/master/cassandra-handler
I generated hive-cassandra-1.2.6.jar, copied it and
cassandra-all-1.2.6.jar, cassandra-thrift-1.2.6.jar to hive lib folder.
Then I started hive and tried the following:
CREATE EXTERNAL TABLE identification.entitylookup(name string, value
string, entity_id binary)
STORED BY 'org.apache.hadoop.hive.cassandra.cql.CqlStorageHandler'
WITH SERDEPROPERTIES(cql.primarykey = name, value,
cassandra.host = localhost, cassandra.port = 9160)
TBLPROPERTIES (cassandra.ks.name = identification,
cassandra.ks.stratOptions='DC1':2,
cassandra.ks.strategy=NetworkTopologyStrategy);
Here is the output:
hive mvalle@mvalle:~/hadoop$ hive
14/05/30 12:02:02 INFO Configuration.deprecation: mapred.reduce.tasks
is deprecated. Instead, use mapreduce.job.reduces
14/05/30 12:02:02 INFO Configuration.deprecation:
mapred.min.split.size is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize
14/05/30 12:02:02 INFO Configuration.deprecation:
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
mapreduce.reduce.speculative
14/05/30 12:02:02 INFO Configuration.deprecation:
mapred.min.split.size.per.node is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.node
14/05/30 12:02:02 INFO Configuration.deprecation:
mapred.input.dir.recursive is deprecated. Instead, use
mapreduce.input.fileinputformat.input.dir.recursive
14/05/30 12:02:02 INFO Configuration.deprecation:
mapred.min.split.size.per.rack is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.rack
14/05/30 12:02:02 INFO Configuration.deprecation:
mapred.max.split.size is deprecated. Instead, use
mapreduce.input.fileinputformat.split.maxsize
14/05/30 12:02:02 INFO Configuration.deprecation:
mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use
mapreduce.job.committer.setup.cleanup.needed
Logging initialized using configuration in
jar:file:/home/mvalle/hadoop/apache-hive-0.13.0-bin/lib/hive-common-0.13.0.jar!/hive-log4j.properties
OpenJDK 64-Bit Server VM warning: You have loaded library
/home/mvalle/hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which
might have disabled stack guard. The VM will try to fix the stack
guard now.
It's highly recommended that you fix the library with 'execstack -c
libfile', or link it with '-z noexecstack'.
hive CREATE EXTERNAL TABLE identification.entitylookup(name string,
value string, entity_id binary)
STORED BY
'org.apache.hadoop.hive.cassandra.cql.CqlStorageHandler' WITH
SERDEPROPERTIES(cql.primarykey = name, value, cassandra.host =
ident.s1mbi0se.com, cassandra.port = 9160)
TBLPROPERTIES (cassandra.ks.name = identification,
cassandra.ks.stratOptions='DC1':2,
cassandra.ks.strategy=NetworkTopologyStrategy);
FAILED: SemanticException [Error 10072]: Database does not exist: identification
Question: how do I do to get more information about what is going wrong? I
tried the same hive command using Identification (capital I), but same
result. Is it possible to access CQL3 column families in cassandra
community? It seems the keyspace has not been mapped, but I don't see how
to map then. In DSE, they are automatically mapped...
Best regards,
Marcelo.
Re: migration to a new model
Hi Jens, Thanks for trying to help. Indeed, I know I can't do it using just CQL. But what would you use to migrate data manually? I tried to create a python program using auto paging, but I am getting timeouts. I also tried Hive, but no success. I only have two nodes and less than 200Gb in this cluster, any simple way to extract the data quickly would be good enough for me. Best regards, Marcelo. 2014-06-02 15:08 GMT-03:00 Jens Rantil jens.ran...@tink.se: Hi Marcelo, Looks like you can't do this without migrating your data manually: https://stackoverflow.com/questions/18421668/alter-cassandra-column-family-primary-key-using-cassandra-cli-or-cql Cheers, Jens On Mon, Jun 2, 2014 at 7:48 PM, Marcelo Elias Del Valle marc...@s1mbi0se.com.br wrote: Hi, I have some cql CFs in a 2 node Cassandra 2.0.8 cluster. I realized I created my column family with the wrong partition. Instead of: CREATE TABLE IF NOT EXISTS entity_lookup ( name varchar, value varchar, entity_id uuid, PRIMARY KEY ((name, value), entity_id)) WITH caching=all; I used: CREATE TABLE IF NOT EXISTS entitylookup ( name varchar, value varchar, entity_id uuid, PRIMARY KEY (name, value, entity_id)) WITH caching=all; Now I need to migrate the data from the second CF to the first one. I am using Data Stax Community Edition. What would be the best way to convert data from one CF to the other? Best regards, Marcelo.
Astyanax TokenAware connection pool
Hello everyone,
I was using astyanax connection pool defined as this:
ipSeeds = LOAD_BALANCER_HOST:9160;
conPool.setSeeds(ipSeeds)
.setDiscoveryType(NodeDiscoveryType.TOKEN_AWARE)
.setConnectionPoolType(ConnectionPoolType.TOKEN_AWARE);
However, my cluster have 4 nodes and I have 8 client machines connecting on
it. LOAD_BALANCER_HOST forwards requests to one of my four nodes.
On a client node, I have:
$netstat -an | grep 9160 | awk '{print $5}' | sort |uniq -c
235 node1:9160
680 node2:9160
4 node3:9160
4 node4:9160
So although the ConnectionPoolType is TOKEN_AWARE, my client seems to be
connecting mainly to node2, sometimes to node1, but almost never to nodes 3
and 4.
Question is:
Why is this happening? Shouldn't a token aware connection pool query the
ring for the node list and connect to all the active nodes using round
robin algorithm?
Best regards,
Marcelo Valle.
PropertiesFileSnitch
Hello everyone, I have a cassandra cluster running at amazon. I am trying to add a new datacenter for this cluster now, outside AWS. I know I could use multiregion, but I would like to be vendor free in terms of cloud. Reading the article http://www.datastax.com/docs/datastax_enterprise3.2/deploy/multi_dc_install, it seems I will need to start using PropertiesFileSnitch instead of Ec2Snitch to do what I want. So here it comes my question: If I set all the seeds on my property file, what will happen if I need to add mores machines and/or seeds to the cluster? Will I need to change the property files on all the nodes of my cluster, or just on the new node? Best regards, Marcelo Valle.
cassandra backup
Hello everyone, I am trying to create backups of my data on AWS. My goal is to store the backups on S3 or glacier, as it's cheap to store this kind of data. So, if I have a cluster with N nodes, I would like to copy data from all N nodes to S3 and be able to restore later. I know Priam does that (we were using it), but I am using the latest cassandra version and we plan to use DSE some time, I am not sure Priam fits this case. I took a look at the docs: http://www.datastax.com/documentation/cassandra/2.0/webhelp/index.html#cassandra/operations/../../cassandra/operations/ops_backup_takes_snapshot_t.html And I am trying to understand if it's really needed to take a snapshot to create my backup. Suppose I do a flush and copy the sstables from each node, 1 by one, to s3. Not all at the same time, but one by one. When I try to restore my backup, data from node 1 will be older than data from node 2. Will this cause problems? AFAIK, if I am using a replication factor of 2, for instance, and Cassandra sees data from node X only, it will automatically copy it to other nodes, right? Is there any chance of cassandra nodes become corrupt somehow if I do my backups this way? Best regards, Marcelo Valle.
io bound model
Hi everyone, I currently have a column family InputCf in production which has 1 data input per row. Everytime I receive new data from web, I insert a row in this CF. Besides that, I have another CF InputCfIndex in which the year/month/day is my row id (MMdd) and I insert the id of InputCf on each column, with no value. At the end of the day, I check all the row inserted that day on InputCf and process it. Reading the id from InputCfIndex is fast, but reading from InputCf uses a lot of IO, because I cannot know in which machine on the cluster the data will be. When I query Cassandra for all the rows inserted today in InputCf, it takes me 100% of Network IO utilization and almost no cpu or memory consumption. I was wondering if there is a way of quering a lot of messages at a time, but multi_get orchestration happens in the client and as data is distributed along the cluster, I am not sure it would help. So here is my question: any ideas of how to change my model to be able to query several inputs at a time, consuming less network IO? I am guessing there must be a way of optimizing it... Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
node dead after restart
Hello, I am having a problem with a node in a test environment I have at amazon. I am using cassandra 1.2.3 in Amazon EC2. Here is my nodetool ring output: $ nodetool ring Note: Ownership information does not include topology; for complete information, specify a keyspace Datacenter: us-east == Address RackStatus State LoadOwns Token 113427455640312821154458202479064646084 10.0.0.76 1b Up Normal 31.34 MB33.33% 1808575600 10.0.0.1461b Up Normal 34.24 MB33.33% 56713727820156410577229101240436610842 10.0.0.111 1b Down Normal 21.19 MB33.33% 113427455640312821154458202479064646084 I logged in 10.0.0.111 machine and restarted cassandra, while looking at the log. Gossip protocol is still up, but the node starts and goes down just after it. Here is what I see in the logs: sudo tail /var/log/cassandra/output.log INFO 12:16:23,084 Node /10.0.0.111 has restarted, now UP INFO 12:16:23,095 InetAddress /10.0.0.111 is now UP INFO 12:16:23,097 Node /10.0.0.111 state jump to normal INFO 12:16:23,105 Not starting native transport as requested. Use JMX (StorageService-startNativeTransport()) to start it INFO 12:16:23,108 Binding thrift service to ip-10-0-0-146.ec2.internal/ 10.0.0.146:9160 INFO 12:16:23,137 Using TFramedTransport with a max frame size of 15728640 bytes. INFO 12:16:23,143 Using synchronous/threadpool thrift server on ip-10-0-0-146.ec2.internal : 9160 INFO 12:16:23,143 Listening for thrift clients... INFO 12:16:30,063 Saved local counter id: 76c1a930-a866-11e2-a3bd-831b111cd74c INFO 12:16:32,860 InetAddress /10.0.0.111 is now dead. I am having no clue of what is wrong. Any hint of what could I do to look for the problem? Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: node dead after restart
Ufff I almost won the Charles Darwin prize now. Thanks! []s 2013/8/22 Hiller, Dean dean.hil...@nrel.gov Isn't this the log file from 10.0.0.146??? And this 10.0.0.146 sees that 10.0.0.111 is up, then sees it dead and in the log we can see it bind with this line INFO 12:16:23,108 Binding thrift service to ip-10-0-0-146.ec2.internal/ 10.0.0.146:9160http://10.0.0.146:9160 What is the log file look like on 10.0.0.111? Thanks, Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Thursday, August 22, 2013 9:19 AM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: node dead after restart Hello, I am having a problem with a node in a test environment I have at amazon. I am using cassandra 1.2.3 in Amazon EC2. Here is my nodetool ring output: $ nodetool ring Note: Ownership information does not include topology; for complete information, specify a keyspace Datacenter: us-east == Address RackStatus State LoadOwns Token 113427455640312821154458202479064646084 10.0.0.76 1b Up Normal 31.34 MB33.33% 1808575600 10.0.0.1461b Up Normal 34.24 MB33.33% 56713727820156410577229101240436610842 10.0.0.111 1b Down Normal 21.19 MB33.33% 113427455640312821154458202479064646084 I logged in 10.0.0.111 machine and restarted cassandra, while looking at the log. Gossip protocol is still up, but the node starts and goes down just after it. Here is what I see in the logs: sudo tail /var/log/cassandra/output.log INFO 12:16:23,084 Node /10.0.0.111http://10.0.0.111 has restarted, now UP INFO 12:16:23,095 InetAddress /10.0.0.111http://10.0.0.111 is now UP INFO 12:16:23,097 Node /10.0.0.111http://10.0.0.111 state jump to normal INFO 12:16:23,105 Not starting native transport as requested. Use JMX (StorageService-startNativeTransport()) to start it INFO 12:16:23,108 Binding thrift service to ip-10-0-0-146.ec2.internal/ 10.0.0.146:9160http://10.0.0.146:9160 INFO 12:16:23,137 Using TFramedTransport with a max frame size of 15728640 bytes. INFO 12:16:23,143 Using synchronous/threadpool thrift server on ip-10-0-0-146.ec2.internal : 9160 INFO 12:16:23,143 Listening for thrift clients... INFO 12:16:30,063 Saved local counter id: 76c1a930-a866-11e2-a3bd-831b111cd74c INFO 12:16:32,860 InetAddress /10.0.0.111http://10.0.0.111 is now dead. I am having no clue of what is wrong. Any hint of what could I do to look for the problem? Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: cassandra 1.2.6 - Start key's token sorts after end token
We managed to make it work with RandomPartitioner... Guess we will rely on it for now... 2013/7/23 Hiller, Dean dean.hil...@nrel.gov Oh, and in the past 0.20.x has been pretty stable by the wayŠ..they finally switched their numbering scheme thank god. Dean On 7/23/13 2:13 PM, Hiller, Dean dean.hil...@nrel.gov wrote: Perhaps try 0.20.2 as 1. The maven pom files have cassandra depending on 0.20.2 2. The 0.20.2 default was murmur and we had to change it to random partitioner or it wouldn't work for us Ie. I suspect they will change the pom file to a more recent version of hadoop at some point but I wonder if test suites suck in 0.20.2 because the pom file points to that versionŠ.depends on if they actually have tests for map/reduce which is probably a bit hard. Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto:mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Tuesday, July 23, 2013 1:54 PM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: cassandra 1.2.6 - Start key's token sorts after end token Dean, I am using hadoop 1.0.3. Indeed, using Cassandra 1.2.3 with Random partitioner, it worked. However, it's the only reason for me to use randompartitioner, I really would like to move forward. Besides, I tried to use Cassandra 1.2.6 with RandomPartitioner and I got problems when inserting data, even stopping Cassandra, cleaning my entire data folder and then starting it again. I am also really curious to know if there is anyone else having these problems or if it is just me... Best regards, Marcelo. 2013/7/23 Hiller, Dean dean.hil...@nrel.govmailto:dean.hil...@nrel.gov Out of curiosity, what version of hadoop are you using with cassandra? I think we are trying 0.20.2 if I remember(I have to ask my guy working on it to be sure). I do remember him saying the cassandra maven dependency was odd in that it is in the older version and not a newer hadoop version. We are using RandomPartitioner though right now which I have personally used in the past with success. We are in the process of map/reducing to a cassandra with MurmurPartitioner (our real reason to map/reduce is some refactorings in our model though and we just thought we would switch to murmur). Has anyone else used map/reduce with murmur partitioner? Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto:mvall...@gmail.commailto:mvall...@gmail.com m ailto:mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.orgmailto: user@c assandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.orgmailto: user@c assandra.apache.orgmailto:user@cassandra.apache.org Date: Monday, July 22, 2013 4:04 PM To: user@cassandra.apache.orgmailto:user@cassandra.apache.orgmailto: user@c assandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.orgmailto: user@c assandra.apache.orgmailto:user@cassandra.apache.org Subject: cassandra 1.2.6 - Start key's token sorts after end token Hello, I am trying to figure what might be cause this error. I am using Cassandra 1.2.6 (tried with 1.2.3 as well) and I am trying to read data from cassandra on hadoop using column family input format. I also got the same error using pure astyanax on a test. I am using Murmur3Partitioner and I created the keyspace using Cassandra 1.2.6, there is nothing from prior versions. I created the keyspace with SimpleStrategy and replication factor 1. Here is the exception I am getting: 2013-07-22 21:53:05,824 WARN org.apache.hadoop.mapred.Child (main): Error running child java.lang.RuntimeException: InvalidRequestException(why:Start key's token sorts after end token) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.maybe Init(ColumnFamilyRecordReader.java:453) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.compu teNext(ColumnFamilyRecordReader.java:459) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.compu teNext(ColumnFamilyRecordReader.java:406) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterat or.java:143) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:1 38) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.getProgress(ColumnFam ilyRecordReader.java:103) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.getProgress(MapTa sk.java:522) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapT ask.java:547) at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143
Re: cassandra 1.2.6 - Start key's token sorts after end token
Dean, I am using hadoop 1.0.3. Indeed, using Cassandra 1.2.3 with Random partitioner, it worked. However, it's the only reason for me to use randompartitioner, I really would like to move forward. Besides, I tried to use Cassandra 1.2.6 with RandomPartitioner and I got problems when inserting data, even stopping Cassandra, cleaning my entire data folder and then starting it again. I am also really curious to know if there is anyone else having these problems or if it is just me... Best regards, Marcelo. 2013/7/23 Hiller, Dean dean.hil...@nrel.gov Out of curiosity, what version of hadoop are you using with cassandra? I think we are trying 0.20.2 if I remember(I have to ask my guy working on it to be sure). I do remember him saying the cassandra maven dependency was odd in that it is in the older version and not a newer hadoop version. We are using RandomPartitioner though right now which I have personally used in the past with success. We are in the process of map/reducing to a cassandra with MurmurPartitioner (our real reason to map/reduce is some refactorings in our model though and we just thought we would switch to murmur). Has anyone else used map/reduce with murmur partitioner? Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Monday, July 22, 2013 4:04 PM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: cassandra 1.2.6 - Start key's token sorts after end token Hello, I am trying to figure what might be cause this error. I am using Cassandra 1.2.6 (tried with 1.2.3 as well) and I am trying to read data from cassandra on hadoop using column family input format. I also got the same error using pure astyanax on a test. I am using Murmur3Partitioner and I created the keyspace using Cassandra 1.2.6, there is nothing from prior versions. I created the keyspace with SimpleStrategy and replication factor 1. Here is the exception I am getting: 2013-07-22 21:53:05,824 WARN org.apache.hadoop.mapred.Child (main): Error running child java.lang.RuntimeException: InvalidRequestException(why:Start key's token sorts after end token) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.maybeInit(ColumnFamilyRecordReader.java:453) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.computeNext(ColumnFamilyRecordReader.java:459) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.computeNext(ColumnFamilyRecordReader.java:406) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.getProgress(ColumnFamilyRecordReader.java:103) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.getProgress(MapTask.java:522) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:547) at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: InvalidRequestException(why:Start key's token sorts after end token) at org.apache.cassandra.thrift.Cassandra$get_paged_slice_result.read(Cassandra.java:14168) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_get_paged_slice(Cassandra.java:769) at org.apache.cassandra.thrift.Cassandra$Client.get_paged_slice(Cassandra.java:753) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.maybeInit(ColumnFamilyRecordReader.java:438) ... 16 more 2013-07-22 21:53:05,828 INFO org.apache.hadoop.mapred.Task (main): Runnning cleanup for the task Any hint? Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
cassandra 1.2.6 - Start key's token sorts after end token
Hello, I am trying to figure what might be cause this error. I am using Cassandra 1.2.6 (tried with 1.2.3 as well) and I am trying to read data from cassandra on hadoop using column family input format. I also got the same error using pure astyanax on a test. I am using Murmur3Partitioner and I created the keyspace using Cassandra 1.2.6, there is nothing from prior versions. I created the keyspace with SimpleStrategy and replication factor 1. Here is the exception I am getting: 2013-07-22 21:53:05,824 WARN org.apache.hadoop.mapred.Child (main): Error running child java.lang.RuntimeException: InvalidRequestException(why:Start key's token sorts after end token) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.maybeInit(ColumnFamilyRecordReader.java:453) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.computeNext(ColumnFamilyRecordReader.java:459) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.computeNext(ColumnFamilyRecordReader.java:406) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.getProgress(ColumnFamilyRecordReader.java:103) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.getProgress(MapTask.java:522) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:547) at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: InvalidRequestException(why:Start key's token sorts after end token) at org.apache.cassandra.thrift.Cassandra$get_paged_slice_result.read(Cassandra.java:14168) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_get_paged_slice(Cassandra.java:769) at org.apache.cassandra.thrift.Cassandra$Client.get_paged_slice(Cassandra.java:753) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.maybeInit(ColumnFamilyRecordReader.java:438) ... 16 more 2013-07-22 21:53:05,828 INFO org.apache.hadoop.mapred.Task (main): Runnning cleanup for the task Any hint? Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
net flix priam - use outside of amazon aws
Hello, I Currently use Netflix Priam to create backups of sstables to Amazon S3. I like how it works, as it can create continuous backups as well as snapshots, but I personally don`t like to be tied to a vendor, in this case, Amazon. Does anybody know if there is some similar tool to do backups for Cassandra and could run in any data center? Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: no other nodes seen on priam cluster
Thanks a lot Ben, actually I managed to make it work erasing the SimpleDB Priam uses to keeps instances... I would pulled the last commit from the repo, not sure if it helped or not. But you message made me curious about something... How do you do to add more Cassandra nodes on the fly? Just update the autoscale properties? I saw instaclustr.com changes the instance type as the number of nodes increase (not sure why the price also becomes higher per instance in this case), I am guessing priam use the data backed up to S3 to restore a node data in another instance, right? []s 2013/2/28 Ben Bromhead b...@relational.io Off the top of my head I would check to make sure the Autoscaling Group you created is restricted to a single Availability Zone, also Priam sets the number of EC2 instances it expects based on the maximum instance count you set on your scaling group (it did this last time i checked a few months ago, it's behaviour may have changed). So I would make your desired, min and max instances for your scaling group are all the same, make sure your ASG is restricted to a single availability zone (e.g. us-east-1b) and then (if you are able to and there is no data in your cluster) delete all the SimpleDB entries Priam has created and then also possibly clear out the cassandra data directory. Other than that I see you've raised it as an issue on the Priam project page , so see what they say ;) Cheers Ben On Thu, Feb 28, 2013 at 3:40 AM, Marcelo Elias Del Valle mvall...@gmail.com wrote: One additional important info, I checked here and the seeds seems really different on each node. The command echo `curl http://127.0.0.1:8080/Priam/REST/v1/cassconfig/get_seeds`http://127.0.0.1:8080/Priam/REST/v1/cassconfig/get_seeds returns ip2 on first node and ip1,ip1 on second node. Any idea why? It's probably what is causing cassandra to die, right? 2013/2/27 Marcelo Elias Del Valle mvall...@gmail.com Hello Ben, Thanks for the willingness to help, 2013/2/27 Ben Bromhead b...@instaclustr.com Have your added the priam java agent to cassandras JVM argurments (e.g. -javaagent:$CASS_HOME/lib/priam-cass-extensions-1.1.15.jar) and does the web container running priam have permissions to write to the cassandra config directory? Also what do the priam logs say? I put the priam log of the first node bellow. Yes, I have added priam-cass-extensions to java args and Priam IS actually writting to cassandra dir. If you want to get up and running quickly with cassandra, AWS and priam quickly check out www.instaclustr.comhttp://www.instaclustr.com/?cid=cass-listyou. We deploy Cassandra under your AWS account and you have full root access to the nodes if you want to explore and play around + there is a free tier which is great for experimenting and trying Cassandra out. That sounded really great. I am not sure if it would apply to our case (will consider it though), but some partners would have a great benefit from it, for sure! I will send your link to them. What priam says: 2013-02-27 14:14:58.0614 INFO pool-2-thread-1 com.netflix.priam.utils.SystemUtils Calling URL API: http://169.254.169.254/latest/meta-data/public-hostname returns: ec2-174-129-59-107.compute-1.amazon aws.com 2013-02-27 14:14:58.0615 INFO pool-2-thread-1 com.netflix.priam.utils.SystemUtils Calling URL API: http://169.254.169.254/latest/meta-data/public-ipv4 returns: 174.129.59.107 2013-02-27 14:14:58.0618 INFO pool-2-thread-1 com.netflix.priam.utils.SystemUtils Calling URL API: http://169.254.169.254/latest/meta-data/instance-id returns: i-88b32bfb 2013-02-27 14:14:58.0618 INFO pool-2-thread-1 com.netflix.priam.utils.SystemUtils Calling URL API: http://169.254.169.254/latest/meta-data/instance-type returns: c1.medium 2013-02-27 14:14:59.0614 INFO pool-2-thread-1 com.netflix.priam.defaultimpl.PriamConfiguration REGION set to us-east-1, ASG Name set to dmp_cluster-useast1b 2013-02-27 14:14:59.0746 INFO pool-2-thread-1 com.netflix.priam.defaultimpl.PriamConfiguration appid used to fetch properties is: dmp_cluster 2013-02-27 14:14:59.0843 INFO pool-2-thread-1 org.quartz.simpl.SimpleThreadPool Job execution threads will use class loader of thread: pool-2-thread-1 2013-02-27 14:14:59.0861 INFO pool-2-thread-1 org.quartz.core.SchedulerSignalerImpl Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl 2013-02-27 14:14:59.0862 INFO pool-2-thread-1 org.quartz.core.QuartzScheduler Quartz Scheduler v.1.7.3 created. 2013-02-27 14:14:59.0864 INFO pool-2-thread-1 org.quartz.simpl.RAMJobStore RAMJobStore initialized. 2013-02-27 14:14:59.0864 INFO pool-2-thread-1 org.quartz.impl.StdSchedulerFactory Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.propertie s' 2013-02-27 14:14:59.0864 INFO pool-2-thread-1 org.quartz.impl.StdSchedulerFactory Quartz scheduler version: 1.7.3 2013-02-27 14
If we Open Source our platform, would it be interesting to you?
Hello All, I’m sending this email because I think it may be interesting for Cassandra users, as this project have a strong usage of Cassandra platform. We are strongly considering opening the source of our DMP (Data Management Platform), if it proves to be technically interesting to other developers / companies. More details: http://www.s1mbi0se.com/s1mbi0se_DMP.html All comments, questions and critics happening at HN: http://news.ycombinator.com/item?id=5251780 Please, feel free to send questions, comments and critics... We will try to reply them all. Regards, Marcelo
Re: Cassandra at Amazon AWS
Everyone, thanks a lot for the answer, they helped me a lot. 2013/1/17 Andrey Ilinykh ailin...@gmail.com I'd recommend Priam. http://techblog.netflix.com/2012/02/announcing-priam.html Andrey On Thu, Jan 17, 2013 at 5:44 AM, Adam Venturella aventure...@gmail.comwrote: Jared, how do you guys handle data backups for your ephemeral based cluster? I'm trying to move to ephemeral drives myself, and that was my last sticking point; asking how others in the community deal with backup in case the VM explodes. On Wed, Jan 16, 2013 at 1:21 PM, Jared Biel jared.b...@bolderthinking.com wrote: We're currently using Cassandra on EC2 at very low scale (a 2 node cluster on m1.large instances in two regions.) I don't believe that EBS is recommended for performance reasons. Also, it's proven to be very unreliable in the past (most of the big/notable AWS outages were due to EBS issues.) We've moved 99% of our instances off of EBS. As other have said, if you require more space in the future it's easy to add more nodes to the cluster. I've found this page (http://www.ec2instances.info/) very useful in determining the amount of space each instance type has. Note that by default only one ephemeral drive is attached and you must specify all ephemeral drives that you want to use at launch time. Also, you can create a RAID 0 of all local disks to provide maximum speed and space. On 16 January 2013 20:42, Marcelo Elias Del Valle mvall...@gmail.com wrote: Hello, I am currently using hadoop + cassandra at amazon AWS. Cassandra runs on EC2 and my hadoop process runs at EMR. For cassandra storage, I am using local EC2 EBS disks. My system is running fine for my tests, but to me it's not a good setup for production. I need my system to perform well for specially for writes on cassandra, but the amount of data could grow really big, taking several Tb of total storage. My first guess was using S3 as a storage and I saw this can be done by using Cloudian package, but I wouldn't like to become dependent on a pre-package solution and I found it's kind of expensive for more than 100Tb: http://www.cloudian.com/pricing.html I saw some discussion at internet about using EBS or ephemeral disks for storage at Amazon too. My question is: does someone on this list have the same problem as me? What are you using as solution to Cassandra's storage when running it at Amazon AWS? Any thoughts would be highly appreciatted. Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: AWS EMR - Cassandra
the Pig integration of Cassandra. Versions: Hadoop 1.0.3, Pig 0.10, Cassandra 1.1.7. I'm 99% sure I have classpaths working (because I didn't at first, and now EMR can find and instantiate CassandraStorage on master and slaves). What isn't working are the system variables. In my DIY cluster, all I needed to do was: --- export PIG_INITIAL_ADDRESS=XXX export PIG_RPC_PORT=9160 export PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner -- And the task trackers somehow magically picked up the values (I never questioned how/why). But, in EMR, they do not. Instead, I get an error from CassandraStorage that the initial address isn't set (on the slave, the master is ok). My DIY cluster used CDH3, which was hadoop 0.20.something. So, maybe the problem is a different version of hadoop? Looking at the CassandraStorage class, I realize I have no idea how it used to work, since it only seems to look at System variables. Those variables are set on the Job.getConfiguration object. I don't know how that part of hadoop works though... do variables that get set on Job on the master get propagated to the task threads? I do know that on my DIY cluster, I do NOT set those system variables on the slaves... Thanks! will -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Cassandra at Amazon AWS
Hello, I am currently using hadoop + cassandra at amazon AWS. Cassandra runs on EC2 and my hadoop process runs at EMR. For cassandra storage, I am using local EC2 EBS disks. My system is running fine for my tests, but to me it's not a good setup for production. I need my system to perform well for specially for writes on cassandra, but the amount of data could grow really big, taking several Tb of total storage. My first guess was using S3 as a storage and I saw this can be done by using Cloudian package, but I wouldn't like to become dependent on a pre-package solution and I found it's kind of expensive for more than 100Tb: http://www.cloudian.com/pricing.html I saw some discussion at internet about using EBS or ephemeral disks for storage at Amazon too. My question is: does someone on this list have the same problem as me? What are you using as solution to Cassandra's storage when running it at Amazon AWS? Any thoughts would be highly appreciatted. Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: AWS EMR - Cassandra
That's good info! Thanks!
2013/1/16 William Oberman ober...@civicscience.com
DataStax recommended (forget the reference) to use the ephemeral disks in
RAID0, which is what I've been running for well over a year now in
production.
In terms of how I'm doing Cassandra/AWS/Hadoop, I started by doing the
split data center thing (one DC for low latency queries, one DC for
hadoop). But, that's a lot of system management. And compute is the most
expensive part of AWS, and you need a LOT of compute to run this setup. I
tried doing Cassandra EC2 cluster - snapshot - clone cluster with hadoop
overlay - ETL to S3 using hadoop - EMR for real work. But that's kind of
a pain too (and the ETL to S3 wasn't very fast).
Now I'm going after the SStables directly(*), which sounds like how
Netflix does it. You can do incremental updates, if you're careful.
(*) Cassandra EC2 - backup to local EBS - remap EBS to another box -
sstable2json over new sstables - S3 (splitting into ~100MB parts), then
use EMR to consume the JSON part files.
will
On Wed, Jan 16, 2013 at 3:30 PM, Marcelo Elias Del Valle
mvall...@gmail.com wrote:
William,
I just saw your message today. I am using Cassandra + Amazon EMR
(hadoop 1.0.3) but I am not using PIG as you are. I set my configuration
vars in Java, as I have a custom jar file and I am using
ColumnFamilyInputFormat.
However, if I understood well your problem, the only thing you have
to do is to set environment vars when running cluster tasks, right? Take a
look a this link:
http://sujee.net/tech/articles/hadoop/amazon-emr-beyond-basics/
As it shows, you can run EMR setting some command line arguments that
specify a script to be executed before the job starts, in each machine in
the cluster. This way, you would be able to correctly set the vars you need.
Out of curiosity, could you share what are you using for cassandra
storage? I am currently using EC2 local disks, but I am looking for an
alternative.
Best regards,
Marcelo.
2013/1/4 William Oberman ober...@civicscience.com
So I've made it work, but I don't get it yet.
I have no idea why my DIY server works when I set the environment
variables on the machine that kicks off pig (master), and in EMR it
doesn't. I recompiled ConfigHelper and CassandraStorage with tons of
debugging, and in EMR I can see the hadoop Configuration object get the
proper values on the master node, and I can see it does NOT propagate to
the task threads.
The other part that was driving me nuts could be made more user
friendly. The issue is this: I started to try to set
cassandra.thrift.address, cassandra.thrift.port,
cassandra.partitioner.class in mapred-site.xml, and it didn't work. After
even more painful debugging, I noticed that the only time Cassandra sets
the input/output versions of those settings (and these input/output
specific versions are the only versions really used!) is when Cassandra
maps the system environment variables. So, having cassandra.thrift.address
in mapred-site.xml does NOTHING, as I needed to
have cassandra.output.thrift.address set. It would be much nicer if the
get{Input/Output}XYZ checked for the existence of getXYZ
if get{Input/Output}XYZ is empty/null. E.g. in getOutputThriftAddress(),
if that setting is null, it would have been nice if that method returned
getThriftAddress(). My problem went away when I put the full cross product
in the XML. E.g. cassandra.input.thrift.address
and cassandra.output.thrift.address (and port, and partitioner).
I still want to know why the old easy way (of setting the 3 system
variables on the pig starter box, and having the config flow into the task
trackers) doesn't work!
will
On Fri, Jan 4, 2013 at 9:04 AM, William Oberman
ober...@civicscience.com wrote:
On all tasktrackers, I see:
java.io.IOException: PIG_OUTPUT_INITIAL_ADDRESS or PIG_INITIAL_ADDRESS
environment variable not set
at
org.apache.cassandra.hadoop.pig.CassandraStorage.setStoreLocation(CassandraStorage.java:821)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setLocation(PigOutputFormat.java:170)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.setUpContext(PigOutputCommitter.java:112)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:86)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.init(PigOutputCommitter.java:67)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:279)
at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:358)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged
Re: Service killed by signal 9
Yes, indeed: Nov 7 20:02:44 ip-10-243-15-139 kernel: [ 4992.839419] Out of memory: Kill process 14183 (jsvc) score 914 or sacrifice child Nov 7 20:02:44 ip-10-243-15-139 kernel: [ 4992.839439] Killed process 14183 (jsvc) total-vm:1181220kB, anon-rss:539164kB, file-rss:12180kB Thanks a lot, at least I can evolute now! :D 2012/11/7 Tristan Seligmann mithra...@mithrandi.net On Wed, Nov 7, 2012 at 9:46 PM, Marcelo Elias Del Valle mvall...@gmail.com wrote: Service killed by signal 9 Signal 9 is SIGKILL. Assuming that you're not killing the process yourself, I guess the most likely cause of this is the OOM killer. If you check /var/log/kern.log or dmesg you should see a message confirming this. -- mithrandi, i Ainil en-Balandor, a faer Ambar -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: ColumnFamilyInputFormat - error when column name is UUID
Answering myself: it seems we can't have any non type 1 UUIDs in column names. I used the UTF8 comparator and saved my UUIDs as strings, it worked. 2012/10/29 Marcelo Elias Del Valle mvall...@gmail.com Hello, I am using ColumnFamilyInputFormat the same way it's described in this example: https://github.com/apache/cassandra/blob/trunk/examples/hadoop_word_count/src/WordCount.java#L215 I have been able to successfully process data in cassandra by using hadoop. However, as this solution doesn't allow me to filter which data in cassandra I want to filter, I decided to create a query column family to list data I want to process in hadoop. This column family is as follows: row key: MM column name: UUID - user ID column value: timestamp - last processed date The problem is, when I run hadoop, I get the exception bellow. Is there any limitation in having UUIDs as column names? I am generating my user IDs with java.util.UUID.randomUUID() for now. I could change the method later, but only type 1 UUIDs are 16 bits longer, isn't it? java.lang.RuntimeException: InvalidRequestException(why:UUIDs must be exactly 16 bytes) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532) at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) Caused by: InvalidRequestException(why:UUIDs must be exactly 16 bytes) at org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12254) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:683) at org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:667) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:356) ... 11 more Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: ColumnFamilyInputFormat - error when column name is UUID
Dean, Are type 1 UUIDs the best ones to use if I want to avoid conflict? I saw this page: http://en.wikipedia.org/wiki/Universally_unique_identifier The only problem with type 1 UUIDs is they are not opaque? I know there is one kind of UUID that can generate two equal values if you generate them at the same milisecond, but I guess I was confusing them... Best regards, Marcelo Valle. 2012/10/29 Hiller, Dean dean.hil...@nrel.gov Hmm, this brings the question of what uuid libraries are others using? I know this one generates type 1 UUIDs with two longs so it is 16 bytes. http://johannburkard.de/software/uuid/ Thanks, Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Monday, October 29, 2012 1:17 PM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: ColumnFamilyInputFormat - error when column name is UUID Answering myself: it seems we can't have any non type 1 UUIDs in column names. I used the UTF8 comparator and saved my UUIDs as strings, it worked. -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: ColumnFamilyInputFormat - error when column name is UUID
Err... Guess you replied in portuguese to the list :D 2012/10/29 Andre Tavares andre...@gmail.com Marcelo, das vezes q tive este problema geralmente era porque o valor UUID sendo tratado para o cassandra não correspondia a um valor exato em UUID, para isso utilizava bastante o UUID.randomUUID() (para gerar um UUID valido) e UUID.fromString(081f4500-047e-401c-8c0b-a41fefd099d7) - este para transformar uma String em UUID valido. Como temos 2 keyspaces no cassandra (dmp_input-Astyanax) e (dmp-PlayOrm) pode acontecer destes frameworks tratarem as chaves UUID de maneira diferentes (em nossa implementação feita ) portanto acho válido a solução que você encontrou (sorry por não ter enxergado o probs antes caso, seja este o seu caso ...) Abs, André 2012/10/29 Marcelo Elias Del Valle mvall...@gmail.com Answering myself: it seems we can't have any non type 1 UUIDs in column names. I used the UTF8 comparator and saved my UUIDs as strings, it worked. 2012/10/29 Marcelo Elias Del Valle mvall...@gmail.com Hello, I am using ColumnFamilyInputFormat the same way it's described in this example: https://github.com/apache/cassandra/blob/trunk/examples/hadoop_word_count/src/WordCount.java#L215 I have been able to successfully process data in cassandra by using hadoop. However, as this solution doesn't allow me to filter which data in cassandra I want to filter, I decided to create a query column family to list data I want to process in hadoop. This column family is as follows: row key: MM column name: UUID - user ID column value: timestamp - last processed date The problem is, when I run hadoop, I get the exception bellow. Is there any limitation in having UUIDs as column names? I am generating my user IDs with java.util.UUID.randomUUID() for now. I could change the method later, but only type 1 UUIDs are 16 bits longer, isn't it? java.lang.RuntimeException: InvalidRequestException(why:UUIDs must be exactly 16 bytes) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532) at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) Caused by: InvalidRequestException(why:UUIDs must be exactly 16 bytes) at org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12254) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:683) at org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:667) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:356) ... 11 more Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: 1000's of column families
Out of curiosity, is it really necessary to have that amount of CFs? I am probably still used to relational databases, where you would use a new table just in case you need to store different kinds of data. As Cassandra stores anything in each CF, it might probably make sense to have a lot of CFs to store your data... But why wouldn't you use a single CF with partitions in these case? Wouldn't it be the same thing? I am asking because I might learn a new modeling technique with the answer. []s 2012/9/26 Hiller, Dean dean.hil...@nrel.gov We are streaming data with 1 stream per 1 CF and we have 1000's of CF. When using the tools they are all geared to analyzing ONE column family at a time :(. If I remember correctly, Cassandra supports as many CF's as you want, correct? Even though I am going to have tons of funs with limitations on the tools, correct? (I may end up wrapping the node tool with my own aggregate calls if needed to sum up multiple column families and such). Thanks, Dean -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: 1000's of column families
Dean, I was used, in the relational world, to use hibernate and O/R mapping. There were times when I used 3 classes (2 inheriting from 1 another) and mapped all of the to 1 table. The common part was in the super class and each sub class had it's own columns. The table, however, use to have all the columns and this design was hard because of that, as creating more subclasses would need changes in the table. However, if you use playOrm and if playOrm has/had a feature to allow inheritance mapping to a CF, it would solve your problem, wouldn't it? Of course it is probably much harder than it might problably appear... :D Best regards, Marcelo Valle. 2012/9/27 Hiller, Dean dean.hil...@nrel.gov We have 1000's of different building devices and we stream data from these devices. The format and data from each one varies so one device has temperature at timeX with some other variables, another device has CO2 percentage and other variables. Every device is unique and streams it's own data. We dynamically discover devices and register them. Basically, one CF or table per thing really makes sense in this environment. While we could try to find out which devices are similar, this would really be a pain and some devices add some new variable into the equation. NOT only that but researchers can register new datasets and upload them as well and each dataset they have they do NOT want to share with other researches necessarily so we have security groups and each CF belongs to security groups. We dynamically create CF's on the fly as people register new datasets. On top of that, when the data sets get too large, we probably want to partition a single CF into time partitions. We could create one CF and put all the data and have a partition per device, but then a time partition will contain multiple devices of data meaning we need to shrink our time partition size where if we have CF per device, the time partition can be larger as it is only for that one device. THEN, on top of that, we have a meta CF for these devices so some people want to query for streams that match criteria AND which returns a CF name and they query that CF name so we almost need a query with variables like select cfName from Meta where x = y and then select * from cfName where x. Which we can do today. Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Thursday, September 27, 2012 8:01 AM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: 1000's of column families Out of curiosity, is it really necessary to have that amount of CFs? I am probably still used to relational databases, where you would use a new table just in case you need to store different kinds of data. As Cassandra stores anything in each CF, it might probably make sense to have a lot of CFs to store your data... But why wouldn't you use a single CF with partitions in these case? Wouldn't it be the same thing? I am asking because I might learn a new modeling technique with the answer. []s 2012/9/26 Hiller, Dean dean.hil...@nrel.govmailto:dean.hil...@nrel.gov We are streaming data with 1 stream per 1 CF and we have 1000's of CF. When using the tools they are all geared to analyzing ONE column family at a time :(. If I remember correctly, Cassandra supports as many CF's as you want, correct? Even though I am going to have tons of funs with limitations on the tools, correct? (I may end up wrapping the node tool with my own aggregate calls if needed to sum up multiple column families and such). Thanks, Dean -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: Correct model
Dean, In the playOrm data modeling, if I understood it correctly, every CF has its own id, right? For instance, User would have its own ID, Activities would have its own id, etc. What if I have a trillion activities? Wouldn't be a problem to have 1 row id for each activity? Cassandra always indexes by row id, right? If I have too many row ids without using composite keys, will it scale the same way? Wouldn't the time to insert an activity be each time longer because I have too many activities? Best regards, Marcelo Valle. 2012/9/25 Hiller, Dean dean.hil...@nrel.gov If you need anything added/fixed, just let PlayOrm know. PlayOrm has been able to quickly add so far…that may change as more and more requests come but so far PlayOrm seems to have managed to keep up. We are using it live by the way already. It works out very well so far for us (We have 5000 column families, obviously dynamically created instead of by hand…a very interesting use case of cassandra). In our live environment we configured astyanax with LocalQUOROM on reads AND writes so CP style so we can afford one node out of 3 to go down but if two go down it stops working THOUGH there is a patch in astyanax to auto switch from LocalQUOROM to ONE NODE read/write when two nodes go down that we would like to suck in eventually so it is always live(I don't think Hector has that and it is a really NICE feature….ie fail localquorm read/write and then try again with consistency level of one). Later, Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Monday, September 24, 2012 1:54 PM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: Correct model Dean, this sounds like magic :D I don't know details about the performance on the index implementations you chose, but it would pay the way to use it in my case, as I don't need the best performance in the world when reading, but I need to assure scalability and have a simple model to maintain. I liked the playOrm concept regarding this. I have more doubts, but I will ask them at stack over flow from now on. 2012/9/24 Hiller, Dean dean.hil...@nrel.govmailto:dean.hil...@nrel.gov PlayOrm will automatically create a CF to index my CF? It creates 3 CF's for all indices, IntegerIndice, DecimalIndice, and StringIndice such that the ad-hoc tool that is in development can display the indices as it knows the prefix of the composite column name is of Integer, Decimal or String and it knows the postfix type as well so it can translate back from bytes to the types and properly display in a GUI (i.e. On top of SELECT, the ad-hoc tool is adding a way to view the induce rows so you can check if they got corrupt or not). Will it auto-manage it, like Cassandra's secondary indexes? YES Further detail… You annotated fields with @NoSqlIndexed and PlayOrm adds/removes from the index as you add/modify/remove the entity…..a modify does a remove old val from index and insert new value into index. An example would be PlayOrm stores all long, int, short, byte in a type that uses the least amount of space so IF you have a long OR BigInteger between –128 to 128 it only ends up storing 1 byte in cassandra(SAVING tons of space!!!). Then if you are indexing a type that is one of those, PlayOrm creates a IntegerIndice table. Right now, another guy is working on playorm-server which is a webgui to allow ad-hoc access to all your data as well so you can ad-hoc queries to see data and instead of showing Hex, it shows the real values by translating the bytes to String for the schema portions that it is aware of that is. Later, Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.commailto:mvall...@gmail.commailto:mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org mailto:user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.orgmailto: user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Monday, September 24, 2012 12:09 PM To: user@cassandra.apache.orgmailto:user@cassandra.apache.orgmailto: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.orgmailto: user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: Correct model Dean, There is one last thing I would like to ask about playOrm by this list, the next questiosn will come by stackOverflow. Just because of the context, I prefer asking this here: When you say playOrm indexes a table (which would be a CF behind the scenes), what do you mean? PlayOrm will automatically create a CF to index my CF? Will it auto-manage it, like Cassandra's
Re: Correct model
2012/9/23 Hiller, Dean dean.hil...@nrel.gov You need to split data among partitions or your query won't scale as more and more data is added to table. Having the partition means you are querying a lot less rows. This will happen in case I can query just one partition. But if I need to query things in multiple partitions, wouldn't it be slower? He means determine the ONE partition key and query that partition. Ie. If you want just latest user requests, figure out the partition key based on which month you are in and query it. If you want the latest independent of user, query the correct single partition for GlobalRequests CF. But in this case, I didn't understand Aaron's model then. My first query is to get all requests for a user. If I did partitions by time, I will need to query all partitions to get the results, right? In his answer it was said I would query ONE partition... If I want all the requests for the user, couldn't I just select all UserRequest records which start with userId? He designed it so the user requests table was completely scalable so he has partitions there. If you don't have partitions, you could run into a row that is t long. You don't need to design it this way if you know none of your users are going to go into the millions as far as number of requests. In his design then, you need to pick the correct partition and query into that partition. You mean too many rows, not a row too long, right? I am assuming each request will be a different row, not a new column. Is having billions of ROWS something non performatic in Cassandra? I know Cassandra allows up to 2 billion columns for a CF, but I am not aware of a limitation for rows... I really didn't understand why to use partitions. Partitions are a way if you know your rows will go into the trillions of breaking them up so each partition has 100k rows or so or even 1 million but maxes out in the millions most likely. Without partitions, you hit a limit in the millions. With partitions, you can keep scaling past that as you can have as many partitions as you want. If I understood it correctly, if I don't specify partitions, Cassandra will store all my data in a single node? I thought Cassandra would automatically distribute my data among nodes as I insert rows into a CF. Of course if I use partitions I understand I could query just one partition (node) to get the data, if I have the partition field, but to the best of my knowledge, this is not what happens in my case, right? In the first query I would have to query all the partitions... Or you are saying partitions have nothing to do with nodes?? I 99,999% of my users will have less than 100k requests, would it make sense to partition by user? A multi-get is a query that finds IN PARALLEL all the rows with the matching keys you send to cassandra. If you do 1000 gets(instead of a multi-get) with 1ms latency, you will find, it takes 1 second+processing time. If you do ONE multi-get, you only have 1 request and therefore 1ms latency. The other solution is you could send 1000 asycnh gets but I have a feeling that would be slower with all the marshalling/unmarshalling of the envelope…..really depends on the envelope size like if we were using http, you would get killed doing 1000 requests instead of 1 with 1000 keys in it. That's cool! :D So if I need to query data split in 10 partitions, for instance, I can perform the query in parallel by using a multiget, right? Out of curiosity, if each get will occur on a different node, I would need to connect to each of the nodes? Or would I query 1 node and it would communicate to others? Later, Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Sunday, September 23, 2012 10:23 AM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: Correct model 2012/9/20 aaron morton aa...@thelastpickle.commailto: aa...@thelastpickle.com I would consider: # User CF * row_key: user_id * columns: user properties, key=value # UserRequests CF * row_key: user_id : partition_start where partition_start is the start of a time partition that makes sense in your domain. e.g. partition monthly. Generally want to avoid rows the grow forever, as a rule of thumb avoid rows more than a few 10's of MB. * columns: two possible approaches: 1) If the requests are immutable and you generally want all of the data store the request in a single column using JSON or similar, with the column name a timestamp. 2) Otherwise use a composite column name of timestamp : request_property to store the request in many columns. * In either case consider using Reversed comparators so the most recent columns are first see http://thelastpickle.com/2011/10/03
Re: Correct model
2012/9/24 Hiller, Dean dean.hil...@nrel.gov I am confused. In this email you say you want get all requests for a user and in a previous one you said Select all the users which has new requests, since date D so let me answer both… I have both needs. These are the two queries I need to perform on the model. For latter, you make ONE query into the latest partition(ONE partition) of the GlobalRequestsCF which gives you the most recent requests ALONG with the user ids of those requests. If you queried all partitions, you would most likely blow out your JVM memory. For the former, you make ONE query to the UserRequestsCF with userid = your user id to get all the requests for that user Now I think I got the main idea! This answered a lot! Sorry, I was skipping some context. A lot of the backing indexing sometimes is done as a long row so in playOrm, too many rows in a partition means == too many columns in the indexing row for that partition. I believe the same is true in cassandra for their indexing. Oh, ok, you were talking about the wide row pattern, right? But playORM is compatible with Aaron's model, isn't it? Can I map exactly this using playORM? The hardest thing for me to use playORM now is I don't know Cassandra well yet, and I know playORM even less. Can I ask playOrm questions in this list? I will try to create a POC here! Only now I am starting to understand what it does ;-) The examples directory is empty for now, I would like to see how to set up the connection with it. Cassandra spreads all your data out on all nodes with or without partitions. A single partition does have it's data co-located though. Now I see. The main advantage of using partitions is keeping the indexes small enough. It has nothing to do with the nodes. Thanks! If you are at 100k(and the requests are rather small), you could embed all the requests in the user or go with Aaron's below suggestion of a UserRequestsCF. If your requests are rather large, you probably don't want to embed them in the User. Either way, it's one query or one row key lookup. I see it now. Multiget ignores partitions…you feed it a LIST of keys and it gets them. It just so happens that partitionId had to be part of your row key. Do you mean I need to load all the keys in memory to do a multiget? I have used Hector and now use Astyanax, I don't worry much about that layer, but I feed astyanax 3 nodes and I believe it discovers some of the other ones. I believe the latter is true but am not 100% sure as I have not looked at that code. Why did you move? Hector is being considered for being the official client for Cassandra, isn't it? I looked at the Astyanax api and it seemed much more high level though As an analogy on the above, if you happen to have used PlayOrm, you would ONLY need one Requests table and you partition by user AND time(two views into the same data partitioned two different ways) and you can do exactly the same thing as Aaron's example. PlayOrm doesn't embed the partition ids in the key leaving it free to partition twice like in your case….and in a refactor, you have to map/reduce A LOT more rows because of rows having the FK of partitionidsubrowkey whereas if you don't have partition id in the key, you only map/reduce the partitioned table in a redesign/refactor. That said, we will be adding support for CQL partitioning in addition to PlayOrm partitioning even though it can be a little less flexible sometimes. I am not sure I understood this part. If I need to refactor, having the partition id in the key would be a bad thing? What would be the alternative? In my case, as I use userId : partitionId as row key, this might be a problem, right? Also, CQL locates all the data on one node for a partition. We have found it can be faster sometimes with the parallelized disks that the partitions are NOT all on one node so PlayOrm partitions are virtual only and do not relate to where the rows are stored. An example on our 6 nodes was a join query on a partition with 1,000,000 rows took 60ms (of course I can't compare to CQL here since it doesn't do joins). It really depends how much data is going to come back in the query though too? There are tradeoff's between disk parallel nodes and having your data all on one node of course. I guess I am still not ready for this level of info. :D In the playORM readme, we have the following: @NoSqlQuery(name=findWithJoinQuery, query=PARTITIONS t(:partId) SELECT t FROM TABLE as t + INNER JOIN t.activityTypeInfo as i WHERE i.type = :type and t.numShares :shares), What would happen behind the scenes when I execute this query? You can only use joins with partition keys, right? In this case, is partId the row id of TABLE CF? Thanks a lot for the answers -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: Correct model
, right? Nope, joins work on anything. You only need to specify the partitionId when you have a partitioned table in the list of join tables. (That is what the PARTITIONS clause is for, to identify partitionId = what?)…it was put BEFORE the SQL instead of within it…CQL took the opposite approach but PlayOrm can also join different partitions together as well ;) ). In this case, is partId the row id of TABLE CF? Nope, partId is one of the columns. There is a test case on this class in PlayOrm …(notice the annotation NoSqlPartitionByThisField on the column/field in the entity)… https://github.com/deanhiller/playorm/blob/master/input/javasrc/com/alvazan/test/db/PartitionedSingleTrade.java PlayOrm allows partitioned tables AND non-partioned tables(non-partitioned tables won't scale but maybe you will never have that many rows). You can join any two combinations(non-partitioned with partitioned, non-partitioned with non-partitioned, partition with another partition). I only prefer stackoverflow as I like referencing links/questions with their urls. To reference this email is very hard later on as I have to find it so in general, I HATE email lists ;) but it seems cassandra prefers them so any questions on PlayOrm you can put there and I am not sure how many on this may or may not be interested so it creates less noise on this list too. Later, Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Monday, September 24, 2012 11:07 AM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: Correct model 2012/9/24 Hiller, Dean dean.hil...@nrel.govmailto:dean.hil...@nrel.gov I am confused. In this email you say you want get all requests for a user and in a previous one you said Select all the users which has new requests, since date D so let me answer both… I have both needs. These are the two queries I need to perform on the model. For latter, you make ONE query into the latest partition(ONE partition) of the GlobalRequestsCF which gives you the most recent requests ALONG with the user ids of those requests. If you queried all partitions, you would most likely blow out your JVM memory. For the former, you make ONE query to the UserRequestsCF with userid = your user id to get all the requests for that user Now I think I got the main idea! This answered a lot! Sorry, I was skipping some context. A lot of the backing indexing sometimes is done as a long row so in playOrm, too many rows in a partition means == too many columns in the indexing row for that partition. I believe the same is true in cassandra for their indexing. Oh, ok, you were talking about the wide row pattern, right? But playORM is compatible with Aaron's model, isn't it? Can I map exactly this using playORM? The hardest thing for me to use playORM now is I don't know Cassandra well yet, and I know playORM even less. Can I ask playOrm questions in this list? I will try to create a POC here! Only now I am starting to understand what it does ;-) The examples directory is empty for now, I would like to see how to set up the connection with it. Cassandra spreads all your data out on all nodes with or without partitions. A single partition does have it's data co-located though. Now I see. The main advantage of using partitions is keeping the indexes small enough. It has nothing to do with the nodes. Thanks! If you are at 100k(and the requests are rather small), you could embed all the requests in the user or go with Aaron's below suggestion of a UserRequestsCF. If your requests are rather large, you probably don't want to embed them in the User. Either way, it's one query or one row key lookup. I see it now. Multiget ignores partitions…you feed it a LIST of keys and it gets them. It just so happens that partitionId had to be part of your row key. Do you mean I need to load all the keys in memory to do a multiget? I have used Hector and now use Astyanax, I don't worry much about that layer, but I feed astyanax 3 nodes and I believe it discovers some of the other ones. I believe the latter is true but am not 100% sure as I have not looked at that code. Why did you move? Hector is being considered for being the official client for Cassandra, isn't it? I looked at the Astyanax api and it seemed much more high level though As an analogy on the above, if you happen to have used PlayOrm, you would ONLY need one Requests table and you partition by user AND time(two views into the same data partitioned two different ways) and you can do exactly the same thing as Aaron's example. PlayOrm doesn't embed the partition ids in the key leaving it free to partition twice like in your case….and in a refactor, you have to map
Re: Correct model
Dean, this sounds like magic :D I don't know details about the performance on the index implementations you chose, but it would pay the way to use it in my case, as I don't need the best performance in the world when reading, but I need to assure scalability and have a simple model to maintain. I liked the playOrm concept regarding this. I have more doubts, but I will ask them at stack over flow from now on. 2012/9/24 Hiller, Dean dean.hil...@nrel.gov PlayOrm will automatically create a CF to index my CF? It creates 3 CF's for all indices, IntegerIndice, DecimalIndice, and StringIndice such that the ad-hoc tool that is in development can display the indices as it knows the prefix of the composite column name is of Integer, Decimal or String and it knows the postfix type as well so it can translate back from bytes to the types and properly display in a GUI (i.e. On top of SELECT, the ad-hoc tool is adding a way to view the induce rows so you can check if they got corrupt or not). Will it auto-manage it, like Cassandra's secondary indexes? YES Further detail… You annotated fields with @NoSqlIndexed and PlayOrm adds/removes from the index as you add/modify/remove the entity…..a modify does a remove old val from index and insert new value into index. An example would be PlayOrm stores all long, int, short, byte in a type that uses the least amount of space so IF you have a long OR BigInteger between –128 to 128 it only ends up storing 1 byte in cassandra(SAVING tons of space!!!). Then if you are indexing a type that is one of those, PlayOrm creates a IntegerIndice table. Right now, another guy is working on playorm-server which is a webgui to allow ad-hoc access to all your data as well so you can ad-hoc queries to see data and instead of showing Hex, it shows the real values by translating the bytes to String for the schema portions that it is aware of that is. Later, Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Monday, September 24, 2012 12:09 PM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: Correct model Dean, There is one last thing I would like to ask about playOrm by this list, the next questiosn will come by stackOverflow. Just because of the context, I prefer asking this here: When you say playOrm indexes a table (which would be a CF behind the scenes), what do you mean? PlayOrm will automatically create a CF to index my CF? Will it auto-manage it, like Cassandra's secondary indexes? In Cassandra, the application is responsible for maintaining the index, right? I might be wrong, but unless I am using secondary indexes I need to update index values manually, right? I got confused when you said PlayOrm indexes the columns you choose. How do I choose and what exactly it means? Best regards, Marcelo Valle. 2012/9/24 Hiller, Dean dean.hil...@nrel.govmailto:dean.hil...@nrel.gov Oh, ok, you were talking about the wide row pattern, right? yes But playORM is compatible with Aaron's model, isn't it? Not yet, PlayOrm supports partitioning one table multiple ways as it indexes the columns(in your case, the userid FK column and the time column) Can I map exactly this using playORM? Not yet, but the plan is to map these typical Cassandra scenarios as well. Can I ask playOrm questions in this list? The best place to ask PlayOrm questions is on stack overflow and tag with PlayOrm though I monitor this list and stack overflow for questions(there are already a few questions on stack overflow). The examples directory is empty for now, I would like to see how to set up the connection with it. Running build or build.bat is always kept working and all 62 tests pass(or we don't merge to master) so to see how to make a connection or run an example 1. Run build.bat or build which generates parsing code 2. Import into eclipse (it already has .classpath and .project for you already there) 3. In FactorySingleton.java you can modify IN_MEMORY to CASSANDRA or not and run any of the tests in-memory or against localhost(We run the test suite also against a 6 node cluster as well and all passes) 4. FactorySingleton probably has the code you are looking for plus you need a class called nosql.Persistence or it won't scan your jar file.(class file not xml file like JPA) Do you mean I need to load all the keys in memory to do a multi get? No, you batch. I am not sure about CQL, but PlayOrm returns a Cursor not the results so you can loop through every key and behind the scenes it is doing batch requests so you can load up 100 keys and make one multi get request for those 100 keys and then can load up the next 100 keys, etc. etc. etc. I need to look
performance for different kinds of row keys
Suppose two cases: 1. I have a Cassandra column family with non-composite row keys = incremental id 2. I have a Cassandra column family with a composite row keys = incremental id 1 : group id Which one will be faster to insert? And which one will be faster to read by incremental id? Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: Correct model
2012/9/20 aaron morton aa...@thelastpickle.com I would consider: # User CF * row_key: user_id * columns: user properties, key=value # UserRequests CF * row_key: user_id : partition_start where partition_start is the start of a time partition that makes sense in your domain. e.g. partition monthly. Generally want to avoid rows the grow forever, as a rule of thumb avoid rows more than a few 10's of MB. * columns: two possible approaches: 1) If the requests are immutable and you generally want all of the data store the request in a single column using JSON or similar, with the column name a timestamp. 2) Otherwise use a composite column name of timestamp : request_property to store the request in many columns. * In either case consider using Reversed comparators so the most recent columns are first see http://thelastpickle.com/2011/10/03/Reverse-Comparators/ # GlobalRequests CF * row_key: partition_start - time partition as above. It may be easier to use the same partition scheme. * column name: timestamp : user_id * column value: empty Ok, I think I understood your suggestion... But the only advantage in this solution is to split data among partitions? I understood how it would work, but I didn't understand why it's better than the other solution, without the GlobalRequests CF - Select all the requests for an user Work out the current partition client side, get the first N columns. Then page. What do you mean here by current partition? You mean I would perform a query for each particition? If I want all the requests for the user, couldn't I just select all UserRequest records which start with userId? I might be missing something here, but in my understanding if I use hector to query a column familly I can do that and Cassandra servers will automatically communicate to each other to get the data I need, right? Is it bad? I really didn't understand why to use partitions. - Select all the users which has new requests, since date D Worm out the current partition client side, get the first N columns from GlobalRequests, make a multi get call to UserRequests NOTE: Assuming the size of the global requests space is not huge. Hope that helps. For sure it is helping a lot. However, I don't know what is a multiget... I saw the hector api reference and found this method, but not sure about what Cassandra would do internally if I do a multiget... Is this expensive in terms of performance and latency? -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: Is Cassandra right for me?
2012/9/20 aaron morton aa...@thelastpickle.com Actually, if I use community edition for now, I wouldn't be able to use hadoop against data stored in CFS? AFAIK DSC is a packaged deployment of Apache Cassandra. You should be ale to use Hadoop against it, in the same way you can use hadoop against Apache Cassandra. You can do anything with computers if you have enough time and patience. DSE reduces the amount of time and patience needed to run Hadoop over Cassandra. Specifically it helps by providing a HDFS and Hive Meta Store that run on Cassandra. This reduces the number of moving parts you need to provision. Can I use BRISK with Apache Cassandra, without changing Brisk or Cassandra's code? To the best of my knowledge, DSE uses Brisk, so I am afraid of writting hadoop process now and have to change them when I hire DSE support. I am not an expert in the Apache 2.0 license, but in my understanding Data Stax modified Apache Cassandra and included modifications to it in the version they sell. At the same time I am interested in hiring their support, I wanna keep compatibility with the open source version distributed in the mainstream, just in case I want to stop hiring their support at any time. -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: Is Cassandra right for me?
Thanks a lot! Things are much more clear now. 2012/9/21 Michael Kjellman mkjell...@barracuda.com Brisk is no longer actively developed by the original author or Datastax. It was left up for the community. https://github.com/steeve/brisk Has a fork that is supposedly compatible with 1.0 API Your more than welcome to fork that and make it work with 1.1 :) DSE != (Cassandra + Brisk) From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Friday, September 21, 2012 10:27 AM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: Is Cassandra right for me? 2012/9/20 aaron morton aa...@thelastpickle.commailto: aa...@thelastpickle.com Actually, if I use community edition for now, I wouldn't be able to use hadoop against data stored in CFS? AFAIK DSC is a packaged deployment of Apache Cassandra. You should be ale to use Hadoop against it, in the same way you can use hadoop against Apache Cassandra. You can do anything with computers if you have enough time and patience. DSE reduces the amount of time and patience needed to run Hadoop over Cassandra. Specifically it helps by providing a HDFS and Hive Meta Store that run on Cassandra. This reduces the number of moving parts you need to provision. Can I use BRISK with Apache Cassandra, without changing Brisk or Cassandra's code? To the best of my knowledge, DSE uses Brisk, so I am afraid of writting hadoop process now and have to change them when I hire DSE support. I am not an expert in the Apache 2.0 license, but in my understanding Data Stax modified Apache Cassandra and included modifications to it in the version they sell. At the same time I am interested in hiring their support, I wanna keep compatibility with the open source version distributed in the mainstream, just in case I want to stop hiring their support at any time. -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr 'Like' us on Facebook for exclusive content and other resources on all Barracuda Networks solutions. Visit http://barracudanetworks.com/facebook -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Data stax community
Not sure if this question should be asked in this list, if this is the wrong place to ask this, please tell me. Does anyone know if Data Stax community edition alows us to run in production? I plan to use the enterprise edition later, but for now even for production I am thinking in using community version. -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: Data stax community
Thanks! 2012/9/19 Tyler Hobbs ty...@datastax.com DataStax Community is free for any type of use, including production. On Wed, Sep 19, 2012 at 1:42 PM, Abhijit Chanda abhijit.chan...@gmail.com wrote: You better ask this question http://www.datastax.com/support-forums/forum/datastax-enterprise there. Any ways as far as i am concern it should not be problematic thing. Regards, Abhijit On Thu, Sep 20, 2012 at 12:07 AM, Marcelo Elias Del Valle mvall...@gmail.com wrote: Not sure if this question should be asked in this list, if this is the wrong place to ask this, please tell me. Does anyone know if Data Stax community edition alows us to run in production? I plan to use the enterprise edition later, but for now even for production I am thinking in using community version. -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr -- Abhijit Chanda Software Developer VeHere Interactive Pvt. Ltd. +91-974395 -- Tyler Hobbs DataStax http://datastax.com/ -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Correct model
I am new to Cassandra and NoSQL at all. I built my first model and any comments would be of great help. I am describing my thoughts bellow. It's a very simple model. I will need to store several users and, for each user, I will need to store several requests. It request has it's insertion time. As the query comes first, here are the only queries I will need to run against this model: - Select all the requests for an user - Select all the users which has new requests, since date D I created the following model: an UserCF, whose key is a userID generated by TimeUUID, and a RequestCF, whose key is composite: UserUUID + timestamp. For each user, I will store basic data and, for each request, I will insert a lot of columns. My questions: - Is the strategy of using a composite key good for this case? I thought in other solutions, but this one seemed to be the best. Another solution would be have a non-composite key of type UUID for the requests, and have another CF to relate user and request. - To perform the second query, instead of selecting if each user has a request inserted after date D, I thought in storing the last request insertion date into the userCF, everytime I have a new insert for the user. It would be a data replication, but I would have no read-before-write and I am guessing the second query would perform faster. Any thoughts? -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: Correct model
2012/9/19 Hiller, Dean dean.hil...@nrel.gov Thinking out loud and I think a bit towards playOrm's model though you don’t' need to use playroom for this. 1. I would probably have a User with the requests either embedded in or the Foreign keys to the requests…either is fine as long as you get the user get ALL FK's and make one request to get the requests for that user This was my first option. However, everytime I have a new request I would need to read the column request_ids, update its value, and them write the result. This would be a read-before-write, which is bad in Cassandra, right? Or you were talking about other kinds of FKs? 2. I would create rows for index and index each month of data OR maybe index each day of data(depends on your system). Then, I can just query into the index for that one month. With playOrm S-SQL, this is a simple PARTITIONS r(:thismonthParititonId) SELECT r FROM Request r where r.date :date OR you just do a column range query doing the same thing into your index. The index is basically the wide row pattern ;) with composite keys of date.rowkey of request I would consider playOrm in a later step in my project, as my understanding now is it is good to store relational data, structured data. I cannot predict which columns I am going to store in requestCF. But regardless, even in Cassandra, you would still use a composite key, but it seems you would create an indexCf using the wide row pattern, and each request would have its own id, right? But why? Wouldn't it be faster to have a composite key in the requestCF itself? From: Marcelo Elias Del Valle mvall...@gmail.commailto:mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Wednesday, September 19, 2012 1:02 PM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Correct model I am new to Cassandra and NoSQL at all. I built my first model and any comments would be of great help. I am describing my thoughts bellow. It's a very simple model. I will need to store several users and, for each user, I will need to store several requests. It request has it's insertion time. As the query comes first, here are the only queries I will need to run against this model: - Select all the requests for an user - Select all the users which has new requests, since date D I created the following model: an UserCF, whose key is a userID generated by TimeUUID, and a RequestCF, whose key is composite: UserUUID + timestamp. For each user, I will store basic data and, for each request, I will insert a lot of columns. My questions: - Is the strategy of using a composite key good for this case? I thought in other solutions, but this one seemed to be the best. Another solution would be have a non-composite key of type UUID for the requests, and have another CF to relate user and request. - To perform the second query, instead of selecting if each user has a request inserted after date D, I thought in storing the last request insertion date into the userCF, everytime I have a new insert for the user. It would be a data replication, but I would have no read-before-write and I am guessing the second query would perform faster. Any thoughts? -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: Correct model
In your first email, you get a request and seem to shove it and a user in
generating the ids which means that user never generates a request ever
again??? If a user sends multiple requests in, how are you looking up his
TimeUUID row key from your first email(I would do the same in my
implementation)?
Actually, I don't get it from Cassandra. I am using Cassandra for the
writes, but to find the userId I look on a pre-indexed structure, because I
think the reads would be faster this way. I need to find the userId by some
key fields, so I use an index like this:
user ID 5596 - { name - john denver, phone - , field3 -
field 3 data, field 10 - field 10 data}
The values are just examples. This part is not implemented yet and I am
looking for alternatives. Currently we have some similar indexes in SOLR,
but we are thinking in keeping the index in memory and replicating manually
in the cluster, or using Voldemort, etc.
I might be wrong, but I think Cassandra is great for writes, but a solution
like this would be better for reads.
If you had an ldap unique username, I would just use that as the primary
key meaning you NEVER have to do reads. If you have a username and need
to lookup a UUID, you would have to do that in both implementationsŠnot a
real big deal thoughŠa quick quick lookup table does the trick there and
in most cases is still fast enough(ie. Read before write here is ok in a
lot of cases).
That X-ref table would simple be rowkey=username and value=users real
primary key
Though again, we use ldap and know no one's username is really going to
change so username is our primary key.
In my case, a single user can have thousands of requests. In my userCF, I
will have just 1 user with uuid X, but I am not sure about what to have in
my requestCF.
--
Marcelo Elias Del Valle
http://mvalle.com - @mvallebr
Re: Is Cassandra right for me?
Aaron, Thank you very much for the answers! Helped me a lot! I would like just a bit more clarification about the points bellow, if you allow me: - You can query your data using Hadoop easily enough. You may want take a look at DSE from http://datastax.com/ it makes using Hadoop and Solr with cassandra easier. Actually, if I use community edition for now, I wouldn't be able to use hadoop against data stored in CFS? We are considering the enterprise edition here, but the best scenario would be using it just when really needed. Would writes on HDFS be so quick as in Cassandra? - It depends on how many moving parts you are comfortable with. Same for the questions about HDFS etc. Start with the smallest about of infrastructure. Sorry, I didn't really understand this part. I am not sure what you wanted to say, but the question was about using nosql instead a relational database in this case. If learning nosql is not a problem, would I have advantages in using Cassandra instead of HBase? If everything in my model fits into a relational database, if my data is structured, would it still be a good idea to use Cassandra? Why? Thanks, Marcelo. 2012/9/18 aaron morton aa...@thelastpickle.com Also, I saw a presentation which said that if I don't have rows with more than a hundred rows in Cassandra, whether I am doing something wrong or I shouldn't be using Cassandra. I do not agree with that statement. (I read that as rows with ore than a hundred _columns_) - I need to support a high volume of writes per second. I might have a billion writes per hour Thats about 280K /sec. Netflix did a benchmark that shows 1.1M/sec http://techblog.netflix.com/2011/11/benchmarking-cassandra-scalability-on.html - I need to write non-structured data that will be processed later by hadoop processes to generate structured data from it. Later, I index the structured data using SOLR or SOLANDRA, so the data can be consulted by my end user application. Is Cassandra recommended for that, or should I be thinking in writting directly to HDFS files, for instance? What's the main advantage I get from storing data in a nosql service like Cassandra, when compared to storing files into HDFS? - You can query your data using Hadoop easily enough. You may want take a look at DSE from http://datastax.com/ it makes using Hadoop and Solr with cassandra easier. - If I don't need to perform complicated queries in Cassandra, should I store the json-like data just as a column value? I am afraid of doing something wrong here, as I would need just to store the json file and some more 5 or 6 fields to query the files later. - Store the data in the way that best supports the read queries you want to make. If you always read all the fields, or it's a canonical record of events storing as JSON may be best. If you often get a few fields, and maybe they are updated, storing each field as a column value may be best. - Does it make sense to you to use hadoop to process data from Cassandra and store the results in a database, like HBase? Once I have structured data, is there any reason I should use Cassandra instead of HBase? - It depends on how many moving parts you are comfortable with. Same for the questions about HDFS etc. Start with the smallest about of infrastructure. Hope that helps. - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 18/09/2012, at 10:28 AM, Marcelo Elias Del Valle mvall...@gmail.com wrote: Hello, I am new to Cassandra and I am in doubt if Cassandra is the right technology to use in the architecture I am defining. Also, I saw a presentation which said that if I don't have rows with more than a hundred rows in Cassandra, whether I am doing something wrong or I shouldn't be using Cassandra. Therefore, it might be the case I am doing something wrong. If you could help me to find out the answer for these questions by giving any feedback, it would be highly appreciated. Here is my need and what I am thinking in using Cassandra for: - I need to support a high volume of writes per second. I might have a billion writes per hour - I need to write non-structured data that will be processed later by hadoop processes to generate structured data from it. Later, I index the structured data using SOLR or SOLANDRA, so the data can be consulted by my end user application. Is Cassandra recommended for that, or should I be thinking in writting directly to HDFS files, for instance? What's the main advantage I get from storing data in a nosql service like Cassandra, when compared to storing files into HDFS? - Usually I will write json data associated to an ID and my hadoop processes will process this data to write data to a database. I have two doubts here: - If I don't need to perform
Re: Is Cassandra right for me?
I will have just 6 columns in my CF, but I will have about a billion writes per hour. In this case, I think Cassandra applies then, by what you are saying. This answer helped a lot too, thanks! 2012/9/18 Hiller, Dean dean.hil...@nrel.gov I wanted to clarify the where that statement comes from on wide rows …. Realize some people make the claim that if you don’t' have 1000's of columns in some rows in cassandra you are doing something wrong. This is not true, BUT it comes from the fact that people are setting up indexes. This is what leads to the very wide row affect. playOrm is one such library using wide rows like this BUT it is NOT necessary for all applications. You can easily use map/reduce on a cassandra cluster. You can map/reduce your dataset into a new model if you make a mistake as well and don't get it right the first time. This wide row affect is 80% of the time used for indexing. I draw off playOrm examples a lot but one table may be partitioned by time so each month of data is in a partition, you can then have indexes on each partition allowing you to do quick queries into partitions. Later, Dean From: Marcelo Elias Del Valle mvall...@gmail.commailto: mvall...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Monday, September 17, 2012 4:28 PM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Is Cassandra right for me? Hello, I am new to Cassandra and I am in doubt if Cassandra is the right technology to use in the architecture I am defining. Also, I saw a presentation which said that if I don't have rows with more than a hundred rows in Cassandra, whether I am doing something wrong or I shouldn't be using Cassandra. Therefore, it might be the case I am doing something wrong. If you could help me to find out the answer for these questions by giving any feedback, it would be highly appreciated. Here is my need and what I am thinking in using Cassandra for: * I need to support a high volume of writes per second. I might have a billion writes per hour * I need to write non-structured data that will be processed later by hadoop processes to generate structured data from it. Later, I index the structured data using SOLR or SOLANDRA, so the data can be consulted by my end user application. Is Cassandra recommended for that, or should I be thinking in writting directly to HDFS files, for instance? What's the main advantage I get from storing data in a nosql service like Cassandra, when compared to storing files into HDFS? * Usually I will write json data associated to an ID and my hadoop processes will process this data to write data to a database. I have two doubts here: * If I don't need to perform complicated queries in Cassandra, should I store the json-like data just as a column value? I am afraid of doing something wrong here, as I would need just to store the json file and some more 5 or 6 fields to query the files later. * Does it make sense to you to use hadoop to process data from Cassandra and store the results in a database, like HBase? Once I have structured data, is there any reason I should use Cassandra instead of HBase? I am sorry if the questions are too dummy, I have been watching a lot of videos and reading a lot of documentation about Cassandra, but honestly, more I read more I have questions. Thanks in advance. Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
Re: Is Cassandra right for me?
You're talking about this project, right? https://github.com/deanhiller/playorm I will take a look. However, I don't think using Cassandra's model itself (with CFs / key-values) would be a problem, I just need to know where the advantage relies on. By your answer, my guess is it relies on better performance and more control. I also saw that if I plan to use Data Stax enterprise to get real time analytics, my data would need to be stored in Cassandra's usual format. It would harder for me use PlayOrm if I am planning to use advanced data stax features, like Solr indexing data on Cassandra without copying columns, realtime, wouldn't it? I don't know much of this Solr feature yet, but my understanding today is it wouldn't be aware of the tables I create with playOrm, just of the column families this framework uses to store the data, right? 2012/9/18 Hiller, Dean dean.hil...@nrel.gov Until Aaron replies, here are my thoughts on the relational piece… If everything in my model fits into a relational database, if my data is structured, would it still be a good idea to use Cassandra? Why? The playOrm project explores exactly this issue……A query on 1,000,000 rows in a single partition only took 60ms AND you can do joins with it's S-SQL language. The answer is a resounding YES, you can put relational data in cassandra. The writes are way faster than a DBMS and joins and SQL can be just as fast and in many cases FASTER on noSQL IF you partition your data properly. A S-SQL statement looks like so on playOrm PARTITIONS t(:partitionId) SELECT t FROM Trades as t where t.numShares 10 You can have as many partitions as you want and a single partition can have millions of rows though I would not exceed 10 million probably. Later, Dean 2012/9/18 aaron morton aa...@thelastpickle.commailto: aa...@thelastpickle.com Also, I saw a presentation which said that if I don't have rows with more than a hundred rows in Cassandra, whether I am doing something wrong or I shouldn't be using Cassandra. I do not agree with that statement. (I read that as rows with ore than a hundred _columns_) * I need to support a high volume of writes per second. I might have a billion writes per hour Thats about 280K /sec. Netflix did a benchmark that shows 1.1M/sec http://techblog.netflix.com/2011/11/benchmarking-cassandra-scalability-on.html * I need to write non-structured data that will be processed later by hadoop processes to generate structured data from it. Later, I index the structured data using SOLR or SOLANDRA, so the data can be consulted by my end user application. Is Cassandra recommended for that, or should I be thinking in writting directly to HDFS files, for instance? What's the main advantage I get from storing data in a nosql service like Cassandra, when compared to storing files into HDFS? * You can query your data using Hadoop easily enough. You may want take a look at DSE from http://datastax.com/ it makes using Hadoop and Solr with cassandra easier. * If I don't need to perform complicated queries in Cassandra, should I store the json-like data just as a column value? I am afraid of doing something wrong here, as I would need just to store the json file and some more 5 or 6 fields to query the files later. * Store the data in the way that best supports the read queries you want to make. If you always read all the fields, or it's a canonical record of events storing as JSON may be best. If you often get a few fields, and maybe they are updated, storing each field as a column value may be best. * Does it make sense to you to use hadoop to process data from Cassandra and store the results in a database, like HBase? Once I have structured data, is there any reason I should use Cassandra instead of HBase? * It depends on how many moving parts you are comfortable with. Same for the questions about HDFS etc. Start with the smallest about of infrastructure. Hope that helps. - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 18/09/2012, at 10:28 AM, Marcelo Elias Del Valle mvall...@gmail.com mailto:mvall...@gmail.com wrote: Hello, I am new to Cassandra and I am in doubt if Cassandra is the right technology to use in the architecture I am defining. Also, I saw a presentation which said that if I don't have rows with more than a hundred rows in Cassandra, whether I am doing something wrong or I shouldn't be using Cassandra. Therefore, it might be the case I am doing something wrong. If you could help me to find out the answer for these questions by giving any feedback, it would be highly appreciated. Here is my need and what I am thinking in using Cassandra for: * I need to support a high volume of writes per second. I might have a billion writes per hour * I need to write non-structured data that will be processed later by hadoop processes
Is Cassandra right for me?
Hello, I am new to Cassandra and I am in doubt if Cassandra is the right technology to use in the architecture I am defining. Also, I saw a presentation which said that if I don't have rows with more than a hundred rows in Cassandra, whether I am doing something wrong or I shouldn't be using Cassandra. Therefore, it might be the case I am doing something wrong. If you could help me to find out the answer for these questions by giving any feedback, it would be highly appreciated. Here is my need and what I am thinking in using Cassandra for: - I need to support a high volume of writes per second. I might have a billion writes per hour - I need to write non-structured data that will be processed later by hadoop processes to generate structured data from it. Later, I index the structured data using SOLR or SOLANDRA, so the data can be consulted by my end user application. Is Cassandra recommended for that, or should I be thinking in writting directly to HDFS files, for instance? What's the main advantage I get from storing data in a nosql service like Cassandra, when compared to storing files into HDFS? - Usually I will write json data associated to an ID and my hadoop processes will process this data to write data to a database. I have two doubts here: - If I don't need to perform complicated queries in Cassandra, should I store the json-like data just as a column value? I am afraid of doing something wrong here, as I would need just to store the json file and some more 5 or 6 fields to query the files later. - Does it make sense to you to use hadoop to process data from Cassandra and store the results in a database, like HBase? Once I have structured data, is there any reason I should use Cassandra instead of HBase? I am sorry if the questions are too dummy, I have been watching a lot of videos and reading a lot of documentation about Cassandra, but honestly, more I read more I have questions. Thanks in advance. Best regards, -- Marcelo Elias Del Valle http://mvalle.com - @mvallebr
