Re: What is 'Read Reuqests' on OpsCenter exaclty?
Let me try to reproduce your test and get back wiith some results. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo http://linkedin.com/in/carlosjuzarterolo* Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649 www.pythian.com On Fri, Apr 24, 2015 at 2:35 AM, Bongseo Jang grayce...@gmail.com wrote: Thanks a lot Carlos, Sebastian :-) My test was with 1 node/1 replica settings, on which I assumed client request = read request on the graph. Because there seems no read_repair and already CL=ONE in my case, I need more explanation, don't I? Or can any other internals be still involved? Do you have more suggestions? I want to design new test narrowing the gap on the suggestions. On 24 April 2015 at 00:23, Sebastian Estevez sebastian.este...@datastax.com wrote: Carlos is right: *Read Requests* - The number of read requests per second on the coordinator nodes, analogous to client reads. Monitoring the number of requests over a given time period reveals system read workload and usage patterns. *Avg* - The average of values recorded during a time interval. A future version of OpsC will include tooltips with these descriptions for better clarity. On Apr 23, 2015 6:30 AM, Carlos Rolo r...@pythian.com wrote: Probably it takes in account the read repair, plus a read that have consistency != 1 will produce reads on other machines (which are taken in account). I don't know the internals of opscenter but I would assume that this is the case. If you want to test it further, disable read_repair, and make all your reads with CL=ONE. Then your client and Opscenter should match. PS: Speculative_retry could also send reads over to more machines. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo http://linkedin.com/in/carlosjuzarterolo* Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649 www.pythian.com On Thu, Apr 23, 2015 at 10:34 AM, Bongseo Jang grayce...@gmail.com wrote: I have cassandra 2.1 + OpsCenter 5.1.1 and test them. When I monitored with opscenter 'read requests' graph, it seems the number on the graph is not what I expected, the number of client requests or responses. I recorded actual number of client request and compare it with graph, then found they're different. The number on the graph is about 4 times larger than what the client claimed. So, my question is what 'Read Reuqests' on OpsCenter counts exaclty ? Thanks ! -- Regards, Jang. a sound mind in a sound body -- -- Regards, Jang. a sound mind in a sound body -- --
Re: What is 'Read Reuqests' on OpsCenter exaclty?
You're quite right. I missed important thing first. I found a mistake in my program while making test case. It turns out that the original program has 3~4 selects for non-existing row keys plus a select for existing row key. It was intended to do nothing but for next tests. My original test counted only selects for existing row key but opscenter graph showed real number of request. That's it Thank you again ~ On 24 April 2015 at 16:01, Carlos Rolo r...@pythian.com wrote: Let me try to reproduce your test and get back wiith some results. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo http://linkedin.com/in/carlosjuzarterolo* Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649 www.pythian.com On Fri, Apr 24, 2015 at 2:35 AM, Bongseo Jang grayce...@gmail.com wrote: Thanks a lot Carlos, Sebastian :-) My test was with 1 node/1 replica settings, on which I assumed client request = read request on the graph. Because there seems no read_repair and already CL=ONE in my case, I need more explanation, don't I? Or can any other internals be still involved? Do you have more suggestions? I want to design new test narrowing the gap on the suggestions. On 24 April 2015 at 00:23, Sebastian Estevez sebastian.este...@datastax.com wrote: Carlos is right: *Read Requests* - The number of read requests per second on the coordinator nodes, analogous to client reads. Monitoring the number of requests over a given time period reveals system read workload and usage patterns. *Avg* - The average of values recorded during a time interval. A future version of OpsC will include tooltips with these descriptions for better clarity. On Apr 23, 2015 6:30 AM, Carlos Rolo r...@pythian.com wrote: Probably it takes in account the read repair, plus a read that have consistency != 1 will produce reads on other machines (which are taken in account). I don't know the internals of opscenter but I would assume that this is the case. If you want to test it further, disable read_repair, and make all your reads with CL=ONE. Then your client and Opscenter should match. PS: Speculative_retry could also send reads over to more machines. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo http://linkedin.com/in/carlosjuzarterolo* Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649 www.pythian.com On Thu, Apr 23, 2015 at 10:34 AM, Bongseo Jang grayce...@gmail.com wrote: I have cassandra 2.1 + OpsCenter 5.1.1 and test them. When I monitored with opscenter 'read requests' graph, it seems the number on the graph is not what I expected, the number of client requests or responses. I recorded actual number of client request and compare it with graph, then found they're different. The number on the graph is about 4 times larger than what the client claimed. So, my question is what 'Read Reuqests' on OpsCenter counts exaclty ? Thanks ! -- Regards, Jang. a sound mind in a sound body -- -- Regards, Jang. a sound mind in a sound body -- -- Regards, Jang. a sound mind in a sound body
Re: Creating 'Put' requests
There's also Achilles: https://github.com/doanduyhai/Achilles On Fri, Apr 24, 2015 at 1:21 PM Jens Rantil jens.ran...@tink.se wrote: Matthew, Maybe this could also be of interest: http://projects.spring.io/spring-data-cassandra/ Cheers, Jens On Fri, Apr 24, 2015 at 12:50 PM, Phil Yang ud1...@gmail.com wrote: 2015-04-23 22:16 GMT+08:00 Matthew Johnson matt.john...@algomi.com: In HBase, we do something like: Put put = new Put(id); put.add(myPojo.getTimestamp(), myPojo.getValue()); put.add(myPojo.getMySecondTimestamp(), myPojo.getSecondValue()); server.put(put); Is there any similar mechanism in Cassandra Java driver for creating these inserts programmatically? Or, can the 'session.execute' take a list of commands so that each column can be inserted as its own insert statement but without the overhead of multiple calls to the server? For your first question, do you mean object-mapping API? http://docs.datastax.com/en/developer/java-driver/2.1/java-driver/reference/crudOperations.html For the second question, C* can execute several commands by unlogged batch, however, because of the distributed nature of Cassandra, there is a better solution, see https://medium.com/@foundev/cassandra-batch-loading-without-the-batch-keyword-40f00e35e23e Thanks! Matt -Original Message- From: Jim Witschey [mailto:jim.witsc...@datastax.com] Sent: 23 April 2015 14:46 To: user@cassandra.apache.org Subject: Re: Creating 'Put' requests Are prepared statements what you're looking for? http://docs.datastax.com/en/developer/java-driver/2.1/java-driver/quick_start/qsSimpleClientBoundStatements_t.html Jim Witschey Software Engineer in Test | jim.witsc...@datastax.com On Thu, Apr 23, 2015 at 9:28 AM, Matthew Johnson matt.john...@algomi.com wrote: Hi all, Currently looking at switching from HBase to Cassandra, and one big difference so far is that in HBase, we create a ‘Put’ object, add to it a set of column/value pairs, and send the Put to the server. So far in Cassandra 2.1.4 the tutorials seem to suggest using CQL3, which I really like for prototyping eg: session.execute(INSERT INTO simplex.playlists (id, song_id, title, album, artist) VALUES (1,1,'La Petite Tonkinoise','Bye Bye Blackbird','Joséphine Baker');); But for more complicated code this will quickly become unmanageable, and doesn’t lend itself well to dynamically creating row data based on various conditions. Is there a way to send a Java object, populated with the desired column/value pairs, to the server instead of executing an insert statement? Would this require some other library, or does the DataStax Java driver support this already? Thanks in advance, Matt -- Thanks, Phil Yang -- Jens Rantil Backend engineer Tink AB Email: jens.ran...@tink.se Phone: +46 708 84 18 32 Web: www.tink.se Facebook https://www.facebook.com/#!/tink.se Linkedin http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_phototrkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary Twitter https://twitter.com/tink
Re: Creating 'Put' requests
To add to Phil's point, there's no circumstance in which I would use an unlogged batch, under load I have yet to hear it do anything other than increase GC pauses. On Fri, Apr 24, 2015 at 11:50 AM Phil Yang ud1...@gmail.com wrote: 2015-04-23 22:16 GMT+08:00 Matthew Johnson matt.john...@algomi.com: In HBase, we do something like: Put put = new Put(id); put.add(myPojo.getTimestamp(), myPojo.getValue()); put.add(myPojo.getMySecondTimestamp(), myPojo.getSecondValue()); server.put(put); Is there any similar mechanism in Cassandra Java driver for creating these inserts programmatically? Or, can the 'session.execute' take a list of commands so that each column can be inserted as its own insert statement but without the overhead of multiple calls to the server? For your first question, do you mean object-mapping API? http://docs.datastax.com/en/developer/java-driver/2.1/java-driver/reference/crudOperations.html For the second question, C* can execute several commands by unlogged batch, however, because of the distributed nature of Cassandra, there is a better solution, see https://medium.com/@foundev/cassandra-batch-loading-without-the-batch-keyword-40f00e35e23e Thanks! Matt -Original Message- From: Jim Witschey [mailto:jim.witsc...@datastax.com] Sent: 23 April 2015 14:46 To: user@cassandra.apache.org Subject: Re: Creating 'Put' requests Are prepared statements what you're looking for? http://docs.datastax.com/en/developer/java-driver/2.1/java-driver/quick_start/qsSimpleClientBoundStatements_t.html Jim Witschey Software Engineer in Test | jim.witsc...@datastax.com On Thu, Apr 23, 2015 at 9:28 AM, Matthew Johnson matt.john...@algomi.com wrote: Hi all, Currently looking at switching from HBase to Cassandra, and one big difference so far is that in HBase, we create a ‘Put’ object, add to it a set of column/value pairs, and send the Put to the server. So far in Cassandra 2.1.4 the tutorials seem to suggest using CQL3, which I really like for prototyping eg: session.execute(INSERT INTO simplex.playlists (id, song_id, title, album, artist) VALUES (1,1,'La Petite Tonkinoise','Bye Bye Blackbird','Joséphine Baker');); But for more complicated code this will quickly become unmanageable, and doesn’t lend itself well to dynamically creating row data based on various conditions. Is there a way to send a Java object, populated with the desired column/value pairs, to the server instead of executing an insert statement? Would this require some other library, or does the DataStax Java driver support this already? Thanks in advance, Matt -- Thanks, Phil Yang
RE: Creating 'Put' requests
The object-mapping API is very interesting, I’ll check that out, thanks. I believe I have found what I was looking for in terms of programmatically inserting data using the following syntax: * Insert builder = QueryBuilder.insertInto(**simplex**, * *mytable1**);* * builder = builder.value(**id**, **myid2**);* * builder = builder.value(**title**, **mytitle2**);* *session**.execute(builder);* Many thanks for all the valuable help so far! Cheers, Matt *From:* Jonathan Haddad [mailto:j...@jonhaddad.com] *Sent:* 24 April 2015 14:15 *To:* user@cassandra.apache.org *Subject:* Re: Creating 'Put' requests To add to Phil's point, there's no circumstance in which I would use an unlogged batch, under load I have yet to hear it do anything other than increase GC pauses. On Fri, Apr 24, 2015 at 11:50 AM Phil Yang ud1...@gmail.com wrote: 2015-04-23 22:16 GMT+08:00 Matthew Johnson matt.john...@algomi.com: In HBase, we do something like: Put put = new Put(id); put.add(myPojo.getTimestamp(), myPojo.getValue()); put.add(myPojo.getMySecondTimestamp(), myPojo.getSecondValue()); server.put(put); Is there any similar mechanism in Cassandra Java driver for creating these inserts programmatically? Or, can the 'session.execute' take a list of commands so that each column can be inserted as its own insert statement but without the overhead of multiple calls to the server? For your first question, do you mean object-mapping API? http://docs.datastax.com/en/developer/java-driver/2.1/java-driver/reference/crudOperations.html For the second question, C* can execute several commands by unlogged batch, however, because of the distributed nature of Cassandra, there is a better solution, see https://medium.com/@foundev/cassandra-batch-loading-without-the-batch-keyword-40f00e35e23e Thanks! Matt -Original Message- From: Jim Witschey [mailto:jim.witsc...@datastax.com] Sent: 23 April 2015 14:46 To: user@cassandra.apache.org Subject: Re: Creating 'Put' requests Are prepared statements what you're looking for? http://docs.datastax.com/en/developer/java-driver/2.1/java-driver/quick_start/qsSimpleClientBoundStatements_t.html Jim Witschey Software Engineer in Test | jim.witsc...@datastax.com On Thu, Apr 23, 2015 at 9:28 AM, Matthew Johnson matt.john...@algomi.com wrote: Hi all, Currently looking at switching from HBase to Cassandra, and one big difference so far is that in HBase, we create a ‘Put’ object, add to it a set of column/value pairs, and send the Put to the server. So far in Cassandra 2.1.4 the tutorials seem to suggest using CQL3, which I really like for prototyping eg: session.execute(INSERT INTO simplex.playlists (id, song_id, title, album, artist) VALUES (1,1,'La Petite Tonkinoise','Bye Bye Blackbird','Joséphine Baker');); But for more complicated code this will quickly become unmanageable, and doesn’t lend itself well to dynamically creating row data based on various conditions. Is there a way to send a Java object, populated with the desired column/value pairs, to the server instead of executing an insert statement? Would this require some other library, or does the DataStax Java driver support this already? Thanks in advance, Matt -- Thanks, Phil Yang
Re: Confirming Repairs
In 3.0, we have system table that stores repair history. https://issues.apache.org/jira/browse/CASSANDRA-5839 So you can just use CQL to check when given ks/cf is repaired. On Sat, Apr 25, 2015 at 5:23 AM, Jeff Ferland j...@tubularlabs.com wrote: The short answer is I used a logstash query to get a list of all repair ranges started and all ranges completed. I then matched the UUID of the start message to the end message and printed out all the ranges that didn't succeed. Then one needs to go a step further than I've coded and match the remaining ranges to at least one node in the ring that would hold a replica for the keyspace. Does anybody have a better way to handle this yet? Will the 3.0 series logging of repairs to the system keyspace be able to give me this same kind of confirmation that everything in a given keyspace was last repaired as of $DATE, or that to repair everything as of $DATE I must repair ranges $X? https://gist.github.com/autocracy/9467eaaff581ff24334c -- Yuki Morishita t:yukim (http://twitter.com/yukim)
