Re: Upgrade 1.1.2 - 1.1.6
This looks like the counters were more out of sync before the upgrade than after? My guess is the update makes some counters over-count since I saw the value of the sum of our daily counter increase by 2000 after each restart at the exact moment that the node is marked as being up. This counter was about 5000, after upgrading the first node 7000, after the second upgrade 9000 and finally 11000. This value shouldn't have increase since we had stopped our storm topology and we were queuing events without writing directly into C*. Do you know if your client is retrying counter operations ? (I saw some dropped messages in the S1 log). I am using phpCassa written by Tyler Hobbs. I have a pool configured like this: $max_retries = 4 $send_timeout = 2000 $recv_timeout = 2000 $pool_size = NULL //max(5, count($servers) * 2) $pool = new ConnectionPool($base, $servers, $pool_size, $max_retries, $send_timeout, $recv_timeout); So I guess my client is retrying count operations like any other operation, 4 times max with 2 sec time out. S1 shows a lot of Commit Log replay going on. Reading your timeline below this sounds like the auto restart catching you out. Is there a way to remove this auto-restart while upgrading from the Datastax repository on Ubuntu ? Let me know if you need something more to understand what happened. Alain
Re: Cassandra nodes failing with OOM
Something that bit us recently was the size of bloom filters: we have a column family which is mostly written to, and only read sequentially, so we were able to free a lot of memory and decrease GC pressure by increasing bloom_filter_fp_chance for that particular CF. This on 1.0.12. /Janne On 18 Nov 2012, at 21:38, aaron morton wrote: 1. How much GCInspector warnings per hour are considered 'normal'? None. A couple during compaction or repair is not the end of the world. But if you have enough to thinking about per hour it's too many. 2. What should be the next thing to check? Try to determine if the GC activity correlates to application workload, compaction or repair. Try to determine what the working set of the server is. Watch the GC activity (via gc logs or JMX) and see what the size of the tenured heap is after a CMS. Or try to calculate it http://www.mail-archive.com/user@cassandra.apache.org/msg25762.html Look at your data model and query patterns for places where very large queries are being made. Or rows that are very long lived with a lot of deletes (prob not as much as an issue with LDB). 3. What are the possible failure reasons and how to prevent those? As above. As a work around sometimes drastically slowing down compaction can help. For LDB try reducing in_memory_compaction_limit_in_mb and compaction_throughput_mb_per_sec Hope that helps. - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 17/11/2012, at 7:07 PM, Ивaн Cобoлeв sobol...@gmail.com wrote: Dear Community, advice from you needed. We have a cluster, 1/6 nodes of which died for various reasons(3 had OOM message). Nodes died in groups of 3, 1, 2. No adjacent died, though we use SimpleSnitch. Version: 1.1.6 Hardware: 12Gb RAM / 8 cores(virtual) Data: 40Gb/node Nodes: 36 nodes Keyspaces:2(RF=3, R=W=2) + 1(OpsCenter) CFs:36, 2 indexes Partitioner: Random Compaction: Leveled(we don't want 2x space for housekeeping) Caching: Keys only All is pretty much standard apart from the one CF receiving writes in 64K chunks and having sstable_size_in_mb=100. No JNA installed - this is to be fixed soon. Checking sysstat/sar I can see 80-90% CPU idle, no anomalies in io and the only change - network activity spiking. All the nodes before dying had the following on logs: INFO [ScheduledTasks:1] 2012-11-15 21:35:05,512 StatusLogger.java (line 72) MemtablePostFlusher 1 4 0 INFO [ScheduledTasks:1] 2012-11-15 21:35:13,540 StatusLogger.java (line 72) FlushWriter 1 3 0 INFO [ScheduledTasks:1] 2012-11-15 21:36:32,162 StatusLogger.java (line 72) HintedHandoff 1 6 0 INFO [ScheduledTasks:1] 2012-11-15 21:36:32,162 StatusLogger.java (line 77) CompactionManager 5 9 GCInspector warnings were there too, they went from ~0.8 to 3Gb heap in 5-10mins. So, could you please give me a hint on: 1. How much GCInspector warnings per hour are considered 'normal'? 2. What should be the next thing to check? 3. What are the possible failure reasons and how to prevent those? Thank you very much in advance, Ivan
Re: Remove crashed node
Quite some time passed, but Alain, it worked. Thank you very much. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl http://goo.gl/Lt7BC Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. On Fri, Nov 9, 2012 at 4:21 PM, Alain RODRIGUEZ arodr...@gmail.com wrote: You have to install mx4j-tools.jar. http://wiki.apache.org/cassandra/Operations#Monitoring_with_MX4J It's a java tool, so it is usable on both windows and linux. Here is the link to dl mx4j-tool.jar : http://www.java2s.com/Code/JarDownload/mx4j/mx4j-tools-3.0.2.jar.zip unzip it and add it to the path of your cassandra libraries, restart the node where it is installed and you should be ok. Alain 2012/11/9 Robin Verlangen ro...@us2.nl Hi Alain, How can I access that? Web browser does not seem to work. Do I need any software to login? If so, what is proper software for Windows? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl http://goo.gl/Lt7BC Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. On Fri, Nov 9, 2012 at 2:50 PM, Alain RODRIGUEZ arodr...@gmail.comwrote: On cassandra-cli if you describe cluster; I guess you will see an UNREACHABLE node. If you do so, there is a way to remove this unreachable node. Go to the JMX management console (ip_of_one_up_node:8081 by default) Then go to the org.apache.cassandra.net:type=Gossiper link and use the unsafeAssassinateEndpoint input. Fill it with the ip of the down node and invoke the function. nodetool gossipinfo should now tell you that this node has left the ring and let you truncate or whatever you need to do. Use this carefully this function is composed of the unsafe and assassinate words because it forces the node to go out of the ring without any check or replication. Alain 2012/11/9 Robin Verlangen ro...@us2.nl Hi there, We have had a crashed node that is currently removed from the rack. However when I try a schema upgrade / truncate operation it complains of the unreachable node. I tried the removetoken, but that didn't resolve. Any ideas on how to fix this? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl http://goo.gl/Lt7BC Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.
Re: SchemaDisagreementException
I was using cassandra direct because it has more performace than using CQL.
Therefore, I am using cassandra because of replication factor and
consistence of data. I am using it as a lib of my app. I only make sample
querys, just use a key to point to a data.
2012/11/16 Everton Lima peitin.inu...@gmail.com
I do that because I need to create a dynamic column families.
I create 2 keyspaces in the start of application, using embedded
cassandra instance too, but it's never throw exception. And than, insert
dynamic column families in this 2 keyspaces.
I put a Thread.sleep(3000); in the middle of the creation column family
code.
int watiTime = 3000;
logger.info(Waiting +(watiTime/1000)+s for synchronizing
...);
Thread.sleep(watiTime);
CassandraHelper.createColumnFamily(CassandraHelper.KEYSPACE,
layer);
logger.info(Waiting +(watiTime/1000)+s for synchronizing
...);
Thread.sleep(watiTime);
I do that, because in the code of CassandraStress, after create a column
family, it do that too. It is wrong or good solution?
Any other idea?
2012/11/14 aaron morton aa...@thelastpickle.com
Out of interest why are you creating column families by making direct
calls on an embedded cassandra instance ? I would guess you life would be
easier if you defined a schema in CQL or CLI.
I already read in the documentation that this error occurs when more than
one thread/processor access the same place in the Cassandra, but I think
this is not occuring.
How may nodes do you have ?
I am using 3 nodes.
What version are you running ?
The version is 1.1.6
It sounds like you have run simultaneous schema updates and the global
schema has diverged.
If you can create your schema in CLI or CQL I would recommend doing that.
If you are trying to do something more complicated you'll need to provide
more information.
Cheers
-
Aaron Morton
Freelance Cassandra Developer
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 15/11/2012, at 3:13 AM, Everton Lima peitin.inu...@gmail.com wrote:
Some times, when I try to insert a data in Cassandra with Method:
static void createColumnFamily(String keySpace, String columnFamily){
synchronized (mutex){
Iface cs = new CassandraServer();
CfDef cfDef = new CfDef(keySpace, columnFamily);
cfDef = cfDef.setComparator_type(comparator.toString());
cfDef = cfDef.setKey_validation_class(UTF8Type.class.getName());
cfDef = cfDef.setCaching(all);
cs.set_keyspace(keySpace);
cs.system_add_column_family(cfDef);
}
}
I run this method in only on process of the distributor system. But in
this process, I run the method two times consecutive in the same thread,
but with keyspace diferent.
I already read in the documentation that this error occurs when more than
one thread/processor access the same place in the Cassandra, but I think
this is not occuring.
Some one cold help me to understand this problem?
Thanks.
--
Everton Lima Aleixo
Bacharel em Ciencia da Computação
Universidade Federal de Goiás
--
Everton Lima Aleixo
Bacharel em Ciencia da Computação
Universidade Federal de Goiás
--
Everton Lima Aleixo
Bacharel em Ciencia da Computação
Universidade Federal de Goiás
RE: Cassandra nodes failing with OOM
We've seen OOM in a situation, when OS was not properly prepared in production. http://www.datastax.com/docs/1.1/install/recommended_settings Best regards / Pagarbiai Viktor Jevdokimov Senior Developer Email: viktor.jevdoki...@adform.commailto:viktor.jevdoki...@adform.com Phone: +370 5 212 3063, Fax +370 5 261 0453 J. Jasinskio 16C, LT-01112 Vilnius, Lithuania Follow us on Twitter: @adforminsiderhttp://twitter.com/#!/adforminsider Take a ride with Adform's Rich Media Suitehttp://vimeo.com/adform/richmedia [Adform News] http://www.adform.com Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. From: some.unique.lo...@gmail.com [mailto:some.unique.lo...@gmail.com] On Behalf Of Ивaн Cобoлeв Sent: Saturday, November 17, 2012 08:08 To: user@cassandra.apache.org Subject: Cassandra nodes failing with OOM Dear Community, advice from you needed. We have a cluster, 1/6 nodes of which died for various reasons(3 had OOM message). Nodes died in groups of 3, 1, 2. No adjacent died, though we use SimpleSnitch. Version: 1.1.6 Hardware: 12Gb RAM / 8 cores(virtual) Data: 40Gb/node Nodes: 36 nodes Keyspaces:2(RF=3, R=W=2) + 1(OpsCenter) CFs:36, 2 indexes Partitioner: Random Compaction: Leveled(we don't want 2x space for housekeeping) Caching: Keys only All is pretty much standard apart from the one CF receiving writes in 64K chunks and having sstable_size_in_mb=100. No JNA installed - this is to be fixed soon. Checking sysstat/sar I can see 80-90% CPU idle, no anomalies in io and the only change - network activity spiking. All the nodes before dying had the following on logs: INFO [ScheduledTasks:1] 2012-11-15 21:35:05,512 StatusLogger.java (line 72) MemtablePostFlusher 1 4 0 INFO [ScheduledTasks:1] 2012-11-15 21:35:13,540 StatusLogger.java (line 72) FlushWriter 1 3 0 INFO [ScheduledTasks:1] 2012-11-15 21:36:32,162 StatusLogger.java (line 72) HintedHandoff 1 6 0 INFO [ScheduledTasks:1] 2012-11-15 21:36:32,162 StatusLogger.java (line 77) CompactionManager 5 9 GCInspector warnings were there too, they went from ~0.8 to 3Gb heap in 5-10mins. So, could you please give me a hint on: 1. How much GCInspector warnings per hour are considered 'normal'? 2. What should be the next thing to check? 3. What are the possible failure reasons and how to prevent those? Thank you very much in advance, Ivan inline: signature-logo49d2.png
Re: Remove crashed node
I am glad to here that. You're very welcome. Alain 2012/11/19 Robin Verlangen ro...@us2.nl Quite some time passed, but Alain, it worked. Thank you very much. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl http://goo.gl/Lt7BC Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. On Fri, Nov 9, 2012 at 4:21 PM, Alain RODRIGUEZ arodr...@gmail.comwrote: You have to install mx4j-tools.jar. http://wiki.apache.org/cassandra/Operations#Monitoring_with_MX4J It's a java tool, so it is usable on both windows and linux. Here is the link to dl mx4j-tool.jar : http://www.java2s.com/Code/JarDownload/mx4j/mx4j-tools-3.0.2.jar.zip unzip it and add it to the path of your cassandra libraries, restart the node where it is installed and you should be ok. Alain 2012/11/9 Robin Verlangen ro...@us2.nl Hi Alain, How can I access that? Web browser does not seem to work. Do I need any software to login? If so, what is proper software for Windows? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl http://goo.gl/Lt7BC Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. On Fri, Nov 9, 2012 at 2:50 PM, Alain RODRIGUEZ arodr...@gmail.comwrote: On cassandra-cli if you describe cluster; I guess you will see an UNREACHABLE node. If you do so, there is a way to remove this unreachable node. Go to the JMX management console (ip_of_one_up_node:8081 by default) Then go to the org.apache.cassandra.net:type=Gossiper link and use the unsafeAssassinateEndpoint input. Fill it with the ip of the down node and invoke the function. nodetool gossipinfo should now tell you that this node has left the ring and let you truncate or whatever you need to do. Use this carefully this function is composed of the unsafe and assassinate words because it forces the node to go out of the ring without any check or replication. Alain 2012/11/9 Robin Verlangen ro...@us2.nl Hi there, We have had a crashed node that is currently removed from the rack. However when I try a schema upgrade / truncate operation it complains of the unreachable node. I tried the removetoken, but that didn't resolve. Any ideas on how to fix this? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl http://goo.gl/Lt7BC Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.
Re: Datatype Conversion in CQL-Client?
I don't think Michael and/or Jonathan have published the CQL java driver yet. (CCing them) Hopefully they'll find a public home for it soon, I hope to include it in the Webinar in December. (http://www.datastax.com/resources/webinars/collegecredit) -brian --- Brian O'Neill Lead Architect, Software Development Health Market Science The Science of Better Results 2700 Horizon Drive King of Prussia, PA 19406 M: 215.588.6024 @boneill42 http://www.twitter.com/boneill42 healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. From: Tommi Laukkanen tlaukka...@gmail.com Reply-To: user@cassandra.apache.org Date: Monday, November 19, 2012 2:36 AM To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? I think Timmy might be referring to the upcoming native CQL Java driver that might be coming with 1.2 - It was mentioned here: http://www.datastax.com/wp-content/uploads/2012/08/7_Datastax_Upcoming_Chang es_in_Drivers.pdf I would also be interested on testing that but I can't find it from repositories. Any hints? Regards, Tommi L. From: Brian O'Neill [mailto:boneil...@gmail.com] On Behalf Of Brian O'Neill Sent: 18. marraskuuta 2012 17:47 To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? Importance: Low If you are talking about the CQL-client that comes with Cassandra (cqlsh), it is actually written in Python: https://github.com/apache/cassandra/blob/trunk/bin/cqlsh For information on datatypes (and conversion) take a look at the CQL definition: http://www.datastax.com/docs/1.0/references/cql/index (Look at the CQL Data Types section) If that's not the client you are referencing, let us know which one you mean: http://brianoneill.blogspot.com/2012/08/cassandra-apis-laundry-list.html -brian On Nov 17, 2012, at 9:54 PM, Timmy Turner wrote: Thanks for the links, however I'm interested in the functionality that the official Cassandra client/API (which is in Java) offers. 2012/11/17 aaron morton aa...@thelastpickle.com Does the official/built-in Cassandra CQL client (in 1.2) What language ? Check the Java http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/ and python http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/ drivers. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com http://www.thelastpickle.com/ On 16/11/2012, at 11:21 AM, Timmy Turner timm.t...@gmail.com wrote: Does the official/built-in Cassandra CQL client (in 1.2) offer any built-in option to get direct values/objects when reading a field, instead of just a byte array? -- Brian ONeill Lead Architect, Health Market Science (http://healthmarketscience.com http://healthmarketscience.com/ ) mobile:215.588.6024 tel:215.588.6024 blog: http://weblogs.java.net/blog/boneill42/ blog: http://brianoneill.blogspot.com/
Re: Datatype Conversion in CQL-Client?
What I meant was the method that the Cassandra-jars give you when you include them in your project: TTransport tr = new TFramedTransport(new TSocket(localhost, 9160)); TProtocol proto = new TBinaryProtocol(tr); Cassandra.Client client = new Cassandra.Client(proto); tr.open(); client.execute_cql_query(ByteBuffer.wrap(cql.getBytes()), Compression.NONE); 2012/11/19 Brian O'Neill b...@alumni.brown.edu I don't think Michael and/or Jonathan have published the CQL java driver yet. (CCing them) Hopefully they'll find a public home for it soon, I hope to include it in the Webinar in December. (http://www.datastax.com/resources/webinars/collegecredit) -brian --- Brian O'Neill Lead Architect, Software Development *Health Market Science* *The Science of Better Results* 2700 Horizon Drive • King of Prussia, PA • 19406 M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 • healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. ** ** From: Tommi Laukkanen tlaukka...@gmail.com Reply-To: user@cassandra.apache.org Date: Monday, November 19, 2012 2:36 AM To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? I think Timmy might be referring to the upcoming native CQL Java driver that might be coming with 1.2 - It was mentioned here: http://www.datastax.com/wp-content/uploads/2012/08/7_Datastax_Upcoming_Changes_in_Drivers.pdf I would also be interested on testing that but I can't find it from repositories. Any hints? Regards, Tommi L. *From:* Brian O'Neill [mailto:boneil...@gmail.com] *On Behalf Of *Brian O'Neill *Sent:* 18. marraskuuta 2012 17:47 *To:* user@cassandra.apache.org *Subject:* Re: Datatype Conversion in CQL-Client? *Importance:* Low ** ** ** ** If you are talking about the CQL-client that comes with Cassandra (cqlsh), it is actually written in Python: https://github.com/apache/cassandra/blob/trunk/bin/cqlsh ** ** For information on datatypes (and conversion) take a look at the CQL definition: http://www.datastax.com/docs/1.0/references/cql/index (Look at the CQL Data Types section) ** ** If that's not the client you are referencing, let us know which one you mean: http://brianoneill.blogspot.com/2012/08/cassandra-apis-laundry-list.html* *** ** ** -brian ** ** On Nov 17, 2012, at 9:54 PM, Timmy Turner wrote: Thanks for the links, however I'm interested in the functionality that the official Cassandra client/API (which is in Java) offers. ** ** 2012/11/17 aaron morton aa...@thelastpickle.com Does the official/built-in Cassandra CQL client (in 1.2) What language ? ** ** Check the Java http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/ and python http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/ drivers.** ** ** ** Cheers ** ** ** ** - Aaron Morton Freelance Cassandra Developer New Zealand ** ** @aaronmorton http://www.thelastpickle.com ** ** On 16/11/2012, at 11:21 AM, Timmy Turner timm.t...@gmail.com wrote: Does the official/built-in Cassandra CQL client (in 1.2) offer any built-in option to get direct values/objects when reading a field, instead of just a byte array? ** ** ** ** ** ** -- Brian ONeill Lead Architect, Health Market Science (http://healthmarketscience.com) mobile:215.588.6024 blog: http://weblogs.java.net/blog/boneill42/ blog: http://brianoneill.blogspot.com/ ** **
Re: Collections, query for contains?
This was my first question after I git the inserts working. Hive has udfs
like array contains. It also has lateral view syntax that is similar to
transposed.
On Monday, November 19, 2012, Timmy Turner timm.t...@gmail.com wrote:
Is there no option to query for the contents of a collection?
Something like
select * from cf where c_list contains('some_value')
or
select * from cf where c_map contains('some_key')
or
select * from cf where c_map['some_key'] contains('some_value')
Re: SchemaDisagreementException
even if you made the calls through cql you would have the same issue since
cql uses thrift. 1.2:0 is supposed to be nicer with concurrent
modifications.
On Monday, November 19, 2012, Everton Lima peitin.inu...@gmail.com wrote:
I was using cassandra direct because it has more performace than using
CQL. Therefore, I am using cassandra because of replication factor and
consistence of data. I am using it as a lib of my app. I only make sample
querys, just use a key to point to a data.
2012/11/16 Everton Lima peitin.inu...@gmail.com
I do that because I need to create a dynamic column families.
I create 2 keyspaces in the start of application, using embedded
cassandra instance too, but it's never throw exception. And than, insert
dynamic column families in this 2 keyspaces.
I put a Thread.sleep(3000); in the middle of the creation column family
code.
int watiTime = 3000;
logger.info(Waiting +(watiTime/1000)+s for synchronizing
...);
Thread.sleep(watiTime);
CassandraHelper.createColumnFamily(CassandraHelper.KEYSPACE,
layer);
logger.info(Waiting +(watiTime/1000)+s for synchronizing
...);
Thread.sleep(watiTime);
I do that, because in the code of CassandraStress, after create a column
family, it do that too. It is wrong or good solution?
Any other idea?
2012/11/14 aaron morton aa...@thelastpickle.com
Out of interest why are you creating column families by making direct
calls on an embedded cassandra instance ? I would guess you life would be
easier if you defined a schema in CQL or CLI.
I already read in the documentation that this error occurs when more than
one thread/processor access the same place in the Cassandra, but I think
this is not occuring.
How may nodes do you have ?
I am using 3 nodes.
What version are you running ?
The version is 1.1.6
It sounds like you have run simultaneous schema updates and the global
schema has diverged.
If you can create your schema in CLI or CQL I would recommend doing that.
If you are trying to do something more complicated you'll need to provide
more information.
Cheers
-
Aaron Morton
Freelance Cassandra Developer
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 15/11/2012, at 3:13 AM, Everton Lima peitin.inu...@gmail.com wrote:
Some times, when I try to insert a data in Cassandra with Method:
static void createColumnFamily(String keySpace, String columnFamily){
synchronized (mutex){
Iface cs = new CassandraServer();
CfDef cfDef = new CfDef(keySpace, colu
Re: Datatype Conversion in CQL-Client?
Gotcha Timmy. That is the Thrift API. You are operating at a pretty low-level. I'm not sure that is considered the official CQL client. IMHO, you might be better off moving up a level. I'd probably either wait for the official CQL Java Driver, or access CQL via a higher-level client like Hector. If you stick with Thrift, I think you can access the Schema metadata: https://github.com/apache/cassandra/blob/trunk/interface/thrift/gen-java/org /apache/cassandra/thrift/CqlMetadata.java (Those are the generated classes for the Thrift interface) But I'm not sure where the code is to apply that metadata to the result set in: https://github.com/apache/cassandra/blob/trunk/interface/thrift/gen-java/org /apache/cassandra/thrift/CqlResult.java -brian --- Brian O'Neill Lead Architect, Software Development Health Market Science The Science of Better Results 2700 Horizon Drive King of Prussia, PA 19406 M: 215.588.6024 @boneill42 http://www.twitter.com/boneill42 healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. From: Timmy Turner timm.t...@gmail.com Reply-To: user@cassandra.apache.org Date: Monday, November 19, 2012 9:48 AM To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? What I meant was the method that the Cassandra-jars give you when you include them in your project: TTransport tr = new TFramedTransport(new TSocket(localhost, 9160)); TProtocol proto = new TBinaryProtocol(tr); Cassandra.Client client = new Cassandra.Client(proto); tr.open(); client.execute_cql_query(ByteBuffer.wrap(cql.getBytes()), Compression.NONE); 2012/11/19 Brian O'Neill b...@alumni.brown.edu I don't think Michael and/or Jonathan have published the CQL java driver yet. (CCing them) Hopefully they'll find a public home for it soon, I hope to include it in the Webinar in December. (http://www.datastax.com/resources/webinars/collegecredit) -brian --- Brian O'Neill Lead Architect, Software Development Health Market Science The Science of Better Results 2700 Horizon Drive King of Prussia, PA 19406 M: 215.588.6024 tel:215.588.6024 @boneill42 http://www.twitter.com/boneill42 healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. From: Tommi Laukkanen tlaukka...@gmail.com Reply-To: user@cassandra.apache.org Date: Monday, November 19, 2012 2:36 AM To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? I think Timmy might be referring to the upcoming native CQL Java driver that might be coming with 1.2 - It was mentioned here: http://www.datastax.com/wp-content/uploads/2012/08/7_Datastax_Upcoming_Changes _in_Drivers.pdf I would also be interested on testing that but I can't find it from repositories. Any hints? Regards, Tommi L. From: Brian O'Neill [mailto:boneil...@gmail.com] On Behalf Of Brian O'Neill Sent: 18. marraskuuta 2012 17:47 To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? Importance: Low If you are talking about the CQL-client that comes with Cassandra (cqlsh), it is actually written in Python: https://github.com/apache/cassandra/blob/trunk/bin/cqlsh For information on datatypes (and conversion) take a look at the CQL definition: http://www.datastax.com/docs/1.0/references/cql/index (Look at the CQL Data Types section) If that's not the client you are referencing, let us know which one you mean: http://brianoneill.blogspot.com/2012/08/cassandra-apis-laundry-list.html -brian On Nov 17, 2012, at 9:54 PM, Timmy Turner wrote: Thanks for the links, however I'm interested in the functionality that the official Cassandra client/API (which is in Java) offers. 2012/11/17 aaron morton aa...@thelastpickle.com Does the official/built-in Cassandra
Re: Datatype Conversion in CQL-Client?
You might want to take look a org.apache.cassandra.transport.SimpleClient and org.apache.cassandra.transport.messages.ResultMessage. On Mon, Nov 19, 2012 at 9:48 AM, Timmy Turner timm.t...@gmail.com wrote: What I meant was the method that the Cassandra-jars give you when you include them in your project: TTransport tr = new TFramedTransport(new TSocket(localhost, 9160)); TProtocol proto = new TBinaryProtocol(tr); Cassandra.Client client = new Cassandra.Client(proto); tr.open(); client.execute_cql_query(ByteBuffer.wrap(cql.getBytes()), Compression.NONE); 2012/11/19 Brian O'Neill b...@alumni.brown.edu I don't think Michael and/or Jonathan have published the CQL java driver yet. (CCing them) Hopefully they'll find a public home for it soon, I hope to include it in the Webinar in December. (http://www.datastax.com/resources/webinars/collegecredit) -brian --- Brian O'Neill Lead Architect, Software Development *Health Market Science* *The Science of Better Results* 2700 Horizon Drive • King of Prussia, PA • 19406 M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 • healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. ** ** From: Tommi Laukkanen tlaukka...@gmail.com Reply-To: user@cassandra.apache.org Date: Monday, November 19, 2012 2:36 AM To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? I think Timmy might be referring to the upcoming native CQL Java driver that might be coming with 1.2 - It was mentioned here: http://www.datastax.com/wp-content/uploads/2012/08/7_Datastax_Upcoming_Changes_in_Drivers.pdf I would also be interested on testing that but I can't find it from repositories. Any hints? Regards, Tommi L. *From:* Brian O'Neill [mailto:boneil...@gmail.com] *On Behalf Of *Brian O'Neill *Sent:* 18. marraskuuta 2012 17:47 *To:* user@cassandra.apache.org *Subject:* Re: Datatype Conversion in CQL-Client? *Importance:* Low ** ** ** ** If you are talking about the CQL-client that comes with Cassandra (cqlsh), it is actually written in Python: https://github.com/apache/cassandra/blob/trunk/bin/cqlsh ** ** For information on datatypes (and conversion) take a look at the CQL definition: http://www.datastax.com/docs/1.0/references/cql/index (Look at the CQL Data Types section) ** ** If that's not the client you are referencing, let us know which one you mean: http://brianoneill.blogspot.com/2012/08/cassandra-apis-laundry-list.html ** ** -brian ** ** On Nov 17, 2012, at 9:54 PM, Timmy Turner wrote: Thanks for the links, however I'm interested in the functionality that the official Cassandra client/API (which is in Java) offers. ** ** 2012/11/17 aaron morton aa...@thelastpickle.com Does the official/built-in Cassandra CQL client (in 1.2) What language ? ** ** Check the Java http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/ and python http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/ drivers.* *** ** ** Cheers ** ** ** ** - Aaron Morton Freelance Cassandra Developer New Zealand ** ** @aaronmorton http://www.thelastpickle.com ** ** On 16/11/2012, at 11:21 AM, Timmy Turner timm.t...@gmail.com wrote:*** * Does the official/built-in Cassandra CQL client (in 1.2) offer any built-in option to get direct values/objects when reading a field, instead of just a byte array? ** ** ** ** ** ** -- Brian ONeill Lead Architect, Health Market Science (http://healthmarketscience.com) mobile:215.588.6024 blog: http://weblogs.java.net/blog/boneill42/ blog: http://brianoneill.blogspot.com/ ** ** -- - John
Re: Collections, query for contains?
It's not supported yet, no, but we have a ticket for it:
https://issues.apache.org/jira/browse/CASSANDRA-4511
On Mon, Nov 19, 2012 at 3:56 PM, Edward Capriolo edlinuxg...@gmail.comwrote:
This was my first question after I git the inserts working. Hive has udfs
like array contains. It also has lateral view syntax that is similar to
transposed.
On Monday, November 19, 2012, Timmy Turner timm.t...@gmail.com wrote:
Is there no option to query for the contents of a collection?
Something like
select * from cf where c_list contains('some_value')
or
select * from cf where c_map contains('some_key')
or
select * from cf where c_map['some_key'] contains('some_value')
Datastax Java Driver
Everyone, We've just open-sourced a new Java driver we have been working on here at DataStax. This driver is CQL3 only and is built to use the new binary protocol that will be introduced with Cassandra 1.2. It will thus only work with Cassandra 1.2 onwards. Currently, it means that testing it requires 1.2.0-beta2. This is also alpha software at this point. You are welcome to try and play with it and we would very much welcome feedback, but be sure that break, it will. The driver is accessible at: http://github.com/datastax/java-driver Today we're open-sourcing the core part of this driver. This main goal of this core module is to handle connections to the Cassandra cluster with all the features that one would expect. The currently supported features are: - Asynchronous: the driver uses the new CQL binary protocol asynchronous capabilities. - Nodes discovery. - Configurable load balancing/routing. - Transparent fail-over. - C* tracing handling. - Convenient schema access. - Configurable retry policy. This core module provides a simple low-level API (that works directly with query strings). We plan to release a higher-level, thin object mapping API based on top of this core shortly. Please refer to the project README for more information. -- The DataStax Team
Re: Datastax Java Driver
Woohoo! Thanks for making this available. --- Brian O'Neill Lead Architect, Software Development Health Market Science The Science of Better Results 2700 Horizon Drive King of Prussia, PA 19406 M: 215.588.6024 @boneill42 http://www.twitter.com/boneill42 healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. From: Sylvain Lebresne sylv...@datastax.com Reply-To: user@cassandra.apache.org Date: Monday, November 19, 2012 1:50 PM To: user@cassandra.apache.org user@cassandra.apache.org Subject: Datastax Java Driver Everyone, We've just open-sourced a new Java driver we have been working on here at DataStax. This driver is CQL3 only and is built to use the new binary protocol that will be introduced with Cassandra 1.2. It will thus only work with Cassandra 1.2 onwards. Currently, it means that testing it requires 1.2.0-beta2. This is also alpha software at this point. You are welcome to try and play with it and we would very much welcome feedback, but be sure that break, it will. The driver is accessible at: http://github.com/datastax/java-driver Today we're open-sourcing the core part of this driver. This main goal of this core module is to handle connections to the Cassandra cluster with all the features that one would expect. The currently supported features are: - Asynchronous: the driver uses the new CQL binary protocol asynchronous capabilities. - Nodes discovery. - Configurable load balancing/routing. - Transparent fail-over. - C* tracing handling. - Convenient schema access. - Configurable retry policy. This core module provides a simple low-level API (that works directly with query strings). We plan to release a higher-level, thin object mapping API based on top of this core shortly. Please refer to the project README for more information. -- The DataStax Team
RE: SchemaDisagreementException
For some reason I can not reply my old thread in that list. So I was creating a new one. The problem is that I do not use thrift to gain in performace. Why it is nicer with concurrent modifications? I do not know why I have falling in the problem of concurrent modification if I was creating 2 keyspaces diferent in only one process with just one thread. Someone knows why? -- Everton Lima Aleixo Bacharel em Ciencia da Computação Universidade Federal de Goiás
Re: Datastax Java Driver
Fantastic! As for the object mapping API, has there been any discussion/consideration of http://www.hibernate.org/subprojects/ogm.html? On Mon, Nov 19, 2012 at 1:50 PM, Sylvain Lebresne sylv...@datastax.comwrote: Everyone, We've just open-sourced a new Java driver we have been working on here at DataStax. This driver is CQL3 only and is built to use the new binary protocol that will be introduced with Cassandra 1.2. It will thus only work with Cassandra 1.2 onwards. Currently, it means that testing it requires 1.2.0-beta2. This is also alpha software at this point. You are welcome to try and play with it and we would very much welcome feedback, but be sure that break, it will. The driver is accessible at: http://github.com/datastax/java-driver Today we're open-sourcing the core part of this driver. This main goal of this core module is to handle connections to the Cassandra cluster with all the features that one would expect. The currently supported features are: - Asynchronous: the driver uses the new CQL binary protocol asynchronous capabilities. - Nodes discovery. - Configurable load balancing/routing. - Transparent fail-over. - C* tracing handling. - Convenient schema access. - Configurable retry policy. This core module provides a simple low-level API (that works directly with query strings). We plan to release a higher-level, thin object mapping API based on top of this core shortly. Please refer to the project README for more information. -- The DataStax Team -- - John
Re: Upgrade 1.1.2 - 1.1.6
On Thu, Nov 15, 2012 at 6:21 AM, Alain RODRIGUEZ arodr...@gmail.com wrote: We had an issue with counters over-counting even using the nodetool drain command before upgrading... You're sure the over-count was caused by the upgrade?Counts can be counted on (heh) to overcount. What is the scale of the over-count? Also, do you realize that drain doesn't actually prevent over-replay, though it should? https://issues.apache.org/jira/browse/CASSANDRA-4446 Check for replayed operations in the system log.. grep -i replay /path/to/system.log (I see further down thread that your logs do in fact contain replay. Please comment to this effect, and how negative the experience is for counts, on 4446?) I saw that the sudo apt-get install cassandra stop the server and restart it automatically. So it updated without draining and restart before I had the time to reconfigure the conf files. Is this normal ? Is there a way to avoid it ? https://issues.apache.org/jira/browse/CASSANDRA-2356 Perhaps if you comment on here you can help convince the debian packager that auto-starting a distributed database when you install or upgrade its package has negative operational characteristics. After both of these updates I saw my current counters increase without any reason. Did I do anything wrong ? Expecting counters to not over-count may qualify as wrong. But your process seems reasonable. =Rob -- =Robert Coli AIMGTALK - rc...@palominodb.com YAHOO - rcoli.palominob SKYPE - rcoli_palominodb
Re: Query regarding SSTable timestamps and counts
On Sun, Nov 18, 2012 at 7:57 PM, Ananth Gundabattula agundabatt...@gmail.com wrote: As per the above url, After running a major compaction, automatic minor compactions are no longer triggered, frequently requiring you to manually run major compactions on a routine basis. ( Just before the heading Tuning Column Family compression in the above link) This inaccurate statement has been questioned a few times on the mailing list. Generally what happens is people discuss it for about 10 emails and then give up because they can't really make sense of it. If you google for cassandra-user and that sentence above, you should find the threads. I suggest mailing d...@datastax.com, explaining your confusion, and asking them to fix it. =Rob -- =Robert Coli AIMGTALK - rc...@palominodb.com YAHOO - rcoli.palominob SKYPE - rcoli_palominodb
Re: SchemaDisagreementException
Have you actually tested to see that the Thrift API is more performant than CQL for your application? As far as I know, CQL almost always has a performance advantage over the Thrift API. On Mon, Nov 19, 2012 at 1:05 PM, Everton Lima peitin.inu...@gmail.comwrote: For some reason I can not reply my old thread in that list. So I was creating a new one. The problem is that I do not use thrift to gain in performace. Why it is nicer with concurrent modifications? I do not know why I have falling in the problem of concurrent modification if I was creating 2 keyspaces diferent in only one process with just one thread. Someone knows why? -- Everton Lima Aleixo Bacharel em Ciencia da Computação Universidade Federal de Goiás -- Tyler Hobbs DataStax http://datastax.com/
Re: Datatype Conversion in CQL-Client?
Do these other clients use the thrift API internaly? 2012/11/19 John Sanda john.sa...@gmail.com You might want to take look a org.apache.cassandra.transport.SimpleClient and org.apache.cassandra.transport.messages.ResultMessage. On Mon, Nov 19, 2012 at 9:48 AM, Timmy Turner timm.t...@gmail.com wrote: What I meant was the method that the Cassandra-jars give you when you include them in your project: TTransport tr = new TFramedTransport(new TSocket(localhost, 9160)); TProtocol proto = new TBinaryProtocol(tr); Cassandra.Client client = new Cassandra.Client(proto); tr.open(); client.execute_cql_query(ByteBuffer.wrap(cql.getBytes()), Compression.NONE); 2012/11/19 Brian O'Neill b...@alumni.brown.edu I don't think Michael and/or Jonathan have published the CQL java driver yet. (CCing them) Hopefully they'll find a public home for it soon, I hope to include it in the Webinar in December. (http://www.datastax.com/resources/webinars/collegecredit) -brian --- Brian O'Neill Lead Architect, Software Development *Health Market Science* *The Science of Better Results* 2700 Horizon Drive • King of Prussia, PA • 19406 M: 215.588.6024 • @boneill42 http://www.twitter.com/boneill42 • healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. ** ** From: Tommi Laukkanen tlaukka...@gmail.com Reply-To: user@cassandra.apache.org Date: Monday, November 19, 2012 2:36 AM To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? I think Timmy might be referring to the upcoming native CQL Java driver that might be coming with 1.2 - It was mentioned here: http://www.datastax.com/wp-content/uploads/2012/08/7_Datastax_Upcoming_Changes_in_Drivers.pdf I would also be interested on testing that but I can't find it from repositories. Any hints? Regards, Tommi L. *From:* Brian O'Neill [mailto:boneil...@gmail.com] *On Behalf Of *Brian O'Neill *Sent:* 18. marraskuuta 2012 17:47 *To:* user@cassandra.apache.org *Subject:* Re: Datatype Conversion in CQL-Client? *Importance:* Low ** ** ** ** If you are talking about the CQL-client that comes with Cassandra (cqlsh), it is actually written in Python: https://github.com/apache/cassandra/blob/trunk/bin/cqlsh ** ** For information on datatypes (and conversion) take a look at the CQL definition: http://www.datastax.com/docs/1.0/references/cql/index (Look at the CQL Data Types section) ** ** If that's not the client you are referencing, let us know which one you mean: http://brianoneill.blogspot.com/2012/08/cassandra-apis-laundry-list.html ** ** -brian ** ** On Nov 17, 2012, at 9:54 PM, Timmy Turner wrote: Thanks for the links, however I'm interested in the functionality that the official Cassandra client/API (which is in Java) offers. ** ** 2012/11/17 aaron morton aa...@thelastpickle.com Does the official/built-in Cassandra CQL client (in 1.2) What language ? ** ** Check the Java http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/ and python http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/ drivers. ** ** Cheers ** ** ** ** - Aaron Morton Freelance Cassandra Developer New Zealand ** ** @aaronmorton http://www.thelastpickle.com ** ** On 16/11/2012, at 11:21 AM, Timmy Turner timm.t...@gmail.com wrote:** ** Does the official/built-in Cassandra CQL client (in 1.2) offer any built-in option to get direct values/objects when reading a field, instead of just a byte array? ** ** ** ** ** ** -- Brian ONeill Lead Architect, Health Market Science (http://healthmarketscience.com) mobile:215.588.6024 blog: http://weblogs.java.net/blog/boneill42/ blog: http://brianoneill.blogspot.com/ ** ** -- - John
Re: Datatype Conversion in CQL-Client?
Hector does, but the newer clients/drivers no longer use Thrift. (Thrift is the legacy protocol) If you are still in early stages and you know you want your primary interface to be CQL, you may want to look at the java driver that Datastax just released. http://github.com/datastax/java-driver -brian --- Brian O'Neill Lead Architect, Software Development Health Market Science The Science of Better Results 2700 Horizon Drive King of Prussia, PA 19406 M: 215.588.6024 @boneill42 http://www.twitter.com/boneill42 healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. From: Timmy Turner timm.t...@gmail.com Reply-To: user@cassandra.apache.org Date: Monday, November 19, 2012 3:37 PM To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? Do these other clients use the thrift API internaly? 2012/11/19 John Sanda john.sa...@gmail.com You might want to take look a org.apache.cassandra.transport.SimpleClient and org.apache.cassandra.transport.messages.ResultMessage. On Mon, Nov 19, 2012 at 9:48 AM, Timmy Turner timm.t...@gmail.com wrote: What I meant was the method that the Cassandra-jars give you when you include them in your project: TTransport tr = new TFramedTransport(new TSocket(localhost, 9160)); TProtocol proto = new TBinaryProtocol(tr); Cassandra.Client client = new Cassandra.Client(proto); tr.open(); client.execute_cql_query(ByteBuffer.wrap(cql.getBytes()), Compression.NONE); 2012/11/19 Brian O'Neill b...@alumni.brown.edu I don't think Michael and/or Jonathan have published the CQL java driver yet. (CCing them) Hopefully they'll find a public home for it soon, I hope to include it in the Webinar in December. (http://www.datastax.com/resources/webinars/collegecredit) -brian --- Brian O'Neill Lead Architect, Software Development Health Market Science The Science of Better Results 2700 Horizon Drive King of Prussia, PA 19406 M: 215.588.6024 tel:215.588.6024 @boneill42 http://www.twitter.com/boneill42 healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. From: Tommi Laukkanen tlaukka...@gmail.com Reply-To: user@cassandra.apache.org Date: Monday, November 19, 2012 2:36 AM To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? I think Timmy might be referring to the upcoming native CQL Java driver that might be coming with 1.2 - It was mentioned here: http://www.datastax.com/wp-content/uploads/2012/08/7_Datastax_Upcoming_Chang es_in_Drivers.pdf I would also be interested on testing that but I can't find it from repositories. Any hints? Regards, Tommi L. From: Brian O'Neill [mailto:boneil...@gmail.com] On Behalf Of Brian O'Neill Sent: 18. marraskuuta 2012 17:47 To: user@cassandra.apache.org Subject: Re: Datatype Conversion in CQL-Client? Importance: Low If you are talking about the CQL-client that comes with Cassandra (cqlsh), it is actually written in Python: https://github.com/apache/cassandra/blob/trunk/bin/cqlsh For information on datatypes (and conversion) take a look at the CQL definition: http://www.datastax.com/docs/1.0/references/cql/index (Look at the CQL Data Types section) If that's not the client you are referencing, let us know which one you mean: http://brianoneill.blogspot.com/2012/08/cassandra-apis-laundry-list.html -brian On Nov 17, 2012, at 9:54 PM, Timmy Turner wrote: Thanks for the links, however I'm interested in the functionality that the official Cassandra client/API (which is in Java) offers. 2012/11/17 aaron morton aa...@thelastpickle.com Does the official/built-in Cassandra CQL client (in 1.2) What language ? Check the Java http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/
Re: row cache re-fill very slow
i was just wondering if anyone else is experiencing very slow ( ~ 3.5 MB/sec ) re-fill of the row cache at start up. It was mentioned the other day. What version are you on ? Do you know how many rows were loaded ? When complete it will log a message with the pattern completed loading (%d ms; %d keys) row cache for %s.%s How is the saved row cache file processed? In Version 1.1, after the SSTables have been opened the keys in the saved row cache are read one at a time and the whole row read into memory. This is a single threaded operation. In 1.2 reading the saved cache is still single threaded, but reading the rows goes through the read thread pool so is in parallel. In both cases I do not believe the cache is stored in token (or key) order. ( Admittedly whatever is going on is still much more preferable to starting with a cold row cache ) row_cache_keys_to_save in yaml may help you find a happy half way point. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 20/11/2012, at 3:17 AM, Andras Szerdahelyi andras.szerdahe...@ignitionone.com wrote: Hey list, i was just wondering if anyone else is experiencing very slow ( ~ 3.5 MB/sec ) re-fill of the row cache at start up. We operate with a large row cache ( 10-15GB currently ) and we already measure startup times in hours :-) How is the saved row cache file processed? Are the cached row keys simply iterated over and their respective rows read from SSTables - possibly creating random reads with small enough sstable files, if the keys were not stored in a manner optimised for a quick re-fill ? - or is there a smarter algorithm ( i.e. scan through one sstable at a time, filter rows that should be in row cache ) at work and this operation is purely disk i/o bound ? ( Admittedly whatever is going on is still much more preferable to starting with a cold row cache ) thanks! Andras Andras Szerdahelyi Solutions Architect, IgnitionOne | 1831 Diegem E.Mommaertslaan 20A M: +32 493 05 50 88 | Skype: sandrew84 C4798BB9-9092-4145-880B-A72C6B7AF9A4[41].png
Re: Invalid argument
Are you running a 32 bit JVM ? What is the full JVM version ? As a work around you can try disabling memory mapped access set disk_access_mode to standard. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 20/11/2012, at 6:27 AM, Alain RODRIGUEZ arodr...@gmail.com wrote: I have backed up production sstables from one of my 3 production nodes (RF=3) and I want to use them on my dev environment.(C* 1.1.6 on both environments) My dev server is a 4 core, 4 GB RAM hardware runing on ubuntu. I have applied the production schema in my dev node and copied all sstable in the appropriated folder and restart my node like I always do. But this time have had the following error (many times and only for ) : INFO [SSTableBatchOpen:4] 2012-11-19 17:52:52,980 SSTableReader.java (line 169) Opening /var/lib/cassandra/data/cassa_teads/data_action/cassa_teads-data_action-hf-660 (7015417424 bytes) ERROR [SSTableBatchOpen:3] 2012-11-19 17:53:17,259 AbstractCassandraDaemon.java (line 135) Exception in thread Thread[SSTableBatchOpen:3,5,main] java.io.IOError: java.io.IOException: Invalid argument at org.apache.cassandra.io.util.MmappedSegmentedFile$Builder.createSegments(MmappedSegmentedFile.java:202) at org.apache.cassandra.io.util.MmappedSegmentedFile$Builder.complete(MmappedSegmentedFile.java:179) at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:429) at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:200) at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:153) at org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:242) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: Invalid argument at sun.nio.ch.FileChannelImpl.truncate0(Native Method) at sun.nio.ch.FileChannelImpl.map(Unknown Source) at org.apache.cassandra.io.util.MmappedSegmentedFile$Builder.createSegments(MmappedSegmentedFile.java:194) ... 11 more If I try with nodetool refresh I have the following error : Exception in thread main java.io.IOError: java.io.IOException: Invalid argument at org.apache.cassandra.io.util.MmappedSegmentedFile$Builder.createSegments(MmappedSegmentedFile.java:202) at org.apache.cassandra.io.util.MmappedSegmentedFile$Builder.complete(MmappedSegmentedFile.java:179) at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:429) at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:200) at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:153) at org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:510) at org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:468) at org.apache.cassandra.service.StorageService.loadNewSSTables(StorageService.java:3089) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source) at com.sun.jmx.mbeanserver.PerInterface.invoke(Unknown Source) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(Unknown Source) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(Unknown Source) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl.access$200(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl.invoke(Unknown Source) at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source)
Re: row cache re-fill very slow
Last time I checked, it took about 120 seconds to load up 21125 keys with total about 500M in memory ( We have a pretty wide row:). So it's about 4 MB/sec. Just curious Andras, how can you manage such a big row cache (10-15GB currently)? By recommendation, you will have 10% of your heap as row cache, so your heap is over 100G?? The largest datastax recommends is 8GB and it seems to be a hardcoded limit in cassandra-env.sh ( # calculate 1/4 ram and cap to 8192MB). Does you GC hold up with such a big heap? In my experience, full GC could take over 20 seconds for such a big heap. Thanks. -Wei From: aaron morton aa...@thelastpickle.com To: user@cassandra.apache.org Sent: Monday, November 19, 2012 1:00 PM Subject: Re: row cache re-fill very slow i was just wondering if anyone else is experiencing very slow ( ~ 3.5 MB/sec ) re-fill of the row cache at start up. It was mentioned the other day. What version are you on ? Do you know how many rows were loaded ? When complete it will log a message with the pattern completed loading (%d ms; %d keys) row cache for %s.%s How is the saved row cache file processed? In Version 1.1, after the SSTables have been opened the keys in the saved row cache are read one at a time and the whole row read into memory. This is a single threaded operation. In 1.2 reading the saved cache is still single threaded, but reading the rows goes through the read thread pool so is in parallel. In both cases I do not believe the cache is stored in token (or key) order. ( Admittedly whatever is going on is still much more preferable to starting with a cold row cache ) row_cache_keys_to_save in yaml may help you find a happy half way point. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 20/11/2012, at 3:17 AM, Andras Szerdahelyi andras.szerdahe...@ignitionone.com wrote: Hey list, i was just wondering if anyone else is experiencing very slow ( ~ 3.5 MB/sec ) re-fill of the row cache at start up. We operate with a large row cache ( 10-15GB currently ) and we already measure startup times in hours :-) How is the saved row cache file processed? Are the cached row keys simply iterated over and their respective rows read from SSTables - possibly creating random reads with small enough sstable files, if the keys were not stored in a manner optimised for a quick re-fill ? - or is there a smarter algorithm ( i.e. scan through one sstable at a time, filter rows that should be in row cache ) at work and this operation is purely disk i/o bound ? ( Admittedly whatever is going on is still much more preferable to starting with a cold row cache ) thanks! Andras Andras Szerdahelyi Solutions Architect, IgnitionOne | 1831 Diegem E.Mommaertslaan 20A M: +32 493 05 50 88 | Skype: sandrew84 C4798BB9-9092-4145-880B-A72C6B7AF9A4[41].png
Re: row cache re-fill very slow
Aaron, What version are you on ? 1.1.5 Do you know how many rows were loaded ? INFO [OptionalTasks:1] 2012-11-19 13:08:58,868 ColumnFamilyStore.java (line 451) completed loading (5175655 ms; 13259976 keys) row cache In both cases I do not believe the cache is stored in token (or key) order. Am i getting this right: the row keys are read and rows are retrieved from SSTables in the order their keys are in the cache file.. Would something like iterating over SSTables instead, and throwing rows at the cache that need to be in there feasible ? If the SSTables themselves are written sequentially at compaction time , which is how i remember they are written, SSTable-sized sequential reads with a filter ( bloom filter for the row cache? :-) ) must be faster than reading from all across the column family ( i have HDDs and about 1k SSTables ) row_cache_keys_to_save in yaml may help you find a happy half way point. If i can keep that high enough, with my data retention requirements, save for the absolute first get on a row, i can operate entirely out of memory. thanks! Andras Andras Szerdahelyi Solutions Architect, IgnitionOne | 1831 Diegem E.Mommaertslaan 20A M: +32 493 05 50 88 | Skype: sandrew84 [cid:7BDF7228-D831-4D98-967A-BE04FEB17544] On 19 Nov 2012, at 22:00, aaron morton aa...@thelastpickle.commailto:aa...@thelastpickle.com wrote: i was just wondering if anyone else is experiencing very slow ( ~ 3.5 MB/sec ) re-fill of the row cache at start up. It was mentioned the other day. What version are you on ? Do you know how many rows were loaded ? When complete it will log a message with the pattern completed loading (%d ms; %d keys) row cache for %s.%s How is the saved row cache file processed? In Version 1.1, after the SSTables have been opened the keys in the saved row cache are read one at a time and the whole row read into memory. This is a single threaded operation. In 1.2 reading the saved cache is still single threaded, but reading the rows goes through the read thread pool so is in parallel. In both cases I do not believe the cache is stored in token (or key) order. ( Admittedly whatever is going on is still much more preferable to starting with a cold row cache ) row_cache_keys_to_save in yaml may help you find a happy half way point. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.comhttp://www.thelastpickle.com/ On 20/11/2012, at 3:17 AM, Andras Szerdahelyi andras.szerdahe...@ignitionone.commailto:andras.szerdahe...@ignitionone.com wrote: Hey list, i was just wondering if anyone else is experiencing very slow ( ~ 3.5 MB/sec ) re-fill of the row cache at start up. We operate with a large row cache ( 10-15GB currently ) and we already measure startup times in hours :-) How is the saved row cache file processed? Are the cached row keys simply iterated over and their respective rows read from SSTables - possibly creating random reads with small enough sstable files, if the keys were not stored in a manner optimised for a quick re-fill ? - or is there a smarter algorithm ( i.e. scan through one sstable at a time, filter rows that should be in row cache ) at work and this operation is purely disk i/o bound ? ( Admittedly whatever is going on is still much more preferable to starting with a cold row cache ) thanks! Andras Andras Szerdahelyi Solutions Architect, IgnitionOne | 1831 Diegem E.Mommaertslaan 20A M: +32 493 05 50 88 | Skype: sandrew84 C4798BB9-9092-4145-880B-A72C6B7AF9A4[41].png inline: C4798BB9-9092-4145-880B-A72C6B7AF9A4[41].png
Re: row cache re-fill very slow
Wei, i'm using the off-heap ( serialised ) row cache and front the entire thing with memcached in the middle layer ( to prevent the most actively requested rows from pressuring the Cassandra heap ). If you ask how much the pointers to the off-heap memory will take.. time will tell ( it should be easy to calculate though ) It's been working out great. Andras Szerdahelyi Solutions Architect, IgnitionOne | 1831 Diegem E.Mommaertslaan 20A M: +32 493 05 50 88 | Skype: sandrew84 [cid:7BDF7228-D831-4D98-967A-BE04FEB17544] On 19 Nov 2012, at 22:23, Wei Zhu wz1...@yahoo.commailto:wz1...@yahoo.com wrote: Just curious Andras, how can you manage such a big row cache (10-15GB currently)? By recommendation, you will have 10% of your heap as row cache, so your heap is over 100G?? The largest datastax recommends is 8GB and it seems to be a hardcoded limit in cassandra-env.sh ( # calculate 1/4 ram and cap to 8192MB). Does you GC hold up with such a big heap? In my experience, full GC could take over 20 seconds for such a big heap. inline: C4798BB9-9092-4145-880B-A72C6B7AF9A4[41].png
Re: row cache re-fill very slow
On Mon, Nov 19, 2012 at 6:17 AM, Andras Szerdahelyi andras.szerdahe...@ignitionone.com wrote: How is the saved row cache file processed? Are the cached row keys simply iterated over and their respective rows read from SSTables - possibly creating random reads with small enough sstable files, if the keys were not stored in a manner optimised for a quick re-fill ? - or is there a smarter algorithm ( i.e. scan through one sstable at a time, filter rows that should be in row cache ) at work and this operation is purely disk i/o bound ? Nope, that's it. I am quite confident that in the version you are running, it just assembles the row from disk, from the relevant SSTables, via the more or less normal read path. The more fragmented your sstables, the more random the i/o. These two 1.2.x era JIRA relate to the row cache startup penalty : https://issues.apache.org/jira/browse/CASSANDRA-4282 # multi-threaded row cache loading at startup https://issues.apache.org/jira/browse/CASSANDRA-3762 # improvements to the AutoSavingCache (which is the base class of AutoSavingRowCache) =Rob -- =Robert Coli AIMGTALK - rc...@palominodb.com YAHOO - rcoli.palominob SKYPE - rcoli_palominodb
Using Hector from .Net
I thought others might be interested to know. If you've ever asked what drivers are available for Cassandra in .Net and can I use Hector? the answer is yes and it works quite well. Just use IKVM with hector peter lin
Re: Datastax Java Driver
We're currently talking with some guys from Hibernate OGM team. This kind of integration is likely to happen on top of this core module. The mapping module that will add soon isn't alike OGM as it'll be a very thin mapping library, covering most use cases with low overhead. Michaël On Mon, Nov 19, 2012 at 8:29 PM, John Sanda john.sa...@gmail.com wrote: Fantastic! As for the object mapping API, has there been any discussion/consideration of http://www.hibernate.org/subprojects/ogm.html? On Mon, Nov 19, 2012 at 1:50 PM, Sylvain Lebresne sylv...@datastax.comwrote: Everyone, We've just open-sourced a new Java driver we have been working on here at DataStax. This driver is CQL3 only and is built to use the new binary protocol that will be introduced with Cassandra 1.2. It will thus only work with Cassandra 1.2 onwards. Currently, it means that testing it requires 1.2.0-beta2. This is also alpha software at this point. You are welcome to try and play with it and we would very much welcome feedback, but be sure that break, it will. The driver is accessible at: http://github.com/datastax/java-driver Today we're open-sourcing the core part of this driver. This main goal of this core module is to handle connections to the Cassandra cluster with all the features that one would expect. The currently supported features are: - Asynchronous: the driver uses the new CQL binary protocol asynchronous capabilities. - Nodes discovery. - Configurable load balancing/routing. - Transparent fail-over. - C* tracing handling. - Convenient schema access. - Configurable retry policy. This core module provides a simple low-level API (that works directly with query strings). We plan to release a higher-level, thin object mapping API based on top of this core shortly. Please refer to the project README for more information. -- The DataStax Team -- - John
Re: Datastax Java Driver
Why is this being released as a separate project, instead of being bundled up with Cassandra? Is it not a part of Cassandra? 2012/11/19 John Sanda john.sa...@gmail.com Fantastic! As for the object mapping API, has there been any discussion/consideration of http://www.hibernate.org/subprojects/ogm.html? On Mon, Nov 19, 2012 at 1:50 PM, Sylvain Lebresne sylv...@datastax.comwrote: Everyone, We've just open-sourced a new Java driver we have been working on here at DataStax. This driver is CQL3 only and is built to use the new binary protocol that will be introduced with Cassandra 1.2. It will thus only work with Cassandra 1.2 onwards. Currently, it means that testing it requires 1.2.0-beta2. This is also alpha software at this point. You are welcome to try and play with it and we would very much welcome feedback, but be sure that break, it will. The driver is accessible at: http://github.com/datastax/java-driver Today we're open-sourcing the core part of this driver. This main goal of this core module is to handle connections to the Cassandra cluster with all the features that one would expect. The currently supported features are: - Asynchronous: the driver uses the new CQL binary protocol asynchronous capabilities. - Nodes discovery. - Configurable load balancing/routing. - Transparent fail-over. - C* tracing handling. - Convenient schema access. - Configurable retry policy. This core module provides a simple low-level API (that works directly with query strings). We plan to release a higher-level, thin object mapping API based on top of this core shortly. Please refer to the project README for more information. -- The DataStax Team -- - John
Re: Datastax Java Driver
The Apache Cassandra project has traditionally not focused on client side. Rather than modifying the scope of the project and jeopardizing the current driver ecosystem we've preferred to open source it this way. Not that this driver's license is Apache License 2 and it will remain so, making it easy to integrate it in most open source projects with no license issues. On Tue, Nov 20, 2012 at 1:19 AM, Timmy Turner timm.t...@gmail.com wrote: Why is this being released as a separate project, instead of being bundled up with Cassandra? Is it not a part of Cassandra? 2012/11/19 John Sanda john.sa...@gmail.com Fantastic! As for the object mapping API, has there been any discussion/consideration of http://www.hibernate.org/subprojects/ogm.html ? On Mon, Nov 19, 2012 at 1:50 PM, Sylvain Lebresne sylv...@datastax.comwrote: Everyone, We've just open-sourced a new Java driver we have been working on here at DataStax. This driver is CQL3 only and is built to use the new binary protocol that will be introduced with Cassandra 1.2. It will thus only work with Cassandra 1.2 onwards. Currently, it means that testing it requires 1.2.0-beta2. This is also alpha software at this point. You are welcome to try and play with it and we would very much welcome feedback, but be sure that break, it will. The driver is accessible at: http://github.com/datastax/java-driver Today we're open-sourcing the core part of this driver. This main goal of this core module is to handle connections to the Cassandra cluster with all the features that one would expect. The currently supported features are: - Asynchronous: the driver uses the new CQL binary protocol asynchronous capabilities. - Nodes discovery. - Configurable load balancing/routing. - Transparent fail-over. - C* tracing handling. - Convenient schema access. - Configurable retry policy. This core module provides a simple low-level API (that works directly with query strings). We plan to release a higher-level, thin object mapping API based on top of this core shortly. Please refer to the project README for more information. -- The DataStax Team -- - John
Re: SchemaDisagreementException
Yes I already have tested. I use the Object CassandraServer to do the operations instead of open conection with CassandraClient. Both of this object implements Iface. I think the performace of use CassandraServer improve because it does not open an connection, and CassandraClient (that use thrift) and CQL open a connection. 2012/11/19 Tyler Hobbs ty...@datastax.com Have you actually tested to see that the Thrift API is more performant than CQL for your application? As far as I know, CQL almost always has a performance advantage over the Thrift API. On Mon, Nov 19, 2012 at 1:05 PM, Everton Lima peitin.inu...@gmail.comwrote: For some reason I can not reply my old thread in that list. So I was creating a new one. The problem is that I do not use thrift to gain in performace. Why it is nicer with concurrent modifications? I do not know why I have falling in the problem of concurrent modification if I was creating 2 keyspaces diferent in only one process with just one thread. Someone knows why? -- Everton Lima Aleixo Bacharel em Ciencia da Computação Universidade Federal de Goiás -- Tyler Hobbs DataStax http://datastax.com/ -- Everton Lima Aleixo Bacharel em Ciencia da Computação Universidade Federal de Goiás
Re: SchemaDisagreementException
While this might not be helpful (I don't have all the thread history here), have you checked that all your servers are properly synced with NTP? From: Everton Lima peitin.inu...@gmail.commailto:peitin.inu...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Monday, November 19, 2012 6:24 PM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: SchemaDisagreementException Yes I already have tested. I use the Object CassandraServer to do the operations instead of open conection with CassandraClient. Both of this object implements Iface. I think the performace of use CassandraServer improve because it does not open an connection, and CassandraClient (that use thrift) and CQL open a connection. 2012/11/19 Tyler Hobbs ty...@datastax.commailto:ty...@datastax.com Have you actually tested to see that the Thrift API is more performant than CQL for your application? As far as I know, CQL almost always has a performance advantage over the Thrift API. On Mon, Nov 19, 2012 at 1:05 PM, Everton Lima peitin.inu...@gmail.commailto:peitin.inu...@gmail.com wrote: For some reason I can not reply my old thread in that list. So I was creating a new one. The problem is that I do not use thrift to gain in performace. Why it is nicer with concurrent modifications? I do not know why I have falling in the problem of concurrent modification if I was creating 2 keyspaces diferent in only one process with just one thread. Someone knows why? -- Everton Lima Aleixo Bacharel em Ciencia da Computação Universidade Federal de Goiás -- Tyler Hobbs DataStaxhttp://datastax.com/ -- Everton Lima Aleixo Bacharel em Ciencia da Computação Universidade Federal de Goiás 'Like' us on Facebook for exclusive content and other resources on all Barracuda Networks solutions. Visit http://barracudanetworks.com/facebook
Re: Upgrade 1.1.2 - 1.1.6
Alain, We performed a 1.1.3 - 1.1.6 upgrade and found that all the logs replayed regardless of the drain. After noticing this on the first node, we did the following: * nodetool flush * nodetool drain * service cassandra stop * mv /path/to/logs/*.log /backup/ * apt-get install cassandra restarts automatically I also agree that starting C* after an upgrade/install seems quite broken if it was already stopped before the install. However annoying, I have found this to be the default for most Ubuntu daemon packages. Mike On Thu, Nov 15, 2012 at 9:21 AM, Alain RODRIGUEZ arodr...@gmail.com wrote: We had an issue with counters over-counting even using the nodetool drain command before upgrading... Here is my bash history 69 cp /etc/cassandra/cassandra.yaml /etc/cassandra/cassandra.yaml.bak 70 cp /etc/cassandra/cassandra-env.sh /etc/cassandra/cassandra-env.sh.bak 71 sudo apt-get install cassandra 72 nodetool disablethrift 73 nodetool drain 74 service cassandra stop 75 cat /etc/cassandra/cassandra-env.sh /etc/cassandra/cassandra-env.sh.bak 76 vim /etc/cassandra/cassandra-env.sh 77 cat /etc/cassandra/cassandra.yaml /etc/cassandra/cassandra.yaml.bak 78 vim /etc/cassandra/cassandra.yaml 79 service cassandra start So I think I followed these steps http://www.datastax.com/docs/1.1/install/upgrading#upgrade-steps I merged my conf files with an external tool so consider I merged my conf files on steps 76 and 78. I saw that the sudo apt-get install cassandra stop the server and restart it automatically. So it updated without draining and restart before I had the time to reconfigure the conf files. Is this normal ? Is there a way to avoid it ? So for the second node I decided to try to stop C*before the upgrade. 125 cp /etc/cassandra/cassandra.yaml /etc/cassandra/cassandra.yaml.bak 126 cp /etc/cassandra/cassandra-env.sh /etc/cassandra/cassandra-env.sh.bak 127 nodetool disablegossip 128 nodetool disablethrift 129 nodetool drain 130 service cassandra stop 131 sudo apt-get install cassandra //131 : This restarted cassandra 132 nodetool disablethrift 133 nodetool disablegossip 134 nodetool drain 135 service cassandra stop 136 cat /etc/cassandra/cassandra-env.sh /etc/cassandra/cassandra-env.sh.bak 137 cim /etc/cassandra/cassandra-env.sh 138 vim /etc/cassandra/cassandra-env.sh 139 cat /etc/cassandra/cassandra.yaml /etc/cassandra/cassandra.yaml.bak 140 vim /etc/cassandra/cassandra.yaml 141 service cassandra start After both of these updates I saw my current counters increase without any reason. Did I do anything wrong ? Alain -- Mike Heffner m...@librato.com Librato, Inc.
Re: SchemaDisagreementException
http://www.acunu.com/2/post/2011/12/cql-benchmarking.html Last I checked, thrift still had an edge over cql due to string serialization and de serialization. Might be even more dramatic for later columns. Not that client speed matters much overall in cassandra's speed, but CQL client does more. On Mon, Nov 19, 2012 at 9:27 PM, Michael Kjellman mkjell...@barracuda.com wrote: While this might not be helpful (I don't have all the thread history here), have you checked that all your servers are properly synced with NTP? From: Everton Lima peitin.inu...@gmail.com Reply-To: user@cassandra.apache.org user@cassandra.apache.org Date: Monday, November 19, 2012 6:24 PM To: user@cassandra.apache.org user@cassandra.apache.org Subject: Re: SchemaDisagreementException Yes I already have tested. I use the Object CassandraServer to do the operations instead of open conection with CassandraClient. Both of this object implements Iface. I think the performace of use CassandraServer improve because it does not open an connection, and CassandraClient (that use thrift) and CQL open a connection. 2012/11/19 Tyler Hobbs ty...@datastax.com Have you actually tested to see that the Thrift API is more performant than CQL for your application? As far as I know, CQL almost always has a performance advantage over the Thrift API. On Mon, Nov 19, 2012 at 1:05 PM, Everton Lima peitin.inu...@gmail.com wrote: For some reason I can not reply my old thread in that list. So I was creating a new one. The problem is that I do not use thrift to gain in performace. Why it is nicer with concurrent modifications? I do not know why I have falling in the problem of concurrent modification if I was creating 2 keyspaces diferent in only one process with just one thread. Someone knows why? -- Everton Lima Aleixo Bacharel em Ciencia da Computação Universidade Federal de Goiás -- Tyler Hobbs DataStax -- Everton Lima Aleixo Bacharel em Ciencia da Computação Universidade Federal de Goiás -- 'Like' us on Facebook for exclusive content and other resources on all Barracuda Networks solutions. Visit http://barracudanetworks.com/facebook
