Re: Cassandra Insert Rate
Thanks for BR for the quick response on this and Appreciate it. That helps for Batch load. If 10 million users are inserting new records at a time ( 1 record for one user ) then how do we increase the same. My sample program assumes request from 10 million records. -Shankar On Tue, Jun 9, 2015 at 1:56 AM, Marcus Olsson marcus.ols...@ericsson.com wrote: Hi Shankar, I would say: * Prepared statements to avoid sending the whole statement with every query and instead just send the values. * Using session.executeAsync() to improve concurrency. So you would start by creating a prepared statement, something like: PreparedStatement ps = session.prepare(INSERT INTO ks.tb (key1,data1,data2) VALUES (?,?,?)); // Only done once And then in loadData(): session.executeAsync(ps.bind(key, 1, 2)); The executeAsync() does not wait for a response for the query, so that should probably be done elsewhere(see the link below for how you can get the results back). http://www.datastax.com/dev/blog/java-driver-async-queries BR Marcus Olsson On 06/09/2015 10:27 AM, SHANKAR REDDY wrote: Team, I have a sample insert query which loads around 10 million records and found that the insert rate is around 1500 per second. This is very slow. The Source code I am using available at the below location. I am using the very latest version 2.1.6 with default seetings and single node VM machine with 20GM RAM and 100 GM SSD disk. https://github.com/shankar-reddy/CassandraSandbox/blob/master/src/main/java/com/itreddys/cassandra/example/BulkLoadTest.java Please suggest on insert rate improvement. -Shankar
Re: How to minimize Cassandra memory usage for test environment?
for a start, maybe you can see the setting use by raspberry pi project, for instance http://ac31004.blogspot.com/2012/05/apache-cassandra-on-raspberry-pi.html you can look at these two files, to tune down the settings for test environment. cassandra-env.sh cassandra.yaml hth jason On Tue, Jun 9, 2015 at 3:59 PM, Eax Melanhovich m...@eax.me wrote: Hello. We are running integration tests, using real Cassandra (not a mock) under Vagrant. MAX_HEAP_SIZE is set to 500M. As I discovered, lower value causes 'out of memory' after some time. Could memory usage be decreased somehow? Developers don't usually have a lot of free RAM and performance obviously is not an issue in this case. -- Best regards, Eax Melanhovich http://eax.me/
How to minimize Cassandra memory usage for test environment?
Hello. We are running integration tests, using real Cassandra (not a mock) under Vagrant. MAX_HEAP_SIZE is set to 500M. As I discovered, lower value causes 'out of memory' after some time. Could memory usage be decreased somehow? Developers don't usually have a lot of free RAM and performance obviously is not an issue in this case. -- Best regards, Eax Melanhovich http://eax.me/
Re: sstableloader usage doubts
Il 08/06/15 20:11, Robert Coli ha scritto: On Mon, Jun 8, 2015 at 6:58 AM, ZeroUno zerozerouno...@gmail.com mailto:zerozerouno...@gmail.com wrote: So... if I stop the two nodes on the first DC, restore their sstables' files, and then restart the nodes, nothing else needs to be done on the first DC? Be careful to avoid bootstrapping, but yes. What do you mean? As far as I read from the docs, bootstrapping happens when adding a new node to the cluster, but in my situation the nodes already exist, I'm only adding data back into them. Also I have all 4 nodes configured as seeds in cassandra.yaml, so if I'm not wrong this should prevent them from auto-bootstrapping. Thanks. Marco -- 01
Cassandra Insert Rate
Team, I have a sample insert query which loads around 10 million records and found that the insert rate is around 1500 per second. This is very slow. The Source code I am using available at the below location. I am using the very latest version 2.1.6 with default seetings and single node VM machine with 20GM RAM and 100 GM SSD disk. https://github.com/shankar-reddy/CassandraSandbox/blob/master/src/main/java/com/itreddys/cassandra/example/BulkLoadTest.java Please suggest on insert rate improvement. -Shankar
Re: Cassandra Insert Rate
Hi Shankar, I would say: * Prepared statements to avoid sending the whole statement with every query and instead just send the values. * Using session.executeAsync() to improve concurrency. So you would start by creating a prepared statement, something like: PreparedStatement ps = session.prepare(INSERT INTO ks.tb (key1,data1,data2) VALUES (?,?,?)); // Only done once And then in loadData(): session.executeAsync(ps.bind(key, 1, 2)); The executeAsync() does not wait for a response for the query, so that should probably be done elsewhere(see the link below for how you can get the results back). http://www.datastax.com/dev/blog/java-driver-async-queries BR Marcus Olsson On 06/09/2015 10:27 AM, SHANKAR REDDY wrote: Team, I have a sample insert query which loads around 10 million records and found that the insert rate is around 1500 per second. This is very slow. The Source code I am using available at the below location. I am using the very latest version 2.1.6 with default seetings and single node VM machine with 20GM RAM and 100 GM SSD disk. https://github.com/shankar-reddy/CassandraSandbox/blob/master/src/main/java/com/itreddys/cassandra/example/BulkLoadTest.java Please suggest on insert rate improvement. -Shankar
Re: Support for ad-hoc query
Cassandra isn¹t great at ad hoc queries. Many of us have paired it with an indexing engine like SOLR or Elastic Search. (built-into the DSE solution) As of late, I think there are a few of us exploring Spark SQL. (which you can then use via JDBC or REST) -brian --- Brian O'Neill Chief Technology Officer Health Market Science, a LexisNexis Company 215.588.6024 Mobile @boneill42 http://www.twitter.com/boneill42 This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. From: Srinivasa T N seen...@gmail.com Reply-To: user@cassandra.apache.org Date: Tuesday, June 9, 2015 at 2:38 AM To: user@cassandra.apache.org user@cassandra.apache.org Subject: Support for ad-hoc query Hi All, I have an web application running with my backend data stored in cassandra. Now I want to do some analysis on the data stored which requires some ad-hoc queries fired on cassandra. How can I do the same? Regards, Seenu.
Re: How to minimize Cassandra memory usage for test environment?
Hi Eax, are you truncating/dropping tables between tests? Are your issues perhaps related to that? If you are, you should disable autoSnapshots and enable -DunsafeSystem=true to make it run smoother. kind regards, Christian On Tue, Jun 9, 2015 at 11:25 AM, Jason Wee peich...@gmail.com wrote: for a start, maybe you can see the setting use by raspberry pi project, for instance http://ac31004.blogspot.com/2012/05/apache-cassandra-on-raspberry-pi.html you can look at these two files, to tune down the settings for test environment. cassandra-env.sh cassandra.yaml hth jason On Tue, Jun 9, 2015 at 3:59 PM, Eax Melanhovich m...@eax.me wrote: Hello. We are running integration tests, using real Cassandra (not a mock) under Vagrant. MAX_HEAP_SIZE is set to 500M. As I discovered, lower value causes 'out of memory' after some time. Could memory usage be decreased somehow? Developers don't usually have a lot of free RAM and performance obviously is not an issue in this case. -- Best regards, Eax Melanhovich http://eax.me/
Support for ad-hoc query
Hi All, I have an web application running with my backend data stored in cassandra. Now I want to do some analysis on the data stored which requires some ad-hoc queries fired on cassandra. How can I do the same? Regards, Seenu.
Re: Hundreds of sstables after every Repair
Hello, Do you have your clocks synced across your cluster? Are you using NTP and have it properly configured? Sometimes clock out of sync can trigger weird behaviour. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo http://linkedin.com/in/carlosjuzarterolo* Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649 www.pythian.com On Tue, Jun 9, 2015 at 5:11 PM, Anuj Wadehra anujw_2...@yahoo.co.in wrote: We were facing dropped mutations earlier and we increased flush writers. Now there are no dropped mutations in tpstats. To repair the damaged vnodes / inconsistent data we executed repair -pr on all nodes. Still, we see the same problem. When we analyze repair logs we see 2 strange things: 1. Out of sync ranges for cf which are not being actively being written/updated while the repair is going on. When we repaired all data by repair -pr on all nodes, why out of sync data? 2. For some cf , repair logs shows that all ranges are consistent. Still we get so many sstables created during repair. When everything is in sync , why repair creates tiny sstables to repair data? Thanks Anuj Wadehra Sent from Yahoo Mail on Android https://overview.mail.yahoo.com/mobile/?.src=Android -- *From*:Ken Hancock ken.hanc...@schange.com *Date*:Tue, 9 Jun, 2015 at 8:24 pm *Subject*:Re: Hundreds of sstables after every Repair I think this came up recently in another thread. If you're getting large numbers of SSTables after repairs, that means that your nodes are diverging from the keys that they're supposed to be having. Likely you're dropping mutations. Do a nodetool tpstats on each of your nodes and look at the mutation droppped counters. If you're seeing dropped message, my money you have a non-zero FlushWriter All time blocked stat which is causing mutations to be dropped. On Tue, Jun 9, 2015 at 10:35 AM, Anuj Wadehra anujw_2...@yahoo.co.in wrote: Any suggestions or comments on this one? Thanks Anuj Wadehra Sent from Yahoo Mail on Android https://overview.mail.yahoo.com/mobile/?.src=Android -- *From*:Anuj Wadehra anujw_2...@yahoo.co.in *Date*:Sun, 7 Jun, 2015 at 1:54 am *Subject*:Hundreds of sstables after every Repair Hi, We are using 2.0.3 and vnodes. After every repair -pr operation 50+ tiny sstables( 10K) get created. And these sstables never get compacted due to coldness issue. I have raised https://issues.apache.org/jira/browse/CASSANDRA-9146 for this issue but I have been told to upgrade. Till we upgrade to latest 2.0.x , we are stuck. Upgrade takes time, testing and planning in Production systems :( I have observed that even if vnodes are NOT damaged, hundreds of tiny sstables are created during repair for a wide row CF. This is beyond my understanding. If everything is consistent, and for the entire repair process Cassandra is saying Endpoints /x.x.x.x and /x.x.x.y are consistent for CF. Whats the need of creating sstables? Is there any alternative to regular major compaction to deal with situation? Thanks Anuj Wadehra -- --
Re: cassandra installation
cool , thank you for the suggestion. 2015-06-09 17:40 GMT+02:00 Carlos Rolo r...@pythian.com: (Some self-promotion here) You can follow this blog to help you out: http://www.pythian.com/blog/from-0-to-cassandra-an-exhaustive-approach-to-installing-cassandra/ Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo http://linkedin.com/in/carlosjuzarterolo* Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649 www.pythian.com On Tue, Jun 9, 2015 at 5:38 PM, ICHIBA Sara ichi.s...@gmail.com wrote: Ok thank you very much. I got my answer :) 2015-06-09 17:34 GMT+02:00 Alex Popescu al...@datastax.com: The JVM is the runtime platform for Cassandra. You can use any driver with it (there's no default driver). DataStax provides quite a few open source drivers [1] and there are also community maintained drivers [2] [1]: https://github.com/datastax/ [2]: http://planetcassandra.org/client-drivers-tools/ On Tue, Jun 9, 2015 at 8:26 AM, ICHIBA Sara ichi.s...@gmail.com wrote: hey there, In order to install cassandra, java has to be installed first. is that mean that the default driver of cassandra is java? what if we wanna use the cpp driver? P.S 1: my questions might be so dump, but I will really appreciate an answer. PS 2: I'm intern and i'm completely new to cassandra so... -- Bests, Alex Popescu | @al3xandru Sen. Product Manager @ DataStax --
Re: cassandra installation
Ok thank you very much. I got my answer :) 2015-06-09 17:34 GMT+02:00 Alex Popescu al...@datastax.com: The JVM is the runtime platform for Cassandra. You can use any driver with it (there's no default driver). DataStax provides quite a few open source drivers [1] and there are also community maintained drivers [2] [1]: https://github.com/datastax/ [2]: http://planetcassandra.org/client-drivers-tools/ On Tue, Jun 9, 2015 at 8:26 AM, ICHIBA Sara ichi.s...@gmail.com wrote: hey there, In order to install cassandra, java has to be installed first. is that mean that the default driver of cassandra is java? what if we wanna use the cpp driver? P.S 1: my questions might be so dump, but I will really appreciate an answer. PS 2: I'm intern and i'm completely new to cassandra so... -- Bests, Alex Popescu | @al3xandru Sen. Product Manager @ DataStax
Re: Hundreds of sstables after every Repair
Any suggestions or comments on this one? Thanks Anuj Wadehra Sent from Yahoo Mail on Android From:Anuj Wadehra anujw_2...@yahoo.co.in Date:Sun, 7 Jun, 2015 at 1:54 am Subject:Hundreds of sstables after every Repair Hi, We are using 2.0.3 and vnodes. After every repair -pr operation 50+ tiny sstables( 10K) get created. And these sstables never get compacted due to coldness issue. I have raised https://issues.apache.org/jira/browse/CASSANDRA-9146 for this issue but I have been told to upgrade. Till we upgrade to latest 2.0.x , we are stuck. Upgrade takes time, testing and planning in Production systems :( I have observed that even if vnodes are NOT damaged, hundreds of tiny sstables are created during repair for a wide row CF. This is beyond my understanding. If everything is consistent, and for the entire repair process Cassandra is saying Endpoints /x.x.x.x and /x.x.x.y are consistent for CF. Whats the need of creating sstables? Is there any alternative to regular major compaction to deal with situation? Thanks Anuj Wadehra
Re: Hundreds of sstables after every Repair
I think this came up recently in another thread. If you're getting large numbers of SSTables after repairs, that means that your nodes are diverging from the keys that they're supposed to be having. Likely you're dropping mutations. Do a nodetool tpstats on each of your nodes and look at the mutation droppped counters. If you're seeing dropped message, my money you have a non-zero FlushWriter All time blocked stat which is causing mutations to be dropped. On Tue, Jun 9, 2015 at 10:35 AM, Anuj Wadehra anujw_2...@yahoo.co.in wrote: Any suggestions or comments on this one? Thanks Anuj Wadehra Sent from Yahoo Mail on Android https://overview.mail.yahoo.com/mobile/?.src=Android -- *From*:Anuj Wadehra anujw_2...@yahoo.co.in *Date*:Sun, 7 Jun, 2015 at 1:54 am *Subject*:Hundreds of sstables after every Repair Hi, We are using 2.0.3 and vnodes. After every repair -pr operation 50+ tiny sstables( 10K) get created. And these sstables never get compacted due to coldness issue. I have raised https://issues.apache.org/jira/browse/CASSANDRA-9146 for this issue but I have been told to upgrade. Till we upgrade to latest 2.0.x , we are stuck. Upgrade takes time, testing and planning in Production systems :( I have observed that even if vnodes are NOT damaged, hundreds of tiny sstables are created during repair for a wide row CF. This is beyond my understanding. If everything is consistent, and for the entire repair process Cassandra is saying Endpoints /x.x.x.x and /x.x.x.y are consistent for CF. Whats the need of creating sstables? Is there any alternative to regular major compaction to deal with situation? Thanks Anuj Wadehra
Re: Hundreds of sstables after every Repair
We were facing dropped mutations earlier and we increased flush writers. Now there are no dropped mutations in tpstats. To repair the damaged vnodes / inconsistent data we executed repair -pr on all nodes. Still, we see the same problem. When we analyze repair logs we see 2 strange things: 1. Out of sync ranges for cf which are not being actively being written/updated while the repair is going on. When we repaired all data by repair -pr on all nodes, why out of sync data? 2. For some cf , repair logs shows that all ranges are consistent. Still we get so many sstables created during repair. When everything is in sync , why repair creates tiny sstables to repair data? Thanks Anuj Wadehra Sent from Yahoo Mail on Android From:Ken Hancock ken.hanc...@schange.com Date:Tue, 9 Jun, 2015 at 8:24 pm Subject:Re: Hundreds of sstables after every Repair I think this came up recently in another thread. If you're getting large numbers of SSTables after repairs, that means that your nodes are diverging from the keys that they're supposed to be having. Likely you're dropping mutations. Do a nodetool tpstats on each of your nodes and look at the mutation droppped counters. If you're seeing dropped message, my money you have a non-zero FlushWriter All time blocked stat which is causing mutations to be dropped. On Tue, Jun 9, 2015 at 10:35 AM, Anuj Wadehra anujw_2...@yahoo.co.in wrote: Any suggestions or comments on this one? Thanks Anuj Wadehra Sent from Yahoo Mail on Android From:Anuj Wadehra anujw_2...@yahoo.co.in Date:Sun, 7 Jun, 2015 at 1:54 am Subject:Hundreds of sstables after every Repair Hi, We are using 2.0.3 and vnodes. After every repair -pr operation 50+ tiny sstables( 10K) get created. And these sstables never get compacted due to coldness issue. I have raised https://issues.apache.org/jira/browse/CASSANDRA-9146 for this issue but I have been told to upgrade. Till we upgrade to latest 2.0.x , we are stuck. Upgrade takes time, testing and planning in Production systems :( I have observed that even if vnodes are NOT damaged, hundreds of tiny sstables are created during repair for a wide row CF. This is beyond my understanding. If everything is consistent, and for the entire repair process Cassandra is saying Endpoints /x.x.x.x and /x.x.x.y are consistent for CF. Whats the need of creating sstables? Is there any alternative to regular major compaction to deal with situation? Thanks Anuj Wadehra
cassandra installation
hey there, In order to install cassandra, java has to be installed first. is that mean that the default driver of cassandra is java? what if we wanna use the cpp driver? P.S 1: my questions might be so dump, but I will really appreciate an answer. PS 2: I'm intern and i'm completely new to cassandra so...
Re: cassandra installation
The JVM is the runtime platform for Cassandra. You can use any driver with it (there's no default driver). DataStax provides quite a few open source drivers [1] and there are also community maintained drivers [2] [1]: https://github.com/datastax/ [2]: http://planetcassandra.org/client-drivers-tools/ On Tue, Jun 9, 2015 at 8:26 AM, ICHIBA Sara ichi.s...@gmail.com wrote: hey there, In order to install cassandra, java has to be installed first. is that mean that the default driver of cassandra is java? what if we wanna use the cpp driver? P.S 1: my questions might be so dump, but I will really appreciate an answer. PS 2: I'm intern and i'm completely new to cassandra so... -- Bests, Alex Popescu | @al3xandru Sen. Product Manager @ DataStax
Re: Restoring all cluster from snapshots
Hi, Not sure why this isn't working. Some thoughts, just in case: - Have you check the files rights / owner ? - Have you tried copying files after directory creation through your cqlsh -f schema step ? - Have you tried without setting tokens manually ? - Are you sure to put the right sstables at the right place (if # nodes prod or RF are different from production) ? - Is trying to use sstableloader an option you want to consider ? Good luck, C*heers, Alain 2015-06-09 15:32 GMT+02:00 Anton Koshevoy nowa...@gmail.com: Ok, new sequence: …. - rm -rf /db/cassandra/cr/data0*/system/* - /etc/init.d/cassandra start - cqlsh -f schema(here is the schema for cr_production keyspace only) - /etc/init.d/cassandra restart I see the new keyspace, but it’s empty: cqlsh describe keyspaces; system_traces cr_production system cqlsh use cr_production ; cqlsh:cr_production select * from cr_production.url_cs ; id | ct | d_id +-+--- (0 rows) = Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens OwnsHost ID Rack UN 10.40.231.3 74.35 KB 256 ? 7fc4f832-9d98-4bfd-81fb-0c361bff487d RACK01 UN 10.40.231.31 86.46 KB 256 ? 782d450d-50d5-4738-a654-bf7398557842 RACK01 Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless then I tried: - nodetool refresh -- cr_production url_cs with the same result. On June 9, 2015 at 1:29:36 AM, Robert Coli (rc...@eventbrite.com) wrote: On Mon, Jun 8, 2015 at 2:52 PM, Sanjay Baronia sanjay.baro...@triliodata.com wrote: Yes, you shouldn’t delete the system directory. Next steps are …reconfigure the test cluster with new IP addresses, clear the gossiping information and then boot the test cluster. If you don't delete the system directory, you run the risk of the test cluster nodes joining the source cluster. Just start a single node on the new cluster, empty, and create the schema on it. Then do the rest of the process. =Rob
Re: Hundreds of sstables after every Repair
Yes. We use NTP. We also thought that drift is creating problems. Our NTP Output is as under: [root@node1 ~]# ntpq -p remote refid st t when poll reach delay offset jitter == +10.x.x.x 10.x.x.x 2 u 237 1024 377 1.199 0.062 0.554 *10.x.x.x 10.x.x.x 2 u 178 1024 377 0.479 -0.350 0.626 [root@node2 ~]# ntpq -p remote refid st t when poll reach delay offset jitter == +10.x.x.x 10.x.x.x 2 u 124 1024 377 0.939 -0.001 0.614 *10.x.x.x 10.x.x.x 2 u 722 1024 377 0.567 -0.241 0.585 [root@node3 ~]# ntpq -p remote refid st t when poll reach delay offset jitter == +10.x.x.x 10.x.x.x 2 u 514 1024 377 0.716 -0.103 1.315 *10.x.x.x 10.x.x.x 2 u 21 1024 377 0.402 -0.262 1.070 ***IPs are masked ThanksAnuj Wadehra On Tuesday, 9 June 2015 9:12 PM, Carlos Rolo r...@pythian.com wrote: Hello, Do you have your clocks synced across your cluster? Are you using NTP and have it properly configured? Sometimes clock out of sync can trigger weird behaviour. Regards, Carlos Juzarte RoloCassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: linkedin.com/in/carlosjuzarteroloMobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649www.pythian.com On Tue, Jun 9, 2015 at 5:11 PM, Anuj Wadehra anujw_2...@yahoo.co.in wrote: | We were facing dropped mutations earlier and we increased flush writers. Now there are no dropped mutations in tpstats. To repair the damaged vnodes / inconsistent data we executed repair -pr on all nodes. Still, we see the same problem. When we analyze repair logs we see 2 strange things: 1. Out of sync ranges for cf which are not being actively being written/updated while the repair is going on. When we repaired all data by repair -pr on all nodes, why out of sync data? 2. For some cf , repair logs shows that all ranges are consistent. Still we get so many sstables created during repair. When everything is in sync , why repair creates tiny sstables to repair data? ThanksAnuj Wadehra Sent from Yahoo Mail on Android | From:Ken Hancock ken.hanc...@schange.com Date:Tue, 9 Jun, 2015 at 8:24 pm Subject:Re: Hundreds of sstables after every Repair I think this came up recently in another thread. If you're getting large numbers of SSTables after repairs, that means that your nodes are diverging from the keys that they're supposed to be having. Likely you're dropping mutations. Do a nodetool tpstats on each of your nodes and look at the mutation droppped counters. If you're seeing dropped message, my money you have a non-zero FlushWriter All time blocked stat which is causing mutations to be dropped. On Tue, Jun 9, 2015 at 10:35 AM, Anuj Wadehra anujw_2...@yahoo.co.in wrote: | Any suggestions or comments on this one? ThanksAnuj Wadehra Sent from Yahoo Mail on Android | From:Anuj Wadehra anujw_2...@yahoo.co.in Date:Sun, 7 Jun, 2015 at 1:54 am Subject:Hundreds of sstables after every Repair Hi, We are using 2.0.3 and vnodes. After every repair -pr operation 50+ tiny sstables( 10K) get created. And these sstables never get compacted due to coldness issue. I have raised https://issues.apache.org/jira/browse/CASSANDRA-9146 for this issue but I have been told to upgrade. Till we upgrade to latest 2.0.x , we are stuck. Upgrade takes time, testing and planning in Production systems :( I have observed that even if vnodes are NOT damaged, hundreds of tiny sstables are created during repair for a wide row CF. This is beyond my understanding. If everything is consistent, and for the entire repair process Cassandra is saying Endpoints /x.x.x.x and /x.x.x.y are consistent for CF. Whats the need of creating sstables? Is there any alternative to regular major compaction to deal with situation? ThanksAnuj Wadehra | | | | | | | | | | | --
Re: C* 2.0.15 - java.lang.NegativeArraySizeException
Quick update, saw the same error on another new node, again the node isn't really misbehaving uptill now. Thanks On Mon, Jun 8, 2015 at 9:48 PM, Aiman Parvaiz ai...@flipagram.com wrote: Hi everyone I am running C* 2.0.9 and decided to do a rolling upgrade. Added a node of C* 2.0.15 in the existing cluster and saw this twice: Jun 9 02:27:20 prod-cass23.localdomain cassandra: 2015-06-09 02:27:20,658 INFO CompactionExecutor:4 CompactionTask.runMayThrow - Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-37-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-40-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-42-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-38-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-39-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-44-Data.db')] Jun 9 02:27:20 prod-cass23.localdomain cassandra: 2015-06-09 02:27:20,669 ERROR CompactionExecutor:4 CassandraDaemon.uncaughtException - Exception in thread Thread[CompactionExecutor:4,1,main] Jun 9 02:27:20 prod-cass23.localdomain *java.lang.NegativeArraySizeException* Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:335) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:462) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:448) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:432) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableReader.getAncestors(SSTableReader.java:1366) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata.createCollector(SSTableMetadata.java:134) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionTask.createCompactionWriter(CompactionTask.java:316) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:162) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.FutureTask.run(FutureTask.java:262) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) Jun 9 02:27:20 prod-cass23.localdomain at java.lang.Thread.run(Thread.java:745) Jun 9 02:27:47 prod-cass23.localdomain cassandra: 2015-06-09 02:27:47,725 INFO main StorageService.setMode - JOINING: Starting to bootstrap... As you can see this happened first time even before Joining. Second occasion stack trace: Jun 9 02:32:15 prod-cass23.localdomain cassandra: 2015-06-09 02:32:15,097 ERROR CompactionExecutor:6 CassandraDaemon.uncaughtException - Exception in thread Thread[CompactionExecutor:6,1,main] Jun 9 02:32:15 prod-cass23.localdomain java.lang.NegativeArraySizeException Jun 9 02:32:15 prod-cass23.localdomain at org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:335) Jun 9 02:32:15 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:462) Jun 9 02:32:15 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:448) Jun 9 02:32:15 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:432) Jun 9 02:32:15 prod-cass23.localdomain at
Re: Restoring all cluster from snapshots
On Tue, Jun 9, 2015 at 6:32 AM, Anton Koshevoy nowa...@gmail.com wrote: - /etc/init.d/cassandra restart Cassandra reads and makes live any files it finds in its data directory. If you write to the keyspace and flush, is it flushing the new file in the directory you've put your sstable in? Permissions? =Rob
RE: C* 2.0.15 - java.lang.NegativeArraySizeException
In my experience, you don’t want to do streaming operations (repairs or bootstraps) with mixed Cassandra versions. Upgrade the ring to the new version, and then add nodes (or add the nodes at the current version, and then upgrade). Sean Durity From: Aiman Parvaiz [mailto:ai...@flipagram.com] Sent: Tuesday, June 09, 2015 1:29 PM To: user@cassandra.apache.org Subject: Re: C* 2.0.15 - java.lang.NegativeArraySizeException Quick update, saw the same error on another new node, again the node isn't really misbehaving uptill now. Thanks On Mon, Jun 8, 2015 at 9:48 PM, Aiman Parvaiz ai...@flipagram.commailto:ai...@flipagram.com wrote: Hi everyone I am running C* 2.0.9 and decided to do a rolling upgrade. Added a node of C* 2.0.15 in the existing cluster and saw this twice: Jun 9 02:27:20 prod-cass23.localdomain cassandra: 2015-06-09 02:27:20,658 INFO CompactionExecutor:4 CompactionTask.runMayThrow - Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-37-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-40-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-42-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-38-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-39-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-44-Data.db')] Jun 9 02:27:20 prod-cass23.localdomain cassandra: 2015-06-09 02:27:20,669 ERROR CompactionExecutor:4 CassandraDaemon.uncaughtException - Exception in thread Thread[CompactionExecutor:4,1,main] Jun 9 02:27:20 prod-cass23.localdomain java.lang.NegativeArraySizeException Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:335) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:462) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:448) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:432) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableReader.getAncestors(SSTableReader.java:1366) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata.createCollector(SSTableMetadata.java:134) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionTask.createCompactionWriter(CompactionTask.java:316) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:162) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.FutureTask.run(FutureTask.java:262) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) Jun 9 02:27:20 prod-cass23.localdomain at java.lang.Thread.run(Thread.java:745) Jun 9 02:27:47 prod-cass23.localdomain cassandra: 2015-06-09 02:27:47,725 INFO main StorageService.setMode - JOINING: Starting to bootstrap... As you can see this happened first time even before Joining. Second occasion stack trace: Jun 9 02:32:15 prod-cass23.localdomain cassandra: 2015-06-09 02:32:15,097 ERROR CompactionExecutor:6 CassandraDaemon.uncaughtException - Exception in thread Thread[CompactionExecutor:6,1,main] Jun 9 02:32:15 prod-cass23.localdomain java.lang.NegativeArraySizeException Jun 9 02:32:15 prod-cass23.localdomain at org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:335) Jun 9 02:32:15 prod-cass23.localdomain at
Re: sstableloader usage doubts
On Tue, Jun 9, 2015 at 1:48 AM, ZeroUno zerozerouno...@gmail.com wrote: As far as I read from the docs, bootstrapping happens when adding a new node to the cluster, but in my situation the nodes already exist, I'm only adding data back into them. If you don't have the contents of the system keyspace, there is a non-zero chance of you bootstrapping in some cases. Also I have all 4 nodes configured as seeds in cassandra.yaml, so if I'm not wrong this should prevent them from auto-bootstrapping. Yes. =Rob
Apache Cassandra 2.2.0-rc1: calling all Windows users
With the upcoming release of Cassandra-2.2 Windows is finally an officially supported operating system. While many months of JIRA tickets https://issues.apache.org/jira/issues/?filter=-1jql=project%20%3D%20cassandra%20and%20labels%20%3D%20Windows, bug fixes, and contributions have gone into making Cassandra on Windows as seamless of an experience as possible, we need your help as the community to further kick the tires and let us know if you run into any problems. Please let us know https://issues.apache.org/jira/browse/CASSANDRA/ if you find anything that's not working on the platform so we can keep Cassandra on Windows running strong! -- Joshua McKenzie
Re: C* 2.0.15 - java.lang.NegativeArraySizeException
On Tue, Jun 9, 2015 at 2:35 PM, Aiman Parvaiz ai...@flipagram.com wrote: Thanks Sean, in this scenario also I would end up running 2 versions of Cassandra as I am planning to do a rolling upgrade and hence zero downtime. Upgrading in place one node at a time would lead to running 2 versions, please let me know if I am missing something here. Running a cluster with nodes of two different versions during a rolling upgrade, for the duration of the upgrade, is (mostly) supported [1]. Modifying cluster topology (adding or removing or replacing nodes) during such an upgrade is not. It is a fair statement that with very large clusters and very slow upgradesstables, the ability of any operator to operate in this manner approaches not-possible. I don't know how people with truly huge clusters deal with this race. =Rob [1] some operations are not supported possible with some combinations of versions. for example one cannot repair in some cases.
Re: C* 2.0.15 - java.lang.NegativeArraySizeException
Thanks Sean, in this scenario also I would end up running 2 versions of Cassandra as I am planning to do a rolling upgrade and hence zero downtime. Upgrading in place one node at a time would lead to running 2 versions, please let me know if I am missing something here. On Tue, Jun 9, 2015 at 2:00 PM, sean_r_dur...@homedepot.com wrote: In my experience, you don’t want to do streaming operations (repairs or bootstraps) with mixed Cassandra versions. Upgrade the ring to the new version, and then add nodes (or add the nodes at the current version, and then upgrade). Sean Durity *From:* Aiman Parvaiz [mailto:ai...@flipagram.com] *Sent:* Tuesday, June 09, 2015 1:29 PM *To:* user@cassandra.apache.org *Subject:* Re: C* 2.0.15 - java.lang.NegativeArraySizeException Quick update, saw the same error on another new node, again the node isn't really misbehaving uptill now. Thanks On Mon, Jun 8, 2015 at 9:48 PM, Aiman Parvaiz ai...@flipagram.com wrote: Hi everyone I am running C* 2.0.9 and decided to do a rolling upgrade. Added a node of C* 2.0.15 in the existing cluster and saw this twice: Jun 9 02:27:20 prod-cass23.localdomain cassandra: 2015-06-09 02:27:20,658 INFO CompactionExecutor:4 CompactionTask.runMayThrow - Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-37-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-40-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-42-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-38-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-39-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-jb-44-Data.db')] Jun 9 02:27:20 prod-cass23.localdomain cassandra: 2015-06-09 02:27:20,669 ERROR CompactionExecutor:4 CassandraDaemon.uncaughtException - Exception in thread Thread[CompactionExecutor:4,1,main] Jun 9 02:27:20 prod-cass23.localdomain *java.lang.NegativeArraySizeException* Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:335) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:462) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:448) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:432) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableReader.getAncestors(SSTableReader.java:1366) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.io.sstable.SSTableMetadata.createCollector(SSTableMetadata.java:134) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionTask.createCompactionWriter(CompactionTask.java:316) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:162) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) Jun 9 02:27:20 prod-cass23.localdomain at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.FutureTask.run(FutureTask.java:262) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) Jun 9 02:27:20 prod-cass23.localdomain at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) Jun 9 02:27:20 prod-cass23.localdomain at java.lang.Thread.run(Thread.java:745) Jun 9 02:27:47 prod-cass23.localdomain cassandra: 2015-06-09 02:27:47,725 INFO main StorageService.setMode - JOINING: Starting to bootstrap... As you can see this happened first time even before Joining. Second occasion stack trace: Jun 9 02:32:15 prod-cass23.localdomain cassandra: 2015-06-09 02:32:15,097 ERROR CompactionExecutor:6 CassandraDaemon.uncaughtException - Exception in thread
Re: Support for ad-hoc query
what do you mean by ad-hoc queries? Do you mean simple queries against a single column family aka table? Or do you mean MDX style queries that looks at multiple tables? if it's MDX style queries, many people extract data from Cassandra into a data warehouse that support multi-dimensional cubes. This works well when the extracted data is a small subset and fits neatly in a data warehouse. As others have stated, Cassandra isn't great at ad-hoc. For MDX style queries, Cassandra wasn't designed for it. One thing we've done for our own project is to combine solr with our own fuzzy index to make ad-hoc queries against a single table more friendly. On Tue, Jun 9, 2015 at 2:38 AM, Srinivasa T N seen...@gmail.com wrote: Hi All, I have an web application running with my backend data stored in cassandra. Now I want to do some analysis on the data stored which requires some ad-hoc queries fired on cassandra. How can I do the same? Regards, Seenu.
Re: Support for ad-hoc query
Thanks guys for the inputs. By ad-hoc queries I mean that I don't know the queries during cf design time. The data may be from single cf or multiple cf. (This feature maybe required if I want to do analysis on the data stored in cassandra, do you have any better ideas)? Regards, Seenu. On Tue, Jun 9, 2015 at 5:57 PM, Peter Lin wool...@gmail.com wrote: what do you mean by ad-hoc queries? Do you mean simple queries against a single column family aka table? Or do you mean MDX style queries that looks at multiple tables? if it's MDX style queries, many people extract data from Cassandra into a data warehouse that support multi-dimensional cubes. This works well when the extracted data is a small subset and fits neatly in a data warehouse. As others have stated, Cassandra isn't great at ad-hoc. For MDX style queries, Cassandra wasn't designed for it. One thing we've done for our own project is to combine solr with our own fuzzy index to make ad-hoc queries against a single table more friendly. On Tue, Jun 9, 2015 at 2:38 AM, Srinivasa T N seen...@gmail.com wrote: Hi All, I have an web application running with my backend data stored in cassandra. Now I want to do some analysis on the data stored which requires some ad-hoc queries fired on cassandra. How can I do the same? Regards, Seenu.
Re: Restoring all cluster from snapshots
Ok, new sequence: …. - rm -rf /db/cassandra/cr/data0*/system/* - /etc/init.d/cassandra start - cqlsh -f schema(here is the schema for cr_production keyspace only) - /etc/init.d/cassandra restart I see the new keyspace, but it’s empty: cqlsh describe keyspaces; system_traces cr_production system cqlsh use cr_production ; cqlsh:cr_production select * from cr_production.url_cs ; id | ct | d_id +-+--- (0 rows) = Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns Host ID Rack UN 10.40.231.3 74.35 KB 256 ? 7fc4f832-9d98-4bfd-81fb-0c361bff487d RACK01 UN 10.40.231.31 86.46 KB 256 ? 782d450d-50d5-4738-a654-bf7398557842 RACK01 Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless then I tried: - nodetool refresh -- cr_production url_cs with the same result. On June 9, 2015 at 1:29:36 AM, Robert Coli (rc...@eventbrite.com) wrote: On Mon, Jun 8, 2015 at 2:52 PM, Sanjay Baronia sanjay.baro...@triliodata.com wrote: Yes, you shouldn’t delete the system directory. Next steps are …reconfigure the test cluster with new IP addresses, clear the gossiping information and then boot the test cluster. If you don't delete the system directory, you run the risk of the test cluster nodes joining the source cluster. Just start a single node on the new cluster, empty, and create the schema on it. Then do the rest of the process. =Rob
