Re: Out of Memory Error While Opening SSTables on Startup
Paul Nickerson; curious, did you get a solution to your problem ? Regards,Jan/ On Tuesday, February 10, 2015 5:48 PM, Flavien Charlon flavien.char...@gmail.com wrote: I already experienced the same problem (hundreds of thousands of SSTables) with Cassandra 2.1.2. It seems to appear when running an incremental repair while there is a medium to high insert load on the cluster. The repair goes in a bad state and starts creating way more SSTables than it should (even when there should be nothing to repair). On 10 February 2015 at 15:46, Eric Stevens migh...@gmail.com wrote: This kind of recovery is definitely not my strong point, so feedback on this approach would certainly be welcome. As I understand it, if you really want to keep that data, you ought to be able to mv it out of the way to get your node online, then move those files in a several thousand at a time, nodetool refresh OpsCenter rollups60 nodetool compact OpsCenter rollups60; rinse and repeat. This should let you incrementally restore the data in that keyspace without putting so many sstables in there that it ooms your cluster again. On Tue, Feb 10, 2015 at 3:38 PM, Chris Lohfink clohfin...@gmail.com wrote: yeah... probably just 2.1.2 things and not compactions. Still probably want to do something about the 1.6 million files though. It may be worth just mv/rm'ing to 60 sec rollup data though unless really attached to it. Chris On Tue, Feb 10, 2015 at 4:04 PM, Paul Nickerson pgn...@gmail.com wrote: I was having trouble with snapshots failing while trying to repair that table (http://www.mail-archive.com/user@cassandra.apache.org/msg40686.html). I have a repair running on it now, and it seems to be going successfully this time. I am going to wait for that to finish, then try a manual nodetool compact. If that goes successfully, then would it be safe to chalk the lack of compaction on this table in the past up to 2.1.2 problems? ~ Paul Nickerson On Tue, Feb 10, 2015 at 3:34 PM, Chris Lohfink clohfin...@gmail.com wrote: Your cluster is probably having issues with compactions (with STCS you should never have this many). I would probably punt with OpsCenter/rollups60. Turn the node off and move all of the sstables off to a different directory for backup (or just rm if you really don't care about 1 minute metrics), than turn the server back on. Once you get your cluster running again go back and investigate why compactions stopped, my guess is you hit an exception in past that killed your CompactionExecutor and things just built up slowly until you got to this point. Chris On Tue, Feb 10, 2015 at 2:15 PM, Paul Nickerson pgn...@gmail.com wrote: Thank you Rob. I tried a 12 GiB heap size, and still crashed out. There are 1,617,289 files under OpsCenter/rollups60. Once I downgraded Cassandra to 2.1.1 (apt-get install cassandra=2.1.1), I was able to start up Cassandra OK with the default heap size formula. Now my cluster is running multiple versions of Cassandra. I think I will downgrade the rest to 2.1.1. ~ Paul Nickerson On Tue, Feb 10, 2015 at 2:05 PM, Robert Coli rc...@eventbrite.com wrote: On Tue, Feb 10, 2015 at 11:02 AM, Paul Nickerson pgn...@gmail.com wrote: I am getting an out of memory error why I try to start Cassandra on one of my nodes. Cassandra will run for a minute, and then exit without outputting any error in the log file. It is happening while SSTableReader is opening a couple hundred thousand things. ... Does anyone know how I might get Cassandra on this node running again? I'm not very familiar with correctly tuning Java memory parameters, and I'm not sure if that's the right solution in this case anyway. Try running 2.1.1, and/or increasing heap size beyond 8gb. Are there actually that many SSTables on disk? =Rob
Re: Adding new node to Cassandra cluster is too slow
You won't lose data unless you have run nodetool cleanup on the existing nodes. Rahul On Mar 19, 2015, at 9:16 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: Also, the new nodes (3 of them, in UJ state) are showing some data size (~10g). Is there any data loss chances with stopping the cassandra on them? On Thu, Mar 19, 2015 at 6:02 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: Thanks Rob, You are right. I am using ReleaseVersion: 2.1.0 What do you mean by point 3? Also, by doing one at a time, does it mean wait till nodetool status of the new node is UN from UJ? On Thu, Mar 19, 2015 at 5:44 PM, Robert Coli rc...@eventbrite.com wrote: On Thu, Mar 19, 2015 at 5:32 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: I have 14 nodes cassandra cluster, each node as around 50gb of data. I added 3 new nodes to the cluster and I can see the status as UJ for the new nodes. They have been in that for almost a day now and their data size seems to be same as well. There is almost no CPU or disk usage either on them. It is not supported to add multiple nodes to a cluster simultaneously until 2.1.1 [1]. Usually what happens is one or more of the bootstraps fails and hangs forever. This seems to be what has happened to you. To resolve : 1) stop each of the bootstrapping nodes 2) wipe their data directories completey 3) verify that they do not show up in gossip on the other nodes 4) bootstrap them again, one at a time =Rob [1] https://issues.apache.org/jira/browse/CASSANDRA-7069
Re: Adding new node to Cassandra cluster is too slow
Thanks Rob. Anyway, Ideally for a new node to join with ~50GB data of it's share, it should be done in couple of minutes or hour tops, right? On Fri, Mar 20, 2015 at 6:07 PM, Robert Coli rc...@eventbrite.com wrote: On Fri, Mar 20, 2015 at 4:08 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: Also, the very fact the new nodes get some data (~10gb) and then get stuck, the configurations and process I am using must be correct, and there is no immediate fix besides upgrade the version? That is correct, it sounds like a correct config and then streaming fails. =Rob
Re: Store data with cassandra
You seem to be missing the point here. Cassandra does not manage files, it manages data in a highly distributed cluster. If you are attempting to manage files, you are quite simply using the wrong tool and Cassandra is not for you. On Fri, Mar 20, 2015 at 9:10 AM, jean paul researche...@gmail.com wrote: I have used this tutoriel to create my data base http://planetcassandra.org/insert-select-records/ /var/lib/cassandra/data# ls demo system system_traces :/var/lib/cassandra/data# cd demo/ :/var/lib/cassandra/data/demo# ls users :/var/lib/cassandra/data/demo# cd users/ :/var/lib/cassandra/data/demo/users# ls :/var/lib/cassandra/data/demo/users# i find nothing in /var/lib/cassandra/data/demo/users! 2015-03-20 13:06 GMT+01:00 jean paul researche...@gmail.com: Hello All; Please, i have created this table. lastname | age | city | email | firstname --+-+---+-+--- Doe | 36 | Beverly Hills | jane...@email.com | Jane Byrne | 24 | San Diego | robby...@email.com | Rob Smith | 46 | Sacramento | johnsm...@email.com | John So, my question, where this data is saved ? in ./var/lib/cassandra/data ? My end goal is to store a file with cassandra and to see on which node my file is stored ? thanks a lot for help Best Regards. -- - michael dykman - mdyk...@gmail.com May the Source be with you.
Re: Store data with cassandra
I have used this tutoriel to create my data base http://planetcassandra.org/insert-select-records/ /var/lib/cassandra/data# ls demo system system_traces :/var/lib/cassandra/data# cd demo/ :/var/lib/cassandra/data/demo# ls users :/var/lib/cassandra/data/demo# cd users/ :/var/lib/cassandra/data/demo/users# ls :/var/lib/cassandra/data/demo/users# i find nothing in /var/lib/cassandra/data/demo/users! 2015-03-20 13:06 GMT+01:00 jean paul researche...@gmail.com: Hello All; Please, i have created this table. lastname | age | city | email | firstname --+-+---+-+--- Doe | 36 | Beverly Hills | jane...@email.com | Jane Byrne | 24 | San Diego | robby...@email.com | Rob Smith | 46 | Sacramento | johnsm...@email.com | John So, my question, where this data is saved ? in ./var/lib/cassandra/data ? My end goal is to store a file with cassandra and to see on which node my file is stored ? thanks a lot for help Best Regards.
Re: Store data with cassandra
What about this so http://www.datastax.com/wp-content/uploads/2012/02/Screen-Shot-2012-02-10-at-11.21.55-AM.png i read also some documents about storing blob with cassandra !! 2015-03-20 15:04 GMT+01:00 Michael Dykman mdyk...@gmail.com: You seem to be missing the point here. Cassandra does not manage files, it manages data in a highly distributed cluster. If you are attempting to manage files, you are quite simply using the wrong tool and Cassandra is not for you. On Fri, Mar 20, 2015 at 9:10 AM, jean paul researche...@gmail.com wrote: I have used this tutoriel to create my data base http://planetcassandra.org/insert-select-records/ /var/lib/cassandra/data# ls demo system system_traces :/var/lib/cassandra/data# cd demo/ :/var/lib/cassandra/data/demo# ls users :/var/lib/cassandra/data/demo# cd users/ :/var/lib/cassandra/data/demo/users# ls :/var/lib/cassandra/data/demo/users# i find nothing in /var/lib/cassandra/data/demo/users! 2015-03-20 13:06 GMT+01:00 jean paul researche...@gmail.com: Hello All; Please, i have created this table. lastname | age | city | email | firstname --+-+---+-+--- Doe | 36 | Beverly Hills | jane...@email.com | Jane Byrne | 24 | San Diego | robby...@email.com | Rob Smith | 46 | Sacramento | johnsm...@email.com | John So, my question, where this data is saved ? in ./var/lib/cassandra/data ? My end goal is to store a file with cassandra and to see on which node my file is stored ? thanks a lot for help Best Regards. -- - michael dykman - mdyk...@gmail.com May the Source be with you.
Re: Store data with cassandra
The files you store have to personally be vetted by the cassandra community. Only if they're found to not contain anything inappropriate, does cassandra let you store them. (A 3/4 majority vote is necessary). Please send your files for approval to j...@reallycereal.com On Fri, Mar 20, 2015 at 8:41 PM, jean paul researche...@gmail.com wrote: What about this so http://www.datastax.com/wp-content/uploads/2012/02/Screen-Shot-2012-02-10-at-11.21.55-AM.png i read also some documents about storing blob with cassandra !! 2015-03-20 15:04 GMT+01:00 Michael Dykman mdyk...@gmail.com: You seem to be missing the point here. Cassandra does not manage files, it manages data in a highly distributed cluster. If you are attempting to manage files, you are quite simply using the wrong tool and Cassandra is not for you. On Fri, Mar 20, 2015 at 9:10 AM, jean paul researche...@gmail.com wrote: I have used this tutoriel to create my data base http://planetcassandra.org/insert-select-records/ /var/lib/cassandra/data# ls demo system system_traces :/var/lib/cassandra/data# cd demo/ :/var/lib/cassandra/data/demo# ls users :/var/lib/cassandra/data/demo# cd users/ :/var/lib/cassandra/data/demo/users# ls :/var/lib/cassandra/data/demo/users# i find nothing in /var/lib/cassandra/data/demo/users! 2015-03-20 13:06 GMT+01:00 jean paul researche...@gmail.com: Hello All; Please, i have created this table. lastname | age | city | email | firstname --+-+---+-+--- Doe | 36 | Beverly Hills | jane...@email.com | Jane Byrne | 24 | San Diego | robby...@email.com | Rob Smith | 46 |Sacramento | johnsm...@email.com | John So, my question, where this data is saved ? in ./var/lib/cassandra/data ? My end goal is to store a file with cassandra and to see on which node my file is stored ? thanks a lot for help Best Regards. -- - michael dykman - mdyk...@gmail.com May the Source be with you.
Re: Store data with cassandra
It has been decided that the file cannot be allowed to be stored, sorry. However, if a sacrifice to the gods is prepared, it may be possible to change things. On Fri, Mar 20, 2015 at 8:49 PM, jean paul researche...@gmail.com wrote: i'd like to store MyFile.txt using cassandra (replicat = 2) and see on what node the file and its replicas are stored on my cluster of 10 nodes it is a simple file with simple content (text) is that possible ? 2015-03-20 16:44 GMT+01:00 Ali Akhtar ali.rac...@gmail.com: The files you store have to personally be vetted by the cassandra community. Only if they're found to not contain anything inappropriate, does cassandra let you store them. (A 3/4 majority vote is necessary). Please send your files for approval to j...@reallycereal.com On Fri, Mar 20, 2015 at 8:41 PM, jean paul researche...@gmail.com wrote: What about this so http://www.datastax.com/wp-content/uploads/2012/02/Screen-Shot-2012-02-10-at-11.21.55-AM.png i read also some documents about storing blob with cassandra !! 2015-03-20 15:04 GMT+01:00 Michael Dykman mdyk...@gmail.com: You seem to be missing the point here. Cassandra does not manage files, it manages data in a highly distributed cluster. If you are attempting to manage files, you are quite simply using the wrong tool and Cassandra is not for you. On Fri, Mar 20, 2015 at 9:10 AM, jean paul researche...@gmail.com wrote: I have used this tutoriel to create my data base http://planetcassandra.org/insert-select-records/ /var/lib/cassandra/data# ls demo system system_traces :/var/lib/cassandra/data# cd demo/ :/var/lib/cassandra/data/demo# ls users :/var/lib/cassandra/data/demo# cd users/ :/var/lib/cassandra/data/demo/users# ls :/var/lib/cassandra/data/demo/users# i find nothing in /var/lib/cassandra/data/demo/users! 2015-03-20 13:06 GMT+01:00 jean paul researche...@gmail.com: Hello All; Please, i have created this table. lastname | age | city | email | firstname --+-+---+-+--- Doe | 36 | Beverly Hills | jane...@email.com | Jane Byrne | 24 | San Diego | robby...@email.com | Rob Smith | 46 |Sacramento | johnsm...@email.com | John So, my question, where this data is saved ? in ./var/lib/cassandra/data ? My end goal is to store a file with cassandra and to see on which node my file is stored ? thanks a lot for help Best Regards. -- - michael dykman - mdyk...@gmail.com May the Source be with you.
Re: Store data with cassandra
i'd like to store MyFile.txt using cassandra (replicat = 2) and see on what node the file and its replicas are stored on my cluster of 10 nodes it is a simple file with simple content (text) is that possible ? 2015-03-20 16:44 GMT+01:00 Ali Akhtar ali.rac...@gmail.com: The files you store have to personally be vetted by the cassandra community. Only if they're found to not contain anything inappropriate, does cassandra let you store them. (A 3/4 majority vote is necessary). Please send your files for approval to j...@reallycereal.com On Fri, Mar 20, 2015 at 8:41 PM, jean paul researche...@gmail.com wrote: What about this so http://www.datastax.com/wp-content/uploads/2012/02/Screen-Shot-2012-02-10-at-11.21.55-AM.png i read also some documents about storing blob with cassandra !! 2015-03-20 15:04 GMT+01:00 Michael Dykman mdyk...@gmail.com: You seem to be missing the point here. Cassandra does not manage files, it manages data in a highly distributed cluster. If you are attempting to manage files, you are quite simply using the wrong tool and Cassandra is not for you. On Fri, Mar 20, 2015 at 9:10 AM, jean paul researche...@gmail.com wrote: I have used this tutoriel to create my data base http://planetcassandra.org/insert-select-records/ /var/lib/cassandra/data# ls demo system system_traces :/var/lib/cassandra/data# cd demo/ :/var/lib/cassandra/data/demo# ls users :/var/lib/cassandra/data/demo# cd users/ :/var/lib/cassandra/data/demo/users# ls :/var/lib/cassandra/data/demo/users# i find nothing in /var/lib/cassandra/data/demo/users! 2015-03-20 13:06 GMT+01:00 jean paul researche...@gmail.com: Hello All; Please, i have created this table. lastname | age | city | email | firstname --+-+---+-+--- Doe | 36 | Beverly Hills | jane...@email.com | Jane Byrne | 24 | San Diego | robby...@email.com | Rob Smith | 46 |Sacramento | johnsm...@email.com | John So, my question, where this data is saved ? in ./var/lib/cassandra/data ? My end goal is to store a file with cassandra and to see on which node my file is stored ? thanks a lot for help Best Regards. -- - michael dykman - mdyk...@gmail.com May the Source be with you.
Store data with cassandra
Hello All; Please, i have created this table. lastname | age | city | email | firstname --+-+---+-+--- Doe | 36 | Beverly Hills | jane...@email.com | Jane Byrne | 24 | San Diego | robby...@email.com | Rob Smith | 46 | Sacramento | johnsm...@email.com | John So, my question, where this data is saved ? in ./var/lib/cassandra/data ? My end goal is to store a file with cassandra and to see on which node my file is stored ? thanks a lot for help Best Regards.
Re: Cassandra cluster Too high DISK IOs
The cluster is processing something like 12k reads and 2k writes/seconds. The disks are locally attached and latency is just fine. It's the number of disk iops that's too high. On Fri, Mar 20, 2015 at 11:05 AM, Ali Akhtar ali.rac...@gmail.com wrote: That probably depends on how many read / write queries your cluster is processing? Also, since you mentioned provisoned IOPS, are you using EBS for storing the data? If so, you probably want to switch to the ephemeral storage since its locally attached to the instance and doesn't require a network call for each lookup. On Fri, Mar 20, 2015 at 10:57 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: Hi All. I am using 15 nodes cassandra cluster(m3.2xlarge) with provisioned IOPS disks (4000). I can see around 12k reads/sec ops on the cassandra cluster. But I see around *~3500 read IOPS* on each of the cassandra nodes. Is that normal? I am using LevelledCompaction and I can see in the histograms that most read requests are coming from 1/2 sstables only. Why are there so many disk IOPS or is the normal for cassandra? -Pranay
Re: Adding new node to Cassandra cluster is too slow
Also, typically how long does it take for a node to join? I have in total 1 TB of data in 15 nodes cassandra cluster. On Fri, Mar 20, 2015 at 10:53 AM, Pranay Agarwal agarwalpran...@gmail.com wrote: Thank Rahul, you are right. Unless the node complete joins the ring, there is no data dependency on them. On Fri, Mar 20, 2015 at 4:46 AM, Rahul Neelakantan ra...@rahul.be wrote: You won't lose data unless you have run nodetool cleanup on the existing nodes. Rahul On Mar 19, 2015, at 9:16 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: Also, the new nodes (3 of them, in *UJ state*) are showing some data size (~10g). Is there any data loss chances with stopping the cassandra on them? On Thu, Mar 19, 2015 at 6:02 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: Thanks Rob, You are right. I am using ReleaseVersion: 2.1.0 What do you mean by point 3? Also, by doing one at a time, does it mean wait till nodetool status of the new node is UN from UJ? On Thu, Mar 19, 2015 at 5:44 PM, Robert Coli rc...@eventbrite.com wrote: On Thu, Mar 19, 2015 at 5:32 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: I have 14 nodes cassandra cluster, each node as around 50gb of data. I added 3 new nodes to the cluster and I can see the status as *UJ *for the new nodes. They have been in that for almost a day now and their data size seems to be same as well. There is almost no CPU or disk usage either on them. It is not supported to add multiple nodes to a cluster simultaneously until 2.1.1 [1]. Usually what happens is one or more of the bootstraps fails and hangs forever. This seems to be what has happened to you. To resolve : 1) stop each of the bootstrapping nodes 2) wipe their data directories completey 3) verify that they do not show up in gossip on the other nodes 4) bootstrap them again, one at a time =Rob [1] https://issues.apache.org/jira/browse/CASSANDRA-7069
Re: Cassandra cluster Too high DISK IOs
That probably depends on how many read / write queries your cluster is processing? Also, since you mentioned provisoned IOPS, are you using EBS for storing the data? If so, you probably want to switch to the ephemeral storage since its locally attached to the instance and doesn't require a network call for each lookup. On Fri, Mar 20, 2015 at 10:57 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: Hi All. I am using 15 nodes cassandra cluster(m3.2xlarge) with provisioned IOPS disks (4000). I can see around 12k reads/sec ops on the cassandra cluster. But I see around *~3500 read IOPS* on each of the cassandra nodes. Is that normal? I am using LevelledCompaction and I can see in the histograms that most read requests are coming from 1/2 sstables only. Why are there so many disk IOPS or is the normal for cassandra? -Pranay
Re: Adding new node to Cassandra cluster is too slow
Thank Rahul, you are right. Unless the node complete joins the ring, there is no data dependency on them. On Fri, Mar 20, 2015 at 4:46 AM, Rahul Neelakantan ra...@rahul.be wrote: You won't lose data unless you have run nodetool cleanup on the existing nodes. Rahul On Mar 19, 2015, at 9:16 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: Also, the new nodes (3 of them, in *UJ state*) are showing some data size (~10g). Is there any data loss chances with stopping the cassandra on them? On Thu, Mar 19, 2015 at 6:02 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: Thanks Rob, You are right. I am using ReleaseVersion: 2.1.0 What do you mean by point 3? Also, by doing one at a time, does it mean wait till nodetool status of the new node is UN from UJ? On Thu, Mar 19, 2015 at 5:44 PM, Robert Coli rc...@eventbrite.com wrote: On Thu, Mar 19, 2015 at 5:32 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: I have 14 nodes cassandra cluster, each node as around 50gb of data. I added 3 new nodes to the cluster and I can see the status as *UJ *for the new nodes. They have been in that for almost a day now and their data size seems to be same as well. There is almost no CPU or disk usage either on them. It is not supported to add multiple nodes to a cluster simultaneously until 2.1.1 [1]. Usually what happens is one or more of the bootstraps fails and hangs forever. This seems to be what has happened to you. To resolve : 1) stop each of the bootstrapping nodes 2) wipe their data directories completey 3) verify that they do not show up in gossip on the other nodes 4) bootstrap them again, one at a time =Rob [1] https://issues.apache.org/jira/browse/CASSANDRA-7069
Cassandra cluster Too high DISK IOs
Hi All. I am using 15 nodes cassandra cluster(m3.2xlarge) with provisioned IOPS disks (4000). I can see around 12k reads/sec ops on the cassandra cluster. But I see around *~3500 read IOPS* on each of the cassandra nodes. Is that normal? I am using LevelledCompaction and I can see in the histograms that most read requests are coming from 1/2 sstables only. Why are there so many disk IOPS or is the normal for cassandra? -Pranay
Re: Store data with cassandra
Jean, I'm not sure you will receive any reply unless you ask specific questions about those links. Cheers, Jens – Skickat från Mailbox On Fri, Mar 20, 2015 at 5:08 PM, Sibbald, Charles charles.sibb...@bskyb.com wrote: Sounds like this is a job for jackrabbit ? http://jackrabbit.apache.org From: Ali Akhtar ali.rac...@gmail.commailto:ali.rac...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Friday, 20 March 2015 15:58 To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: Store data with cassandra ( I apologize, I'm only joking. To answer your question, Cassandra tends to cache the first 300MB or so of data in memory, only when it grows beyond that does it start to write it to files. But, Cassandra is not the write choice for storing files. In the screenshot you linked, its only storing the filenames, not the actual contents of the files). On Fri, Mar 20, 2015 at 8:54 PM, Ali Akhtar ali.rac...@gmail.commailto:ali.rac...@gmail.com wrote: It has been decided that the file cannot be allowed to be stored, sorry. However, if a sacrifice to the gods is prepared, it may be possible to change things. On Fri, Mar 20, 2015 at 8:49 PM, jean paul researche...@gmail.commailto:researche...@gmail.com wrote: i'd like to store MyFile.txt using cassandra (replicat = 2) and see on what node the file and its replicas are stored on my cluster of 10 nodes it is a simple file with simple content (text) is that possible ? 2015-03-20 16:44 GMT+01:00 Ali Akhtar ali.rac...@gmail.commailto:ali.rac...@gmail.com: The files you store have to personally be vetted by the cassandra community. Only if they're found to not contain anything inappropriate, does cassandra let you store them. (A 3/4 majority vote is necessary). Please send your files for approval to j...@reallycereal.commailto:j...@reallycereal.com On Fri, Mar 20, 2015 at 8:41 PM, jean paul researche...@gmail.commailto:researche...@gmail.com wrote: What about this so http://www.datastax.com/wp-content/uploads/2012/02/Screen-Shot-2012-02-10-at-11.21.55-AM.png i read also some documents about storing blob with cassandra !! 2015-03-20 15:04 GMT+01:00 Michael Dykman mdyk...@gmail.commailto:mdyk...@gmail.com: You seem to be missing the point here. Cassandra does not manage files, it manages data in a highly distributed cluster. If you are attempting to manage files, you are quite simply using the wrong tool and Cassandra is not for you. On Fri, Mar 20, 2015 at 9:10 AM, jean paul researche...@gmail.commailto:researche...@gmail.com wrote: I have used this tutoriel to create my data base http://planetcassandra.org/insert-select-records/ /var/lib/cassandra/data# ls demo system system_traces :/var/lib/cassandra/data# cd demo/ :/var/lib/cassandra/data/demo# ls users :/var/lib/cassandra/data/demo# cd users/ :/var/lib/cassandra/data/demo/users# ls :/var/lib/cassandra/data/demo/users# i find nothing in /var/lib/cassandra/data/demo/users! 2015-03-20 13:06 GMT+01:00 jean paul researche...@gmail.commailto:researche...@gmail.com: Hello All; Please, i have created this table. lastname | age | city | email | firstname --+-+---+-+--- Doe | 36 | Beverly Hills | jane...@email.commailto:jane...@email.com | Jane Byrne | 24 | San Diego | robby...@email.commailto:robby...@email.com | Rob Smith | 46 | Sacramento | johnsm...@email.commailto:johnsm...@email.com | John So, my question, where this data is saved ? in ./var/lib/cassandra/data ? My end goal is to store afile with cassandra and to see on which node my file is stored ? thanks a lot for help Best Regards. -- - michael dykman - mdyk...@gmail.commailto:mdyk...@gmail.com May the Source be with you. Information in this email including any attachments may be privileged, confidential and is intended exclusively for the addressee. The views expressed may not be official policy, but the personal views of the originator. If you have received it in error, please notify the sender by return e-mail and delete it from your system. You should not reproduce, distribute, store, retransmit, use or disclose its contents to anyone. Please note we reserve the right to monitor all e-mail communication through our internal and external networks. SKY and the SKY marks are trademarks of Sky plc and Sky International AG and are used under licence. Sky UK Limited (Registration No. 2906991), Sky-In-Home Service Limited (Registration No. 2067075) and Sky Subscribers Services Limited (Registration No. 2340150) are direct or indirect subsidiaries of Sky plc (Registration No. 2247735). All of the companies mentioned in this paragraph are incorporated in
Re: Store data with cassandra
Please, can you explain details in this two links: https://wiki.apache.org/cassandra/FAQ#large_file_and_blob_storage http://stackoverflow.com/questions/9219222/how-to-store-small-files-in-cassandra
Re: Store data with cassandra
Please, can you explain details in this two links: https://wiki.apache.org/cassandra/FAQ#large_file_and_blob_storage http://stackoverflow.com/questions/9219222/how-to-store-small-files-in-cassandra 2015-03-20 17:07 GMT+01:00 Sibbald, Charles charles.sibb...@bskyb.com: Sounds like this is a job for jackrabbit ? http://jackrabbit.apache.org From: Ali Akhtar ali.rac...@gmail.com Reply-To: user@cassandra.apache.org user@cassandra.apache.org Date: Friday, 20 March 2015 15:58 To: user@cassandra.apache.org user@cassandra.apache.org Subject: Re: Store data with cassandra ( I apologize, I'm only joking. To answer your question, Cassandra tends to cache the first 300MB or so of data in memory, only when it grows beyond that does it start to write it to files. But, Cassandra is not the write choice for storing files. In the screenshot you linked, its only storing the filenames, not the actual contents of the files). On Fri, Mar 20, 2015 at 8:54 PM, Ali Akhtar ali.rac...@gmail.com wrote: It has been decided that the file cannot be allowed to be stored, sorry. However, if a sacrifice to the gods is prepared, it may be possible to change things. On Fri, Mar 20, 2015 at 8:49 PM, jean paul researche...@gmail.com wrote: i'd like to store MyFile.txt using cassandra (replicat = 2) and see on what node the file and its replicas are stored on my cluster of 10 nodes it is a simple file with simple content (text) is that possible ? 2015-03-20 16:44 GMT+01:00 Ali Akhtar ali.rac...@gmail.com: The files you store have to personally be vetted by the cassandra community. Only if they're found to not contain anything inappropriate, does cassandra let you store them. (A 3/4 majority vote is necessary). Please send your files for approval to j...@reallycereal.com On Fri, Mar 20, 2015 at 8:41 PM, jean paul researche...@gmail.com wrote: What about this so http://www.datastax.com/wp-content/uploads/2012/02/Screen-Shot-2012-02-10-at-11.21.55-AM.png i read also some documents about storing blob with cassandra !! 2015-03-20 15:04 GMT+01:00 Michael Dykman mdyk...@gmail.com: You seem to be missing the point here. Cassandra does not manage files, it manages data in a highly distributed cluster. If you are attempting to manage files, you are quite simply using the wrong tool and Cassandra is not for you. On Fri, Mar 20, 2015 at 9:10 AM, jean paul researche...@gmail.com wrote: I have used this tutoriel to create my data base http://planetcassandra.org/insert-select-records/ /var/lib/cassandra/data# ls demo system system_traces :/var/lib/cassandra/data# cd demo/ :/var/lib/cassandra/data/demo# ls users :/var/lib/cassandra/data/demo# cd users/ :/var/lib/cassandra/data/demo/users# ls :/var/lib/cassandra/data/demo/users# i find nothing in /var/lib/cassandra/data/demo/users! 2015-03-20 13:06 GMT+01:00 jean paul researche...@gmail.com: Hello All; Please, i have created this table. lastname | age | city | email | firstname --+-+---+-+--- Doe | 36 | Beverly Hills | jane...@email.com | Jane Byrne | 24 | San Diego | robby...@email.com | Rob Smith | 46 |Sacramento | johnsm...@email.com | John So, my question, where this data is saved ? in ./var/lib/cassandra/data ? My end goal is to store afile with cassandra and to see on which node my file is stored ? thanks a lot for help Best Regards. -- - michael dykman - mdyk...@gmail.com May the Source be with you. Information in this email including any attachments may be privileged, confidential and is intended exclusively for the addressee. The views expressed may not be official policy, but the personal views of the originator. If you have received it in error, please notify the sender by return e-mail and delete it from your system. You should not reproduce, distribute, store, retransmit, use or disclose its contents to anyone. Please note we reserve the right to monitor all e-mail communication through our internal and external networks. SKY and the SKY marks are trademarks of Sky plc and Sky International AG and are used under licence. Sky UK Limited (Registration No. 2906991), Sky-In-Home Service Limited (Registration No. 2067075) and Sky Subscribers Services Limited (Registration No. 2340150) are direct or indirect subsidiaries of Sky plc (Registration No. 2247735). All of the companies mentioned in this paragraph are incorporated in England and Wales and share the same registered office at Grant Way, Isleworth, Middlesex TW7 5QD.
Re: Deleted snapshot files filling up /var/lib/cassandra
Sorry for the late reply. To immediately solve the problem you can restart Cassandra and all the open file descriptors to the deleted snapshots should disappear. As for why it happened I would first address the disk space issue and see if the snapshot errors + open file descriptors issue still occurs (I am unclear as to whether you got the snapshot exception after the disk filled up or before), if you still have issues with repair not letting go of snapshotted files even with free disk space I would look to raise a ticket in Jira. On 17 March 2015 at 12:46, David Wahler dwah...@indeed.com wrote: On Mon, Mar 16, 2015 at 6:51 PM, Ben Bromhead b...@instaclustr.com wrote: If you are running a sequential repair (or have previously run a sequential repair that is still running) Cassandra will still have the file descriptors open for files in the snapshot it is using for the repair operation. Yeah, that aligns with my understanding of how the repair process works. But the cluster has no repair sessions active (I think; when I run nodetool tpstats, the AntiEntropyStage and AntiEntropySessions values are zero on all nodes) and the space still hasn't been freed. -- Ben Bromhead Instaclustr | www.instaclustr.com | @instaclustr http://twitter.com/instaclustr | (650) 284 9692
Re: Store data with cassandra
Sounds like this is a job for jackrabbit ? http://jackrabbit.apache.org From: Ali Akhtar ali.rac...@gmail.commailto:ali.rac...@gmail.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Friday, 20 March 2015 15:58 To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: Store data with cassandra ( I apologize, I'm only joking. To answer your question, Cassandra tends to cache the first 300MB or so of data in memory, only when it grows beyond that does it start to write it to files. But, Cassandra is not the write choice for storing files. In the screenshot you linked, its only storing the filenames, not the actual contents of the files). On Fri, Mar 20, 2015 at 8:54 PM, Ali Akhtar ali.rac...@gmail.commailto:ali.rac...@gmail.com wrote: It has been decided that the file cannot be allowed to be stored, sorry. However, if a sacrifice to the gods is prepared, it may be possible to change things. On Fri, Mar 20, 2015 at 8:49 PM, jean paul researche...@gmail.commailto:researche...@gmail.com wrote: i'd like to store MyFile.txt using cassandra (replicat = 2) and see on what node the file and its replicas are stored on my cluster of 10 nodes it is a simple file with simple content (text) is that possible ? 2015-03-20 16:44 GMT+01:00 Ali Akhtar ali.rac...@gmail.commailto:ali.rac...@gmail.com: The files you store have to personally be vetted by the cassandra community. Only if they're found to not contain anything inappropriate, does cassandra let you store them. (A 3/4 majority vote is necessary). Please send your files for approval to j...@reallycereal.commailto:j...@reallycereal.com On Fri, Mar 20, 2015 at 8:41 PM, jean paul researche...@gmail.commailto:researche...@gmail.com wrote: What about this so http://www.datastax.com/wp-content/uploads/2012/02/Screen-Shot-2012-02-10-at-11.21.55-AM.png i read also some documents about storing blob with cassandra !! 2015-03-20 15:04 GMT+01:00 Michael Dykman mdyk...@gmail.commailto:mdyk...@gmail.com: You seem to be missing the point here. Cassandra does not manage files, it manages data in a highly distributed cluster. If you are attempting to manage files, you are quite simply using the wrong tool and Cassandra is not for you. On Fri, Mar 20, 2015 at 9:10 AM, jean paul researche...@gmail.commailto:researche...@gmail.com wrote: I have used this tutoriel to create my data base http://planetcassandra.org/insert-select-records/ /var/lib/cassandra/data# ls demo system system_traces :/var/lib/cassandra/data# cd demo/ :/var/lib/cassandra/data/demo# ls users :/var/lib/cassandra/data/demo# cd users/ :/var/lib/cassandra/data/demo/users# ls :/var/lib/cassandra/data/demo/users# i find nothing in /var/lib/cassandra/data/demo/users! 2015-03-20 13:06 GMT+01:00 jean paul researche...@gmail.commailto:researche...@gmail.com: Hello All; Please, i have created this table. lastname | age | city | email | firstname --+-+---+-+--- Doe | 36 | Beverly Hills | jane...@email.commailto:jane...@email.com | Jane Byrne | 24 | San Diego | robby...@email.commailto:robby...@email.com | Rob Smith | 46 |Sacramento | johnsm...@email.commailto:johnsm...@email.com | John So, my question, where this data is saved ? in ./var/lib/cassandra/data ? My end goal is to store afile with cassandra and to see on which node my file is stored ? thanks a lot for help Best Regards. -- - michael dykman - mdyk...@gmail.commailto:mdyk...@gmail.com May the Source be with you. Information in this email including any attachments may be privileged, confidential and is intended exclusively for the addressee. The views expressed may not be official policy, but the personal views of the originator. If you have received it in error, please notify the sender by return e-mail and delete it from your system. You should not reproduce, distribute, store, retransmit, use or disclose its contents to anyone. Please note we reserve the right to monitor all e-mail communication through our internal and external networks. SKY and the SKY marks are trademarks of Sky plc and Sky International AG and are used under licence. Sky UK Limited (Registration No. 2906991), Sky-In-Home Service Limited (Registration No. 2067075) and Sky Subscribers Services Limited (Registration No. 2340150) are direct or indirect subsidiaries of Sky plc (Registration No. 2247735). All of the companies mentioned in this paragraph are incorporated in England and Wales and share the same registered office at Grant Way, Isleworth, Middlesex TW7 5QD.
Re: Logging client ID for YCSB workloads on Cassandra?
HI Jatin; besides enabling Tracing, is there any other way to get the task done ? (to log the client ID for every operation)Please share with the community the solution, so that we could collectively learn from your experience. cheersJan/ On Friday, February 20, 2015 12:48 PM, Jatin Ganhotra jatin.ganho...@gmail.com wrote: Never mind, got it working. Thanks :) — Jatin GanhotraGraduate Student, Computer ScienceUniversity of Illinois at Urbana Champaignhttp://jatinganhotra.comhttp://linkedin.com/in/jatinganhotra On Wed, Feb 18, 2015 at 7:09 PM, Jatin Ganhotra jatin.ganho...@gmail.com wrote: Hi, I'd like to log the client ID for every operation performed by the YCSB on my Cassandra cluster. The purpose is to identify analyze various other consistency measures other than eventual consistency. I wanted to know if people have done something similar in the past. Or am I missing something really basic here? Please let me know if you need more information. Thanks — Jatin Ganhotra
Re: Cassandra cluster Too high DISK IOs
On 20/03/15 19:34, Pranay Agarwal wrote: The cluster is processing something like 12k reads and 2k writes/seconds. The disks are locally attached and latency is just fine. It's the number of disk iops that's too high. Maybe each read is accessing many sstables. Ciao, Duncan.
Re: Cassandra cluster Too high DISK IOs
No. as shown in the histograms, 99% of reads are using 2 or less number of tables. What's typical usually? Can anyone share from experience? On Fri, Mar 20, 2015 at 1:12 PM, Duncan Sands duncan.sa...@gmail.com wrote: On 20/03/15 19:34, Pranay Agarwal wrote: The cluster is processing something like 12k reads and 2k writes/seconds. The disks are locally attached and latency is just fine. It's the number of disk iops that's too high. Maybe each read is accessing many sstables. Ciao, Duncan.
RE: Problems after trying a migration
Hi Jan, Thank you for your help, we’ll see during next week. Have a nice day. Best regards, [cid:image001.png@01D062FA.DDD7FC50] David CHARBONNIER Sysadmin T : +33 411 934 200 david.charbonn...@rgsystem.commailto:david.charbonn...@rgsystem.com ZAC Aéroport 125 Impasse Adam Smith 34470 Pérols - France www.rgsystem.comhttp://www.rgsystem.com/ [cid:image002.png@01D062FA.DDD7FC50] De : Jan [mailto:cne...@yahoo.com] Envoyé : jeudi 19 mars 2015 05:09 À : user@cassandra.apache.org Objet : Re: Problems after trying a migration Hi David; some input to get back to where you were : a)Start with the French cluster only and get it working with DSE 4.5.1 b)Opscenter keyspace is by default RF1; alter the keyspace to RF3 c)Take a full snapshot of all your nodes copy the files to a safe location on all the nodes To migrate the data into new cluster: a)Use the same version DSE 4.5.1 in Luxembourg bring up 1 node at a time.Check that the node has comeup in the new Datacenter. b)Bring up new nodes into the new Datacenter one at a time c)After all your new nodes are UP in Luxembourg, conduct a 'nodetool repair -parallel' d) Check in OpsCenter that you have all your nodes showing up (new and old) e)Start taking down your nodes in France, one at a time f) After all the nodes in France are down, conduct a 'nodetool repair -parallel' again g)Upgrade the nodes in Luxembourg to DSE 4.6.1 h) conduct a 'nodetool repair -parallel' again i) Upgrade to OpsCenter 5.1 Best of luck, hope this helps. Jan/ On Wednesday, March 18, 2015 1:01 PM, Robert Coli rc...@eventbrite.commailto:rc...@eventbrite.com wrote: On Wed, Mar 18, 2015 at 9:05 AM, David CHARBONNIER david.charbonn...@rgsystem.commailto:david.charbonn...@rgsystem.com wrote: - New nodes in the other country have been installed like French nodes except for Datastax Enterprise version (4.5.1 in France and 4.6.1 in the other country which means Cassandra version 2.0.8.39 in France and 2.0.12.200 in the other country) This is officially unsupported, and might cause of problems during this process. =Rob
Re: Adding new node to Cassandra cluster is too slow
On Thu, Mar 19, 2015 at 6:02 PM, Pranay Agarwal agarwalpran...@gmail.com wrote: What do you mean by point 3? Also, by doing one at a time, does it mean wait till nodetool status of the new node is UN from UJ? point 3 is look at nodetool status/nodetool ring/nodetool info/etc. on other cluster nodes and make sure the node you just stopped isn't in their list of, for example, UJ hosts. One at a time means, if you can afford it, I would wait for the node to be UN. If not, wait a few minutes between each join. Also, 2.1.0 is super broken, read this and consider using 1.2.x. https://engineering.eventbrite.com/what-version-of-cassandra-should-i-run/ =Rob
Re: Store data with cassandra
On Fri, Mar 20, 2015 at 5:06 AM, jean paul researche...@gmail.com wrote: Btw, Jean Paul... are you sure you don't want this? https://github.com/mogilefs/ As a general statement, storing files in Cassandra seems like a bad idea to me. =Rob
