Hi Peter, It sounds like Cassandra was failing to start while reading the saved-off copy of its key cache. This can happen if the file has become corrupt, although it sounds here like you are hitting an out-of-memory issue. Deleting the cache file is quite safe, and may be the only option here. The impact will be that Cassandra will start up without anything pre-populated in its key cache. As the node receives load, it will automatically repopulate the cache. Until it has done so, you might see reduced Cassandra performance.
It sounds like this was a sensible thing to do, and that no further action is required. This isn’t something that we’ve seen before. Do you know of anything strange that you may have done before turning off your VM? Did you turn it off, or suspend it? It would be great to get a reliable reproduction scenario so that we can fix this issue once and for all. Yours, Chris From: Clearwater [mailto:[email protected]] On Behalf Of Peter Skrzynski Sent: 03 August 2016 02:47 To: [email protected] Subject: [Project Clearwater] Homestead not starting due to cassandra problem Hi, I am running a Release-89 bono/sprout/homestead/ibcf/dns configuration. After my virtual machines had been turned off for a while (few days), then restarted, the homestead node was not running up correctly, due to homestead and cassandra not running up. Doing various monit and service commands could not get it going. Looking in /var/log/cassandra/system.log, I see that it failed due to… - reading saved cache /var/lib/cassandra/saved_caches/system-sstable_activity-KeyCache-b.db - exception encountered during startup - java.lang.OutOfMemoryError: Java heap space Changing the HEAP size in /etc/Cassandra/Cassandra-env.sh did not fix the problem. But… After deleting the file /var/lib/cassandra/saved_caches/system-sstable_activity-KeyCache-b.db, then everything ran up OK. So, my question is… Is it acceptable to simply delete the file /var/lib/cassandra/saved_caches/system-sstable_activity-KeyCache-b.db? Or do I need to rebuild that db file somehow, how would I do that? Many thanks for any assistance, Regards, Peter Peter Skrzynski Technical Lead NEC New Zealand Limited NEC House, Level 6, 40 Taranaki Street, PO Box: 1936, Wellington 6011, New Zealand T: 043816257, M: 0274849530, F: +6443811110 [email protected]<mailto:[email protected]> nz.nec.com<http://nz.nec.com> [cid:[email protected]] Please consider the environment before printing this email Attention: The information contained in this message and or attachments is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination, copying or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any system and destroy any copies. NEC has no liability for any act or omission in reliance on the email or any attachment. Before opening this email or any attachment(s), please check them for viruses. NEC is not responsible for any viruses in this email or any attachment(s); any changes made to this email or any attachment(s) after they are sent; or any effects this email or any attachment(s) have on your network or computer system.
_______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org
