We are using cassandra 1.2.10 (With JNA installed) on ubuntu 12.04.3 and are
running our instances in Amazon Web Services.
What I am trying to do.
Our cassandra systems data is on an EBS volume so we can take snapshots of the
data and create volumes based on those snapshots and restore them
On Fri, Mar 28, 2014 at 11:15 AM, Russ Lavoie ussray...@yahoo.com wrote:
We are using cassandra 1.2.10 (With JNA installed) on ubuntu 12.04.3 and
are running our instances in Amazon Web Services.
Our cassandra systems data is on an EBS volume
Best practice for Cassandra on AWS is to run
Thank you for your quick response.
Is there a way to tell when a snapshot is completely done?
On Friday, March 28, 2014 1:30 PM, Robert Coli rc...@eventbrite.com wrote:
On Fri, Mar 28, 2014 at 11:15 AM, Russ Lavoie ussray...@yahoo.com wrote:
We are using cassandra 1.2.10 (With JNA
In your step 4, be sure you create a consistent EBS snapshot. You may have
pieces of your sstables that have not actually been flushed all the way to
EBS.
See https://github.com/alestic/ec2-consistent-snapshot
ml
On Fri, Mar 28, 2014 at 3:21 PM, Russ Lavoie ussray...@yahoo.com wrote:
Thank
On Fri, Mar 28, 2014 at 12:21 PM, Russ Lavoie ussray...@yahoo.com wrote:
Thank you for your quick response.
Is there a way to tell when a snapshot is completely done?
IIRC, the JMX call blocks until the snapshot completes. It should be done
when nodetool returns.
=Rob
Robert,
That is what I thought as well. But apparently something is happening. The
only way I can get away with doing this is adding a sleep 60 right after the
nodetool snapshot is executed. I can reproduce this 100% of the time by not
issuing a sleep after nodetool snapshot.
This is the
I have a nagging memory of reading about issues with virtualization and not
actually having durable versions of your data even after an fsync (within
the VM). Googling around lead me to this post:
http://petercai.com/virtualization-is-bad-for-database-integrity/
It's possible you're hitting this
I will +1 the recommendation on using tablesnap over EBS. S3 is at least
predictable.
Additionally, from a practical standpoint, you may want to back up your
sstables somewhere. If you use S3, it's easy to pull just the new tables
out via aws-cli tools (s3 sync), to your remote, non-aws server,
As I tried to say, EBS snapshots require much care or you get corruption
such as you have encountered.
Does Cassandra quiesce the file system after a snapshot using fsfreeze or
xfs_freeze? Somehow I doubt it...
On Fri, Mar 28, 2014 at 4:17 PM, Jonathan Haddad j...@jonhaddad.com wrote:
I have
+1 for tablesnap
On Fri, Mar 28, 2014 at 4:28 PM, Jonathan Haddad j...@jonhaddad.com wrote:
I will +1 the recommendation on using tablesnap over EBS. S3 is at least
predictable.
Additionally, from a practical standpoint, you may want to back up your
sstables somewhere. If you use S3,
Another thing to keep in mind is that if you are hitting the issue I
described, waiting 60 seconds will not absolutely solve your problem, it
will only make it less likely to occur. If a memtable has been partially
flushed at the 60 second mark you will end up with the same corrupt sstable.
On
11 matches
Mail list logo
