Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Tom Buskey
Sorry, POC == Proof of Concept. In this case, it's an all in one (AIO) cloud with 1 physical box that's not really suitable for production. On Thu, Sep 28, 2017 at 3:02 PM, Ted Roche wrote: > On Thu, Sep 28, 2017 at 2:32 PM, Tom Buskey wrote: > > > > I misspoke about LVM for Glance/Swift. The

Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Ted Roche
On Thu, Sep 28, 2017 at 2:32 PM, Tom Buskey wrote: > > I misspoke about LVM for Glance/Swift. The backend for the images are on > top of a filesystem in the POC clouds. LVM is used for Cinder, the block > image store. Ceph is often used to drop in replace LVM for Cinder and files > for Swift ob

Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Tom Buskey
On Thu, Sep 28, 2017 at 2:00 PM, Joshua Judson Rosen wrote: > On 09/28/2017 01:46 PM, Tom Buskey wrote: > > I work with OpenStack. It manages images in Glance which sit above its > object storage, Swift. > > > > On the POC clouds, you can use LVM as a backend for Glance. > Snapshotting is *very*

Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Joshua Judson Rosen
On 09/28/2017 02:14 PM, mark wrote: > AWS/EBS is not LVM under the covers, it's more like NFS; and snapshots are > more like VMware & how it does snapshots. I have never used VMWare and have no idea how it does anything. Can you provide more insight on what that means? > The OS cache exclusion

Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread mark
AWS/EBS is not LVM under the covers, it's more like NFS; and snapshots are more like VMware & how it does snapshots. The OS cache exclusion refers to read-ahead and write caching going on in RAM. Mark On Sep 28, 2017 1:17 PM, "Joshua Judson Rosen" wrote: > I'm working on a project that uses Amaz

Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Joshua Judson Rosen
On 09/28/2017 01:32 PM, Ken D'Ambrosio wrote: > I would say it's unlikely to be LVM, because LVM is content-ignorant; it > snapshots the entire volume, which is > inefficient, and when you're Amazon, you care a LOT about being efficient.  > Instead, I imagine they're using some > content-aware Co

Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Joshua Judson Rosen
On 09/28/2017 01:48 PM, Bill Ricker wrote: > The lack of coherence due to OS cave not being flushed should still be a > concern. In the general case, yes. In my particular case I'm specifically concerned only with data that's stored transactionally to the extent that (and I really hope that I'm

Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Joshua Judson Rosen
On 09/28/2017 01:46 PM, Tom Buskey wrote: > I work with OpenStack.  It manages images in Glance which sit above its > object storage, Swift. > > On the POC clouds, you can use LVM as a backend for Glance.  Snapshotting is > *very* slow.  30 minutes for a snap of a > 80GB VM that's shutdown. OK.

Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Bill Ricker
The lack of coherence due to OS cave not being flushed should still be a concern. OTOH I saw a storage level replication system propagate corruption to the remote site's copy of the Production DBMS ... So it perfectly replicated the primary's failure. Oops. Easiest recovery was restoring a nightly

Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Tom Buskey
I work with OpenStack. It manages images in Glance which sit above its object storage, Swift. On the POC clouds, you can use LVM as a backend for Glance. Snapshotting is *very* slow. 30 minutes for a snap of a 80GB VM that's shutdown. You can use other storage backends in OpenStack that are fa

Re: Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Ken D'Ambrosio
I would say it's unlikely to be LVM, because LVM is content-ignorant; it snapshots the entire volume, which is inefficient, and when you're Amazon, you care a LOT about being efficient. Instead, I imagine they're using some content-aware CoW solution such as ZFS. But, whatever mechanism, I ag

Is Amazon AWS/EBS snapshotting just LVM, or what?

2017-09-28 Thread Joshua Judson Rosen
I'm working on a project that uses Amazon AWS-provided VPS instances, and the other guy on the project is telling me that "snapshotting hourly may degrade performance", and I'm trying to determine where that's actually true. My gut feeling is that it sounds kind of bogus. >From the information I