On Oct 25, 2011, at 7:51 PM, [email protected] wrote:
>
> The snapshot has no effect on the master, and yes, we've already said and
> we already know it is a weakness in LVM that if you don't extend your
> snapshots you lose them. This can be mitigated by monitoring and automatic
> volume extension.
You missed it. This isn't about what happens to master. It's what happens to
b when a disappears. If master<-a<-b and a disappears due to reaping then b
becomes useless. Or b is reaped, too. Either way you're dealing with data
loss. This is why LVM will not do what you originally asked about.
Monitoring has problems. If the volume fills up faster than the monitor polls
capacity then you lose your data. If the volume fills up faster than it can be
extended then you lose your data. If the volume cannot be extended because the
volume group has no more extents available then you lose your data. Like I
wrote at the start: LVM will quite happily bite your face off.
Now, to address your most recent question:
How do I back up a 1TB disk. Think about this: how do you intend to do a
restore from this backup? The most important part of a backup system is being
able to restore from backup in a timely fashion.
I have in production a compute server with two 8TB file systems and a 9TB file
system, all sitting on LVM volumes. I have an automated backup that runs every
night on this server. It's an incremental file system backup so I'm only
backing up the changes every night. This is, as you might expect, quite faster
than trying to do full backups of 25TB every night -- which I can't because it
would take three days to do it.
On smaller capacity volumes, in the several hundred GB range, I use rsnapshot
to do incremental file snapshots to a storage server. Again, I don't back up
the raw disk partitions every time. I only back up the changed files.
In both cases -- and in fact with all my backups -- they are file level
backups. The reason being that if I need to restore a single file or directory
then I don't have to rebuild the entire volume to do so. I can restore as
little or as much as I need to recover from a mistake or a disaster.
Suppose the case of a live volume that needs to be in a frozen state for doing
a backup. Database servers are prime examples of this. Here, I would freeze
the database, make a snapshot of the underlying volume, and then thaw the
database. Now I can do my backup of the read-only snapshot volume without
interfering with the running system. I would delete the snapshot when the
backup is complete.
If I were using plain LVM and ext3 for my users' home directories then I would
do something similar with read-only snapshots. There would be no freeze step,
and I would keep several days worth of snapshots on the file server to make
error recovery faster than going to tape or network storage. As it is, I use
OpenAFS which has file system snapshots so I don't need to do any of this and
users can go back in time just by looking in .clone in their home directories.
I still have nightly backups to tape for long-term archives.
Now, time to poke holes in your proposal. I have a physics graduate student
doing his thesis research project on a shared compute server along with a dozen
others. They collectively have 7.5TB of data on there. This is a real-world
case on the aforementioned compute server. Said student accidentally wipes out
his entire thesis project, 200GB worth of files. It's 9:30 PM and he needs his
files by 8am or he fails his thesis defense, doesn't graduate and I'm looking
for a new job.
With my file level backup system I can have his files restored within a couple
of hours at the outside without affecting anyone else's work.
With your volume level backup system I would spend the night on Monster looking
for a new job. The problem with it is that I can't restore individual files
because it isn't individual files that are backed up. It's the disk blocks. I
can't just drop those backed-up blocks onto the volume. Here:
master->changes->changes->changes
\->backup
If I dumped the backup blocks onto the volume then I'd scramble the file
system. Restoration would require me to replicate the entire volume at the
block level as it was when the backup was made. This would destroy all the
other researchers' work done in the past however many hours since that backup
was made. I would fire myself for gross incompetence if I were relying on this
kind of backup system. It's that bad.
It gets worse. What happens when the whole thing fails outright? Total
disaster on your 1TB disk. Now it's not just 29 minutes to restore last
night's blocks. It's two hours to restore the initial replica and then 30
minutes times however many deltas have been made. Six deltas means 5 hours to
do a full rebuild. I can do a complete restore from TSM or rsnapshot in half
that time, maybe less depending on how much data was actually there.
Data point: It takes ~19 hours to restore 7.5TB from enterprise-class Tivoli
Storage Manager over 1GB Ethernet to a 12x1TB SATA (3Gb/s) RAID 6 volume. I
had to do it this past spring after the RAID controller on that volume went
stupid and corrupted the whole thing.
--Rich P.
_______________________________________________
Discuss mailing list
[email protected]
http://lists.blu.org/mailman/listinfo/discuss
