Re: [gentoo-user] Back up a server in real-time
On Monday 17 November 2008, Joerg Schilling wrote: William Kenworthy [EMAIL PROTECTED] wrote: dirvish is in portage. tar/rsync/cp are not really backups but manual copies. If you want true backup you need scripts to handle the extra functionality for things like versioning, archive management and the all important restore. star has everything you need for backups and it is based on the standard archive format. Thanks Joerg, I have just had a quick look at star. Would you know if there is a LiveCD that has star - or will I be able to restore a fs from an archive created with star, but using tar to do it as it comes in e.g. Knoppix? -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Back up a server in real-time
Mick [EMAIL PROTECTED] wrote: On Monday 17 November 2008, Joerg Schilling wrote: William Kenworthy [EMAIL PROTECTED] wrote: dirvish is in portage. tar/rsync/cp are not really backups but manual copies. If you want true backup you need scripts to handle the extra functionality for things like versioning, archive management and the all important restore. star has everything you need for backups and it is based on the standard archive format. Thanks Joerg, I have just had a quick look at star. Would you know if there is a LiveCD that has star - or will I be able to restore a fs from an archive created with star, but using tar to do it as it comes in e.g. Knoppix? Did you check grml? Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [gentoo-user] Back up a server in real-time
On Sat, Nov 15, 2008 at 07:45:04PM +, Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? I like rdiff-backup, which gives incremental backups over rsync. Mark pgpHGlsB6HS3f.pgp Description: PGP signature
Re: [gentoo-user] Back up a server in real-time
William Kenworthy [EMAIL PROTECTED] wrote: dirvish is in portage. tar/rsync/cp are not really backups but manual copies. If you want true backup you need scripts to handle the extra functionality for things like versioning, archive management and the all important restore. star has everything you need for backups and it is based on the standard archive format. Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [gentoo-user] Back up a server in real-time
dirvish is in portage. tar/rsync/cp are not really backups but manual copies. If you want true backup you need scripts to handle the extra functionality for things like versioning, archive management and the all important restore. Dirvish is excellent on all but restore. BillK On Sat, 2008-11-15 at 19:45 +, Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? -- William Kenworthy [EMAIL PROTECTED] Home in Perth!
Re: [gentoo-user] Back up a server in real-time
On Sonntag 16 November 2008, William Kenworthy wrote: dirvish is in portage. tar/rsync/cp are not really backups but manual copies. If you want true backup you need scripts to handle the extra functionality for things like versioning, archive management and the all important restore. archives are managed by my tapelib, thank you very much. Restore is as easy as backup with tar. And if you start depending on something complex, you will be screwed at some point. Every system and every livecd has tar. amanda? mondo? devilish? not.
Re: [gentoo-user] Back up a server in real-time
On Sunday 16 November 2008 02:08:42 Mick wrote: On Saturday 15 November 2008, Dale wrote: Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? I keep mine simple, cp -auv paths/you/want/to/backup back/up/to It has works so far. Thought about doing a cron job but that complicates things. :/ Thank you all for the suggestions and for the link to the wiki! I've got some reading to do. ;-) Whenever I have used tar to back up a whole OS I used it with a LiveCD. This was to make sure that files and their metadata were not being changed while I was tar'ing them. Are you saying that I can actually fire up tar/rsync and back up in real time? Yes. Unix does some RealSmartThings(tm) when using files. The name is just a pointer to the actual file, represented by an inode. Once you have an inode open, it stays open until everything using it closes it. So you can add/delete/copy/move files by name with impunity as you then just move names around. Contrast this with other inferior systems, like say Windows for example, which has a built-in self-destruct button when you try this... I was gravitating towards using LVM snapshot and then tar'ing that to an external USB drive. This is the preferred way, as you get a consistent snapshot frozen at a point in time. This deals nicely with inconsistencies caused by files changing while you are backing up other ones. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Back up a server in real-time
On Sunday 16 November 2008 11:04:41 William Kenworthy wrote: dirvish is in portage. tar/rsync/cp are not really backups but manual copies. If you want true backup you need scripts to handle the extra functionality for things like versioning, archive management and the all important restore. Dirvish is excellent on all but restore. Which raises the question: What's the point of it then? OTOH, it has to be better than legato. Nothing could be worse than legato. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Back up a server in real-time
Am Samstag, 15. November 2008 20:45:04 schrieb Mick: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? I guess I would use rsnapshot to backup to another disk/machine if I had only a small number of machines to backup. If the server is rather critical, or there more than a handfull of machines to backup, I would think about a full featured backup suite like bacula. Both can eventually combined with using LVM snapshots. Bye... Dirk
Re: [gentoo-user] Back up a server in real-time
On Sunday 16 November 2008 12:31:16 William Kenworthy wrote: * app-backup/dirvish Latest version available: 1.2.1 Latest version installed: 1.2.1 Size of downloaded files: [no/bad digest] Homepage: http://www.dirvish.org/ Description: Dirvish is a fast, disk based, rotating network backup system. License: OSL-2.0 Its worth a read on how its done - using smarts rather than brute force! So the basic premise is that backups are done frequently and need to be efficient, whereas restores are only done in the event of a mistake. Less efficient restores is then a fair trade? I see people get this wrong all the time at work. They assume that backups are some sort of rapid shared storage system. Which of course it isn't. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Back up a server in real-time
It actually does a very fast, space efficient backup management. As for restore, each backup is effectively an uncompressed fully accessible mirror image so you can manually copy/tar/... etc back for the full system, or individual files. The developers basically say we have a working backup system where you can restore using standard tools if needed, but we'll make it easier when we can get around to it. Having versioned backups can be REALLY handy at times. Space efficiency typically stabilises at 2x archive size depending on delta. Downside it can hammer a disk - reiserfs is fine but ext2/3 was flaky. moriah ~ # esearch dirvish [ Results for search key : dirvish ] [ Applications found : 1 ] * app-backup/dirvish Latest version available: 1.2.1 Latest version installed: 1.2.1 Size of downloaded files: [no/bad digest] Homepage:http://www.dirvish.org/ Description: Dirvish is a fast, disk based, rotating network backup system. License: OSL-2.0 Its worth a read on how its done - using smarts rather than brute force! BillK On Sun, 2008-11-16 at 11:58 +0200, Alan McKinnon wrote: On Sunday 16 November 2008 11:04:41 William Kenworthy wrote: dirvish is in portage. tar/rsync/cp are not really backups but manual copies. If you want true backup you need scripts to handle the extra functionality for things like versioning, archive management and the all important restore. Dirvish is excellent on all but restore. Which raises the question: What's the point of it then? OTOH, it has to be better than legato. Nothing could be worse than legato. -- William Kenworthy [EMAIL PROTECTED] Home in Perth!
Re: [gentoo-user] Back up a server in real-time
On Sunday 16 November 2008, Alan McKinnon wrote: On Sunday 16 November 2008 02:08:42 Mick wrote: On Saturday 15 November 2008, Dale wrote: Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? I keep mine simple, cp -auv paths/you/want/to/backup back/up/to It has works so far. Thought about doing a cron job but that complicates things. :/ Thank you all for the suggestions and for the link to the wiki! I've got some reading to do. ;-) Whenever I have used tar to back up a whole OS I used it with a LiveCD. This was to make sure that files and their metadata were not being changed while I was tar'ing them. Are you saying that I can actually fire up tar/rsync and back up in real time? Yes. Unix does some RealSmartThings(tm) when using files. The name is just a pointer to the actual file, represented by an inode. Once you have an inode open, it stays open until everything using it closes it. So you can add/delete/copy/move files by name with impunity as you then just move names around. Contrast this with other inferior systems, like say Windows for example, which has a built-in self-destruct button when you try this... Sure, but isn't there a problem with atime mtime metadata when you carry out a backup in real time and then restore from it? I was gravitating towards using LVM snapshot and then tar'ing that to an external USB drive. This is the preferred way, as you get a consistent snapshot frozen at a point in time. This deals nicely with inconsistencies caused by files changing while you are backing up other ones. Right, that's what I was thinking too. What does restoring from a backed up snapshot involve? -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Back up a server in real-time
On Sun, 2008-11-16 at 12:39 +0200, Alan McKinnon wrote: On Sunday 16 November 2008 12:31:16 William Kenworthy wrote: * app-backup/dirvish Latest version available: 1.2.1 Latest version installed: 1.2.1 Size of downloaded files: [no/bad digest] Homepage:http://www.dirvish.org/ Description: Dirvish is a fast, disk based, rotating network backup system. License: OSL-2.0 Its worth a read on how its done - using smarts rather than brute force! So the basic premise is that backups are done frequently and need to be efficient, whereas restores are only done in the event of a mistake. Less efficient restores is then a fair trade? I see people get this wrong all the time at work. They assume that backups are some sort of rapid shared storage system. Which of course it isn't. I wouldnt call it fair trade, rather its working now and we'll make it better it eventually. I agree, backups should be just that - protection from mistakes/lost data. However this is from someone who has the archives mounted via nfs and can browse via filemanager an retrieve files that way via a desktop. Currently I have my freeruner mobile phone, mythtv critical files and a number of systems such as laptops backed up this way and its been very reliable and useful, both as automatic/unattended and manually run. Its neat in that you only need rsync and ssh (though its possible to use rsh and the like) on the client. You only need dirvish on the server. Starting to sound like a salesman for it! - but I have been very happy with it. I used to use custom rsync scripts, different backup managers and even tivoli backup at work. Dirvish is best for my usage. BillK
Re: [gentoo-user] Back up a server in real-time
On Sunday 16 November 2008 13:51:23 Mick wrote: Yes. Unix does some RealSmartThings(tm) when using files. The name is just a pointer to the actual file, represented by an inode. Once you have an inode open, it stays open until everything using it closes it. So you can add/delete/copy/move files by name with impunity as you then just move names around. Contrast this with other inferior systems, like say Windows for example, which has a built-in self-destruct button when you try this... Sure, but isn't there a problem with atime mtime metadata when you carry out a backup in real time and then restore from it? With a restore, you have really just two options: - consider the file being restored to be a new file and set the *time to now - consider it a full restore and set them the same as what's in the backup root or the file's owner is permitted to do the latter There's a third option which makes little sense: set the *time of the restored file to be the same as whatever file it is repalcing on disk. But this is mostly silly as the file data is now inconsistent with the recorded times I was gravitating towards using LVM snapshot and then tar'ing that to an external USB drive. This is the preferred way, as you get a consistent snapshot frozen at a point in time. This deals nicely with inconsistencies caused by files changing while you are backing up other ones. Right, that's what I was thinking too. What does restoring from a backed up snapshot involve? The backup is just a backup, the fact that it was made from a frozen disk snapshot is irrelevant. So you would restore it in the usual manner for the backup format/method in use -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Back up a server in real-time
On Sunday 16 November 2008 14:36:22 William Kenworthy wrote: I wouldnt call it fair trade, rather its working now and we'll make it better it eventually. I agree, backups should be just that - protection from mistakes/lost data. However this is from someone who has the archives mounted via nfs and can browse via filemanager an retrieve files that way via a desktop. Currently I have my freeruner mobile phone, mythtv critical files and a number of systems such as laptops backed up this way and its been very reliable and useful, both as automatic/unattended and manually run. OK I see where you are coming from. When you say backup/restore to me, I think in terms of what I'm used to - massive tape archives holding many multi-TB of data for thousands of hosts. What you are describing is indeed a backup, but in my world I'd probably call it something else. Perhaps I should have been more explicit and said Disaster Recovery Archival Backup instead of backup... :-) -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Back up a server in real-time
On Sat, Nov 15, 2008 at 10:08 PM, Mick [EMAIL PROTECTED] wrote: On Saturday 15 November 2008, Dale wrote: Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? I keep mine simple, cp -auv paths/you/want/to/backup back/up/to It has works so far. Thought about doing a cron job but that complicates things. :/ Thank you all for the suggestions and for the link to the wiki! I've got some reading to do. ;-) Whenever I have used tar to back up a whole OS I used it with a LiveCD. This was to make sure that files and their metadata were not being changed while I was tar'ing them. Are you saying that I can actually fire up tar/rsync and back up in real time? Please read tar's texinfo manual, at least the section 5 Performing Backups and Restoring Files It is short, and nicely divided in subsections making it is easy to select you need/want to read. Not like a man page, which for me is useful as a reference but horrible at teaching how to use the program (unless the program is quite simple). -- Software is like sex: it is better when it is free - Linus Torvalds
[gentoo-user] Back up a server in real-time
Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Back up a server in real-time
On Samstag 15 November 2008, Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? tar
Re: [gentoo-user] Back up a server in real-time
On Saturday 15 November 2008 21:45:04 Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? rsync -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] Back up a server in real-time
On Saturday 15 November 2008 02:45:04 pm Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? What's wrong with gentoo-wiki.info? As for backup yes tar is good, but how about rsync? -- * From the desk of: Jerome D. McBride 15:28:45 up 20 days, 4:03, 2 users, load average: 1.18, 1.05, 1.03 *
Re: [gentoo-user] Back up a server in real-time
Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? I keep mine simple, cp -auv paths/you/want/to/backup back/up/to It has works so far. Thought about doing a cron job but that complicates things. :/ Dale :-) :-)
Re: [gentoo-user] Back up a server in real-time
On Saturday 15 November 2008, Dale wrote: Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? I keep mine simple, cp -auv paths/you/want/to/backup back/up/to It has works so far. Thought about doing a cron job but that complicates things. :/ Thank you all for the suggestions and for the link to the wiki! I've got some reading to do. ;-) Whenever I have used tar to back up a whole OS I used it with a LiveCD. This was to make sure that files and their metadata were not being changed while I was tar'ing them. Are you saying that I can actually fire up tar/rsync and back up in real time? I was gravitating towards using LVM snapshot and then tar'ing that to an external USB drive. -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Back up a server in real-time
Mick wrote: On Saturday 15 November 2008, Dale wrote: Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? I keep mine simple, cp -auv paths/you/want/to/backup back/up/to It has works so far. Thought about doing a cron job but that complicates things. :/ Thank you all for the suggestions and for the link to the wiki! I've got some reading to do. ;-) Whenever I have used tar to back up a whole OS I used it with a LiveCD. This was to make sure that files and their metadata were not being changed while I was tar'ing them. Are you saying that I can actually fire up tar/rsync and back up in real time? I was gravitating towards using LVM snapshot and then tar'ing that to an external USB drive. I have read that you can use tar while the system is running. I wouldn't do that during say a emerge or something tho. I would try to keep the system somewhat idle as far as changing files. My cp command works fine on a running system with the same advice as above on the system being idle tho. Dale :-) :-)
Re: [gentoo-user] Back up a server in real-time
On Saturday 15 November 2008 07:08:42 pm Mick wrote: On Saturday 15 November 2008, Dale wrote: Mick wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? I keep mine simple, cp -auv paths/you/want/to/backup back/up/to It has works so far. Thought about doing a cron job but that complicates things. :/ Thank you all for the suggestions and for the link to the wiki! I've got some reading to do. ;-) Whenever I have used tar to back up a whole OS I used it with a LiveCD. This was to make sure that files and their metadata were not being changed while I was tar'ing them. Are you saying that I can actually fire up tar/rsync and back up in real time? I was gravitating towards using LVM snapshot and then tar'ing that to an external USB drive. If you need to guarantee a backup without the data being changed during the process... you gotta take the server down. The easiest way is to init 1 to single user mode , make your backup, then init x back to what ever runlevel you were in... -- * From the desk of: Jerome D. McBride 21:54:45 up 5:40, 4 users, load average: 0.00, 0.01, 0.00 *
Re: [gentoo-user] Back up a server in real-time
rsync is nice way On Sun, Nov 16, 2008 at 3:45 AM, Mick [EMAIL PROTECTED] wrote: Without gentoo-wiki my knowledge level is rather poor (just like my memory!) What would you use to back up a running server without taking it off line? -- Regards, Mick -- Only freebsd openbsd gentoo-linux windows2008
