Re: [flexbackup-help] What a shame flexbackup is *almost* there - but I have to reject it
Hi I didn't mean to be rude either, it's just that having a backupsystem that really works like it should be just doesn't exist currently (to my knowledge). For you're example, (a.conf, b.conf) you need some filesystem support to be able to replicate your changes to another system/backup. Currently there's no method/sw that I know of that is supporting windows/unix/linux and is in common practise (otherwise it's difficult to use in an enterprise environment due to the shere amount of servers and different OS's). What you maybe are searching maybe FAM (File Alteration Monitor)?!? http://oss.sgi.com/projects/fam/faq.html And use of a FAM mirror http://www.linuxfocus.org/common/src/article199/fam_mirror Unless you set up something like that, you're not getting any solution to your a.conf/b.conf problem, the old methods of full/incremental/differential backups just doesn't provide that functionality. Also isn't a tape backup system flexible enough, or fast enough to support that. I will not trust my system+data to such a backup-system. Are *you* comfortable with that? What is the down side of being able to restore reliably? Rather too much than too little data protection is what keeps an enterprise datacenter running... You are right, I don't know EMC Networker, Veritas Netbackup, HP Data Protector, but I'm willing to bet they can restore a directory containing *exactly* b.conf. (Am I right?) They can, if you really have an full backup just before the crash/logical accident happens. For incrementals, it's just to restore full+incrementals and get all those a.conf's as well ;-( --Robert On Fri 4 August 2006 09:50, Peter Valdemar Mørch wrote: flexbackup-at-worreby.ch |Lists| wrote: 1. Have you the slightest idea how it's done in the reality? That's standard procedure to restore full backup plus all incrementals you have, if you invented some other schema, please let me know (at work we're currently backing up between 15-20 TB / night) and 2. What's worse? Dataloss or too much data!!! Let us first agree that data loss is totally unacceptable. I'm not arguing that data loss is a good thing. (Am i? Where?) But for me, too much data is *ALSO* unacceptable, because it does not represent reality. Let me illustrate with a scenario: * I start with a directory e.g. under Apache's configuration that has a single file a.conf * I make a full backup. * rm a.conf * Add a file b.conf * (Notice that at no point in time was there ever more than one file in the directory) * I make an incremental backup. * Hard disk crash * Restore full backup * Restore incremental backup Result with current flexbackup: A directory containing both a.conf AND b.conf. Wouldn't you rather end up with a directory containing *exactly* b.conf - the exact contents of the directory when the incremental backup was made? Ok, so in a directory where there is supposed to be 1 single file you may be able to remember yourself to delete a.conf (because it is *not* supposed to be there). Lets just hope I slept well and didn't remove b.conf instead by accident because I was tired... ;) But with between 15-20 TB of data as you put it? Who can remember which of the gazillion files to delete? No, I don't want that to be a guessing game. In some (most?) cases, too much data is just annoying - not really catastrophic. But in some cases (e.g. conf.d, cron.d, *.d directories, or a file containing sensitive data that was deleted) this *is* catastrophic. I could end up with a system that won't boot or misbehaves or is dangerous after a full+incremental restore. I will not trust my system+data to such a backup-system. Are *you* comfortable with that? What is the down side of being able to restore reliably? You are right, I don't know EMC Networker, Veritas Netbackup, HP Data Protector, but I'm willing to bet they can restore a directory containing *exactly* b.conf. (Am I right?) Peter -- --Robert Robert Worreby Birkenweg 82 CH-3123 Belp http://counter.li.org pgpEvGqhQntHR.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ flexbackup-help mailing list flexbackup-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flexbackup-help
Re: [flexbackup-help] What a shame flexbackup is *almost* there - but I have to reject it
flexbackup-at-worreby.ch |Lists| wrote: 1. Have you the slightest idea how it's done in the reality? That's standard procedure to restore full backup plus all incrementals you have, if you invented some other schema, please let me know (at work we're currently backing up between 15-20 TB / night) and 2. What's worse? Dataloss or too much data!!! Let us first agree that data loss is totally unacceptable. I'm not arguing that data loss is a good thing. (Am i? Where?) But for me, too much data is *ALSO* unacceptable, because it does not represent reality. Let me illustrate with a scenario: * I start with a directory e.g. under Apache's configuration that has a single file a.conf * I make a full backup. * rm a.conf * Add a file b.conf * (Notice that at no point in time was there ever more than one file in the directory) * I make an incremental backup. * Hard disk crash * Restore full backup * Restore incremental backup Result with current flexbackup: A directory containing both a.conf AND b.conf. Wouldn't you rather end up with a directory containing *exactly* b.conf - the exact contents of the directory when the incremental backup was made? Ok, so in a directory where there is supposed to be 1 single file you may be able to remember yourself to delete a.conf (because it is *not* supposed to be there). Lets just hope I slept well and didn't remove b.conf instead by accident because I was tired... ;) But with between 15-20 TB of data as you put it? Who can remember which of the gazillion files to delete? No, I don't want that to be a guessing game. In some (most?) cases, too much data is just annoying - not really catastrophic. But in some cases (e.g. conf.d, cron.d, *.d directories, or a file containing sensitive data that was deleted) this *is* catastrophic. I could end up with a system that won't boot or misbehaves or is dangerous after a full+incremental restore. I will not trust my system+data to such a backup-system. Are *you* comfortable with that? What is the down side of being able to restore reliably? You are right, I don't know EMC Networker, Veritas Netbackup, HP Data Protector, but I'm willing to bet they can restore a directory containing *exactly* b.conf. (Am I right?) Peter -- Peter Valdemar Mørch http://www.morch.com - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ flexbackup-help mailing list flexbackup-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flexbackup-help
Re: [flexbackup-help] What a shame flexbackup is *almost* there - but I have to reject it
On Fri, 4 Aug 2006, Peter Valdemar Mørch wrote: flexbackup-at-worreby.ch |Lists| wrote: 1. Have you the slightest idea how it's done in the reality? That's standard procedure to restore full backup plus all incrementals you have... That is standard procedure, unless you have fancy expensive tools. Wouldn't you rather end up with a directory containing *exactly* b.conf - the exact contents of the directory when the incremental backup was made? Sure, but to create such a thing, you need a considerably more sophisticated process for creating incremental backups. The 'standard procedure' backs up all data created or modified since the last full backup. You only need the metadata of all the current data to do that, and that's all in the file system. You want deletion records for all data deleted since the last backup. You need both old and new metadata to construct that. Only the new metadata is contained in the file system. You also need an archive structure which contains deletion records. AFAIK tar, cpio etc don't handle that. You are right, I don't know EMC Networker, Veritas Netbackup, HP Data Protector, but I'm willing to bet they can restore a directory containing *exactly* b.conf. (Am I right?) I don't know. Is that question relevant to this list? Are you offering to sponsor the development of the features you find missing? Or are you complaining because the gift you received isn't exactly what you want? -- Charlie - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ flexbackup-help mailing list flexbackup-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flexbackup-help
Re: [flexbackup-help] What a shame flexbackup is *almost* there - but I have to reject it
Hi A couple of comments on your *concerns* 1. Have you the slightest idea how it's done in the reality? That's standard procedure to restore full backup plus all incrementals you have, if you invented some other schema, please let me know (at work we're currently backing up between 15-20 TB / night) 2. What's worse? Dataloss or too much data!!! ;-) It's up to you to decide that, we do prefer to have more data than less data, that can always be overcome to reduce the duplicate ones, the non-existing ones are a lot more difficult to get. Regarding finding files, so is flexbackup similar to how the EMC Networker, Veritas Netbackup, HP Data Protector is working, they are all depending on your index/internal database over all files to get to them quickly. 3. All help is welcome, the open-source community needs people that make helpful work. Regarding Tape management so privately I'm just backing up to disk, is a lot easier and cheaper than to buy a tape library at home. (do got several DAT Exchanger at home, but is not just using them) --Robert On Fri 28 July 2006 14:23, Peter Valdemar Mørch wrote: I'm searching for a linux backup solution. And flexbackup is almost it! There are three things that make me continue looking for other solutions: 1) Crashev has asked these questions (twice): My question is - how to restore fully latest backup ? I mean if my system fails today and I want to have the freshest backup restored - how to do it with flexbackup? Should I extract fullbackup with *.0.* and then overwrite files with other days backups or how does it work? The other thing is how to to tell apart differential from incremental in such scenerio? These are *very* reasonable questions to be asking of my backup system. How do I reliably recreate the last good state? Any backup system that can't answer these questions are not up to the job, IMHO. 2) Pablo Godel asked: incremental backups and deleted files Basically, if I delete files and make incremental backups, the deletion of these files is not recorded in the incremental backups. So no, it is in fact impossible to reliably get to the last good state. Also, FAQ How do I find out which archive(s) contain a certain file? on http://www.edwinh.org/flexbackup/faq.html mentions: If you don't know in which archive to find a certain file, look at the log files. As long as you have verbose turned on (default), you can just 'zgrep filename /var/log/flexbackup/*.gz', and that works pretty well. No, that doesn't work very well. It *sucks*! (Especially if you have to restore 12342345 files and you don't know what they are.) There seems to be no way to automatically and reliably restore/extract a backup set. Adding just a little meta-data to the created backups could enable this functionality: I'm thinking a -search option to find the relevant archives and -restore to extract from the relevant archives. 3) There are three Project Admins: edwinh, jjreynold and pholcomb. EdwinH, the only one to ever post to the mailing list, last did so 2004/01/30 - over two years ago. Basically the project is without contributors and maintainers. I'm tempted to take a stab at adding the meta data to the created and backuped data and implementing -restore and -search. I've just never used a tape backup system, only to-disk. Would anybody we willing to test any such enhancements on a tape-drive? Peter - #!/bin/bash # I'd like to be able to restore src with newest file2, # file3 and file4 in it. (And no file1) # Impossible with current flexbackup, though. mkdir src echo line src/file1 echo line src/file2 echo line src/file3 echo line src/file4 flexbackup -c flexbackup.conf -set test -level 0 rm src/file1 echo anotherline src/file2 chmod go-rwx src/file3 ls -l src sleep 60 flexbackup -c flexbackup.conf -set test -level 1 -- --Robert Robert Worreby Birkenweg 82 CH-3123 Belp http://counter.li.org pgpOtyJqSTmDp.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ flexbackup-help mailing list flexbackup-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flexbackup-help
[flexbackup-help] What a shame flexbackup is *almost* there - but I have to reject it
I'm searching for a linux backup solution. And flexbackup is almost it! There are three things that make me continue looking for other solutions: 1) Crashev has asked these questions (twice): My question is - how to restore fully latest backup ? I mean if my system fails today and I want to have the freshest backup restored - how to do it with flexbackup? Should I extract fullbackup with *.0.* and then overwrite files with other days backups or how does it work? The other thing is how to to tell apart differential from incremental in such scenerio? These are *very* reasonable questions to be asking of my backup system. How do I reliably recreate the last good state? Any backup system that can't answer these questions are not up to the job, IMHO. 2) Pablo Godel asked: incremental backups and deleted files Basically, if I delete files and make incremental backups, the deletion of these files is not recorded in the incremental backups. So no, it is in fact impossible to reliably get to the last good state. Also, FAQ How do I find out which archive(s) contain a certain file? on http://www.edwinh.org/flexbackup/faq.html mentions: If you don't know in which archive to find a certain file, look at the log files. As long as you have verbose turned on (default), you can just 'zgrep filename /var/log/flexbackup/*.gz', and that works pretty well. No, that doesn't work very well. It *sucks*! (Especially if you have to restore 12342345 files and you don't know what they are.) There seems to be no way to automatically and reliably restore/extract a backup set. Adding just a little meta-data to the created backups could enable this functionality: I'm thinking a -search option to find the relevant archives and -restore to extract from the relevant archives. 3) There are three Project Admins: edwinh, jjreynold and pholcomb. EdwinH, the only one to ever post to the mailing list, last did so 2004/01/30 - over two years ago. Basically the project is without contributors and maintainers. I'm tempted to take a stab at adding the meta data to the created and backuped data and implementing -restore and -search. I've just never used a tape backup system, only to-disk. Would anybody we willing to test any such enhancements on a tape-drive? Peter - #!/bin/bash # I'd like to be able to restore src with newest file2, # file3 and file4 in it. (And no file1) # Impossible with current flexbackup, though. mkdir src echo line src/file1 echo line src/file2 echo line src/file3 echo line src/file4 flexbackup -c flexbackup.conf -set test -level 0 rm src/file1 echo anotherline src/file2 chmod go-rwx src/file3 ls -l src sleep 60 flexbackup -c flexbackup.conf -set test -level 1 -- Peter Valdemar Mørch http://www.morch.com - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ flexbackup-help mailing list flexbackup-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flexbackup-help
