Re: [flexbackup-help] What a shame flexbackup is *almost* there - but I have to reject it
Hi I didn't mean to be rude either, it's just that having a backupsystem that really works like it "should be" just doesn't exist currently (to my knowledge). For you're example, (a.conf, b.conf) you need some filesystem support to be able to replicate your changes to another system/backup. Currently there's no method/sw that I know of that is supporting windows/unix/linux and is in common practise (otherwise it's difficult to use in an enterprise environment due to the shere amount of servers and different OS's). What you maybe are searching maybe FAM (File Alteration Monitor)?!? http://oss.sgi.com/projects/fam/faq.html And use of a FAM mirror http://www.linuxfocus.org/common/src/article199/fam_mirror Unless you set up something like that, you're not getting any solution to your a.conf/b.conf problem, the "old" methods" of full/incremental/differential backups just doesn't provide that functionality. Also isn't a tape backup system flexible enough, or fast enough to support that. > I will not trust my system+data to such a backup-system. Are *you* > comfortable with that? What is the down side of being able to restore > reliably? Rather too much than too little data protection is what keeps an enterprise datacenter running... > You are right, I don't know "EMC Networker, Veritas Netbackup, HP Data > Protector", but I'm willing to bet they can restore a directory > containing *exactly* b.conf. (Am I right?) They can, if you really have an full backup just before the crash/logical accident happens. For incrementals, it's just to restore full+incrementals and get all those a.conf's as well ;-( --Robert On Fri 4 August 2006 09:50, Peter Valdemar Mørch wrote: > flexbackup-at-worreby.ch |Lists| wrote: > > 1. Have you the slightest idea how it's done in the reality? > > > > That's standard procedure to restore full backup plus all > > incrementals you have, if you invented some other schema, > > please let me know > > (at work we're currently backing up between 15-20 TB / night) > > and > > > 2. What's worse? Dataloss or too much data!!! > > Let us first agree that data loss is totally unacceptable. > I'm not arguing that data loss is a good thing. (Am i? Where?) > > But for me, "too much data" is *ALSO* unacceptable, because it does not > represent reality. Let me illustrate with a scenario: > > * I start with a directory e.g. under Apache's configuration that >has a single file a.conf > * I make a full backup. > * rm a.conf > * Add a file b.conf > * (Notice that at no point in time was there ever more >than one file in the directory) > * I make an incremental backup. > * Hard disk crash > * Restore full backup > * Restore incremental backup > > Result with current flexbackup: > > A directory containing both a.conf AND b.conf. > > Wouldn't you rather end up with a directory containing *exactly* b.conf > - the exact contents of the directory when the incremental backup was made? > > Ok, so in a directory where there is supposed to be 1 single file you > may be able to remember yourself to delete a.conf (because it is *not* > supposed to be there). Lets just hope I slept well and didn't remove > b.conf instead by accident because I was tired... ;) > > But with "between 15-20 TB" of data as you put it? Who can remember > which of the gazillion files to delete? > > No, I don't want that to be a guessing game. > > In some (most?) cases, "too much data" is just annoying - not really > catastrophic. But in some cases (e.g. conf.d, cron.d, *.d directories, > or a file containing sensitive data that was deleted) this *is* > catastrophic. I could end up with a system that won't boot or misbehaves > or is dangerous after a full+incremental restore. > > I will not trust my system+data to such a backup-system. Are *you* > comfortable with that? What is the down side of being able to restore > reliably? > > You are right, I don't know "EMC Networker, Veritas Netbackup, HP Data > Protector", but I'm willing to bet they can restore a directory > containing *exactly* b.conf. (Am I right?) > > Peter -- --Robert Robert Worreby Birkenweg 82 CH-3123 Belp http://counter.li.org pgpEvGqhQntHR.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV___ flexbackup-help mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/flexbackup-help
Re: [flexbackup-help] What a shame flexbackup is *almost* there - but I have to reject it
On Fri, 4 Aug 2006, Peter Valdemar Mørch wrote: > flexbackup-at-worreby.ch |Lists| wrote: > > 1. Have you the slightest idea how it's done in the reality? > > That's standard procedure to restore full backup plus all > > incrementals you have... That is standard procedure, unless you have fancy expensive tools. > Wouldn't you rather end up with a directory containing *exactly* b.conf > - the exact contents of the directory when the incremental backup was made? Sure, but to create such a thing, you need a considerably more sophisticated process for creating incremental backups. The 'standard procedure' backs up all data created or modified since the last full backup. You only need the metadata of all the current data to do that, and that's all in the file system. You want deletion records for all data deleted since the last backup. You need both old and new metadata to construct that. Only the new metadata is contained in the file system. You also need an archive structure which contains deletion records. AFAIK tar, cpio etc don't handle that. > You are right, I don't know "EMC Networker, Veritas Netbackup, HP Data > Protector", but I'm willing to bet they can restore a directory > containing *exactly* b.conf. (Am I right?) I don't know. Is that question relevant to this list? Are you offering to sponsor the development of the features you find missing? Or are you complaining because the gift you received isn't exactly what you want? -- Charlie - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ flexbackup-help mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/flexbackup-help
Re: [flexbackup-help] What a shame flexbackup is *almost* there - but I have to reject it
flexbackup-at-worreby.ch |Lists| wrote: > 1. Have you the slightest idea how it's done in the reality? > That's standard procedure to restore full backup plus all > incrementals you have, if you invented some other schema, > please let me know > (at work we're currently backing up between 15-20 TB / night) and > 2. What's worse? Dataloss or too much data!!! Let us first agree that data loss is totally unacceptable. I'm not arguing that data loss is a good thing. (Am i? Where?) But for me, "too much data" is *ALSO* unacceptable, because it does not represent reality. Let me illustrate with a scenario: * I start with a directory e.g. under Apache's configuration that has a single file a.conf * I make a full backup. * rm a.conf * Add a file b.conf * (Notice that at no point in time was there ever more than one file in the directory) * I make an incremental backup. * Hard disk crash * Restore full backup * Restore incremental backup Result with current flexbackup: A directory containing both a.conf AND b.conf. Wouldn't you rather end up with a directory containing *exactly* b.conf - the exact contents of the directory when the incremental backup was made? Ok, so in a directory where there is supposed to be 1 single file you may be able to remember yourself to delete a.conf (because it is *not* supposed to be there). Lets just hope I slept well and didn't remove b.conf instead by accident because I was tired... ;) But with "between 15-20 TB" of data as you put it? Who can remember which of the gazillion files to delete? No, I don't want that to be a guessing game. In some (most?) cases, "too much data" is just annoying - not really catastrophic. But in some cases (e.g. conf.d, cron.d, *.d directories, or a file containing sensitive data that was deleted) this *is* catastrophic. I could end up with a system that won't boot or misbehaves or is dangerous after a full+incremental restore. I will not trust my system+data to such a backup-system. Are *you* comfortable with that? What is the down side of being able to restore reliably? You are right, I don't know "EMC Networker, Veritas Netbackup, HP Data Protector", but I'm willing to bet they can restore a directory containing *exactly* b.conf. (Am I right?) Peter -- Peter Valdemar Mørch http://www.morch.com - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ flexbackup-help mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/flexbackup-help
Re: [flexbackup-help] What a shame flexbackup is *almost* there - but I have to reject it
Hi A couple of comments on your *concerns* 1. Have you the slightest idea how it's done in the reality? That's standard procedure to restore full backup plus all incrementals you have, if you invented some other schema, please let me know (at work we're currently backing up between 15-20 TB / night) 2. What's worse? Dataloss or too much data!!! ;-) It's up to you to decide that, we do prefer to have more data than less data, that can always be overcome to reduce the duplicate ones, the non-existing ones are a lot more difficult to get. Regarding finding files, so is flexbackup similar to how the EMC Networker, Veritas Netbackup, HP Data Protector is working, they are all depending on your index/internal database over all files to get to them quickly. 3. All help is welcome, the open-source community needs people that make helpful work. Regarding Tape management so privately I'm just backing up to disk, is a lot easier and cheaper than to buy a tape library at home. (do got several DAT Exchanger at home, but is not just using them) --Robert On Fri 28 July 2006 14:23, Peter Valdemar Mørch wrote: > I'm searching for a linux backup solution. And flexbackup is almost it! > There are three things that make me continue looking for other solutions: > > 1) Crashev has asked these questions (twice): > > My question is - how to restore fully latest backup ? I mean if my > > system fails today and I want to have the freshest backup restored - > > how to do it with flexbackup? Should I extract fullbackup with *.0.* > > and then overwrite files with other days backups or how does it work? > > The other thing is how to to tell apart differential from > > incremental in such scenerio? > > These are *very* reasonable questions to be asking of my backup system. > How do I reliably recreate the "last good state"? Any backup system that > can't answer these questions are not up to the job, IMHO. > > 2) Pablo Godel asked: incremental backups and deleted files > > Basically, if I delete files and make incremental backups, the deletion > of these files is not recorded in the incremental backups. So no, it is > in fact impossible to reliably get to the last good state. > > Also, FAQ "How do I find out which archive(s) contain a certain file?" > > on http://www.edwinh.org/flexbackup/faq.html mentions: > > If you don't know in which archive to find a certain file, look at > > the log files. As long as you have verbose turned on (default), you > > can just 'zgrep filename /var/log/flexbackup/*.gz', and that works > > pretty well. > > No, that doesn't work very well. It *sucks*! (Especially if you have to > restore 12342345 files and you don't know what they are.) There seems > to be no way to automatically and reliably restore/extract a backup set. > > Adding just a little meta-data to the created backups could enable this > functionality: I'm thinking a -search option to find the relevant > archives and -restore to extract from the relevant archives. > > 3) There are three Project Admins: edwinh, jjreynold and pholcomb. > EdwinH, the only one to ever post to the mailing list, last did so > 2004/01/30 - over two years ago. Basically the project is without > contributors and maintainers. > > I'm tempted to take a stab at adding the meta data to the created and > backuped data and implementing -restore and -search. I've just never > used a tape backup system, only to-disk. Would anybody we willing to > test any such enhancements on a tape-drive? > > Peter > > - > #!/bin/bash > # I'd like to be able to restore src with newest file2, > # file3 and file4 in it. (And no file1) > # Impossible with current flexbackup, though. > > mkdir src > > echo line > src/file1 > echo line > src/file2 > echo line > src/file3 > echo line > src/file4 > flexbackup -c flexbackup.conf -set test -level 0 > > rm src/file1 > echo anotherline > src/file2 > chmod go-rwx src/file3 > ls -l src > > sleep 60 > flexbackup -c flexbackup.conf -set test -level 1 -- --Robert Robert Worreby Birkenweg 82 CH-3123 Belp http://counter.li.org pgpOtyJqSTmDp.pgp Description: PGP signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV___ flexbackup-help mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/flexbackup-help
