Hello, On 07/24/2012 05:36 PM, Bastian Friedrich wrote: > > Am Dienstag, 24. Juli 2012, 16:39:29 schrieb Eric Bollengier: >> >> On 07/24/2012 03:28 PM, Bastian Friedrich wrote: >>> we are using Bacula's raw block device backup features for backing up >>> virtual machines that use raw devices (logical volumes, or DRBD) as their >>> storage. As advised by the bacula documentation (obviously, for the >>> purpose of sparing storage space), we set "sparse = yes" for the >>> respective file sets. >> >> Can you be more specific on where the documentation says that we advise >> to use sparse option with a raw device ? At least, it needs to be fixed. > > http://www.bacula.org/5.2.x- > manuals/en/main/main/Configuring_Director.html#SECTION002370000000000000000 > (the section about configuring filesets in the director) > says: > > "If you explicitly specify a block device such as /dev/hda1, then Bacula > (starting with version 1.28) will assume that this is a raw partition to be > backed up. In this case, you are strongly urged to specify a sparse=yes > include option, otherwise, you will save the whole partition rather than just > the actual data that the partition contains."
It looks to be a very old comment, maybe some tricks that I'm not aware of :-) But I doubt. >> What you describe is exactly what the code is supposed to do, but it >> works well only if you restore to a filesystem that supports "sparse >> blocks", and a raw device doesn't support it. > > ACK. I haven't found any hints in that sense in the documentation. > > As said, not only the restore device needs to support sparse blocks, but also > the backup device ... No, the backup device doesn't need to support the sparse block interface, which is not something easy to implement (because it's not standard across systems), so Bacula implements a simple sparse block detection. In an other hand, as Bacula doesn't check if the destination supports sparse block, it leads to problems when restoring. >>> Recently, we discovered a problem with systems backed up in this way. >>> >>> Bacula does not only back up sparse data; it also restores the data in >>> such a way, which can result in problems. Imagine a virtual machine that >>> has a (non- sparse!) file with large chunks of zero data in its file >>> system. When these data are backed up, these null data are stored as a >>> sparse stream. >>> >>> During restore, however, this can result in a problem: The sparse data >>> (all >>> zeros) are not written to the disk; instead, the respective chunk is just >>> skipped (via seek). As a result, the virtual machine now has random data >>> in >>> its (formerly all-zero) file. >>> >>> While Bacula's auto-detection of sparse (32k-)blocks is more than welcome >>> in our scenario, it can result in real data corruption. The solution -- >>> as configured and implemented by us, though -- is flawed. >>> >>> Would you regard this behavior as a bug? Restored data are not identical >>> with what was backed up. Or is this inteded behavior (which one should, I >>> guess, document in a better fashion), and we are using the technology in >>> a wrong way? >> >> If you restore the device image to the filesystem then you copy it with >> dd to your device, it will work. > > As long as said device has been nulled before. Not in this case (not easy to implement that saids), when restoring to the filesystem, empty blocks will contains zeros, then when you copy the result file to the raw device, dd will copy zero as well. >>> Btw: When restoring a block device that has a number of zeros at its end >>> into a file (e.g., by pre-creating such a file -- see mail thread >>> "Behavior w.r.t. block device backup and restore", March 2011, for more >>> information) differs from a file created e.g. by "dd if=/dev/foo >>> of=output" directly. That being said, zeroing a raw block device before >>> restoring is not a solution for all cases. >> >> If you don't want to initialize your disk before doing the restore, you >> should switch off the sparse option. The compression will still work >> pretty very well on blocks that contains only zeros. > > Not for the first time it's a problem for us (and I mean: for us. This is not > a rant, or a wish you should change anything) that its not easily possible to > run any commands prior to a _restore_ job. I think that if Bacula was able to write zeros on sparse blocks when restoring to a raw device, it would be nice, however, it might be a bit tricky to implement, because you can write to the empty area only after you received a new sparse stream and you detected a sparse area. Maybe you can start by filling a feature request, and discuss implementation details with Kern. Bye -- Need professional help and support for Bacula ? Visit http://www.baculasystems.com ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
