I'm sorry about that. The version is 1.38.9. Thanks for the feedback. It sounds like Fifo is EOL'd as far as support in the code goes?
Actually, my purpose in using Fifo's is to use bacula for archiving to
DVD in addition to doing backups.
The problem I'm solving is that I've already been burned by DVD's that,
for whatever, reason, have an error or two on them after some time in
storage. Rather than make 3 (or however many) DVD images of whatever I'm
archiving, I've taken to using cpio, and piping the output of that into
a program that performs a FEC encoding based on Vandermonde matrices.
This, in turn, is streamed directly onto a DVD (no filesystem--just a
stream of data.)
The resulting DVD can be fed back into the FEC decoding quite well using
dd if=/dev/dvd, and then that, of course, is piped back into cpio.
Using this method, I like to make regular "permanent snapshots" of
my /home filesystem, in additional to occasional snapshots of certain
directories, like my digital photos.
Yeah, I know it violates the principle that an "archive" should be more
random-accessible than that (I have to play back through the whole
stream to get any particular file) but it gives me the "best"
combination of ease of use and administration. Using my technique (along
with another program for splitting the streams across multiple deivces),
I don't have to worry about figuring out which and how many files (e.g.,
pictures) to put onto which DVD. All I need to know is the order of
DVD's and what is in that particular stream. (kind of like a backup :-).
My hope was to use bacula to make the overall operation go even more
smoothly, basically using bacula to replace cpio + my manual
manipulation of filenames to backup. When I saw that bacula supported a
fifo device type, I was very excited because of the possibilities this
opened up.
Another thing that would work just as well--probably even better--is if
bacula provided a hook for some kind of manipulation of the data stream.
That manipulation could then be a user-supplied compression, FEC, or
whatever. It might even make sense as an additional option in the
FileSet definition.
Anyway, that's my story. Perhaps there's a better way, with bacula as it
exists, to achieve what I want? I'm open to suggestions, but the key
thing is that I want to be able to use some kind of self-supplied or
third-party FEC coding to make it so that even if there are read errors
from the DVD volume, I can still ultimately recover all of my data.
Thanks,
-Dan
On Sun, 2006-10-22 at 17:40 +0200, Kern Sibbald wrote:
> I wonder if we couldn't require a $10 donation to the project every time
> someone forgets to include the Bacula version? I'd probably be a
> multi-millionaire by now. :-)
>
> It looks like you are running some 1.39 version, and any FIFO use for the
> Device type is likely to be broken as there is no regression test for it.
> In any case, I cannot imagine any reason to use a FIFO device unless you
> have some sort of pseudo attached tape device. You are probably wasting
> your time trying to get it to work, unless there is no other way to do
> what you want.
>
> If you are trying to write DVD images, a disk Volume would do quite well.
> If you want to write directly to DVD, the current beta seems to be in
> pretty good shape.
>
> Regards,
>
> Kern
>
>
> > I'm trying to use a fifo for backup/restore.
> >
> > I have the storage set up for min and max block size = 1024.
> >
> > My runbeforejob script uses dd with ibs=1024 obs=1024 conv=sync, to pull
> > the data from the fifo and save it to a file (saving to the file is just
> > an interim test setup; I'll do something else with the data later). I
> > tried it with and without the conv=sync.
> >
> > The first few blocks seem fine. However, it always seems to write fewer
> > bytes out for the fifth block, resulting in a sixth block that starts at
> > the wrong place. This shows up in the log as something like:
> >
> > Enter dispatch_msg type=4 msg=monica-sd:
> > RestoreFiles.2006-10-22_10.40.43 Error: block.c:263 Volume data error at
> > 0:6143! Wanted ID: "BB02", got "". Buffer discarded.
> >
> > This almost feels like a block sync error inside of bacula. If bacula
> > was actually outputing a smaller block size, then the conv=sync would
> > have taken care of it, so that on restore the beginning of each block
> > would at least look right.
> >
> > Can anyone shed any light on this? Here is my storage definition:
> >
> > Device {
> > Name = SafeDVD
> > Archive Device = "/tmp/backup_fifo"
> > Device Type = Fifo
> > Media Type = "Safe DVD"
> > Minimum block size = 1024
> > Maximum block size = 1024
> > AlwaysOpen = no
> > LabelMedia = no
> > Automatic Mount = no
> > Random Access = no
> > RequiresMount = no
> > RemovableMedia = no
> > VolumePollInterval = 10
> > }
> >
> > And my scripts for backup and restore runbeforejob:
> >
> > Here's the one for backup:
> >
> > #!/bin/bash
> > #
> >
> > # create a fifo
> > #
> > /bin/rm -f /tmp/backup_fifo
> > /bin/mkfifo --mode=777 /tmp/backup_fifo
> >
> > # start the backup process reading from it
> >
> > ( nohup dd if=/tmp/backup_fifo of=/var/tmp/mybackup_ \
> > ibs=1024 obs=1024 conv=sync ) &>/dev/null </dev/zero &
> >
> >
> > And here's the one for restore:
> >
> > #!/bin/bash
> > #
> >
> > # create a fifo
> > #
> > /bin/rm -f /tmp/backup_fifo
> > /bin/mkfifo --mode=777 /tmp/backup_fifo
> >
> > # start the backup process reading from it
> >
> > ( nohup dd if=/var/tmp/mybackup_ of=/tmp/backup_fifo \
> > bs=1024 ) &>/tmp/restore.err </dev/zero &
> >
> >
> >
>
--
.
signature.asc
Description: This is a digitally signed message part
------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
