Hello,
Yes, something like this could work.
However, here are some thoughts about the changes and challenges:
1. New code would be needed to be added to the Director so that it knows which
SDs to contact.
2. Only the appropriate part of the bsr should be sent to the FD for each SD,
or the FD would need to know how to parse the bsr file. In any case, either
the DIR, FD, or SD would need to know only to use a particular part of the
bsr -- this seems a bit tricky to me. It would probably be better to have a
chain of bsr files to be processed and the Director would process them one at
a time. Each file would deal only with one SD.
3. Error handling is a bit more complicated:
a. The FD is contacted, then it must wait, until the Dir
contacts the SD. If either have problems, everything must
be properly cleaned up.
b. The Director currently prints the SD status in the job report,
I suppose it would need some way to print an arbitrary number of
SD job statuses with the Names of each SD.
4. The FD has to be taught how to contact multiple SDs before terminating and
it must know that each is a continuation of the current job. This is not
currently the case.
5. It may or may not be simple to make this upward compatible -- in
otherwords, it would not be very acceptable if a user was required to upgrade
all his FDs if he/she did not want to use this feature. There are
installations of Bacula that consist of several thousand clients.
There may be other details to consider that I have not thought about.
Best regards,
Kern
On Friday 13 March 2009 12:38:14 Graham Keeling wrote:
> Hello,
> Here are my answers to the rest of your comments.
>
> On Thu, Mar 12, 2009 at 06:44:54PM +0100, Kern Sibbald wrote:
> > Hello,
> >
> > I think what you are asking for is probably a much bigger issue than you
> > might imagine. Hopefully what follows will give a rough overview the
> > issues.
> >
> > The scheme you are suggesting has a number of things that worry me:
> >
> > 1. As a general principle, it is not a good idea to be cycling through
> > storage daemons seeing if they work or not, which is what I think you are
> > proposing unless I misunderstood it.
> >
> > 2. Under the current design, the Director *must* first connect to the SD
> > then connect to the FD. This is required to prepare the SD for accepting
> > a communication from the FD. Changing this would require a major design
> > review as it involves the low level details of ensuring security, and
> > before undertaking such a change, I would really need to be convinced of
> > the need to make such a change ...
>
> Under the current design:
>
> The director connects to the storage daemon, and gets an sd_auth_key.
> The director then connects to the file daemon, and gives it the
> sd_auth_key with the 'jobcmd'.
> (restoring of files happens)
> The director does a 'wait_for_storage_daemon_termination()'.
> The director waits for the file daemon to indicate the end of the job.
>
> With my idea:
>
> The director connects to the file daemon.
> Then, for each storage daemon in the .bsr file... {
> The director connects to the storage daemon, and gets an sd_auth_key.
> The director then connects to the file daemon, and gives it the
> sd_auth_key with the 'storaddr' command.
> (restoring of files happens)
> The director does a 'wait_for_storage_daemon_termination()'.
> The director waits for the file daemon to indicate the end of the
> work on this storage.
> }
> The director tells the file daemon that there are no more storages to
> contact. The director waits for the file daemon to indicate the end of the
> job.
>
> As you can see, each restore between the file daemon and storage daemon is
> handled in the same way that it is currently handled, using the same method
> for authentication, except that the sd_auth_key is moved from the 'jobcmd'
> to the 'storaddr' command - where it logically belongs.
>
> > 3. I am not convinced that any given bootstrap file should contain a
> > reference to more than one SD or what it would mean.
>
> ...[snip]...
>
> > 5. I have always planned to have SD <-> SD communications. For example a
> > migration job that migrates from one SD to another. Any addition of
> > Storage= in bsr files would need to be carefully designed not to
> > interfere with this. If this feature is properly implemented, it might
> > even be capable of giving you what you want -- in fact, it goes more in
> > the direction that David Boyes has been advocating for some time to have
> > a sort of Virtual Storage Manager or Virtual Volume Manager interposed
> > between the Director and the Storage daemon(s).
>
> As it currently stands, the director may find that it needs to restore from
> more than one storage daemon. The bootstrap file might currently look like
> this:
>
> Volume="backup-0001"
> MediaType="File"
> Device="Dev 1.0"
> VolSessionId=1
> VolSessionTime=1236768303
> VolAddr=245-956251
> FileIndex=39-45
> FileIndex=425
> ...(more FileIndexes)...
> Count=110
> Volume="backup-0003"
> MediaType="File"
> Device="Dev 2.0"
> VolSessionId=3
> VolSessionTime=1236768303
> VolAddr=245-608410
> FileIndex=6-44
> FileIndex=55-92
> ...(more FileIndexes)...
> Count=442
>
> However, "Dev 1.0" and "Dev 2.0" are actually devices controlled by
> different storage daemons. The first only knows about "Dev 1.0", and the
> second only knows about "Dev 2.0".
> Therefore, it is pointless to send the "Dev 2.0" section to the first
> storage daemon, and pointless to send the "Dev 1.0" section to the second
> storage daemon. Currently, both sections will be sent to the first storage
> daemon, and the restore will fail.
>
> With my idea, the bootstrap file would look like this:
>
> Storage="SD 1"
> Volume="backup-0001"
> MediaType="File"
> Device="Dev 1.0"
> ...(and so on)...
> Storage="SD 2"
> Volume="backup-0003"
> MediaType="File"
> Device="Dev 2.0"
> ...(and so on)...
>
> When the director reads this file, it knows exactly which storage daemons
> to point the file daemon to.
>
> It is worth noting that what I am suggesting here can probably be seen as
> a kind of 'super.bsr' file that only the director understands.
> I am not saying that the storage daemon receives one of these. The storage
> daemon just receives something that looks exactly like the current .bsr
> files.
>
> As it currently stands, a .bsr file read by the director cannot cover some
> cases because the storage daemons aren't specified.
>
> > 4. Any change in how the Storage daemon is specified needs to be
> > carefully thought out, because the current specification allows for
> > multiple storage devices (within one storage daemon) to be specified --
> > this is already implemented and I believe it works, but it is not
> > documented, and it also permits specifying multiple storage daemons, but
> > the code for this is probably not complete and not working. (see below
> > for more "specifics").
>
> I have not proposed any changes in how the storage daemon is specified.
> I cannot see how it makes any difference at all to multiple storage devices
> within one storage daemon.
>
>
> Thanks,
> Graham.
>
>
> ---------------------------------------------------------------------------
>--- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> Bacula-devel mailing list
> Bacula-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-devel
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Bacula-devel mailing list
Bacula-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel
