Re: [Bacula-users] [Bacula-devel] Restoring from volumes on multiple storages
On Wednesday 04 March 2009 18:31:06 Graham Keeling wrote: On Thu, Feb 26, 2009 at 02:23:04PM +, Graham Keeling wrote: One kludgy solution that I may have a go at trying is to make the director write out a separate .bsr file containing the correct set of volumes that it needs for each Storage, and then doing multiple run job= commands. This will set off multiple restores and you'd end up with multiple confirmation messages. It might all go horribly wrong if you're using plugins, mind. After trying to make the above work and not liking it very much, I realised that there might be a better way. The way that a restore job works is that the director... a) opens a connection to the storage daemon b) opens a connection to the file daemon c) sends the storage daemon address to the file daemon and waits for a response (file daemon contacts the storage daemon) d) sends the bootstrap file to the file daemon e) sends the restore commands to the file daemon (file daemon sends the bootstrap file to the storage daemon and starts receiving data) f) waits for the file daemon to finish g) waits for the storage daemon to finish I think it's possible to change the process so that the director and file daemons open connections to a series of storage daemons during one job, and the file daemon sends each storage daemon the relevant segment of .bsr file. The director could insert 'Storage =' lines into the .bsr file and the file daemon could parse the individual chunks out to the storage daemons. I may have a go at making it do this. Since I am buried in emails, I am probably reading this out of context, and will attempt to find the start of the thread a bit later today. So I apologize in advance if I misunderstand your request. Please note that when discussing this subject, it is important to distinguish between multiple Devices within a given Storage daemon, and multiple Storage daemons (different binary files). The director directives currently used can easily lead to confusion of these two different concepts. My first point is that at the current time, Bacula does not support multiple storage daemons for a single Job. It will happily write jobs to different storage daemons, but has no current way of doing a restore from multiple storage daemons. In fact, it was only in 2.2.x that we introduced the ability for the Director to switch drives within a single Storage daemon during *restore*, and though this code seems to work, I would hesitate to say that it is currently officially supported. Note, once a jobs starts *writing* on a given drive, the current Bacula cannot switch drives later (we have a design project to fix this). Now, if the problem you are reporting has to do with multiple or different devices within a single Storage daemon, then it falls under the new 2.2.x code, which should work. If you are really talking about restoring a single job from multiple different storage daemons, then it seems to me that we should start from a Feature Request, and either in that request or as an additional document write a detailed technical document on how we plan to do this -- only after that is thouroughly reviewed should we begin working on coding -- you may not know it, because a some of these design details are done off line (often in kernstodo), but this is how we do all major development projects. Certainly the ability to work (both backup and restore) with multiple storage daemons within a single job could be a very useful feature, but let's define clearly the need and then design the implementation. Best regards, Kern -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula-devel] Restoring from volumes on multiple storages
On Thu, Feb 26, 2009 at 02:23:04PM +, Graham Keeling wrote: One kludgy solution that I may have a go at trying is to make the director write out a separate .bsr file containing the correct set of volumes that it needs for each Storage, and then doing multiple run job= commands. This will set off multiple restores and you'd end up with multiple confirmation messages. It might all go horribly wrong if you're using plugins, mind. After trying to make the above work and not liking it very much, I realised that there might be a better way. The way that a restore job works is that the director... a) opens a connection to the storage daemon b) opens a connection to the file daemon c) sends the storage daemon address to the file daemon and waits for a response (file daemon contacts the storage daemon) d) sends the bootstrap file to the file daemon e) sends the restore commands to the file daemon (file daemon sends the bootstrap file to the storage daemon and starts receiving data) f) waits for the file daemon to finish g) waits for the storage daemon to finish I think it's possible to change the process so that the director and file daemons open connections to a series of storage daemons during one job, and the file daemon sends each storage daemon the relevant segment of .bsr file. The director could insert 'Storage =' lines into the .bsr file and the file daemon could parse the individual chunks out to the storage daemons. I may have a go at making it do this. -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula-devel] Restoring from volumes on multiple storages
On Thu, Feb 26, 2009 at 02:23:04PM +, Graham Keeling wrote: I've now looked at the bacula code a bit, and what I think it does is this (bear in mind that I'm not one of the proper bacula development guys, so I might be talking utter rubbish here): When you tell it to restore something, it creates a single .bsr file. Though the director knows the Storage that has each volume, the .bsr file does not contain that information. The director then takes that .bsr file and sends a run job= command back through the bconsole interface, like this (with the %s bits filled in - src/dird/ua_restore.c): run job=\%s\ client=\%s\ restoreclient=\%s\ storage=\%s\ bootstrap=\%s\ files=%u catalog=\%s\ As you can see, the Storage is set outside of the bootstrap file, and I think you can only specify one of them. So, that single Storage gets told to restore from all of the volumes, even if the director knows that some of the volumes shouldn't be there. I suppose that it's possible that if there was a volume in that place with the correct label, it would happily restore from it and give you odd files. One kludgy solution that I may have a go at trying is to make the director write out a separate .bsr file containing the correct set of volumes that it needs for each Storage, and then doing multiple run job= commands. This will set off multiple restores and you'd end up with multiple confirmation messages. It might all go horribly wrong if you're using plugins, mind. ...and another thing: I just noticed that the Virtual Backup / Consolidation stuff seems to work in a similar fashion. Unless I'm missing something, it looks like it just creates a single .bsr file and sends it off to a single Storage. -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
