Hello, Thanks to those who sent in suggestions. While talking to Eric about this, I came up with the solution that appeals to me the most:
We add new functionality between the FD and the SD. 1. FD asks SD to Open named Spool file (all subsequent data will go there) 2. SD sends back spool name. 3. FD can ask to stop spooling 4. The FD can then start spooling with a new name (item 1) if it wants. 5. The FD can ask the SD to commit a specified named spool file. 6. The SD sends back the status. 7. For the moment, only one spool can be open at a time. This resolves the problem since the FD can begin spooling into a named spool file for the File data, when it is done, it stops spooling and simply writes in non-spooled mode, then commits the file data. Result, they are in order, and if anything goes wrong during spooling of the file data, the FD is notified and the VSS backup is canceled properly. One minor variation would have the FD start spooling for the XML data, then stop spooling when it is done, and despool the XML data, then despool the file data -- resulting in the correct order for restore. Constraint: the SD must have sufficient spool space, which is generally never a problem. I am going to think about this a day or two, wait for James' reaction, then begin implementation of it. Best regards, Kern On Friday 02 April 2010 13:13:55 Kern Sibbald wrote: > Hello, > > Over the past several weeks James and I have been discussing a rather > sticky point with doing System State backup and restores via a Bacula > plugin. > > The basic problem works down to the fact that we do the backup, we first > have the names of the files to backup, which we do, then we get some > special writer XML data to save. This is no problem. > > However, on the restore, to properly restore to writers, we must first > present the XML data, then we can write the file data. This is a real > problem because Bacula puts data sequentially on a Volume (tape or disk) > and then restores it sequently. There is no mechanism in Bacula to write > some data after backing up files, then get that data *before* restoring the > files. > > For a typical system state backup, this XML metadata represents something > like the following in bytes: > > 2604 > 463820 > 754 > 3378 > 1610 > 4636 > 1414 > > there are a number of meta data files, because the system state backup > consists of backing a number of different "components" > > Now, we could compress this data, and store it in the catalog, but that is > a bit "ugly" and requires transmitting it back to the FD in some way during > a restore. Compressing the data reduces it to about 20-30KB rather then > the 500KB. > > Another idea is to spool the file data in either the File daemon or Storage > daemon, until we receive and write the meta data to the Volume, and then > despool the file data -- thus during a restore the xml meta data would be > restored (given to the plugin) first. > > On James' system the file data backed up during s systems state is about > 200MB (not a lot to spool). However, he believes that he has seen > something like 1GB of data produced by ntbackup, so that would be a lot > more data to spool. One of the problems of spooling, is that if it fails, > it will be relatively serious, because in order to get the meta data, we > need to tell VSS that the backup of the files was complete, which changes > the state of VSS (i.e. if you are doing an incremental backup of the system > log files, you cannot just repeat the operation). > > Does anyone have any ideas or suggestions preferences? > > Best regards, > > Kern ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Bacula-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-devel
