Re: [Bacula-users] Restoring from a copy job
On 11/30/21 14:54, Josip Deanovic wrote: On Tuesday 2021-11-30 12:07:37 Phil Stracchino wrote: The question then ceases to be "I have twenty possible volumes from which your restore could be performed, which five of the twenty volumes do you want to use?", and instead becomes, "I have four possible SOURCES to perform this restore from, please pick *one restore source*." The restore command of the bconsole tool already has the "pool" command line argument which could be used for volume selection. In my case with Bacula 9.6.7 it didn't work with copy jobs. It would be nice to make it work with newer Bacula versions (e.g. 11.x). Eric mentioned that MediaType might potentially be used in the selection algorithm. I am not sure if this is necessary if we get a working "pool=" argument. Indeed, and Pool is probably a better discriminator in this case than MediaType. In my case, the two relevant pools for Full backups are Full-Disk and Full-Archive, and both are Media Type = File. In any random hypothetical environment with copy jobs, I would expect it to be far more likely that Copy jobs will be in a different Pool than that they will have a different media type. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restoring from a copy job
On Tuesday 2021-11-30 12:07:37 Phil Stracchino wrote: > > However, in a case where there are two or more complete sets of jobs on > different pools or devices, both containing all of the jobs necessary to > perform the desired restore, it SEEMS that it should be conceptually > simple to list the jobs and sources and allow the user/administrator to > select a restore SOURCE, then automatically select the volumes and jobs > required to perform the restore from that source. This seems much > simpler than trying to choose from all possible sets of jobs/volumes. > > The question then ceases to be "I have twenty possible volumes from > which your restore could be performed, which five of the twenty volumes > do you want to use?", and instead becomes, "I have four possible SOURCES > to perform this restore from, please pick *one restore source*." The restore command of the bconsole tool already has the "pool" command line argument which could be used for volume selection. In my case with Bacula 9.6.7 it didn't work with copy jobs. It would be nice to make it work with newer Bacula versions (e.g. 11.x). Eric mentioned that MediaType might potentially be used in the selection algorithm. I am not sure if this is necessary if we get a working "pool=" argument. -- Josip Deanovic ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restoring from a copy job
On 11/30/21 11:01, Bill Arlofski via Bacula-users wrote: On 11/30/21 01:38, Eric Bollengier wrote: Note, that it's also possible to have multiple copies of each job, on different kind of devices, and some jobs involved may not have copies. Now, if we can determine a selection algorithm (via MediaType?), we can probably implement it. Bconsole is not the best tool for advanced selection screens, however Baculum GUI has an excellent supports the copy job selection and can also display the different versions of a file, and let you restore the one you want for example. Yes, this is true, and something people (like me for example) often forget - A backup job may be copied many times, and to every different storage type - So making a guess which copy the user wants to restore from would be impossible. :) So, the workflow as I described in my follow-up replay last night currently is the right one and it makes more sense considering the multiple copies scenario. However, in a case where there are two or more complete sets of jobs on different pools or devices, both containing all of the jobs necessary to perform the desired restore, it SEEMS that it should be conceptually simple to list the jobs and sources and allow the user/administrator to select a restore SOURCE, then automatically select the volumes and jobs required to perform the restore from that source. This seems much simpler than trying to choose from all possible sets of jobs/volumes. The question then ceases to be "I have twenty possible volumes from which your restore could be performed, which five of the twenty volumes do you want to use?", and instead becomes, "I have four possible SOURCES to perform this restore from, please pick *one restore source*." -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restoring from a copy job
On 11/30/21 01:38, Eric Bollengier wrote: > Hello, > > On 11/30/21 04:52, Bill Arlofski via Bacula-users wrote: >> >> >> So far, so good... BUT it gets better because Bacula notices that I have >> copies of these five jobs! Excellent! >> >> So it prints: >> >> These JobIds have copies as follows: >> ++---+---+---+ >> | jobid | job | copyjobid | mediatype | >> ++---+---+---+ >> | 43,803 | Speedy.2021-11-29_03.15.00_39 |43,813 | Synology-File | >> | 43,777 | Speedy.2021-11-28_03.15.00_39 |43,788 | Synology-File | >> | 43,751 | Speedy.2021-11-27_03.15.00_39 |43,762 | Synology-File | >> | 43,725 | Speedy.2021-11-26_03.15.00_39 |43,736 | Synology-File | >> | 43,699 | Speedy.2021-11-25_03.15.00_08 |43,710 | Synology-File | >> ++---+---+---+ >> >> But then this is where the breakdown is... >> >> Bacula ignores those five copy jobids, and reports that it is building the >> restore tree from the original backups jobids: >> >> You have selected the following JobIds: 43699,43725,43751,43777,43803 >> >> Building directory tree for JobId(s) 43699,43725,43751,43777,43803 ... >> + >> 554,180 files inserted into the tree. >> >> >> So, I think I need to report this. (or amend the Mantis if one exists :) >> > > The command prints the alternative to the different jobs, I don't think it can > choose for you, here in this example, we have 2^5 different possibilities to > restore the files, I'm not sure Bacula can guess automatically which one do > you want in the mix. > > The basic idea is that the original job is usually stored on fast and > available > disks, the copy is more likely stored on tape or cloud kind of slow access > device. > > When the original job is purged (usually you have a smaller retention), then > the > copy job is "promoted", and the next restore command will select it > automatically. > > Note, that it's also possible to have multiple copies of each job, on > different > kind of devices, and some jobs involved may not have copies. Now, if we can > determine a selection algorithm (via MediaType?), we can probably implement > it. > > Bconsole is not the best tool for advanced selection screens, however Baculum > GUI has an excellent supports the copy job selection and can also display the > different versions of a file, and let you restore the one you want for > example. > > Best Regards, > Eric > Hello Eric! Thank you for the detailed reply. Yes, this is true, and something people (like me for example) often forget - A backup job may be copied many times, and to every different storage type - So making a guess which copy the user wants to restore from would be impossible. :) So, the workflow as I described in my follow-up replay last night currently is the right one and it makes more sense considering the multiple copies scenario. Best regards, Bill -- Bill Arlofski w...@protonmail.com ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] META: Repeated posts
Hello Gary, > I don't know if this is down to Sourceforge or the OP, but every ten > minutes my filters catch a duplicate delivery of Bill Arlofski's message > with the Message ID: . Not me. I use an aaS Zimbra hosted in Deutschland and administrated by a friend in Brazil: https://www.kyahosting.com/ (@anahuac on Telegram), and I can't see anything weird on the https://spfbl.net/en/ filter. Regards, -- MSc Heitor Faria (Miami/USA) Bacula LATAM CEO mobile1: + 1 909 655-8971 mobile2: + 55 61 98268-4220 [ https://www.linkedin.com/in/msc-heitor-faria-5ba51b3 ] [ http://www.bacula.com.br/ ] América Latina [ http://bacula.lat/ | bacula.lat ] | [ http://www.bacula.com.br/ | bacula.com.br ] ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] META: Repeated posts
Hi, I don't know if this is down to Sourceforge or the OP, but every ten minutes my filters catch a duplicate delivery of Bill Arlofski's message with the Message ID: . So either Bill's mail system has the hiccups, or Sourceforge has been into the cheap meth. I can't see multiple copies in the archives, so I'd put my money on Sourceforge... Cheers, GaryB-) ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Restore Session
Josip DeanovicOn Tuesday 2021-11-30 10:27:08 wrote: > On Tuesday 2021-11-30 15:29:42 Gary R. Schmidt wrote: > > Also, be aware that you have created a completely new file system, all > > the inodes will be different, and the access/modify/change/birth times > > on all the directories will be set to when the new directory was > > created - i.e. when it started to be populated by the restore. > > > > And everything will have different dev and rdev values. > > The documentation says that Bacula decides what files to backup for > Incremental and Differential backup by comparing the change (st_ctime) > and modification (st_mtime) times of the file to the time the last > backup completed. > > Could it be that the "Accurate" option changes that behavior when used > in the Job resource? > > If that is the case then it could be controlled by the "Accurate" option > used in the FileSet resource. Just did a simple test. What changes during the restore is the change time (st_ctime). That's why Bacula decides to backup restored files with the next Incremental backup. There is an option "mtimeonly=yes" which could be used in the FileSet. This would solve the problem Neil currently has to deal with. I would suggest to remove that option once it is not needed any more as it could have undesirable effects on the long run. Adding and removing of this option to the FileSet resource will not trigger promotion of Incremental job to Full unless Include or Exclude lists are changed as well. Regards! -- Josip Deanovic ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restoring from a copy job
On Tuesday 2021-11-30 04:54:35 Bill Arlofski via Bacula-users wrote: > Hello Josip, > > I did a little research and found the old Mantis ticket I was referring > to... > > However, the result from the conversation with the developers was as > follows: > > > The "copies" option is doing only what is described in the manual: > 8< > If the keyword copies is present on the command line, Bacula > Enterprise will DISPLAY the list of all copies for selected jobs. > 8< > > So, if you actually you want to restore from the copies, you need to > cancel the restore at the point where the copy jobids are displayed, > then run the restore job again but instead of specifying the client and > fileset you can specify the copy jobids as follows: > > * restore jobid=1,2,3,4,5 (where 1,2,3,4,5 are the jobids of the copy > jobs) > > It is not the way I would expect or hope the 'copies' option would > function, but at least it is "functioning as documented"™ :) > > Hope this helps. :) Thank you Bill, it helped and restore from copy jobs from tape pool works without issues now when I am properly using it. I expected the "copies" argument to work exactly as the documentation stated: "display the list of all copies for selected jobs" but I missed and completely ignored the "copyjobid" column given in the list. I was steered in the wrong direction by the part of the documentation covering restore command line arguments which says: "- pool=pool-name - specify a Pool name to be used for selection of Volumes when specifying option 5 and 6 (restore current system, and restore current system before given date). This permits you to have several Pools, possibly one offsite, and to select the Pool to be used for restoring." That part with specifying pool (in my case tape1) doesn't work although I am using separate storage, device and media type for that specific pool. Maybe there is a bug regarding that because when I specify a pool, after I select option 5 and then client and its particular job, it immediately returns this output: "No Full backup before 2021-11-30 09:42:58 found." Thanks again, you helped a lot and I can confirm that restoring from copy jobs is working with Bacula 9.6.7 when jobid= is used with the values from the "copyjobid" column listed by "copies" argument. Regards! -- Josip Deanovic ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Restore Session
On Tuesday 2021-11-30 15:29:42 Gary R. Schmidt wrote: > Also, be aware that you have created a completely new file system, all > the inodes will be different, and the access/modify/change/birth times > on all the directories will be set to when the new directory was > created - i.e. when it started to be populated by the restore. > > And everything will have different dev and rdev values. The documentation says that Bacula decides what files to backup for Incremental and Differential backup by comparing the change (st_ctime) and modification (st_mtime) times of the file to the time the last backup completed. Could it be that the "Accurate" option changes that behavior when used in the Job resource? If that is the case then it could be controlled by the "Accurate" option used in the FileSet resource. Regards! -- Josip Deanovic ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restoring from a copy job
On Tuesday 2021-11-30 08:51:40 Radosław Korzeniewski wrote: > Hello, > > wt., 30 lis 2021 o 06:01 Bill Arlofski via Bacula-users < > > bacula-users@lists.sourceforge.net> napisał(a): > > It is not the way I would expect or hope the 'copies' option would > > function, but at least it is "functioning as documented"™ :) > > The way my customers are using it is when "originals" are not available > for restore, then restore from copies works perfectly as expected. So, > when doing copy for offline, off-site storage, disaster recovery, long > term, low-cost storage, etc. etc. Yes. In my case I needed it to test the offsite storage, media and backup itself. While I was googling for the solution, I have found that people often misinterpret the documentation which says that Bacula will automatically promote copy jobs if the records about original jobs are purged from the catalog. It seems that people read the last part as: "if the original jobs are not available" which is not correct. Their records must be deleted from the database. > It is not perfect and does not cover all use cases you can imagine for > sure. But do the real work when required. I am fine with it as long there is a way to perform restore from copy jobs. I don't mind if it takes an extra step as long as it works. :-) Regards! -- Josip Deanovic ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restoring from a copy job
On Tuesday 2021-11-30 09:19:51 Marcin Haba wrote: > Hello Josip, > > Baculum supports restore from copy jobs. It is a function in the > restore wizard that default is enabled. So you can just select copy > jobs and restore from them all or selected files. This function is > available from Baculum version 9.6.0. > > If you prefer doing it using Bconsole, you can use Bvfs restore > commands for that. But please be careful with .bvfs_get_jobids command > in version lower than 11.0.3, because there was a bug in this command > with copy jobs: > > https://bugs.bacula.org/view.php?id=2500 > > Besides that it works well. Thank you for confirming it works on 9.6.x. I am currently not using Baculum but Bill helped me to find the solution. In short, the solution is: 1. use "restore copies" to get the list of IDs of copy jobs 2. use "jobid=copyjob1,copyjob2" to select the copy jobs listed in the copyjobid column Regards! -- Josip Deanovic ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restoring from a copy job
Hello, On 11/30/21 04:52, Bill Arlofski via Bacula-users wrote: So far, so good... BUT it gets better because Bacula notices that I have copies of these five jobs! Excellent! So it prints: These JobIds have copies as follows: ++---+---+---+ | jobid | job | copyjobid | mediatype | ++---+---+---+ | 43,803 | Speedy.2021-11-29_03.15.00_39 |43,813 | Synology-File | | 43,777 | Speedy.2021-11-28_03.15.00_39 |43,788 | Synology-File | | 43,751 | Speedy.2021-11-27_03.15.00_39 |43,762 | Synology-File | | 43,725 | Speedy.2021-11-26_03.15.00_39 |43,736 | Synology-File | | 43,699 | Speedy.2021-11-25_03.15.00_08 |43,710 | Synology-File | ++---+---+---+ But then this is where the breakdown is... Bacula ignores those five copy jobids, and reports that it is building the restore tree from the original backups jobids: You have selected the following JobIds: 43699,43725,43751,43777,43803 Building directory tree for JobId(s) 43699,43725,43751,43777,43803 ... + 554,180 files inserted into the tree. So, I think I need to report this. (or amend the Mantis if one exists :) The command prints the alternative to the different jobs, I don't think it can choose for you, here in this example, we have 2^5 different possibilities to restore the files, I'm not sure Bacula can guess automatically which one do you want in the mix. The basic idea is that the original job is usually stored on fast and available disks, the copy is more likely stored on tape or cloud kind of slow access device. When the original job is purged (usually you have a smaller retention), then the copy job is "promoted", and the next restore command will select it automatically. Note, that it's also possible to have multiple copies of each job, on different kind of devices, and some jobs involved may not have copies. Now, if we can determine a selection algorithm (via MediaType?), we can probably implement it. Bconsole is not the best tool for advanced selection screens, however Baculum GUI has an excellent supports the copy job selection and can also display the different versions of a file, and let you restore the one you want for example. Best Regards, Eric ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restoring from a copy job
Hello Josip, Baculum supports restore from copy jobs. It is a function in the restore wizard that default is enabled. So you can just select copy jobs and restore from them all or selected files. This function is available from Baculum version 9.6.0. If you prefer doing it using Bconsole, you can use Bvfs restore commands for that. But please be careful with .bvfs_get_jobids command in version lower than 11.0.3, because there was a bug in this command with copy jobs: https://bugs.bacula.org/view.php?id=2500 Besides that it works well. Best regards, Marcin Haba (gani) On Tue, 30 Nov 2021 at 08:54, Radosław Korzeniewski wrote: > > Hello, > > wt., 30 lis 2021 o 06:01 Bill Arlofski via Bacula-users > napisał(a): >> >> >> It is not the way I would expect or hope the 'copies' option would function, >> but at least it is "functioning as documented"™ :) > > > The way my customers are using it is when "originals" are not available for > restore, then restore from copies works perfectly as expected. > So, when doing copy for offline, off-site storage, disaster recovery, long > term, low-cost storage, etc. etc. > > It is not perfect and does not cover all use cases you can imagine for sure. > But do the real work when required. > > best regards > -- > Radosław Korzeniewski > rados...@korzeniewski.net > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
