Oops, I made an error. In point 1, your FileCP pool should point to its own storage resource, which from your source job, appears to be "file2"
I always define the storage at the pool level. Retention is defined there too. Robert Gerber 402-237-8692 [email protected] On Sun, Dec 7, 2025, 10:52 PM Rob Gerber <[email protected]> wrote: > Copy jobs are tricky to wrap ones head around, as you noted. I know I had > trouble at first. > > The first things I'd try are: > > 1. The FileCP pool doesn't have a storage resource defined. In this case I > think you'd want one pointing to FileCP. The FileCP pool cannot express > which storage resource is appropriate for it, so when that pool is > referenced the copy control job MigrateDCIM-to-Tape cannot figure out the > correct device. It would be nice if the copy control job figured this out > by looking through records for JobID 4, but that doesn't appear to be > happening here. > > 2. The tape drive appears not to have a suitable tape labeled and > appendable, though it's possible that the misconfiguration that caused all > this tied up a suitable tape while attempting to use the same resource as > both source and destination (valid if you wanted to copy a tape, but not in > your case). > > 3. You might want selection criterion "PoolUncopiedJobs". I suspect the > selection tool criterion you've provided will copy jobs matching the mask > you provided, no matter whether they've previously been copied or not. Yes, > it did match your target job correctly, but might lead to extra job copies > later. > > 4. Make sure that retention matches for original and copy job definitions. > I had an issue where, during testing, I set up longer retention A for my > intial source jobs (local storage). I made full backups on the local > storage file writers. At some point, I decided to lower the retention > period, but I DIDNT forcibly purge the initial full backups with longer > retention periods. Then I set up copy jobs to upload local jobs to the > cloud, with shorter retention period B (at that time matching all other > pools in the system, but this new shorter retention was NOT applied to the > existing jobs). The copy jobs worked and I walked away. Then, a few months > later, the cloud volume retention expired, the copies of the first full > backups were pruned from cloud storage... And were then promptly pruned > from the cloud on completion, because they inherited their retention period > from the start or end date associated with the original job. Meanwhile, the > local full backups weren't set to expire anytime soon. Copy after copy was > uploaded to the cloud before I figured out the mistake. Oof. > > 5. As far as I can tell, when a copy control job like MigrateDCIM-to-Tape > launches, any selection criterion like PoolUncopiedJobs operates at time of > job launch, even if the job waits for other higher priority jobs to finish > first. In other words, if you launch your file backup jobs, then 1 minute > later launch your copy jobs with a lower priority while the file jobs > complete, the only valid targets for the copy job will be last night's file > storage jobs, not the yet-to-be-completed jobs from tonight. The solution I > found to this problem was to make an admin job that launches the copy jobs > using an echo command to bconsole. So the admin job CopyLauncher sits in > the queue waiting for file jobs to finish, then once activated it > immediately spawns copy control jobs and exits. Those copy control jobs see > all tonights completed jobs that are eligible for a copy, and act > accordingly. > > > Robert Gerber > 402-237-8692 > [email protected] > > On Sun, Dec 7, 2025, 10:02 PM Kevin Buckley < > [email protected]> wrote: > >> Hi there, >> >> trying to get my head around the "Migrate/Copy" examples, specifically >> a Copy, and running the jobs manually whilst testing and familiarising, >> against 15.0.3. >> >> FWIW, the names of the Pools and Jobs have been chosen so as not to >> clash with other testing components still in the config, and so are >> dedicated to this particular testing effort, hence the explicit >> addition of the JobDefs attributes, into the Job stanzas, so as to >> avoid getting any "unwanted defaults" >> >> Pool { >> Name = FileCP >> Pool Type = Backup >> Recycle = yes # Bacula can automatically recycle >> Volumes >> AutoPrune = yes # Prune expired volumes >> Volume Retention = 365 days # one year >> Maximum Volume Bytes = 50G # Limit Volume size to something >> reasonable >> Maximum Volumes = 100 # Limit number of Volumes in Pool >> Label Format = "VolCP-" # Auto label >> Next Pool = TapeCP >> } >> >> >> # TapeCP Pool definition >> Pool { >> Name = TapeCP >> Pool Type = Backup >> Recycle = yes >> AutoPrune = yes >> Storage = TS4300 >> } >> >> Job { >> Name = "BackupDCIM-to-FileCP" >> #-JobDefs = "DefaultJob" >> Type = Backup >> Level = Incremental >> Client = client-fd >> FileSet = "DCIM Fileset" >> Schedule = "WeeklyCycle" >> Storage = File2 >> # Messages = Standard >> Messages = Daemon >> Pool = FileCP >> SpoolAttributes = yes >> Priority = 10 >> Write Bootstrap = "/opt/bacula/working/%c.bsr" >> #-Attributes not defined in JobDefs come after >> Enabled = yes >> } >> >> >> >> Job { >> Name = "MigrateDCIM-to-Tape" >> #-JobDefs = "DefaultJob" >> Type = Copy >> Level = Full >> Client = client-fd >> FileSet = "DCIM Fileset" >> Schedule = "WeeklyCycle" >> Messages = Daemon >> Pool = FileCP >> #-Attributes not defined in JobDefs come after >> Storage = TS4300 >> Selection Type = Job >> Selection Pattern = ".*FileCP" >> Enabled = yes >> } >> >> >> I run the first job >> >> Select Job resource (1-7): 4 >> Run Backup job >> JobName: BackupDCIM-to-FileCP >> Level: Incremental >> Client: client-fd >> FileSet: DCIM Fileset >> Pool: FileCP (From Job resource) >> Storage: File2 (From Job resource) >> When: 2025-12-04 16:24:59 >> Priority: 10 >> OK to run? (Yes/mod/no): Yes >> Job queued. JobId=4 >> You have messages. >> >> >> and that produces a file, VolCP-0009, on the filesystem >> >> When I come to run the "Copy" however, >> >> Select Job resource (1-7): 5 >> Run Copy job >> JobName: MigrateDCIM-to-Tape >> Bootstrap: *None* >> Client: client-fd >> FileSet: DCIM Fileset >> Pool: FileCP (From Job resource) >> NextPool: TapeCP (From Job Pool's NextPool resource) >> Read Storage: TS4300 (From Job resource) >> Write Storage: TS4300 (From Job Pool's NextPool resource) >> JobId: *None* >> When: 2025-12-04 16:30:09 >> Catalog: MyCatalog >> Priority: 10 >> OK to run? (Yes/mod/no):Yes >> Job queued. JobId=5 >> You have messages. >> >> >> it appears to be trying to "Read" from a Tape Caddy drive, rather >> then from the File2 storage, and fails with >> >> testinst-dir JobId 5: The following 1 JobId was chosen to be copied: 4 >> testinst-dir JobId 5: Copying using JobId=4 >> Job=BackupDCIM-to-FileCP.2025-12-04_16.25.22_03 >> testinst-dir JobId 5: Start Copying JobId 5, >> Job=MigrateDCIM-to-Tape.2025-12-04_16.31.01_05 >> testinst-dir JobId 5: Connected to Storage "TS4300" at >> testinst.pawsey.org.au:9103 with TLS >> testinst-dir JobId 5: Connected to Storage "TS4300" at >> testinst.pawsey.org.au:9103 with TLS >> testinst-dir JobId 5: Using Device "TS4300-Drive1" to read. >> testinst-dir JobId 6: Using Device "TS4300-Drive2" to write. >> testinst-sd JobId 6: Connected to Storage at testinst.pawsey.org.au:9103 >> with TLS >> testinst-sd JobId 6: Job BackupDCIM-to-FileCP.2025-12-04_16.31.01_06 is >> waiting. Cannot find any appendable volumes. >> Please use the "label" command to create a new Volume for: >> Storage: "TS4300-Drive2" (/dev/IBMtape0n) >> Pool: TapeCP >> Media type: ULT3580-HH9 >> >> but I can't see why the Copy Job wants to read (Read Storage) from a >> Tape Caddy DRive, given that it has identified the FileCP Pool correctly. >> >> If, however, I take the TS4300 Storage attribute out of the TapeCP >> Pool stanza (in the belief that it might then just inherit the right >> target, as per the Pool), then the Copy Job doesn't run, giving >> the message: >> >> Select Job resource (1-7): 5 >> Job not run. >> You have messages. >> *messages >> 08-Dec 11:27 testinst-dir JobId 0: Fatal error: No Storage specification >> found in Next Pool "TapeCP". >> >> >> >> I am assuming I have managed to mess up the Pool, and/or Storage, >> definitions, but what may I have missed? >> >> >> For completeness, here are the Autochanger-related stanzas: >> >> bacula-dir.conf >> >> Autochanger { >> Name = TS4300 >> Address = testinst.pawsey.org.au >> SDPort = 9103 >> # password for Storage daemon >> Password = "somepasswdstring" >> # Device must be same as Device in Storage daemon >> Device = TS4300-Drive1 >> Device = TS4300-Drive2 >> # Media Type must be same as MediaType in Storage daemon >> Media Type = ULT3580-HH9 >> # enable for autochanger device >> Autochanger = TS4300 >> Maximum Concurrent Jobs = 10 >> } >> >> bacula-sd.conf >> >> Autochanger { >> Name = TS4300 >> Device = TS4300-Drive1 >> Device = TS4300-Drive2 >> Changer Command = "/opt/bacula/scripts/mtx-changer %c %o %S %a %d" >> Changer Device = /dev/sg3 >> } >> >> Device { >> Name = TS4300-Drive1 >> Drive Index = 0 >> Media Type = ULT3580-HH9 >> Archive Device = /dev/IBMtape1n >> AutomaticMount = yes; # when device opened, read it >> AlwaysOpen = yes; >> RemovableMedia = yes; >> RandomAccess = no; >> AutoChanger = yes >> Control Device = /dev/sg3 >> Alert Command = "/opt/bacula/scripts/tapealert %l" >> } >> >> Device { >> Name = TS4300-Drive2 >> Drive Index = 1 >> Media Type = ULT3580-HH9 >> Archive Device = /dev/IBMtape0n >> AutomaticMount = yes; # when device opened, read it >> AlwaysOpen = yes; >> RemovableMedia = yes; >> RandomAccess = no; >> AutoChanger = yes >> Control Device = /dev/sg3 >> Alert Command = "/opt/bacula/scripts/tapealert %l" >> } >> >> >> Started thinking, from reading other parts of the Bacula docs, >> that it might be possible to target just ONE of the Autochanger >> Drives, but not clear how to do that, should it even be the >> "way to go" >> >> As I say though, I'm sure I've missed something in reading the >> docs a few times now, >> Kevin Buckley >> >> >> >> _______________________________________________ >> Bacula-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/bacula-users >> >
_______________________________________________ Bacula-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-users
