Thanks, I did just remove those from the job. I wonder what else could be causing it to error, the backup server is also the job's client, it connects to ceph and creates a snapshot of the vm's disk and then pulls the snapshot to it's local file system (in /var/lib/bareos/vmname_snap), and then extracts that snapshot into /var/lib/bareos/vmname_snap_fs_0/ (which it creates using libguestfs).
The same process works for the other 150+ vms it's just always this one vm that causes the issue. the size of the snapshot is 58GB and the extracted snapshot filesystem is currently at about 20GB, it backs up much larger snapshots ok, just this one. It's so frustrating. On Friday, 11 November 2022 at 14:36:09 UTC [email protected] wrote: > I would remove those max runtime / wait etc until you have it working. > There is often confusion with those settings. > > I have had jobs run for over a day no issue. > > Sent from my iPhone > Brock Palen > > On Nov 11, 2022, at 5:42 AM, 'Christian Sacks' via bareos-users < > [email protected]> wrote: > > I was premature, it was a one off fluke. Back to the drawing board. > > > > On Thursday, 10 November 2022 at 15:31:41 UTC Christian Sacks wrote: > >> I have got it working, finally!!! >> >> # grep -ir heartbeat|grep -v \# >> bareos-sd.d/storage/bareos-sd.conf: Heartbeat Interval = 60 seconds >> bareos-dir.d/director/bareos-dir.conf: Heartbeat Interval = 60 seconds >> bareos-fd.d/client/myself.conf: Heartbeat Interval = 60 seconds >> >> Woohoo!! >> >> >> On Thursday, 10 November 2022 at 14:00:42 UTC Christian Sacks wrote: >> >>> Hi, thought I'd check back and realised I hadn't posted my reply before; >>> >>> I have these in the job's .conf file; >>> >>> MaxRunSchedTime = 2 hours >>> MaxRunTime = 1 hours >>> MaxWaitTime = 1 hours >>> >>> Yet still as soon as it gets to >>> Elapsed time: 32 mins 20 secs >>> it just automatically cancels the job and fails with >>> FD termination status: Fatal Error >>> >>> I'm so stumped. >>> Thanks for your helps. >>> >>> On Friday, 8 July 2022 at 13:53:05 UTC+1 [email protected] wrote: >>> >>>> Check if parameter set Max Run Sched Time in >>>> bareos-dir.d/job/your_job.conf >>>> >>>> понедельник, 27 июня 2022 г. в 17:40:39 UTC+3, Bruno Friedmann: >>>> >>>>> Didn't you have some firewall just cutting the connection because they >>>>> believe it is hang out ? >>>>> >>>>> Did you already tried to setup HeartbeatInterval >>>>> https://docs.bareos.org/Configuration/Director.html#config-Dir_Client_HeartbeatInterval >>>>> >>>>> instead of ConnectTimeout ? >>>>> >>>>> On Friday, June 24, 2022 at 12:01:07 PM UTC+2 [email protected] >>>>> wrote: >>>>> >>>>>> Hello there, >>>>>> >>>>>> I am about done pulling my hair out trying to find where I need to >>>>>> adjust a timeout setting for a job that takes longer than 30 minutes to >>>>>> run. >>>>>> >>>>>> I have a job that before running will create a snapshot in ceph for a >>>>>> vm's disks, pull the snapshot to the backup server, extract it into a >>>>>> folder and then backup the contents of that folder however the disk is >>>>>> quite large that it backs up and so takes longer than 30 minutes to >>>>>> create/pull/extract the snapshot. >>>>>> >>>>>> I have added the "FD Connect Timeout" and "SD Connect Timeout" with a >>>>>> value of 14400 seconds to the following files; >>>>>> >>>>>> # grep -ir timeout >>>>>> bareos-sd.d/storage/bareos-sd.conf: FD Connect Timeout = 14400 >>>>>> seconds >>>>>> bareos-sd.d/storage/bareos-sd.conf: SD Connect Timeout = 14400 >>>>>> seconds >>>>>> bareos-dir.d/director/bareos-dir.conf: FD Connect Timeout = 14400 >>>>>> seconds >>>>>> bareos-dir.d/director/bareos-dir.conf: SD Connect Timeout = 14400 >>>>>> seconds >>>>>> bareos-fd.d/client/myself.conf: SD Connect Timeout = 14400 seconds >>>>>> bareos-fd.d/client/myself.conf: FD Connect Timeout = 14400 seconds >>>>>> >>>>>> The job still times out after 30 minutes. >>>>>> >>>>>> Scheduled time: 23-Jun-2022 23:00:00 >>>>>> Start time: 23-Jun-2022 23:00:00 >>>>>> End time: 23-Jun-2022 23:30:31 >>>>>> Elapsed time: 30 mins 31 secs >>>>>> FD termination status: Fatal Error >>>>>> SD termination status: Waiting on FD >>>>>> >>>>>> I don't know where else to add an increased timeout, please can >>>>>> someone tell me where I need to look to add this timeout increased >>>>>> figure. >>>>>> >>>>>> What is interesting is if I run the backup manually, it works fine >>>>>> and does it in under 30 minutes, but because the automatic schedule >>>>>> starts >>>>>> at 11pm every night, other jobs run at the same time, and so it takes >>>>>> longer to run it and hits this bizarre 30 minute timeout. >>>>>> >>>>>> Thanks in advance. >>>>>> >>>>> -- > You received this message because you are subscribed to the Google Groups > "bareos-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/bareos-users/46a99a3a-8625-41bf-ba09-9fb96acf9a26n%40googlegroups.com > > <https://groups.google.com/d/msgid/bareos-users/46a99a3a-8625-41bf-ba09-9fb96acf9a26n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/226921d4-9307-406e-bebd-89e41a6d4e68n%40googlegroups.com.
