It works for me on FreeBSD with Bacula 15 from git. Can you attach gdb to the bacula-fd while it is running and issue the gdb command:
thread apply all bt Also, try running bacula-fd with -d 150 -dt -v -fP which will make it print the debug info to the terminal. Level 150 should show what it is doing for the fifo. __Martin >>>>> On Tue, 13 Feb 2024 09:07:43 -0500, Peter Sjoberg said: > > On 2024-02-13 02:49, Eric Bollengier wrote: > > Hello Peter, > > > > Without the ReadFifo directive, it's unlikely to cause a problem, > Unlikely maybe but that is the problem and I can even reproduce it! > My setup is based on ubuntu 22.04 LTS (was trying debian but align is > broken there) using the community repo > > debhttps://www.bacula.org/packages/<redacted>/debs/13.0.3 jammy main > > > and the file daemon output is pretty clear, we are not at this file. > > The file daemon output shows last file that worked, not the file it is > trying to backup. > > To reproduce I did > > *1 - create a fileset that backups just /tmp/debug* > > FileSet { > Name = "debugfs2" > Ignore FileSet Changes = yes > Include { > Options { > signature=MD5 > honor nodump flag=yes > noatime=yes > keepatime = no > sparse=yes > exclude = yes > wild = *~ > wild = *.tmp > } > File = "/tmp/debug" > } > } > > *2 - create a pipe ("mkfifo random_pipe") and a plane file ("date > >a_red_herring") in /tmp/debug* > > peters@quark:/tmp/debug$ find /tmp/debug/ -ls > 11017 0 drwxr-xr-x 2 peters peters 80 Feb 13 08:47 > /tmp/debug/ > 11031 4 -rw-r--r-- 1 peters peters 32 Feb 13 08:47 > /tmp/debug/a_red_herring > 11029 0 prw-r--r-- 1 peters peters 0 Feb 13 08:46 > /tmp/debug/random_pipe > peters@quark:/tmp/debug$ > > *3 - start a backup; > * > > root@quark:~#echo run BackupQ_quark FileSet="debugfs2" Level=Full yes|bconsole > > *4 - confirm it hangs* > > root@quark:~# echo stat client=quark-fd|bconsole #CLIENTSTAT > Connecting to Director quark:9101 > 1000 OK: 10002 techwiz-dir Version: 13.0.3 (02 May 2023) > Enter a period to cancel a command. > stat client=quark-fd > Connecting to Client quark-fd at quark:9102 > > quark-fd Version: 13.0.3 (02 May 2023) x86_64-pc-linux-gnu-bacula-enterprise > ubuntu 22.04 > Daemon started 12-Feb-24 23:55. Jobs: run=5 running=1. > Heap: heap=856,064 smbytes=603,907 max_bytes=1,219,047 bufs=178 max_bufs=429 > Sizes: boffset_t=8 size_t=8 debug=0 trace=0 mode=0,0 bwlimit=0kB/s > Crypto: fips=N/A crypto=OpenSSL 3.0.2 15 Mar 2022 > Plugin: bpipe-fd.so(2) > > Running Jobs: > JobId 315 Job BackupQ_quark.2024-02-13_08.48.07_46 is running. > Full Backup Job started: 13-Feb-24 08:48 > Files=1 Bytes=40 AveBytes/sec=0 LastBytes/sec=2 Errors=0 > Bwlimit=0 ReadBytes=32 > Files: Examined=1 Backed up=1 > Processing file: /tmp/debug/a_red_herring > SDReadSeqNo=8 fd=5 SDtls=1 > Director connected using TLS at: 13-Feb-24 08:55 > ==== > > *5 - release the job by sending something to the pipe; * > > root@quark:~# echo >/tmp/debug/random_pipe > > *6 - confirm the job finished* > > root@quark:~# echo 'llist jobid=315'|bconsole > Connecting to Director quark:9101 > 1000 OK: 10002 techwiz-dir Version: 13.0.3 (02 May 2023) > Enter a period to cancel a command. > llist jobid=315 > Automatically selected Catalog: MyCatalog > Using Catalog "MyCatalog" > JobId: 315 > Job: BackupQ_quark.2024-02-13_08.48.07_46 > Name: BackupQ_quark > PurgedFiles: 0 > Type: B > Level: F > ClientId: 32 > ClientName: quark-fd > JobStatus: T > SchedTime: 2024-02-13 08:48:07 > StartTime: 2024-02-13 08:48:09 > EndTime: 2024-02-13 08:56:40 > RealEndTime: 2024-02-13 08:56:40 > JobTDate: 1,707,832,600 > VolSessionId: 66 > VolSessionTime: 1,707,779,559 > JobFiles: 3 > JobBytes: 40 > ReadBytes: 32 > JobErrors: 0 > JobMissingFiles: 0 > PoolId: 2 > PoolName: File-Full > PriorJobId: 0 > PriorJob: > FileSetId: 7 > FileSet: debugfs2 > HasCache: 0 > Comment: > Reviewed: 0 > > You have messages. > root@quark:~# > > /ps > > > > > The problem can be somewhere else, and a good start is a "status dir" > > and "status storage". > > > > Best Regards, > > Eric > > > > On 2/13/24 06:23, Peter Sjoberg wrote: > >> > >> Actually, I think I found the root cause - a pipe! > >> > >> The file listed in the client status is not the problem but close to > >> it is a pipe (maybe next file) and that is what causing the issue in > >> all cases. > >> I striped down the directory to just one file and it still fails > >> > >> root@defiant1:/home/debug# find .zoom -ls > >> 23855106 4 drwx------ 4 sys adm 4096 Feb 13 > >> 00:11 .zoom > >> 23855107 4 drwxrwxr-x 2 ba ba 4096 Feb 13 > >> 00:11 .zoom/data > >> 23855116 0 prw-r--r-- 1 ba ba 0 Mar 12 > >> 2021 .zoom/data/com.zoom.ipc.confapp__res > >> 23855110 4 drwxrwxr-x 2 sys adm 4096 May 5 > >> 2020 .zoom/reports > >> root@defiant1:/home/debug# > >> > >> > >> and if I send something to the pipe the job finish ok and restoring > >> the job did include the pipe. > >> > >> /ps > >> > >> On 2024-02-12 23:36, Peter Sjoberg wrote: > >>> > >>> In short - no and no, no special files > >>> Also, while I haven't waited forever I have left it for several > >>> hours so it's not like it's just some house cleaning left. > >>> It does happen on different servers, sample below is my laptop. > >>> > >>> =============== My fileset: > >>> FileSet { > >>> Name = "debugfs2" > >>> Ignore FileSet Changes = yes > >>> Include { > >>> Options { > >>> signature=MD5 > >>> honor nodump flag=yes > >>> noatime=yes > >>> keepatime = no > >>> sparse=yes > >>> exclude = yes > >>> wild = *~ > >>> wild = *.tmp > >>> } > >>> File = "/home/ba/.zoom" > >>> } > >>> } > >>> > >>> ============= run command > >>> echo run BackupQ_defiant1 FileSet="debugfs2" Level=Full yes|bconsole > >>> > >>> > >>> ============= client status when hung > >>> root@quark:~# echo stat client=defiant1-fd|bconsole #CLIENTSTAT > >>> Connecting to Director quark:9101 > >>> 1000 OK: 10002 techwiz-dir Version: 13.0.3 (02 May 2023) > >>> Enter a period to cancel a command. > >>> stat client=defiant1-fd > >>> Connecting to Client defiant1-fd at defiant1:9102 > >>> > >>> defiant1-fd Version: 13.0.3 (02 May 2023) > >>> x86_64-pc-linux-gnu-bacula-enterprise ubuntu 22.04 > >>> Daemon started 12-Feb-24 23:22. Jobs: run=2 running=1. > >>> Heap: heap=856,064 smbytes=606,583 max_bytes=794,675 bufs=188 > >>> max_bufs=203 > >>> Sizes: boffset_t=8 size_t=8 debug=0 trace=0 mode=0,0 bwlimit=0kB/s > >>> Crypto: fips=N/A crypto=OpenSSL 3.0.2 15 Mar 2022 > >>> Plugin: bpipe-fd.so(2) > >>> > >>> Running Jobs: > >>> JobId 255 Job BackupQ_defiant1.2024-02-12_23.29.48_18 is running. > >>> Full Backup Job started: 12-Feb-24 23:29 > >>> Files=2 Bytes=0 AveBytes/sec=0 LastBytes/sec=0 Errors=0 > >>> Bwlimit=0 ReadBytes=0 > >>> Files: Examined=2 Backed up=2 > >>> Processing file: /home/ba/.zoom/logs > >>> SDReadSeqNo=8 fd=5 SDtls=1 > >>> Director connected using TLS at: 12-Feb-24 23:30 > >>> ==== > >>> > >>> > >>> ================ Content of that directory > >>> > >>> root@defiant1:~# find /home/ba/.zoom -ls > >>> 15728688 4 drwx------ 7 ba ba 4096 Apr 28 > >>> 2021 /home/ba/.zoom > >>> 16130303 4 drwx------ 2 ba ba 4096 Apr 28 > >>> 2021 /home/ba/.zoom/screenCapture > >>> 16130301 4 drwxrwxr-x 2 ba ba 4096 Feb 12 > >>> 18:47 /home/ba/.zoom/logs > >>> 16023538 4 drwxrwxr-x 4 ba ba 4096 Apr 28 > >>> 2021 /home/ba/.zoom/data > >>> 16023540 0 prw-r--r-- 1 ba ba 0 May 5 > >>> 2020 /home/ba/.zoom/data/com.zoom.ipc.assistantapp__res > >>> 16023541 0 prw-r--r-- 1 ba ba 0 Mar 12 > >>> 2021 /home/ba/.zoom/data/com.zoom.ipc.confapp__req > >>> 16023539 0 prw-r--r-- 1 ba ba 0 Apr 28 > >>> 2021 /home/ba/.zoom/data/com.zoom.ipc.assistantapp__req > >>> 16130305 4 drwx------ 2 ba ba 4096 Mar 23 > >>> 2021 /home/ba/.zoom/data/VirtualBkgnd_Custom > >>> 16131475 1564 -rw------- 1 ba ba 1597940 Mar 23 > >>> 2021 > >>> /home/ba/.zoom/data/VirtualBkgnd_Custom/{ff6d8a57-d810-4dd2-bf1b-8366c063728f} > >>> 16023542 0 prw-r--r-- 1 ba ba 0 Mar 12 > >>> 2021 /home/ba/.zoom/data/com.zoom.ipc.confapp__res > >>> 16023545 52 -rw------- 1 ba ba 53248 Apr 28 > >>> 2021 /home/ba/.zoom/data/zoomus.enc.db > >>> 16130304 4 drwx------ 2 ba ba 4096 Mar 23 > >>> 2021 /home/ba/.zoom/data/ConfAvatar > >>> 16131472 36 -rw------- 1 ba ba 36397 Mar 23 > >>> 2021 > >>> /home/ba/.zoom/data/ConfAvatar/conf_avatar_6c72761c1ad5cc6f485dce3966cbb705_100 > >>> 16131473 4 -rw------- 1 ba ba 1020 Mar 23 > >>> 2021 > >>> /home/ba/.zoom/data/ConfAvatar/conf_avatar_9e6b3f01c5d33a2052c2681a42b4e659_100 > >>> 16131474 4 -rw------- 1 ba ba 1020 Mar 23 > >>> 2021 > >>> /home/ba/.zoom/data/ConfAvatar/conf_avatar_e977cbed2632f5b11882e92e31f32516_100 > >>> 16023544 8 -rw------- 1 ba ba 5120 Mar 23 > >>> 2021 /home/ba/.zoom/data/zoommeeting.enc.db > >>> 16130302 4 drwxrwxr-x 2 ba ba 4096 May 5 > >>> 2020 /home/ba/.zoom/reports > >>> 16130300 4 drwx------ 2 ba ba 4096 Apr 28 > >>> 2021 /home/ba/.zoom/im > >>> root@defiant1:~# > >>> > >>> /ps > >>> > >>> > >>> On 2024-02-12 20:03, Gary R. Schmidt wrote: > >>>> On 13/02/2024 11:08, Phil Stracchino wrote: >>>>> On 2/12/24 18:35, Peter Sjoberg wrote: > >>>>>> Hi all > >>>>>> > >>>>>> I have a strange problem and (on my system) reproducible problem. > >>>>>> When I do backup of some directories then bacula-fd just hangs > >>>>>> and never complete. > >>>>>> The directories in question are not very strange and backup of > >>>>>> them works find with older versions of -fd > >>>>> > >>>>> >>>>> Silly question: Do the problem directories contain named pipes or >>>>> sockets? > >>>>> > >>>> Another possibly silly question: Are there any soft links that may > >>>> cause a loop? > >>>> > >>>> Cheers, > >>>> Gary B-) > >>>> > >>>> > >>>> _______________________________________________ > >>>> Bacula-users mailing list > >>>> Bacula-users@lists.sourceforge.net > >>>> https://lists.sourceforge.net/lists/listinfo/bacula-users > >>> > >>> > >>> > >>> > >>> _______________________________________________ > >>> Bacula-users mailing list > >>> Bacula-users@lists.sourceforge.net > >>> https://lists.sourceforge.net/lists/listinfo/bacula-users > >> > >> > >> > >> > >> _______________________________________________ > >> Bacula-users mailing list > >> Bacula-users@lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/bacula-users > > > _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
