On 2024-02-13 02:49, Eric Bollengier wrote:
Hello Peter,
Without the ReadFifo directive, it's unlikely to cause a problem,
Unlikely maybe but that is the problem and I can even reproduce it!
My setup is based on ubuntu 22.04 LTS (was trying debian but align is
broken there) using the community repo
debhttps://www.bacula.org/packages/<redacted>/debs/13.0.3 jammy main
and the file daemon output is pretty clear, we are not at this file.
The file daemon output shows last file that worked, not the file it is
trying to backup.
To reproduce I did
*1 - create a fileset that backups just /tmp/debug*
FileSet {
Name = "debugfs2"
Ignore FileSet Changes = yes
Include {
Options {
signature=MD5
honor nodump flag=yes
noatime=yes
keepatime = no
sparse=yes
exclude = yes
wild = *~
wild = *.tmp
}
File = "/tmp/debug"
}
}
*2 - create a pipe ("mkfifo random_pipe") and a plane file ("date
>a_red_herring") in /tmp/debug*
peters@quark:/tmp/debug$ find /tmp/debug/ -ls
11017 0 drwxr-xr-x 2 peters peters 80 Feb 13 08:47
/tmp/debug/
11031 4 -rw-r--r-- 1 peters peters 32 Feb 13 08:47
/tmp/debug/a_red_herring
11029 0 prw-r--r-- 1 peters peters 0 Feb 13 08:46
/tmp/debug/random_pipe
peters@quark:/tmp/debug$
*3 - start a backup;
*
root@quark:~#echo run BackupQ_quark FileSet="debugfs2" Level=Full yes|bconsole
*4 - confirm it hangs*
root@quark:~# echo stat client=quark-fd|bconsole #CLIENTSTAT
Connecting to Director quark:9101
1000 OK: 10002 techwiz-dir Version: 13.0.3 (02 May 2023)
Enter a period to cancel a command.
stat client=quark-fd
Connecting to Client quark-fd at quark:9102
quark-fd Version: 13.0.3 (02 May 2023) x86_64-pc-linux-gnu-bacula-enterprise
ubuntu 22.04
Daemon started 12-Feb-24 23:55. Jobs: run=5 running=1.
Heap: heap=856,064 smbytes=603,907 max_bytes=1,219,047 bufs=178 max_bufs=429
Sizes: boffset_t=8 size_t=8 debug=0 trace=0 mode=0,0 bwlimit=0kB/s
Crypto: fips=N/A crypto=OpenSSL 3.0.2 15 Mar 2022
Plugin: bpipe-fd.so(2)
Running Jobs:
JobId 315 Job BackupQ_quark.2024-02-13_08.48.07_46 is running.
Full Backup Job started: 13-Feb-24 08:48
Files=1 Bytes=40 AveBytes/sec=0 LastBytes/sec=2 Errors=0
Bwlimit=0 ReadBytes=32
Files: Examined=1 Backed up=1
Processing file: /tmp/debug/a_red_herring
SDReadSeqNo=8 fd=5 SDtls=1
Director connected using TLS at: 13-Feb-24 08:55
====
*5 - release the job by sending something to the pipe; *
root@quark:~# echo >/tmp/debug/random_pipe
*6 - confirm the job finished*
root@quark:~# echo 'llist jobid=315'|bconsole
Connecting to Director quark:9101
1000 OK: 10002 techwiz-dir Version: 13.0.3 (02 May 2023)
Enter a period to cancel a command.
llist jobid=315
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
JobId: 315
Job: BackupQ_quark.2024-02-13_08.48.07_46
Name: BackupQ_quark
PurgedFiles: 0
Type: B
Level: F
ClientId: 32
ClientName: quark-fd
JobStatus: T
SchedTime: 2024-02-13 08:48:07
StartTime: 2024-02-13 08:48:09
EndTime: 2024-02-13 08:56:40
RealEndTime: 2024-02-13 08:56:40
JobTDate: 1,707,832,600
VolSessionId: 66
VolSessionTime: 1,707,779,559
JobFiles: 3
JobBytes: 40
ReadBytes: 32
JobErrors: 0
JobMissingFiles: 0
PoolId: 2
PoolName: File-Full
PriorJobId: 0
PriorJob:
FileSetId: 7
FileSet: debugfs2
HasCache: 0
Comment:
Reviewed: 0
You have messages.
root@quark:~#
/ps
The problem can be somewhere else, and a good start is a "status dir"
and "status storage".
Best Regards,
Eric
On 2/13/24 06:23, Peter Sjoberg wrote:
Actually, I think I found the root cause - a pipe!
The file listed in the client status is not the problem but close to
it is a pipe (maybe next file) and that is what causing the issue in
all cases.
I striped down the directory to just one file and it still fails
root@defiant1:/home/debug# find .zoom -ls
23855106 4 drwx------ 4 sys adm 4096 Feb 13
00:11 .zoom
23855107 4 drwxrwxr-x 2 ba ba 4096 Feb 13
00:11 .zoom/data
23855116 0 prw-r--r-- 1 ba ba 0 Mar 12
2021 .zoom/data/com.zoom.ipc.confapp__res
23855110 4 drwxrwxr-x 2 sys adm 4096 May 5
2020 .zoom/reports
root@defiant1:/home/debug#
and if I send something to the pipe the job finish ok and restoring
the job did include the pipe.
/ps
On 2024-02-12 23:36, Peter Sjoberg wrote:
In short - no and no, no special files
Also, while I haven't waited forever I have left it for several
hours so it's not like it's just some house cleaning left.
It does happen on different servers, sample below is my laptop.
=============== My fileset:
FileSet {
Name = "debugfs2"
Ignore FileSet Changes = yes
Include {
Options {
signature=MD5
honor nodump flag=yes
noatime=yes
keepatime = no
sparse=yes
exclude = yes
wild = *~
wild = *.tmp
}
File = "/home/ba/.zoom"
}
}
============= run command
echo run BackupQ_defiant1 FileSet="debugfs2" Level=Full yes|bconsole
============= client status when hung
root@quark:~# echo stat client=defiant1-fd|bconsole #CLIENTSTAT
Connecting to Director quark:9101
1000 OK: 10002 techwiz-dir Version: 13.0.3 (02 May 2023)
Enter a period to cancel a command.
stat client=defiant1-fd
Connecting to Client defiant1-fd at defiant1:9102
defiant1-fd Version: 13.0.3 (02 May 2023)
x86_64-pc-linux-gnu-bacula-enterprise ubuntu 22.04
Daemon started 12-Feb-24 23:22. Jobs: run=2 running=1.
Heap: heap=856,064 smbytes=606,583 max_bytes=794,675 bufs=188
max_bufs=203
Sizes: boffset_t=8 size_t=8 debug=0 trace=0 mode=0,0 bwlimit=0kB/s
Crypto: fips=N/A crypto=OpenSSL 3.0.2 15 Mar 2022
Plugin: bpipe-fd.so(2)
Running Jobs:
JobId 255 Job BackupQ_defiant1.2024-02-12_23.29.48_18 is running.
Full Backup Job started: 12-Feb-24 23:29
Files=2 Bytes=0 AveBytes/sec=0 LastBytes/sec=0 Errors=0
Bwlimit=0 ReadBytes=0
Files: Examined=2 Backed up=2
Processing file: /home/ba/.zoom/logs
SDReadSeqNo=8 fd=5 SDtls=1
Director connected using TLS at: 12-Feb-24 23:30
====
================ Content of that directory
root@defiant1:~# find /home/ba/.zoom -ls
15728688 4 drwx------ 7 ba ba 4096 Apr 28
2021 /home/ba/.zoom
16130303 4 drwx------ 2 ba ba 4096 Apr 28
2021 /home/ba/.zoom/screenCapture
16130301 4 drwxrwxr-x 2 ba ba 4096 Feb 12
18:47 /home/ba/.zoom/logs
16023538 4 drwxrwxr-x 4 ba ba 4096 Apr 28
2021 /home/ba/.zoom/data
16023540 0 prw-r--r-- 1 ba ba 0 May 5
2020 /home/ba/.zoom/data/com.zoom.ipc.assistantapp__res
16023541 0 prw-r--r-- 1 ba ba 0 Mar 12
2021 /home/ba/.zoom/data/com.zoom.ipc.confapp__req
16023539 0 prw-r--r-- 1 ba ba 0 Apr 28
2021 /home/ba/.zoom/data/com.zoom.ipc.assistantapp__req
16130305 4 drwx------ 2 ba ba 4096 Mar 23
2021 /home/ba/.zoom/data/VirtualBkgnd_Custom
16131475 1564 -rw------- 1 ba ba 1597940 Mar 23
2021
/home/ba/.zoom/data/VirtualBkgnd_Custom/{ff6d8a57-d810-4dd2-bf1b-8366c063728f}
16023542 0 prw-r--r-- 1 ba ba 0 Mar 12
2021 /home/ba/.zoom/data/com.zoom.ipc.confapp__res
16023545 52 -rw------- 1 ba ba 53248 Apr 28
2021 /home/ba/.zoom/data/zoomus.enc.db
16130304 4 drwx------ 2 ba ba 4096 Mar 23
2021 /home/ba/.zoom/data/ConfAvatar
16131472 36 -rw------- 1 ba ba 36397 Mar 23
2021
/home/ba/.zoom/data/ConfAvatar/conf_avatar_6c72761c1ad5cc6f485dce3966cbb705_100
16131473 4 -rw------- 1 ba ba 1020 Mar 23
2021
/home/ba/.zoom/data/ConfAvatar/conf_avatar_9e6b3f01c5d33a2052c2681a42b4e659_100
16131474 4 -rw------- 1 ba ba 1020 Mar 23
2021
/home/ba/.zoom/data/ConfAvatar/conf_avatar_e977cbed2632f5b11882e92e31f32516_100
16023544 8 -rw------- 1 ba ba 5120 Mar 23
2021 /home/ba/.zoom/data/zoommeeting.enc.db
16130302 4 drwxrwxr-x 2 ba ba 4096 May 5
2020 /home/ba/.zoom/reports
16130300 4 drwx------ 2 ba ba 4096 Apr 28
2021 /home/ba/.zoom/im
root@defiant1:~#
/ps
On 2024-02-12 20:03, Gary R. Schmidt wrote:
On 13/02/2024 11:08, Phil Stracchino wrote:
On 2/12/24 18:35, Peter Sjoberg wrote:
Hi all
I have a strange problem and (on my system) reproducible problem.
When I do backup of some directories then bacula-fd just hangs
and never complete.
The directories in question are not very strange and backup of
them works find with older versions of -fd
Silly question: Do the problem directories contain named pipes or
sockets?
Another possibly silly question: Are there any soft links that may
cause a loop?
Cheers,
Gary B-)
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
