Kern Sibbald wrote:
> OK, I think the solution is for Hugh to:
> 1. Figure out why his alert command is broken
> 2. Create a script with a timer
> 3. Disable the alert
Here's what I've done:
-- Ran backups, no change; got a hang. Restarted sd and director.
-- Commented out the "Alert" sections in bacula-sd for the two tape
drives. Ran backups, no hang.
-- Changed the Alert section to:
Alert Command = "sh -c '/etc/bacula/alert_debugging.pl %c'"
which was a very simple perl script (attached). I reran backups and
got a hang. Here's what was logged:
Mar 19 15:30:24 agnatha hugh[29410]: Parent here...waiting patiently
Mar 19 15:30:24 agnatha hugh[29409]: About to run /usr/sbin/smartctl -H
-l error -q errorsonly -d scsi /dev/changer
Mar 19 15:30:24 agnatha hugh[29414]: Done: exit status 0
This was the same entries as seen before, since a bunch of jobs ran
and finished before the hang, but now I've got the two processes
again:
bacula 28827 1.1 0.0 248588 6220 ? Ssl 15:24 0:18
/usr/sbin/bacula-sd -u bacula -g disk -c /etc/bacula/bacula-sd.conf
bacula 29422 0.0 0.0 258816 4492 ? S 15:30 0:00
/usr/sbin/bacula-sd -u bacula -g disk -c /etc/bacula/bacula-sd.conf
I ran btraceback on both (attached). I can't find any mention of
PID 29422 (the child) in the traceback for 28827 (the parent). The
child appears to be hung at closelog(), which matches what I had in
the original traceback I sent with my first message to the list. I
ran kill -6 on both, but only the parent produced a lock dump
(attached). If I should add these files to the bug, let me know.
The bacula I'm running now is compiled from the sourceforge SRPM, but
with the changes detailed in bug #1527:
-- the patch removing the "debug_list_volumes" line
-- enabling the lock manager in the args to configure
-- and enabling developer mode in version.h
I thought maybe that last one was causing problems, since it results
in stdout not being closed, but I was having this problem with the
stock (though still locally-compiled) SRPM.
I'm thoroughly confused. For the weekend I'll just be removing the
alert section and letting things run.
--
Hugh Brown, Systems Manager
The Centre for High-Throughput Biology
[email protected]
[Thread debugging using libthread_db enabled]
[New Thread 0x2ad819585560 (LWP 28827)]
[New Thread 0x448ac940 (LWP 29426)]
[New Thread 0x41297940 (LWP 29425)]
[New Thread 0x43eab940 (LWP 29424)]
[New Thread 0x47ab1940 (LWP 29416)]
[New Thread 0x470b0940 (LWP 29415)]
[New Thread 0x42aa9940 (LWP 29399)]
[New Thread 0x4e8bc940 (LWP 29393)]
[New Thread 0x466af940 (LWP 29390)]
[New Thread 0x452ad940 (LWP 29371)]
[New Thread 0x4a2b5940 (LWP 29333)]
[New Thread 0x4acb6940 (LWP 29329)]
[New Thread 0x434aa940 (LWP 28832)]
0x00000035aa4cced2 in select () from /lib64/libc.so.6
$1 = "cbs-01-sd", '\0' <repeats 20 times>
$2 = 0x100f0528 "bacula-sd"
$3 = 0x100f0568 "/usr/sbin/"
$4 = 0x0
$5 = 0x31b0a450e6 "5.0.1 (24 February 2010)"
$6 = 0x31b0a450ff "x86_64-redhat-linux-gnu"
$7 = 0x31b0a45117 "redhat"
$8 = 0x31b0a44ddd ""
$9 = "agnatha.chibi.ubc.ca", '\0' <repeats 29 times>
#0 0x00000035aa4cced2 in select () from /lib64/libc.so.6
#1 0x00000031b0a1603b in bnet_thread_server (addrs=<value optimized out>,
max_clients=<value optimized out>, client_wq=<value optimized out>,
handle_client_request=<value optimized out>) at bnet_server.c:161
#2 0x000000000040796d in main (argc=<value optimized out>, argv=<value
optimized out>) at stored.c:312
Thread 13 (Thread 0x434aa940 (LWP 28832)):
#0 0x00000035ab00af70 in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00000031b0a3fa01 in bthread_cond_timedwait_p (cond=<value optimized out>,
m=<value optimized out>, abstime=<value optimized out>, file=<value optimized
out>, line=<value optimized out>) at lockmgr.c:786
#2 0x00000031b0a3ad1a in watchdog_thread (arg=<value optimized out>) at
watchdog.c:308
#3 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#4 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#5 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 12 (Thread 0x4acb6940 (LWP 29329)):
#0 0x00000035ab00d2e4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00000035ab008c3a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x00000035ab008afc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00000031b0a3fc7b in lmgr_p (m=<value optimized out>) at lockmgr.c:95
#4 0x00000031b0a3fded in bthread_mutex_lock_p (m=<value optimized out>,
file=<value optimized out>, line=<value optimized out>) at lockmgr.c:718
#5 0x00000000004277f9 in DEVICE::r_dlock (this=0x10112a58, locked=<value
optimized out>) at lock.c:222
#6 0x000000000042796b in DEVICE::dblock (this=0x10112a90, why=7) at lock.c:145
#7 0x00000000004336c4 in despool_data (dcr=0x101179f8, commit=true) at
dev.h:557
#8 0x0000000000434062 in commit_data_spool (dcr=0x101179f8) at spool.c:136
#9 0x000000000040f829 in do_append_data (jcr=0x10129e08) at append.c:318
#10 0x0000000000422def in append_data_cmd (jcr=0x10129e08) at fd_cmds.c:203
#11 0x00000000004225a3 in do_fd_commands (jcr=0x10129e08) at fd_cmds.c:162
#12 0x0000000000422d09 in run_job (jcr=0x10129e08) at fd_cmds.c:124
#13 0x0000000000423383 in run_cmd (jcr=0x10129e08) at job.c:225
#14 0x000000000041fbe8 in handle_connection_request (arg=0x10236928) at
dircmd.c:233
#15 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#16 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#17 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#18 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 11 (Thread 0x4a2b5940 (LWP 29333)):
#0 0x00000035ab00d2e4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00000035ab008c3a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x00000035ab008afc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00000031b0a3fc7b in lmgr_p (m=<value optimized out>) at lockmgr.c:95
#4 0x00000031b0a3fded in bthread_mutex_lock_p (m=<value optimized out>,
file=<value optimized out>, line=<value optimized out>) at lockmgr.c:718
#5 0x00000000004277f9 in DEVICE::r_dlock (this=0x10112a58, locked=<value
optimized out>) at lock.c:222
#6 0x000000000042796b in DEVICE::dblock (this=0x10112a90, why=7) at lock.c:145
#7 0x00000000004336c4 in despool_data (dcr=0x101122d8, commit=true) at
dev.h:557
#8 0x0000000000434062 in commit_data_spool (dcr=0x101122d8) at spool.c:136
#9 0x000000000040f829 in do_append_data (jcr=0x101159e8) at append.c:318
#10 0x0000000000422def in append_data_cmd (jcr=0x101159e8) at fd_cmds.c:203
#11 0x00000000004225a3 in do_fd_commands (jcr=0x101159e8) at fd_cmds.c:162
#12 0x0000000000422d09 in run_job (jcr=0x101159e8) at fd_cmds.c:124
#13 0x0000000000423383 in run_cmd (jcr=0x101159e8) at job.c:225
#14 0x000000000041fbe8 in handle_connection_request (arg=0x10111a08) at
dircmd.c:233
#15 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#16 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#17 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#18 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 10 (Thread 0x452ad940 (LWP 29371)):
#0 0x00000035ab00d2e4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00000035ab008c3a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x00000035ab008afc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00000031b0a3fc7b in lmgr_p (m=<value optimized out>) at lockmgr.c:95
#4 0x00000031b0a3fded in bthread_mutex_lock_p (m=<value optimized out>,
file=<value optimized out>, line=<value optimized out>) at lockmgr.c:718
#5 0x00000000004277f9 in DEVICE::r_dlock (this=0x10112a58, locked=<value
optimized out>) at lock.c:222
#6 0x000000000042796b in DEVICE::dblock (this=0x10112a90, why=7) at lock.c:145
#7 0x00000000004336c4 in despool_data (dcr=0x10122198, commit=true) at
dev.h:557
#8 0x0000000000434062 in commit_data_spool (dcr=0x10122198) at spool.c:136
#9 0x000000000040f829 in do_append_data (jcr=0x101163c8) at append.c:318
#10 0x0000000000422def in append_data_cmd (jcr=0x101163c8) at fd_cmds.c:203
#11 0x00000000004225a3 in do_fd_commands (jcr=0x101163c8) at fd_cmds.c:162
#12 0x0000000000422d09 in run_job (jcr=0x101163c8) at fd_cmds.c:124
#13 0x0000000000423383 in run_cmd (jcr=0x101163c8) at job.c:225
#14 0x000000000041fbe8 in handle_connection_request (arg=0x1011c288) at
dircmd.c:233
#15 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#16 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#17 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#18 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 9 (Thread 0x466af940 (LWP 29390)):
#0 0x00000035aa4c5f3b in read () from /lib64/libc.so.6
#1 0x00000035aa46cc07 in _IO_new_file_underflow () from /lib64/libc.so.6
#2 0x00000035aa46d5ce in _IO_default_uflow_internal () from /lib64/libc.so.6
#3 0x00000035aa468e8b in getc () from /lib64/libc.so.6
#4 0x00000031b0a135c0 in bfgets (s=<value optimized out>, size=<value
optimized out>, fd=<value optimized out>) at bsys.c:617
#5 0x000000000040d432 in release_device (dcr=0x1011e1e8) at acquire.c:533
#6 0x000000000040f83b in do_append_data (jcr=0x1012e7d8) at append.c:329
#7 0x0000000000422def in append_data_cmd (jcr=0x1012e7d8) at fd_cmds.c:203
#8 0x00000000004225a3 in do_fd_commands (jcr=0x1012e7d8) at fd_cmds.c:162
#9 0x0000000000422d09 in run_job (jcr=0x1012e7d8) at fd_cmds.c:124
#10 0x0000000000423383 in run_cmd (jcr=0x1012e7d8) at job.c:225
#11 0x000000000041fbe8 in handle_connection_request (arg=0x10130038) at
dircmd.c:233
#12 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#13 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#14 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#15 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 8 (Thread 0x4e8bc940 (LWP 29393)):
#0 0x00000035ab00d2e4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00000035ab008c3a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x00000035ab008afc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00000031b0a3fc7b in lmgr_p (m=<value optimized out>) at lockmgr.c:95
#4 0x00000031b0a3fded in bthread_mutex_lock_p (m=<value optimized out>,
file=<value optimized out>, line=<value optimized out>) at lockmgr.c:718
#5 0x00000000004277f9 in DEVICE::r_dlock (this=0x10112a58, locked=<value
optimized out>) at lock.c:222
#6 0x000000000042796b in DEVICE::dblock (this=0x10112a90, why=7) at lock.c:145
#7 0x00000000004336c4 in despool_data (dcr=0x10122d88, commit=true) at
dev.h:557
#8 0x0000000000434062 in commit_data_spool (dcr=0x10122d88) at spool.c:136
#9 0x000000000040f829 in do_append_data (jcr=0x1012da08) at append.c:318
#10 0x0000000000422def in append_data_cmd (jcr=0x1012da08) at fd_cmds.c:203
#11 0x00000000004225a3 in do_fd_commands (jcr=0x1012da08) at fd_cmds.c:162
#12 0x0000000000422d09 in run_job (jcr=0x1012da08) at fd_cmds.c:124
#13 0x0000000000423383 in run_cmd (jcr=0x1012da08) at job.c:225
#14 0x000000000041fbe8 in handle_connection_request (arg=0x10132958) at
dircmd.c:233
#15 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#16 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#17 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#18 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 7 (Thread 0x42aa9940 (LWP 29399)):
#0 0x00000035ab00d2e4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00000035ab008c3a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x00000035ab008afc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00000031b0a3fc7b in lmgr_p (m=<value optimized out>) at lockmgr.c:95
#4 0x00000031b0a3fded in bthread_mutex_lock_p (m=<value optimized out>,
file=<value optimized out>, line=<value optimized out>) at lockmgr.c:718
#5 0x00000000004277f9 in DEVICE::r_dlock (this=0x10112a58, locked=<value
optimized out>) at lock.c:222
#6 0x000000000042796b in DEVICE::dblock (this=0x10112a90, why=7) at lock.c:145
#7 0x00000000004336c4 in despool_data (dcr=0x10119de8, commit=true) at
dev.h:557
#8 0x0000000000434062 in commit_data_spool (dcr=0x10119de8) at spool.c:136
#9 0x000000000040f829 in do_append_data (jcr=0x101257d8) at append.c:318
#10 0x0000000000422def in append_data_cmd (jcr=0x101257d8) at fd_cmds.c:203
#11 0x00000000004225a3 in do_fd_commands (jcr=0x101257d8) at fd_cmds.c:162
#12 0x0000000000422d09 in run_job (jcr=0x101257d8) at fd_cmds.c:124
#13 0x0000000000423383 in run_cmd (jcr=0x101257d8) at job.c:225
#14 0x000000000041fbe8 in handle_connection_request (arg=0x1012d5b8) at
dircmd.c:233
#15 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#16 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#17 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#18 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x470b0940 (LWP 29415)):
#0 0x00000035ab00d2e4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00000035ab008c3a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x00000035ab008afc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00000031b0a3fc7b in lmgr_p (m=<value optimized out>) at lockmgr.c:95
#4 0x00000031b0a3fded in bthread_mutex_lock_p (m=<value optimized out>,
file=<value optimized out>, line=<value optimized out>) at lockmgr.c:718
#5 0x00000000004277f9 in DEVICE::r_dlock (this=0x10112a58, locked=<value
optimized out>) at lock.c:222
#6 0x000000000042796b in DEVICE::dblock (this=0x10112a90, why=7) at lock.c:145
#7 0x00000000004336c4 in despool_data (dcr=0x10131a48, commit=true) at
dev.h:557
#8 0x0000000000434062 in commit_data_spool (dcr=0x10131a48) at spool.c:136
#9 0x000000000040f829 in do_append_data (jcr=0x1011aa88) at append.c:318
#10 0x0000000000422def in append_data_cmd (jcr=0x1011aa88) at fd_cmds.c:203
#11 0x00000000004225a3 in do_fd_commands (jcr=0x1011aa88) at fd_cmds.c:162
#12 0x0000000000422d09 in run_job (jcr=0x1011aa88) at fd_cmds.c:124
#13 0x0000000000423383 in run_cmd (jcr=0x1011aa88) at job.c:225
#14 0x000000000041fbe8 in handle_connection_request (arg=0x100f37e8) at
dircmd.c:233
#15 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#16 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#17 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#18 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x47ab1940 (LWP 29416)):
#0 0x00000035ab00d2e4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00000035ab008c3a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x00000035ab008afc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00000031b0a3fc7b in lmgr_p (m=<value optimized out>) at lockmgr.c:95
#4 0x00000031b0a3fded in bthread_mutex_lock_p (m=<value optimized out>,
file=<value optimized out>, line=<value optimized out>) at lockmgr.c:718
#5 0x00000000004277f9 in DEVICE::r_dlock (this=0x10112a58, locked=<value
optimized out>) at lock.c:222
#6 0x000000000042796b in DEVICE::dblock (this=0x10112a90, why=7) at lock.c:145
#7 0x00000000004336c4 in despool_data (dcr=0x101548f8, commit=true) at
dev.h:557
#8 0x0000000000434062 in commit_data_spool (dcr=0x101548f8) at spool.c:136
#9 0x000000000040f829 in do_append_data (jcr=0x10297c08) at append.c:318
#10 0x0000000000422def in append_data_cmd (jcr=0x10297c08) at fd_cmds.c:203
#11 0x00000000004225a3 in do_fd_commands (jcr=0x10297c08) at fd_cmds.c:162
#12 0x0000000000422d09 in run_job (jcr=0x10297c08) at fd_cmds.c:124
#13 0x0000000000423383 in run_cmd (jcr=0x10297c08) at job.c:225
#14 0x000000000041fbe8 in handle_connection_request (arg=0x10237058) at
dircmd.c:233
#15 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#16 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#17 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#18 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x43eab940 (LWP 29424)):
#0 0x00000035ab00ad09 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00000031b0a30016 in rwl_writelock (rwl=<value optimized out>) at
rwlock.c:242
#2 0x0000000000438217 in _lock_volumes () at vol_mgr.c:112
#3 0x00000000004383f6 in dup_vol_list (jcr=0x650a5c) at vol_mgr.c:766
#4 0x0000000000431645 in find_suitable_device_for_job (jcr=0x10154ed8,
rc...@0x43eaac10) at reserve.c:430
#5 0x00000000004323c7 in use_cmd (jcr=0x10154ed8) at reserve.c:317
#6 0x000000000041fbe8 in handle_connection_request (arg=0x10125d88) at
dircmd.c:233
#7 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#8 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#9 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#10 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x41297940 (LWP 29425)):
#0 0x00000035ab00ad09 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00000031b0a30016 in rwl_writelock (rwl=<value optimized out>) at
rwlock.c:242
#2 0x0000000000438217 in _lock_volumes () at vol_mgr.c:112
#3 0x00000000004387f7 in list_volumes (sendit=0x434ee0 <sendit>,
arg=0x41296cd0) at vol_mgr.c:226
#4 0x00000000004369c6 in output_status (sp=0x41296cd0) at status.c:103
#5 0x0000000000436a90 in status_cmd (jcr=<value optimized out>) at status.c:680
#6 0x000000000041fbe8 in handle_connection_request (arg=0x10116878) at
dircmd.c:233
#7 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#8 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#9 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#10 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x448ac940 (LWP 29426)):
#0 0x00000035ab00ad09 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00000031b0a30016 in rwl_writelock (rwl=<value optimized out>) at
rwlock.c:242
#2 0x0000000000431bc7 in _lock_reservations () at reserve.c:108
#3 0x0000000000432286 in use_cmd (jcr=0x10305c58) at reserve.c:275
#4 0x000000000041fbe8 in handle_connection_request (arg=0x10236f18) at
dircmd.c:233
#5 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#6 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#7 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#8 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x2ad819585560 (LWP 28827)):
#0 0x00000035aa4cced2 in select () from /lib64/libc.so.6
#1 0x00000031b0a1603b in bnet_thread_server (addrs=<value optimized out>,
max_clients=<value optimized out>, client_wq=<value optimized out>,
handle_client_request=<value optimized out>) at bnet_server.c:161
#2 0x000000000040796d in main (argc=<value optimized out>, argv=<value
optimized out>) at stored.c:312
#0 0x00000035aa4cced2 in select () from /lib64/libc.so.6
No symbol table info available.
#1 0x00000031b0a1603b in bnet_thread_server (addrs=<value optimized out>,
max_clients=<value optimized out>, client_wq=<value optimized out>,
handle_client_request=<value optimized out>) at bnet_server.c:161
161 if ((stat = select(maxfd + 1, &sockset, NULL, NULL, NULL)) < 0) {
Current language: auto; currently c++
maxfd = <value optimized out>
sockset = Could not find the frame base for "bnet_thread_server(dlist*, int,
workq_tag*, void* (*)(void*))".
newsockfd = <value optimized out>
stat = <value optimized out>
clilen = <value optimized out>
cli_addr = Could not find the frame base for "bnet_thread_server(dlist*, int,
workq_tag*, void* (*)(void*))".
tlog = <value optimized out>
turnon = 1
p = <value optimized out>
fd_ptr = <value optimized out>
buf = Could not find the frame base for "bnet_thread_server(dlist*, int,
workq_tag*, void* (*)(void*))".
sockfds = Could not find the frame base for "bnet_thread_server(dlist*, int,
workq_tag*, void* (*)(void*))".
allbuf = Could not find the frame base for "bnet_thread_server(dlist*, int,
workq_tag*, void* (*)(void*))".
#2 0x000000000040796d in main (argc=<value optimized out>, argv=<value
optimized out>) at stored.c:312
312 &dird_workq, handle_connection_request);
ch = <value optimized out>
no_signals = false
test_config = false
thid = 1118476608
uid = 0x7fffddc1de8a "bacula"
gid = 0x7fffddc1de94 "disk"
#0 0x0000000000000000 in ?? ()
No symbol table info available.
#0 0x0000000000000000 in ?? ()
No symbol table info available.
#0 0x0000000000000000 in ?? ()
No symbol table info available.
#0 0x0000000000000000 in ?? ()
No symbol table info available.
#0 0x0000000000000000 in ?? ()
No symbol table info available.
[Thread debugging using libthread_db enabled]
[New Thread 0x466af940 (LWP 29422)]
0x00000035aa4df9ee in __lll_lock_wait_private () from /lib64/libc.so.6
$1 = "cbs-01-sd", '\0' <repeats 20 times>
$2 = 0x100f0528 "bacula-sd"
$3 = 0x100f0568 "/usr/sbin/"
$4 = 0x0
$5 = 0x31b0a450e6 "5.0.1 (24 February 2010)"
$6 = 0x31b0a450ff "x86_64-redhat-linux-gnu"
$7 = 0x31b0a45117 "redhat"
$8 = 0x31b0a44ddd ""
$9 = "agnatha.chibi.ubc.ca", '\0' <repeats 29 times>
#0 0x00000035aa4df9ee in __lll_lock_wait_private () from /lib64/libc.so.6
#1 0x00000035aa4d06a5 in _L_lock_1206 () from /lib64/libc.so.6
#2 0x00000035aa4d05be in closelog () from /lib64/libc.so.6
#3 0x00000031b0a19d35 in open_bpipe (prog=<value optimized out>, wait=<value
optimized out>, mode=<value optimized out>) at bpipe.c:138
#4 0x000000000040d3f1 in release_device (dcr=0x1011e1e8) at acquire.c:531
#5 0x000000000040f83b in do_append_data (jcr=0x1012e7d8) at append.c:329
#6 0x0000000000422def in append_data_cmd (jcr=0x1012e7d8) at fd_cmds.c:203
#7 0x00000000004225a3 in do_fd_commands (jcr=0x1012e7d8) at fd_cmds.c:162
#8 0x0000000000422d09 in run_job (jcr=0x1012e7d8) at fd_cmds.c:124
#9 0x0000000000423383 in run_cmd (jcr=0x1012e7d8) at job.c:225
#10 0x000000000041fbe8 in handle_connection_request (arg=0x10130038) at
dircmd.c:233
#11 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#12 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#13 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#14 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x466af940 (LWP 29422)):
#0 0x00000035aa4df9ee in __lll_lock_wait_private () from /lib64/libc.so.6
#1 0x00000035aa4d06a5 in _L_lock_1206 () from /lib64/libc.so.6
#2 0x00000035aa4d05be in closelog () from /lib64/libc.so.6
#3 0x00000031b0a19d35 in open_bpipe (prog=<value optimized out>, wait=<value
optimized out>, mode=<value optimized out>) at bpipe.c:138
#4 0x000000000040d3f1 in release_device (dcr=0x1011e1e8) at acquire.c:531
#5 0x000000000040f83b in do_append_data (jcr=0x1012e7d8) at append.c:329
#6 0x0000000000422def in append_data_cmd (jcr=0x1012e7d8) at fd_cmds.c:203
#7 0x00000000004225a3 in do_fd_commands (jcr=0x1012e7d8) at fd_cmds.c:162
#8 0x0000000000422d09 in run_job (jcr=0x1012e7d8) at fd_cmds.c:124
#9 0x0000000000423383 in run_cmd (jcr=0x1012e7d8) at job.c:225
#10 0x000000000041fbe8 in handle_connection_request (arg=0x10130038) at
dircmd.c:233
#11 0x00000031b0a3b5a2 in workq_server (arg=<value optimized out>) at
workq.c:346
#12 0x00000031b0a406d3 in lmgr_thread_launcher (x=<value optimized out>) at
lockmgr.c:882
#13 0x00000035ab006617 in start_thread () from /lib64/libpthread.so.0
#14 0x00000035aa4d3c2d in clone () from /lib64/libc.so.6
#0 0x00000035aa4df9ee in __lll_lock_wait_private () from /lib64/libc.so.6
No symbol table info available.
#1 0x00000035aa4d06a5 in _L_lock_1206 () from /lib64/libc.so.6
No symbol table info available.
#2 0x00000035aa4d05be in closelog () from /lib64/libc.so.6
No symbol table info available.
#3 0x00000031b0a19d35 in open_bpipe (prog=<value optimized out>, wait=<value
optimized out>, mode=<value optimized out>) at bpipe.c:138
138 closelog(); /* close syslog if open */
Current language: auto; currently c++
bargv = Could not find the frame base for "open_bpipe(char*, int, char const*)".
bargc = <value optimized out>
i = <value optimized out>
readp = Could not find the frame base for "open_bpipe(char*, int, char const*)".
writep = Could not find the frame base for "open_bpipe(char*, int, char
const*)".
tprog = Could not find the frame base for "open_bpipe(char*, int, char const*)".
mode_read = <value optimized out>
bpipe = <value optimized out>
save_errno = <value optimized out>
#4 0x000000000040d3f1 in release_device (dcr=0x1011e1e8) at acquire.c:531
531 bpipe = open_bpipe(alert, 0, "r");
alert = (POOLMEM *) 0x35aa753c90 "\002"
status = <value optimized out>
bpipe = <value optimized out>
line =
"8\000\023\020\000\000\000\000d\000\000\000\000\000\000\000p?d\000\000\000\000\000?v*\020\000\000\000\000??\021\020\000\000\000\000?\220??1\000\000\000p?h\020\000\000\000\000p#r?5\000\000\000c\000\001\000\000\000\000\000??\022\020\000\000\000\000?m??\000\000\000\000c\000\001\000\000\000\000\0008\000\023\020\000\000\000\000d\000\001\000\000\000\000\000p?d\000\000\000\000\000?v*\020\000\000\000\000??\021\020\000\000\000\000?\223??1\000\000\00080911\000\000\000??jf\000\000\000\000??jf\000\000\000\...@\\*\020\000\000\000\000(\\*\020\000\000\000\000\000??\000\...@\225}c\000\001\000\000\000\000\000"...
jcr = (JCR *) 0x1012e7d8
dev = (DEVICE *) 0x10112a58
tbuf =
"\by+\020\000\000\000\000?V??1\000\000\000??\022\020\000\000\000\000????1\000\000\000?$Ű1\000\000\000?V??1",
'\0' <repeats 11 times>,
"?\000\000\000\000\000\000\000?$Ű1\000\000\000?\000??1\000\000\000??\021\020\000\000\000\000??\021\020\000\000\000\000\000\000\000"
#5 0x000000000040f83b in do_append_data (jcr=0x1012e7d8) at append.c:329
329 release_device(dcr);
n = <value optimized out>
file_index = 0
stream = -1331425024
last_file_index = 0
fd = <value optimized out>
ok = true
rec = {link = {next = 0x0, prev = 0x0}, File = 0, Block = 0, VolSessionId = 0,
VolSessionTime = 0, FileIndex = 0, Stream = 0, data_len = 0, remainder = 0,
state = 0, bsr = 0x0, ser_buf = '\0' <repeats 11 times>, data = 0x0,
match_stat = 0, last_VolSessionId = 0, last_VolSessionTime = 0,
last_FileIndex = 0}
buf1 =
"\000?jf\000\000\000\000c\000\001\000\000\000\000\0008\000\023\020\000\000\000\000d\000\001\000\000\000\000\000\026\000\000\000\000\000\000\000n\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000??\000\...@\225}\000\000\000\000\000\000\000\000\032\002\000\000\000\000\000\000?
??1", '\0' <repeats 12 times>, "\203\021\020"
buf2 =
"??\022\020\000\000\000\000\000??\000\...@\225}c\000\001\000\000\000\000\000\001\000\000\000\000\000\000\000?\034Ű1\000\000\000\224'??1\000\000\000??\021\020\000\000\000\000\000\002\000\000\032\002\000\000?
??1\000\000\000??\022\020\000\000\000\000??\021\020\000\000\000\000?\223??1\000\000\000\030\000\000"
dcr = (DCR *) 0x1011e1e8
dev = (DEVICE *) 0x10112a58
ec = "0 \000\000\000\000\000\000?(??1", '\0' <repeats 12 times>,
"/Ű1\000\000\000??\021\020\000\000\000\000|\217??1\000\000\000??"
job_elapsed = <value optimized out>
#6 0x0000000000422def in append_data_cmd (jcr=0x1012e7d8) at fd_cmds.c:203
203 if (do_append_data(jcr)) {
fd = (BSOCK *) 0x1011bcb8
#7 0x00000000004225a3 in do_fd_commands (jcr=0x1012e7d8) at fd_cmds.c:162
162 if (!fd_cmds[i].func(jcr)) { /* do command */
stat = <value optimized out>
i = 1
fd = (BSOCK *) 0x1011bcb8
Attempt to dump locks
threadid=0x448ac940 max=1 current=0
lock=0x6506c0 state=Wanted priority=0 rwlock.c:237
threadid=0x41297940 max=1 current=0
lock=0x650a00 state=Wanted priority=10 rwlock.c:237
threadid=0x43eab940 max=1 current=1
lock=0x6506c0 state=Granted priority=0 rwlock.c:237
lock=0x650a00 state=Wanted priority=10 rwlock.c:237
threadid=0x47ab1940 max=3 current=0
lock=0x10112a90 state=Wanted priority=5 lock.c:222
threadid=0x470b0940 max=3 current=0
lock=0x10112a90 state=Wanted priority=5 lock.c:222
threadid=0x42aa9940 max=3 current=0
lock=0x10112a90 state=Wanted priority=5 lock.c:222
threadid=0x4e8bc940 max=3 current=0
lock=0x10112a90 state=Wanted priority=5 lock.c:222
threadid=0x466af940 max=3 current=1
lock=0x10112a90 state=Granted priority=5 dev.h:462
lock=0x650a00 state=Granted priority=10 rwlock.c:237
threadid=0x452ad940 max=3 current=0
lock=0x10112a90 state=Wanted priority=5 lock.c:222
threadid=0x4a2b5940 max=3 current=0
lock=0x10112a90 state=Wanted priority=5 lock.c:222
threadid=0x4acb6940 max=3 current=0
lock=0x10112a90 state=Wanted priority=5 lock.c:222
threadid=0x434aa940 max=2 current=-1
threadid=0x2ad819585560 max=0 current=-1
Attempt to dump current JCRs
JCR=0x10129e08 JobId=11922 name=noc-var.2010-03-19_15.25.01_28 JobStatus=l
use_count=1
JobType=B JobLevel=I
sched_time=19-Mar-2010 15:30 start_time=19-Mar-2010 15:30
end_time=31-Dec-1969 16:00 wait_time=31-Dec-1969 16:00
db=(nil) db_batch=(nil) batch_started=0
JCR=0x101159e8 JobId=11899 name=agnatha-var.2010-03-19_15.25.01_05 JobStatus=l
use_count=1
JobType=B JobLevel=I
sched_time=19-Mar-2010 15:30 start_time=19-Mar-2010 15:30
end_time=31-Dec-1969 16:00 wait_time=31-Dec-1969 16:00
db=(nil) db_batch=(nil) batch_started=0
JCR=0x101163c8 JobId=11925 name=cs-01-etc.2010-03-19_15.25.01_31 JobStatus=l
use_count=1
JobType=B JobLevel=I
sched_time=19-Mar-2010 15:30 start_time=19-Mar-2010 15:30
end_time=31-Dec-1969 16:00 wait_time=31-Dec-1969 16:00
db=(nil) db_batch=(nil) batch_started=0
JCR=0x1012e7d8 JobId=11908 name=vs-02-home_vmail.2010-03-19_15.25.01_14
JobStatus=R
use_count=1
JobType=B JobLevel=I
sched_time=19-Mar-2010 15:30 start_time=19-Mar-2010 15:30
end_time=31-Dec-1969 16:00 wait_time=31-Dec-1969 16:00
db=(nil) db_batch=(nil) batch_started=0
JCR=0x1012da08 JobId=11928 name=jgdesk-01-etc.2010-03-19_15.25.01_34 JobStatus=l
use_count=1
JobType=B JobLevel=I
sched_time=19-Mar-2010 15:30 start_time=19-Mar-2010 15:30
end_time=31-Dec-1969 16:00 wait_time=31-Dec-1969 16:00
db=(nil) db_batch=(nil) batch_started=0
JCR=0x101257d8 JobId=11931 name=jgdesk-02-etc.2010-03-19_15.25.01_37 JobStatus=l
use_count=1
JobType=B JobLevel=I
sched_time=19-Mar-2010 15:30 start_time=19-Mar-2010 15:30
end_time=31-Dec-1969 16:00 wait_time=31-Dec-1969 16:00
db=(nil) db_batch=(nil) batch_started=0
JCR=0x1011aa88 JobId=11912 name=amniota-var.2010-03-19_15.25.01_18 JobStatus=l
use_count=1
JobType=B JobLevel=I
sched_time=19-Mar-2010 15:30 start_time=19-Mar-2010 15:30
end_time=31-Dec-1969 16:00 wait_time=31-Dec-1969 16:00
db=(nil) db_batch=(nil) batch_started=0
JCR=0x10297c08 JobId=11924 name=quimby-var.2010-03-19_15.25.01_30 JobStatus=l
use_count=1
JobType=B JobLevel=I
JobType=B JobLevel=I
sched_time=19-Mar-2010 15:30 start_time=31-Dec-1969 16:00
end_time=31-Dec-1969 16:00 wait_time=31-Dec-1969 16:00
db=(nil) db_batch=(nil) batch_started=0
Attempt to dump plugins. Hook count=0
------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Bacula-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-devel
