Hi all,
This morning I got a regression test failure on both mysql and sqlite3
versions of Bacula (postgresql ran fine) on Ubuntu 1004 (Lucid).
Details can be found in the regression dashboard at
mysql - http://regress.bacula.org/testDetails.php?test=402703&build=5591
the failure symptoms were the same on both. A signal 11 - Segmentation
violation.
@sleep 2
run comment="Should work" level=Full job=CancelLowerLevelDuplicatesYes yes
Bacula interrupted by signal 11: Segmentation violation
Kaboom! bacula-sd, localhost-sd got signal 11 - Segmentation violation.
Attempting traceback.
Kaboom! exepath=/home/regress/bacula/regress/bin/
Calling: /home/regress/bacula/regress/bin/btraceback
/home/regress/bacula/regress/bin/bacula-sd 30586
/home/regress/bacula/regress/working
Bacula emailed me a GDB traceback for both events however the MySQL
version failed to catch the process. The sqlite3 version looks to be a
complete trace however (full trace attached).
Looking at the trace I'm guessing that the problem might be somewhere in
libz/openssl because that is the only bit of code that is not waiting.
Thread 3 (Thread 0x7fcfc77fe710 (LWP 24355)):
#0 0x00007fcfcf7f4941 in ?? () from /lib/libz.so.1
#1 0x00007fcfcf7f3281 in deflate () from /lib/libz.so.1
#2 0x00007fcfcecfaa52 in ?? () from /lib/libcrypto.so.0.9.8
#3 0x00007fcfcecf9e10 in COMP_compress_block () from
/lib/libcrypto.so.0.9.8
#4 0x00007fcfcef93ffe in ssl3_do_compress () from /lib/libssl.so.0.9.8
#5 0x00007fcfcef9413e in ?? () from /lib/libssl.so.0.9.8
#6 0x00007fcfcef94656 in ssl3_write_bytes () from /lib/libssl.so.0.9.8
#7 0x00007fcfcfa3dfce in openssl_bsock_readwrite (bsock=0x2491278,
ptr=0x2551cbc "", nbytes=4) at tls.c:626
#8 tls_bsock_writen (bsock=0x2491278, ptr=0x2551cbc "", nbytes=4) at
tls.c:704
If it is libz/openssl related then I'm guessing that there isn't much
that needs fixing in Bacula itself.
Do anyone have any comments?
Regards
Richard
P.S. I think I may have had this same failure before in the past few
weeks so it is likely that it isn't a recent regression. I just didn't
have time to look at it then.
[Thread debugging using libthread_db enabled]
[New Thread 0x7fcfcd693710 (LWP 24357)]
[New Thread 0x7fcfc77fe710 (LWP 24355)]
[New Thread 0x7fcfcce92710 (LWP 24292)]
[New Thread 0x7fcfcde94710 (LWP 24288)]
0x00007fcfce18dff3 in select () from /lib/libc.so.6
$1 = '\000' <repeats 29 times>
$2 = 0x2459088 "bacula-sd"
$3 = 0x24590c8 "/home/regress/bacula/regress/bin/bacula-sd"
$4 = 0x0
$5 = 0x7fcfcfa5261b "5.1.7 (24 July 2010)"
$6 = 0x7fcfcfa5263d "x86_64-unknown-linux-gnu"
$7 = 0x7fcfcfa52656 "ubuntu"
$8 = 0x7fcfcfa52637 "10.04"
$9 = "royalcounty", '\000' <repeats 38 times>
$10 = 0x7fcfcfa52630 "ubuntu 10.04"
$11 = 0
TestName = duplicate-job-test
#0 0x00007fcfce18dff3 in select () from /lib/libc.so.6
#1 0x00007fcfcfa1df64 in bnet_thread_server (addrs=<value optimized out>,
max_clients=<value optimized out>, client_wq=<value optimized out>,
handle_client_request=<value optimized out>) at bnet_server.c:161
#2 0x00000000004084ea in main (argc=<value optimized out>,
argv=<value optimized out>) at stored.c:313
Thread 5 (Thread 0x7fcfcde94710 (LWP 24288)):
#0 0x00007fcfcf5e011d in nanosleep () from /lib/libpthread.so.0
#1 0x00007fcfcfa1b4f7 in bmicrosleep (sec=30, usec=0) at bsys.c:63
#2 0x00007fcfcfa4c2fb in check_deadlock () at lockmgr.c:574
#3 0x00007fcfcf5d79ca in start_thread () from /lib/libpthread.so.0
#4 0x00007fcfce19570d in clone () from /lib/libc.so.6
#5 0x0000000000000000 in ?? ()
Thread 4 (Thread 0x7fcfcce92710 (LWP 24292)):
#0 0x00007fcfcf5dcbc9 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
from /lib/libpthread.so.0
#1 0x00007fcfcfa4b290 in bthread_cond_timedwait_p (cond=0x7fcfcfc619a0,
m=0x7fcfcfc61960, abstime=0x7fcfcce91da0,
file=0x7fcfcfa5578a "watchdog.c", line=321) at lockmgr.c:824
#2 0x00007fcfcfa46378 in watchdog_thread (arg=<value optimized out>)
at watchdog.c:321
#3 0x00007fcfcfa4c242 in lmgr_thread_launcher (x=<value optimized out>)
at lockmgr.c:939
#4 0x00007fcfcf5d79ca in start_thread () from /lib/libpthread.so.0
#5 0x00007fcfce19570d in clone () from /lib/libc.so.6
#6 0x0000000000000000 in ?? ()
Thread 3 (Thread 0x7fcfc77fe710 (LWP 24355)):
#0 0x00007fcfcf7f4941 in ?? () from /lib/libz.so.1
#1 0x00007fcfcf7f3281 in deflate () from /lib/libz.so.1
#2 0x00007fcfcecfaa52 in ?? () from /lib/libcrypto.so.0.9.8
#3 0x00007fcfcecf9e10 in COMP_compress_block () from /lib/libcrypto.so.0.9.8
#4 0x00007fcfcef93ffe in ssl3_do_compress () from /lib/libssl.so.0.9.8
#5 0x00007fcfcef9413e in ?? () from /lib/libssl.so.0.9.8
#6 0x00007fcfcef94656 in ssl3_write_bytes () from /lib/libssl.so.0.9.8
#7 0x00007fcfcfa3dfce in openssl_bsock_readwrite (bsock=0x2491278,
ptr=0x2551cbc "", nbytes=4) at tls.c:626
#8 tls_bsock_writen (bsock=0x2491278, ptr=0x2551cbc "", nbytes=4) at tls.c:704
#9 0x00007fcfcfa1cf53 in write_nbytes (bsock=0x23e, ptr=0x2551cbc "",
nbytes=4) at bnet.c:131
#10 0x00007fcfcfa209a6 in BSOCK::send (this=0x2491278) at bsock.c:379
#11 0x000000000041f414 in handle_connection_request (arg=0x2491278)
at dircmd.c:251
#12 0x00007fcfcfa4682d in workq_server (arg=<value optimized out>)
at workq.c:346
#13 0x00007fcfcfa4c242 in lmgr_thread_launcher (x=<value optimized out>)
at lockmgr.c:939
#14 0x00007fcfcf5d79ca in start_thread () from /lib/libpthread.so.0
#15 0x00007fcfce19570d in clone () from /lib/libc.so.6
#16 0x0000000000000000 in ?? ()
Thread 2 (Thread 0x7fcfcd693710 (LWP 24357)):
#0 0x00007fcfcf5e048d in waitpid () from /lib/libpthread.so.0
#1 0x00007fcfcfa3c2da in signal_handler (sig=11) at signal.c:229
#2 <signal handler called>
#3 0x00007fcfcf7f4680 in ?? () from /lib/libz.so.1
#4 0x00007fcfcf7f3281 in deflate () from /lib/libz.so.1
#5 0x00007fcfcecfaa52 in ?? () from /lib/libcrypto.so.0.9.8
#6 0x00007fcfcecf9e10 in COMP_compress_block () from /lib/libcrypto.so.0.9.8
#7 0x00007fcfcef93ffe in ssl3_do_compress () from /lib/libssl.so.0.9.8
#8 0x00007fcfcef9413e in ?? () from /lib/libssl.so.0.9.8
#9 0x00007fcfcef94656 in ssl3_write_bytes () from /lib/libssl.so.0.9.8
#10 0x00007fcfcfa3dfce in openssl_bsock_readwrite (bsock=0x2491278,
ptr=0x2551cbc "", nbytes=202) at tls.c:626
#11 tls_bsock_writen (bsock=0x2491278, ptr=0x2551cbc "", nbytes=202)
at tls.c:704
#12 0x00007fcfcfa1cf53 in write_nbytes (bsock=0x3218, ptr=0x2551cbc "",
nbytes=202) at bnet.c:131
#13 0x00007fcfcfa209a6 in BSOCK::send (this=0x2491278) at bsock.c:379
#14 0x00007fcfcfa20d43 in BSOCK::fsend (this=0x2491278,
fmt=0x7fcfcfa529d0 "Jmsg Job=%s type=%d level=%lld %s") at bsock.c:434
#15 0x00007fcfcfa31b21 in dispatch_message (jcr=<value optimized out>,
type=<value optimized out>, mtime=1291179811, msg=<value optimized out>)
at message.c:888
#16 0x00007fcfcfa32590 in Jmsg (jcr=0x2549618, type=6, mtime=0,
fmt=0x442880 "JobId=%d Job=\"%s\" marked to be canceled.\n")
at message.c:1296
#17 0x000000000041ec55 in cancel_cmd (cjcr=<value optimized out>)
at dircmd.c:344
#18 0x000000000041f527 in handle_connection_request (arg=0x2494478)
at dircmd.c:233
#19 0x00007fcfcfa4682d in workq_server (arg=<value optimized out>)
at workq.c:346
#20 0x00007fcfcfa4c242 in lmgr_thread_launcher (x=<value optimized out>)
at lockmgr.c:939
#21 0x00007fcfcf5d79ca in start_thread () from /lib/libpthread.so.0
#22 0x00007fcfce19570d in clone () from /lib/libc.so.6
#23 0x0000000000000000 in ?? ()
Thread 1 (Thread 0x7fcfd0274720 (LWP 24286)):
#0 0x00007fcfce18dff3 in select () from /lib/libc.so.6
#1 0x00007fcfcfa1df64 in bnet_thread_server (addrs=<value optimized out>,
max_clients=<value optimized out>, client_wq=<value optimized out>,
handle_client_request=<value optimized out>) at bnet_server.c:161
#2 0x00000000004084ea in main (argc=<value optimized out>,
argv=<value optimized out>) at stored.c:313
#0 0x00007fcfce18dff3 in select () from /lib/libc.so.6
No symbol table info available.
#1 0x00007fcfcfa1df64 in bnet_thread_server (addrs=<value optimized out>,
max_clients=<value optimized out>, client_wq=<value optimized out>,
handle_client_request=<value optimized out>) at bnet_server.c:161
warning: Source file is more recent than executable.
161 if ((stat = select(maxfd + 1, &sockset, NULL, NULL, NULL)) < 0) {
maxfd = 3
sockset = {fds_bits = {8, 0 <repeats 15 times>}}
clilen = 16
turnon = 1
p = <value optimized out>
buf = "127.0.0.1\000\000\000\000\000\000\000\370\004e", '\000' <repeats 37
times>"\321, \266\a\320\317\177\000\000\005", '\000' <repeats 15 times>"\212,
W\245\317\317\177\000\000(\310\240\317\317\177\000\000\000\000\000\000\000\000\000\000
\364\330k\377\177\000\000$\000\000\000\000\000\000\000\001\000\000\000\000\000\000"
allbuf =
"\273\333\321\316\317\177\000\000n\001\000\000\000\000\000\000\360\207g\002\000\000\000\000\003\000\000\000\000\000\000\000\273\333\321\316\317\177\000\000\003\000\000\000\000\000\000\000p\000\000\000\000\000\000\000\003\000\000\000\000\000\000\000\225ng\002\000\000\000\000\002\000\000\000\317\177\000\...@\000\000\000\000\000\000\000\002\000\000\000\317\177\000\000@\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\020\235g\002\000\000\000\...@\336b\316\317\177\000\000\273\333\321\316\317\177\000\000@\336B\316\317\177\000\000!\362\321\316\317\177\000\000\n\000\000\000\000\000\000\000\215nG\002\000\000\000\000\060\235G\002\000\000\000\000\v\000\000\000\000\000\000\000\216\265\022\316\317\177\000\000\273\333\321\316\317\177\000\000N\001\000\000\000\000\000\000\020\235G\002\000\000\000\000\003\000\000\000\000\000\000\000\273\333\321\316\317\177\000\000\n\000\000\000\000\000\000\000!\362\321\316\317\177\000\000w\001\000\000\000\000\000\000\215"...
stat = <value optimized out>
tlog = <value optimized out>
fd_ptr = 0x0
sockfds = {<SMARTALLOC> = {<No data fields>}, head = 0x7fff6bd8e350,
tail = 0x7fff6bd8e350, loffset = 0, num_items = 1}
newsockfd = <value optimized out>
cli_addr = {sa_family = 2,
sa_data = "\341\210\177\000\000\001\000\000\000\000\000\000\000"}
request = {fd = 5, user = '\000' <repeats 127 times>,
daemon = "localhost-sd", '\000' <repeats 115 times>,
pid = "24286\000\000\000\000", client = {{name = '\000' <repeats 127 times>,
addr = '\000' <repeats 127 times>, sin = 0x7fcfcf3cbb20, unit = 0x0,
request = 0x7fff6bd8ee90}}, server = {{
name = '\000' <repeats 127 times>, addr = '\000' <repeats 127 times>,
sin = 0x7fcfcf3cbaa0, unit = 0x0, request = 0x7fff6bd8ee90}}, sink = 0,
hostname = 0x7fcfcf1c83d0 <sock_hostname>,
hostaddr = 0x7fcfcf1c81b0 <sock_hostaddr>, cleanup = 0, config = 0x0}
#2 0x00000000004084ea in main (argc=<value optimized out>,
argv=<value optimized out>) at stored.c:313
313 &dird_workq, handle_connection_request);
test_config = false
ch = <value optimized out>
no_signals = false
thid = 140530481182480
uid = 0x0
gid = 0x0
#0 0x0000000000000000 in ?? ()
No symbol table info available.
#0 0x0000000000000000 in ?? ()
No symbol table info available.
#0 0x0000000000000000 in ?? ()
No symbol table info available.
#0 0x0000000000000000 in ?? ()
No symbol table info available.
#0 0x0000000000000000 in ?? ()
No symbol table info available.
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Bacula-devel mailing list
Bacula-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel
