A useful trick for finding similar reports is to search the internet for the functions listed near the top of the backtrace. In this case, googling for:
bacula "clear_slot" "set_slot" leads to this bug report: https://bugs.bacula.org/view.php?id=2231 where kern suggests compiling with -O0 as a workaround. __Martin >>>>> On Sat, 7 Sep 2019 14:24:43 -0700, Chandler said: > > Hey all, I'm trying to configure a new storage demon on a new server with > Debian 10 (buster) but need to use the 5.2 version for compatibility. I used > configure "--with-sqlite3 --without-openssl" and "make" and didn't have any > problems, just some warnings. When I start bacula-sd I see a signal 11 > segmentation violation. The last lines of bacula-sd debug log are printed > below and the backtrace is below that. Is the problem because the OS/C > libraries are too new? Or is there too much RAM (192 GB) or too much storage > in /not1/bacula (140 TB)? Would appreciate a solution with this version > since I'd rather not update 20 other servers to the 9.x brach if I can get > this working. > > Thanks, > Chandler / Systems Administrator > Arizona Genomics Institute > University Of Arizona > www.genome.arizona.edu > -- > > # bacula-sd -d 10000 -c bacula-sd.conf > [snip] > > Device: name=Not1-ZFS4-restore MediaType=File Device=/not1/bacula LabelType=0 > rew_wait=300 min_bs=0 max_bs=0 chgr_wait=300 > max_jobs=0 max_files=0 max_size=0 > max_file_size=1000000000 capacity=0 > spool_directory=*None* > max_spool_size=0 max_job_spool_size=0 > CAP_EOF CAP_BSR CAP_BSF CAP_FSR CAP_FSF CAP_EOM CAP_RACCESS > CAP_AUTOMOUNT CAP_LABEL > dump_resource type=3004 > Messages: name=Standard > Warning: no "autochanger" resource (3005) defined. > bacula-sd: lex.c:131-0 Close lex file: /etc/bacula/bacula-sd.conf > bacula-sd: lex.c:140-0 Close cfg file /etc/bacula/bacula-sd.conf > bacula-sd: smartall.c:190-0 sm_free 68 at 55eff32e1f18 from lex.c:173 > bacula-sd: smartall.c:190-0 sm_free 5065 at 55eff32e37d8 from lex.c:210 > bacula-sd: mem_pool.c:235-0 free_pool_memory 55eff32e37d8 pool=0 from > lex.c:142 > bacula-sd: smartall.c:190-0 sm_free 681 at 55eff32e1f68 from lex.c:186 > bacula-sd: parse_conf.c:1033-0 Leave parse_config() > bacula-sd: message.c:504-0 Close_msg jcr=0 > bacula-sd: message.c:529-0 ===Begin close msg resource at 55eff32e1168 > bacula-sd: mem_pool.c:123-0 sm_get_pool_memory reuse 55eff32e71f8 to > message.c:530 > bacula-sd: mem_pool.c:235-0 free_pool_memory 55eff32e71f8 pool=3 from > message.c:636 > bacula-sd: message.c:637-0 Done walking message chain. > bacula-sd: message.c:644-0 ===End close msg resource > bacula-sd: smartall.c:147-0 smalloc 97 at 55eff32e37d8 from message.c:317 > bacula-sd: smartall.c:147-0 smalloc 50 at 55eff32e74b8 from message.c:326 > bacula-sd: smartall.c:190-0 sm_free 97 at 55eff32e1238 from message.c:409 > bacula-sd: smartall.c:190-0 sm_free 185 at 55eff32e1168 from message.c:303 > bacula-sd: smartall.c:147-0 smalloc 185 at 55eff32e1168 from message.c:341 > bacula-sd: message.c:347-0 Copy message resource 55eff32e74f8 to 55eff32e37d8 > not1-sd: daemon.c:62-0 Enter daemon_start > not1-sd: daemon.c:135-0 Exit daemon_start > not1-sd: smartall.c:147-0 smalloc 321 at 55eff32e3848 from bsys.c:457 > not1-sd: mem_pool.c:139-0 sm_get_pool_memory give 55eff32e3848 to bsys.c:457 > not1-sd: mem_pool.c:235-0 free_pool_memory 55eff32e3848 pool=2 from bsys.c:500 > not1-sd: mem_pool.c:123-0 sm_get_pool_memory reuse 55eff32e3848 to bsys.c:546 > not1-sd: mem_pool.c:123-0 sm_get_pool_memory reuse 55eff32e23c8 to > ../lib/berrno.h:77 > not1-sd: bsys.c:556-0 Could not open state file. sfd=-1 size=192: ERR=No such > file or directory > not1-sd: mem_pool.c:235-0 free_pool_memory 55eff32e23c8 pool=4 from > ../lib/berrno.h:84 > not1-sd: mem_pool.c:235-0 free_pool_memory 55eff32e3848 pool=2 from bsys.c:587 > not1-sd: smartall.c:147-0 smalloc 65 at 55eff32e3ce8 from lockmgr.c:617 > not1-sd: smartall.c:147-0 smalloc 1921 at 55eff32e3e68 from lockmgr.c:643 > not1-sd: sd_plugins.c:173-0 Load sd plugins > not1-sd: sd_plugins.c:175-0 No sd plugin dir! > not1-sd: priv.c:60-0 uname=NONE gname=NONE > not1-sd: mem_pool.c:123-0 sm_get_pool_memory reuse 55eff32e71f8 to > stored.c:470 > not1-sd: smartall.c:147-0 smalloc 1921 at 7f6b08000b48 from lockmgr.c:643 > not1-sd: smartall.c:147-0 smalloc 577 at 55eff32e45f8 from stored.c:471 > not1-sd: mem_pool.c:139-0 sm_get_pool_memory give 55eff32e45f8 to stored.c:471 > not1-sd: mem_pool.c:123-0 sm_get_pool_memory reuse 55eff32e23c8 to > ../lib/berrno.h:77 > not1-sd: smartall.c:147-0 smalloc 2345 at 55eff32f0248 from stored.c:508 > not1-sd: stored.c:516-0 Skipped: bacula.5990.traceback > not1-sd: stored.c:516-0 Skipped: .. > not1-sd: stored.c:516-0 Skipped: bacula.1762.traceback > not1-sd: stored.c:516-0 Skipped: . > not1-sd: smartall.c:190-0 sm_free 2345 at 55eff32f0248 from stored.c:508 > not1-sd: mem_pool.c:235-0 free_pool_memory 55eff32e71f8 pool=3 from > stored.c:534 > not1-sd: mem_pool.c:235-0 free_pool_memory 55eff32e45f8 pool=3 from > stored.c:535 > not1-sd: mem_pool.c:235-0 free_pool_memory 55eff32e23c8 pool=4 from > ../lib/berrno.h:84 > not1-sd: smartall.c:147-0 smalloc 65 at 55eff32e5298 from vol_mgr.c:693 > not1-sd: smartall.c:147-0 smalloc 65 at 55eff32f1ef8 from vol_mgr.c:696 > not1-sd: smartall.c:147-0 smalloc 57 at 55eff32f15d8 from lockmgr.c:951 > not1-sd: watchdog.c:91-0 Initialising NicB-hacked watchdog thread > not1-sd: smartall.c:147-0 smalloc 65 at 55eff32e4c58 from watchdog.c:99 > not1-sd: smartall.c:147-0 smalloc 65 at 55eff32e5708 from watchdog.c:100 > not1-sd: smartall.c:147-0 smalloc 57 at 55eff32e56b8 from lockmgr.c:951 > not1-sd: smartall.c:147-0 smalloc 1921 at 7f6b00000b48 from lockmgr.c:643 > not1-sd: smartall.c:190-0 sm_free 57 at 55eff32f15d8 from lockmgr.c:951 > not1-sd: jcr.c:339-0 Enter new_jcr > not1-sd: smartall.c:147-0 smalloc 1193 at 7f6b000012d8 from jcr.c:345 > not1-sd: smartall.c:147-0 smalloc 65 at 55eff32f15d8 from jcr.c:347 > not1-sd: mem_pool.c:123-0 sm_get_pool_memory reuse 55eff32e3848 to jcr.c:358 > not1-sd: mem_pool.c:123-0 sm_get_pool_memory reuse 55eff32e45f8 to jcr.c:360 > not1-sd: smartall.c:147-0 smalloc 321 at 7f6b00001798 from jcr.c:362 > not1-sd: mem_pool.c:139-0 sm_get_pool_memory give 7f6b00001798 to jcr.c:362 > not1-sd: jcr.c:910-0 set_jcr_job_status(*System*, C) > not1-sd: jcr.c:919-0 OnEntry JobStatus=not1-sd: jcr.c:930-0 Set new stat. > old: not1-sd: jcr.c:935-0 leave setJobStatus old=not1-sd: smartall.c:147-0 > smalloc 105 at 55eff32e5b48 from watchdog.c:172 > not1-sd: stored.c:564-0 calling init_dev /not1/bacula > not1-sd: smartall.c:147-0 smalloc 2441 at 7f6b000018e8 from dev.c:177 > not1-sd: watchdog.c:206-0 Registered watchdog 55eff32e5b48, interval 30 > not1-sd: signal.c:142-0 sig=11 Segmentation violation > Bacula interrupted by signal 11: Segmentation violation > not1-sd: smartall.c:147-0 smalloc 1921 at 7f6b04000b48 from lockmgr.c:643 > Kaboom! bacula-sd, not1-sd got signal 11 - Segmentation violation. Attempting > traceback. > Kaboom! exepath=/usr/sbin > not1-sd: smartall.c:190-0 sm_free 57 at 55eff32e56b8 from lockmgr.c:951 > not1-sd: signal.c:205-0 Working=/opt/bacula/working > not1-sd: signal.c:206-0 btpath=/usr/sbin/btraceback > not1-sd: signal.c:207-0 exepath=/usr/sbin/bacula-sd > not1-sd: watchdog.c:263-0 NicB-reworked watchdog thread entered > not1-sd: watchdog.c:318-0 pthread_cond_timedwait 30 > not1-sd: signal.c:236-0 Doing waitpid > not1-sd: bnet_server.c:112-0 Addresses host[ipv4:150.135.49.195:9103] > host[ipv4:150.135.62.66:9103] > Calling: /usr/sbin/btraceback /usr/sbin/bacula-sd 29938 /opt/bacula/working > not1-sd: signal.c:238-0 Done waitpid > It looks like the traceback worked... > Dumping: /opt/bacula/working/not1-sd.29938.bactrace > > root@not1:/opt/bacula/working# cat bacula.9923.traceback > [New LWP 9924] > [New LWP 9925] > [New LWP 9926] > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > 0x00007ffff7924037 in __GI___select (nfds=nfds@entry=7, > readfds=readfds@entry=0x7fffffffd9d0, writefds=writefds@entry=0x0, > exceptfds=exceptfds@entry=0x0, timeout=timeout@entry=0x0) at > ../sysdeps/unix/sysv/linux/select.c:41 > 41 ../sysdeps/unix/sysv/linux/select.c: No such file or directory. > $1 = 0x5555555b25c0 <my_name> "not1-sd" > $2 = 0x5555555c6038 "bacula-sd" > $3 = 0x5555555c6078 "/usr/sbin/bacula-sd" > $4 = 0x0 > $5 = 0x7ffff7fa05a9 "5.2.13 (19 February 2013)" > $6 = 0x7ffff7fa0584 "x86_64-unknown-linux-gnu" > $7 = 0x7ffff7fa057d "debian" > $8 = 0x7ffff7fa05a4 "10.0" > $9 = "not1", '\000' <repeats 45 times> > $10 = 0x7ffff7fa059d "debian 10.0" > $11 = 0 > Environment variable "TestName" not defined. > #0 0x00007ffff7924037 in __GI___select (nfds=nfds@entry=7, > readfds=readfds@entry=0x7fffffffd9d0, writefds=writefds@entry=0x0, > exceptfds=exceptfds@entry=0x0, timeout=timeout@entry=0x0) at > ../sysdeps/unix/sysv/linux/select.c:41 > #1 0x00007ffff7f701f8 in bnet_thread_server (addr_list=<optimized out>, > max_clients=<optimized out>, client_wq=0x5555555b26e0 <dird_workq>, > handle_client_request=0x5555555789d0 <handle_connection_request(void*)>) at > bnet_server.c:177 > #2 0x000055555555e8f7 in main (argc=<optimized out>, argv=<optimized out>) > at stored.c:284 > > Thread 4 (Thread 0x7ffff6546700 (LWP 9926)): > #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffff6545e00, > expected=0, futex_word=0x7ffff7fb25a8 <_ZL5timer+40>) at > ../sysdeps/unix/sysv/linux/futex-internal.h:205 > #1 __pthread_cond_wait_common (abstime=0x7ffff6545e00, mutex=0x7ffff7fb25c0 > <_ZL11timer_mutex>, cond=0x7ffff7fb2580 <_ZL5timer>) at > pthread_cond_wait.c:539 > #2 __pthread_cond_timedwait (cond=cond@entry=0x7ffff7fb2580 <_ZL5timer>, > mutex=mutex@entry=0x7ffff7fb25c0 <_ZL11timer_mutex>, > abstime=abstime@entry=0x7ffff6545e00) at pthread_cond_wait.c:667 > #3 0x00007ffff7f9a6fc in bthread_cond_timedwait_p > (cond=cond@entry=0x7ffff7fb2580 <_ZL5timer>, m=m@entry=0x7ffff7fb25c0 > <_ZL11timer_mutex>, abstime=abstime@entry=0x7ffff6545e00, > file=file@entry=0x7ffff7fa35c2 "watchdog.c", line=line@entry=321) at > lockmgr.c:824 > #4 0x00007ffff7f942ca in watchdog_thread (arg=arg@entry=0x0) at > watchdog.c:321 > #5 0x00007ffff7f9a1fc in lmgr_thread_launcher (x=0x5555555c9ec8) at > lockmgr.c:939 > #6 0x00007ffff7d23fa3 in start_thread (arg=<optimized out>) at > pthread_create.c:486 > #7 0x00007ffff792c4cf in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 > > Thread 3 (Thread 0x7ffff6d47700 (LWP 9925)): > #0 0x00007ffff7d2e0ca in __waitpid (pid=pid@entry=9927, > stat_loc=stat_loc@entry=0x7ffff6d45bcc, options=options@entry=0) at > ../sysdeps/unix/sysv/linux/waitpid.c:30 > #1 0x00007ffff7f8b953 in signal_handler (sig=11) at signal.c:237 > #2 <signal handler called> > #3 VOLRES::set_slot (slot=-1, this=0x5555555555555555) at vol_mgr.h:88 > #4 DEVICE::clear_slot (this=0x7fffe80018e8) at dev.c:1444 > #5 m_init_dev (jcr=0x0, device=0x5555555caad8, new_init=false) at dev.c:183 > #6 0x000055555555f170 in device_initialization (arg=arg@entry=0x0) at > stored.c:565 > #7 0x00007ffff7f9a1fc in lmgr_thread_launcher (x=0x5555555d5db8) at > lockmgr.c:939 > #8 0x00007ffff7d23fa3 in start_thread (arg=<optimized out>) at > pthread_create.c:486 > #9 0x00007ffff792c4cf in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 > > Thread 2 (Thread 0x7ffff7548700 (LWP 9924)): > #0 0x00007ffff7d2dbf0 in __GI___nanosleep > (requested_time=requested_time@entry=0x7ffff7547e00, > remaining=remaining@entry=0x0) at ../sysdeps/unix/sysv/linux/nanosleep.c:28 > #1 0x00007ffff7f6d248 in bmicrosleep (sec=sec@entry=30, usec=usec@entry=0) > at bsys.c:106 > #2 0x00007ffff7f9a0a9 in check_deadlock () at lockmgr.c:574 > #3 0x00007ffff7d23fa3 in start_thread (arg=<optimized out>) at > pthread_create.c:486 > #4 0x00007ffff792c4cf in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 > > Thread 1 (Thread 0x7ffff782e740 (LWP 9923)): > #0 0x00007ffff7924037 in __GI___select (nfds=nfds@entry=7, > readfds=readfds@entry=0x7fffffffd9d0, writefds=writefds@entry=0x0, > exceptfds=exceptfds@entry=0x0, timeout=timeout@entry=0x0) at > ../sysdeps/unix/sysv/linux/select.c:41 > #1 0x00007ffff7f701f8 in bnet_thread_server (addr_list=<optimized out>, > max_clients=<optimized out>, client_wq=0x5555555b26e0 <dird_workq>, > handle_client_request=0x5555555789d0 <handle_connection_request(void*)>) at > bnet_server.c:177 > #2 0x000055555555e8f7 in main (argc=<optimized out>, argv=<optimized out>) > at stored.c:284 > #0 0x00007ffff7924037 in __GI___select (nfds=nfds@entry=7, > readfds=readfds@entry=0x7fffffffd9d0, writefds=writefds@entry=0x0, > exceptfds=exceptfds@entry=0x0, timeout=timeout@entry=0x0) at > ../sysdeps/unix/sysv/linux/select.c:41 > 41 in ../sysdeps/unix/sysv/linux/select.c > resultvar = 18446744073709551102 > sc_cancel_oldtype = 0 > sc_ret = <optimized out> > #1 0x00007ffff7f701f8 in bnet_thread_server (addr_list=<optimized out>, > max_clients=<optimized out>, client_wq=0x5555555b26e0 <dird_workq>, > handle_client_request=0x5555555789d0 <handle_connection_request(void*)>) at > bnet_server.c:177 > 177 if ((stat = select(maxfd + 1, &sockset, NULL, NULL, NULL)) < 0) { > maxfd = 6 > sockset = {fds_bits = {80, 0 <repeats 15 times>}} > newsockfd = <optimized out> > stat = <optimized out> > clilen = 1431655224 > cli_addr = {sa_family = 46489, sa_data = > "\222\001\000\000\000\000\020\332\377\377\377\177\000"} > tlog = <optimized out> > turnon = 1 > ipaddr = <optimized out> > next = <optimized out> > fd_ptr = 0x0 > buf = > "\003\000\000\000\000\000\000\000If\255d\000\000\000\000\360\344\377\367\377\177\000\000\250\332\377\377\377\177\000\000\220\341\377\367\377\177", > '\000' <repeats 18 times>, "\217\363\375\367\377\177\000\000\003", '\000' > <repeats 15 times>, "\005", '\000' <repeats 15 times>, > "\001\000\000\000\000\000\000\000\220\341\377\367\377\177\000\000\302tUUUU\000\000\000;\376\367\001\000\000" > sockfds = {<SMARTALLOC> = {<No data fields>}, head = 0x7fffffffd890, tail = > 0x7fffffffd870, loffset = 0, num_items = 2} > allbuf = "\340&[UUU\000\000)\000\000\000\000\000\000\000(n\\UUU", '\000' > <repeats 11 times>, "gT\366\377\177", '\000' <repeats 34 times>, > "\200\037\000\000\377\377\000\000", 'U' <repeats 16 times>, > "\000\000\000\000\000\000\000\000\000\377\000\000\000\000\000\000\002\000\000\000\000\000\000\000\356\000\000\000\000\000\000\200", > '\000' <repeats 16 times>, 'U' <repeats 16 times>, > "`\365\236\367\377\177\000\000\000)\237\367\377\177\000\000\000\020", '\000' > <repeats 16 times>, "\200", '\000' <repeats 13 times>... > #2 0x000055555555e8f7 in main (argc=<optimized out>, argv=<optimized out>) > at stored.c:284 > 284 bnet_thread_server(me->sdaddrs, me->max_concurrent_jobs * 2 + 1, > ch = <optimized out> > no_signals = <optimized out> > test_config = false > thid = 140737334507264 > uid = 0x0 > gid = 0x0 > #0 0x0000000000000000 in ?? () > No symbol table info available. > #0 0x0000000000000000 in ?? () > No symbol table info available. > #0 0x0000000000000000 in ?? () > No symbol table info available. > #0 0x0000000000000000 in ?? () > No symbol table info available. > #0 0x0000000000000000 in ?? () > No symbol table info available. > [Inferior 1 (process 9923) detached] > > > root@not1:/opt/bacula/working# cat not1-sd.9923.bactrace > Attempt to dump locks > threadid=0x7ffff6546700 max=0 current=-1 > threadid=0x7ffff6d47700 max=2 current=0 > lock=0x7ffff7fbf540 state=Granted priority=0 res.c:64 > threadid=0x7ffff7548700 max=0 current=-1 > threadid=0x7ffff782e740 max=0 current=-1 > > > > > _______________________________________________ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
