On 18-9-2014 10:14, [email protected] [firebird-support] wrote: > here is a stack trace from Java app server. After that you will find > back trace from some of fb_inet processes and a list of all live > processes. The mssing back traces are the same like for PID Attaching to > process 16275. > > After that is the output of fb_lock_print and then is the output from > monitoring tables for all running queries.
Well, the 'good' news is that I don't see anything strange in the Java stacktrace. All Firebird connections are waiting for a response from the server. I don't know enough about reading below The Firebird stacktraces, but it looks like apart for process 14830 that they are all waiting for a lock. > Attaching to process 14830 > > #0 0x00007fb6da7905b3 in __select_nocancel () at > ../sysdeps/unix/syscall-template.S:82 > #1 0x00007fb6db7eace1 in packet_receive (port=0x7fb6dba93080, > buffer=0x7fb6dba93222 "", buffer_length=8192, length=0x7fff4400f60e) at > ../src/remote/inet.cpp:3689 > #2 0x00007fb6db7eaac1 in inet_read (xdrs=0x7fb6dba93148) at > ../src/remote/inet.cpp:3369 > #3 0x00007fb6db7ea856 in inet_getbytes (xdrs=0x7fb6dba93148, > buff=0x7fff4400f674 "", count=0) at ../src/remote/inet.cpp:3112 > #4 0x00007fb6db7ea89f in inet_getlong (xdrs=0x1, lp=0x7fff4400f694) at > ../src/remote/inet.cpp:3134 > #5 0x00007fb6db7fff0e in xdr_enum (xdrs=0x1, ip=0x7fff4400f760) at > ../src/remote/xdr.cpp:433 > #6 0x00007fb6db7fd277 in xdr_protocol (xdrs=0x7fb6dba93148, > p=0x7fff4400f760) at ../src/remote/protocol.cpp:299 > #7 0x00007fb6db7ede1d in receive (main_port=0x7fb6dba93080, > packet=0x7fff4400f760) at ../src/remote/inet.cpp:2436 > #8 0x00007fb6db7ff90a in rem_port::receive (this=0xfffffffffffffdfe, > pckt=0x7fff4400f0c0) at ../src/remote/remote.cpp:780 > #9 0x00007fb6db80f820 in SRVR_main (main_port=0x7fb6dba93080, > flags=61632) at ../src/remote/server.cpp:290 > #10 0x00007fb6db80f2ee in server_main (argc=1, argv=0x7fff44010380) at > ../src/remote/inet_server.cpp:450 > #11 0x0000000000400709 in main (argc=1, argv=0x7fff4400f0c0) at > ../src/remote/server_stub.cpp:12 > > > Attaching to process 16275 > > #0 0x00007f1c3c7ec187 in semop () at ../sysdeps/unix/syscall-template.S:82 > #1 0x00007f1c3d678dee in ISC_event_wait (count=1, > events=0x7fffe206c690, values=0x7fffe206c69c, micro_seconds=10000000, > timeout_handler= > 0x7f1c3d83a510 <lock_alarm_handler(void*)>, > handler_arg=0x7f1c3bd19e04) at ../src/jrd/isc_sync.cpp:4583 > #2 0x00007f1c3d83bcad in wait_for_request (request=0x7f1c3b587d78, > lck_wait=1, status_vector=0x7f1c3bd30000) at ../src/lock/lock.cpp:5059 > #3 0x00007f1c3d83a28e in grant_or_que (request=0x7f1c3b587d78, > lock=0x43fd78, lck_wait=1) at ../src/lock/lock.cpp:3006 > #4 0x00007f1c3d83775a in LOCK_enq (prior_request=995655032, > parent_request=994547264, series=4, value=0x7fffe206c8f0 "r\267\b\001", > length=4, type=3 '\003', > ast_routine=0xfffffffffffffffc, ast_argument=0xfffffffffffffffc, > data=0, lck_wait=1, status_vector=0x7fffe206ca50, > owner_offset=991199232) at ../src/lock/lock.cpp:567 > #5 0x00007f1c3d773801 in LCK_lock (tdbb=0x0, lock=0x7fffe206c860, > level=3, wait=1) at ../src/jrd/lck.cpp:1084 > #6 0x00007f1c3d773900 in LCK_lock_non_blocking (tdbb=0x218005, > lock=0x7fffe206c860, level=3, wait=1) at ../src/jrd/lck.cpp:810 > #7 0x00007f1c3d7c3605 in TRA_wait (tdbb=0x7fffe206f320, > trans=0x7f1c3b98ec48, number=17348466, wait=jrd_tra::tra_wait) at > ../src/jrd/tra.cpp:2099 > #8 0x00007f1c3d7cc071 in VIO_get_current (tdbb=0x7fffe206f320, > old_rpb=0x7fffe206c3b0, rpb=0x7fffe206cce0, transaction=0x7f1c3b98ec48, > pool=0x7f1c3b98e1a0, > foreign_key=114, has_old_values=@0xfffffffffffffffc) at > ../src/jrd/vio.cpp:1941 > #9 0x00007f1c3d74a041 in check_duplicates (tdbb=0x7fffe206f320, > record=0x7f1c3b98c8c0, record_idx=0x7fffe206df70, > insertion=0x7fffe206df20, relation_2=0x0) > at ../src/jrd/jrd.h:905 > #10 0x00007f1c3d749cbe in insert_key (tdbb=0x7fffe206f320, > relation=0x7f1c3ba95b58, record=0x7f1c3b98c8c0, > transaction=0x7f1c3b98ec48, window_ptr=0x7fffe206def0, > insertion=0x7fffe206df20, bad_relation=0x7fffe206dea0, > bad_index=0xfffffffffffffffc) at ../src/jrd/idx.cpp:1650 > #11 0x00007f1c3d74907a in IDX_store (tdbb=0x7fffe206f320, > rpb=0x7f1c3b98d6a8, transaction=0x7f1c3b98ec48) at ../src/jrd/idx.cpp:1062 > #12 0x00007f1c3d73a498 in store (tdbb=0x7fffe206f320, > node=0x7f1c3b9a7328, which_trig=0) at ../src/jrd/exe.cpp:3685 > #13 0x00007f1c3d738295 in looper (tdbb=0x7fffe206f320, > request=0x7f1c3b98d278, in_node=0x7f1c3d738268) at ../src/jrd/exe.cpp:2635 > #14 0x00007f1c3d737016 in execute_looper (tdbb=0x7fffe206f320, > request=0x7f1c3b98d278, transaction=0x7f1c3b98ec48, > next_state=jrd_req::req_proceed) > at ../src/jrd/exe.cpp:1443 > #15 0x00007f1c3d7367a0 in EXE_send (tdbb=0x7fffe206f320, > request=0x7f1c3b98d278, msg=6, length=80, buffer=0x7f1c3b9a1f78 "\t") at > ../src/jrd/exe.cpp:993 > #16 0x00007f1c3d76ad04 in jrd8_start_and_send > (user_status=0x7fffe206fa40, req_handle=0x7f1c3b98d278, > tra_handle=0x7f1c3b98ec48, msg_type=0, msg_length=40, > msg=0x7f1c3b7a1f98 "", level=0) at ../src/jrd/jrd.cpp:3789 > #17 0x00007f1c3d68c153 in isc_start_and_send (user_status=0x218005, > req_handle=0x7f1c3bd3ca18, tra_handle=0x7fffe206fa40, msg_type=0, > msg_length=40, > msg=0x7f1c3b7a1f98 "", level=0) at ../src/jrd/why.cpp:450 > #18 0x00007f1c3d80414f in execute_request (request=0x7f1c3b7a2da8, > trans_handle=0x7f1c3b7a28c8, in_blr_length=20, in_blr=0x7f1c3bd3bd00 > "\005\002\004", in_msg_length=34, > in_msg=0x7f1c3bd3d4a8 "", out_blr_length=0, out_blr=0x0, > out_msg_length=0, out_msg=0x0, singleton=false) at ../src/dsql/dsql.cpp:3397 > #19 0x00007f1c3d801f7f in GDS_DSQL_EXECUTE_CPP (user_status=0x218005, > trans_handle=0x7fffe206fa3c, req_handle=0x20d8b, in_blr_length=20, > in_blr=0x7f1c3bd3bd00 "\005\002\004", in_msg_type=0, > in_msg_length=34, in_msg=0x7f1c3bd3d4a8 "", out_blr_length=0, > out_blr=0xfffffffffffffffc <Address 0xfffffffffffffffc out of > bounds>, out_msg_type=0, out_msg_length=0, > out_msg=0xfffffffffffffffc <Address 0xfffffffffffffffc out of > bounds>) at ../src/dsql/dsql.cpp:561 > #20 0x00007f1c3d685688 in isc_dsql_execute2_m (user_status=0x218005, > tra_handle=0x7fffe206fa3c, stmt_handle=0x7f1c3bd3cb30, in_blr_length=20, > in_blr=0x7f1c3bd3bd00 "\005\002\004", in_msg_type=0, > in_msg_length=34, in_msg=0x7f1c3bd3d4a8 "", out_blr_length=34, > out_blr=0x0, out_msg_type=54440, > out_msg_length=0, out_msg=0x0) at ../src/jrd/why.cpp:450 > #21 0x00007f1c3d8632b3 in rem_port::execute_statement > (this=0x7f1c3dae5080, op=op_execute, sqldata=0x218005, > sendL=0x7fffe20702f0) at ../src/remote/server.cpp:2180 > #22 0x00007f1c3d8656b4 in process_packet2 (port=0x7f1c3dae5080, > sendL=0x7fffe20702f0, receive=0x7fffe206fe90, result=0x7fffe206fe88) at > ../src/remote/server.cpp:3644 > #23 0x00007f1c3d8650d2 in process_packet (port=0x7f1c3dae5080, > sendL=0x7fffe206c3b0, receive=0x1, result=0x7fffe206fe88) at > ../src/remote/server.cpp:3393 > #24 0x00007f1c3d861841 in SRVR_main (main_port=0x7f1c3dae5080, > flags=50096) at ../src/remote/server.cpp:294 > #25 0x00007f1c3d8612ee in server_main (argc=2195461, > argv=0x7fffe2070ab0) at ../src/remote/inet_server.cpp:450 > #26 0x0000000000400709 in main (argc=2195461, argv=0x7fffe206c3b0) at > ../src/remote/server_stub.cpp:12 > > > Attaching to process 16276 > Reading symbols from /opt/firebird/bin/fb_inet_server...Reading symbols > from /opt/firebird/bin/.debug/fb_inet_server.debug...done. > done. > Reading symbols from /opt/firebird/lib/libfbembed.so.2.1...Reading > symbols from /opt/firebird/lib/.debug/libfbembed.so.2.1.6.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libfbembed.so.2.1 > Reading symbols from /lib64/libdl.so.2...Reading symbols from > /usr/lib/debug/lib64/libdl-2.12.so.debug...done. > done. > Loaded symbols for /lib64/libdl.so.2 > Reading symbols from /lib64/libncurses.so.5...(no debugging symbols > found)...done. > Loaded symbols for /lib64/libncurses.so.5 > Reading symbols from /usr/lib64/libstdc++.so.5...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib64/libstdc++.so.5 > Reading symbols from /lib64/libm.so.6...Reading symbols from > /usr/lib/debug/lib64/libm-2.12.so.debug...done. > done. > Loaded symbols for /lib64/libm.so.6 > Reading symbols from /lib64/libgcc_s.so.1...Reading symbols from > /usr/lib/debug/lib64/libgcc_s-4.4.7-20120601.so.1.debug...done. > done. > Loaded symbols for /lib64/libgcc_s.so.1 > Reading symbols from /lib64/libc.so.6...Reading symbols from > /usr/lib/debug/lib64/libc-2.12.so.debug...done. > done. > Loaded symbols for /lib64/libc.so.6 > Reading symbols from /opt/firebird/lib/libicuuc.so.30...Reading symbols > from /opt/firebird/lib/.debug/libicuuc.so.30.0.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libicuuc.so.30 > Reading symbols from /opt/firebird/lib/libicudata.so.30...Reading > symbols from /opt/firebird/lib/.debug/libicudata.so.30.0.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libicudata.so.30 > Reading symbols from /opt/firebird/lib/libicui18n.so.30...Reading > symbols from /opt/firebird/lib/.debug/libicui18n.so.30.0.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libicui18n.so.30 > Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from > /usr/lib/debug/lib64/ld-2.12.so.debug...done. > done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /lib64/libtinfo.so.5...(no debugging symbols > found)...done. > Loaded symbols for /lib64/libtinfo.so.5 > Reading symbols from /lib64/libpthread.so.0...Reading symbols from > /usr/lib/debug/lib64/libpthread-2.12.so.debug...done. > [Thread debugging using libthread_db enabled] > done. > Loaded symbols for /lib64/libpthread.so.0 > Reading symbols from /opt/firebird/intl/fbintl...Reading symbols from > /opt/firebird/intl/.debug/fbintl.debug...done. > done. > Loaded symbols for /opt/firebird/intl/fbintl > 0x00007fda6896b187 in semop () at ../sysdeps/unix/syscall-template.S:82 > 82 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) > Missing separate debuginfos, use: debuginfo-install > compat-libstdc++-33-3.2.3-69.el6.x86_64 > ncurses-libs-5.7-3.20090208.el6.x86_64 > (gdb) bt > #0 0x00007fda6896b187 in semop () at ../sysdeps/unix/syscall-template.S:82 > #1 0x00007fda697f7dee in ISC_event_wait (count=1, > events=0x7fff9e2f8800, values=0x7fff9e2f880c, micro_seconds=10000000, > timeout_handler= > 0x7fda699b9510 <lock_alarm_handler(void*)>, > handler_arg=0x7fda67e98fa4) at ../src/jrd/isc_sync.cpp:4583 > #2 0x00007fda699bacad in wait_for_request (request=0x7fda677c96c8, > lck_wait=1, status_vector=0x0) at ../src/lock/lock.cpp:5059 > #3 0x00007fda699b928e in grant_or_que (request=0x7fda677c96c8, > lock=0x5026c8, lck_wait=1) at ../src/lock/lock.cpp:3006 > #4 0x00007fda699b675a in LOCK_enq (prior_request=1736218312, > parent_request=1734313536, series=4, value=0x7fff9e2f8a60 "r\267\b\001", > length=4, type=3 '\003', > ast_routine=0xfffffffffffffffc, ast_argument=0xfffffffffffffffc, > data=0, lck_wait=1, status_vector=0x7fff9e2f8bc0, > owner_offset=1730965504) at ../src/lock/lock.cpp:567 > #5 0x00007fda698f2801 in LCK_lock (tdbb=0x0, lock=0x7fff9e2f89d0, > level=3, wait=1) at ../src/jrd/lck.cpp:1084 > #6 0x00007fda698f2900 in LCK_lock_non_blocking (tdbb=0x218005, > lock=0x7fff9e2f89d0, level=3, wait=1) at ../src/jrd/lck.cpp:810 > #7 0x00007fda69942605 in TRA_wait (tdbb=0x7fff9e2fb490, > trans=0x7fda67949258, number=17348466, wait=jrd_tra::tra_wait) at > ../src/jrd/tra.cpp:2099 > #8 0x00007fda6994b071 in VIO_get_current (tdbb=0x7fff9e2fb490, > old_rpb=0x7fff9e2f8520, rpb=0x7fff9e2f8e50, transaction=0x7fda67949258, > pool=0x7fda67ac15e8, foreign_key=114, > has_old_values=@0xfffffffffffffffc) at ../src/jrd/vio.cpp:1941 > #9 0x00007fda698c9041 in check_duplicates (tdbb=0x7fff9e2fb490, > record=0x7fda67ac1448, record_idx=0x7fff9e2fa0e0, > insertion=0x7fff9e2fa090, relation_2=0x0) > at ../src/jrd/jrd.h:905 > #10 0x00007fda698c8cbe in insert_key (tdbb=0x7fff9e2fb490, > relation=0x7fda67c14b58, record=0x7fda67ac1448, > transaction=0x7fda67949258, window_ptr=0x7fff9e2fa060, > insertion=0x7fff9e2fa090, bad_relation=0x7fff9e2fa010, > bad_index=0xfffffffffffffffc) at ../src/jrd/idx.cpp:1650 > #11 0x00007fda698c807a in IDX_store (tdbb=0x7fff9e2fb490, > rpb=0x7fda67ac32b0, transaction=0x7fda67949258) at ../src/jrd/idx.cpp:1062 > #12 0x00007fda698b9498 in store (tdbb=0x7fff9e2fb490, > node=0x7fda67944960, which_trig=0) at ../src/jrd/exe.cpp:3685 > #13 0x00007fda698b7295 in looper (tdbb=0x7fff9e2fb490, > request=0x7fda67ac2e80, in_node=0x7fda698b7268) at ../src/jrd/exe.cpp:2635 > #14 0x00007fda698b6016 in execute_looper (tdbb=0x7fff9e2fb490, > request=0x7fda67ac2e80, transaction=0x7fda67949258, > next_state=jrd_req::req_proceed) at ../src/jrd/exe.cpp:1443 > #15 0x00007fda698b57a0 in EXE_send (tdbb=0x7fff9e2fb490, > request=0x7fda67ac2e80, msg=6, length=80, buffer=0x7fda6794f180 "\t") at > ../src/jrd/exe.cpp:993 > #16 0x00007fda698e9d04 in jrd8_start_and_send > (user_status=0x7fff9e2fbbb0, req_handle=0x7fda67ac2e80, > tra_handle=0x7fda67949258, msg_type=0, msg_length=40, > msg=0x7fda66291f98 "", level=0) at ../src/jrd/jrd.cpp:3789 > #17 0x00007fda6980b153 in isc_start_and_send (user_status=0x218005, > req_handle=0x7fda67ebc5a8, tra_handle=0x7fff9e2fbbb0, msg_type=0, > msg_length=40, msg=0x7fda66291f98 "", > level=0) at ../src/jrd/why.cpp:450 > #18 0x00007fda6998314f in execute_request (request=0x7fda66292da8, > trans_handle=0x7fda662928c8, in_blr_length=20, in_blr=0x7fda67eba1e0 > "\005\002\004", in_msg_length=34, > in_msg=0x7fda67ebc0b0 "", out_blr_length=0, out_blr=0x0, > out_msg_length=0, out_msg=0x0, singleton=false) at ../src/dsql/dsql.cpp:3397 > #19 0x00007fda69980f7f in GDS_DSQL_EXECUTE_CPP (user_status=0x218005, > trans_handle=0x7fff9e2fbbac, req_handle=0x16a43, in_blr_length=20, > in_blr=0x7fda67eba1e0 "\005\002\004", in_msg_type=0, > in_msg_length=34, in_msg=0x7fda67ebc0b0 "", out_blr_length=0, > out_blr=0xfffffffffffffffc <Address 0xfffffffffffffffc out of > bounds>, out_msg_type=0, out_msg_length=0, > out_msg=0xfffffffffffffffc <Address 0xfffffffffffffffc out of > bounds>) at ../src/dsql/dsql.cpp:561 > #20 0x00007fda69804688 in isc_dsql_execute2_m (user_status=0x218005, > tra_handle=0x7fff9e2fbbac, stmt_handle=0x7fda67ebc198, in_blr_length=20, > in_blr=0x7fda67eba1e0 "\005\002\004", in_msg_type=0, > in_msg_length=34, in_msg=0x7fda67ebc0b0 "", out_blr_length=34, > out_blr=0x0, out_msg_type=49328, out_msg_length=0, > out_msg=0x0) at ../src/jrd/why.cpp:450 > #21 0x00007fda699e22b3 in rem_port::execute_statement > (this=0x7fda69c64080, op=op_execute, sqldata=0x218005, > sendL=0x7fff9e2fc460) at ../src/remote/server.cpp:2180 > #22 0x00007fda699e46b4 in process_packet2 (port=0x7fda69c64080, > sendL=0x7fff9e2fc460, receive=0x7fff9e2fc000, result=0x7fff9e2fbff8) at > ../src/remote/server.cpp:3644 > #23 0x00007fda699e40d2 in process_packet (port=0x7fda69c64080, > sendL=0x7fff9e2f8520, receive=0x1, result=0x7fff9e2fbff8) at > ../src/remote/server.cpp:3393 > #24 0x00007fda699e0841 in SRVR_main (main_port=0x7fda69c64080, > flags=34080) at ../src/remote/server.cpp:294 > #25 0x00007fda699e02ee in server_main (argc=2195461, > argv=0x7fff9e2fcc20) at ../src/remote/inet_server.cpp:450 > #26 0x0000000000400709 in main (argc=2195461, argv=0x7fff9e2f8520) at > ../src/remote/server_stub.cpp:12 > > > > Attaching to process 17159 > Reading symbols from /opt/firebird/bin/fb_inet_server...Reading symbols > from /opt/firebird/bin/.debug/fb_inet_server.debug...done. > done. > Reading symbols from /opt/firebird/lib/libfbembed.so.2.1...Reading > symbols from /opt/firebird/lib/.debug/libfbembed.so.2.1.6.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libfbembed.so.2.1 > Reading symbols from /lib64/libdl.so.2...Reading symbols from > /usr/lib/debug/lib64/libdl-2.12.so.debug...done. > done. > Loaded symbols for /lib64/libdl.so.2 > Reading symbols from /lib64/libncurses.so.5...(no debugging symbols > found)...done. > Loaded symbols for /lib64/libncurses.so.5 > Reading symbols from /usr/lib64/libstdc++.so.5...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib64/libstdc++.so.5 > Reading symbols from /lib64/libm.so.6...Reading symbols from > /usr/lib/debug/lib64/libm-2.12.so.debug...done. > done. > Loaded symbols for /lib64/libm.so.6 > Reading symbols from /lib64/libgcc_s.so.1...Reading symbols from > /usr/lib/debug/lib64/libgcc_s-4.4.7-20120601.so.1.debug...done. > done. > Loaded symbols for /lib64/libgcc_s.so.1 > Reading symbols from /lib64/libc.so.6...Reading symbols from > /usr/lib/debug/lib64/libc-2.12.so.debug...done. > done. > Loaded symbols for /lib64/libc.so.6 > Reading symbols from /opt/firebird/lib/libicuuc.so.30...Reading symbols > from /opt/firebird/lib/.debug/libicuuc.so.30.0.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libicuuc.so.30 > Reading symbols from /opt/firebird/lib/libicudata.so.30...Reading > symbols from /opt/firebird/lib/.debug/libicudata.so.30.0.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libicudata.so.30 > Reading symbols from /opt/firebird/lib/libicui18n.so.30...Reading > symbols from /opt/firebird/lib/.debug/libicui18n.so.30.0.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libicui18n.so.30 > Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from > /usr/lib/debug/lib64/ld-2.12.so.debug...done. > done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /lib64/libtinfo.so.5...(no debugging symbols > found)...done. > Loaded symbols for /lib64/libtinfo.so.5 > Reading symbols from /lib64/libpthread.so.0...Reading symbols from > /usr/lib/debug/lib64/libpthread-2.12.so.debug...done. > [Thread debugging using libthread_db enabled] > done. > Loaded symbols for /lib64/libpthread.so.0 > Reading symbols from /opt/firebird/intl/fbintl...Reading symbols from > /opt/firebird/intl/.debug/fbintl.debug...done. > done. > Loaded symbols for /opt/firebird/intl/fbintl > 0x00007f04c1018187 in semop () at ../sysdeps/unix/syscall-template.S:82 > 82 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) > Missing separate debuginfos, use: debuginfo-install > compat-libstdc++-33-3.2.3-69.el6.x86_64 > ncurses-libs-5.7-3.20090208.el6.x86_64 > (gdb) bt > #0 0x00007f04c1018187 in semop () at ../sysdeps/unix/syscall-template.S:82 > #1 0x00007f04c1ea4dee in ISC_event_wait (count=1, > events=0x7fff0754dfd0, values=0x7fff0754dfdc, micro_seconds=10000000, > timeout_handler= > 0x7f04c2066510 <lock_alarm_handler(void*)>, > handler_arg=0x7f04c054538c) at ../src/jrd/isc_sync.cpp:4583 > #2 0x00007f04c2067cad in wait_for_request (request=0x7f04bff0f2f0, > lck_wait=1, status_vector=0x0) at ../src/lock/lock.cpp:5059 > #3 0x00007f04c206628e in grant_or_que (request=0x7f04bff0f2f0, > lock=0x59b2f0, lck_wait=1) at ../src/lock/lock.cpp:3006 > #4 0x00007f04c206375a in LOCK_enq (prior_request=-1074728208, > parent_request=-1077258688, series=4, value=0x7fff0754e230 > "r\267\b\001", length=4, type=3 '\003', > ast_routine=0xfffffffffffffffc, ast_argument=0xfffffffffffffffc, > data=0, lck_wait=1, status_vector=0x7fff0754e390, > owner_offset=-1080606720) at ../src/lock/lock.cpp:567 > #5 0x00007f04c1f9f801 in LCK_lock (tdbb=0x0, lock=0x7fff0754e1a0, > level=3, wait=1) at ../src/jrd/lck.cpp:1084 > #6 0x00007f04c1f9f900 in LCK_lock_non_blocking (tdbb=0x218005, > lock=0x7fff0754e1a0, level=3, wait=1) at ../src/jrd/lck.cpp:810 > #7 0x00007f04c1fef605 in TRA_wait (tdbb=0x7fff07550c60, > trans=0x7f04bffa2978, number=17348466, wait=jrd_tra::tra_wait) at > ../src/jrd/tra.cpp:2099 > #8 0x00007f04c1ff8071 in VIO_get_current (tdbb=0x7fff07550c60, > old_rpb=0x7fff0754dcf0, rpb=0x7fff0754e620, transaction=0x7f04bffa2978, > pool=0x7f04c01f02b0, foreign_key=114, > has_old_values=@0xfffffffffffffffc) at ../src/jrd/vio.cpp:1941 > #9 0x00007f04c1f76041 in check_duplicates (tdbb=0x7fff07550c60, > record=0x7f04bfffeec8, record_idx=0x7fff0754f8b0, > insertion=0x7fff0754f860, relation_2=0x0) > at ../src/jrd/jrd.h:905 > #10 0x00007f04c1f75cbe in insert_key (tdbb=0x7fff07550c60, > relation=0x7f04c02c1b58, record=0x7f04bfffeec8, > transaction=0x7f04bffa2978, window_ptr=0x7fff0754f830, > insertion=0x7fff0754f860, bad_relation=0x7fff0754f7e0, > bad_index=0xfffffffffffffffc) at ../src/jrd/idx.cpp:1650 > #11 0x00007f04c1f7507a in IDX_store (tdbb=0x7fff07550c60, > rpb=0x7f04bfffdb38, transaction=0x7f04bffa2978) at ../src/jrd/idx.cpp:1062 > #12 0x00007f04c1f66498 in store (tdbb=0x7fff07550c60, > node=0x7f04bffab758, which_trig=0) at ../src/jrd/exe.cpp:3685 > #13 0x00007f04c1f64295 in looper (tdbb=0x7fff07550c60, > request=0x7f04bfffd708, in_node=0x7f04c1f64268) at ../src/jrd/exe.cpp:2635 > #14 0x00007f04c1f63016 in execute_looper (tdbb=0x7fff07550c60, > request=0x7f04bfffd708, transaction=0x7f04bffa2978, > next_state=jrd_req::req_proceed) at ../src/jrd/exe.cpp:1443 > #15 0x00007f04c1f627a0 in EXE_send (tdbb=0x7fff07550c60, > request=0x7f04bfffd708, msg=6, length=80, buffer=0x7f04bffa6558 "\t") at > ../src/jrd/exe.cpp:993 > #16 0x00007f04c1f96d04 in jrd8_start_and_send > (user_status=0x7fff07551380, req_handle=0x7f04bfffd708, > tra_handle=0x7f04bffa2978, msg_type=0, msg_length=40, > msg=0x7f04c007df98 "", level=0) at ../src/jrd/jrd.cpp:3789 > #17 0x00007f04c1eb8153 in isc_start_and_send (user_status=0x218005, > req_handle=0x7f04c05695a8, tra_handle=0x7fff07551380, msg_type=0, > msg_length=40, msg=0x7f04c007df98 "", > level=0) at ../src/jrd/why.cpp:450 > #18 0x00007f04c203014f in execute_request (request=0x7f04c007eda8, > trans_handle=0x7f04c007e8c8, in_blr_length=20, in_blr=0x7f04c0567aa0 > "\005\002\004", in_msg_length=34, > in_msg=0x7f04c0569368 "", out_blr_length=0, out_blr=0x0, > out_msg_length=0, out_msg=0x0, singleton=false) at ../src/dsql/dsql.cpp:3397 > #19 0x00007f04c202df7f in GDS_DSQL_EXECUTE_CPP (user_status=0x218005, > trans_handle=0x7fff0755137c, req_handle=0x6c0a, in_blr_length=20, > in_blr=0x7f04c0567aa0 "\005\002\004", > in_msg_type=0, in_msg_length=34, in_msg=0x7f04c0569368 "", > out_blr_length=0, out_blr=0xfffffffffffffffc <Address 0xfffffffffffffffc > out of bounds>, out_msg_type=0, > out_msg_length=0, out_msg=0xfffffffffffffffc <Address > 0xfffffffffffffffc out of bounds>) at ../src/dsql/dsql.cpp:561 > #20 0x00007f04c1eb1688 in isc_dsql_execute2_m (user_status=0x218005, > tra_handle=0x7fff0755137c, stmt_handle=0x7f04c05693d8, in_blr_length=20, > in_blr=0x7f04c0567aa0 "\005\002\004", in_msg_type=0, > in_msg_length=34, in_msg=0x7f04c0569368 "", out_blr_length=34, > out_blr=0x0, out_msg_type=37736, out_msg_length=0, > out_msg=0x0) at ../src/jrd/why.cpp:450 > #21 0x00007f04c208f2b3 in rem_port::execute_statement > (this=0x7f04c2311080, op=op_execute, sqldata=0x218005, > sendL=0x7fff07551c30) at ../src/remote/server.cpp:2180 > #22 0x00007f04c20916b4 in process_packet2 (port=0x7f04c2311080, > sendL=0x7fff07551c30, receive=0x7fff075517d0, result=0x7fff075517c8) at > ../src/remote/server.cpp:3644 > #23 0x00007f04c20910d2 in process_packet (port=0x7f04c2311080, > sendL=0x7fff0754dcf0, receive=0x1, result=0x7fff075517c8) at > ../src/remote/server.cpp:3393 > #24 0x00007f04c208d841 in SRVR_main (main_port=0x7f04c2311080, > flags=56560) at ../src/remote/server.cpp:294 > #25 0x00007f04c208d2ee in server_main (argc=2195461, > argv=0x7fff075523f0) at ../src/remote/inet_server.cpp:450 > #26 0x0000000000400709 in main (argc=2195461, argv=0x7fff0754dcf0) at > ../src/remote/server_stub.cpp:12 > > > > Attaching to process 17483 > Reading symbols from /opt/firebird/bin/fb_inet_server...Reading symbols > from /opt/firebird/bin/.debug/fb_inet_server.debug...done. > done. > Reading symbols from /opt/firebird/lib/libfbembed.so.2.1...Reading > symbols from /opt/firebird/lib/.debug/libfbembed.so.2.1.6.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libfbembed.so.2.1 > Reading symbols from /lib64/libdl.so.2...Reading symbols from > /usr/lib/debug/lib64/libdl-2.12.so.debug...done. > done. > Loaded symbols for /lib64/libdl.so.2 > Reading symbols from /lib64/libncurses.so.5...(no debugging symbols > found)...done. > Loaded symbols for /lib64/libncurses.so.5 > Reading symbols from /usr/lib64/libstdc++.so.5...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib64/libstdc++.so.5 > Reading symbols from /lib64/libm.so.6...Reading symbols from > /usr/lib/debug/lib64/libm-2.12.so.debug...done. > done. > Loaded symbols for /lib64/libm.so.6 > Reading symbols from /lib64/libgcc_s.so.1...Reading symbols from > /usr/lib/debug/lib64/libgcc_s-4.4.7-20120601.so.1.debug...done. > done. > Loaded symbols for /lib64/libgcc_s.so.1 > Reading symbols from /lib64/libc.so.6...Reading symbols from > /usr/lib/debug/lib64/libc-2.12.so.debug...done. > done. > Loaded symbols for /lib64/libc.so.6 > Reading symbols from /opt/firebird/lib/libicuuc.so.30...Reading symbols > from /opt/firebird/lib/.debug/libicuuc.so.30.0.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libicuuc.so.30 > Reading symbols from /opt/firebird/lib/libicudata.so.30...Reading > symbols from /opt/firebird/lib/.debug/libicudata.so.30.0.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libicudata.so.30 > Reading symbols from /opt/firebird/lib/libicui18n.so.30...Reading > symbols from /opt/firebird/lib/.debug/libicui18n.so.30.0.debug...done. > done. > Loaded symbols for /opt/firebird/lib/libicui18n.so.30 > Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from > /usr/lib/debug/lib64/ld-2.12.so.debug...done. > done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /lib64/libtinfo.so.5...(no debugging symbols > found)...done. > Loaded symbols for /lib64/libtinfo.so.5 > Reading symbols from /lib64/libpthread.so.0...Reading symbols from > /usr/lib/debug/lib64/libpthread-2.12.so.debug...done. > [Thread debugging using libthread_db enabled] > done. > Loaded symbols for /lib64/libpthread.so.0 > Reading symbols from /opt/firebird/intl/fbintl...Reading symbols from > /opt/firebird/intl/.debug/fbintl.debug...done. > done. > Loaded symbols for /opt/firebird/intl/fbintl > 0x00007f1b479a3187 in semop () at ../sysdeps/unix/syscall-template.S:82 > 82 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) > Missing separate debuginfos, use: debuginfo-install > compat-libstdc++-33-3.2.3-69.el6.x86_64 > ncurses-libs-5.7-3.20090208.el6.x86_64 > (gdb) bt > #0 0x00007f1b479a3187 in semop () at ../sysdeps/unix/syscall-template.S:82 > #1 0x00007f1b4882fdee in ISC_event_wait (count=1, > events=0x7fff989956b0, values=0x7fff989956bc, micro_seconds=10000000, > timeout_handler= > 0x7f1b489f1510 <lock_alarm_handler(void*)>, > handler_arg=0x7f1b46ed0e04) at ../src/jrd/isc_sync.cpp:4583 > #2 0x00007f1b489f2cad in wait_for_request (request=0x7f1b467223f8, > lck_wait=1, status_vector=0x0) at ../src/lock/lock.cpp:5059 > #3 0x00007f1b489f128e in grant_or_que (request=0x7f1b467223f8, > lock=0x4233f8, lck_wait=1) at ../src/lock/lock.cpp:3006 > #4 0x00007f1b489ee75a in LOCK_enq (prior_request=1181885432, > parent_request=1180894784, series=4, value=0x7fff98995910 "r\267\b\001", > length=4, type=3 '\003', > ast_routine=0xfffffffffffffffc, ast_argument=0xfffffffffffffffc, > data=0, lck_wait=1, status_vector=0x7fff98995a70, > owner_offset=1177546752) at ../src/lock/lock.cpp:567 > #5 0x00007f1b4892a801 in LCK_lock (tdbb=0x0, lock=0x7fff98995880, > level=3, wait=1) at ../src/jrd/lck.cpp:1084 > #6 0x00007f1b4892a900 in LCK_lock_non_blocking (tdbb=0x218005, > lock=0x7fff98995880, level=3, wait=1) at ../src/jrd/lck.cpp:810 > #7 0x00007f1b4897a605 in TRA_wait (tdbb=0x7fff98998340, > trans=0x7f1b46a8dcd8, number=17348466, wait=jrd_tra::tra_wait) at > ../src/jrd/tra.cpp:2099 > #8 0x00007f1b48983071 in VIO_get_current (tdbb=0x7fff98998340, > old_rpb=0x7fff989953d0, rpb=0x7fff98995d00, transaction=0x7f1b46a8dcd8, > pool=0x7f1b46a8df90, foreign_key=114, > has_old_values=@0xfffffffffffffffc) at ../src/jrd/vio.cpp:1941 > #9 0x00007f1b48901041 in check_duplicates (tdbb=0x7fff98998340, > record=0x7f1b46ad6a90, record_idx=0x7fff98996f90, > insertion=0x7fff98996f40, relation_2=0x0) > at ../src/jrd/jrd.h:905 > #10 0x00007f1b48900cbe in insert_key (tdbb=0x7fff98998340, > relation=0x7f1b46c4cb58, record=0x7f1b46ad6a90, > transaction=0x7f1b46a8dcd8, window_ptr=0x7fff98996f10, > insertion=0x7fff98996f40, bad_relation=0x7fff98996ec0, > bad_index=0xfffffffffffffffc) at ../src/jrd/idx.cpp:1650 > #11 0x00007f1b4890007a in IDX_store (tdbb=0x7fff98998340, > rpb=0x7f1b46a94998, transaction=0x7f1b46a8dcd8) at ../src/jrd/idx.cpp:1062 > #12 0x00007f1b488f1498 in store (tdbb=0x7fff98998340, > node=0x7f1b46add278, which_trig=0) at ../src/jrd/exe.cpp:3685 > #13 0x00007f1b488ef295 in looper (tdbb=0x7fff98998340, > request=0x7f1b46a94568, in_node=0x7f1b488ef268) at ../src/jrd/exe.cpp:2635 > #14 0x00007f1b488ee016 in execute_looper (tdbb=0x7fff98998340, > request=0x7f1b46a94568, transaction=0x7f1b46a8dcd8, > next_state=jrd_req::req_proceed) at ../src/jrd/exe.cpp:1443 > #15 0x00007f1b488ed7a0 in EXE_send (tdbb=0x7fff98998340, > request=0x7f1b46a94568, msg=6, length=80, buffer=0x7f1b46a8d170 "\t") at > ../src/jrd/exe.cpp:993 > #16 0x00007f1b48921d04 in jrd8_start_and_send > (user_status=0x7fff98998a60, req_handle=0x7f1b46a94568, > tra_handle=0x7f1b46a8dcd8, msg_type=0, msg_length=40, > msg=0x7f1b46928f98 "", level=0) at ../src/jrd/jrd.cpp:3789 > #17 0x00007f1b48843153 in isc_start_and_send (user_status=0x218005, > req_handle=0x7f1b46ef45a8, tra_handle=0x7fff98998a60, msg_type=0, > msg_length=40, msg=0x7f1b46928f98 "", > level=0) at ../src/jrd/why.cpp:450 > #18 0x00007f1b489bb14f in execute_request (request=0x7f1b46929da8, > trans_handle=0x7f1b469298c8, in_blr_length=20, in_blr=0x7f1b46ef2a50 > "\005\002\004", in_msg_length=34, > in_msg=0x7f1b46ef4068 "", out_blr_length=0, out_blr=0x0, > out_msg_length=0, out_msg=0x0, singleton=false) at ../src/dsql/dsql.cpp:3397 > #19 0x00007f1b489b8f7f in GDS_DSQL_EXECUTE_CPP (user_status=0x218005, > trans_handle=0x7fff98998a5c, req_handle=0xad18, in_blr_length=20, > in_blr=0x7f1b46ef2a50 "\005\002\004", > in_msg_type=0, in_msg_length=34, in_msg=0x7f1b46ef4068 "", > out_blr_length=0, out_blr=0xfffffffffffffffc <Address 0xfffffffffffffffc > out of bounds>, out_msg_type=0, > out_msg_length=0, out_msg=0xfffffffffffffffc <Address > 0xfffffffffffffffc out of bounds>) at ../src/dsql/dsql.cpp:561 > #20 0x00007f1b4883c688 in isc_dsql_execute2_m (user_status=0x218005, > tra_handle=0x7fff98998a5c, stmt_handle=0x7f1b46ef4508, in_blr_length=20, > in_blr=0x7f1b46ef2a50 "\005\002\004", in_msg_type=0, > in_msg_length=34, in_msg=0x7f1b46ef4068 "", out_blr_length=34, > out_blr=0x0, out_msg_type=16488, out_msg_length=0, > out_msg=0x0) at ../src/jrd/why.cpp:450 > #21 0x00007f1b48a1a2b3 in rem_port::execute_statement > (this=0x7f1b48c9c080, op=op_execute, sqldata=0x218005, > sendL=0x7fff98999310) at ../src/remote/server.cpp:2180 > #22 0x00007f1b48a1c6b4 in process_packet2 (port=0x7f1b48c9c080, > sendL=0x7fff98999310, receive=0x7fff98998eb0, result=0x7fff98998ea8) at > ../src/remote/server.cpp:3644 > #23 0x00007f1b48a1c0d2 in process_packet (port=0x7f1b48c9c080, > sendL=0x7fff989953d0, receive=0x1, result=0x7fff98998ea8) at > ../src/remote/server.cpp:3393 > #24 0x00007f1b48a18841 in SRVR_main (main_port=0x7f1b48c9c080, > flags=21456) at ../src/remote/server.cpp:294 > #25 0x00007f1b48a182ee in server_main (argc=2195461, > argv=0x7fff98999ad0) at ../src/remote/inet_server.cpp:450 > #26 0x0000000000400709 in main (argc=2195461, argv=0x7fff989953d0) at > ../src/remote/server_stub.cpp:12 > > > [root@provisioning ~]# ps ax |grep fb_in > 14830 ? Ss 0:46 fb_inet_server > 16275 ? Ss 0:21 fb_inet_server > 16276 ? Ss 0:17 fb_inet_server > 17159 ? Ss 0:05 fb_inet_server > 17483 ? Ss 0:07 fb_inet_server > 17484 ? Ss 0:20 fb_inet_server > 17950 ? Ss 0:01 fb_inet_server > 18564 ? Ss 0:02 fb_inet_server > 20447 pts/1 S+ 0:00 grep fb_in > [root@provisioning ~]# > > > LOCK_HEADER BLOCK > Version: 144, Active owner: 0, Length: 6343488, Used: 6320424 > Lock manager pid: 435 > Semmask: 0x2A320, Flags: 0x0001 > Enqs: 178045374, Converts: 2877880, Rejects: 2021295, Blocks: > 10077157 > Deadlock scans: 7, Deadlocks: 0, Scan interval: 10 > Acquires: 263029525, Acquire blocks: 2825140, Spin count: 0 > Mutex wait: 1.1% > Hash slots: 20011, Hash lengths (min/avg/max): 0/ 1/ 6 > Remove node: 0, Insert queue: 0, Insert prior: 0 > Owners (10): forward: 173984, backward: 202488 > Free owners (39): forward: 4557136, backward: 6320056 > Free locks (12468): forward: 654352, backward: 1970488 > Free requests (27976): forward: 5451080, backward: 4703800 > Lock Ordering: Enabled > > OWNER BLOCK 173984 > Owner id: 435, type: 1, flags: 0x04, pending: 0, > semid: 1 > Process id: 435, UID: 0x0 Alive > Flags: 0x04 > Requests: *empty* > Blocks: *empty* > > OWNER BLOCK 263312 > Owner id: 14830, type: 3, flags: 0x20, pending: 0, > semid: 9 (available) > Process id: 14830, UID: 0x54 Alive > Flags: 0x20 wake > Requests (4368): forward: 3069384, backward: 6187376 > Blocks: *empty* > > OWNER BLOCK 6319536 > Owner id: 16275, type: 3, flags: 0x02, pending: 4455800, > semid: 10 > Process id: 16275, UID: 0x54 Alive > Flags: 0x02 scan > Requests (3687): forward: 4236408, backward: 4455800 > Blocks: *empty* > > OWNER BLOCK 6319952 > Owner id: 16276, type: 3, flags: 0x02, pending: 5252808, > semid: 8 > Process id: 16276, UID: 0x54 Alive > Flags: 0x02 scan > Requests (4239): forward: 2304312, backward: 5252808 > Blocks: *empty* > > OWNER BLOCK 172856 > Owner id: 17159, type: 3, flags: 0x02, pending: 5878512, > semid: 4 > Process id: 17159, UID: 0x54 Alive > Flags: 0x02 scan > Requests (3646): forward: 5991664, backward: 5878512 > Blocks: *empty* > > OWNER BLOCK 298416 > Owner id: 17483, type: 3, flags: 0x02, pending: 4338680, > semid: 15 > Process id: 17483, UID: 0x54 Alive > Flags: 0x02 scan > Requests (4233): forward: 5393480, backward: 4338680 > Blocks: *empty* > > OWNER BLOCK 3289272 > Owner id: 17484, type: 3, flags: 0x02, pending: 5772232, > semid: 2 > Process id: 17484, UID: 0x54 Alive > Flags: 0x02 scan > Requests (3950): forward: 2876824, backward: 5772232 > Blocks: *empty* > > OWNER BLOCK 5860872 > Owner id: 17950, type: 3, flags: 0x02, pending: 5143560, > semid: 5 > Process id: 17950, UID: 0x54 Alive > Flags: 0x02 scan > Requests (4016): forward: 2903064, backward: 5143560 > Blocks: *empty* > > OWNER BLOCK 338592 > Owner id: 18564, type: 3, flags: 0x02, pending: 4223480, > semid: 25 > Process id: 18564, UID: 0x54 Alive > Flags: 0x02 scan > Requests (4390): forward: 1977400, backward: 4223480 > Blocks: *empty* > > OWNER BLOCK 202488 > Owner id: 19980, type: 3, flags: 0x00, pending: 0, > semid: 3 (available) > Process id: 19980, UID: 0x54 Alive > Flags: 0x00 > Requests (140): forward: 2053880, backward: 1340344 > Blocks: *empty* > > Event log: > DEL_OWNER: owner = 4557032, lock = 4557032, request = 0 > DEL_OWNER: owner = 217312, lock = 217312, request = 0 > DEL_OWNER: owner = 280800, lock = 280800, request = 0 > DEL_OWNER: owner = 356352, lock = 356352, request = 0 > DEL_OWNER: owner = 274872, lock = 274872, request = 0 > DEL_OWNER: owner = 310792, lock = 310792, request = 0 > DEL_OWNER: owner = 6319536, lock = 6319536, request = 0 > DEL_OWNER: owner = 6319952, lock = 6319952, request = 0 > DEL_OWNER: owner = 3289272, lock = 3289272, request = 0 > DEL_OWNER: owner = 4557136, lock = 4557136, request = 0 > DEL_OWNER: owner = 6318808, lock = 6318808, request = 0 > DEL_OWNER: owner = 234624, lock = 234624, request = 0 > DEL_OWNER: owner = 309976, lock = 309976, request = 0 > DEL_OWNER: owner = 6318704, lock = 6318704, request = 0 > DEL_OWNER: owner = 174912, lock = 174912, request = 0 > DEL_OWNER: owner = 338592, lock = 338592, request = 0 > DEL_OWNER: owner = 6319744, lock = 6319744, request = 0 > DEL_OWNER: owner = 6319848, lock = 6319848, request = 0 > DEL_OWNER: owner = 241064, lock = 241064, request = 0 > DEL_OWNER: owner = 6319640, lock = 6319640, request = 0 > DEL_OWNER: owner = 6319432, lock = 6319432, request = 0 > DEL_OWNER: owner = 172856, lock = 172856, request = 0 > DEL_OWNER: owner = 4557240, lock = 4557240, request = 0 > DEL_OWNER: owner = 6319224, lock = 6319224, request = 0 > DEL_OWNER: owner = 247056, lock = 247056, request = 0 > DEL_OWNER: owner = 344688, lock = 344688, request = 0 > DEL_OWNER: owner = 217312, lock = 217312, request = 0 > DEL_OWNER: owner = 356352, lock = 356352, request = 0 > DEL_OWNER: owner = 310792, lock = 310792, request = 0 > DEL_OWNER: owner = 6319952, lock = 6319952, request = 0 > DEL_OWNER: owner = 4557136, lock = 4557136, request = 0 > DEL_OWNER: owner = 6319328, lock = 6319328, request = 0 > DEL_OWNER: owner = 350488, lock = 350488, request = 0 > DEL_OWNER: owner = 6318912, lock = 6318912, request = 0 > DEL_OWNER: owner = 341640, lock = 341640, request = 0 > DEL_OWNER: owner = 369080, lock = 369080, request = 0 > DEL_OWNER: owner = 298416, lock = 298416, request = 0 > DEL_OWNER: owner = 263312, lock = 263312, request = 0 > DEL_OWNER: owner = 234624, lock = 234624, request = 0 > DEL_OWNER: owner = 208368, lock = 208368, request = 0 > DEL_OWNER: owner = 5860872, lock = 5860872, request = 0 > DEL_OWNER: owner = 311160, lock = 311160, request = 0 > DEL_OWNER: owner = 292424, lock = 292424, request = 0 > DEL_OWNER: owner = 6318704, lock = 6318704, request = 0 > DEL_OWNER: owner = 338592, lock = 338592, request = 0 > DEL_OWNER: owner = 6319848, lock = 6319848, request = 0 > DEL_OWNER: owner = 6319640, lock = 6319640, request = 0 > DEL_OWNER: owner = 172856, lock = 172856, request = 0 > DEL_OWNER: owner = 6319224, lock = 6319224, request = 0 > DEL_OWNER: owner = 344688, lock = 344688, request = 0 > DEL_OWNER: owner = 356352, lock = 356352, request = 0 > DEL_OWNER: owner = 6319952, lock = 6319952, request = 0 > DEL_OWNER: owner = 6319328, lock = 6319328, request = 0 > DEL_OWNER: owner = 218512, lock = 218512, request = 0 > DEL_OWNER: owner = 326048, lock = 326048, request = 0 > DEL_OWNER: owner = 211160, lock = 211160, request = 0 > DEL_OWNER: owner = 202488, lock = 202488, request = 0 > DEL_OWNER: owner = 4557032, lock = 4557032, request = 0 > DEL_OWNER: owner = 6318912, lock = 6318912, request = 0 > DEL_OWNER: owner = 369080, lock = 369080, request = 0 > DEL_OWNER: owner = 263312, lock = 263312, request = 0 > DEL_OWNER: owner = 208368, lock = 208368, request = 0 > DEL_OWNER: owner = 311160, lock = 311160, request = 0 > DEL_OWNER: owner = 6319016, lock = 6319016, request = 0 > DEL_OWNER: owner = 6318600, lock = 6318600, request = 0 > DEL_OWNER: owner = 274872, lock = 274872, request = 0 > DEL_OWNER: owner = 280800, lock = 280800, request = 0 > DEL_OWNER: owner = 3289272, lock = 3289272, request = 0 > DEL_OWNER: owner = 6318704, lock = 6318704, request = 0 > DEL_OWNER: owner = 6319848, lock = 6319848, request = 0 > DEL_OWNER: owner = 172856, lock = 172856, request = 0 > DEL_OWNER: owner = 344688, lock = 344688, request = 0 > DEL_OWNER: owner = 6319952, lock = 6319952, request = 0 > DEL_OWNER: owner = 218512, lock = 218512, request = 0 > DEL_OWNER: owner = 326048, lock = 326048, request = 0 > DEL_OWNER: owner = 211160, lock = 211160, request = 0 > DEL_OWNER: owner = 174912, lock = 174912, request = 0 > DEL_OWNER: owner = 362280, lock = 362280, request = 0 > DEL_OWNER: owner = 6319744, lock = 6319744, request = 0 > DEL_OWNER: owner = 6319120, lock = 6319120, request = 0 > DEL_OWNER: owner = 6319432, lock = 6319432, request = 0 > DEL_OWNER: owner = 4557240, lock = 4557240, request = 0 > DEL_OWNER: owner = 247056, lock = 247056, request = 0 > DEL_OWNER: owner = 217312, lock = 217312, request = 0 > DEL_OWNER: owner = 310792, lock = 310792, request = 0 > DEL_OWNER: owner = 6318808, lock = 6318808, request = 0 > DEL_OWNER: owner = 4557136, lock = 4557136, request = 0 > DEL_OWNER: owner = 350488, lock = 350488, request = 0 > DEL_OWNER: owner = 4557032, lock = 4557032, request = 0 > DEL_OWNER: owner = 369080, lock = 369080, request = 0 > DEL_OWNER: owner = 208368, lock = 208368, request = 0 > DEL_OWNER: owner = 6319016, lock = 6319016, request = 0 > DEL_OWNER: owner = 274872, lock = 274872, request = 0 > DEL_OWNER: owner = 3289272, lock = 3289272, request = 0 > DEL_OWNER: owner = 6319848, lock = 6319848, request = 0 > DEL_OWNER: owner = 344688, lock = 344688, request = 0 > DEL_OWNER: owner = 218512, lock = 218512, request = 0 > DEL_OWNER: owner = 211160, lock = 211160, request = 0 > DEL_OWNER: owner = 362280, lock = 362280, request = 0 > DEL_OWNER: owner = 6319120, lock = 6319120, request = 0 > DEL_OWNER: owner = 202488, lock = 202488, request = 0 > DEL_OWNER: owner = 341640, lock = 341640, request = 0 > DEL_OWNER: owner = 309976, lock = 309976, request = 0 > DEL_OWNER: owner = 6319536, lock = 6319536, request = 0 > DEL_OWNER: owner = 6320056, lock = 6320056, request = 0 > DEL_OWNER: owner = 241064, lock = 241064, request = 0 > DEL_OWNER: owner = 5860872, lock = 5860872, request = 0 > DEL_OWNER: owner = 4557240, lock = 4557240, request = 0 > DEL_OWNER: owner = 217312, lock = 217312, request = 0 > DEL_OWNER: owner = 6318808, lock = 6318808, request = 0 > DEL_OWNER: owner = 350488, lock = 350488, request = 0 > DEL_OWNER: owner = 369080, lock = 369080, request = 0 > DEL_OWNER: owner = 6319016, lock = 6319016, request = 0 > DEL_OWNER: owner = 338592, lock = 338592, request = 0 > DEL_OWNER: owner = 298416, lock = 298416, request = 0 > DEL_OWNER: owner = 356352, lock = 356352, request = 0 > DEL_OWNER: owner = 6319328, lock = 6319328, request = 0 > DEL_OWNER: owner = 292424, lock = 292424, request = 0 > DEL_OWNER: owner = 3289272, lock = 3289272, request = 0 > DEL_OWNER: owner = 344688, lock = 344688, request = 0 > DEL_OWNER: owner = 211160, lock = 211160, request = 0 > DEL_OWNER: owner = 6319120, lock = 6319120, request = 0 > DEL_OWNER: owner = 341640, lock = 341640, request = 0 > DEL_OWNER: owner = 6318912, lock = 6318912, request = 0 > DEL_OWNER: owner = 6319640, lock = 6319640, request = 0 > DEL_OWNER: owner = 6318600, lock = 6318600, request = 0 > DEL_OWNER: owner = 280800, lock = 280800, request = 0 > DEL_OWNER: owner = 6318704, lock = 6318704, request = 0 > DEL_OWNER: owner = 6319536, lock = 6319536, request = 0 > DEL_OWNER: owner = 172856, lock = 172856, request = 0 > DEL_OWNER: owner = 6319952, lock = 6319952, request = 0 > DEL_OWNER: owner = 326048, lock = 326048, request = 0 > DEL_OWNER: owner = 174912, lock = 174912, request = 0 > DEL_OWNER: owner = 6319432, lock = 6319432, request = 0 > DEL_OWNER: owner = 241064, lock = 241064, request = 0 > DEL_OWNER: owner = 4557240, lock = 4557240, request = 0 > DEL_OWNER: owner = 6318808, lock = 6318808, request = 0 > DEL_OWNER: owner = 369080, lock = 369080, request = 0 > DEL_OWNER: owner = 338592, lock = 338592, request = 0 > DEL_OWNER: owner = 356352, lock = 356352, request = 0 > DEL_OWNER: owner = 292424, lock = 292424, request = 0 > DEL_OWNER: owner = 344688, lock = 344688, request = 0 > DEL_OWNER: owner = 6319120, lock = 6319120, request = 0 > DEL_OWNER: owner = 247056, lock = 247056, request = 0 > DEL_OWNER: owner = 310792, lock = 310792, request = 0 > DEL_OWNER: owner = 311160, lock = 311160, request = 0 > DEL_OWNER: owner = 4557136, lock = 4557136, request = 0 > DEL_OWNER: owner = 234624, lock = 234624, request = 0 > DEL_OWNER: owner = 208368, lock = 208368, request = 0 > DEL_OWNER: owner = 6319744, lock = 6319744, request = 0 > DEL_OWNER: owner = 274872, lock = 274872, request = 0 > DEL_OWNER: owner = 6318912, lock = 6318912, request = 0 > DEL_OWNER: owner = 6318600, lock = 6318600, request = 0 > DEL_OWNER: owner = 6318704, lock = 6318704, request = 0 > DEL_OWNER: owner = 172856, lock = 172856, request = 0 > DEL_OWNER: owner = 326048, lock = 326048, request = 0 > DEL_OWNER: owner = 6319432, lock = 6319432, request = 0 > DEL_OWNER: owner = 4557240, lock = 4557240, request = 0 > DEL_OWNER: owner = 369080, lock = 369080, request = 0 > DEL_OWNER: owner = 6319848, lock = 6319848, request = 0 > DEL_OWNER: owner = 6320056, lock = 6320056, request = 0 > DEL_OWNER: owner = 4557032, lock = 4557032, request = 0 > DEL_OWNER: owner = 202488, lock = 202488, request = 0 > DEL_OWNER: owner = 356352, lock = 356352, request = 0 > DEL_OWNER: owner = 344688, lock = 344688, request = 0 > DEL_OWNER: owner = 247056, lock = 247056, request = 0 > DEL_OWNER: owner = 311160, lock = 311160, request = 0 > DEL_OWNER: owner = 5860872, lock = 5860872, request = 0 > DEL_OWNER: owner = 362280, lock = 362280, request = 0 > DEL_OWNER: owner = 218512, lock = 218512, request = 0 > DEL_OWNER: owner = 6319224, lock = 6319224, request = 0 > DEL_OWNER: owner = 350488, lock = 350488, request = 0 > DEL_OWNER: owner = 6319016, lock = 6319016, request = 0 > DEL_OWNER: owner = 234624, lock = 234624, request = 0 > DEL_OWNER: owner = 298416, lock = 298416, request = 0 > DEL_OWNER: owner = 6319328, lock = 6319328, request = 0 > DEL_OWNER: owner = 3289272, lock = 3289272, request = 0 > DEL_OWNER: owner = 309976, lock = 309976, request = 0 > DEL_OWNER: owner = 341640, lock = 341640, request = 0 > DEL_OWNER: owner = 6319744, lock = 6319744, request = 0 > DEL_OWNER: owner = 6318912, lock = 6318912, request = 0 > DEL_OWNER: owner = 6318704, lock = 6318704, request = 0 > DEL_OWNER: owner = 326048, lock = 326048, request = 0 > DEL_OWNER: owner = 4557240, lock = 4557240, request = 0 > DEL_OWNER: owner = 6319848, lock = 6319848, request = 0 > DEL_OWNER: owner = 4557032, lock = 4557032, request = 0 > DEL_OWNER: owner = 356352, lock = 356352, request = 0 > DEL_OWNER: owner = 247056, lock = 247056, request = 0 > DEL_OWNER: owner = 5860872, lock = 5860872, request = 0 > DEL_OWNER: owner = 280800, lock = 280800, request = 0 > DEL_OWNER: owner = 6319640, lock = 6319640, request = 0 > DEL_OWNER: owner = 241064, lock = 241064, request = 0 > DEL_OWNER: owner = 217312, lock = 217312, request = 0 > DEL_OWNER: owner = 211160, lock = 211160, request = 0 > DEL_OWNER: owner = 338592, lock = 338592, request = 0 > DEL_OWNER: owner = 218512, lock = 218512, request = 0 > DEL_OWNER: owner = 350488, lock = 350488, request = 0 > DEL_OWNER: owner = 234624, lock = 234624, request = 0 > DEL_OWNER: owner = 6319328, lock = 6319328, request = 0 > DEL_OWNER: owner = 309976, lock = 309976, request = 0 > DEL_OWNER: owner = 6319744, lock = 6319744, request = 0 > DEL_OWNER: owner = 6318704, lock = 6318704, request = 0 > DEL_OWNER: owner = 326048, lock = 326048, request = 0 > DEL_OWNER: owner = 4557240, lock = 4557240, request = 0 > DEL_OWNER: owner = 310792, lock = 310792, request = 0 > DEL_OWNER: owner = 174912, lock = 174912, request = 0 > DEL_OWNER: owner = 6318808, lock = 6318808, request = 0 > DEL_OWNER: owner = 4557032, lock = 4557032, request = 0 > DEL_OWNER: owner = 247056, lock = 247056, request = 0 > DEL_OWNER: owner = 280800, lock = 280800, request = 0 > DEL_OWNER: owner = 6319848, lock = 6319848, request = 0 > DEL_OWNER: owner = 241064, lock = 241064, request = 0 > DEL_OWNER: owner = 211160, lock = 211160, request = 0 > DEL_OWNER: owner = 6319432, lock = 6319432, request = 0 > DEL_OWNER: owner = 369080, lock = 369080, request = 0 > DEL_OWNER: owner = 6320056, lock = 6320056, request = 0 > DEL_OWNER: owner = 202488, lock = 202488, request = 0 > DEL_OWNER: owner = 4557136, lock = 4557136, request = 0 > DEL_OWNER: owner = 311160, lock = 311160, request = 0 > DEL_OWNER: owner = 362280, lock = 362280, request = 0 > DEL_OWNER: owner = 218512, lock = 218512, request = 0 > DEL_OWNER: owner = 234624, lock = 234624, request = 0 > DEL_OWNER: owner = 309976, lock = 309976, request = 0 > DEL_OWNER: owner = 6318704, lock = 6318704, request = 0 > DEL_OWNER: owner = 4557240, lock = 4557240, request = 0 > DEL_OWNER: owner = 174912, lock = 174912, request = 0 > DEL_OWNER: owner = 4557032, lock = 4557032, request = 0 > DEL_OWNER: owner = 280800, lock = 280800, request = 0 > DEL_OWNER: owner = 344688, lock = 344688, request = 0 > DEL_OWNER: owner = 341640, lock = 341640, request = 0 > DEL_OWNER: owner = 241064, lock = 241064, request = 0 > DEL_OWNER: owner = 6319432, lock = 6319432, request = 0 > DEL_OWNER: owner = 6319848, lock = 6319848, request = 0 > DEL_OWNER: owner = 274872, lock = 274872, request = 0 > DEL_OWNER: owner = 217312, lock = 217312, request = 0 > DEL_OWNER: owner = 356352, lock = 356352, request = 0 > DEL_OWNER: owner = 6318600, lock = 6318600, request = 0 > DEL_OWNER: owner = 247056, lock = 247056, request = 0 > DEL_OWNER: owner = 377088, lock = 377088, request = 0 > DEL_OWNER: owner = 368272, lock = 368272, request = 0 > DEL_OWNER: owner = 350488, lock = 350488, request = 0 > DEL_OWNER: owner = 6319328, lock = 6319328, request = 0 > DEL_OWNER: owner = 6319744, lock = 6319744, request = 0 > DEL_OWNER: owner = 326048, lock = 326048, request = 0 > DEL_OWNER: owner = 211160, lock = 211160, request = 0 > DEL_OWNER: owner = 6319016, lock = 6319016, request = 0 > DEL_OWNER: owner = 6319120, lock = 6319120, request = 0 > DEL_OWNER: owner = 6319224, lock = 6319224, request = 0 > DEL_OWNER: owner = 208368, lock = 208368, request = 0 > DEL_OWNER: owner = 292424, lock = 292424, request = 0 > DEL_OWNER: owner = 6318912, lock = 6318912, request = 0 > DEL_OWNER: owner = 6319640, lock = 6319640, request = 0 > DEL_OWNER: owner = 6318808, lock = 6318808, request = 0 > DEL_OWNER: owner = 310792, lock = 310792, request = 0 > DEL_OWNER: owner = 369080, lock = 369080, request = 0 > DEL_OWNER: owner = 6320056, lock = 6320056, request = 0 > > > MON$ATTACHMENT_ID MON$SERVER_PID MON$STATE MON$TIMESTAMP > MON$STAT_ID MON$TIMESTAMP_1 MON$SQL_TEXT > ---------------------------------------------------------------------------------------------------------- > 14786 16275 1 18.9.2014 г. 07:35:39 > 113 18.9.2014 г. 08:05:00 INSERT INTO CASH_CASHBOX_DAY_AMMOUNT > (CASHBOXID, DATE_TIME, AMMOUNT) VALUES (?, ?, ?) > 14798 17483 1 18.9.2014 г. 07:58:09 > 71 18.9.2014 г. 08:05:31 INSERT INTO CASH_CASHBOX_DAY_AMMOUNT > (CASHBOXID, DATE_TIME, AMMOUNT) VALUES (?, ?, ?) > 14804 17950 1 18.9.2014 г. 08:05:40 > 29 18.9.2014 г. 08:09:02 INSERT INTO CASH_CASHBOX_DAY_AMMOUNT > (CASHBOXID, DATE_TIME, AMMOUNT) VALUES (?, ?, ?) > 14796 17159 1 18.9.2014 г. 07:50:39 > 57 18.9.2014 г. 08:12:25 INSERT INTO CASH_CASHBOX_DAY_AMMOUNT > (CASHBOXID, DATE_TIME, AMMOUNT) VALUES (?, ?, ?) > 14799 17484 1 18.9.2014 г. 07:58:09 > 85 18.9.2014 г. 08:19:17 INSERT INTO CASH_CASHBOX_DAY_AMMOUNT > (CASHBOXID, DATE_TIME, AMMOUNT) VALUES (?, ?, ?) > 14787 16276 1 18.9.2014 г. 07:35:39 > 99 18.9.2014 г. 08:20:34 INSERT INTO CASH_CASHBOX_DAY_AMMOUNT > (CASHBOXID, DATE_TIME, AMMOUNT) VALUES (?, ?, ?) > 14807 18564 1 18.9.2014 г. 08:12:29 > 43 18.9.2014 г. 08:21:43 INSERT INTO CASH_CASHBOX_DAY_AMMOUNT > (CASHBOXID, DATE_TIME, AMMOUNT) VALUES (?, ?, ?) > 14814 20475 1 18.9.2014 г. 08:37:38 > 2 18.9.2014 г. 08:37:39 select att.MON$ATTACHMENT_ID, > att.MON$SERVER_PID, att.MON$STATE, att.MON$TIMESTAMP, att.MON$STAT_ID, > st.MON$TIMESTAMP, MON$SQL_TEXT > from MON$ATTACHMENTS att > join mon$STATEMENTS st on st.MON$ATTACHMENT_ID = att.MON$ATTACHMENT_ID > where mon$SQL_TEXT is not null > order by st.MON$TIMESTAMP > > I hope this can give you an idea what is happening here. The problem now > shows every 2-3 days on only one installation from 20. > > I'll try to upgrade installation to the most recent kernel available for > CentOS 6 later today. -- Mark Rotteveel
