tsearle created an issue (kamailio/kamailio#4414)
### Description
A crash randomly occured in our production environment
the crash back trace is similar to the one described in #3878
#### Debugging Data
bt full output
```
GNU gdb (Debian 13.1-3) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/kamailio...
Reading symbols from
/usr/lib/debug/.build-id/24/bb830d80441ecf857819d5cdaac404e1ff987b.debug...
[New LWP 1583]
[New LWP 1618]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f
/etc/kamailio/kamailio.cfg'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 link_profile (linker=linker@entry=0xffff68d91dc0,
vkey=vkey@entry=0xffff69019458) at ./src/modules/dialog/dlg_profile.c:492
[Current thread is 1 (Thread 0xffff8c6d9020 (LWP 1583))]
(gdb) #0 link_profile (linker=linker@entry=0xffff68d91dc0,
vkey=vkey@entry=0xffff69019458) at ./src/modules/dialog/dlg_profile.c:492
hash = <optimized out>
p_entry = 0xffff68c1ae90
#1 0x0000ffff88b089f4 in link_dlg_profile (linker=linker@entry=0xffff68d91dc0,
dlg=dlg@entry=0xffff690193e0) at ./src/modules/dialog/dlg_profile.c:529
d_entry = <optimized out>
#2 0x0000ffff88b0a8e4 in set_dlg_profile (msg=0xffff8b661750, value=<optimized
out>, profile=<optimized out>) at ./src/modules/dialog/dlg_profile.c:620
dlg = 0xffff690193e0
linker = 0xffff68d91dc0
__func__ = "set_dlg_profile"
#3 0x0000ffff88ab88dc in w_set_dlg_profile_helper (msg=<optimized out>,
profile=<optimized out>, value=<optimized out>) at
./src/modules/dialog/dialog.c:899
__func__ = "w_set_dlg_profile_helper"
#4 0x0000aaaac1bfca68 in sr_kemi_exec_func (ket=ket@entry=0xffff88b60800
<sr_kemi_dialog_exports+936>, msg=msg@entry=0xffff8b661750, pno=pno@entry=2,
vps=vps@entry=0xfffff7b61488)
at core/kemiexec.c:102
ret = <optimized out>
__func__ = "sr_kemi_exec_func"
#5 0x0000ffff89591d40 in sr_apy_kemi_exec_func_ex (ket=0xffff88b60800
<sr_kemi_dialog_exports+936>, self=<optimized out>, args=0xffff6661c140,
idx=<optimized out>)
at ./src/modules/app_python3/apy_kemi.c:329
fname = {s = 0xffff88b300e0 "set_dlg_profile", len = 15}
i = <optimized out>
ret = <optimized out>
vps = {{vtype = 2, v = {n = 1749996576, l = 281472431739936, s = {s =
0xffff684ed420 "ingress_ip", len = 10}, dict = 0xffff684ed420}}, {vtype = 2, v
= {n = 1749584160,
l = 281472431327520, s = {s = 0xffff68488920 "23.94.26.58", len =
11}, dict = 0xffff68488920}}, {vtype = 0, v = {n = 0, l = 0, s = {s = 0x0, len
= 0}, dict = 0x0}}, {vtype = 0,
v = {n = 0, l = 0, s = {s = 0x0, len = 0}, dict = 0x0}}, {vtype =
0, v = {n = 0, l = 0, s = {s = 0x0, len = 0}, dict = 0x0}}, {vtype = 0, v = {n
= 0, l = 0, s = {s = 0x0,
len = 0}, dict = 0x0}}}
env_P = <optimized out>
lmsg = 0xffff8b661750
xret = <optimized out>
slen = 11
alen = 2
pobj = <optimized out>
__func__ = "sr_apy_kemi_exec_func_ex"
#6 0x0000ffff8959370c in sr_apy_kemi_exec_func (self=0xffff68615440,
args=0xffff6661c140, idx=659) at ./src/modules/app_python3/apy_kemi.c:358
ket = 0xffff88b60800 <sr_kemi_dialog_exports+936>
ret = 0x0
pstate = 0x0
pframe = 0x0
pcode = 0x0
tvb = {tv_sec = 0, tv_usec = 0}
tve = {tv_sec = 0, tv_usec = 0}
tz = {tz_minuteswest = 10, tz_dsttime = 0}
tdiff = <optimized out>
__func__ = "sr_apy_kemi_exec_func"
#7 0x0000ffff88fa5d88 in ?? () from /lib/aarch64-linux-gnu/libpython3.11.so.1.0
No symbol table info available.
#8 0x0000ffff88f571d8 in _PyObject_MakeTpCall () from
/lib/aarch64-linux-gnu/libpython3.11.so.1.0
No symbol table info available.
#9 0x0000ffff88eed03c in _PyEval_EvalFrameDefault () from
/lib/aarch64-linux-gnu/libpython3.11.so.1.0
No symbol table info available.
#10 0x0000ffff890440fc in ?? () from /lib/aarch64-linux-gnu/libpython3.11.so.1.0
No symbol table info available.
#11 0x0000ffff88f5a728 in ?? () from /lib/aarch64-linux-gnu/libpython3.11.so.1.0
No symbol table info available.
#12 0x0000ffff89598c44 in apy_exec (_msg=<optimized out>, fname=0xaaaac1eb8b20
"ksr_request_route", fparam=fparam@entry=0x0, emode=emode@entry=1)
at ./src/modules/app_python3/python_exec.c:157
pFunc = 0xffff67b824c0
pArgs = 0xffff687ad4e0
pValue = <optimized out>
pResult = <optimized out>
pmsg = 0xffff668158f0
rval = -1
bmsg = 0x0
gstate = PyGILState_LOCKED
locked = 1
__func__ = "apy_exec"
#13 0x0000ffff89590c98 in sr_kemi_config_engine_python (msg=<optimized out>,
rtype=1, rname=0x0, rparam=<optimized out>) at
./src/modules/app_python3/apy_kemi.c:68
ret = -1
__func__ = "sr_kemi_config_engine_python"
#14 0x0000aaaac1bfaae8 in sr_kemi_route (keng=0xaaaac1faf8e0
<_sr_kemi_eng_list>, msg=msg@entry=0xffff8b661750, rtype=rtype@entry=1,
ename=ename@entry=0x0, edata=edata@entry=0x0)
at core/kemi.c:3853
sfbk = 0
ret = <optimized out>
#15 0x0000aaaac1c704f4 in receive_msg (
buf=buf@entry=0xaaaac2064550 <buf> "INVITE
sip:[email protected]:5080;transport=UDP SIP/2.0\r\nRecord-Route:
<sip:10.153.16.54:5080;lr=on;stamp=9616eded056fc0110b8fa0da983b32f9825820fdc153661bbfb0d1707e2ee863>\r\nVia:
SIP/2.0/UDP 10."..., len=<optimized out>,
rcv_info=rcv_info@entry=0xfffff7b61ff8) at core/receive.c:507
msg = 0xffff8b661750
ctx = {rec_lev = 0, run_flags = 0, last_retcode = 0, jmp_env =
{{__jmpbuf = {544, 4294967295, 281473035184984, 281473037789088, 16777216,
14728432, 1586192, 2048784, 281474837652176,
281473037621132, 281473020408336, 281474837651448,
281474837651296, 187650375533040, 187650374803456, 187650376419056,
187650375533040, 197, 187650374661248, 281473035184984,
281473036099216, 14021539797695420928}, __mask_was_saved =
-139059536, __saved_mask = {__val = {187650373409376, 281473020408512,
281473019950128, 128, 128, 281473020408336,
281474837651292, 65535, 0, 13, 187650376353104,
281474837652176, 187650373237272, 281473020408336, 281474837651448,
281474837651056}}}}}
bctx = 0x0
ret = <optimized out>
tvb = {tv_sec = 0, tv_usec = 0}
tve = {tv_sec = 0, tv_usec = 0}
diff = 0
inb = {
s = 0xaaaac2064550 <buf> "INVITE
sip:[email protected]:5080;transport=UDP SIP/2.0\r\nRecord-Route:
<sip:10.153.16.54:5080;lr=on;stamp=9616eded056fc0110b8fa0da983b32f9825820fdc153661bbfb0d1707e2ee863>\r\nVia:
SIP/2.0/UDP 10."..., len = 1070}
netinfo = {data = {s = 0x0, len = 0}, bufsize = 0, rcv = 0x0, dst = 0x0}
keng = <optimized out>
evp = {data = 0xfffff7b61c48, obuf = {s = 0x0, len = 0}, rcv =
0xfffff7b61ff8, dst = 0x0, req = 0x0, rpl = 0x0, rplcode = 0, mode = 0}
cidlockidx = 0
cidlockset = 0
errsipmsg = 0
exectime = 0
__func__ = "receive_msg"
#16 0x0000aaaac1d6aec8 in udp_rcv_loop () at core/udp_server.c:770
len = 1070
buf = "INVITE sip:[email protected]:5080;transport=UDP
SIP/2.0\r\nRecord-Route:
<sip:10.153.16.54:5080;lr=on;stamp=9616eded056fc0110b8fa0da983b32f9825820fdc153661bbfb0d1707e2ee863>\r\nVia:
SIP/2.0/UDP 10."...
tmp = <optimized out>
fromaddr = 0xffff8b653210
fromaddrlen = 16
rcvi = {src_ip = {af = 2, len = 4, u = {addrl = {907057418, 0}, addr32
= {907057418, 0, 0, 0}, addr16 = {39178, 13840, 0, 0, 0, 0, 0, 0},
addr = "\n\231\0206", '\000' <repeats 11 times>}}, dst_ip = {af =
2, len = 4, u = {addrl = {3910179082, 0}, addr32 = {3910179082, 0, 0, 0},
addr16 = {39178, 59664, 0, 0, 0, 0,
0, 0}, addr = "\n\231\020\351", '\000' <repeats 11 times>}},
src_port = 5080, dst_port = 5060, proto_reserved1 = 0, proto_reserved2 = 0,
src_su = {s = {sa_family = 2,
sa_data = "\023\330\n\231\0206\000\000\000\000\000\000\000"}, sin
= {sin_family = 2, sin_port = 55315, sin_addr = {s_addr = 907057418},
sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family =
2, sin6_port = 55315, sin6_flowinfo = 907057418, sin6_addr = {__in6_u = {
__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0,
0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}, sas =
{ss_family = 2,
__ss_padding = "\023\330\n\231\0206", '\000' <repeats 111 times>,
__ss_align = 0}}, bind_address = 0xffff8b5e8ad0, rflags = 0, proto = 1 '\001',
proto_pad0 = 0 '\000',
proto_pad1 = 0}
evp = {data = 0x0, obuf = {s = 0x0, len = 0}, rcv = 0x0, dst = 0x0, req
= 0x0, rpl = 0x0, rplcode = 0, mode = 0}
printbuf =
"\\\375\266\301\252\252\000\000\000\000\000\000\000\000\000\000\360\301\371\301\252\252\000\000P\271\374\301\252\252\000\000\360\301\371\301\252\252\000\000\000\000\000\000\000\000\000\000\350#\266\367\377\377\000\000\340\"\266\367\377\377\000\000d\376\266\301\252\252\000\000\360\301\371\301\252\252\000\000\000Z\243\035(\204\226°!\266\367\377\377\000\000\370\276\316\301\252\252\000\000\260!\266\367\377\377\000\000t3\341\301\252\252\000\0008$\352\301\252\252\000\000\024",
'\000' <repeats 16 times>,
"Z\243\035(\204\226\302\300!\266\367\377\377\000\000DF\341\301\252\252\000\000\004\000\000\000\000\000\000\000\340!\266\367\377\377\000\000l,[\201\000\000\000\000\001\000\000\000\000\000\000\000\001"...
i = <optimized out>
j = <optimized out>
l = <optimized out>
__func__ = "udp_rcv_loop"
__llevel = <optimized out>
__kld = <optimized out>
#17 0x0000aaaac1b5be18 in main_loop () at ./src/main.c:1895
i = <optimized out>
pid = <optimized out>
si = <optimized out>
sx = <optimized out>
si_desc = "udp receiver child=0
sock=10.153.16.233:5060\000\000\000\000\260\037\371\301\252\252", '\000'
<repeats 18 times>,
"ansaction=>size=\000\000\000\000\000\000\000\000\000\377\000\000\000\000\000\377\000\000\000\000\000\000\000\000\000\377\000\000\000\000\000\37733333333"
nrprocs = <optimized out>
woneinit = 0
agfound = <optimized out>
__func__ = "main_loop"
error = <optimized out>
#18 0x0000aaaac1b4dffc in main (argc=<optimized out>, argv=<optimized out>) at
./src/main.c:3406
cfg_stream = <optimized out>
c = <optimized out>
r = <optimized out>
tmp = 0xfffff7b62e5d ""
tmp_len = 65535
port = 5060
proto = 0
aproto = 0
ahost = 0x0
socket_name = 0x0
aport = 0
listen_field_count = <optimized out>
listen_fields = {0x0, 0xffff8c6ed028 <_rtld_global>
"\200\343n\214\377\377", 0xaaaac1f8e6e8 "\200\037\265\301\252\252"}
options = 0xaaaac1e9ec00
":f:cm:M:dVIhEeb:B:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
ret = -1
seed = 2178755526
rfd = <optimized out>
debug_save = <optimized out>
debug_flag = <optimized out>
dont_fork_cnt = <optimized out>
n_lst = <optimized out>
p = <optimized out>
tbuf = 0x0
tbuf_tmp = <optimized out>
st = {st_dev = 24, st_ino = 992, st_mode = 16888, st_nlink = 2, st_uid
= 107, st_gid = 112, st_rdev = 0, __pad1 = 0, st_size = 40, st_blksize = 4096,
__pad2 = 0, st_blocks = 0,
st_atim = {tv_sec = 1757332083, tv_nsec = 450116302}, st_mtim =
{tv_sec = 1757332083, tv_nsec = 450116302}, st_ctim = {tv_sec = 1757332083,
tv_nsec = 450116302},
__glibc_reserved = {0, 0}}
l1 = <optimized out>
lim = {rlim_cur = 1024, rlim_max = 524288}
option_index = 9
long_options = {{name = 0xaaaac1ea0788 "help", has_arg = 0, flag = 0x0,
val = 104}, {name = 0xaaaac1ea0790 "version", has_arg = 0, flag = 0x0, val =
118}, {
name = 0xaaaac1eb8908 "alias", has_arg = 1, flag = 0x0, val =
1024}, {name = 0xaaaac1ea0798 "subst", has_arg = 1, flag = 0x0, val = 1025},
{name = 0xaaaac1ea07a0 "substdef",
has_arg = 1, flag = 0x0, val = 1026}, {name = 0xaaaac1ea07b0
"substdefs", has_arg = 1, flag = 0x0, val = 1027}, {name = 0xaaaac1ea07c0
"server-id", has_arg = 1, flag = 0x0,
val = 1028}, {name = 0xaaaac1ea07d0 "loadmodule", has_arg = 1, flag
= 0x0, val = 1029}, {name = 0xaaaac1ea07e0 "modparam", has_arg = 1, flag = 0x0,
val = 1030}, {
name = 0xaaaac1ea07f0 "log-engine", has_arg = 1, flag = 0x0, val =
1031}, {name = 0xaaaac1ea0800 "debug", has_arg = 1, flag = 0x0, val = 1032}, {
name = 0xaaaac1ea0808 "cfg-print", has_arg = 0, flag = 0x0, val =
1033}, {name = 0xaaaac1ea0818 "atexit", has_arg = 1, flag = 0x0, val = 1034}, {
name = 0xaaaac1ea0820 "all-errors", has_arg = 0, flag = 0x0, val =
1035}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}}
__func__ = "main"
(gdb)
```
info locals
```
(gdb) info locals
hash = <optimized out>
p_entry = 0xffff68c1ae90
(gdb) print *p_entry
$1 = {first = 0xffff6900e1b0, content = 1}
(gdb) print *p_entry->first
$2 = {value = {s = 0x1136900000d <error: Cannot access memory at address
0x1136900000d>, len = 1761447408}, dlg = 0xffff6900e1e0,
puid =
"\004\000\000\000-68b\345\341\000i\377\377\000\000$\000\000\000e9\000\000uuid\00031110fa2-43a6-45d4-8387-6bf989507906",
puid_len = 22, expires = 3233857728, flags = -1412567059,
linker = 0xffff6900e1b0, next = 0xffff6900e1b0, prev = 0x60, hash = 10}
(gdb) print *vkey
$3 = {s = 0xffff69019558 "uwLrPZ8JybtF5he4yTxai2..", len = 24}
(gdb) print *linker
$4 = {hash_linker = {value = {s = 0xffff68d91e60 "23.94.26.58\016", len = 11},
dlg = 0xffff690193e0, puid = "dlgp-68befede-62f-c5e9", '\000' <repeats 43
times>, puid_len = 22, expires = 0,
flags = 0, linker = 0xffff68d91dc0, next = 0xffff6900e1b0, prev = 0x60,
hash = 10}, next = 0x0, profile = 0xffff68c1adc0}
```
list output
```
(gdb) list
487 in ./src/modules/dialog/dlg_profile.c
```
### Log outputs
it may or may not related, while the set dlg profile above looks like it
properly has string as a key, I do sometimes have the following in the logs
```
unable to get pv value for [$(dlg_var(dst_uri){uri.domain})]
non-string parameter - func: set_dlg_profile idx: 1 argc: 2
```
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
```
version: kamailio 6.0.1 (aarch64/linux)
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE,
USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC,
F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT-NOSMP,
USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES,
TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE
262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled with gcc 12.2.0
```
* **Operating System**:
<!--
Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu
16.04, CentOS 7.1, ...), MacOS, xBSD, Solaris, ...;
Kernel details (output of `lsb_release -a` and `uname -a`)
-->
```
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 12 (bookworm)
Release: 12
Codename: bookworm
Linux ******** 6.1.0-38-cloud-arm64 #1 SMP Debian 6.1.147-1 (2025-08-02)
aarch64 GNU/Linux
```
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/4414
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/[email protected]>_______________________________________________
Kamailio - Development Mailing List -- [email protected]
To unsubscribe send an email to [email protected]
Important: keep the mailing list in the recipients, do not reply only to the
sender!
