Hi everyone, we are starting to have some strange issues with mediaproxy. Our setup is kamailio 4.2 + mediaproxy 2.6.1 on CentOS 6.6.
Recently we have two incidents were the communication between the dispatcher and the relays (we got two of them, one in the same server) get lost, digging in the logs we see this: Jun 25 13:07:22 kamailio media-dispatcher[14459]: session with call_id [email protected]:5060 from relay AAA.AAA.AAA.AAA did timeout Jun 25 13:07:22 kamailio media-dispatcher[14459]: Traceback (most recent call last): Jun 25 13:07:22 kamailio media-dispatcher[14459]: File "/usr/lib64/python2.6/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite Jun 25 13:07:22 kamailio media-dispatcher[14459]: why = selectable.doRead() Jun 25 13:07:22 kamailio media-dispatcher[14459]: File "/usr/lib64/python2.6/site-packages/gnutls/interfaces/twisted/__init__.py", line 79, in doRead Jun 25 13:07:22 kamailio media-dispatcher[14459]: return tcp.Connection.doRead(self) Jun 25 13:07:22 kamailio media-dispatcher[14459]: File "/usr/lib64/python2.6/site-packages/twisted/internet/tcp.py", line 214, in doRead Jun 25 13:07:22 kamailio media-dispatcher[14459]: return self._dataReceived(data) Jun 25 13:07:22 kamailio media-dispatcher[14459]: File "/usr/lib64/python2.6/site-packages/twisted/internet/tcp.py", line 220, in _dataReceived Jun 25 13:07:22 kamailio media-dispatcher[14459]: rval = self.protocol.dataReceived(data) Jun 25 13:07:22 kamailio media-dispatcher[14459]: File "/usr/lib64/python2.6/site-packages/twisted/protocols/basic.py", line 454, in dataReceived Jun 25 13:07:22 kamailio media-dispatcher[14459]: self.lineReceived(line) Jun 25 13:07:22 kamailio media-dispatcher[14459]: File "/usr/lib64/python2.6/site-packages/mediaproxy/dispatcher.py", line 239, in lineReceived Jun 25 13:07:22 kamailio media-dispatcher[14459]: session.expire_time = time() Jun 25 13:07:22 kamailio media-dispatcher[14459]: TypeError: 'module' object is not callable Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Connection with relay at AAA.AAA.AAA.AAA was lost: 'module' object is not callable Jun 25 13:07:22 kamailio media-relay[19712]: error: Connection with dispatcher at AAA.AAA.AAA.AAA:25060 was lost: A TLS packet with unexpected length was received. Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: failed to end dialog: 404 Requested Dialog not found Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Relay for this session (AAA.AAA.AAA.AAA) is no longer connected Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Relay for this session (AAA.AAA.AAA.AAA) is no longer connected Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Relay for this session (AAA.AAA.AAA.AAA) is no longer connected Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Relay for this session (AAA.AAA.AAA.AAA) is no longer connected Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Relay for this session (AAA.AAA.AAA.AAA) is no longer connected Jun 25 13:07:22 kamailio media-dispatcher[14459]: warning: user requested media_relay AAA.AAA.AAA.AAA is not available Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: No suitable relay found Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Relay for this session (AAA.AAA.AAA.AAA) is no longer connected Jun 25 13:07:22 kamailio media-dispatcher[14459]: warning: user requested media_relay AAA.AAA.AAA.AAA is not available Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: No suitable relay found Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Relay for this session (AAA.AAA.AAA.AAA) is no longer connected Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Relay for this session (AAA.AAA.AAA.AAA) is no longer connected Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Relay for this session (AAA.AAA.AAA.AAA) is no longer connected Jun 25 13:07:22 kamailio media-dispatcher[14459]: error: Relay for this session (AAA.AAA.AAA.AAA) is no longer connected And after that we had to restart the dispatcher. The connections to the management interface stopped working too. We don't have more calls or traffic than other days and the resources on the machines seems fine (this is a "top" when the incident happened, I hope that doesn't get too much distorted in the mail): top - 10:19:53 up 30 days, 9:34, 7 users, load average: 0.07, 0.05, 0.02 Tasks: 288 total, 2 running, 286 sleeping, 0 stopped, 0 zombie Cpu(s): 1.8%us, 0.3%sy, 0.0%ni, 97.7%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%st Mem: 16315980k total, 970248k used, 15345732k free, 59004k buffers Swap: 4194300k total, 57876k used, 4136424k free, 455560k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18896 root 20 0 261m 29m 1956 S 14.6 0.2 6:58.93 media-relay 18903 root 20 0 411m 24m 1992 S 7.3 0.2 2:48.80 media-dispatcher 25921 root 20 0 244m 5228 404 S 1.3 0.0 210:41.31 rsyslogd 18856 root 20 0 1198m 38m 33m S 0.7 0.2 0:11.34 kamailio 18860 root 20 0 1198m 31m 26m S 0.7 0.2 0:11.36 kamailio 31712 root 20 0 192m 8296 8184 S 0.7 0.1 108:03.22 captagent 18846 root 20 0 1198m 31m 26m S 0.3 0.2 0:11.68 kamailio 18847 root 20 0 1198m 38m 33m S 0.3 0.2 0:11.38 kamailio We are starting to suspect that maybe the old python version that Centos 6 uses or from other packages (twisted?) may be the cause but we don't discard other options (accouting, some sysctl fine tunning option...) Any ideas would be appreciated. Thanks! ======================================== Koldo Aingeru Marcos Fdez. Ingeniería de Sistemas Sarenet S.A. 944209470 Parque Tecnológico, Edificio 103 48170 Zamudio, Bizkaia www.sarenet.es ========================================
_______________________________________________ Users mailing list [email protected] http://lists.opensips.org/cgi-bin/mailman/listinfo/users
