[asterisk-users] Asterisk 1.4.x segfaulting daily
Hello list, An Asterisk installation that was doing fine suddenly stared segfaulting a couple of times per day. I enabled all the logging and debugging to try to find a pattern but there was too much information to see exactly where it broke. So I enabled core dump and did backtraces and all of them seem to break on ast_setstate, setting the state to AST_STATE_DOWN. That's pretty much the only thing I can make of it, don't even know if that's correct. Does anyone have any ideas on why this is happening? The backtrace is attached. P.S.: I've switched the whole hardware already, including the BRI card (B400P, OpenVox). Also tried different versions of Asterisk, Dahdi and mISDN. I'm stuck with 1.4 Asterisk branch and mISDN v1. Best regards, Paulo Santos Core was generated by `/usr/sbin/asterisk'. Program terminated with signal 11, Segmentation fault. [New process 21726] [New process 24376] [New process 24375] [New process 24374] [New process 24371] [New process 24344] [New process 23560] [New process 22868] [New process 22329] [New process 22327] [New process 22325] [New process 22324] [New process 22323] [New process 22322] [New process 22321] [New process 22320] [New process 22319] [New process 22318] [New process 22317] [New process 22316] [New process 22315] [New process 22259] [New process 22208] [New process 22203] [New process 22185] [New process 22184] [New process 22160] [New process 21515] [New process 21725] [New process 21687] [New process 21686] [New process 21685] [New process 21681] [New process 21659] [New process 21658] [New process 21648] [New process 21647] [New process 21609] [New process 21594] [New process 21542] [New process 21540] [New process 21516] #0 0x080851ee in ast_setstate (chan=0xb3401c00, state=AST_STATE_DOWN) at /usr/src/asterisk-1.4.42/include/asterisk/strings.h:37 37 return (!s || (*s == '\0')); #0 0x080851ee in ast_setstate (chan=0xb3401c00, state=AST_STATE_DOWN) at /usr/src/asterisk-1.4.42/include/asterisk/strings.h:37 name = mISDN/4\000u\000ݴ��\177�\020\000@�\b(@�H0ݴf\211q�\020\000@�\b(@�\000\000@�Хe�\b(@�X\b@�h0ݴ\203\225a�P[ 3] \000\000\000\000\000 #1 0xb561975d in release_chan (ch=0xb3400858, bc=0x88e8f5c) at chan_misdn.c:3750 ast = (struct ast_channel *) 0xb3401c00 #2 0xb5622275 in cb_events (event=EVENT_CLEANUP, bc=0x88e8f5c, user_data=0x0) at chan_misdn.c:4845 msn_valid = -1287644160 held_ch = value optimized out ch = (struct chan_list *) 0xb3400858 __PRETTY_FUNCTION__ = cb_events #3 0xb5632d9f in handle_cr (stack=0x88e82d8, frm=value optimized out) at misdn/isdn_lib.c:1684 channel = 255 bc = (struct misdn_bchannel *) 0x88e8f5c dummybc = {send_lock = 0xb67feff4, dummy = -1260570753, nt = -1260572104, pri = -1234083825, port = -1260572068, b_stid = -1260571776, layer_id = -1260570753, layer = -1234274741, need_disconnect = -1233129484, need_release = -1260572068, need_release_complete = -1260571776, dec = -1260571832, l3_id = -1234111388, pid = -1260572068, ces = -1251638304, restart_channel = -1260570700, channel = -1260571776, channel_preselected = 0, in_use = -1260571908, last_used = {tv_sec = 1023, tv_usec = -72515583}, cw = -1260571776, addr = -1260571776, bframe = 0xb4dd3380 handle_frm: frm-addr:42000303 frm-prim:3f182\n, bframe_len = -1260571776, time_usec = -1260571729, astbuf = 0xb4dd377f, misdnbuf = 0xb4dd3380, te_choose_channel = -1260570753, early_bconnect = 0, dtmf = 0, send_dtmf = 0, need_more_infos = 0, sending_complete = 0, nodsp = 1635021600, nojitter = 0, dnumplan = NUMPLAN_UNKNOWN, rnumplan = 1308622848, onumplan = NUMPLAN_UNKNOWN, cpnnumplan = NUMPLAN_UNINITIALIZED, progress_coding = 824193585, progress_location = 942881334, progress_indicator = 3617594, fac_in = {Function = Fac_GetSupportedServices, u = {Listen = {NotificationMask = 21}, Suspend = { CallIdentity = \025\000\000\000\000\000\000\000\000\000\000}, Resume = { CallIdentity = \025\000\000\000\000\000\000\000\000\000\000}, CFActivate = {Handle = 21, Procedure = 0, BasicService = 0, ServedUserNumber = \000\000\000\000Хe�\001\000\000, ForwardedToNumber = @�\177�\000\000\000\000�wa�\0203ݴ, ForwardedToSubaddress = \000\004\000\000�ze�7ݴ@�\177�}, CFDeactivate = {Handle = 21, Procedure = 0, BasicService = 0, ServedUserNumber = \000\000\000\000Хe�\001\000\000}, CFInterrogateParameters = {Handle = 21, Procedure = 0, BasicService = 0, ServedUserNumber = \000\000\000\000Хe�\001\000\000}, CFInterrogateNumbers = {Handle = 21}, CDeflection = { PresentationAllowed = 21, DeflectedToNumber = \000\000\000\000\000\000\000\000\000\000Х, DeflectedToSubaddress = e�\001\000\000\000@�\177�\000\000\000\000�w}, AOCDchu = {chargeNotAvailable = 21, freeOfCharge = 0, recordedUnits = 0, typeOfChargingInfo = -1, billingId = 0}, AOCDcur = {chargeNotAvailable
Re: [asterisk-users] Asterisk 1.4.x segfaulting daily
On 14 December 2011 12:56, Paulo Santos paulo.r.san...@sapo.pt wrote: Hello list, An Asterisk installation that was doing fine suddenly stared segfaulting a couple of times per day. I enabled all the logging and debugging to try to find a pattern but there was too much information to see exactly where it broke. So I enabled core dump and did backtraces and all of them seem to break on ast_setstate, setting the state to AST_STATE_DOWN. That's pretty much the only thing I can make of it, don't even know if that's correct. Does anyone have any ideas on why this is happening? The backtrace is attached. P.S.: I've switched the whole hardware already, including the BRI card (B400P, OpenVox). Also tried different versions of Asterisk, Dahdi and mISDN. I'm stuck with 1.4 Asterisk branch and mISDN v1. If I was guessing, I'd say that the channel structure that is being modified by the ast_setstate() call is incomplete, and contains some garbage pointers. If I was guessing further, I'd say that the callerID pointers are the most likely candidate - Does the issue happen when a caller-id withheld call is hung-up? hung-up before being answered perhaps? You'd need to add some debug reporting into ast_setstate() to know for sure. Just my 2p - 1.4.42 is an old version, so the chance of a solid answer is fairly low. Steve -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Asterisk 1.4.x segfaulting daily
On 14-12-11 13:56, Paulo Santos wrote: Hello list, An Asterisk installation that was doing fine suddenly stared segfaulting a couple of times per day. I enabled all the logging and debugging to try to find a pattern but there was too much information to see exactly where it broke. So I enabled core dump and did backtraces and all of them seem to break on ast_setstate, setting the state to AST_STATE_DOWN. That's pretty much the only thing I can make of it, don't even know if that's correct. Does anyone have any ideas on why this is happening? The backtrace is attached. P.S.: I've switched the whole hardware already, including the BRI card (B400P, OpenVox). Also tried different versions of Asterisk, Dahdi and mISDN. I'm stuck with 1.4 Asterisk branch and mISDN v1. If the suggestion from Steve Davies doesn't work out for you then my suggestion would be to try out the latest DAHDI libpri with the latest Asterisk 1.8 because those versions have built-in support for the 4x BRI HFC chipset which can be found on the Digium b410p and the Openvox B400P. This way you no longer need mISDN V1 and have recent versions with tons of bugs fixed. Here are instructions from Openvox: http://wiki.openvox.cn/index.php/OpenVox_B400P_User_Manual_for_dahdi Please note that in the instructions they use older versions. I would use the latest DAHDI, libpri (don't forget this one) and asterisk 1.8 available here: https://www.asterisk.org/downloads Regards, Patrick -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Asterisk 1.4.x segfaulting daily
Hello, Thank you all for the replies. Steve Davies wrote: If I was guessing, I'd say that the channel structure that is being modified by the ast_setstate() call is incomplete, and contains some garbage pointers. If I was guessing further, I'd say that the callerID pointers are the most likely candidate - Does the issue happen when a caller-id withheld call is hung-up? hung-up before being answered perhaps? It was an outgoing call that tried to call through the port 2, then 1 and finally 3. The third port has a quite different debug output than the other 2. Maybe it's a problem on that connection, appears to be common on all segfaults. Apparently that third port is something of a strange group of BRI lines between that one and the line on the second port, but behaves differently. I'll try to find out more about it. Patrick Lists wrote: If the suggestion from Steve Davies doesn't work out for you then my suggestion would be to try out the latest DAHDI libpri with the latest Asterisk 1.8 because those versions have built-in support for the 4x BRI HFC chipset which can be found on the Digium b410p and the Openvox B400P. This way you no longer need mISDN V1 and have recent versions with tons of bugs fixed. Unfortunately I can't do that, at least not now. I will, however, migrate it eventually to either mISDN v2 or Dahdi, depending on the state of Dahdi then. P.S.: Attached the log. Best regards, Paulo Santos [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: = No match Their Call ID: 333232837-5062-310@192.168.0.8 Their Tag 1036797295 Our tag: as5b7769e2 [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: = No match Their Call ID: 1693981358-5068-505@192.168.0.7 Their Tag 692402733 Our tag: as170cc25e [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: = No match Their Call ID: 1394539361-5064-828@192.168.0.7 Their Tag 1627163612 Our tag: as5f15bf50 [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: = No match Their Call ID: 1708030692-5060-122@192.168.0.8 Their Tag 52015999 Our tag: as24b80c2d [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Setting NAT on RTP to Off [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Setting NAT on UDPTL to Off [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Allocating new SIP dialog for 1547819775-5062-295@192.168.0.7 - INVITE (With RTP) [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Received INVITE (5) - Command in SIP INVITE [Dec 12 16:38:36] DEBUG[22160] acl.c: # Testing 192.168.0.7 with 0.0.0.0 [Dec 12 16:38:36] DEBUG[22160] acl.c: # Testing 192.168.0.7 with 192.168.0.0 [Dec 12 16:38:36] DEBUG[22160] acl.c: # Testing 192.168.0.7 with 10.0.0.0 [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Setting NAT on RTP to On [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Setting NAT on UDPTL to On [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: = Found Their Call ID: 1547819775-5062-295@192.168.0.7 Their Tag 2074339809 Our tag: as2515e4b3 [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Received ACK (6) - Command in SIP ACK [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Stopping retransmission on '1547819775-5062-295@192.168.0.7' of Response 2940: Match Found [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: = Found Their Call ID: 1547819775-5062-295@192.168.0.7 Their Tag 2074339809 Our tag: as2515e4b3 [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Received INVITE (5) - Command in SIP INVITE [Dec 12 16:38:36] DEBUG[22160] acl.c: # Testing 192.168.0.7 with 0.0.0.0 [Dec 12 16:38:36] DEBUG[22160] acl.c: # Testing 192.168.0.7 with 192.168.0.0 [Dec 12 16:38:36] DEBUG[22160] acl.c: # Testing 192.168.0.7 with 10.0.0.0 [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Setting NAT on RTP to On [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Setting NAT on UDPTL to On [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing session-level SDP v=0... UNSUPPORTED. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing session-level SDP o=11 8002 8000 IN IP4 192.168.0.7... UNSUPPORTED. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing session-level SDP s=SIP Call... UNSUPPORTED. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing session-level SDP c=IN IP4 192.168.0.7... OK. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing session-level SDP t=0 0... UNSUPPORTED. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing media-level (audio) SDP a=sendrecv... OK. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing media-level (audio) SDP a=rtpmap:0 PCMU/8000... OK. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing media-level (audio) SDP a=ptime:20... OK. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing media-level (audio) SDP a=rtpmap:8 PCMA/8000... OK. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing media-level (audio) SDP a=rtpmap:4 G723/8000... OK. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing media-level (audio) SDP a=rtpmap:18 G729/8000... OK. [Dec 12 16:38:36] DEBUG[22160] chan_sip.c: Processing media-level (audio) SDP a=rtpmap:2 G726-32/8000... OK. [Dec 12 16:38:36] DEBUG[22160]