Kannel 1.5 DLR - segmentation fault
Hello everyone, I have a problem with the DLR on kannel 1.5 The remote SMSC sending DLRs which are not found in our DLR table as you can see in attached logs. This causes segmentation fault end kills kannel. I've found a patch to solve this problem which trying to find the DLR in table for X times, but after X attempts the problem is same: DLR not found braerbox stops and segmentation fault Anyone can help me with it? Thanks Adam Logs: smsbox.log 2013-05-16 11:03:34 [14915] [0] ERROR: connect to 127.0.0.1 failed 2013-05-16 11:03:34 [14915] [0] ERROR: System error 111: Connection refused 2013-05-16 11:03:34 [14915] [0] DEBUG: Connecting to 127.0.0.1 2013-05-16 11:03:34 [14915] [0] ERROR: connect to 127.0.0.1 failed 2013-05-16 11:03:34 [14915] [0] ERROR: System error 111: Connection refused 2013-05-16 11:03:34 [14915] [0] ERROR: error connecting to server `localhost' at port `29005' 2013-05-16 11:03:34 [14915] [0] PANIC: Couldn't connect to the bearerbox. 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox(gw_backtrace+0xad) [0x808b44d] 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox(gw_panic+0xe8) [0x808b548] 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox(main+0x1040) [0x8055dd0] 2013-05-16 11:03:34 [14915] [0] PANIC: /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe0) [0xb7112450] 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox [0x8054531] kannel.log - 2013-05-16 11:03:23 [14504] [8] DEBUG: boxc_receiver: sms received 2013-05-16 11:03:23 [14504] [8] DEBUG: send_msg: sending msg to box: 127.0.0.1 2013-05-16 11:03:23 [14504] [8] DEBUG: boxc_receiver: sms received 2013-05-16 11:03:23 [14504] [8] DEBUG: send_msg: sending msg to box: 127.0.0.1 2013-05-16 11:03:23 [14504] [8] DEBUG: boxc_receiver: sms received 2013-05-16 11:03:23 [14504] [8] DEBUG: send_msg: sending msg to box: 127.0.0.1 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80fc18d] 2013-05-16 11:03:23 [14504] [6] PANIC: [0xb77c5440] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80a338f] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80a4156] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80a523d] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80e5edc] 2013-05-16 11:03:23 [14504] [6] PANIC: /lib/tls/i686/cmov/libpthread.so.0 [0xb752a4fb] Smsc.log -- 2013-05-16 11:03:23 [14504] [6] DEBUG: DLR from SMSCremotesmsc for DST06301234567. Attempt 19 of 20. 2013-05-16 11:03:23 [14504] [6] DEBUG: Sleeping for 1.000 seconds 2013-05-16 11:03:23 [14504] [6] DEBUG: sql: SELECT `mask`, `service`, `url`, `source`, `destination`, `boxc` FROM `dlrtablename` WHERE `smsc`=? AND `ts`=? AND `destination` LIKE CONCAT('%', ?) LIMIT 1 2013-05-16 11:03:23 [14504] [6] DEBUG: column=mask buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=service buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=url buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=source buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=destination buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=boxc buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: DLR from SMSCremotesmsc for DST06301234567. Attempt 20 of 20. 2013-05-16 11:03:23 [14504] [6] WARNING: DLR[mysql]: DLR from SMSCremotesmsc for DST06301234567 not found after 20 attempts. 2013-05-16 11:03:44 [15036] [6] INFO: EMI2[remotesmsc]: connecting to Primary SMSC
Re: Kannel 1.5 DLR - segmentation fault
there's two issues to solve here: a) why are you getting a DLR for a message you have sent but don't remember. b) why is it bailing out at this time because this should simply ignore the DLR in that case. for b) we need a backtrace, a PANIC log or something like that to find out what is creating the segfault (PANIC does it on purpose but the panic output is what we must be able to see). You should compile kannel with symbols on or run it in gdb and do a backtrace there. for a) this could be a config issue of all kinds or simply a bug but I guess it's the first. So how this is configured exactly has to be analyzed. Maybe you can share an example of the exact URL's as passed out and back into kannel. On 16.05.2013, at 13:04, Adam Pinter adam.pin...@dream.hu wrote: Hello everyone, I have a problem with the DLR on kannel 1.5 The remote SMSC sending DLRs which are not found in our DLR table as you can see in attached logs. This causes segmentation fault end kills kannel. I've found a patch to solve this problem which trying to find the DLR in table for X times, but after X attempts the problem is same: DLR not found braerbox stops and segmentation fault Anyone can help me with it? Thanks Adam Logs: smsbox.log 2013-05-16 11:03:34 [14915] [0] ERROR: connect to 127.0.0.1 failed 2013-05-16 11:03:34 [14915] [0] ERROR: System error 111: Connection refused 2013-05-16 11:03:34 [14915] [0] DEBUG: Connecting to 127.0.0.1 2013-05-16 11:03:34 [14915] [0] ERROR: connect to 127.0.0.1 failed 2013-05-16 11:03:34 [14915] [0] ERROR: System error 111: Connection refused 2013-05-16 11:03:34 [14915] [0] ERROR: error connecting to server `localhost' at port `29005' 2013-05-16 11:03:34 [14915] [0] PANIC: Couldn't connect to the bearerbox. 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox(gw_backtrace+0xad) [0x808b44d] 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox(gw_panic+0xe8) [0x808b548] 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox(main+0x1040) [0x8055dd0] 2013-05-16 11:03:34 [14915] [0] PANIC: /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe0) [0xb7112450] 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox [0x8054531] kannel.log - 2013-05-16 11:03:23 [14504] [8] DEBUG: boxc_receiver: sms received 2013-05-16 11:03:23 [14504] [8] DEBUG: send_msg: sending msg to box: 127.0.0.1 2013-05-16 11:03:23 [14504] [8] DEBUG: boxc_receiver: sms received 2013-05-16 11:03:23 [14504] [8] DEBUG: send_msg: sending msg to box: 127.0.0.1 2013-05-16 11:03:23 [14504] [8] DEBUG: boxc_receiver: sms received 2013-05-16 11:03:23 [14504] [8] DEBUG: send_msg: sending msg to box: 127.0.0.1 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80fc18d] 2013-05-16 11:03:23 [14504] [6] PANIC: [0xb77c5440] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80a338f] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80a4156] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80a523d] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80e5edc] 2013-05-16 11:03:23 [14504] [6] PANIC: /lib/tls/i686/cmov/libpthread.so.0 [0xb752a4fb] Smsc.log -- 2013-05-16 11:03:23 [14504] [6] DEBUG: DLR from SMSCremotesmsc for DST06301234567. Attempt 19 of 20. 2013-05-16 11:03:23 [14504] [6] DEBUG: Sleeping for 1.000 seconds 2013-05-16 11:03:23 [14504] [6] DEBUG: sql: SELECT `mask`, `service`, `url`, `source`, `destination`, `boxc` FROM `dlrtablename` WHERE `smsc`=? AND `ts`=? AND `destination` LIKE CONCAT('%', ?) LIMIT 1 2013-05-16 11:03:23 [14504] [6] DEBUG: column=mask buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=service buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=url buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=source buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=destination buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=boxc buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: DLR from SMSCremotesmsc for DST06301234567. Attempt 20 of 20. 2013-05-16 11:03:23 [14504] [6] WARNING: DLR[mysql]: DLR from SMSCremotesmsc for DST06301234567 not found after 20 attempts. 2013-05-16 11:03:44 [15036] [6] INFO: EMI2[remotesmsc]: connecting to Primary SMSC
Re: Kannel 1.5 DLR - segmentation fault
1. Did you modify kannel source? 2. What's the revision from SVN are you using? Is it latest? 2013/5/16 Andreas Fink af...@list.fink.org there's two issues to solve here: a) why are you getting a DLR for a message you have sent but don't remember. b) why is it bailing out at this time because this should simply ignore the DLR in that case. for b) we need a backtrace, a PANIC log or something like that to find out what is creating the segfault (PANIC does it on purpose but the panic output is what we must be able to see). You should compile kannel with symbols on or run it in gdb and do a backtrace there. for a) this could be a config issue of all kinds or simply a bug but I guess it's the first. So how this is configured exactly has to be analyzed. Maybe you can share an example of the exact URL's as passed out and back into kannel. On 16.05.2013, at 13:04, Adam Pinter adam.pin...@dream.hu wrote: Hello everyone, I have a problem with the DLR on kannel 1.5 The remote SMSC sending DLRs which are not found in our DLR table as you can see in attached logs. This causes segmentation fault end kills kannel. I've found a patch to solve this problem which trying to find the DLR in table for X times, but after X attempts the problem is same: DLR not found braerbox stops and segmentation fault Anyone can help me with it? Thanks Adam Logs: smsbox.log 2013-05-16 11:03:34 [14915] [0] ERROR: connect to 127.0.0.1 failed 2013-05-16 11:03:34 [14915] [0] ERROR: System error 111: Connection refused 2013-05-16 11:03:34 [14915] [0] DEBUG: Connecting to 127.0.0.1 2013-05-16 11:03:34 [14915] [0] ERROR: connect to 127.0.0.1 failed 2013-05-16 11:03:34 [14915] [0] ERROR: System error 111: Connection refused 2013-05-16 11:03:34 [14915] [0] ERROR: error connecting to server `localhost' at port `29005' 2013-05-16 11:03:34 [14915] [0] PANIC: Couldn't connect to the bearerbox. 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox(gw_backtrace+0xad) [0x808b44d] 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox(gw_panic+0xe8) [0x808b548] 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox(main+0x1040) [0x8055dd0] 2013-05-16 11:03:34 [14915] [0] PANIC: /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe0) [0xb7112450] 2013-05-16 11:03:34 [14915] [0] PANIC: /usr/local/kannel15/sbin/smsbox [0x8054531] kannel.log - 2013-05-16 11:03:23 [14504] [8] DEBUG: boxc_receiver: sms received 2013-05-16 11:03:23 [14504] [8] DEBUG: send_msg: sending msg to box: 127.0.0.1 2013-05-16 11:03:23 [14504] [8] DEBUG: boxc_receiver: sms received 2013-05-16 11:03:23 [14504] [8] DEBUG: send_msg: sending msg to box: 127.0.0.1 2013-05-16 11:03:23 [14504] [8] DEBUG: boxc_receiver: sms received 2013-05-16 11:03:23 [14504] [8] DEBUG: send_msg: sending msg to box: 127.0.0.1 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80fc18d] 2013-05-16 11:03:23 [14504] [6] PANIC: [0xb77c5440] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80a338f] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80a4156] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80a523d] 2013-05-16 11:03:23 [14504] [6] PANIC: /usr/local/kannel15/sbin/bearerbox [0x80e5edc] 2013-05-16 11:03:23 [14504] [6] PANIC: /lib/tls/i686/cmov/libpthread.so.0 [0xb752a4fb] Smsc.log -- 2013-05-16 11:03:23 [14504] [6] DEBUG: DLR from SMSCremotesmsc for DST06301234567. Attempt 19 of 20. 2013-05-16 11:03:23 [14504] [6] DEBUG: Sleeping for 1.000 seconds 2013-05-16 11:03:23 [14504] [6] DEBUG: sql: SELECT `mask`, `service`, `url`, `source`, `destination`, `boxc` FROM `dlrtablename` WHERE `smsc`=? AND `ts`=? AND `destination` LIKE CONCAT('%', ?) LIMIT 1 2013-05-16 11:03:23 [14504] [6] DEBUG: column=mask buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=service buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=url buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=source buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=destination buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: column=boxc buffer_type=252 max_length=0 length=65535 2013-05-16 11:03:23 [14504] [6] DEBUG: DLR from SMSCremotesmsc for DST06301234567. Attempt 20 of 20. 2013-05-16 11:03:23 [14504] [6] WARNING: DLR[mysql]: DLR from SMSCremotesmsc for DST06301234567 not found after 20 attempts. 2013-05-16 11:03:44 [15036] [6] INFO: EMI2[remotesmsc]: connecting to Primary SMSC