No, no. I was wrong. It wasn't kannel. The issue was in the SIGTRAN SMSC. Tuned some parameters and its now happily sending at 350 sms/sec, sustained.
Will now start tuning kannel, smsbox/dlr-url sqlbox/mysqld to try and push the 4-node cluster past 2000 sms/sec. Many thanks all. -wasim On Thu, May 31, 2012 at 4:46 PM, spameden <[email protected]> wrote: > Also interested in this issue, but I'm not so good in C. Played a lot with > this in the past. > > 2012/5/31 Wasim Baig <[email protected]> > >> Replaced Kannel with an IBM/UNICA SMPP, and the sending rate shot up to >> 320/sec and stayed there. A short test of 10k sms, finished in just under >> 32 secs. >> >> This leads me to believe more and more that the issue I'm facing is >> within Kannel, and specifically the throttlling code and pending submits. >> Maybe not on the submit_sm side, maybe on the dlr side. Not sure. >> >> I'm going to try and fiddle with it, but sure would appreciate someone >> more knowledgeable taking a look. I have a test bed ready so we can quickly >> isolate the issue. Thanks. >> >> -wasim >> >> >> On Tue, May 29, 2012 at 6:38 AM, Wasim Baig <[email protected]> wrote: >> >>> Is there a configurable timeout for a pending submit? In the guide there >>> are >>> >>> idle-timeout >>> remote-timeout >>> connection-timeout >>> mo-timeout >>> >>> Nothing for mt or an otherwise response timeout? I tried another test >>> with 50k sms and the following: >>> >>> sms-resend-retry=0 >>> ... >>> throughput=200 >>> max-pending-submits=500 >>> >>> Ended up with a net throughput of 100 sms/sec, 50k sms in 500 secs. In >>> the pcap, most responses are within .001 secs >>> >>> Attached are the smpp summary and a graph of the total smpp packets and >>> submit_sm rate. >>> >>> Today, when the sending window (0900 to 2100 hours) opens, I will try a >>> throttle at 400 and see what happens. >>> >>> Appreciate any bits of advice from other high throughput kannel users to >>> tackle this. >>> >>> -wasim >>> >>> >>> On Mon, May 28, 2012 at 10:08 PM, Alvaro Cornejo < >>> [email protected]> wrote: >>> >>>> Hi Wasim >>>> >>>> The submit_sm_resp comes from your provider, the pcap is taken >>>> direclty on the ethernet layer. ie before it reaches kannel. It shows >>>> the 10K submit but only 99K resp. Therefore I'll tend to think is >>>> either a problem with the carrier or the link. >>>> >>>> Hope helps >>>> >>>> Alvaro >>>> >>>> On 5/28/12, Wasim Baig <[email protected]> wrote: >>>> > Hi Alvaro: >>>> > >>>> > There is no limitation on at the carrier, a Mobicents JSS7/SMSC with >>>> > CloudSMPP connected directly to GSM network. The only throttling we're >>>> > doing is on the kannel side. >>>> > >>>> > The connection between bearerbox and CloudSMPP is on gigabit >>>> ethernet, same >>>> > vlan. I've even tested by putting the bearerbox on the same box as the >>>> > smsc. Same results. I've tried separate bearerbox, sqlbox, smsbox, db >>>> etc. >>>> > Same results. >>>> > >>>> > I have a feeling its just pending submits over time. If you see the >>>> > attached screenshot of a 10,000 sms pcap, wireshark says we only got >>>> 9982 >>>> > submit_sm_resp back to 10k submit_sm. Over time these would add up. >>>> > >>>> > It could be dumpcap dropping the packets, but I doubt that. >>>> > >>>> > My aim is to get this up to 500 sms/sec, and then cluster 4 of those >>>> > puppies together for a nice little 2k sms/sec kannel implementation. I >>>> > would be very happy to paypal for some advice on this. >>>> > >>>> > -wasim >>>> > >>>> > >>>> > On Mon, May 28, 2012 at 7:48 PM, Alvaro Cornejo >>>> > <[email protected]>wrote: >>>> > >>>> >> Hi >>>> >> >>>> >> What is the throughput allowed you by your carrier? >>>> >> >>>> >> Usually kannel is able to send much more messages than the carriers >>>> >> allows you; so check with them. >>>> >> >>>> >> Also, as you are connecting through internet, your bandwidth is >>>> >> important for submitting sms as well as receiving dlrs. >>>> >> >>>> >> Regards >>>> >> >>>> >> Alvaro >>>> >> >>>> >> >>>> >> >>>> >> On 5/28/12, Wasim Baig <[email protected]> wrote: >>>> >> > using fakesmsc my setup happily does 2500/sec (SQLBOX MT inserts, >>>> DLR >>>> >> > to >>>> >> > smsbox and get-url updates a DB) >>>> >> > plug in the real SMSC, throttle down to 200 sms/sec and initially >>>> its >>>> >> fine >>>> >> > but after a few minutes slows down to 50-60 sms/sec... >>>> >> > >>>> >> > I never see bearerbox sending at 200 sms/sec, after a few minutes >>>> its >>>> >> down >>>> >> > to 120-130 ... >>>> >> > >>>> >> > 2012-05-28 18:20:13 [15234] [6] DEBUG: SMPP[smsc]: throughput >>>> >> > (132.00,200.00) >>>> >> > 2012-05-28 18:20:15 [15234] [6] DEBUG: SMPP[smsc]: throughput >>>> >> > (125.00,200.00) >>>> >> > >>>> >> > and after some time, even lower >>>> >> > >>>> >> > 2012-05-28 18:45:39 [22134] [6] DEBUG: SMPP[smsc]: throughput >>>> >> > (47.00,200.00) >>>> >> > 2012-05-28 18:45:40 [22134] [6] DEBUG: SMPP[smsc]: throughput >>>> >> > (42.00,200.00) >>>> >> > >>>> >> > could this be pending submits? >>>> >> > >>>> >> > -- >>>> >> > wasim h. baig | +92 30 0850 8070 | peace be upon you ... >>>> >> > >>>> >> >>>> >> >>>> >> -- >>>> >> >>>> >> >>>> |-----------------------------------------------------------------------------------------------------------------| >>>> >> Envíe y Reciba Datos y mensajes de Texto (SMS) hacia y desde >>>> cualquier >>>> >> celular y Nextel >>>> >> en el Perú, México y en mas de 180 paises. Use aplicaciones 2 vias >>>> via >>>> >> SMS y GPRS online >>>> >> Visitenos en www.perusms.NET www.smsglobal.com.mx y >>>> >> www.pravcom.com >>>> >> >>>> > >>>> > >>>> > >>>> > -- >>>> > wasim h. baig | +92 30 0850 8070 | peace be upon you ... >>>> > >>>> >>>> >>>> -- >>>> >>>> |-----------------------------------------------------------------------------------------------------------------| >>>> Envíe y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier >>>> celular y Nextel >>>> en el Perú, México y en mas de 180 paises. Use aplicaciones 2 vias via >>>> SMS y GPRS online >>>> Visitenos en www.perusms.NET www.smsglobal.com.mx y >>>> www.pravcom.com >>>> >>> >>> >>> >>> -- >>> wasim h. baig | +92 30 0850 8070 | peace be upon you ... >>> >>> >> >> >> -- >> wasim h. baig | +92 30 0850 8070 | peace be upon you ... >> >> > -- wasim h. baig | +92 30 0850 8070 | peace be upon you ...
