Hi Ahmed I don't know how to help you or where the problem is. Based on what you just explained, it should be very fast. Sorry I can't help you. But let me know what the problem was when you find it! Regards
On Tue, Jun 10, 2014 at 3:31 AM, Ahmed BOUDHRAA < [email protected]> wrote: > Hi > I hope i m not bothering you, > > first the kannel servers, we choose 8GB but its over estimated in our test > we never reached 5 GB, and never go ferther of 150 httpd process even if we > have set it in kannel side to reach 1000 httpd process. > For the web app server 32 GB is litle over estimated too, in our test with > fakesms we reached the 10 GB, and yes for tunning the web side we use > apache bench (ab) the values i have given where choosed based on ab > benchmarking ( setting up the 4500 max httpd process) as i said to not > complicate more things i m quite sure that the web side is well done > because we have worked years now and we know how to tune it up system and > apache sides > With fakesms we never reached the values we got with ab and has the web > server working fine that's why the bottleneck could never be the web server > We tuned even kannel apache side as i said in all the tests we never seen > the kannel server reach one of his limits, RAM, CPU, nember of httpd > process running simultanitly > imagine a state that you have a kannel how can process 1000 process httpd > but he is running just 100 with low memory and CPU usage, the web server / > db server can handle 4500 httpd process and 4500 postgres process and they > are both far from their capabilitys, in our tests with fakesms when sending > the 30000 sms we reached the 4000 httpd process in web app but still as i > said with apache bench (ab) test the servers handled quite fine 4500 ones > so 4000 should be fine too so how could be after like 15000 sms the > incoming sms drop radically to 1sms/5-6 seconds ????? > > PS : for the web side we could go ferver up with ab test and increasing > the nembrer of httpd process like 6000 7000 and even further withaout > running out of ressources but we dont see the use for it > > > ----- Mail Original ----- > De: "DHC Admin" <[email protected]> > À: "Ahmed BOUDHRAA" <[email protected]> > Cc: "spameden" <[email protected]>, [email protected] > Envoyé: Lundi 9 Juin 2014 17:20:53 > > Objet: Re: Tunning up kannel > > Ahmed, I understand that each server has 8Gb, each Apache client, with the > usual WEB server stuff is around 10Mb, each client. > How fast is your process for each MO? Because I can't see how can a server > with 8Gb hold up 2000 clients at the same time (assuming the process takes > 0.5sec) > Have you checked the "apachectl status" while sending the messages to > check how busy it is? > > Have you tried using "ab" to stress the web server and be 100% sure it can > handle that amount of traffic? > > Sorry I can not help you further on this, I don't know where is the > problem, but I think it might not be a pure kannel thing, but a combination > of the whole system. > > > > On Mon, Jun 9, 2014 at 12:55 PM, Ahmed BOUDHRAA < > [email protected]> wrote: > >> Yes yes this is what we think there will be anonther queue in smsbox, and >> yes we have enough resources as i said configuring the web at 4000 requests >> / s its about the 1/4 of his hardware capability and with that >> configuration we havent reached the 4000 request / s >> for the the MO Queue that i mentioned its in the png attached its taked >> from the web interface kannel status >> >> >> ----- Mail Original ----- >> De: "DHC Admin" <[email protected]> >> À: "Ahmed BOUDHRAA" <[email protected]> >> Cc: "spameden" <[email protected]>, [email protected] >> Envoyé: Lundi 9 Juin 2014 16:39:24 >> >> Objet: Re: Tunning up kannel >> >> Hi Ahmed >> Maybe the Bearebox has emptied the its queue and the queue moves to the >> smsbox? Are you sure you have enough resources to receive that much number >> of SMS at the same time? The Apache web server news to spawn (create) a big >> number of clients to handle the traffic, it might run out of resources for >> a few seconds and the smsbox could be resending the MOs later on, based on >> this configuration: >> >> http-request-retryinteger If set, specifies how many retries should be >> performed for failing HTTP requests of sms-services. Defaults to 0, which >> means no retries should be performed and hence no HTTP request queuing is >> done. http-queue-delayinteger If set, specifies how many seconds should >> pass within the HTTP queuing thread for retrying a failed HTTP request. >> Defaults to 10 sec. and is only obeyed if http-request-retry is set to a >> non-zero value. >> >> Have you tried to disable the HTTP retry and see if you loose any MO? >> Maybe they are getting retried. >> >> Please tell me where is that you see the MO Queue that you mention, are >> you just checking the status by console or using a HTML page? >> >> >> >> >> On Mon, Jun 9, 2014 at 11:51 AM, Ahmed BOUDHRAA < >> [email protected]> wrote: >> >>> Thinx spameden >>> >>> Our plateform is made for publishing mainlly baccalaureate results, and >>> tow other grades this is the main use right now, all those are about 240 >>> 000 canditates the fact is they may look like not much, but its a very >>> important event that every parents/student attends the problematic its not >>> about the nember but about the ammount of incoming/outgoing sms in a short >>> time, last year we have reached the 800 sms /s in the operator side so you >>> can imagine how it could be stressfull for us / the operators / the >>> parents-students we cant allow any unavalability or too much wait, you can >>> add to that that our plateform will do other things in the future :) >>> >>> for the throughput between as and the operators we fixied it in >>> accordance with them every operator have specific capabilities but as i >>> said our plateform can manage them at ease, what we want is to prepare our >>> self, and to understand all the possible problem/solutions that we may have >>> in using kannel. >>> >>> For the web server side, i cant assure that he is far from his real >>> capabilities we are using RHEL web servers from long now. for the current >>> configuration our web server can serve about 4000 requests / s i ll try >>> below to describe a simple test we made usining fakesms: >>> >>> We are launching this test from our 3 kannel server simultanisouly : >>> >>> ./fakesmsc -i 0.001 -m 10000 "100 200 text test 000000*00000000" ==> >>> this mean about 1000 sms /s from each kannel technically we could say that >>> our portal should manage 3000 sms / s >>> >>> when we llaunch the test we can see on the kannel status in the box >>> connections rubrique the "Queued (MO) started" 10000 x 3 and it take about >>> 5 minutes to empty the three queue now what we want to understand is how >>> the viewed queue is empty, the web server is far from his treatement >>> capabilities and still messages are comming about 1-2 sms /s after about >>> 15000 have reached the database. I dont know if i m clear normally when the >>> queue is empty we have to have 30000 sms on the database side, wich mean >>> there is another queue somewhere else >>> >>> >>> this is our smskannel.conf : we are not using dlr nor internal storage: >>> >>> >>> >>> group = core >>> admin-port = 13000 >>> smsbox-port = 2776 >>> admin-password = 123456 >>> box-deny-ip = "*.*.*.*" >>> box-allow-ip = "127.0.0.1" >>> log-file = /var/log/kannel/kannel.log >>> log-level = 0 >>> access-log = /var/log/kannel/access_kannel.log >>> access-log-clean = true >>> access-log-format= SMS %t %l %i %p %P %b %F %I %k >>> store-file =/var/log/kannel/sms.store >>> dlr-storage = internal >>> store-dump-freq = 5 >>> sms-resend-freq = 60 >>> sms-resend-retry = -1 >>> >>> >>> #--------------------------------------------- >>> # SMSC CONNECTIONS >>> # >>> # SMSC connections are created in bearerbox and they handle SMSC specific >>> # protocol and message relying. You need these to actually receive and >>> send >>> # messages to handset, but can use GSM modems as virtual SMSCs >>> >>> >>> group = smsc >>> smsc = fake >>> smsc-id = fake >>> port = 10000 >>> >>> >>> >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt1.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt2.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> #service-type = 'test' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt3.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt4.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt5.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt6.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt7.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt8.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt9.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt10.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> group = smsc >>> smsc = smpp >>> smsc-id = TT >>> host = x.x.x.x >>> port = xxxx >>> log-file = /var/log/kannel/tt11.log >>> log-level = 0 >>> transceiver-mode = 1 >>> receive-port = xxxx >>> smsc-username = user >>> smsc-password = pass >>> system-type = 'VMA' >>> interface-version = 34 >>> preferred-smsc-id = TT >>> >>> >>> # This is a fake smsc connection, _only_ used to test the system and >>> services. >>> # It really cannot relay messages to actual handsets! >>> >>> >>> >>> #--------------------------------------------- >>> # SMSBOX SETUP >>> # >>> # Smsbox(es) do higher-level SMS handling after they have been received >>> from >>> # SMS centers by bearerbox, or before they are given to bearerbox for >>> delivery >>> >>> group = smsbox >>> bearerbox-host = 127.0.0.1 >>> sendsms-port = 8080 >>> #global-sender = 13013 >>> log-file = /var/log/kannel/smsbox.log >>> log-level = 0 >>> mo-recode = true >>> reply-couldnotfetch = "----------------------------------" >>> reply-couldnotrepresent = "-----------" >>> reply-requestfailed = "-------------------------" >>> reply-emptymessage = "*---------------------*" >>> max-pending-requests = 1200 >>> http-queue-delay = 2 >>> http-request-retry = 100 >>> >>> #--------------------------------------------- >>> # SEND-SMS USERS >>> # >>> # These users are used when Kannel smsbox sendsms interface is used to >>> # send PUSH sms messages, i.e. calling URL like >>> # >>> http://kannel.machine:13013/cgi-bin/sendsms?username=tester&password=foobar. >>> .. >>> >>> group = sendsms-user >>> username = user >>> password = pass >>> user-allow-ip = ".e.e.e.e" >>> forced-smsc = TT >>> default-sender = 111111 >>> max-messages = 4 >>> concatenation = true >>> >>> #--------------------------------------------- >>> # SERVICES >>> # >>> # These are 'responses' to sms PULL messages, i.e. messages arriving from >>> # handsets. The response is based on message content. Only one >>> sms-service is >>> # applied, using the first one to match. >>> >>> group = sms-service >>> keyword = default >>> accept-x-kannel-headers = true >>> catch-all = true >>> max-messages =3 >>> get-url = " >>> http://site.site.tn/default.php?phone=%p&text=%a&smsc=%i&corp=%s&time=%t&smsid=%I&drv=%d&drs=%A&shortcode=%P >>> " >>> concatenation = true >>> >>> group = sms-service >>> keyword = test >>> accept-x-kannel-headers = true >>> >>> catch-all = true >>> max-messages =0 >>> get-url = " >>> http://site.site.tn/test.php?phone=%p&text=%a&smsc=%i&corp=%s&time=%t&smsid=%I&drv=%d&drs=%A&shortcode=%P >>> " >>> concatenation = true >>> >>> group = sms-service >>> keyword = bac >>> accept-x-kannel-headers = true >>> catch-all = true >>> max-messages = 4 # it's better to put this parameter to 0 or you will >>> have a lot Ack in your network >>> get-url = " >>> http://site.site.tn/default.php?phone=%p&text=%a&smsc=%i&corp=%s&time=%t&smsid=%I&drv=%d&drs=%A&shortcode=%P >>> " >>> concatenation = true >>> >>> group = sms-service >>> keyword = neuf >>> accept-x-kannel-headers = true >>> #text = "No service specified" >>> catch-all = true >>> max-messages = 4 # it's better to put this parameter to 0 or you will >>> have a lot Ack in your network >>> get-url = " >>> http://site.site.tn/default.php?phone=%p&text=%a&smsc=%i&corp=%s&time=%t&smsid=%I&drv=%d&drs=%A&shortcode=%P >>> " >>> concatenation = true >>> >>> group = sms-service >>> keyword = six >>> accept-x-kannel-headers = true >>> #text = "No service specified" >>> catch-all = true >>> max-messages = 4 # it's better to put this parameter to 0 or you will >>> have a lot Ack in your network >>> >>> get-url = " >>> http://site.site.tn/default.php?phone=%p&text=%a&smsc=%i&corp=%s&time=%t&smsid=%I&drv=%d&drs=%A&shortcode=%P >>> " >>> concatenation = true >>> >>> group = sms-service >>> keyword = ent >>> accept-x-kannel-headers = true >>> catch-all = true >>> max-messages = 4 # it's better to put this parameter to 0 or you will >>> have a lot Ack in your network >>> get-url = " >>> http://site.site.tn/entreceive.php?phone=%p&text=%a&smsc=%i&corp=%s&time=%t&smsid=%I&drv=%d&drs=%B&shortcode=%P >>> " >>> concatenation = true >>> >>> group = sms-service >>> keyword = edu >>> accept-x-kannel-headers = true >>> catch-all = true >>> max-messages = 5 # it's better to put this parameter to 0 or you will >>> have a lot Ack in your network >>> get-url = " >>> http://site.site.tn/edureceive.php?phone=%p&text=%a&smsc=%i&corp=%s&time=%t&smsid=%I&drv=%d&drs=%B&shortcode=%P >>> " >>> concatenation = true >>> >>> >>> >>> >>> >>> >>> >>> >>> thx for your patience >>> >>> >>> >>> ----- Mail Original ----- >>> De: "spameden" <[email protected]> >>> À: "Ahmed BOUDHRAA" <[email protected]> >>> Cc: "DHC Admin" <[email protected]>, [email protected] >>> Envoyé: Lundi 9 Juin 2014 14:40:50 >>> >>> Objet: Re: Tunning up kannel >>> >>> >>> >>> >>> 2014-06-09 17:08 GMT+04:00 Ahmed BOUDHRAA < >>> [email protected]>: >>> >>>> HI >>>> thx for the relpay i was wondering if someone will help :) >>>> the fact is that we tunned the max-pending-requests and we ve concluded >>>> that about 800 is the best value and if we go up the results go worst, but >>>> the smsbox-max-pending we havent added such parametre; can you please >>>> explain it, is it, the smsbox-max-pending, is the queue between the smsbox >>>> and our portal? >>>> >>> >>> I can't imagine why do you need such high rate of sending MT messages >>> for university unless you're spamming your students every second .. >>> >>> The speed also very much depends on your SMSC upstream providers, >>> network link between you and them, TCP RTT (re-transmissions), etc. >>> >>> The general settings for throughput between you and smsc are: >>> >>> 1) throughput -- limits MT/sec >>> 2) max-pending-submits -- unofficial parameter, controls how many >>> "outstanding" operations between you and SMSC (e.g. unacknowledged >>> submit_sm packets) >>> >>> Try tuning these two parameters to get maximum of your upstream SMSC. Do >>> it carefully to not get THROTTLED errors or MAX_QUEUE errors. Consulting >>> your provider is highly recommended. >>> >>> >>> >>> >>> >>>> if i make it more simple: >>>> >>>> what is the difference between the tow variable that you gived to us : >>>> smsbox-max-pending >>>> and max-pending-requests ? and where are they placed, i mean if there >>>> is tow pending queue, i think one is in front of the bearerbox ( can i >>>> assume it is the max-pending-requests ?) and one between the bearerbox >>>> and the smsbox ? ( should it be the smsbox-max-pending ???) if my >>>> assumption are right i could be the bottleneck coz we havent touch such >>>> parametre, and if so the default value is about how much? >>>> >>> >>> check user-guide, it's available on the WEB: >>> http://kannel.org/download/kannel-userguide-snapshot/userguide.html >>> >>> bottleneck is most likely your code in DLR-url parsing script not the >>> kannel itself. >>> >>> try testing with different conditions: >>> >>> 1) without DLR reports at all >>> 2) with DLR but without your url hit >>> 3) with DLR and your URL >>> >>> Also, you might want to consider switching over to sqlbox, so you won't >>> need to drag web-server each time report comes. >>> >>> >>>> >>>> thinx again for the help >>>> >>>> >>>> >>>> ----- Mail Original ----- >>>> De: "DHC Admin" <[email protected]> >>>> À: "Ahmed BOUDHRAA" <[email protected]> >>>> Cc: [email protected] >>>> Envoyé: Lundi 9 Juin 2014 13:50:46 >>>> Objet: Re: Tunning up kannel >>>> >>>> >>>> Hi Ahmed >>>> Have you tweaked the smsbox-max-pending and/or max-pending-requests? >>>> Check for them on the user guide >>>> >>>> >>>> On Wed, Jun 4, 2014 at 11:33 AM, Ahmed BOUDHRAA < >>>> [email protected]> wrote: >>>> >>>>> Hi >>>>> we are using kannel about 2 years in our institution and its woriking >>>>> like a charm. we have high load traffic with 3 operators with 3 kannel >>>>> gatways, our configuration is like this: >>>>> >>>>> - kannel 1 : Operator 1 : VM 2 x CPU 3 Ghz 8 Go Ram >>>>> - kannel 2 : Operator 2 : VM 2 x CPU 3 Ghz 8 Go Ram >>>>> - kannel 3 : Operator 3 : VM 2 x CPU 3 Ghz 8 Go Ram >>>>> - Web Portal: for all kannel : VM 8 x CPU 3 Ghz 32 Go RAM >>>>> - Postgres Database behind the web server : VM 8 x CPU 3 Ghz 32 Go RAM >>>>> >>>>> All those server are running under ESXi hypervisor in SAN >>>>> environnement. Any way we are tunning all the plateform with several tools >>>>> ( ApacheBench for web load and fakesms for kannel) >>>>> we want to profit of all the hardware ressources with optimizing all >>>>> componenents to reach the limits of the hardware, but still are still far >>>>> behind the real capability of the hardware. >>>>> The fact is we have reached 800 sms /s with the operators and its >>>>> fine, but we want more not that we need it right now but just to master >>>>> how >>>>> every things work... >>>>> The WEB/DB side tunning is "well" done due to fact that we have >>>>> "mastered" it in our webs servers but the kannel side is litle different. >>>>> with fakesms we are sending about 1000 sms / s from each kannel so we want >>>>> to manage 3000 sms / s in our portal but all the kannel dosent send them >>>>> with the speed we want even if they are not even litle solicited, we >>>>> noticed that the bottle neck is between the smsbox and the bearerbox we >>>>> dont know exactly but all 1000 sms arrive instantelly to the bearerbox in >>>>> each kannel and a queue is formed between the bearerbox and smsbox, the >>>>> smsbox send to the web server ( the portal ) about 1200 request / s wich >>>>> is >>>>> far from his limit ( we have tunning it to manage 3500 request / s ) I ve >>>>> talked a lot but the main question is how we remove that so called queue >>>>> smsbox/bearebox? to make the kannels send more than 1200 request/s so we >>>>> can reach the limits in all the equippements all the system is running far >>>>> behind his capabilitys. >>>>> thinx for the help >>>>> >>>> >>>> >>> >> >
