Good to hear, I’ll start increasing the children to see what happens. It’s a good start point.
On the other hand, we have different Kamailio implementations where no http or any kind of ‘external integration’ is involved and we get a lot more calls, it’s very clear where the problem is :) Thanks for you help (again) Enviado desde dispositivo móvil > El 25 ago 2024, a la(s) 1:47 a. m., Alex Balashov via sr-users > <[email protected]> escribió: > > A few hundred CPS per node is impressive, but especially so when anything > with HTTP is involved! Kamailio has HTTP client modules, but there's no > pretending that being an HTTP client is natural or particularly performant > for Kamailio. > > Since most--not all, but most--of the workload is I/O wait, you could get > away with just adding more children. The general principle that children > should not exceed CPUs, because they'd just fight for CPU, applies when the > workload is CPU-bound, or computational in nature. However, if most of the > time is spent waiting on something to come across a socket, e.g. from a > database or an HTTP server, you can overbook quite a bit relative to the CPUs. > > You still don't want too many child processes, and I would be hard-pressed to > say exactly how many is too many, but 2:1 or 3:1 should be safe. > > -- Alex > >> On Aug 24, 2024, at 10:35 PM, Alexis Fidalgo <[email protected]> wrote: >> >> it is. im dealing with this issue since a few weeks, i can push and add more >> cpu’s and more children and play with queues, timeouts, etc etc. but i know, >> im pretty sure that there’s a limit i can't surpass. >> >> at this point im struggling on how to modify the http part (the api server) >> it would be great (and easy for me) if the execution pipelines can at least >> have some part where in can execute in parallel, but … a pipeline, why to >> execute following steps if not needed vs ‘execute and discard’ (and worst, >> some consumed webservices in the pipeline are transactional charged). >> >> im happy, more that that, a docker swarm with 4 nodes with 10cpu/children >> each are handling ~1300 call attempls (invite,100,302,ack) per second, thats >> more than ok (and cpu/mem are low, really low, problem is the wait, no the >> power), but we need more (ill move to more cores/childrens and appeal to >> brute force by now to gain some time) >> >> still seeing there’s no point to start using TM for a 100% stateless flow >> >> >>>> On 24 Aug 2024, at 9:02 PM, Alex Balashov via sr-users >>>> <[email protected]> wrote: >>> >>> Not overtly related to your most immediate question, but: >>> >>> 1) "we're hitting a limit where all children become busy waiting for the >>> API to answer." >>> >>> 2) "So i decided to move to http_async_client" >>> >>> I'm not sure this is really going to solve your problem. You've hit >>> fundamental, thermodynamic kind of limits here. Using async here just >>> squeezes the balloon in one place and causes it to inflate in another. >>> >>> It would be different if some of your workload were HTTP-dependent and >>> other parts of the workload were not. Doing the HTTP queries would free up >>> the core SIP workers to process other kinds of requests. That doesn't sound >>> like it's the case, so all you're really liberating is reply processing, >>> which, if this is a redirect server, is nonexistent anyway. >>> >>> This matter might occasion some deeper reflection. >>> >>> -- Alex >>> >>>> On Aug 24, 2024, at 6:48 PM, alexis via sr-users >>>> <[email protected]> wrote: >>>> >>>> Hello all, context first, we have an REST API that performs queries to >>>> external devices in the network (diameter to DRA's, REST to different >>>> servers) and based on n conditions returns the content for a Contact >>>> header to be used in a SIP 302. >>>> >>>> Now we're consuming this API with http_client (synchronously) and as >>>> there's no way to speed up the API (pipeline executions, delays on >>>> external api's etc etc) we're hitting a limit where all children become >>>> busy waiting for the API to answer. >>>> >>>> So i decided to move to http_async_client and started working on it on the >>>> lab with this first and base concept to test. >>>> >>>> request_route { >>>> >>>> >>>> #for testing purposes only >>>> if(is_method("ACK")){ >>>> exit; >>>> } >>>> $http_req(all) = $null; >>>> $http_req(suspend) = 1; >>>> $http_req(timeout) = 500; >>>> $http_req(method) = "POST"; >>>> $http_req(hdr) = "Content-Type: application/json"; >>>> jansson_set("string", "event", "sip-routing", "$var(cre_query)"); >>>> xlog("L_INFO","API ASYNC ROUTING REQUEST: $var(cre_query)\n"); >>>> $http_req(body) = $var(cre_query); >>>> t_newtran(); >>>> http_async_query("http://192.168.86.128:8000/";, "CRE_RESPONSE"); >>>> } >>>> >>>> http://192.168.86.128:8000/ receives the POST, randomly creates a delay >>>> between 0.5 and 1 second and responds (simulating the real api with an >>>> excess delay to probe the concept) >>>> >>>> Then >>>> >>>> route[CRE_RESPONSE] { >>>> if ($http_ok && $http_rs == 200) { >>>> xlog("L_INFO","CRE RESPONSE: $http_rb\n"); >>>> # for testing purpose, Contact content will be replaced from the received >>>> api response >>>> append_to_reply("Contact: <sip:[email protected]>\r\n"); >>>> send_reply(302,"Moved Temporarily"); >>>> exit; >>>> } >>>> send_reply(500, "Internal error"); >>>> exit; >>>> } >>>> >>>> INVITE is received and processed, API is called, after API response, 302 >>>> is replied and then an ACK (ignored by now). >>>> >>>> Situation is that the 302 retransmitted >>>> >>>> 37 1519.846253067 192.168.86.34 → 192.168.86.128 SIP/SDP 585 Request: >>>> INVITE sip:[email protected]:5060 | >>>> 38 1519.848100380 192.168.86.128 → 192.168.86.34 SIP 318 Status: 100 >>>> Trying | >>>> 39 1520.094997642 192.168.86.128 → 192.168.86.34 SIP 407 Status: 302 Moved >>>> Temporarily | >>>> 40 1520.102323728 192.168.86.34 → 192.168.86.128 SIP 453 Request: ACK >>>> sip:[email protected]:5060 | >>>> 41 1520.591300933 192.168.86.128 → 192.168.86.34 SIP 407 Status: 302 Moved >>>> Temporarily | >>>> 42 1521.591061065 192.168.86.128 → 192.168.86.34 SIP 407 Status: 302 Moved >>>> Temporarily | >>>> 43 1523.591227956 192.168.86.128 → 192.168.86.34 SIP 407 Status: 302 Moved >>>> Temporarily | >>>> >>>> >>>> 18(24) DEBUG: tm [t_reply.c:1703]: t_retransmit_reply(): reply >>>> retransmitted. buf=0x7f6d79745dc0: SIP/2.0 3..., shmem=0x7f6d75187fd8: >>>> SIP/2.0 3 >>>> 18(24) DEBUG: tm [t_reply.c:1703]: t_retransmit_reply(): reply >>>> retransmitted. buf=0x7f6d79745dc0: SIP/2.0 3..., shmem=0x7f6d75187fd8: >>>> SIP/2.0 3 >>>> 18(24) DEBUG: tm [t_reply.c:1703]: t_retransmit_reply(): reply >>>> retransmitted. buf=0x7f6d79745dc0: SIP/2.0 3..., shmem=0x7f6d75187fd8: >>>> SIP/2.0 3 >>>> 18(24) DEBUG: tm [timer.c:634]: wait_handler(): finished transaction: >>>> 0x7f6d75184cc8 (p:0x7f6d74f600c8/n:0x7f6d74f600c8) >>>> 18(24) DEBUG: tm [h_table.c:132]: free_cell_helper(): freeing transaction >>>> 0x7f6d75184cc8 from timer.c:643 >>>> >>>> Any help to avoid the retransmission and make the transaction just finish >>>> right after the 302 will be appreciated. >>>> >>>> regards >>>> >>>> >>>> >>>> >>>> >>>> __________________________________________________________ >>>> Kamailio - Users Mailing List - Non Commercial Discussions >>>> To unsubscribe send an email to [email protected] >>>> Important: keep the mailing list in the recipients, do not reply only to >>>> the sender! >>>> Edit mailing list options or unsubscribe: >>> >>> -- >>> Alex Balashov >>> Principal Consultant >>> Evariste Systems LLC >>> Web: https://evaristesys.com >>> Tel: +1-706-510-6800 >>> >>> __________________________________________________________ >>> Kamailio - Users Mailing List - Non Commercial Discussions >>> To unsubscribe send an email to [email protected] >>> Important: keep the mailing list in the recipients, do not reply only to >>> the sender! >>> Edit mailing list options or unsubscribe: >> > > -- > Alex Balashov > Principal Consultant > Evariste Systems LLC > Web: https://evaristesys.com > Tel: +1-706-510-6800 > > __________________________________________________________ > Kamailio - Users Mailing List - Non Commercial Discussions > To unsubscribe send an email to [email protected] > Important: keep the mailing list in the recipients, do not reply only to the > sender! > Edit mailing list options or unsubscribe: __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to [email protected] Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
