I suspect that what Jawaid meant by "naive" was that, in Kamailio, there is not a central distributor thread which aims to buffer, serialise and otherwise order packets prior to distribution into worker processes or threads, as is common in many other multiprocess systems.
I agree that "naive" is perhaps not the best choice of vocabulary for it, and of course there would be performance downsides to such a centralised pipeline. But I think that's what was probably meant. -- Alex > On Dec 6, 2022, at 4:15 AM, Daniel-Constantin Mierla <mico...@gmail.com> > wrote: > > Hello, > actually your expectation that packets should come in order is "naive", just > think about how internet is constructed and IP routing works. In the past it > was easy to reproduce on mobile networks scenarios when sending CANCEL very > quickly after the INVITE resulted in CANCEL arriving first at the proxy, then > the INVITE. > Probably you don't get it in your lab environment where you have sipp on the > same system as the sip server or one network segment away, but over the > internet the packets can get in different order because of network > transmission (different IP routing paths). It is the reason in some cases > there are jitter buffers and sequence numbers (e.g., in RTP and SIP (CSeq)). > In other words, the protocols like RTP or SIP were designed to deal with > these out-of-order packets. > Ans here is a wikipedia short article enumerating what can cause out of order: > - https://en.wikipedia.org/wiki/Out-of-order_delivery > That said, if you missed in the message from mailing list archive that you > linked to, there is a global parameter that should reduce the risk of sending > out of order sip packets to the minimum that can be done from SIP processing > point of view based on call-id, but there are still chances that on multi-cpu > systems the packets read from the network can get to be processed in > different order because of how read on udp sockets is done by kernel/libc and > how cpu scheduler allocate cycles to running application processes. > Cheers, > Daniel > On 05.12.22 19:34, Jawaid Bazyar wrote: >> Hi, >> I have experienced out of order packet processing when testing a simple >> Kamailio config. >> The configuration is as follows, basically: >> request_route{ >> record_route(); >> enum_query(); >> xlog("INVITE ENUM query - To URI $tU"); >> forward(); >> } >> I saw this thread from 2020: >> https://www.mail-archive.com/sr-users@lists.kamailio.org/msg11786.html >> The issue seems to be that kamailio process scheduling is naïve – i.e., >> incoming SIP packets are processed without regard to whether packets >> received before this one, are currently being processed. This means any >> packets after the initial INVITE that require more processing, can get >> reordered. In my test lab I have: >> SIPp – UAC >> Kamailio Proxy >> SIPp – UAS >> The proxy uses enum NAPR lookups to route calls to +13038151000 to the UAS. >> Now, if I do SIPp UAC o SIPp UAS directly, I have no problems – no out of >> order packets. >> It is only when I introduce Kamailio in the middle that I get OOO packets. >> See the images attached: uac-to-proxy, proxy, and proxy-to-uas. >> In this example, Kamailio OOO causes SIPp UAC to fail to “generate audio” – >> because UAC does not see packets in the correct order, I never turns on the >> simulated audio. Calls that have in-order dialogs, work fine, and SIPP UAC >> “pauses” 10 seconds to simulate a phone call. >> So far, the error rate runs from 1/1000 to around 1/200 – pretty bad. >> In the thread, a few things were suggested. >> Have fewer kamailio processes than cores: >> Did not resolve issue. >> Try route_locks_size = 256 >> Did not resolve issue. Though, it did alter it somewhat. >> But, it is not clear to me how this works – would this setting restrict the >> number of open calls to 256? >> Have only one kamailio process: (“children=1”) >> This works. “Works”, I should say, because this would greatly >> restrict total platform scalability to a point where it is probably useless >> for my application. >> I saw the same issue discussed in the OpenSIPS mailing list from 2010, and >> the response was “this is not a bug”. >> Well, I respectfully beg to differ with both OpenSIPS and Kamailio – it IS >> a bug. I don’t think a proxy should reorder packets involved in a call in a >> non-deterministic way. In the Kamailio list thread, Alex Balashov discusses >> the contortions he has to go through to avoid repercussions from this issue. >> Kamailio as-is probably works fine for relatively low-volume operations. >> And a lot of the feedback is “why are out of order packets a problem?” OK, >> sure, in the very specific example given in the 2020 thread, maybe who >> cares. But in my thinking, there is absolutely nothing preventing Kamailio >> from generating much more serious OOO scenarios that would cause calls to >> fail. In my example, Kamailio OOO causes SIPp to fail to “generate audio”. >> Who knows how other SIP stacks will behave? >> But the more one will try to scale Kamailio, the more significantly this >> out of order processing issue will become. >> The solution to this seems very simple and straightforward – put packets to >> be processed into a queue PER Call-ID, or something along those lines. >> In short, the parallelism should be by call, not by packet. >> What say ye? Have I misunderstood something here? >> Cheers, >> Jawaid >> >> __________________________________________________________ >> Kamailio - Users Mailing List - Non Commercial Discussions >> sr-users@lists.kamailio.org >> Important: keep the mailing list in the recipients, do not reply only to the >> sender! >> Edit mailing list options or unsubscribe: >> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >> > -- > Daniel-Constantin Mierla -- www.asipto.com > www.twitter.com/miconda -- www.linkedin.com/in/miconda > __________________________________________________________ > Kamailio - Users Mailing List - Non Commercial Discussions > sr-users@lists.kamailio.org > Important: keep the mailing list in the recipients, do not reply only to the > sender! > Edit mailing list options or unsubscribe: > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users -- Alex Balashov | Principal | Evariste Systems LLC Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/ __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions sr-users@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe: https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
