Fix implemented on trunk, 3.1 and 3.0. We should update the fix on rtpproxy to match rtpengine and move the OSIP_IOV_MAX define somewhere upper in the code tree to be visible by both rtpproxy and rtpengine modules. For trunk, maybe we should add a new param.
Should we backport this fix to 2.4? Thanks, Ovidiu On Fri, Jun 26, 2020 at 9:35 AM Ovidiu Sas <[email protected]> wrote: > > Hello Razvan, > > On ubuntu we have a failure for more then 1024 buffers. There is something > defined in some experimental headers (as __IOV_MAX), but the IOV_MAX define > is not visible in the rtpengine module. > > And yes, there is an extra buffer. I guess the length of that extra buffer > was zero and that’s why it didn’t create issues. But as soon as I > concatenated the buffer, the issue showed up and I had an extra trailing > string at the end of the rtpengine command that was sent on the wire. > After reducing the number of buffers by one, all was good on both scenarios: > sending with and without concatenated buffers. > I will prepare a fix for it and push it. > > -ovidiu > > On Fri, Jun 26, 2020 at 03:55 Răzvan Crainea <[email protected]> wrote: >> >> Hi, Ovidiu! >> >> So you're saying that the IOV_MAX is not explicitly defined, but it does >> fail after 1024 buffers, correct? If so, perhaps we should limit the >> number of buffers to 1024, if not already defined. >> I did notice the extra vcnt as well, but I though that was related to >> the fact that it allocates one extra iovec (at the headr) for the >> cookie, and sometime uses it, sometime doesn't. Nevertheless, it is >> consistent on both cases, UNIX & UDP, both have that vcnt + 1. Now I >> haven't checked whether this is correct or not, could you please confirm? >> >> Best regards, >> >> Răzvan Crainea >> OpenSIPS Core Developer >> http://www.opensips-solutions.com >> >> On 6/26/20 2:29 AM, Ovidiu Sas wrote: >> > It looks like the hidden IOV_MAX is set to 1024 in debian. >> > I tested a patch and all looks good. >> > It seems that there is an error in the code: writev has the wrong >> > number of iovcnt (should be one less). >> > I tested and all looks ok (the code in rtpproxy.c has the proper iovcnt). >> > >> > -ovidiu >> > >> > On Thu, Jun 25, 2020 at 1:03 PM Ovidiu Sas <[email protected]> wrote: >> >> >> >> We will need to add a param to control the max number of buffers. >> >> >> >> -ovidiu >> >> >> >> On Thu, Jun 25, 2020 at 1:02 PM Ovidiu Sas <[email protected]> wrote: >> >>> >> >>> It seems that when we have more than roughly 1000 buffers, the send >> >>> fails. >> >>> >> >>> -ovidiu >> >>> >> >>> On Wed, Jun 24, 2020 at 8:54 AM Ovidiu Sas <[email protected]> wrote: >> >>>> >> >>>> Hello Razvan, >> >>>> >> >>>> The system is a debian buster one. >> >>>> I patched the code: >> >>>> #ifdef IOV_MAX >> >>>> LM_NOTICE("IOV_MAX=[%d]\n", IOV_MAX); >> >>>> #else >> >>>> LM_NOTICE("no IOV_MAX\n"); >> >>>> #endif >> >>>> >> >>>> and I get: >> >>>> NOTICE:rtpengine:send_rtpe_command: no IOV_MAX >> >>>> in the logs. >> >>>> >> >>>> Then I patched the code again to check how many buffers are being used: >> >>>> The max so far was 73: >> >>>> LM_NOTICE("writev(rtpe_socks[node->idx], v , %d)\n", vcnt + 1); >> >>>> got me: >> >>>> NOTICE:rtpengine:send_rtpe_command: writev(rtpe_socks[node->idx], v , >> >>>> 73) >> >>>> >> >>>> I will continue to monitor the system to see if there is a correlation >> >>>> between the error and the number of buffers. >> >>>> >> >>>> Thanks, >> >>>> Ovidiu >> >>>> >> >>>> On Wed, Jun 24, 2020 at 1:59 AM Răzvan Crainea <[email protected]> >> >>>> wrote: >> >>>>> >> >>>>> Hi, Ovidiu! >> >>>>> >> >>>>> I doubt this is a problem of OpenSIPS version, but rather of the OS you >> >>>>> are running on. I suspect that error comes from the fact that the bson >> >>>>> resulted has more than IOV_MAX elements, which if I recall correctly it >> >>>>> was 15 on some OSes. >> >>>>> We had a similar problem in rtpproy[1], where we had merged the buffers >> >>>>> in a single one just to pass over this limitation. Could you check if >> >>>>> you are facing a similar issue? >> >>>>> >> >>>>> [1] >> >>>>> https://github.com/OpenSIPS/opensips/blob/master/modules/rtpproxy/rtpproxy.c#L2031 >> >>>>> >> >>>>> Răzvan Crainea >> >>>>> OpenSIPS Core Developer >> >>>>> http://www.opensips-solutions.com >> >>>>> >> >>>>> On 6/24/20 7:32 AM, Ovidiu Sas wrote: >> >>>>>> This is happening also on the latest 3.0. >> >>>>>> The weird thing is that opensips doesn't send anything to rtpengine. >> >>>>>> The first opensips/rtpengine exchange on the initial INVITE works ok, >> >>>>>> but the opensips/rtpengine exchange on the 200ok fails (no command is >> >>>>>> sent by opensips - confirmed by running ngrep on the loopback >> >>>>>> interface). >> >>>>>> On the next call, the initial offer works fine, but the answer fails >> >>>>>> due to no command issued by opensips. >> >>>>>> >> >>>>>> version: opensips 3.0.2 (x86_64/linux) >> >>>>>> flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, >> >>>>>> Q_MALLOC, F_MALLOC, HP_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT >> >>>>>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, >> >>>>>> MAX_URI_SIZE 1024, BUF_SIZE 65535 >> >>>>>> poll method support: poll, epoll, sigio_rt, select. >> >>>>>> git revision: 3a8f6f137 >> >>>>>> main.c compiled on 09:36:42 Jun 22 2020 with gcc 9 >> >>>>>> >> >>>>>> -ovidiu >> >>>>>> >> >>>>>> >> >>>>>> On Wed, Jun 10, 2020 at 2:49 PM Ovidiu Sas <[email protected]> >> >>>>>> wrote: >> >>>>>>> >> >>>>>>> Hello all, >> >>>>>>> >> >>>>>>> I'm running opensips 3.1.0-beta (latest version) and experiencing >> >>>>>>> connectivity issues to the rtpengine daemon running on the same host: >> >>>>>>> ERROR:rtpengine:send_rtpe_command: can't send command to a RTP proxy >> >>>>>>> (22:Invalid argument) >> >>>>>>> ERROR:rtpengine:send_rtpe_command: timeout waiting reply from a RTP >> >>>>>>> proxy >> >>>>>>> ERROR:rtpengine:send_rtpe_command: proxy <udp:127.0.0.1:2223> does >> >>>>>>> not >> >>>>>>> respond, disable it >> >>>>>>> ERROR:rtpengine:rtpe_function_call: no available proxies >> >>>>>>> >> >>>>>>> After an opensips restart, everything comes to normal. >> >>>>>>> I was running previously opensips 3.1.0-dev and everything was >> >>>>>>> working fine. >> >>>>>>> >> >>>>>>> The issue starts showing up after a few days with very little >> >>>>>>> traffic. >> >>>>>>> Is there anyone experiencing this issue? >> >>>>>>> >> >>>>>>> >> >>>>>>> Thanks, >> >>>>>>> Ovidiu >> >>>>>>> >> >>>>>>> -- >> >>>>>>> VoIP Embedded, Inc. >> >>>>>>> http://www.voipembedded.com >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>> >> >>>>> _______________________________________________ >> >>>>> Users mailing list >> >>>>> [email protected] >> >>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> VoIP Embedded, Inc. >> >>>> http://www.voipembedded.com >> >>> >> >>> >> >>> >> >>> -- >> >>> VoIP Embedded, Inc. >> >>> http://www.voipembedded.com >> >> >> >> >> >> >> >> -- >> >> VoIP Embedded, Inc. >> >> http://www.voipembedded.com >> > >> > >> > >> >> _______________________________________________ >> Users mailing list >> [email protected] >> http://lists.opensips.org/cgi-bin/mailman/listinfo/users > > -- > VoIP Embedded, Inc. > http://www.voipembedded.com -- VoIP Embedded, Inc. http://www.voipembedded.com _______________________________________________ Users mailing list [email protected] http://lists.opensips.org/cgi-bin/mailman/listinfo/users
