Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Yes do be careful to think about your TCP implementation we had issues with it as well. so opensips to opensips it works good so far, but any type of fan out OpenSips to many clients or backend servers started to cause problems.buffers, lag in responses, sorry don't remember them all but you will need to think about it a bit. Please let us know what you find might be able to help figure something out. 21.05.2020 00:10 に Maxim Sobolev さんは書きました: Well, beware TCP is second-class citizen in the SIP land, for the very same reasons HTTP is moving away from it in HTTP3/QUIC. -Max On Wed., May 20, 2020, 10:52 p.m. Olle Frimanson, wrote: Thanks for the tips will give it a try to see what happens, but I guess TCP is the solution. Br Olle Skickat från min iPhone 21 maj 2020 kl. 07:41 skrev junkmail : Yea that is it. so if you are doing something like tcpdump udp port 5060 or udp port 5080 etc. you would have to change it to the specific host IP that you are testing with. so more like tcpdump host 10.1.1.1 or something like that to make sure that you see all the traffic. Or I am sure there is a way to tell TCPdump to capture fragments but I am a bit too lazy to look it up. But that was why I was not seeing the fragments. 20.05.2020 15:46 に Maxim Sobolev さんは書きました: Olle, this is what he has been warning you about. See, the fragmentation is done at IP level, so that if your UDP packet gets fragmented, only the first fragment is going to contain a UDP header with a port number. Therefore, if you are using a port number(s) as a capture filter with your tcpdump then you won't see those subsequent fragment(s). You should be using IP with destination address as a filter for example and not UDP with a port number(s). Or combine udp rule with rule that would only match IP fragment(s). -Max On Wed, May 20, 2020 at 12:57 PM Olle Frimanson wrote: Hi thanks for the tip, how dit you find it? I just capture 3 ports in my tcpdump. Br Olle Skickat från min iPhone 20 maj 2020 kl. 19:18 skrev junkmail : Sorry that is what I was trying to let you know. Is that I had thought the same thing that the Fragment was not even sent, but it was just because of the tcpdump filter I had not that it actually wasn't being sent. If you have not try capturing all IP traffic to a host IP and see if you see it. 20.05.2020 11:11 に Olle Frimanson さんは書きました: Hi the issue on my side is that it’s not the network that is the problem the second fragment is not even sent. I also kind on lean to TCP at the moment but it would be good to get a comment from Opensips team on this if and how they setup the sockets and if there is a difference on different routes Br Olle Skickat från min iPhone 20 maj 2020 kl. 17:14 skrev junkmail : Hello, I had run into the same issue. One thing I was a bit mistaken because I was using tcpdump and doing a capture filter of port 5060 or the such. So I was missing the Fragment in my sniff as it does not include the UDP header. Just something to be aware of. But I was having problems specifically traffic inside of GCP < google cloud. As well as traffic traversing the VPN to GCP. I am not certain about what changed for internal to GCP but that started working and now the only thing using TCP is over VPNs. Sorry not a lot of information here. but my best guess is either the firewall/router on my side or Googles is dropping the UDP fragment. I didn't dig into it much further as TCP fixed the issue and this was just a transit between opensisps systems. 19.05.2020 01:21 に o...@zaark.com さんは書きました: Hi, this happens one single opensips instance server it receives the inbound packet fine, then when its send out on the same interface it’s fragmented, so I don’t think it’s network or router switch related. Have seen such problems in the past in virtual environments but this is not the case now. My prime suspect is Centos since it send out the first part of the fragmented packet but not the following part that would complete the packet. But indeed it is a strange bug, since it does not always happen. BR/Olle FRÅN: Users FÖR Giovanni Maruzzelli SKICKAT: den 19 maj 2020 09:13 TILL: OpenSIPS users mailling list ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes Can be a problem of the virtual env, and/or the router/switch... Try substitute real hardware to virtual, and different models of router/switch In a LAN, UDP fragmentation is not supposed to be a problem at all... answered from mobile, please pardon terseness and typos, -giovanni On Tue, May 19, 2020, 08:05 wrote: Thanks for the reply Max, we are doing all we can to make the packets smaller, but before we move over to TCP, which is most likely our next step, I wanted to explore what could be happening. AFAIK the application have some control of this since these are parameters that partly can be set when you open a socket, that’s wh
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Well, beware TCP is second-class citizen in the SIP land, for the very same reasons HTTP is moving away from it in HTTP3/QUIC. -Max On Wed., May 20, 2020, 10:52 p.m. Olle Frimanson, wrote: > Thanks for the tips will give it a try to see what happens, but I guess > TCP is the solution. > > Br Olle > > Skickat från min iPhone > > > 21 maj 2020 kl. 07:41 skrev junkmail : > > > > Yea that is it. > > > > so if you are doing something like tcpdump udp port 5060 or udp port > 5080 etc. you would have to change it to the specific host IP that you are > testing with. > > > > so more like tcpdump host 10.1.1.1 or something like that to make sure > that you see all the traffic. Or I am sure there is a way to tell TCPdump > to capture fragments but I am a bit too lazy to look it up. But that was > why I was not seeing the fragments. > > > > > > > > 20.05.2020 15:46 に Maxim Sobolev さんは書きました: > >> Olle, this is what he has been warning you about. See, the > >> fragmentation is done at IP level, so that if your UDP packet gets > >> fragmented, only the first fragment is going to contain a UDP header > >> with a port number. Therefore, if you are using a port number(s) as a > >> capture filter with your tcpdump then you won't see those subsequent > >> fragment(s). You should be using IP with destination address as a > >> filter for example and not UDP with a port number(s). Or combine udp > >> rule with rule that would only match IP fragment(s). > >> -Max > >>> On Wed, May 20, 2020 at 12:57 PM Olle Frimanson > >>> wrote: > >>> Hi thanks for the tip, how dit you find it? I just capture 3 ports > >>> in my tcpdump. > >>> Br Olle > >>> Skickat från min iPhone > >>>> 20 maj 2020 kl. 19:18 skrev junkmail : > >>>> Sorry that is what I was trying to let you know. Is that I had > >>> thought the same thing that the Fragment was not even sent, but it > >>> was just because of the tcpdump filter I had not that it actually > >>> wasn't being sent. If you have not try capturing all IP traffic to > >>> a host IP and see if you see it. > >>>> 20.05.2020 11:11 に Olle Frimanson さんは書きました: > >>>>> Hi the issue on my side is that it’s not the network that is > >>> the > >>>>> problem the second fragment is not even sent. I also kind on lean > >>> to > >>>>> TCP at the moment but it would be good to get a comment from > >>> Opensips > >>>>> team on this if and how they setup the sockets and if there is a > >>>>> difference on different routes > >>>>> Br Olle > >>>>> Skickat från min iPhone > >>>>>>> 20 maj 2020 kl. 17:14 skrev junkmail : > >>>>>> Hello, I had run into the same issue. One thing I was a bit > >>> mistaken because I was using tcpdump and doing a capture filter of > >>> port 5060 or the such. So I was missing the Fragment in my sniff as > >>> it does not include the UDP header. Just something to be aware of. > >>> But I was having problems specifically traffic inside of GCP < > >>> google cloud. As well as traffic traversing the VPN to GCP. I am > >>> not certain about what changed for internal to GCP but that started > >>> working and now the only thing using TCP is over VPNs. Sorry not a > >>> lot of information here. but my best guess is either the > >>> firewall/router on my side or Googles is dropping the UDP fragment. > >>> I didn't dig into it much further as TCP fixed the issue and this > >>> was just a transit between opensisps systems. > >>>>>> 19.05.2020 01:21 に o...@zaark.com さんは書きました: > >>>>>>> Hi, this happens one single opensips instance server it > >>> receives the > >>>>>>> inbound packet fine, then when its send out on the same > >>> interface > >>>>>>> it’s fragmented, so I don’t think it’s network or router > >>> switch > >>>>>>> related. Have seen such problems in the past in virtual > >>> environments > >>>>>>> but this is not the case now. > >>>>>>> My prime suspect is Centos since it send out the first part of > >>> the > >>>>>>> fragmented packet but not the following part that would > >>> complete the > >>>>>>>
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Thanks for the tips will give it a try to see what happens, but I guess TCP is the solution. Br Olle Skickat från min iPhone > 21 maj 2020 kl. 07:41 skrev junkmail : > > Yea that is it. > > so if you are doing something like tcpdump udp port 5060 or udp port 5080 > etc. you would have to change it to the specific host IP that you are > testing with. > > so more like tcpdump host 10.1.1.1 or something like that to make sure that > you see all the traffic. Or I am sure there is a way to tell TCPdump to > capture fragments but I am a bit too lazy to look it up. But that was why I > was not seeing the fragments. > > > > 20.05.2020 15:46 に Maxim Sobolev さんは書きました: >> Olle, this is what he has been warning you about. See, the >> fragmentation is done at IP level, so that if your UDP packet gets >> fragmented, only the first fragment is going to contain a UDP header >> with a port number. Therefore, if you are using a port number(s) as a >> capture filter with your tcpdump then you won't see those subsequent >> fragment(s). You should be using IP with destination address as a >> filter for example and not UDP with a port number(s). Or combine udp >> rule with rule that would only match IP fragment(s). >> -Max >>> On Wed, May 20, 2020 at 12:57 PM Olle Frimanson >>> wrote: >>> Hi thanks for the tip, how dit you find it? I just capture 3 ports >>> in my tcpdump. >>> Br Olle >>> Skickat från min iPhone >>>> 20 maj 2020 kl. 19:18 skrev junkmail : >>>> Sorry that is what I was trying to let you know. Is that I had >>> thought the same thing that the Fragment was not even sent, but it >>> was just because of the tcpdump filter I had not that it actually >>> wasn't being sent. If you have not try capturing all IP traffic to >>> a host IP and see if you see it. >>>> 20.05.2020 11:11 に Olle Frimanson さんは書きました: >>>>> Hi the issue on my side is that it’s not the network that is >>> the >>>>> problem the second fragment is not even sent. I also kind on lean >>> to >>>>> TCP at the moment but it would be good to get a comment from >>> Opensips >>>>> team on this if and how they setup the sockets and if there is a >>>>> difference on different routes >>>>> Br Olle >>>>> Skickat från min iPhone >>>>>>> 20 maj 2020 kl. 17:14 skrev junkmail : >>>>>> Hello, I had run into the same issue. One thing I was a bit >>> mistaken because I was using tcpdump and doing a capture filter of >>> port 5060 or the such. So I was missing the Fragment in my sniff as >>> it does not include the UDP header. Just something to be aware of. >>> But I was having problems specifically traffic inside of GCP < >>> google cloud. As well as traffic traversing the VPN to GCP. I am >>> not certain about what changed for internal to GCP but that started >>> working and now the only thing using TCP is over VPNs. Sorry not a >>> lot of information here. but my best guess is either the >>> firewall/router on my side or Googles is dropping the UDP fragment. >>> I didn't dig into it much further as TCP fixed the issue and this >>> was just a transit between opensisps systems. >>>>>> 19.05.2020 01:21 に o...@zaark.com さんは書きました: >>>>>>> Hi, this happens one single opensips instance server it >>> receives the >>>>>>> inbound packet fine, then when its send out on the same >>> interface >>>>>>> it’s fragmented, so I don’t think it’s network or router >>> switch >>>>>>> related. Have seen such problems in the past in virtual >>> environments >>>>>>> but this is not the case now. >>>>>>> My prime suspect is Centos since it send out the first part of >>> the >>>>>>> fragmented packet but not the following part that would >>> complete the >>>>>>> packet. >>>>>>> But indeed it is a strange bug, since it does not always >>> happen. >>>>>>> BR/Olle >>>>>>> FRÅN: Users FÖR Giovanni >>>>>>> Maruzzelli >>>>>>> SKICKAT: den 19 maj 2020 09:13 >>>>>>> TILL: OpenSIPS users mailling list >>>>>>> ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes >>>>>>> Can be a problem of the virtual env, and/or the >>> router/switch... >
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Yea that is it. so if you are doing something like tcpdump udp port 5060 or udp port 5080 etc. you would have to change it to the specific host IP that you are testing with. so more like tcpdump host 10.1.1.1 or something like that to make sure that you see all the traffic. Or I am sure there is a way to tell TCPdump to capture fragments but I am a bit too lazy to look it up. But that was why I was not seeing the fragments. 20.05.2020 15:46 に Maxim Sobolev さんは書きました: Olle, this is what he has been warning you about. See, the fragmentation is done at IP level, so that if your UDP packet gets fragmented, only the first fragment is going to contain a UDP header with a port number. Therefore, if you are using a port number(s) as a capture filter with your tcpdump then you won't see those subsequent fragment(s). You should be using IP with destination address as a filter for example and not UDP with a port number(s). Or combine udp rule with rule that would only match IP fragment(s). -Max On Wed, May 20, 2020 at 12:57 PM Olle Frimanson wrote: Hi thanks for the tip, how dit you find it? I just capture 3 ports in my tcpdump. Br Olle Skickat från min iPhone 20 maj 2020 kl. 19:18 skrev junkmail : Sorry that is what I was trying to let you know. Is that I had thought the same thing that the Fragment was not even sent, but it was just because of the tcpdump filter I had not that it actually wasn't being sent. If you have not try capturing all IP traffic to a host IP and see if you see it. 20.05.2020 11:11 に Olle Frimanson さんは書きました: Hi the issue on my side is that it’s not the network that is the problem the second fragment is not even sent. I also kind on lean to TCP at the moment but it would be good to get a comment from Opensips team on this if and how they setup the sockets and if there is a difference on different routes Br Olle Skickat från min iPhone 20 maj 2020 kl. 17:14 skrev junkmail : Hello, I had run into the same issue. One thing I was a bit mistaken because I was using tcpdump and doing a capture filter of port 5060 or the such. So I was missing the Fragment in my sniff as it does not include the UDP header. Just something to be aware of. But I was having problems specifically traffic inside of GCP < google cloud. As well as traffic traversing the VPN to GCP. I am not certain about what changed for internal to GCP but that started working and now the only thing using TCP is over VPNs. Sorry not a lot of information here. but my best guess is either the firewall/router on my side or Googles is dropping the UDP fragment. I didn't dig into it much further as TCP fixed the issue and this was just a transit between opensisps systems. 19.05.2020 01:21 に o...@zaark.com さんは書きました: Hi, this happens one single opensips instance server it receives the inbound packet fine, then when its send out on the same interface it’s fragmented, so I don’t think it’s network or router switch related. Have seen such problems in the past in virtual environments but this is not the case now. My prime suspect is Centos since it send out the first part of the fragmented packet but not the following part that would complete the packet. But indeed it is a strange bug, since it does not always happen. BR/Olle FRÅN: Users FÖR Giovanni Maruzzelli SKICKAT: den 19 maj 2020 09:13 TILL: OpenSIPS users mailling list ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes Can be a problem of the virtual env, and/or the router/switch... Try substitute real hardware to virtual, and different models of router/switch In a LAN, UDP fragmentation is not supposed to be a problem at all... answered from mobile, please pardon terseness and typos, -giovanni On Tue, May 19, 2020, 08:05 wrote: Thanks for the reply Max, we are doing all we can to make the packets smaller, but before we move over to TCP, which is most likely our next step, I wanted to explore what could be happening. AFAIK the application have some control of this since these are parameters that partly can be set when you open a socket, that’s why I wonders if Opensips might use those parameters or not, especially since we have so very different behaviour in different directions. BR/Olle FRÅN: Users FÖR Maxim Sobolev SKICKAT: den 18 maj 2020 22:03 TILL: OpenSIPS users mailling list ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes Smells like a OS/kernel bug to me. There is little application can do in that regard, UDP fragmentation/reassembly happens at much lower layers of the OSI stack. However, as a workaround as long as SIP goes you can try to reduce your SIP signalling packet size by using compact version of SIP headers, as well as dropping headers that are not used. That would save you 100-150 bytes per SIP message perhaps. I don't know if OpenSIP can do that in the proxy mode out of the box though, so you might want to add b2b into the flow. -Max On Mon.,
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Olle, this is what he has been warning you about. See, the fragmentation is done at IP level, so that if your UDP packet gets fragmented, only the first fragment is going to contain a UDP header with a port number. Therefore, if you are using a port number(s) as a capture filter with your tcpdump then you won't see those subsequent fragment(s). You should be using IP with destination address as a filter for example and not UDP with a port number(s). Or combine udp rule with rule that would only match IP fragment(s). -Max On Wed, May 20, 2020 at 12:57 PM Olle Frimanson wrote: > Hi thanks for the tip, how dit you find it? I just capture 3 ports in my > tcpdump. > > Br Olle > > Skickat från min iPhone > > > 20 maj 2020 kl. 19:18 skrev junkmail : > > > > Sorry that is what I was trying to let you know. Is that I had thought > the same thing that the Fragment was not even sent, but it was just because > of the tcpdump filter I had not that it actually wasn't being sent. If you > have not try capturing all IP traffic to a host IP and see if you see it. > > > > > > 20.05.2020 11:11 に Olle Frimanson さんは書きました: > >> Hi the issue on my side is that it’s not the network that is the > >> problem the second fragment is not even sent. I also kind on lean to > >> TCP at the moment but it would be good to get a comment from Opensips > >> team on this if and how they setup the sockets and if there is a > >> difference on different routes > >> Br Olle > >> Skickat från min iPhone > >>>> 20 maj 2020 kl. 17:14 skrev junkmail : > >>> Hello, I had run into the same issue. One thing I was a bit mistaken > because I was using tcpdump and doing a capture filter of port 5060 or the > such. So I was missing the Fragment in my sniff as it does not include the > UDP header. Just something to be aware of. But I was having problems > specifically traffic inside of GCP < google cloud. As well as traffic > traversing the VPN to GCP. I am not certain about what changed for > internal to GCP but that started working and now the only thing using TCP > is over VPNs. Sorry not a lot of information here. but my best guess is > either the firewall/router on my side or Googles is dropping the UDP > fragment. I didn't dig into it much further as TCP fixed the issue and > this was just a transit between opensisps systems. > >>> 19.05.2020 01:21 に o...@zaark.com さんは書きました: > >>>> Hi, this happens one single opensips instance server it receives the > >>>> inbound packet fine, then when its send out on the same interface > >>>> it’s fragmented, so I don’t think it’s network or router switch > >>>> related. Have seen such problems in the past in virtual environments > >>>> but this is not the case now. > >>>> My prime suspect is Centos since it send out the first part of the > >>>> fragmented packet but not the following part that would complete the > >>>> packet. > >>>> But indeed it is a strange bug, since it does not always happen. > >>>> BR/Olle > >>>> FRÅN: Users FÖR Giovanni > >>>> Maruzzelli > >>>> SKICKAT: den 19 maj 2020 09:13 > >>>> TILL: OpenSIPS users mailling list > >>>> ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes > >>>> Can be a problem of the virtual env, and/or the router/switch... > >>>> Try substitute real hardware to virtual, and different models of > >>>> router/switch > >>>> In a LAN, UDP fragmentation is not supposed to be a problem at all... > >>>> answered from mobile, please pardon terseness and typos, > >>>> -giovanni > >>>>> On Tue, May 19, 2020, 08:05 wrote: > >>>>> Thanks for the reply Max, > >>>>> we are doing all we can to make the packets smaller, but before we > >>>>> move over to TCP, which is most likely our next step, I wanted to > >>>>> explore what could be happening. > >>>>> AFAIK the application have some control of this since these are > >>>>> parameters that partly can be set when you open a socket, that’s > >>>>> why I wonders if Opensips might use those parameters or not, > >>>>> especially since we have so very different behaviour in different > >>>>> directions. > >>>>> BR/Olle > >>>>> FRÅN: Users FÖR Maxim Sobolev > >>>>> SKICKAT: den 18 maj 2020 22:03 > >>>>> TILL: OpenSIPS users mailling list > >>>&
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Hi thanks for the tip, how dit you find it? I just capture 3 ports in my tcpdump. Br Olle Skickat från min iPhone > 20 maj 2020 kl. 19:18 skrev junkmail : > > Sorry that is what I was trying to let you know. Is that I had thought the > same thing that the Fragment was not even sent, but it was just because of > the tcpdump filter I had not that it actually wasn't being sent. If you have > not try capturing all IP traffic to a host IP and see if you see it. > > > 20.05.2020 11:11 に Olle Frimanson さんは書きました: >> Hi the issue on my side is that it’s not the network that is the >> problem the second fragment is not even sent. I also kind on lean to >> TCP at the moment but it would be good to get a comment from Opensips >> team on this if and how they setup the sockets and if there is a >> difference on different routes >> Br Olle >> Skickat från min iPhone >>>> 20 maj 2020 kl. 17:14 skrev junkmail : >>> Hello, I had run into the same issue. One thing I was a bit mistaken >>> because I was using tcpdump and doing a capture filter of port 5060 or the >>> such. So I was missing the Fragment in my sniff as it does not include the >>> UDP header. Just something to be aware of. But I was having problems >>> specifically traffic inside of GCP < google cloud. As well as traffic >>> traversing the VPN to GCP. I am not certain about what changed for >>> internal to GCP but that started working and now the only thing using TCP >>> is over VPNs. Sorry not a lot of information here. but my best guess is >>> either the firewall/router on my side or Googles is dropping the UDP >>> fragment. I didn't dig into it much further as TCP fixed the issue and >>> this was just a transit between opensisps systems. >>> 19.05.2020 01:21 に o...@zaark.com さんは書きました: >>>> Hi, this happens one single opensips instance server it receives the >>>> inbound packet fine, then when its send out on the same interface >>>> it’s fragmented, so I don’t think it’s network or router switch >>>> related. Have seen such problems in the past in virtual environments >>>> but this is not the case now. >>>> My prime suspect is Centos since it send out the first part of the >>>> fragmented packet but not the following part that would complete the >>>> packet. >>>> But indeed it is a strange bug, since it does not always happen. >>>> BR/Olle >>>> FRÅN: Users FÖR Giovanni >>>> Maruzzelli >>>> SKICKAT: den 19 maj 2020 09:13 >>>> TILL: OpenSIPS users mailling list >>>> ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes >>>> Can be a problem of the virtual env, and/or the router/switch... >>>> Try substitute real hardware to virtual, and different models of >>>> router/switch >>>> In a LAN, UDP fragmentation is not supposed to be a problem at all... >>>> answered from mobile, please pardon terseness and typos, >>>> -giovanni >>>>> On Tue, May 19, 2020, 08:05 wrote: >>>>> Thanks for the reply Max, >>>>> we are doing all we can to make the packets smaller, but before we >>>>> move over to TCP, which is most likely our next step, I wanted to >>>>> explore what could be happening. >>>>> AFAIK the application have some control of this since these are >>>>> parameters that partly can be set when you open a socket, that’s >>>>> why I wonders if Opensips might use those parameters or not, >>>>> especially since we have so very different behaviour in different >>>>> directions. >>>>> BR/Olle >>>>> FRÅN: Users FÖR Maxim Sobolev >>>>> SKICKAT: den 18 maj 2020 22:03 >>>>> TILL: OpenSIPS users mailling list >>>>> ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes >>>>> Smells like a OS/kernel bug to me. There is little application can >>>>> do in that regard, UDP fragmentation/reassembly happens at much >>>>> lower layers of the OSI stack. >>>>> However, as a workaround as long as SIP goes you can try to reduce >>>>> your SIP signalling packet size by using compact version of SIP >>>>> headers, as well as dropping headers that are not used. That would >>>>> save you 100-150 bytes per SIP message perhaps. I don't know if >>>>> OpenSIP can do that in the proxy mode out of the box though, so you >>>>> might want to add b2b into the fl
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Sorry that is what I was trying to let you know. Is that I had thought the same thing that the Fragment was not even sent, but it was just because of the tcpdump filter I had not that it actually wasn't being sent. If you have not try capturing all IP traffic to a host IP and see if you see it. 20.05.2020 11:11 に Olle Frimanson さんは書きました: Hi the issue on my side is that it’s not the network that is the problem the second fragment is not even sent. I also kind on lean to TCP at the moment but it would be good to get a comment from Opensips team on this if and how they setup the sockets and if there is a difference on different routes Br Olle Skickat från min iPhone 20 maj 2020 kl. 17:14 skrev junkmail : Hello, I had run into the same issue. One thing I was a bit mistaken because I was using tcpdump and doing a capture filter of port 5060 or the such. So I was missing the Fragment in my sniff as it does not include the UDP header. Just something to be aware of. But I was having problems specifically traffic inside of GCP < google cloud. As well as traffic traversing the VPN to GCP. I am not certain about what changed for internal to GCP but that started working and now the only thing using TCP is over VPNs. Sorry not a lot of information here. but my best guess is either the firewall/router on my side or Googles is dropping the UDP fragment. I didn't dig into it much further as TCP fixed the issue and this was just a transit between opensisps systems. 19.05.2020 01:21 に o...@zaark.com さんは書きました: Hi, this happens one single opensips instance server it receives the inbound packet fine, then when its send out on the same interface it’s fragmented, so I don’t think it’s network or router switch related. Have seen such problems in the past in virtual environments but this is not the case now. My prime suspect is Centos since it send out the first part of the fragmented packet but not the following part that would complete the packet. But indeed it is a strange bug, since it does not always happen. BR/Olle FRÅN: Users FÖR Giovanni Maruzzelli SKICKAT: den 19 maj 2020 09:13 TILL: OpenSIPS users mailling list ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes Can be a problem of the virtual env, and/or the router/switch... Try substitute real hardware to virtual, and different models of router/switch In a LAN, UDP fragmentation is not supposed to be a problem at all... answered from mobile, please pardon terseness and typos, -giovanni On Tue, May 19, 2020, 08:05 wrote: Thanks for the reply Max, we are doing all we can to make the packets smaller, but before we move over to TCP, which is most likely our next step, I wanted to explore what could be happening. AFAIK the application have some control of this since these are parameters that partly can be set when you open a socket, that’s why I wonders if Opensips might use those parameters or not, especially since we have so very different behaviour in different directions. BR/Olle FRÅN: Users FÖR Maxim Sobolev SKICKAT: den 18 maj 2020 22:03 TILL: OpenSIPS users mailling list ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes Smells like a OS/kernel bug to me. There is little application can do in that regard, UDP fragmentation/reassembly happens at much lower layers of the OSI stack. However, as a workaround as long as SIP goes you can try to reduce your SIP signalling packet size by using compact version of SIP headers, as well as dropping headers that are not used. That would save you 100-150 bytes per SIP message perhaps. I don't know if OpenSIP can do that in the proxy mode out of the box though, so you might want to add b2b into the flow. -Max On Mon., May 18, 2020, 12:34 p.m. Olle Frimanson, wrote: Hi, We have an issue on our home proxy (opensips 2.4.6), when it receives 200 OK (over UDP) from our Freeswitch and the package size is higher than the MTU size , we sometimes get fragmentation of the UDP packets, but only the first part of the fragmented package is sent to our edge proxy. Is this a known issue or is it a OS bug? We have not yet spotted any pattern on this and in most cases bigger packets with MTU around 1600 bytes gets through without an issue. I can add that in the other direction in the normal request routes we don’t have any issue at all can have packets > 2000 bytes without any issues. Does Opensips use IP_MTU_DISCOVER or how is fragmentation controlled and is it expected to have different behavior in reply routes vs other routes? We use Centos 7 in a virtual server environment. I was hoping someone can share some light on this strange issue. BR/Olle ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Hi the issue on my side is that it’s not the network that is the problem the second fragment is not even sent. I also kind on lean to TCP at the moment but it would be good to get a comment from Opensips team on this if and how they setup the sockets and if there is a difference on different routes Br Olle Skickat från min iPhone > 20 maj 2020 kl. 17:14 skrev junkmail : > > Hello, I had run into the same issue. One thing I was a bit mistaken > because I was using tcpdump and doing a capture filter of port 5060 or the > such. So I was missing the Fragment in my sniff as it does not include the > UDP header. Just something to be aware of. But I was having problems > specifically traffic inside of GCP < google cloud. As well as traffic > traversing the VPN to GCP. I am not certain about what changed for internal > to GCP but that started working and now the only thing using TCP is over > VPNs. Sorry not a lot of information here. but my best guess is either the > firewall/router on my side or Googles is dropping the UDP fragment. I didn't > dig into it much further as TCP fixed the issue and this was just a transit > between opensisps systems. > > > 19.05.2020 01:21 に o...@zaark.com さんは書きました: >> Hi, this happens one single opensips instance server it receives the >> inbound packet fine, then when its send out on the same interface >> it’s fragmented, so I don’t think it’s network or router switch >> related. Have seen such problems in the past in virtual environments >> but this is not the case now. >> My prime suspect is Centos since it send out the first part of the >> fragmented packet but not the following part that would complete the >> packet. >> But indeed it is a strange bug, since it does not always happen. >> BR/Olle >> FRÅN: Users FÖR Giovanni >> Maruzzelli >> SKICKAT: den 19 maj 2020 09:13 >> TILL: OpenSIPS users mailling list >> ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes >> Can be a problem of the virtual env, and/or the router/switch... >> Try substitute real hardware to virtual, and different models of >> router/switch >> In a LAN, UDP fragmentation is not supposed to be a problem at all... >> answered from mobile, please pardon terseness and typos, >> -giovanni >>> On Tue, May 19, 2020, 08:05 wrote: >>> Thanks for the reply Max, >>> we are doing all we can to make the packets smaller, but before we >>> move over to TCP, which is most likely our next step, I wanted to >>> explore what could be happening. >>> AFAIK the application have some control of this since these are >>> parameters that partly can be set when you open a socket, that’s >>> why I wonders if Opensips might use those parameters or not, >>> especially since we have so very different behaviour in different >>> directions. >>> BR/Olle >>> FRÅN: Users FÖR Maxim Sobolev >>> SKICKAT: den 18 maj 2020 22:03 >>> TILL: OpenSIPS users mailling list >>> ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes >>> Smells like a OS/kernel bug to me. There is little application can >>> do in that regard, UDP fragmentation/reassembly happens at much >>> lower layers of the OSI stack. >>> However, as a workaround as long as SIP goes you can try to reduce >>> your SIP signalling packet size by using compact version of SIP >>> headers, as well as dropping headers that are not used. That would >>> save you 100-150 bytes per SIP message perhaps. I don't know if >>> OpenSIP can do that in the proxy mode out of the box though, so you >>> might want to add b2b into the flow. >>> -Max >>> On Mon., May 18, 2020, 12:34 p.m. Olle Frimanson, >>> wrote: >>>> Hi, >>>> We have an issue on our home proxy (opensips 2.4.6), when it >>>> receives 200 OK (over UDP) from our Freeswitch and the package >>>> size is higher than the MTU size , we sometimes get fragmentation >>>> of the UDP packets, but only the first part of the fragmented >>>> package is sent to our edge proxy. Is this a known issue or is it >>>> a OS bug? >>>> We have not yet spotted any pattern on this and in most cases >>>> bigger packets with MTU around 1600 bytes gets through without an >>>> issue. >>>> I can add that in the other direction in the normal request routes >>>> we don’t have any issue at all can have packets > 2000 bytes >>>> without any issues. >>>> Does Opensips use IP_MTU_DISCOVER or
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Hello, I had run into the same issue. One thing I was a bit mistaken because I was using tcpdump and doing a capture filter of port 5060 or the such. So I was missing the Fragment in my sniff as it does not include the UDP header. Just something to be aware of. But I was having problems specifically traffic inside of GCP < google cloud. As well as traffic traversing the VPN to GCP. I am not certain about what changed for internal to GCP but that started working and now the only thing using TCP is over VPNs. Sorry not a lot of information here. but my best guess is either the firewall/router on my side or Googles is dropping the UDP fragment. I didn't dig into it much further as TCP fixed the issue and this was just a transit between opensisps systems. 19.05.2020 01:21 に o...@zaark.com さんは書きました: Hi, this happens one single opensips instance server it receives the inbound packet fine, then when its send out on the same interface it’s fragmented, so I don’t think it’s network or router switch related. Have seen such problems in the past in virtual environments but this is not the case now. My prime suspect is Centos since it send out the first part of the fragmented packet but not the following part that would complete the packet. But indeed it is a strange bug, since it does not always happen. BR/Olle FRÅN: Users FÖR Giovanni Maruzzelli SKICKAT: den 19 maj 2020 09:13 TILL: OpenSIPS users mailling list ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes Can be a problem of the virtual env, and/or the router/switch... Try substitute real hardware to virtual, and different models of router/switch In a LAN, UDP fragmentation is not supposed to be a problem at all... answered from mobile, please pardon terseness and typos, -giovanni On Tue, May 19, 2020, 08:05 wrote: Thanks for the reply Max, we are doing all we can to make the packets smaller, but before we move over to TCP, which is most likely our next step, I wanted to explore what could be happening. AFAIK the application have some control of this since these are parameters that partly can be set when you open a socket, that’s why I wonders if Opensips might use those parameters or not, especially since we have so very different behaviour in different directions. BR/Olle FRÅN: Users FÖR Maxim Sobolev SKICKAT: den 18 maj 2020 22:03 TILL: OpenSIPS users mailling list ÄMNE: Re: [OpenSIPS-Users] UDP fragmentation in reply routes Smells like a OS/kernel bug to me. There is little application can do in that regard, UDP fragmentation/reassembly happens at much lower layers of the OSI stack. However, as a workaround as long as SIP goes you can try to reduce your SIP signalling packet size by using compact version of SIP headers, as well as dropping headers that are not used. That would save you 100-150 bytes per SIP message perhaps. I don't know if OpenSIP can do that in the proxy mode out of the box though, so you might want to add b2b into the flow. -Max On Mon., May 18, 2020, 12:34 p.m. Olle Frimanson, wrote: Hi, We have an issue on our home proxy (opensips 2.4.6), when it receives 200 OK (over UDP) from our Freeswitch and the package size is higher than the MTU size , we sometimes get fragmentation of the UDP packets, but only the first part of the fragmented package is sent to our edge proxy. Is this a known issue or is it a OS bug? We have not yet spotted any pattern on this and in most cases bigger packets with MTU around 1600 bytes gets through without an issue. I can add that in the other direction in the normal request routes we don’t have any issue at all can have packets > 2000 bytes without any issues. Does Opensips use IP_MTU_DISCOVER or how is fragmentation controlled and is it expected to have different behavior in reply routes vs other routes? We use Centos 7 in a virtual server environment. I was hoping someone can share some light on this strange issue. BR/Olle ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Hi, this happens one single opensips instance server it receives the inbound packet fine, then when its send out on the same interface it’s fragmented, so I don’t think it’s network or router switch related. Have seen such problems in the past in virtual environments but this is not the case now. My prime suspect is Centos since it send out the first part of the fragmented packet but not the following part that would complete the packet. But indeed it is a strange bug, since it does not always happen. BR/Olle Från: Users För Giovanni Maruzzelli Skickat: den 19 maj 2020 09:13 Till: OpenSIPS users mailling list Ämne: Re: [OpenSIPS-Users] UDP fragmentation in reply routes Can be a problem of the virtual env, and/or the router/switch... Try substitute real hardware to virtual, and different models of router/switch In a LAN, UDP fragmentation is not supposed to be a problem at all... answered from mobile, please pardon terseness and typos, -giovanni On Tue, May 19, 2020, 08:05 mailto:o...@zaark.com> > wrote: Thanks for the reply Max, we are doing all we can to make the packets smaller, but before we move over to TCP, which is most likely our next step, I wanted to explore what could be happening. AFAIK the application have some control of this since these are parameters that partly can be set when you open a socket, that’s why I wonders if Opensips might use those parameters or not, especially since we have so very different behaviour in different directions. BR/Olle Från: Users mailto:users-boun...@lists.opensips.org> > För Maxim Sobolev Skickat: den 18 maj 2020 22:03 Till: OpenSIPS users mailling list mailto:Users@lists.opensips.org> > Ämne: Re: [OpenSIPS-Users] UDP fragmentation in reply routes Smells like a OS/kernel bug to me. There is little application can do in that regard, UDP fragmentation/reassembly happens at much lower layers of the OSI stack. However, as a workaround as long as SIP goes you can try to reduce your SIP signalling packet size by using compact version of SIP headers, as well as dropping headers that are not used. That would save you 100-150 bytes per SIP message perhaps. I don't know if OpenSIP can do that in the proxy mode out of the box though, so you might want to add b2b into the flow. -Max On Mon., May 18, 2020, 12:34 p.m. Olle Frimanson, mailto:o...@zaark.com> > wrote: Hi, We have an issue on our home proxy (opensips 2.4.6), when it receives 200 OK (over UDP) from our Freeswitch and the package size is higher than the MTU size , we sometimes get fragmentation of the UDP packets, but only the first part of the fragmented package is sent to our edge proxy. Is this a known issue or is it a OS bug? We have not yet spotted any pattern on this and in most cases bigger packets with MTU around 1600 bytes gets through without an issue. I can add that in the other direction in the normal request routes we don’t have any issue at all can have packets > 2000 bytes without any issues. Does Opensips use IP_MTU_DISCOVER or how is fragmentation controlled and is it expected to have different behavior in reply routes vs other routes? We use Centos 7 in a virtual server environment. I was hoping someone can share some light on this strange issue. BR/Olle ___ Users mailing list Users@lists.opensips.org <mailto:Users@lists.opensips.org> http://lists.opensips.org/cgi-bin/mailman/listinfo/users ___ Users mailing list Users@lists.opensips.org <mailto:Users@lists.opensips.org> http://lists.opensips.org/cgi-bin/mailman/listinfo/users ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Can be a problem of the virtual env, and/or the router/switch... Try substitute real hardware to virtual, and different models of router/switch In a LAN, UDP fragmentation is not supposed to be a problem at all... answered from mobile, please pardon terseness and typos, -giovanni On Tue, May 19, 2020, 08:05 wrote: > Thanks for the reply Max, > > > > we are doing all we can to make the packets smaller, but before we move > over to TCP, which is most likely our next step, I wanted to explore what > could be happening. > > > > AFAIK the application have some control of this since these are parameters > that partly can be set when you open a socket, that’s why I wonders if > Opensips might use those parameters or not, especially since we have so > very different behaviour in different directions. > > > > BR/Olle > > > > *Från:* Users *För *Maxim Sobolev > *Skickat:* den 18 maj 2020 22:03 > *Till:* OpenSIPS users mailling list > *Ämne:* Re: [OpenSIPS-Users] UDP fragmentation in reply routes > > > > Smells like a OS/kernel bug to me. There is little application can do in > that regard, UDP fragmentation/reassembly happens at much lower layers of > the OSI stack. > > > > However, as a workaround as long as SIP goes you can try to reduce your > SIP signalling packet size by using compact version of SIP headers, as well > as dropping headers that are not used. That would save you 100-150 bytes > per SIP message perhaps. I don't know if OpenSIP can do that in the proxy > mode out of the box though, so you might want to add b2b into the flow. > > > > -Max > > > > On Mon., May 18, 2020, 12:34 p.m. Olle Frimanson, wrote: > > Hi, > > > > We have an issue on our home proxy (opensips 2.4.6), when it receives 200 > OK (over UDP) from our Freeswitch and the package size is higher than the > MTU size , we sometimes get fragmentation of the UDP packets, but only the > first part of the fragmented package is sent to our edge proxy. Is this a > known issue or is it a OS bug? > > > > We have not yet spotted any pattern on this and in most cases bigger > packets with MTU around 1600 bytes gets through without an issue. > > > > I can add that in the other direction in the normal request routes we > don’t have any issue at all can have packets > 2000 bytes without any > issues. > > Does Opensips use IP_MTU_DISCOVER or how is fragmentation controlled and is > it expected to have different behavior in reply routes vs other routes? > > We use Centos 7 in a virtual server environment. > > > > I was hoping someone can share some light on this strange issue. > > > > BR/Olle > > ___ > Users mailing list > Users@lists.opensips.org > http://lists.opensips.org/cgi-bin/mailman/listinfo/users > > ___ > Users mailing list > Users@lists.opensips.org > http://lists.opensips.org/cgi-bin/mailman/listinfo/users > ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Thanks for the reply Max, we are doing all we can to make the packets smaller, but before we move over to TCP, which is most likely our next step, I wanted to explore what could be happening. AFAIK the application have some control of this since these are parameters that partly can be set when you open a socket, that’s why I wonders if Opensips might use those parameters or not, especially since we have so very different behaviour in different directions. BR/Olle Från: Users För Maxim Sobolev Skickat: den 18 maj 2020 22:03 Till: OpenSIPS users mailling list Ämne: Re: [OpenSIPS-Users] UDP fragmentation in reply routes Smells like a OS/kernel bug to me. There is little application can do in that regard, UDP fragmentation/reassembly happens at much lower layers of the OSI stack. However, as a workaround as long as SIP goes you can try to reduce your SIP signalling packet size by using compact version of SIP headers, as well as dropping headers that are not used. That would save you 100-150 bytes per SIP message perhaps. I don't know if OpenSIP can do that in the proxy mode out of the box though, so you might want to add b2b into the flow. -Max On Mon., May 18, 2020, 12:34 p.m. Olle Frimanson, mailto:o...@zaark.com> > wrote: Hi, We have an issue on our home proxy (opensips 2.4.6), when it receives 200 OK (over UDP) from our Freeswitch and the package size is higher than the MTU size , we sometimes get fragmentation of the UDP packets, but only the first part of the fragmented package is sent to our edge proxy. Is this a known issue or is it a OS bug? We have not yet spotted any pattern on this and in most cases bigger packets with MTU around 1600 bytes gets through without an issue. I can add that in the other direction in the normal request routes we don’t have any issue at all can have packets > 2000 bytes without any issues. Does Opensips use IP_MTU_DISCOVER or how is fragmentation controlled and is it expected to have different behavior in reply routes vs other routes? We use Centos 7 in a virtual server environment. I was hoping someone can share some light on this strange issue. BR/Olle ___ Users mailing list Users@lists.opensips.org <mailto:Users@lists.opensips.org> http://lists.opensips.org/cgi-bin/mailman/listinfo/users ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Re: [OpenSIPS-Users] UDP fragmentation in reply routes
Smells like a OS/kernel bug to me. There is little application can do in that regard, UDP fragmentation/reassembly happens at much lower layers of the OSI stack. However, as a workaround as long as SIP goes you can try to reduce your SIP signalling packet size by using compact version of SIP headers, as well as dropping headers that are not used. That would save you 100-150 bytes per SIP message perhaps. I don't know if OpenSIP can do that in the proxy mode out of the box though, so you might want to add b2b into the flow. -Max On Mon., May 18, 2020, 12:34 p.m. Olle Frimanson, wrote: > Hi, > > > > We have an issue on our home proxy (opensips 2.4.6), when it receives 200 > OK (over UDP) from our Freeswitch and the package size is higher than the > MTU size , we sometimes get fragmentation of the UDP packets, but only the > first part of the fragmented package is sent to our edge proxy. Is this a > known issue or is it a OS bug? > > > > We have not yet spotted any pattern on this and in most cases bigger > packets with MTU around 1600 bytes gets through without an issue. > > > > I can add that in the other direction in the normal request routes we > don’t have any issue at all can have packets > 2000 bytes without any > issues. > > Does Opensips use IP_MTU_DISCOVER or how is fragmentation controlled and is > it expected to have different behavior in reply routes vs other routes? > > We use Centos 7 in a virtual server environment. > > > > I was hoping someone can share some light on this strange issue. > > > > BR/Olle > ___ > Users mailing list > Users@lists.opensips.org > http://lists.opensips.org/cgi-bin/mailman/listinfo/users > ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
