Re: [Sip-implementors] Reinvite/ACK race

[Alex Balashov]

On Tue, Oct 31, 2017 at 10:38:00PM -0400, Dale R. Worley wrote:

> As others have said, the problem must be solved by the UAs, not the
> proxy.  Indeed, what with possible network delays and re-sending of
> lost packets, there's no way to guarantee end-to-end sequencing of
> messages.  Even with TCP service, there's no guarantee that none of
> the intermediate proxies aren't using multiple TCP connections to
> carry messages between themselves.

That's certainly true. But in this case I do control the entire setup
topology end-to-end, and implicit in my thesis was the desire to assert
control over that over which control can be asserted. :-) 

> You're correct to look to 5407 (which is important enough to be known as
> just "Hasebe") for guidance.  And it does look like section 3.1.4 is the
> relevant case, or very close to it.
> 
>This scenario illustrates the race condition that occurs when a UAS
>in the Moratorium state [having sent 2xx but not received ACK]
>receives a re-INVITE sent by a UAC in the Established state.
> 
> And as the text goes on, sending 2xx is recommended, rather than 491,
> because all of the state changes within the UA associated with the first
> re-INVITE have already been completed.  But a 491 response is also
> permissible.

The reason that I did not think § 3.1.4 was exactly pertinent was due to
this formulation:

   The UAS receives a re-INVITE (with offer2) before receiving an ACK
   for the ini-INVITE (with offer1).  The UAS sends a 200 OK (with
   answer2) to the re-INVITE (F8) because it has sent a 200 OK (with
   answer1) to the ini-INVITE (F3, F5) and the dialog has already been
   established.  (Because F5 is a retransmission of F3, SDP negotiation
   is not performed here.)

The scenario described in the foregoing is for a reinvite occuring after
an *initial* INVITE cycle but arriving before the ACK for that initial
INVITE. 

The scenario I've got going on is a reinvite following another reinvite.

As you suggest, there are few reasons to think that the reasoning should
be different, or at any rate substantially different, for a consecutive
reinvite vs. a reinvite following an initial INVITE. Nevertheless, that
was what motivated the perception that § 3.1.4 may not speak to the fine
letter of the exact scenario in play.

-- Alex

-- 
Alex Balashov | Principal | Evariste Systems LLC

Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) 
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
___
Sip-implementors mailing list
Sip-implementors@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors

Re: [Sip-implementors] Reinvite/ACK race

[Dale R. Worley]

Alex Balashov  writes:
> 5. UA B sends end-to-end ACK for reinvite #1 and almost
>simultaneously sends reinvite #2. The temporal delta is
>between reinvite #2 and ACK for reinvite #1 on the wire
>is 3 ms.
>
> The issue is that the concurrency characteristics of proxy P are such
> that its worker threads are very loosely coupled, and there's no
> synchronisation among them for message ordering. Transport is UDP,
> naturally.

As others have said, the problem must be solved by the UAs, not the
proxy.  Indeed, what with possible network delays and re-sending of lost
packets, there's no way to guarantee end-to-end sequencing of messages.
Even with TCP service, there's no guarantee that none of the
intermediate proxies aren't using multiple TCP connections to carry
messages between themselves.

> So, the result -- for all kinds of stochastic processing and userspace
> scheduling type reasons -- is that the reinvite is forwarded first,
> before the ACK.  That leads to a 500 / 491 scenario UA A.
>
> Is there any general guidance on what to do with these scenarios? I
> looked at RFC 5407 [section] 3.1.4, which appears to describe a
> similar, but not identical scenario involving an initial INVITE and
> subsequent reinvite.  As far as I can tell, the recommendation in that
> standard is "space the messaging out more in time".

You're correct to look to 5407 (which is important enough to be known as
just "Hasebe") for guidance.  And it does look like section 3.1.4 is the
relevant case, or very close to it.

   This scenario illustrates the race condition that occurs when a UAS
   in the Moratorium state [having sent 2xx but not received ACK]
   receives a re-INVITE sent by a UAC in the Established state.

And as the text goes on, sending 2xx is recommended, rather than 491,
because all of the state changes within the UA associated with the first
re-INVITE have already been completed.  But a 491 response is also
permissible.

Dale
___
Sip-implementors mailing list
Sip-implementors@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors

Re: [Sip-implementors] Reinvite/ACK race

[Alex Balashov]

Paul,

I'm not a SIP standards expert by any stretch, but yours is the interpretation 
I tend to favour and which sits best with me.

Since in-dialog requests are originated by the dialog parties, timing issues 
belong with them, not with intermediate proxies IMHO. 

On October 31, 2017 10:50:45 AM EDT, Paul Kyzivat  wrote:
>On 10/31/17 8:44 AM, Liviu Chircu wrote:
>> Hi Alex,
>> 
>> IMO, building logic into the proxy which encourages/mends the proper 
>> sequencing of UDP messages does nothing more than to hide the
>underlying 
>> problem, i.e. "UDP does not guarantee message sequencing in the first
>
>> place" *. There is also a subtle point to be made here: your proxy's 
>> loosely coupled/parallelized way of handling the two transactions is 
>> effectively breaking the RFC some % of the time, since the proxy's TU
>is 
>> generating a new INVITE while another INVITE transaction is in
>progress 
>> (strictly judging by the network timestamps).
>
>One can argue that the proxy is doing nothing wrong and perhaps being 
>helpful by exposing bugs in handling of reordered UDP messages.
>
>   Thanks,
>   Paul
>
>> * From this angle, the only sane thing left to do is to have the
>proxy 
>> retry the re-INVITE for UASs who are properly responding with 491 and
>
>> propagate any other 4xx/5xx error replies back to the UAC (UA B in
>our 
>> case) in the hope that they retry their re-INVITE.
>> 
>> Best regards,
>> 
>> Liviu Chircu
>> OpenSIPS Developer
>> http://www.opensips-solutions.com
>> 
>> On 30.10.2017 19:11, Alex Balashov wrote:
>>> Hi,
>>>
>>> I've got a scenario like so:
>>>
>>>     UA A -> Proxy P > UA B
>>>
>>> 1. UA A initiates call through Proxy P;
>>>
>>> 2. Dialog is established and confirmed, with Record-Route;
>>>
>>> 3. UA B sends reinvite #1 through P to A;
>>>
>>> 4. UA B sends 2xx reply;
>>>
>>> 5. UA B sends end-to-end ACK for reinvite #1 and almost
>>>     simultaneously sends reinvite #2. The temporal delta is
>>>     between reinvite #2 and ACK for reinvite #1 on the wire
>>>     is 3 ms.
>>>
>>> The issue is that the concurrency characteristics of proxy P are
>such
>>> that its worker threads are very loosely coupled, and there's no
>>> synchronisation among them for message ordering. Transport is UDP,
>>> naturally.
>>>
>>> So, the result — for all kinds of stochastic processing and
>userspace
>>> scheduling type reasons — is that the reinvite is forwarded first,
>>> before the ACK.  That leads to a 500 / 491 scenario UA A.
>>>
>>> Is there any general guidance on what to do with these scenarios? I
>>> looked at RFC 5407 § 3.1.4, which appears to describe a similar, but
>not
>>> identical scenario involving an initial INVITE and subsequent
>reinvite.
>>> As far as I can tell, the recommendation in that standard is "space
>the
>>> messaging out more in time".
>>>
>>> Switching to TCP would presumably help, since any given flow would
>>> involve a single connection to a single worker thread and the
>transport
>>> would guarantee ordering. However, that's not really feasible in
>this
>>> implementation for a host of reasons.
>>>
>>> Any other thoughts welcome!
>>>
>>> Cheers,
>>>
>>> -- Alex
>>>
>> 
>> ___
>> Sip-implementors mailing list
>> Sip-implementors@lists.cs.columbia.edu
>> https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors
>
>___
>Sip-implementors mailing list
>Sip-implementors@lists.cs.columbia.edu
>https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors


-- Alex

--
Sent via mobile, please forgive typos and brevity. 
___
Sip-implementors mailing list
Sip-implementors@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors

Re: [Sip-implementors] Reinvite/ACK race

[Paul Kyzivat]

On 10/31/17 8:44 AM, Liviu Chircu wrote:

Hi Alex,

IMO, building logic into the proxy which encourages/mends the proper 
sequencing of UDP messages does nothing more than to hide the underlying 
problem, i.e. "UDP does not guarantee message sequencing in the first 
place" *. There is also a subtle point to be made here: your proxy's 
loosely coupled/parallelized way of handling the two transactions is 
effectively breaking the RFC some % of the time, since the proxy's TU is 
generating a new INVITE while another INVITE transaction is in progress 
(strictly judging by the network timestamps).


One can argue that the proxy is doing nothing wrong and perhaps being 
helpful by exposing bugs in handling of reordered UDP messages.


Thanks,
Paul

* From this angle, the only sane thing left to do is to have the proxy 
retry the re-INVITE for UASs who are properly responding with 491 and 
propagate any other 4xx/5xx error replies back to the UAC (UA B in our 
case) in the hope that they retry their re-INVITE.


Best regards,

Liviu Chircu
OpenSIPS Developer
http://www.opensips-solutions.com

On 30.10.2017 19:11, Alex Balashov wrote:

Hi,

I've got a scenario like so:

    UA A -> Proxy P > UA B

1. UA A initiates call through Proxy P;

2. Dialog is established and confirmed, with Record-Route;

3. UA B sends reinvite #1 through P to A;

4. UA B sends 2xx reply;

5. UA B sends end-to-end ACK for reinvite #1 and almost
    simultaneously sends reinvite #2. The temporal delta is
    between reinvite #2 and ACK for reinvite #1 on the wire
    is 3 ms.

The issue is that the concurrency characteristics of proxy P are such
that its worker threads are very loosely coupled, and there's no
synchronisation among them for message ordering. Transport is UDP,
naturally.

So, the result — for all kinds of stochastic processing and userspace
scheduling type reasons — is that the reinvite is forwarded first,
before the ACK.  That leads to a 500 / 491 scenario UA A.

Is there any general guidance on what to do with these scenarios? I
looked at RFC 5407 § 3.1.4, which appears to describe a similar, but not
identical scenario involving an initial INVITE and subsequent reinvite.
As far as I can tell, the recommendation in that standard is "space the
messaging out more in time".

Switching to TCP would presumably help, since any given flow would
involve a single connection to a single worker thread and the transport
would guarantee ordering. However, that's not really feasible in this
implementation for a host of reasons.

Any other thoughts welcome!

Cheers,

-- Alex



___
Sip-implementors mailing list
Sip-implementors@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors


___
Sip-implementors mailing list
Sip-implementors@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors

Re: [Sip-implementors] Reinvite/ACK race

[Liviu Chircu]

Hi Alex,

IMO, building logic into the proxy which encourages/mends the proper 
sequencing of UDP messages does nothing more than to hide the underlying 
problem, i.e. "UDP does not guarantee message sequencing in the first 
place" *. There is also a subtle point to be made here: your proxy's 
loosely coupled/parallelized way of handling the two transactions is 
effectively breaking the RFC some % of the time, since the proxy's TU is 
generating a new INVITE while another INVITE transaction is in progress 
(strictly judging by the network timestamps).


* From this angle, the only sane thing left to do is to have the proxy 
retry the re-INVITE for UASs who are properly responding with 491 and 
propagate any other 4xx/5xx error replies back to the UAC (UA B in our 
case) in the hope that they retry their re-INVITE.


Best regards,

Liviu Chircu
OpenSIPS Developer
http://www.opensips-solutions.com

On 30.10.2017 19:11, Alex Balashov wrote:

Hi,

I've got a scenario like so:

UA A -> Proxy P > UA B

1. UA A initiates call through Proxy P;

2. Dialog is established and confirmed, with Record-Route;

3. UA B sends reinvite #1 through P to A;

4. UA B sends 2xx reply;

5. UA B sends end-to-end ACK for reinvite #1 and almost
simultaneously sends reinvite #2. The temporal delta is
between reinvite #2 and ACK for reinvite #1 on the wire
is 3 ms.

The issue is that the concurrency characteristics of proxy P are such
that its worker threads are very loosely coupled, and there's no
synchronisation among them for message ordering. Transport is UDP,
naturally.

So, the result — for all kinds of stochastic processing and userspace
scheduling type reasons — is that the reinvite is forwarded first,
before the ACK.  That leads to a 500 / 491 scenario UA A.

Is there any general guidance on what to do with these scenarios? I
looked at RFC 5407 § 3.1.4, which appears to describe a similar, but not
identical scenario involving an initial INVITE and subsequent reinvite.
As far as I can tell, the recommendation in that standard is "space the
messaging out more in time".

Switching to TCP would presumably help, since any given flow would
involve a single connection to a single worker thread and the transport
would guarantee ordering. However, that's not really feasible in this
implementation for a host of reasons.

Any other thoughts welcome!

Cheers,

-- Alex



___
Sip-implementors mailing list
Sip-implementors@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors