Re: [tipc-discussion] EXTERNAL: Re: DGRAM/STREAM Crossover on Debian?

Wed, 09 Jun 2021 11:11:52 -0700 


On 6/9/21 1:07 PM, Duzan, Gary D wrote:

   For the record, as I got some of my application issues out of the 
way and actually started using the STREAM socket, I did encounter some 
odd behavior (connection timeout) which disappeared when I switched to 
using a different service type. So the presence of a listening STREAM 
socket didn't cause trouble for the DGRAM activity (except on 
kernel-4.19), but something about the DGRAM activity appeared to cause 
trouble for the STREAMs.

We are still in the process of evaluating this, and what is a reasonable 
solution. We want to be as permissive as possible, but of course it has 
to work "as expected" without surprises.



   As an aside, at least some of our systems threw an EMSGSIZE when I 
tried to sendmsg() on a STREAM socket with an oversized message, which 
was at least a little surprising.

The first message cannot be oversized, as it (in reality a data-carrying 
SYN) is routed as a regular SEQPACKET/DGRAM message to the server 
socket. Not sure if this is well documented, though.



Using explicit connect()/writev() worked, but having the implicit 
connect work Would Be Nice. This is another case where knowing what is 
expected to work would help, as I could always try the sendmsg() first 
and fall back to connect()/writev() on EMSGSIZE, but if it is never 
expected to work, then there would be no point.

It is supposed to work, but I haven't heard about anybody actually using 
this feature in a product, and it is not part of our regular test suite. 
So, testing this has probably slipped though our filter at some moment. 
We will have to look into this too, and add it to our test suite.


Regards
///jon



   Thanks.

Gary Duzan
FIS - GT.M Core


------------------------------------------------------------------------

*From:* Duzan, Gary D via tipc-discussion 
<tipc-discussion@lists.sourceforge.net>

*Sent:* Thursday, June 3, 2021 10:57 AM
*To:* Jon Maloy <jma...@redhat.com>; Xin Long <lucien....@gmail.com>

*Cc:* tipc-discussion@lists.sourceforge.net 
<tipc-discussion@lists.sourceforge.net>
*Subject:* Re: [tipc-discussion] EXTERNAL: Re: DGRAM/STREAM Crossover 
on Debian?



I can frankly admit that this is a problem we never came across 
before, and that we haven´t really spent much time considering this 
kind of issues.
   That's why you need crazy developers trying to do weird stuff with 
it. 🙂



You are right that this needs to be at least clarified in the 
documentation.
But we should also run some internal tests to identify what is working 
and what is not, and then decide what is the correct approach.

Thank you for making us aware of this.

   Thanks for your efforts in making TIPC a great tool for datacenter 
communication.


                              Gary Duzan
                              FIS - GT.M Core


///jon




________________________________
From: Jon Maloy <jma...@redhat.com>
Sent: Thursday, June 3, 2021 10:19 AM

To: Duzan, Gary D <gary.du...@fisglobal.com>; Xin Long 
<lucien....@gmail.com>
Cc: tipc-discussion@lists.sourceforge.net 
<tipc-discussion@lists.sourceforge.net>
Subject: Re: EXTERNAL: Re: [tipc-discussion] DGRAM/STREAM Crossover on 
Debian?




On 6/3/21 9:55 AM, Duzan, Gary D wrote:

Contrary to UDP vs TCP, TIPC is only one protocol, so you
cannot bind the same service type/instance to different socket types
without risking problems.
The SYN bit will prevent a connection from being established with a
socket of the wrong type, but it will not stop the binding table lookup
from selecting such a socket, since it knows nothing about socket types.
I am actually surprised that this works even on the non-Debian machines.
Maybe the secondary lookup mechanism is saving the day.

This could of course be fixed without too much effort, but the question
is if that is the right way to go. At least we would have to carefully
consider possible compatibility issues.


Would it be a problem for you to just choose different service 
types/ranges?


///jon


   Interesting. So, accidents of implementation aside, you would 
expect DGRAM, STREAM, and SEQPACKET sockets to be able to communicate 
with each other? I'm not using SEQPACKET (yet), but it sounds like one 
might be able to connect() between STREAM and SEQPACKET sockets today, 
though presumably the way that send()s on the STREAM side get 
"packetized" on the SEQPACKET end is not clearly defined. If this is 
what is intended, I think some clarification in the documentation 
would be helpful, as would testing to ensure expected behavior across 
socket types. It is true that bringing TCP/UDP thinking to TIPC is 
going to lead to errors, but it isn't going to be rare. If it is 
straightforward to keep communication between different socket types 
separate, then I suspect that would produce the least surprise.



   As for my own project, logically I'm dealing with the same service 
either way; it is only the DGRAM message size limit that is pushing 
the STREAM fallback option. I could require a separate service type 
for the STREAM implementation, but that is another piece of 
configuration which I'd rather avoid, if practical. I expect I will 
keep it the way I have it while I'm in experimental mode, as it seems 
to be working as I expect on the target kernels, but I'll need to be 
sure I'm on solid ground if we ever manage to get the thing to production.


   Thanks.

                              Gary Duzan
                              FIS - GT.M Core


I can frankly admit that this is a problem we never came across 
before, and that we haven´t really spent much time considering this 
kind of issues.
You are right that this needs to be at least clarified in the 
documentation.
But we should also run some internal tests to identify what is working 
and what is not, and then decide what is the correct approach.

Thank you for making us aware of this.

///jon


________________________________
From: Jon Maloy <jma...@redhat.com><mailto:jma...@redhat.com>
Sent: Wednesday, June 2, 2021 8:56 PM

To: Xin Long <lucien....@gmail.com><mailto:lucien....@gmail.com>; 
Duzan, Gary D <gary.du...@fisglobal.com><mailto:gary.du...@fisglobal.com>
Cc: 
tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net> 
<tipc-discussion@lists.sourceforge.net><mailto:tipc-discussion@lists.sourceforge.net>

Subject: EXTERNAL: Re: [tipc-discussion] DGRAM/STREAM Crossover on Debian?


CAUTION: This email originated from outside of the company. Do not 
click links or open attachments unless you recognize the sender and 
know the content is safe.




On 6/2/21 4:14 PM, Xin Long wrote:
> On Wed, May 26, 2021 at 11:38 AM Duzan, Gary D via tipc-discussion

> 
<tipc-discussion@lists.sourceforge.net><mailto:tipc-discussion@lists.sourceforge.net> 
wrote:
>>     I'm in the process of enhancing a TIPC DGRAM-based RPC-ish 
service to include TIPC STREAM transport for larger messages. To 
simplify configuration, I have the server process(es) bind() the same 
type/range for both DGRAM and STREAM sockets (poll()ing to see which 
have incoming requests), then choose which to use on the client. This 
seems to work on most of my Linux systems (RHEL-8, Ubuntu 20.04/21.04, 
Fedora 34, Debian 11), but on my Debian 10 system (4.19.181-1 kernel) 
I am seeing messages from a DGRAM client appearing on an accept()ed 
STREAM socket on the server. I have confirmed that the client is not 
sending anything on a STREAM socket, and the message received by the 
server is formatted as a DGRAM message (without the message framing 
header).

> When you start two scoket on the server: DGRAM and STREAM, in the
> client's nametable there will be 2 sockets with different portids:
> # tipc nametable show
> Type       Lower      Upper      Scope    Port Node
> 18888      17         17         cluster  4063960415
> 18888      17         17         cluster  1106254118
>
> When the client calls sendmsg()/connect() to send msg to the server,
> it will choose one of them by the rule of "local, closest-first or
> round-robin".
> The client doesn't know if the peer is a DGRAM socket or STREAM
> socket. In your case, it should go round-robin.
>
> Without this commit:
>
> commit 25b9221b959483f17c2964d0922869e16caa86b5

> Author: Jon Maloy 
<jon.ma...@ericsson.com><mailto:jon.ma...@ericsson.com>

> Date:   Fri Sep 28 20:23:21 2018 +0200
>
>      tipc: add SYN bit to connection setup messages
>
> The SYN msg for STREAM is no different from the DATA msg for DGRAM.
> that's what you're seeing in kernel-4.19
>

>>     Debian isn't a target platform for production, so I don't need 
a specific fix, but it is still surprising and a bit disturbing. Was 
this a known problem with the 4.19 kernel? Are there particular 
reasons why using this pattern is a bad idea?

> I think it may not work as expected if you create 2 different types of
> TIPC sockets binding to the same address.
> At least on the latest kernel, once the DGRAM client chooses the
> STREAM socket, the DATA msg will be dropped.
>
> Thanks.
Exactly. Contrary to UDP vs TCP, TIPC is only one protocol, so you
cannot bind the same service type/instance to different socket types
without risking problems.
The SYN bit will prevent a connection from being established with a
socket of the wrong type, but it will not stop the binding table lookup
from selecting such a socket, since it knows nothing about socket types.
I am actually surprised that this works even on the non-Debian machines.
Maybe the secondary lookup mechanism is saving the day.

This could of course be fixed without too much effort, but the question
is if that is the right way to go. At least we would have to carefully
consider possible compatibility issues.


Would it be a problem for you to just choose different service 
types/ranges?


///jon

>
>>     Thanks.
>>
>> Gary Duzan
>> FIS - GT.M Core
>>

>> The information contained in this message is proprietary and/or 
confidential. If you are not the intended recipient, please: (i) 
delete the message and all copies; (ii) do not disclose, distribute or 
use the message in any manner; and (iii) notify the sender 
immediately. In addition, please be aware that any message addressed 
to our domain is subject to archiving and review by persons other than 
the intended recipient. Thank you.

>>
>> _______________________________________________
>> tipc-discussion mailing list

>> 
tipc-discussion@lists.sourceforge.net<mailto:tipc-discussion@lists.sourceforge.net>
>> 
https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&amp;data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&amp;reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&amp;data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&amp;reserved=0 
<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&amp;data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&amp;reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&amp;data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&amp;reserved=0>>



The information contained in this message is proprietary and/or 
confidential. If you are not the intended recipient, please: (i) 
delete the message and all copies; (ii) do not disclose, distribute or 
use the message in any manner; and (iii) notify the sender 
immediately. In addition, please be aware that any message addressed 
to our domain is subject to archiving and review by persons other than 
the intended recipient. Thank you.



The information contained in this message is proprietary and/or 
confidential. If you are not the intended recipient, please: (i) 
delete the message and all copies; (ii) do not disclose, distribute or 
use the message in any manner; and (iii) notify the sender 
immediately. In addition, please be aware that any message addressed 
to our domain is subject to archiving and review by persons other than 
the intended recipient. Thank you.


_______________________________________________
tipc-discussion mailing list
tipc-discussion@lists.sourceforge.net

https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&amp;data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&amp;reserved=0 
<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&amp;data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&amp;reserved=0>
The information contained in this message is proprietary and/or 
confidential. If you are not the intended recipient, please: (i) 
delete the message and all copies; (ii) do not disclose, distribute or 
use the message in any manner; and (iii) notify the sender 
immediately. In addition, please be aware that any message addressed 
to our domain is subject to archiving and review by persons other than 
the intended recipient. Thank you. 



_______________________________________________
tipc-discussion mailing list
tipc-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/tipc-discussion






















					Reply via email to