For the record, as I got some of my application issues out of the way and
actually started using the STREAM socket, I did encounter some odd behavior
(connection timeout) which disappeared when I switched to using a different
service type. So the presence of a listening STREAM socket didn't cause trouble
for the DGRAM activity (except on kernel-4.19), but something about the DGRAM
activity appeared to cause trouble for the STREAMs.
As an aside, at least some of our systems threw an EMSGSIZE when I tried to
sendmsg() on a STREAM socket with an oversized message, which was at least a
little surprising. Using explicit connect()/writev() worked, but having the
implicit connect work Would Be Nice. This is another case where knowing what is
expected to work would help, as I could always try the sendmsg() first and fall
back to connect()/writev() on EMSGSIZE, but if it is never expected to work,
then there would be no point.
Thanks.
Gary Duzan
FIS - GT.M Core
________________________________
From: Duzan, Gary D via tipc-discussion <[email protected]>
Sent: Thursday, June 3, 2021 10:57 AM
To: Jon Maloy <[email protected]>; Xin Long <[email protected]>
Cc: [email protected]
<[email protected]>
Subject: Re: [tipc-discussion] EXTERNAL: Re: DGRAM/STREAM Crossover on Debian?
I can frankly admit that this is a problem we never came across before, and
that we haven´t really spent much time considering this kind of issues.
That's why you need crazy developers trying to do weird stuff with it. 🙂
You are right that this needs to be at least clarified in the documentation.
But we should also run some internal tests to identify what is working and what
is not, and then decide what is the correct approach.
Thank you for making us aware of this.
Thanks for your efforts in making TIPC a great tool for datacenter
communication.
Gary Duzan
FIS - GT.M Core
///jon
________________________________
From: Jon Maloy <[email protected]>
Sent: Thursday, June 3, 2021 10:19 AM
To: Duzan, Gary D <[email protected]>; Xin Long <[email protected]>
Cc: [email protected]
<[email protected]>
Subject: Re: EXTERNAL: Re: [tipc-discussion] DGRAM/STREAM Crossover on Debian?
On 6/3/21 9:55 AM, Duzan, Gary D wrote:
Contrary to UDP vs TCP, TIPC is only one protocol, so you
cannot bind the same service type/instance to different socket types
without risking problems.
The SYN bit will prevent a connection from being established with a
socket of the wrong type, but it will not stop the binding table lookup
from selecting such a socket, since it knows nothing about socket types.
I am actually surprised that this works even on the non-Debian machines.
Maybe the secondary lookup mechanism is saving the day.
This could of course be fixed without too much effort, but the question
is if that is the right way to go. At least we would have to carefully
consider possible compatibility issues.
Would it be a problem for you to just choose different service types/ranges?
///jon
Interesting. So, accidents of implementation aside, you would expect DGRAM,
STREAM, and SEQPACKET sockets to be able to communicate with each other? I'm
not using SEQPACKET (yet), but it sounds like one might be able to connect()
between STREAM and SEQPACKET sockets today, though presumably the way that
send()s on the STREAM side get "packetized" on the SEQPACKET end is not clearly
defined. If this is what is intended, I think some clarification in the
documentation would be helpful, as would testing to ensure expected behavior
across socket types. It is true that bringing TCP/UDP thinking to TIPC is going
to lead to errors, but it isn't going to be rare. If it is straightforward to
keep communication between different socket types separate, then I suspect that
would produce the least surprise.
As for my own project, logically I'm dealing with the same service either
way; it is only the DGRAM message size limit that is pushing the STREAM
fallback option. I could require a separate service type for the STREAM
implementation, but that is another piece of configuration which I'd rather
avoid, if practical. I expect I will keep it the way I have it while I'm in
experimental mode, as it seems to be working as I expect on the target kernels,
but I'll need to be sure I'm on solid ground if we ever manage to get the thing
to production.
Thanks.
Gary Duzan
FIS - GT.M Core
I can frankly admit that this is a problem we never came across before, and
that we haven´t really spent much time considering this kind of issues.
You are right that this needs to be at least clarified in the documentation.
But we should also run some internal tests to identify what is working and what
is not, and then decide what is the correct approach.
Thank you for making us aware of this.
///jon
________________________________
From: Jon Maloy <[email protected]><mailto:[email protected]>
Sent: Wednesday, June 2, 2021 8:56 PM
To: Xin Long <[email protected]><mailto:[email protected]>; Duzan, Gary D
<[email protected]><mailto:[email protected]>
Cc:
[email protected]<mailto:[email protected]>
<[email protected]><mailto:[email protected]>
Subject: EXTERNAL: Re: [tipc-discussion] DGRAM/STREAM Crossover on Debian?
CAUTION: This email originated from outside of the company. Do not click links
or open attachments unless you recognize the sender and know the content is
safe.
On 6/2/21 4:14 PM, Xin Long wrote:
> On Wed, May 26, 2021 at 11:38 AM Duzan, Gary D via tipc-discussion
> <[email protected]><mailto:[email protected]>
> wrote:
>> I'm in the process of enhancing a TIPC DGRAM-based RPC-ish service to
>> include TIPC STREAM transport for larger messages. To simplify
>> configuration, I have the server process(es) bind() the same type/range for
>> both DGRAM and STREAM sockets (poll()ing to see which have incoming
>> requests), then choose which to use on the client. This seems to work on
>> most of my Linux systems (RHEL-8, Ubuntu 20.04/21.04, Fedora 34, Debian 11),
>> but on my Debian 10 system (4.19.181-1 kernel) I am seeing messages from a
>> DGRAM client appearing on an accept()ed STREAM socket on the server. I have
>> confirmed that the client is not sending anything on a STREAM socket, and
>> the message received by the server is formatted as a DGRAM message (without
>> the message framing header).
> When you start two scoket on the server: DGRAM and STREAM, in the
> client's nametable there will be 2 sockets with different portids:
> # tipc nametable show
> Type Lower Upper Scope Port Node
> 18888 17 17 cluster 4063960415
> 18888 17 17 cluster 1106254118
>
> When the client calls sendmsg()/connect() to send msg to the server,
> it will choose one of them by the rule of "local, closest-first or
> round-robin".
> The client doesn't know if the peer is a DGRAM socket or STREAM
> socket. In your case, it should go round-robin.
>
> Without this commit:
>
> commit 25b9221b959483f17c2964d0922869e16caa86b5
> Author: Jon Maloy <[email protected]><mailto:[email protected]>
> Date: Fri Sep 28 20:23:21 2018 +0200
>
> tipc: add SYN bit to connection setup messages
>
> The SYN msg for STREAM is no different from the DATA msg for DGRAM.
> that's what you're seeing in kernel-4.19
>
>> Debian isn't a target platform for production, so I don't need a
>> specific fix, but it is still surprising and a bit disturbing. Was this a
>> known problem with the 4.19 kernel? Are there particular reasons why using
>> this pattern is a bad idea?
> I think it may not work as expected if you create 2 different types of
> TIPC sockets binding to the same address.
> At least on the latest kernel, once the DGRAM client chooses the
> STREAM socket, the DATA msg will be dropped.
>
> Thanks.
Exactly. Contrary to UDP vs TCP, TIPC is only one protocol, so you
cannot bind the same service type/instance to different socket types
without risking problems.
The SYN bit will prevent a connection from being established with a
socket of the wrong type, but it will not stop the binding table lookup
from selecting such a socket, since it knows nothing about socket types.
I am actually surprised that this works even on the non-Debian machines.
Maybe the secondary lookup mechanism is saving the day.
This could of course be fixed without too much effort, but the question
is if that is the right way to go. At least we would have to carefully
consider possible compatibility issues.
Would it be a problem for you to just choose different service types/ranges?
///jon
>
>> Thanks.
>>
>> Gary Duzan
>> FIS - GT.M Core
>>
>> The information contained in this message is proprietary and/or
>> confidential. If you are not the intended recipient, please: (i) delete the
>> message and all copies; (ii) do not disclose, distribute or use the message
>> in any manner; and (iii) notify the sender immediately. In addition, please
>> be aware that any message addressed to our domain is subject to archiving
>> and review by persons other than the intended recipient. Thank you.
>>
>> _______________________________________________
>> tipc-discussion mailing list
>> [email protected]<mailto:[email protected]>
>> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&reserved=0>
The information contained in this message is proprietary and/or confidential.
If you are not the intended recipient, please: (i) delete the message and all
copies; (ii) do not disclose, distribute or use the message in any manner; and
(iii) notify the sender immediately. In addition, please be aware that any
message addressed to our domain is subject to archiving and review by persons
other than the intended recipient. Thank you.
The information contained in this message is proprietary and/or confidential.
If you are not the intended recipient, please: (i) delete the message and all
copies; (ii) do not disclose, distribute or use the message in any manner; and
(iii) notify the sender immediately. In addition, please be aware that any
message addressed to our domain is subject to archiving and review by persons
other than the intended recipient. Thank you.
_______________________________________________
tipc-discussion mailing list
[email protected]
https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&reserved=0
The information contained in this message is proprietary and/or confidential.
If you are not the intended recipient, please: (i) delete the message and all
copies; (ii) do not disclose, distribute or use the message in any manner; and
(iii) notify the sender immediately. In addition, please be aware that any
message addressed to our domain is subject to archiving and review by persons
other than the intended recipient. Thank you.
_______________________________________________
tipc-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/tipc-discussion
