[Development] Multiclient UDP server

[Thiago Macieira]

Related to the DTLS discussion, but I'll try to keep this free from crypto for 
now.

The question is how to write a multi-client server operating on UDP, not TCP. 
It would be nice to understand SCTP case too (and maybe DCCP in the future).

On TCP, this is easy:
1) you create a socket, bind it and then put it in LISTENING mode
2) every time a new TCP connection arrives, you accept() a new socket
3) the server socket is from now on independent of the client

In our API, the QTcpServer creates new QTcpSockets and they're from that point 
on independent.

In UDP, that's different. There are two ways to proceed:

A) single UDP socket in unconnected mode

The code identifies each different client by the source address. 
QNetworkDatagram makes this easy, since it holds for you the source address 
and using makeReply() creates a QNetworkDatagram that you can use to reply.

Pros: uses a single socket, so limits the resource usage.

Cons: scalability. Since it's the same socket, only one thread can read from 
it at a time. You may pass the QNetworkDatagram to another thread to process 
the data, but you need to centralise sending of replies (QUdpSocket 
limitation, not of the OS API).

We don't need to implement anything to support this: that is simply QUdpSocket 
as it is today.

B) multiple UDP sockets

This is similar to TCP. Whenever a new client is detected, we create a new UDP 
socket and put it in connected mode with that particular client. Once you do 
that, this socket will only receive datagrams from that client, whereas the 
unconnected socket will not.

Pros: scalability, since you can move the QUdpSocket to another thread; the 
connected datagram socket has another advantage that it can relay ICMP errors 
like "port unreachable" to the upper level, whereas the unconnected one won't. 
I also believe the connected mode socket can do Path MTU calculation.

Cons: more resources, but more importantly, there's a race condition: the 
unconnected socket which you received the initial packet from the new client 
on may have queued more datagrams. Those datagrams can be:
 - from the same client
 - from other clients

So we are in a catch-22 situation: if you connect() this socket and create a 
new unconnected one, any packets from other clients will not be seen. If you 
create a new socket and connect() it, you may miss any packets from the same 
client. I think that for both CoAP and DTLS, we'd opt for connect()ing the new 
socket, since the client wouldn't be sending new commands until it receives a 
reply from us. That reply resolves the problem of race condition.

[food for thought: what happens if we receive a retransmission?]


So questions:
 - do we attempt to provide a "QUdpServer" or augment QUdpSocket's API to 
support (B) above? Or do we provide only a QDtlsServer API that implements 
(B)? This "QUdpServer" would be useful to implement a unencrypted CoAP server 
("coap"). It's not very difficult to rewrite the code, but the choice becomes 
hairy if we want to support both unencrypted and encrypted CoAP ("coaps").

 - do we need a QDtlsSocket at all, or just a wrapper on top of QUdpSocket? 
One advantage of having the class is that you could implement both "coap" and 
"coaps" clients on top of a plain QUdpSocket pointer, not having to worry  
about the encryption.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] DTLS support in Qt

[Thiago Macieira]

On Friday, 18 August 2017 13:07:13 PDT Timur Pocheptsov wrote:
> > Should *I* contribute it?
> 
> Well, yes, please, otherwise I'll do it myself 

I think I need an answer from Lars or Tuukka, because of the legal 
implications.

> > If NO:
> 
> Then who will write it? When?
> 
> I will, 5.11.

Cool! I'll start a new thread discussing the non-crypto related parts until we 
get the conclusion of what I can or cannot be part of.

> > Then what module should it be in?
> 
> Option 'a' is optimal.

No doubt.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] DTLS support in Qt

[Timur Pocheptsov]

> Should *I* contribute it?


Well, yes, please, otherwise I'll do it myself 


> If NO:
Then who will write it? When?


I will, 5.11.


> Then what module should it be in?


Option 'a' is optimal.


From: Development  
on behalf of Thiago Macieira 
Sent: Friday, August 18, 2017 8:57:09 PM
To: development@qt-project.org
Subject: [Development] DTLS support in Qt

Hello

Last year at QCS, I joined the Networking discussions and one of my requests
was DTLS support. Everything else needed to support IoT was already in place,
in flight (the QNetworkDatagram class) or I could do it myself (new API for
QNetworkInterface coming soon).

DTLS was the only thing I wasn't allowed to contribute and no one else has
stepped up in the last year.

So I decided to implement it myself. I've now got a proof of concept to
support DTLS over QUdpSocket and it already manages to connect one client and
server, verify the certificate (haven't tested failure) and communicate with
itself, with the "openssl" binary and with "gnutls-serv" binary. I've got
approval from Intel to contribute it.

I'd like Qt to have DTLS support. Should *I* contribute it? This question is
important because there used to be restrictions on "US persons" contributing
cryptography-related code. I need an answer from the Qt Project.

If NO:
Then who will write it? When? Can you finish it by Qt 5.11 feature freeze?

If YES:
Then what module should it be in?
 a) QtNetwork
Would be ideal, as there are quite a few changes to QSslSocketBackend,
QSslContext, etc. that are required. We'd also reuse the dynamic OpenSSL
loading. If I can implement DTLS support in QtNetwork, I can make these
changes myself.
 b) another module, outside of qtbase
This module would be licenced LGPLv3, no commercial.
Not ideal, but workable. The changes I mentioned above would still need 
to
be implemented, so we'd need a volunteer to implement them and work with
me. It shouldn't be too difficult.
 c) not in Qt
Really not ideal. Would make for a crappy API and would increase the
development time at least threefold, probably more.

I'll be really disappointed if the answer is "no, we won't accept this
contribution and we won't develop it for 5.11 either".

--
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

[Development] DTLS support in Qt

[Thiago Macieira]

Hello

Last year at QCS, I joined the Networking discussions and one of my requests 
was DTLS support. Everything else needed to support IoT was already in place, 
in flight (the QNetworkDatagram class) or I could do it myself (new API for 
QNetworkInterface coming soon).

DTLS was the only thing I wasn't allowed to contribute and no one else has 
stepped up in the last year.

So I decided to implement it myself. I've now got a proof of concept to 
support DTLS over QUdpSocket and it already manages to connect one client and 
server, verify the certificate (haven't tested failure) and communicate with 
itself, with the "openssl" binary and with "gnutls-serv" binary. I've got 
approval from Intel to contribute it.

I'd like Qt to have DTLS support. Should *I* contribute it? This question is 
important because there used to be restrictions on "US persons" contributing 
cryptography-related code. I need an answer from the Qt Project.

If NO:
Then who will write it? When? Can you finish it by Qt 5.11 feature freeze?

If YES:
Then what module should it be in?
 a) QtNetwork
Would be ideal, as there are quite a few changes to QSslSocketBackend, 
QSslContext, etc. that are required. We'd also reuse the dynamic 
OpenSSL 
loading. If I can implement DTLS support in QtNetwork, I can make these 
changes myself.
 b) another module, outside of qtbase
This module would be licenced LGPLv3, no commercial.
Not ideal, but workable. The changes I mentioned above would still need 
to 
be implemented, so we'd need a volunteer to implement them and work 
with
me. It shouldn't be too difficult.
 c) not in Qt
Really not ideal. Would make for a crappy API and would increase the 
development time at least threefold, probably more.

I'll be really disappointed if the answer is "no, we won't accept this 
contribution and we won't develop it for 5.11 either".

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Coding style for lambdas, particularly their open-brace

[Mark Gaiser]

On Fri, Aug 18, 2017 at 2:17 PM, Edward Welbourne 
wrote:

> Hi all,
>
> We have a draft policy for lambdas at [0], in a section that begins with
>
>   Note: This section is not an accepted convention yet.
>   This section serves as baseline for further discussions.
>
> That section is now a quarter decade old; it's had a few updates since
> it was added (2015-02-27), so may fairly be said to be evolving (albeit
> it's had more formatting changes than substantive ones); but perhaps
> it's about time we agreed that at least its low-level bits about
> formatting can be promoted to [1], without such a caveat.
>
> * [0] https://wiki.qt.io/Coding_Conventions#Conventions_for_C.
> 2B.2B11_usage
> * [1] https://wiki.qt.io/Qt_Coding_Style#Braces
>
> In particular, I'd like to (at least) amend the first exception in [1],
>
>   Function implementations and class declarations always have the left
>   brace on the start of a line:
>
> to include "(but not lambdas)" in a judicious place, so that lambdas are
> excluded from the exception and fit into our general pattern, putting
> the open-brace on the same line as its controlling preamble: e.g.
>
>   Function implementations (but not lambdas) and class declarations
>   always have the left brace on the start of a line:
>
> Does anyone object to this minimal change ?
> (How long do I have to wait before I can claim lazy consensus ?)
>
> Eddy.
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>

The explicit return type rule should probably be changed, as that was for
VS2010. And that compiler has been dropped in 5.7.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

[Development] Coding style for lambdas, particularly their open-brace

[Edward Welbourne]

Hi all,

We have a draft policy for lambdas at [0], in a section that begins with

  Note: This section is not an accepted convention yet.
  This section serves as baseline for further discussions.

That section is now a quarter decade old; it's had a few updates since
it was added (2015-02-27), so may fairly be said to be evolving (albeit
it's had more formatting changes than substantive ones); but perhaps
it's about time we agreed that at least its low-level bits about
formatting can be promoted to [1], without such a caveat.

* [0] https://wiki.qt.io/Coding_Conventions#Conventions_for_C.2B.2B11_usage
* [1] https://wiki.qt.io/Qt_Coding_Style#Braces

In particular, I'd like to (at least) amend the first exception in [1],

  Function implementations and class declarations always have the left
  brace on the start of a line:

to include "(but not lambdas)" in a judicious place, so that lambdas are
excluded from the exception and fit into our general pattern, putting
the open-brace on the same line as its controlling preamble: e.g.

  Function implementations (but not lambdas) and class declarations
  always have the left brace on the start of a line:

Does anyone object to this minimal change ?
(How long do I have to wait before I can claim lazy consensus ?)

Eddy.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Qt Creator 4.4 RC released

[Iikka Eklund]

> I was wondering whether there are any plans to make future pre-releases 
> installable via the Qt installer.


This is actually planned already. Exact schedule is not know yet, but that is 
quite high on the current work list.




Iikka Eklund
Senior Software Engineer

The Qt Company
Elektroniikkatie 13
90590, Oulu, Finland
iikka.ekl...@qt.io
http://qt.io





From: Development  on 
behalf of Bullinger, Julius 
Sent: Friday, August 18, 2017 12:50:57 PM
To: development@qt-project.org
Subject: Re: [Development] Qt Creator 4.4 RC released

-Original Message-
From: Development 
[mailto:development-bounces+julius.bullinger=intel@qt-project.org] On 
Behalf Of List for announcements regarding Qt releases and development
Sent: Thursday, August 17, 2017 14:09
To: annou...@qt-project.org
Subject: [Development] [Announce] Qt Creator 4.4 RC released

> We are happy to announce the release of Qt Creator 4.4 RC!
>
> http://blog.qt.io/blog/2017/08/17/qt-creator-4-4-rc-released/

Hi Eike,

That looks like a great release. I was wondering whether there are any plans to 
make future pre-releases installable via the Qt installer.

At the moment, there's a bit of pain involved when you have to maintain two 
separate installations with separate user profiles (since the Creator instance 
bundled with Qt cannot be removed easily).

It would be really great if we were able to just upgrade and/or parallel 
install pre-releases via the Maintenance Tool.

Anyway, thanks a lot for your work and congratulations for the upcoming release.

Best regards,
Julius
Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Qt Creator 4.4 RC released

[Bullinger, Julius]

-Original Message-
From: Development 
[mailto:development-bounces+julius.bullinger=intel@qt-project.org] On 
Behalf Of List for announcements regarding Qt releases and development
Sent: Thursday, August 17, 2017 14:09
To: annou...@qt-project.org
Subject: [Development] [Announce] Qt Creator 4.4 RC released

> We are happy to announce the release of Qt Creator 4.4 RC!
> 
> http://blog.qt.io/blog/2017/08/17/qt-creator-4-4-rc-released/

Hi Eike,

That looks like a great release. I was wondering whether there are any plans to 
make future pre-releases installable via the Qt installer.

At the moment, there's a bit of pain involved when you have to maintain two 
separate installations with separate user profiles (since the Creator instance 
bundled with Qt cannot be removed easily).

It would be really great if we were able to just upgrade and/or parallel 
install pre-releases via the Maintenance Tool.

Anyway, thanks a lot for your work and congratulations for the upcoming release.

Best regards,
Julius
Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development