Re: [zeromq-dev] Is there a reasonable way to use an existing PKI and D/TLS with 0MQ?

[paddor]

Hi John,

One cool thing about Curve25519 is that, given the secret key, the public key 
can be derived from it. And it seems that you actually you can use random bytes 
for the secret key (almost). Only a few bits in the first and last byte are 
fixed. Here an excerpt from [1]:

> Computing secret keys. Inside your program, to generate a 32-byte Curve25519 
> secret key, start by generating 32 secret random bytes from a 
> cryptographically safe source: mysecret[0], mysecret[1], ..., mysecret[31]. 
> Then do
> 
>  mysecret[0] &= 248;
>  mysecret[31] &= 127;
>  mysecret[31] |= 64;
> 
> to create a 32-byte Curve25519 secret key mysecret[0], mysecret[1], ..., 
> mysecret[31].

Of course the conversion tool would have to print the public key so you can 
collect those centrally.

Yes, it might be too much effort if you want to keep using the existing PKI 
as-is. Having used ZMQ for a while now, I'd say it's worth the effort! ;-)

Regards,
Patrik

[1] https://cr.yp.to/ecdh.html


> On 14 Feb 2018, at 08:32, John Lane Schultz  
> wrote:
> 
> Hi Patrik,
> 
> I’m not terribly familiar with the details of elliptic curve cryptography 
> myself.
> 
> Even so, there must be a strong EC relationship (inversion?) between the 
> public and private parts of each key pair, just like there is for RSA key 
> pairs.  I highly doubt that simply using a random-ish transform of RSA 
> parameters will yield proper ECC key pairs.
> 
> I probably could automate the creation of a parallel PKI system, but then it 
> would need to be mapped to, maintained, watered, and fed alongside our X509 / 
> CA PKI infrastructure.  I doubt I want to take on that burden simply to 
> reduce my socket programming pains.
> 
> Thanks,
> John
> 
> On Feb 13, 2018, at 8:06 PM, Patrik VV.  wrote:
> 
> Hi
> 
> WARNING: I’m no infosec expert.
> 
> I’m just thinking out loud, so please bear with me. To my understanding, the 
> keys used by the crypto primitives (NaCl/TweetNaCl) used in CurveZMQ are just 
> 32 secret bytes. You already have about 2000 secrets for those 2000 
> identities, their current format just isn’t usable. Can’t you just convert 
> those secrets to 32 secret bytes each using a cryptographically secure hash 
> function? Of course it should be a hash function that gives you a hash of 32 
> bytes or more to avoid losing entropy.
> 
> I’m sure you could automate this process and store the converted identities 
> as zcerts using CZMQ. See [1]
> 
> Regards,
> Patrik
> 
> [1] http://czmq.zeromq.org/czmq-master:zcert
> 
> On 13 Feb 2018, at 23:42, Luca Boccassi  wrote:
> 
>> On Tue, 2018-02-13 at 17:03 -0500, John Lane Schultz wrote:
>>> I’m new to 0MQ, so please excuse my ignorance about it.  I read the
>>> guide book, the blog entries on CurveZMQ, looked at the directory of
>>> 0MQ GitHub repositories, and googled around to see if anyone had
>>> added a D/TLS layer into 0MQ. I didn’t find much of anything.
>>> 
>>> I did find the email below from 4 years ago, that seems to answer a
>>> similar question from back then.  Is it still the situation that 0MQ
>>> doesn’t readily support using D/TLS to secure its communications?
>>> 
>>> I ask because we have an existing PKI managing more than 2000
>>> distributed identities and we use D/TLS to secure our mutually
>>> authenticated (the “Ironhouse Pattern") point-to-point
>>> communications.  If it matters, we use Ephemeral Diffie-Hellman key
>>> exchanges with long term RSA keys.
>>> 
>>> From my reading, 0MQ looks really appealing to get away from low-
>>> level programming OpenSSL D/TLS, UDP, and TCP sockets, but not being
>>> able to use our existing security infrastructure would probably be a
>>> deal breaker.
>>> 
>>> I’d greatly appreciate any information on the current state of
>>> affairs of 0MQ security layers and whether or not adding D/TLS
>>> support to 0MQ is reasonable or not.
>>> 
>>> Thanks!
>>> John
>> 
>> Hi,
>> 
>> The situation is the same - security is only supported through Curve or
>> Kerberos.
>> 
>> Not for any particular reason if not that nobody has contributed any
>> other implementation. This is probably due to the fact that SSL is
>> awfully, awfully complex (but I understand your requirements.
>> 
>>> From ph at imatix.com  Fri Oct  4 00:46:06 2013
>>> From: ph at imatix.com (Pieter Hintjens)
>>> Date: Fri, 4 Oct 2013 00:46:06 +0200
>>> Subject: [zeromq-dev] Using other kinds of certificates with CurveZMQ
>>> In-Reply-To: >> y.corp.yahoo.com>
>>> References: >> .corp.yahoo.com>
>>> Message-ID: 

[zeromq-dev] Bidirectional Async Over TCP Without Multiple Binds

[Jake]

What do would be an appropriate pattern to achieve bidirectional,
asynchronous messages between two nodes over TCP transport without binding
more than one TCP port?  The two nodes would be in different processes at
least and possibly on different machines.  I can imagine that a pair of
PUSH->PULL sockets would enable this, where Node A and Node B each bind a
TCP port with a PULL socket and connect to each other with a PUSH socket.
However, I will have a one-to-many relationship between an A node and
several B nodes, so needing a distinct port on A for each B limits the
design.  Ideally, only B nodes would bind a port for their A node to
connect to.

-Jake Cobb
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Is there a reasonable way to use an existing PKI and D/TLS with 0MQ?

[Luca Boccassi]

On Wed, 2018-02-14 at 00:17 -0500, John Lane Schultz wrote:
> Hi Luca,
> 
> Thanks for the update.  I was holding out hope that someone might
> have taken a crack at it already.
> 
> Yes, TLS is a pretty complicated protocol that supports tons of
> different cipher suites and options.  I certainly wouldn’t recommend
> reimplementing the protocol natively inside 0MQ.
> 
> I’m just curious about how hard it would be for 0MQ to use (and
> expose) a D/TLS implementation instead of TCP / UDP?
> 
> I’m guessing the primary obstacles would be dependence on an external
> D/TLS library (which maybe could be addressed with conditional
> compilation), how to have a simple 0MQ API that exposes the
> functionality / configurability of the D/TLS transport, and possibly
> objections to the security models (e.g. - X509 certs, CAs, HMAC then
> encrypt, etc.) of TLS itself.
> 
> Thanks,
> John

We already support a number of optional transports that require
external libraries, like PGM and NORM, so that would not be a problem.
If anyone wants to implement it, they would be most welcome.

In terms of difficulty, plugging in a new transport is not simple
plug but neither is too hard - apart from the new mechanism
subclass, which should be not too difficult as the interface is fairly
small, it would need some changes in a few other places - with those we
can help eventually.

> On Feb 13, 2018, at 5:42 PM, Luca Boccassi 
> wrote:
> 
> Hi,
> 
> The situation is the same - security is only supported through Curve
> or
> Kerberos.
> 
> Not for any particular reason if not that nobody has contributed any
> other implementation. This is probably due to the fact that SSL is
> awfully, awfully complex (but I understand your requirements.
> 

-- 
Kind regards,
Luca Boccassi

signature.asc
Description: This is a digitally signed message part
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Bidirectional Async Over TCP Without Multiple Binds

[Justin Azoff]

You would most likely use a ZMQ_ROUTER socket on both ends:

Summary of ZMQ_ROUTER characteristics:
Compatible peer sockets ZMQ_DEALER, ZMQ_REQ, ZMQ_ROUTER
Direction Bidirectional


On Wed, Feb 14, 2018 at 9:08 AM, Jake  wrote:
> What do would be an appropriate pattern to achieve bidirectional,
> asynchronous messages between two nodes over TCP transport without binding
> more than one TCP port?  The two nodes would be in different processes at
> least and possibly on different machines.  I can imagine that a pair of
> PUSH->PULL sockets would enable this, where Node A and Node B each bind a
> TCP port with a PULL socket and connect to each other with a PUSH socket.
> However, I will have a one-to-many relationship between an A node and
> several B nodes, so needing a distinct port on A for each B limits the
> design.  Ideally, only B nodes would bind a port for their A node to connect
> to.
>
> -Jake Cobb
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>



-- 
- Justin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Bidirectional Async Over TCP Without Multiple Binds

[Patrik VV.]

Hi

Not sure why Justin suggested using a ROUTER-Socket on each side. 

I think having the A node bind a ROUTER socket and each B node connect using a 
DEALER socket is enough. One single TCP port would have to be bound. The ROUTER 
socket on the A node is needed to distinguish between the different B nodes 
when receiving/sending messages. The B nodes don’t need to make that 
distinction because it’ll always be the A node implicitly. 

Regards,
Patrik

> On 14 Feb 2018, at 15:14, Justin Azoff  wrote:
> 
> You would most likely use a ZMQ_ROUTER socket on both ends:
> 
> Summary of ZMQ_ROUTER characteristics:
> Compatible peer sockets ZMQ_DEALER, ZMQ_REQ, ZMQ_ROUTER
> Direction Bidirectional
> 
> 
>> On Wed, Feb 14, 2018 at 9:08 AM, Jake  wrote:
>> What do would be an appropriate pattern to achieve bidirectional,
>> asynchronous messages between two nodes over TCP transport without binding
>> more than one TCP port?  The two nodes would be in different processes at
>> least and possibly on different machines.  I can imagine that a pair of
>> PUSH->PULL sockets would enable this, where Node A and Node B each bind a
>> TCP port with a PULL socket and connect to each other with a PUSH socket.
>> However, I will have a one-to-many relationship between an A node and
>> several B nodes, so needing a distinct port on A for each B limits the
>> design.  Ideally, only B nodes would bind a port for their A node to connect
>> to.
>> 
>> -Jake Cobb
>> 
>> ___
>> zeromq-dev mailing list
>> zeromq-dev@lists.zeromq.org
>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> 
> 
> 
> 
> -- 
> - Justin
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Bidirectional Async Over TCP Without Multiple Binds

[Patrik Wenger]

Hi Jake

In fact, most ZMQ socket types do some sort of load balancing when 
connected with multiple other sockets. See [1], especially the "Outgoing 
routing strategy" and "Incoming routing strategy" in the table for each 
socket type. TL;DR if the socket allows receiving messages and is 
connected to multiple other sockets, it will try to read from all of 
those sockets "fairly". If the socket allows sending messages and is 
connected to multiple other sockets, it will usually round-robin when 
sending.


Of course PUB (and RADIO) will not round-robin, but "fan out" instead, 
meaning all connected sockets will receive every message.


And ROUTER is an exception too: When sending, it uses the first message 
part (set by the application) to determine the correct receiver. This 
allows you to send a message to one specific connected socket. When 
receiving, it prepends a message part that identifies the socket which 
sent the message. Your application will have to inspect that first part 
to know who is its sender. These two things will have to happen in your 
A node. The DEALER sockets in your B nodes will have to set an identity 
to be distinguishable in A's ROUTER socket.


Regards,
Patrik

[1] http://api.zeromq.org/4-2:zmq-socket


On 14.02.2018 20:17, Jake wrote:
Isn't ROUTER-DEALER load balancing though?  I've used it in a 
REQ-ROUTER-DEALER-REP set up
but in that case the REQ -> ROUTER send does not target a particular 
handler on the other side, it
just goes to an arbitrary one and the ROUTER makes sure the reply goes 
back to the original sender.
How would my sender (an A node) target a particular B node to receive 
a message?


-Jake

On Wed, Feb 14, 2018 at 12:34 PM, Justin Azoff > wrote:


On Wed, Feb 14, 2018 at 12:28 PM, Patrik VV. > wrote:
> Hi
>
> Not sure why Justin suggested using a ROUTER-Socket on each side.

Ah yes, I misread the original message. For "one-to-many" you would
use router-dealer.  router+router would be more for many-to-many where
each node runs the same code and there is no obvious master node.


--
- Justin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org 
https://lists.zeromq.org/mailman/listinfo/zeromq-dev





___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Bidirectional Async Over TCP Without Multiple Binds

[Justin Azoff]

On Wed, Feb 14, 2018 at 12:28 PM, Patrik VV.  wrote:
> Hi
>
> Not sure why Justin suggested using a ROUTER-Socket on each side.

Ah yes, I misread the original message. For "one-to-many" you would
use router-dealer.  router+router would be more for many-to-many where
each node runs the same code and there is no obvious master node.


-- 
- Justin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Is there a reasonable way to use an existing PKI and D/TLS with 0MQ?

[John Lane Schultz]

That’s very cool that (almost) any random number can be used for the private 
key and the public key can be easily derived from the private one!

See?  I am quite clueless about the details of ECC. ;)

John

On Feb 14, 2018, at 4:05 AM, paddor  wrote:

Hi John,

One cool thing about Curve25519 is that, given the secret key, the public key 
can be derived from it. And it seems that you actually you can use random bytes 
for the secret key (almost). Only a few bits in the first and last byte are 
fixed. Here an excerpt from [1]:

> Computing secret keys. Inside your program, to generate a 32-byte Curve25519 
> secret key, start by generating 32 secret random bytes from a 
> cryptographically safe source: mysecret[0], mysecret[1], ..., mysecret[31]. 
> Then do
> 
> mysecret[0] &= 248;
> mysecret[31] &= 127;
> mysecret[31] |= 64;
> 
> to create a 32-byte Curve25519 secret key mysecret[0], mysecret[1], ..., 
> mysecret[31].

Of course the conversion tool would have to print the public key so you can 
collect those centrally.

Yes, it might be too much effort if you want to keep using the existing PKI 
as-is. Having used ZMQ for a while now, I'd say it's worth the effort! ;-)

Regards,
Patrik

[1] https://cr.yp.to/ecdh.html

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Bidirectional Async Over TCP Without Multiple Binds

[Jake]

Isn't ROUTER-DEALER load balancing though?  I've used it in a
REQ-ROUTER-DEALER-REP set up
but in that case the REQ -> ROUTER send does not target a particular
handler on the other side, it
just goes to an arbitrary one and the ROUTER makes sure the reply goes back
to the original sender.
How would my sender (an A node) target a particular B node to receive a
message?

-Jake

On Wed, Feb 14, 2018 at 12:34 PM, Justin Azoff 
wrote:

> On Wed, Feb 14, 2018 at 12:28 PM, Patrik VV.  wrote:
> > Hi
> >
> > Not sure why Justin suggested using a ROUTER-Socket on each side.
>
> Ah yes, I misread the original message. For "one-to-many" you would
> use router-dealer.  router+router would be more for many-to-many where
> each node runs the same code and there is no obvious master node.
>
>
> --
> - Justin
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Bidirectional Async Over TCP Without Multiple Binds

[Jake]

Ok, thanks for the explanation and doc reference.  My A node spawns the B
nodes in the
first place, so I guess I can either (1) assign the identity out-of-band at
spawn time and
have the B use it, or (2) have B handshake once at start up to allow A to
associate the
zmq-assigned identity.  After that B just receives and sends on DEALER;
without other
sockets connected there is no load balancing on B's end.  Node A uses the
identity association
to send to a particular B and to recognize which B is sending when it
receives a message.

-Jake

On Wed, Feb 14, 2018 at 4:25 PM, Patrik Wenger  wrote:

> Hi Jake
>
> In fact, most ZMQ socket types do some sort of load balancing when
> connected with multiple other sockets. See [1], especially the "Outgoing
> routing strategy" and "Incoming routing strategy" in the table for each
> socket type. TL;DR if the socket allows receiving messages and is connected
> to multiple other sockets, it will try to read from all of those sockets
> "fairly". If the socket allows sending messages and is connected to
> multiple other sockets, it will usually round-robin when sending.
>
> Of course PUB (and RADIO) will not round-robin, but "fan out" instead,
> meaning all connected sockets will receive every message.
>
> And ROUTER is an exception too: When sending, it uses the first message
> part (set by the application) to determine the correct receiver. This
> allows you to send a message to one specific connected socket. When
> receiving, it prepends a message part that identifies the socket which sent
> the message. Your application will have to inspect that first part to know
> who is its sender. These two things will have to happen in your A node. The
> DEALER sockets in your B nodes will have to set an identity to be
> distinguishable in A's ROUTER socket.
>
> Regards,
> Patrik
>
> [1] http://api.zeromq.org/4-2:zmq-socket
>
> On 14.02.2018 20:17, Jake wrote:
>
> Isn't ROUTER-DEALER load balancing though?  I've used it in a
> REQ-ROUTER-DEALER-REP set up
> but in that case the REQ -> ROUTER send does not target a particular
> handler on the other side, it
> just goes to an arbitrary one and the ROUTER makes sure the reply goes
> back to the original sender.
> How would my sender (an A node) target a particular B node to receive a
> message?
>
> -Jake
>
> On Wed, Feb 14, 2018 at 12:34 PM, Justin Azoff 
> wrote:
>
>> On Wed, Feb 14, 2018 at 12:28 PM, Patrik VV.  wrote:
>> > Hi
>> >
>> > Not sure why Justin suggested using a ROUTER-Socket on each side.
>>
>> Ah yes, I misread the original message. For "one-to-many" you would
>> use router-dealer.  router+router would be more for many-to-many where
>> each node runs the same code and there is no obvious master node.
>>
>>
>> --
>> - Justin
>> ___
>> zeromq-dev mailing list
>> zeromq-dev@lists.zeromq.org
>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
>
>
> ___
> zeromq-dev mailing 
> listzeromq-dev@lists.zeromq.orghttps://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Bidirectional Async Over TCP Without Multiple Binds

[Patrik Wenger]

Ah right, I guess there will be a ZMQ-assigned identity if you don't set 
one. Wasn't able to find that in the docs right away though. I found it 
in the ZGuide now though. Seems like the ROUTER itself will invent those 
for each connected socket that doesn't announce its own identity.


Happy hacking!

Regards,
Patrik


On 14.02.2018 22:40, Jake wrote:
Ok, thanks for the explanation and doc reference. My A node spawns the 
B nodes in the
first place, so I guess I can either (1) assign the identity 
out-of-band at spawn time and
have the B use it, or (2) have B handshake once at start up to allow A 
to associate the
zmq-assigned identity.  After that B just receives and sends on 
DEALER; without other
sockets connected there is no load balancing on B's end.  Node A uses 
the identity association
to send to a particular B and to recognize which B is sending when it 
receives a message.


-Jake

On Wed, Feb 14, 2018 at 4:25 PM, Patrik Wenger > wrote:


Hi Jake

In fact, most ZMQ socket types do some sort of load balancing when
connected with multiple other sockets. See [1], especially the
"Outgoing routing strategy" and "Incoming routing strategy" in the
table for each socket type. TL;DR if the socket allows receiving
messages and is connected to multiple other sockets, it will try
to read from all of those sockets "fairly". If the socket allows
sending messages and is connected to multiple other sockets, it
will usually round-robin when sending.

Of course PUB (and RADIO) will not round-robin, but "fan out"
instead, meaning all connected sockets will receive every message.

And ROUTER is an exception too: When sending, it uses the first
message part (set by the application) to determine the correct
receiver. This allows you to send a message to one specific
connected socket. When receiving, it prepends a message part that
identifies the socket which sent the message. Your application
will have to inspect that first part to know who is its sender.
These two things will have to happen in your A node. The DEALER
sockets in your B nodes will have to set an identity to be
distinguishable in A's ROUTER socket.

Regards,
Patrik

[1] http://api.zeromq.org/4-2:zmq-socket



On 14.02.2018 20:17, Jake wrote:

Isn't ROUTER-DEALER load balancing though?  I've used it in a
REQ-ROUTER-DEALER-REP set up
but in that case the REQ -> ROUTER send does not target a
particular handler on the other side, it
just goes to an arbitrary one and the ROUTER makes sure the reply
goes back to the original sender.
How would my sender (an A node) target a particular B node to
receive a message?

-Jake

On Wed, Feb 14, 2018 at 12:34 PM, Justin Azoff
> wrote:

On Wed, Feb 14, 2018 at 12:28 PM, Patrik VV.
> wrote:
> Hi
>
> Not sure why Justin suggested using a ROUTER-Socket on each
side.

Ah yes, I misread the original message. For "one-to-many" you
would
use router-dealer.  router+router would be more for
many-to-many where
each node runs the same code and there is no obvious master node.


--
- Justin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org 
https://lists.zeromq.org/mailman/listinfo/zeromq-dev





___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org 
https://lists.zeromq.org/mailman/listinfo/zeromq-dev




___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org 
https://lists.zeromq.org/mailman/listinfo/zeromq-dev





___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev