Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Jabrane Hamdi]

Hi Lionel,

Thank you for your feedback. I think that my real problem is Go and not the
network.
In fact, when i use python, the work is done. But with Go, i have the
problem. You find below the code complete :


//  File Transfer model #1
//
//  In which the server sends the entire file to the client in
//  large chunks with no attempt at flow control.

package main

import (
zmq "github.com/pebbe/zmq4"

"fmt"
"io"
"os"
//"runtime"
)

const (
CHUNK_SIZE = 25
)

func client_thread(pipe chan<- string) {
context,_ := zmq.NewContext()
dealer, _ := context.NewSocket(zmq.DEALER)
dealer.Connect("tcp://127.0.0.1:6000")

dealer.Send("fetch", 0)
total := 0  //  Total bytes received
chunks := 0 //  Total chunks received

for {
frame, err := dealer.RecvBytes(0)
if err != nil {
break //  Shutting down, quit
}
chunks++
size := len(frame)
total += size
if size == 0 {
break //  Whole file received
}
}
fmt.Printf("%v chunks received, %v bytes\n", chunks, total)
pipe <- "OK"
}

//  The server thread reads the file from disk in chunks, and sends
//  each chunk to the client as a separate message. We only have one
//  test file, so open that once and then serve it out as needed:

func server_thread() {
file, err := os.Open("testdata_1G")
if err != nil {
panic(err)
}
context,_ := zmq.NewContext()
router, _ := context.NewSocket(zmq.ROUTER)
//  Default HWM is 1000, which will drop messages here
//  since we send more than 1,000 chunks of test data,
//  so set an infinite HWM as a simple, stupid solution:
router.SetRcvhwm(0)
router.SetSndhwm(0)
router.Bind("tcp://*:6000")
for {
//  First frame in each message is the sender identity
identity, err := router.Recv(0)
if err != nil {
break //  Shutting down, quit
}

//  Second frame is "fetch" command
command, _ := router.Recv(0)
if command != "fetch" {
panic("command != \"fetch\"")
}

chunk := make([]byte, CHUNK_SIZE)
for {
n, _ := io.ReadFull(file, chunk)
router.SendMessage(identity, chunk[:n])
if n == 0 {
break //  Always end with a zero-size frame
}
}
}
file.Close()
}

//  The main task starts the client and server threads; it's easier
//  to test this as a single process with threads, than as multiple
//  processes:

func main() {
pipe1 := make(chan string)
//  Start child threads
go server_thread()
go client_thread(pipe1)
//  Loop until client tells us it's done
<-pipe1
}

2018-05-07 14:47 GMT+02:00 Lionel Flandrin :

> On Wed, May 02, 2018 at 07:33:36PM +0200, Lionel Flandrin wrote:
> > On Sun, Apr 29, 2018 at 08:45:20AM +0300, Doron Somech wrote:
> > > I can try and help regarding adding IPv6 to RADIO/DISH UDP protocol.
> > >
> > > I don't think it should very complicated. I can try and point to you
> to the
> > > places in the code that need to have the support.
> > >
> >
> > Since my PR for the IP parsing code has just been merged I'm going to
> > start working on the UDP code momentarily, first by replacing the
> > existing address parsing code with the new ip_resolver API and then
> > trying to add IPv6 support. Let me know if you have any remarks or
> > complaints.
> >
> > Also I'm on the IRC channel if you want something more interactive for
> > discussion.
>
> I've made a PR for my work in progress to add IPv6 UDP support,
> feedback very welcome:
>
> https://github.com/zeromq/libzmq/pull/3081
>
> In particular there's the raw socket code I'm not sure I understand
> completely (see my comments in the PR).
>
> --
> Lionel Flandrin
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Lionel Flandrin]

On Wed, May 02, 2018 at 07:33:36PM +0200, Lionel Flandrin wrote:
> On Sun, Apr 29, 2018 at 08:45:20AM +0300, Doron Somech wrote:
> > I can try and help regarding adding IPv6 to RADIO/DISH UDP protocol.
> > 
> > I don't think it should very complicated. I can try and point to you to the
> > places in the code that need to have the support.
> > 
> 
> Since my PR for the IP parsing code has just been merged I'm going to
> start working on the UDP code momentarily, first by replacing the
> existing address parsing code with the new ip_resolver API and then
> trying to add IPv6 support. Let me know if you have any remarks or
> complaints.
> 
> Also I'm on the IRC channel if you want something more interactive for
> discussion.

I've made a PR for my work in progress to add IPv6 UDP support,
feedback very welcome:

https://github.com/zeromq/libzmq/pull/3081

In particular there's the raw socket code I'm not sure I understand
completely (see my comments in the PR).

-- 
Lionel Flandrin


signature.asc
Description: PGP signature
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Lionel Flandrin]

On Sun, Apr 29, 2018 at 08:45:20AM +0300, Doron Somech wrote:
> I can try and help regarding adding IPv6 to RADIO/DISH UDP protocol.
> 
> I don't think it should very complicated. I can try and point to you to the
> places in the code that need to have the support.
> 

Since my PR for the IP parsing code has just been merged I'm going to
start working on the UDP code momentarily, first by replacing the
existing address parsing code with the new ip_resolver API and then
trying to add IPv6 support. Let me know if you have any remarks or
complaints.

Also I'm on the IRC channel if you want something more interactive for
discussion.

Cheers,
-- 
Lionel Flandrin


signature.asc
Description: PGP signature
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Lionel Flandrin]

On Sun, Apr 29, 2018 at 08:45:20AM +0300, Doron Somech wrote:
> I can try and help regarding adding IPv6 to RADIO/DISH UDP protocol.
> 
> I don't think it should very complicated. I can try and point to you to the
> places in the code that need to have the support.

Thank you, I've started by factoring the TCP hostname/url parsing code
to be able to reuse it in the UDP code (including IPv6 support and
hostname resolution). It seems to be working but I need to add more
tests before I submit the PR. Once that's done and if it's deemed
acceptable I'll try to work the UDP code to add IPv6 support.

-- 
Lionel Flandrin


signature.asc
Description: PGP signature
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Doron Somech]

I can try and help regarding adding IPv6 to RADIO/DISH UDP protocol.

I don't think it should very complicated. I can try and point to you to the
places in the code that need to have the support.

On Fri, Apr 27, 2018 at 4:09 PM, Luca Boccassi 
wrote:

> On Fri, 2018-04-27 at 15:00 +0200, Lionel Flandrin wrote:
> > On Fri, Apr 27, 2018 at 01:36:13PM +0100, Luca Boccassi wrote:
> > > On Fri, 2018-04-27 at 14:29 +0200, Lionel Flandrin wrote:
> > > > On Thu, Apr 26, 2018 at 10:58:33AM +0100, Luca Boccassi wrote:
> > > > > On Thu, 2018-04-26 at 10:55 +0200, Lionel Flandrin wrote:
> > > > > > On Thu, Apr 26, 2018 at 09:23:13AM +0100, Luca Boccassi
> > > > > > wrote:
> > > > > > > On Thu, 2018-04-26 at 10:00 +0200, Lionel Flandrin wrote:
> > > > > > > > Hello everyone,
> > > > > > > >
> > > > > > > > I'm trying to build a multicast protocol on top of an
> > > > > > > > IPv6-
> > > > > > > > only
> > > > > > > > network. I found that the draft RADIO/DISH sockets seem
> > > > > > > > to do
> > > > > > > > exactly
> > > > > > > > what I want, however the zmq_udp man page doesn't
> > > > > > > > explicitely
> > > > > > > > mention
> > > > > > > > supporting IPv6 multicast and I couldn't get pyzmq to
> > > > > > > > bind an
> > > > > > > > IPv6
> > > > > > > > multicast DISH with any URL format I've tried.
> > > > > > > >
> > > > > > > > Is IPv6 multicast simply currently unsupported for UDP
> > > > > > > > sockets?
> > > > > > > > If
> > > > > > > > that's the case is it because of a technical difficulty
> > > > > > > > or
> > > > > > > > simply
> > > > > > > > because nobody bothered to implement it?
> > > > > > > >
> > > > > > > > Thank you for your help (and your great library),
> > > > > > >
> > > > > > > UDP right now supports only ipv4 - it's a work in progress:
> > > > > > > https://github.com/zeromq/libzmq/issues/2891
> > > > > >
> > > > > > Ah, I see, thank you for confirming that. Do you think adding
> > > > > > IPv6
> > > > > > support would be a huge amount of work for somebody not
> > > > > > familiar
> > > > > > with
> > > > > > ZMQ's codebase? Is it just about adding a few branches
> > > > > > changing
> > > > > > AF_INET to AF_INET6 or am I being ridiculously naive?
> > > > > >
> > > > > > My current backup solution if I can't get ZMQ to do what I
> > > > > > want
> > > > > > is to
> > > > > > write my own IPv6 multicast code using BSD sockets directly,
> > > > > > if
> > > > > > hacking ZMQ's code to add support is not too daunting I could
> > > > > > consider
> > > > > > doing that instead.
> > > > >
> > > > > I'm not too familiar with that module - but it shouldn't be too
> > > > > much
> > > > > work. Address support is the first thing, and then the right
> > > > > socket
> > > > > options for V6 multicast in the engine I suppose:
> > > > >
> > > > > https://github.com/zeromq/libzmq/blob/master/src/udp_address.cp
> > > > > p
> > > > > https://github.com/zeromq/libzmq/blob/master/src/udp_engine.cpp
> > > > >
> > > > > Remember to add unit tests as the very first thing.
> > > >
> > > > I've started doing that and I'm noticing that currently the UDP
> > > > addressing code uses inet_addr to parse the IPv4 address instead
> > > > of
> > > > getaddrinfo like the TCP and PGM code. Is there a reason to avoid
> > > > the
> > > > additional functionality of getaddrinfo here or can I safely
> > > > switch
> > > > to
> > > > it in the UDP code? Do we want to avoid resolving hostnames here
> > > > for
> > > > some reason?
> > >
> > > You can switch, I imagine it was done that way as it was quicker.
> > > Remember to add tests.
> >
> > Got it, thanks.
> >
> > I've already refactored the existing tests to add IPv6:
> >
> > https://github.com/simias/libzmq/commit/16834fd4d2dee3460e3c46f44241e
> > 73f2a3633f8
> >
> > I'm thinking about adding multicast tests as well but I wonder if
> > there are some caveats if I try to send and receive multicast traffic
> > in the test suite? Am I allowed to subscribe to a random multicast
> > address or is it forbidden to genererate external traffic in the
> > tests?
>
> Perhaps add a similar check to the one already present for ipv6 - on
> some build systems networking other than loopback might be blocked, so
> it would fail the tests.
>
> --
> Kind regards,
> Luca Boccassi
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Luca Boccassi]

On Fri, 2018-04-27 at 15:00 +0200, Lionel Flandrin wrote:
> On Fri, Apr 27, 2018 at 01:36:13PM +0100, Luca Boccassi wrote:
> > On Fri, 2018-04-27 at 14:29 +0200, Lionel Flandrin wrote:
> > > On Thu, Apr 26, 2018 at 10:58:33AM +0100, Luca Boccassi wrote:
> > > > On Thu, 2018-04-26 at 10:55 +0200, Lionel Flandrin wrote:
> > > > > On Thu, Apr 26, 2018 at 09:23:13AM +0100, Luca Boccassi
> > > > > wrote:
> > > > > > On Thu, 2018-04-26 at 10:00 +0200, Lionel Flandrin wrote:
> > > > > > > Hello everyone,
> > > > > > > 
> > > > > > > I'm trying to build a multicast protocol on top of an
> > > > > > > IPv6-
> > > > > > > only
> > > > > > > network. I found that the draft RADIO/DISH sockets seem
> > > > > > > to do
> > > > > > > exactly
> > > > > > > what I want, however the zmq_udp man page doesn't
> > > > > > > explicitely
> > > > > > > mention
> > > > > > > supporting IPv6 multicast and I couldn't get pyzmq to
> > > > > > > bind an
> > > > > > > IPv6
> > > > > > > multicast DISH with any URL format I've tried.
> > > > > > > 
> > > > > > > Is IPv6 multicast simply currently unsupported for UDP
> > > > > > > sockets?
> > > > > > > If
> > > > > > > that's the case is it because of a technical difficulty
> > > > > > > or
> > > > > > > simply
> > > > > > > because nobody bothered to implement it?
> > > > > > > 
> > > > > > > Thank you for your help (and your great library),
> > > > > > 
> > > > > > UDP right now supports only ipv4 - it's a work in progress:
> > > > > > https://github.com/zeromq/libzmq/issues/2891
> > > > > 
> > > > > Ah, I see, thank you for confirming that. Do you think adding
> > > > > IPv6
> > > > > support would be a huge amount of work for somebody not
> > > > > familiar
> > > > > with
> > > > > ZMQ's codebase? Is it just about adding a few branches
> > > > > changing
> > > > > AF_INET to AF_INET6 or am I being ridiculously naive?
> > > > > 
> > > > > My current backup solution if I can't get ZMQ to do what I
> > > > > want
> > > > > is to
> > > > > write my own IPv6 multicast code using BSD sockets directly,
> > > > > if
> > > > > hacking ZMQ's code to add support is not too daunting I could
> > > > > consider
> > > > > doing that instead.
> > > > 
> > > > I'm not too familiar with that module - but it shouldn't be too
> > > > much
> > > > work. Address support is the first thing, and then the right
> > > > socket
> > > > options for V6 multicast in the engine I suppose:
> > > > 
> > > > https://github.com/zeromq/libzmq/blob/master/src/udp_address.cp
> > > > p
> > > > https://github.com/zeromq/libzmq/blob/master/src/udp_engine.cpp
> > > > 
> > > > Remember to add unit tests as the very first thing.
> > > 
> > > I've started doing that and I'm noticing that currently the UDP
> > > addressing code uses inet_addr to parse the IPv4 address instead
> > > of
> > > getaddrinfo like the TCP and PGM code. Is there a reason to avoid
> > > the
> > > additional functionality of getaddrinfo here or can I safely
> > > switch
> > > to
> > > it in the UDP code? Do we want to avoid resolving hostnames here
> > > for
> > > some reason?
> > 
> > You can switch, I imagine it was done that way as it was quicker.
> > Remember to add tests.
> 
> Got it, thanks.
> 
> I've already refactored the existing tests to add IPv6:
> 
> https://github.com/simias/libzmq/commit/16834fd4d2dee3460e3c46f44241e
> 73f2a3633f8
> 
> I'm thinking about adding multicast tests as well but I wonder if
> there are some caveats if I try to send and receive multicast traffic
> in the test suite? Am I allowed to subscribe to a random multicast
> address or is it forbidden to genererate external traffic in the
> tests?

Perhaps add a similar check to the one already present for ipv6 - on
some build systems networking other than loopback might be blocked, so
it would fail the tests.

-- 
Kind regards,
Luca Boccassi

signature.asc
Description: This is a digitally signed message part
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Lionel Flandrin]

On Fri, Apr 27, 2018 at 01:36:13PM +0100, Luca Boccassi wrote:
> On Fri, 2018-04-27 at 14:29 +0200, Lionel Flandrin wrote:
> > On Thu, Apr 26, 2018 at 10:58:33AM +0100, Luca Boccassi wrote:
> > > On Thu, 2018-04-26 at 10:55 +0200, Lionel Flandrin wrote:
> > > > On Thu, Apr 26, 2018 at 09:23:13AM +0100, Luca Boccassi wrote:
> > > > > On Thu, 2018-04-26 at 10:00 +0200, Lionel Flandrin wrote:
> > > > > > Hello everyone,
> > > > > > 
> > > > > > I'm trying to build a multicast protocol on top of an IPv6-
> > > > > > only
> > > > > > network. I found that the draft RADIO/DISH sockets seem to do
> > > > > > exactly
> > > > > > what I want, however the zmq_udp man page doesn't explicitely
> > > > > > mention
> > > > > > supporting IPv6 multicast and I couldn't get pyzmq to bind an
> > > > > > IPv6
> > > > > > multicast DISH with any URL format I've tried.
> > > > > > 
> > > > > > Is IPv6 multicast simply currently unsupported for UDP
> > > > > > sockets?
> > > > > > If
> > > > > > that's the case is it because of a technical difficulty or
> > > > > > simply
> > > > > > because nobody bothered to implement it?
> > > > > > 
> > > > > > Thank you for your help (and your great library),
> > > > > 
> > > > > UDP right now supports only ipv4 - it's a work in progress:
> > > > > https://github.com/zeromq/libzmq/issues/2891
> > > > 
> > > > Ah, I see, thank you for confirming that. Do you think adding
> > > > IPv6
> > > > support would be a huge amount of work for somebody not familiar
> > > > with
> > > > ZMQ's codebase? Is it just about adding a few branches changing
> > > > AF_INET to AF_INET6 or am I being ridiculously naive?
> > > > 
> > > > My current backup solution if I can't get ZMQ to do what I want
> > > > is to
> > > > write my own IPv6 multicast code using BSD sockets directly, if
> > > > hacking ZMQ's code to add support is not too daunting I could
> > > > consider
> > > > doing that instead.
> > > 
> > > I'm not too familiar with that module - but it shouldn't be too
> > > much
> > > work. Address support is the first thing, and then the right socket
> > > options for V6 multicast in the engine I suppose:
> > > 
> > > https://github.com/zeromq/libzmq/blob/master/src/udp_address.cpp
> > > https://github.com/zeromq/libzmq/blob/master/src/udp_engine.cpp
> > > 
> > > Remember to add unit tests as the very first thing.
> > 
> > I've started doing that and I'm noticing that currently the UDP
> > addressing code uses inet_addr to parse the IPv4 address instead of
> > getaddrinfo like the TCP and PGM code. Is there a reason to avoid the
> > additional functionality of getaddrinfo here or can I safely switch
> > to
> > it in the UDP code? Do we want to avoid resolving hostnames here for
> > some reason?
> 
> You can switch, I imagine it was done that way as it was quicker.
> Remember to add tests.

Got it, thanks.

I've already refactored the existing tests to add IPv6:

https://github.com/simias/libzmq/commit/16834fd4d2dee3460e3c46f44241e73f2a3633f8

I'm thinking about adding multicast tests as well but I wonder if
there are some caveats if I try to send and receive multicast traffic
in the test suite? Am I allowed to subscribe to a random multicast
address or is it forbidden to genererate external traffic in the
tests?

-- 
Lionel Flandrin


signature.asc
Description: PGP signature
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Luca Boccassi]

On Fri, 2018-04-27 at 14:29 +0200, Lionel Flandrin wrote:
> On Thu, Apr 26, 2018 at 10:58:33AM +0100, Luca Boccassi wrote:
> > On Thu, 2018-04-26 at 10:55 +0200, Lionel Flandrin wrote:
> > > On Thu, Apr 26, 2018 at 09:23:13AM +0100, Luca Boccassi wrote:
> > > > On Thu, 2018-04-26 at 10:00 +0200, Lionel Flandrin wrote:
> > > > > Hello everyone,
> > > > > 
> > > > > I'm trying to build a multicast protocol on top of an IPv6-
> > > > > only
> > > > > network. I found that the draft RADIO/DISH sockets seem to do
> > > > > exactly
> > > > > what I want, however the zmq_udp man page doesn't explicitely
> > > > > mention
> > > > > supporting IPv6 multicast and I couldn't get pyzmq to bind an
> > > > > IPv6
> > > > > multicast DISH with any URL format I've tried.
> > > > > 
> > > > > Is IPv6 multicast simply currently unsupported for UDP
> > > > > sockets?
> > > > > If
> > > > > that's the case is it because of a technical difficulty or
> > > > > simply
> > > > > because nobody bothered to implement it?
> > > > > 
> > > > > Thank you for your help (and your great library),
> > > > 
> > > > UDP right now supports only ipv4 - it's a work in progress:
> > > > https://github.com/zeromq/libzmq/issues/2891
> > > 
> > > Ah, I see, thank you for confirming that. Do you think adding
> > > IPv6
> > > support would be a huge amount of work for somebody not familiar
> > > with
> > > ZMQ's codebase? Is it just about adding a few branches changing
> > > AF_INET to AF_INET6 or am I being ridiculously naive?
> > > 
> > > My current backup solution if I can't get ZMQ to do what I want
> > > is to
> > > write my own IPv6 multicast code using BSD sockets directly, if
> > > hacking ZMQ's code to add support is not too daunting I could
> > > consider
> > > doing that instead.
> > 
> > I'm not too familiar with that module - but it shouldn't be too
> > much
> > work. Address support is the first thing, and then the right socket
> > options for V6 multicast in the engine I suppose:
> > 
> > https://github.com/zeromq/libzmq/blob/master/src/udp_address.cpp
> > https://github.com/zeromq/libzmq/blob/master/src/udp_engine.cpp
> > 
> > Remember to add unit tests as the very first thing.
> 
> I've started doing that and I'm noticing that currently the UDP
> addressing code uses inet_addr to parse the IPv4 address instead of
> getaddrinfo like the TCP and PGM code. Is there a reason to avoid the
> additional functionality of getaddrinfo here or can I safely switch
> to
> it in the UDP code? Do we want to avoid resolving hostnames here for
> some reason?

You can switch, I imagine it was done that way as it was quicker.
Remember to add tests.

-- 
Kind regards,
Luca Boccassi

signature.asc
Description: This is a digitally signed message part
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Lionel Flandrin]

On Thu, Apr 26, 2018 at 10:58:33AM +0100, Luca Boccassi wrote:
> On Thu, 2018-04-26 at 10:55 +0200, Lionel Flandrin wrote:
> > On Thu, Apr 26, 2018 at 09:23:13AM +0100, Luca Boccassi wrote:
> > > On Thu, 2018-04-26 at 10:00 +0200, Lionel Flandrin wrote:
> > > > Hello everyone,
> > > > 
> > > > I'm trying to build a multicast protocol on top of an IPv6-only
> > > > network. I found that the draft RADIO/DISH sockets seem to do
> > > > exactly
> > > > what I want, however the zmq_udp man page doesn't explicitely
> > > > mention
> > > > supporting IPv6 multicast and I couldn't get pyzmq to bind an
> > > > IPv6
> > > > multicast DISH with any URL format I've tried.
> > > > 
> > > > Is IPv6 multicast simply currently unsupported for UDP sockets?
> > > > If
> > > > that's the case is it because of a technical difficulty or simply
> > > > because nobody bothered to implement it?
> > > > 
> > > > Thank you for your help (and your great library),
> > > 
> > > UDP right now supports only ipv4 - it's a work in progress:
> > > https://github.com/zeromq/libzmq/issues/2891
> > 
> > Ah, I see, thank you for confirming that. Do you think adding IPv6
> > support would be a huge amount of work for somebody not familiar with
> > ZMQ's codebase? Is it just about adding a few branches changing
> > AF_INET to AF_INET6 or am I being ridiculously naive?
> > 
> > My current backup solution if I can't get ZMQ to do what I want is to
> > write my own IPv6 multicast code using BSD sockets directly, if
> > hacking ZMQ's code to add support is not too daunting I could
> > consider
> > doing that instead.
> 
> I'm not too familiar with that module - but it shouldn't be too much
> work. Address support is the first thing, and then the right socket
> options for V6 multicast in the engine I suppose:
> 
> https://github.com/zeromq/libzmq/blob/master/src/udp_address.cpp
> https://github.com/zeromq/libzmq/blob/master/src/udp_engine.cpp
> 
> Remember to add unit tests as the very first thing.

I've started doing that and I'm noticing that currently the UDP
addressing code uses inet_addr to parse the IPv4 address instead of
getaddrinfo like the TCP and PGM code. Is there a reason to avoid the
additional functionality of getaddrinfo here or can I safely switch to
it in the UDP code? Do we want to avoid resolving hostnames here for
some reason?

> > > PUB/SUB with PGM or NORM should do what you are looking for.
> > 
> > I considered EPGM briefly but the zmq_pgm man page seems to say that
> > only IPv4 is supported, so I didn't even attempt it:
> > 
> >   A multicast address is specified by an IPv4 multicast address in
> > its
> >   numeric representation.
> > 
> > Maybe I shoudln't have been so quick to dismiss it, I'm going to give
> > it a try. That being said I don't actually need the guarantees of
> > PGM/NORM, in particular I don't care if a frame gets mangled or
> > dropped and I don't need SNDMORE.
> > 
> > Thank you,
> 
> IIRC both libraries should support IPv6 just fine. The manpage is just
> older - if you find it works and want to update it with some v6
> examples, feel free to send a PR.

-- 
Lionel Flandrin


signature.asc
Description: PGP signature
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Lionel Flandrin]

On Thu, Apr 26, 2018 at 08:49:17AM -0400, Brian Adamson wrote:
> Hi Lionel,
> 
> Although it isn’t yet supported with the ZMQ API’s, the NORM
> protocol can be configured with receiver feedback disabled to
> provide an unreliable UDP-like best effort service with the added
> benefit of its message fragmentation and reassembly when messages
> are larger than the network MTU.  Additionally the FEC-based packet
> erasure coding can also be invoked if desired to provide a
> quasi-reliable “better than best effort” service where a number of
> FEC parity packets per coding block (where a “block” is a block of
> packets) can be set that are sent in addition to the user data
> packets.  It would be interesting to explore adding some additional
> options for the NORM binding.  I haven’t had time to spend on that
> but would be happy to help someone who wanted to take a shot at it.

Hello Brian,

It does sound enticing, my main worry is that I run on an embedded
system with relatively limited RAM resources and I try to avoid
unnecessary overhead wherever I can. Well, the fact that I'm using
python shows that it's not entirely true but that's code I can rewrite
in a future update if I want to save a few MB, the communication
protocol I'll have to stick with for the lifetime of the product so
I'm in a bit of a "premature optimization" mindset.

I've been reading about NORM and while it does seem very nice I only
need a tiny subset of the functionality (effectively UDP RADIO/DISH)
so I'm a bit on the fence about adding such a large dependency just
because ZMQ's UDP sockets don't yet support IPv6.

I think I need to give it a little more thought before I commit either
way but I do appreciate your feedback.

> best regards,
> 
> Brian 
> 
> > On Apr 26, 2018, at 4:55 AM, Lionel Flandrin  wrote:
> > 
> > On Thu, Apr 26, 2018 at 09:23:13AM +0100, Luca Boccassi wrote:
> >> On Thu, 2018-04-26 at 10:00 +0200, Lionel Flandrin wrote:
> >>> Hello everyone,
> >>> 
> >>> I'm trying to build a multicast protocol on top of an IPv6-only
> >>> network. I found that the draft RADIO/DISH sockets seem to do exactly
> >>> what I want, however the zmq_udp man page doesn't explicitely mention
> >>> supporting IPv6 multicast and I couldn't get pyzmq to bind an IPv6
> >>> multicast DISH with any URL format I've tried.
> >>> 
> >>> Is IPv6 multicast simply currently unsupported for UDP sockets? If
> >>> that's the case is it because of a technical difficulty or simply
> >>> because nobody bothered to implement it?
> >>> 
> >>> Thank you for your help (and your great library),
> >> 
> >> UDP right now supports only ipv4 - it's a work in progress:
> >> https://github.com/zeromq/libzmq/issues/2891
> > 
> > Ah, I see, thank you for confirming that. Do you think adding IPv6
> > support would be a huge amount of work for somebody not familiar with
> > ZMQ's codebase? Is it just about adding a few branches changing
> > AF_INET to AF_INET6 or am I being ridiculously naive?
> > 
> > My current backup solution if I can't get ZMQ to do what I want is to
> > write my own IPv6 multicast code using BSD sockets directly, if
> > hacking ZMQ's code to add support is not too daunting I could consider
> > doing that instead.
> > 
> >> PUB/SUB with PGM or NORM should do what you are looking for.
> > 
> > I considered EPGM briefly but the zmq_pgm man page seems to say that
> > only IPv4 is supported, so I didn't even attempt it:
> > 
> >  A multicast address is specified by an IPv4 multicast address in its
> >  numeric representation.
> > 
> > Maybe I shoudln't have been so quick to dismiss it, I'm going to give
> > it a try. That being said I don't actually need the guarantees of
> > PGM/NORM, in particular I don't care if a frame gets mangled or
> > dropped and I don't need SNDMORE.
> > 
> > Thank you,


-- 
Lionel Flandrin


signature.asc
Description: PGP signature
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Brian Adamson]

Hi Lionel,

Although it isn’t yet supported with the ZMQ API’s, the NORM protocol can be 
configured with receiver feedback disabled to provide an unreliable UDP-like 
best effort service with the added benefit of its message fragmentation and 
reassembly when messages are larger than the network MTU.  Additionally the 
FEC-based packet erasure coding can also be invoked if desired to provide a 
quasi-reliable “better than best effort” service where a number of FEC parity 
packets per coding block (where a “block” is a block of packets) can be set 
that are sent in addition to the user data packets.  It would be interesting to 
explore adding some additional options for the NORM binding.  I haven’t had 
time to spend on that but would be happy to help someone who wanted to take a 
shot at it.

best regards,

Brian 

> On Apr 26, 2018, at 4:55 AM, Lionel Flandrin  wrote:
> 
> On Thu, Apr 26, 2018 at 09:23:13AM +0100, Luca Boccassi wrote:
>> On Thu, 2018-04-26 at 10:00 +0200, Lionel Flandrin wrote:
>>> Hello everyone,
>>> 
>>> I'm trying to build a multicast protocol on top of an IPv6-only
>>> network. I found that the draft RADIO/DISH sockets seem to do exactly
>>> what I want, however the zmq_udp man page doesn't explicitely mention
>>> supporting IPv6 multicast and I couldn't get pyzmq to bind an IPv6
>>> multicast DISH with any URL format I've tried.
>>> 
>>> Is IPv6 multicast simply currently unsupported for UDP sockets? If
>>> that's the case is it because of a technical difficulty or simply
>>> because nobody bothered to implement it?
>>> 
>>> Thank you for your help (and your great library),
>> 
>> UDP right now supports only ipv4 - it's a work in progress:
>> https://github.com/zeromq/libzmq/issues/2891
> 
> Ah, I see, thank you for confirming that. Do you think adding IPv6
> support would be a huge amount of work for somebody not familiar with
> ZMQ's codebase? Is it just about adding a few branches changing
> AF_INET to AF_INET6 or am I being ridiculously naive?
> 
> My current backup solution if I can't get ZMQ to do what I want is to
> write my own IPv6 multicast code using BSD sockets directly, if
> hacking ZMQ's code to add support is not too daunting I could consider
> doing that instead.
> 
>> PUB/SUB with PGM or NORM should do what you are looking for.
> 
> I considered EPGM briefly but the zmq_pgm man page seems to say that
> only IPv4 is supported, so I didn't even attempt it:
> 
>  A multicast address is specified by an IPv4 multicast address in its
>  numeric representation.
> 
> Maybe I shoudln't have been so quick to dismiss it, I'm going to give
> it a try. That being said I don't actually need the guarantees of
> PGM/NORM, in particular I don't care if a frame gets mangled or
> dropped and I don't need SNDMORE.
> 
> Thank you,
> -- 
> Lionel Flandrin
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org 
> https://lists.zeromq.org/mailman/listinfo/zeromq-dev 
> 
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Luca Boccassi]

On Thu, 2018-04-26 at 10:55 +0200, Lionel Flandrin wrote:
> On Thu, Apr 26, 2018 at 09:23:13AM +0100, Luca Boccassi wrote:
> > On Thu, 2018-04-26 at 10:00 +0200, Lionel Flandrin wrote:
> > > Hello everyone,
> > > 
> > > I'm trying to build a multicast protocol on top of an IPv6-only
> > > network. I found that the draft RADIO/DISH sockets seem to do
> > > exactly
> > > what I want, however the zmq_udp man page doesn't explicitely
> > > mention
> > > supporting IPv6 multicast and I couldn't get pyzmq to bind an
> > > IPv6
> > > multicast DISH with any URL format I've tried.
> > > 
> > > Is IPv6 multicast simply currently unsupported for UDP sockets?
> > > If
> > > that's the case is it because of a technical difficulty or simply
> > > because nobody bothered to implement it?
> > > 
> > > Thank you for your help (and your great library),
> > 
> > UDP right now supports only ipv4 - it's a work in progress:
> > https://github.com/zeromq/libzmq/issues/2891
> 
> Ah, I see, thank you for confirming that. Do you think adding IPv6
> support would be a huge amount of work for somebody not familiar with
> ZMQ's codebase? Is it just about adding a few branches changing
> AF_INET to AF_INET6 or am I being ridiculously naive?
> 
> My current backup solution if I can't get ZMQ to do what I want is to
> write my own IPv6 multicast code using BSD sockets directly, if
> hacking ZMQ's code to add support is not too daunting I could
> consider
> doing that instead.

I'm not too familiar with that module - but it shouldn't be too much
work. Address support is the first thing, and then the right socket
options for V6 multicast in the engine I suppose:

https://github.com/zeromq/libzmq/blob/master/src/udp_address.cpp
https://github.com/zeromq/libzmq/blob/master/src/udp_engine.cpp

Remember to add unit tests as the very first thing.

> > PUB/SUB with PGM or NORM should do what you are looking for.
> 
> I considered EPGM briefly but the zmq_pgm man page seems to say that
> only IPv4 is supported, so I didn't even attempt it:
> 
>   A multicast address is specified by an IPv4 multicast address in
> its
>   numeric representation.
> 
> Maybe I shoudln't have been so quick to dismiss it, I'm going to give
> it a try. That being said I don't actually need the guarantees of
> PGM/NORM, in particular I don't care if a frame gets mangled or
> dropped and I don't need SNDMORE.
> 
> Thank you,

IIRC both libraries should support IPv6 just fine. The manpage is just
older - if you find it works and want to update it with some v6
examples, feel free to send a PR.

-- 
Kind regards,
Luca Boccassi

signature.asc
Description: This is a digitally signed message part
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Lionel Flandrin]

On Thu, Apr 26, 2018 at 09:23:13AM +0100, Luca Boccassi wrote:
> On Thu, 2018-04-26 at 10:00 +0200, Lionel Flandrin wrote:
> > Hello everyone,
> > 
> > I'm trying to build a multicast protocol on top of an IPv6-only
> > network. I found that the draft RADIO/DISH sockets seem to do exactly
> > what I want, however the zmq_udp man page doesn't explicitely mention
> > supporting IPv6 multicast and I couldn't get pyzmq to bind an IPv6
> > multicast DISH with any URL format I've tried.
> > 
> > Is IPv6 multicast simply currently unsupported for UDP sockets? If
> > that's the case is it because of a technical difficulty or simply
> > because nobody bothered to implement it?
> > 
> > Thank you for your help (and your great library),
> 
> UDP right now supports only ipv4 - it's a work in progress:
> https://github.com/zeromq/libzmq/issues/2891

Ah, I see, thank you for confirming that. Do you think adding IPv6
support would be a huge amount of work for somebody not familiar with
ZMQ's codebase? Is it just about adding a few branches changing
AF_INET to AF_INET6 or am I being ridiculously naive?

My current backup solution if I can't get ZMQ to do what I want is to
write my own IPv6 multicast code using BSD sockets directly, if
hacking ZMQ's code to add support is not too daunting I could consider
doing that instead.

> PUB/SUB with PGM or NORM should do what you are looking for.

I considered EPGM briefly but the zmq_pgm man page seems to say that
only IPv4 is supported, so I didn't even attempt it:

  A multicast address is specified by an IPv4 multicast address in its
  numeric representation.

Maybe I shoudln't have been so quick to dismiss it, I'm going to give
it a try. That being said I don't actually need the guarantees of
PGM/NORM, in particular I don't care if a frame gets mangled or
dropped and I don't need SNDMORE.

Thank you,
-- 
Lionel Flandrin


signature.asc
Description: PGP signature
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] IPv6 multicast RADIO/DISH

[Luca Boccassi]

On Thu, 2018-04-26 at 10:00 +0200, Lionel Flandrin wrote:
> Hello everyone,
> 
> I'm trying to build a multicast protocol on top of an IPv6-only
> network. I found that the draft RADIO/DISH sockets seem to do exactly
> what I want, however the zmq_udp man page doesn't explicitely mention
> supporting IPv6 multicast and I couldn't get pyzmq to bind an IPv6
> multicast DISH with any URL format I've tried.
> 
> Is IPv6 multicast simply currently unsupported for UDP sockets? If
> that's the case is it because of a technical difficulty or simply
> because nobody bothered to implement it?
> 
> Thank you for your help (and your great library),

UDP right now supports only ipv4 - it's a work in progress:
https://github.com/zeromq/libzmq/issues/2891

PUB/SUB with PGM or NORM should do what you are looking for.

-- 
Kind regards,
Luca Boccassi

signature.asc
Description: This is a digitally signed message part
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev