Re: [PD-dev] SIGPIPE on iemnet's tcpserver
hi, thanks for that, I'm currently testing the Iohannes version that fix the SIGPIPE crash I realized my patches crash more often since I'm using [tcpserver], this is why I tested it alone a lot and find some issues i tried to overcome I think I will run the server in a separate instance of Pd to avoid crash and blocked ports + a -- do it yourself http://antoine.villeret.free.fr 2013/7/4 Roman Haefeli reduz...@gmail.com On Don, 2013-07-04 at 14:20 +0200, Antoine Villeret wrote: ok thanks for the explanation so, for now, there is now way to setup a working [udpserver] in pd right ? if so, I will stay with [tcpserver] even if I don't need tcp... I think that is your best bet. I'm still not quite clear what problems you're trying to overcome. Have you tried my patches? Do they work? If not, in what way do they fail? In the meanwhile I came to the conclusion it's probably not worth putting too much time into a server failure proof design. If the sole purpose of the server is to act as a message relay between clients, it will most likely run stable enough. For instance, the netpd server which is also basically a relay for messages usually runs for months without crashing. If you have a stable server, many of the workarounds you had to implement are not necessary. 'Simple' often equals stable. Roman ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
On Fre, 2013-07-05 at 17:30 +0200, Antoine Villeret wrote: I think I will run the server in a separate instance of Pd to avoid crash and blocked ports This is exactly what I meant. Free the server of any other role, just keep the message relay part and isolate it from the rest of your setup (own Pd instance). Once this part is stable, the rest is much easier to handle. Roman ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
2013/7/3 IOhannes m zmoelnig zmoel...@iem.at -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-03 17:33, Antoine Villeret wrote: so it could be difficult to use a server which doesn't accept more than one connection... no that's not what i meant. you can have as many connections as you want, but they cannot be maintained at the same time. simple example: - - both clientA and clientB send a a query to the server - - to complicate things, they do so at exactly the same time - - but since IP is a serial protocol, they will somehow arrive one after each other - let's assume clientB was faster - - [udpserver] will output the query from clientB - - if the server-patch now immediately responds to that query, the response will be sent back to clientB - - then [udpserver] will output the query from clientA. - - [udpserver] will forget everything about clientB - - if the server-patch responds immediately to that query, the response will be sent back to clientB - - if you later send something from the server, it will still be sent to clientB (because clientB is the last known connection) so in this case, if I understand correctly, udpserver never send an answer to clientA ? I have to disconnect clientB *before* connecting clientA ? but how clientA will know this is time to connect ? should it try until the connection is accepted ? so you can have as many connections as you want, but they have to be handled atomically - on after another. you cannot have clientA and clientB connect to the server, and make the server send info to both simultaneously. instead you have to adapt a query/response scheme, where each client asks the server for piece of information. this what I'm doing now, the client connects then ask for something and when it receive the answer it disconnects if there is no answer it disconnect after xx second concerning [udpserver], I try to reproduce the scenario above with your github's version : I start 3 pd : one with udpserver-help.pd, and the 2 other with udpclient-help.pd I connect the clientA, send some data, the server receives them and the server sends back something else to the client then I disconnect the clientA and connect clientB, but when clientB tries to send data I got in the pd console on server side : [udpserver] received disconnection event and in the command line : iemnet:sender-shutdown: Transport endpoint is not connected and on the client side, in the terminal I got : iemnet:sender-shutdown: Bad file descriptor I know [udpserver] does not work yet but as we are talking about it, I want to try your version I just file a report for that thanks for all antoine fgamsdr IOhannes -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHUSAkACgkQkX2Xpv6ydvQ7dQCfWd6Ms++xbvpFYHMUArPILPeA fA8AoOo0uQuuQnJV5FBafbRsggXOqtzx =29SY -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-04 13:55, Antoine Villeret wrote: 2013/7/3 IOhannes m zmoelnig zmoel...@iem.at On 2013-07-03 17:33, Antoine Villeret wrote: so it could be difficult to use a server which doesn't accept more than one connection... no that's not what i meant. you can have as many connections as you want, but they cannot be maintained at the same time. simple example: - both clientA and clientB send a a query to the server - to complicate things, they do so at exactly the same time - but since IP is a serial protocol, they will somehow arrive one after each other - let's assume clientB was faster - [udpserver] will output the query from clientB - if the server-patch now immediately responds to that query, the response will be sent back to clientB - then [udpserver] will output the query from clientA. - [udpserver] will forget everything about clientB - if the server-patch responds immediately to that query, the response will be sent back to clientB - if you later send something from the server, it will still be sent to clientB (because clientB is the last known connection) so in this case, if I understand correctly, udpserver never send an answer to clientA ? no, it does send and answer back. I have to disconnect clientB *before* connecting clientA ? no. UDP doesn't know about connections. but how clientA will know this is time to connect ? should it try until the connection is accepted ? again, UDP is connection-less so there is no connection. i think the main problem here comes from the use of the symbol connect for interfacing with e.g. [udpsend]. this message is named connect mainly for consistency with the tcp/ip objects. anyhow, when you connect a client to server, the client will open a socket for this connection. the server won't know anything about this connection, but it will receive data on a it's listening socket. it can use the socket to send data back (if you have routers/switches/... inbetween, this sending back will only work for a limited amount of time). since the server doesn't know anything about the connection-state of the clients, you don't have to disconnect anything. fgmasdr IOhannes -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHVZt0ACgkQkX2Xpv6ydvS1DQCg3OWfgTqDbH6P52s+1S5FOoJt q3IAn3fxytsJlhVAmepjfXsakZIYGFiT =djBg -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
ok thanks for the explanation so, for now, there is now way to setup a working [udpserver] in pd right ? if so, I will stay with [tcpserver] even if I don't need tcp... but if I could help in debugging [udpserver], I'll be happy to do so, just let me know what can I do cheers a -- do it yourself http://antoine.villeret.free.fr 2013/7/4 IOhannes m zmoelnig zmoel...@iem.at -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-04 13:55, Antoine Villeret wrote: 2013/7/3 IOhannes m zmoelnig zmoel...@iem.at On 2013-07-03 17:33, Antoine Villeret wrote: so it could be difficult to use a server which doesn't accept more than one connection... no that's not what i meant. you can have as many connections as you want, but they cannot be maintained at the same time. simple example: - both clientA and clientB send a a query to the server - to complicate things, they do so at exactly the same time - but since IP is a serial protocol, they will somehow arrive one after each other - let's assume clientB was faster - [udpserver] will output the query from clientB - if the server-patch now immediately responds to that query, the response will be sent back to clientB - then [udpserver] will output the query from clientA. - [udpserver] will forget everything about clientB - if the server-patch responds immediately to that query, the response will be sent back to clientB - if you later send something from the server, it will still be sent to clientB (because clientB is the last known connection) so in this case, if I understand correctly, udpserver never send an answer to clientA ? no, it does send and answer back. I have to disconnect clientB *before* connecting clientA ? no. UDP doesn't know about connections. but how clientA will know this is time to connect ? should it try until the connection is accepted ? again, UDP is connection-less so there is no connection. i think the main problem here comes from the use of the symbol connect for interfacing with e.g. [udpsend]. this message is named connect mainly for consistency with the tcp/ip objects. anyhow, when you connect a client to server, the client will open a socket for this connection. the server won't know anything about this connection, but it will receive data on a it's listening socket. it can use the socket to send data back (if you have routers/switches/... inbetween, this sending back will only work for a limited amount of time). since the server doesn't know anything about the connection-state of the clients, you don't have to disconnect anything. fgmasdr IOhannes -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHVZt0ACgkQkX2Xpv6ydvS1DQCg3OWfgTqDbH6P52s+1S5FOoJt q3IAn3fxytsJlhVAmepjfXsakZIYGFiT =djBg -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
On Don, 2013-07-04 at 14:20 +0200, Antoine Villeret wrote: ok thanks for the explanation so, for now, there is now way to setup a working [udpserver] in pd right ? if so, I will stay with [tcpserver] even if I don't need tcp... I think that is your best bet. I'm still not quite clear what problems you're trying to overcome. Have you tried my patches? Do they work? If not, in what way do they fail? In the meanwhile I came to the conclusion it's probably not worth putting too much time into a server failure proof design. If the sole purpose of the server is to act as a message relay between clients, it will most likely run stable enough. For instance, the netpd server which is also basically a relay for messages usually runs for months without crashing. If you have a stable server, many of the workarounds you had to implement are not necessary. 'Simple' often equals stable. Roman ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
Hi Martin, thanks for that, but I don't know how many clients there are so it's quite difficult to figure out port number in this case. My problem is that I can't listen several time on the same port with [udpreceive] but with multicasting, that's why I switched to [udpserver]. thanks a -- do it yourself http://antoine.villeret.free.fr 2013/7/2 Martin Peach martin.pe...@sympatico.ca On 2013-07-02 14:38, Antoine Villeret wrote: all of this is quite complex for a not so difficult task : I want to make several instance of Pd talking to each other on one machine I started with udpsend/udpreceive but I found no way to do broadcast on localhost and I do need to have lots of clients You don't need to broadcast on a single machine, just use different port numbers. You can send to several ports with several [udpsend]s. Martin ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
2013/7/3 Roman Haefeli reduz...@gmail.com On Die, 2013-07-02 at 20:38 +0200, Antoine Villeret wrote: Hi roman, the code around networking object are some workarounds for differents bugs, the first was a crash when client number reach 32, but it seems to be fixed, I can't reproduce this anymore the second was something strange if several clients connect to the same server at the same time (when patch launch for example), so I add a timeout, if no answer in this timeout, the client disconnect there is also a mechanism to choose an available port at startup in server and the client try different port until it get a connection... Ok, I think I understand a bit more now. The server tries a different port when the first port is not available right? It scans until it finds one. I don't see what the client is supposed to do. There is some code to scan through ports, but actually when I tested your client, it always only tried 1. the client try to connect to a port if there is a server on that port, it gets a connection if there is no server, it can't get connected and so the client tries another port. at least it's supposed to do so... I don't think that dynamic ports are a good solution. If you find a proper way to disconnect the clients, then you shouldn't experience the problem of a busy port when restarting the server. This only happens when you close the server patch without disconnecting the clients. Anyway, the port scanning of the clients would be so slow that probably by the time they found the correct port, the initial port would be free again (depending on OS, though). yep, I'm aware that dynamic port finding is not the best solution but due to bugs related before, iemnet's [tcpserver] used to crash pd. And so port stay busy until a certain timeout (here 60 sec), finding a new available port restores the installation in a working state more quickly. Then I think your way of testing might expose bugs that you wouldn't experience in real life. I mean that is good for bug reporting, but it isn't really meaningful to connect many clients in the same instance of Pd, even less so to connect them all in zero logical time. If you would run your clients in different instances of Pd (what I think is your plan anyway), you wouldn't have to find workarounds for issues exposed by running clients in the same instance. yes of course my patches stress the server a lot but it's only to delimit where the bug is. and at least it works for that :-) I figured you can only properly disconnect all clients with [net/tcpserver]. Unlike [iemnet/tcpserver], it has no 'disconnect' method. So you have to disconnect each client separately. [iemnet/tcpserver] is supposed to disconnect all clients with 'disconnect', but actually the [iemnet/tcpclient]s seem to ignore it and keep their connected state (outlet 3 doesn't switch to 0). Finishing sessions seems generally to be still a problem with iemnet. Also when a client disconnects, the server often doesn't change the number of connected clients. The net library seems much more robust in this regard. I simplified your patches a lot and as simple as they are, they seem to work fine here. Note about the server: you need to disconnect all clients before you close the server patch. This will the leave the network socket in a clean state and you will be able to occupy that port immediately again. The client tries to connect at a rate of one try per second. In case it gets disconnected by the server, it waits 5 seconds before trying to connect again. This gives you time to disconnect all clients and shutdown the server properly. hum, I can't be sure that all clients are properly disconnected because the pd where the server is could crash at anytime for lots of other reasons... Behind that there is a shared memory id manager. On the server side, I generate ids then try if I can get memory segment, if so I send this id to all clients that need it. When a client gets connected, it asks for an id for a give path (a string like /video_server or /cam1) if it exist the server send it back to the client, if no, it generates a new id. all of this is quite complex for a not so difficult task : I want to make several instance of Pd talking to each other on one machine I started with udpsend/udpreceive but I found no way to do broadcast on localhost You mean broadcast on all ports? I don't think that something like that exists. I mean listening several clients on the same port on the same machine (that is a server, right ?) and I do need to have lots of clients multicasting is not a solution because it need to have a working networking interface If you want to run many instances of Pd on localhost, then multicast can't help you. It is about sending data to many clients on different IPs at the same time. You want to send to many clients on localhost. then I tried udpserver which
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
yep, a software shouldn't crash in theory but in real life, I should take crash into account and find some way to restart the system in a working state as fast as possible (so that, nobody suffers from the crash). another solution is to correct all bugs, but it's too hard for me :-) (and many are not in Pd but in separate library like v4l2...) thanks for all, a -- do it yourself http://antoine.villeret.free.fr 2013/7/3 Roman Haefeli reduz...@gmail.com On Mit, 2013-07-03 at 00:16 +0200, Roman Haefeli wrote: then I tried udpserver which doesn't work (at least the version in the pd's SVN) then I switch to tcpserver and I got a lots of troubles... Things look good as long as you think you only need streams. As soon as you figure out that you need to send packets between your instances of Pd, things get really complex wit [tcpserver]. Have you also tried [maxlib/netserver]? It is the oldest of all and does only send FUDI. For connecting many Pds together this might be the easiest to use. It's actually quite stable since it isn't threaded, but it blocks Pd if its buffer is full. This might not be an issue under normal circumstances as the buffer should get emptied quickly for connections on localhost. Problems arise when a client disappear without disconnecting properly. [maxlib/netserver] wants to still send data to such a vanished client and so the buffer is filled until it blocks Pd and also the traffic for all other clients. If that happens, there is nothing much you can do about but closing Pd. This means your clients must not crash and must not be closed without closing the connection. Networking doesn't come for free in Pd. Roman ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
On Wed, 2013-07-03 at 12:56 +0200, Antoine Villeret wrote: then I tried udpserver which doesn't work (at least the version in the pd's SVN) then I switch to tcpserver and I got a lots of troubles... Things look good as long as you think you only need streams. As soon as you figure out that you need to send packets between your instances of Pd, things get really complex wit [tcpserver]. no, I don't really need stream I choose [udpserver] because it's the only server in the iemnet's folder that works yep, it's a stupid reason... I should better try another udpserver somewhere else... :-) I think iemnet is the only library with a [udpserver] implementation. It doesn't seem to be working and also the help-file reflects that. Actually, a working [udpserver] would be practical when dealing with packets, much more so than a [tcpserver]. I will try out iohannes version of iemnet and I'll also investigate those bugs My _personal_ opinion is that helping fix bugs for iemnet is rather worth effort than for net. I gave up on net, especially on [net/tcpserver] and am exclusively using iemnet. iemnet's design is a bit different in that it puts each socket into its own thread which has some advantages: * iemnet hardly ever blocks Pd * iemnet is quite fast performance-wise * iemnet hasn't exposed any data integrity issues cons: * connection state is not reported correctly (incl. num of connected clients) * is by design more prone to crashes. However, I haven't experienced them for quite a while. I consider it quite stable nowadays I haven't found a version of [net/tcpserver] that reliably ensures data integrity. Under load I always received mixed up or even lost chunks of data. This was on Linux. Your mileage may vary if you are on Windows, though. Roman ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
On 2013-07-03 06:37, Antoine Villeret wrote: Hi Martin, thanks for that, but I don't know how many clients there are so it's quite difficult to figure out port number in this case. My problem is that I can't listen several time on the same port with [udpreceive] but with multicasting, that's why I switched to [udpserver]. Not sure what you want to do here. Do you want several [udpreceive]s listening to the same port, or a single [udpreceive] that listens on multiple ports, or a single [udpreceive] that listens on the same port for messages from multiple sources? Or something else... Martin ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
sorry I was not clear enough I need a server : listening on one port and sending data to client on different port i first use only udpsend/udpreceive and the 'server' was sending to a multicast group, each client join this group and receive all the data this is not possible with unicast while I can't listen several time on one port but multicasting need a working network interface (the computer need to be in a network) antoine -- do it yourself http://antoine.villeret.free.fr 2013/7/3 Martin Peach martin.pe...@sympatico.ca On 2013-07-03 06:37, Antoine Villeret wrote: Hi Martin, thanks for that, but I don't know how many clients there are so it's quite difficult to figure out port number in this case. My problem is that I can't listen several time on the same port with [udpreceive] but with multicasting, that's why I switched to [udpserver]. Not sure what you want to do here. Do you want several [udpreceive]s listening to the same port, or a single [udpreceive] that listens on multiple ports, or a single [udpreceive] that listens on the same port for messages from multiple sources? Or something else... Martin ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-03 15:58, Antoine Villeret wrote: sorry I was not clear enough I need a server : listening on one port and sending data to client on different port i first use only udpsend/udpreceive and the 'server' was sending to a multicast group, each client join this group and receive all the data this is not possible with unicast while I can't listen several time on one port but multicasting need a working network interface (the computer need to be in a network) i don't think so. but multicast doesn't solve your problem, just as broadcast doesn't solve it. *cast works on an the IP-level, it allows you to address multiple hosts with a single packet. but afaiu, you want to address multiple applications on a single host, which is not so trivial, as each application needs a separate (receiving) port. and there is no multiportcast. so on a single host you have to address each application directly. fgmasdr IOhannes -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHUMD8ACgkQkX2Xpv6ydvTeGwCg8RgxCCzGiUnPAnCCQGIRIZyp QUgAoNrrfQTaViIcg58r5Tsb7Cnvln08 =V9PY -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-03 16:12, IOhannes m zmoelnig wrote: i guess you meant [tcpserver] instead of [udpserver]. in any case, i'm thinking about removing the multi-client feature of iemnet's [udpserver] just to make sure: i did mean [udpserver] (which does not exist in mrpeach/net) in this mail. fgmadsr IOhannes -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHUMc8ACgkQkX2Xpv6ydvRxMQCfbA2cV46KuZNa2ihrn1CqYXLq 0DIAnRdUGfG43AIjtL2OsecJmErZhkhG =QW+N -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
thanks for that, in fact, some of my clients need the same infos and this is why I first try to do multicasting but reaching multiple clients on the same host is different than addressing one packet to different hosts... for now all my clients try to connect around each 10 second to server to query some infos and to update their state but I could guarantee that only one client will try to connect at a time so it could be difficult to use a server which doesn't accept more than one connection... or maybe i'm wrong and there is another solution, not based on networking ? -- do it yourself http://antoine.villeret.free.fr 2013/7/3 IOhannes m zmoelnig zmoel...@iem.at -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-03 12:37, Antoine Villeret wrote: that's why I switched to [udpserver]. i guess you meant [tcpserver] instead of [udpserver]. in any case, i'm thinking about removing the multi-client feature of iemnet's [udpserver], so you could only send replies to the *last* client that connected. the idea is, that you could use it as a server that can be queried by clients (in a ping/pong sort of ways: the clients sends a query packet and is immediately answered with an answer packet), similarily to DNS. maintaining a number of connections is simply impossible with a connectionless protocol. fgamsdr IOhannes -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHUMTUACgkQkX2Xpv6ydvQMpQCeOx09OJYP3LszFo39rqGjjYoL l5gAoN5Y6sdSXR06w4hBSw2+WZKGS6HH =WJYl -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
Well [udpreceive] should be able to receive from many different senders, no? (It's a bug if not...) Based on what the [udpreceive] receives, route your replies to one or more [udpsend]s based on info in the incoming packets, or set the port of a single [udpsend] before sending. UDP is connectionless, it doesn't know or care if the other end exists. Martin On 2013-07-03 11:33, Antoine Villeret wrote: thanks for that, in fact, some of my clients need the same infos and this is why I first try to do multicasting but reaching multiple clients on the same host is different than addressing one packet to different hosts... for now all my clients try to connect around each 10 second to server to query some infos and to update their state but I could guarantee that only one client will try to connect at a time so it could be difficult to use a server which doesn't accept more than one connection... or maybe i'm wrong and there is another solution, not based on networking ? -- do it yourself http://antoine.villeret.free.fr 2013/7/3 IOhannes m zmoelnig zmoel...@iem.at mailto:zmoel...@iem.at -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-03 12:37, Antoine Villeret wrote: that's why I switched to [udpserver]. i guess you meant [tcpserver] instead of [udpserver]. in any case, i'm thinking about removing the multi-client feature of iemnet's [udpserver], so you could only send replies to the *last* client that connected. the idea is, that you could use it as a server that can be queried by clients (in a ping/pong sort of ways: the clients sends a query packet and is immediately answered with an answer packet), similarily to DNS. maintaining a number of connections is simply impossible with a connectionless protocol. fgamsdr IOhannes -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHUMTUACgkQkX2Xpv6ydvQMpQCeOx09OJYP3LszFo39rqGjjYoL l5gAoN5Y6sdSXR06w4hBSw2+WZKGS6HH =WJYl -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at mailto:Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-03 17:33, Antoine Villeret wrote: so it could be difficult to use a server which doesn't accept more than one connection... no that's not what i meant. you can have as many connections as you want, but they cannot be maintained at the same time. simple example: - - both clientA and clientB send a a query to the server - - to complicate things, they do so at exactly the same time - - but since IP is a serial protocol, they will somehow arrive one after each other - let's assume clientB was faster - - [udpserver] will output the query from clientB - - if the server-patch now immediately responds to that query, the response will be sent back to clientB - - then [udpserver] will output the query from clientA. - - [udpserver] will forget everything about clientB - - if the server-patch responds immediately to that query, the response will be sent back to clientB - - if you later send something from the server, it will still be sent to clientB (because clientB is the last known connection) so you can have as many connections as you want, but they have to be handled atomically - on after another. you cannot have clientA and clientB connect to the server, and make the server send info to both simultaneously. instead you have to adapt a query/response scheme, where each client asks the server for piece of information. fgamsdr IOhannes -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHUSAkACgkQkX2Xpv6ydvQ7dQCfWd6Ms++xbvpFYHMUArPILPeA fA8AoOo0uQuuQnJV5FBafbRsggXOqtzx =29SY -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-03 17:44, Martin Peach wrote: Well [udpreceive] should be able to receive from many different senders, no? (It's a bug if not...) Based on what the [udpreceive] receives, route your replies to one or more [udpsend]s based on info in the incoming packets, or set the port of a single [udpsend] before sending. th problem with [udpsend]/[udpreceive] is, that you need a [udpreceive] for each client that wants to receive data, which means possibly a lot of different ports, i all [udpreceive]s are sitting on the same host. [udpclient]/[udpserver] allows to have all this with a single known listening port. fgmasdr IOhannes -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHUSLgACgkQkX2Xpv6ydvQNegCgsajUa/8ymMet0oicFIRKxYRZ 8aQAmwcPmpU/pZACSkpwjw88QHYCTMBw =qNOr -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
hi again, just saw this thread right after posting mine : http://lists.puredata.info/pipermail/pd-list/2013-07/103236.html sorry for bothering here is attached three small patches that make PD crash raw_client and raw_server work well together but when I try to connect more client (eg. 10 with 10_raw_client) crash happens... I realize that with iemnet's version of tcpclient/tcpserver, if two client connect at the same time to server, only on receive data not the other, so I put a timeout to disconnect the client if no answer was received in a certain time and then reconnect i first make this with iemnet's tcpserver and I got a SIGPIPE on the server side (see my previous post) while I got SIGSEGV on the client side, here is the gdb backtrace : [New Thread 0x7fff7bfff700 (LWP 4478)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffc8ff9700 (LWP 4477)] 0x00472963 in clock_set () (gdb) watchdog: signaling pd... I think in the server side a signal(SIGPIPE, SIG_IGN); could help but I don't know where to put it (in tcpserver.c ? in iemlnet_sender.c or somewhere else ?) also I tested it with the mrpeach's version, it doesn't crash but the GUI hangs gdb doesn't tell anything, it continue to show thread creation and exiting also I'm using iemnet's first because it has a [port( method to change the binding port on the fly and I made a rebinding routing to choose an available port in a certain range both in server and in client side to prenvent connection error if port is still used after a crash for example I don't know how to go further with this, But I really need a reliable server for some project and for now I just have an headache :-) please tell me how i can help fixing this (and please note that I don't know anything on tcp communication...) cheers antoine -- do it yourself http://antoine.villeret.free.fr 2013/7/2 Antoine Villeret antoine.ville...@gmail.com hi all, I got some crashes with iemnet's tcpserver gdb tells : [New Thread 0x7fffb9ffb700 (LWP 7828)] Program received signal SIGPIPE, Broken pipe. [Switching to Thread 0x7fffea57a700 (LWP 7713)] 0x773b52cc in __libc_send (fd=optimized out, buf=optimized out, n=optimized out, flags=optimized out) and it happends when several (10) clients are connected at the same time and send some data I guess tcpserver is trying to write to a broken pipe and receive a SIGPIPE signal which is not handle and then exit - so crashes pd am I right ? if so, is it possible to set the SIG_PIPE handler to SIG_IGN to avoid crash ? I saw that here [1]. should I file a bug report ? if so where ? best regards antoine [1] : http://stackoverflow.com/questions/108183/how-to-prevent-sigpipes-or-handle-them-properly -- do it yourself http://antoine.villeret.free.fr 10_raw_client.pd Description: Binary data raw_client.pd Description: Binary data raw_server.pd Description: Binary data ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-02 13:39, Antoine Villeret wrote: I realize that with iemnet's version of tcpclient/tcpserver, if two client connect at the same time to server, only on receive data not the other, that's a different bug, please report it. (please also report *this* bug in the sourceforge bugtracker for pure-data) so I put a timeout to disconnect the client if no answer was received in a certain time and then reconnect i first make this with iemnet's tcpserver and I got a SIGPIPE on the server side (see my previous post) while I got SIGSEGV on the client side, here is the gdb backtrace : [New Thread 0x7fff7bfff700 (LWP 4478)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffc8ff9700 (LWP 4477)] 0x00472963 in clock_set () (gdb) watchdog: signaling pd... tip: when running Pd in a debugger, always use -nrt. general remark: to get a backtrace, please run bt in the debugger (after the crash). I think in the server side a signal(SIGPIPE, SIG_IGN); could help but I don't know where to put it (in tcpserver.c ? in iemlnet_sender.c or somewhere else ?) i don't think this is a good solution. i would prefer something along the lines of setsockopt(SO_NOSIGPIPE) and/or send(..., MSG_NOSIGNAL) - a solution that does not have side-effects on the entire Pd. also I tested it with the mrpeach's version, it doesn't crash but the GUI hangs gdb doesn't tell anything, it continue to show thread creation and exiting also I'm using iemnet's first because it has a [port( method to change the binding port on the fly and I made a rebinding routing to choose an available port in a certain range both in server and in client side to prenvent connection error if port is still used after a crash for example I don't know how to go further with this, But I really need a reliable server for some project and for now I just have an headache :-) please tell me how i can help fixing this (and please note that I don't know anything on tcp communication...) btw, my experimental repository for iemnet is at [1]. i added the MSG_NOSIGNAL flag (currently this is linux only), and the server does not crash anymore, but the clients still do. fgmasdr IOhannes [1] https://github.com/umlaeute/pd-iemnet -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHSxSoACgkQkX2Xpv6ydvRiAwCgrn20fLBsSDaDxDODerVSEGiw AG0An0u5PY21NryZawi/JdH3U02NOYAe =mQ4d -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
On Die, 2013-07-02 at 13:39 +0200, Antoine Villeret wrote: hi again, just saw this thread right after posting mine : http://lists.puredata.info/pipermail/pd-list/2013-07/103236.html sorry for bothering here is attached three small patches that make PD crash raw_client and raw_server work well together but when I try to connect more client (eg. 10 with 10_raw_client) crash happens... What is all the code around the networking objects about, specially in raw_client.pd? Do you mind explaining what it does as it is not really fun to look at. Or probably more interesting: What is it trying to achieve? Even when only loading one instance of raw_client.pd, I only get: iemnet:sender-shutdown: Transport endpoint is not connected Roman ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
2013/7/2 IOhannes m zmoelnig zmoel...@iem.at -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-07-02 13:39, Antoine Villeret wrote: I realize that with iemnet's version of tcpclient/tcpserver, if two client connect at the same time to server, only on receive data not the other, that's a different bug, please report it. strangely I cannot reproduce it now, but when it happens i'll report it (please also report *this* bug in the sourceforge bugtracker for pure-data) ok, i'll do sorry, I was thinking pure-data bug tracker was only for pure-data bugs not for externals ones... so I put a timeout to disconnect the client if no answer was received in a certain time and then reconnect i first make this with iemnet's tcpserver and I got a SIGPIPE on the server side (see my previous post) while I got SIGSEGV on the client side, here is the gdb backtrace : [New Thread 0x7fff7bfff700 (LWP 4478)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffc8ff9700 (LWP 4477)] 0x00472963 in clock_set () (gdb) watchdog: signaling pd... tip: when running Pd in a debugger, always use -nrt. general remark: to get a backtrace, please run bt in the debugger (after the crash). ok thanks for that, gdb is still a bit confusing for me I think in the server side a signal(SIGPIPE, SIG_IGN); could help but I don't know where to put it (in tcpserver.c ? in iemlnet_sender.c or somewhere else ?) i don't think this is a good solution. i would prefer something along the lines of setsockopt(SO_NOSIGPIPE) and/or send(..., MSG_NOSIGNAL) - a solution that does not have side-effects on the entire Pd. ok also I tested it with the mrpeach's version, it doesn't crash but the GUI hangs gdb doesn't tell anything, it continue to show thread creation and exiting also I'm using iemnet's first because it has a [port( method to change the binding port on the fly and I made a rebinding routing to choose an available port in a certain range both in server and in client side to prenvent connection error if port is still used after a crash for example I don't know how to go further with this, But I really need a reliable server for some project and for now I just have an headache :-) please tell me how i can help fixing this (and please note that I don't know anything on tcp communication...) btw, my experimental repository for iemnet is at [1]. i added the MSG_NOSIGNAL flag (currently this is linux only), and the server does not crash anymore, but the clients still do. ok thanks I'll check it best regards a fgmasdr IOhannes [1] https://github.com/umlaeute/pd-iemnet -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlHSxSoACgkQkX2Xpv6ydvRiAwCgrn20fLBsSDaDxDODerVSEGiw AG0An0u5PY21NryZawi/JdH3U02NOYAe =mQ4d -END PGP SIGNATURE- ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
On 2013-07-02 14:38, Antoine Villeret wrote: all of this is quite complex for a not so difficult task : I want to make several instance of Pd talking to each other on one machine I started with udpsend/udpreceive but I found no way to do broadcast on localhost and I do need to have lots of clients You don't need to broadcast on a single machine, just use different port numbers. You can send to several ports with several [udpsend]s. Martin ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
On Die, 2013-07-02 at 20:38 +0200, Antoine Villeret wrote: Hi roman, the code around networking object are some workarounds for differents bugs, the first was a crash when client number reach 32, but it seems to be fixed, I can't reproduce this anymore the second was something strange if several clients connect to the same server at the same time (when patch launch for example), so I add a timeout, if no answer in this timeout, the client disconnect there is also a mechanism to choose an available port at startup in server and the client try different port until it get a connection... Ok, I think I understand a bit more now. The server tries a different port when the first port is not available right? It scans until it finds one. I don't see what the client is supposed to do. There is some code to scan through ports, but actually when I tested your client, it always only tried 1. I don't think that dynamic ports are a good solution. If you find a proper way to disconnect the clients, then you shouldn't experience the problem of a busy port when restarting the server. This only happens when you close the server patch without disconnecting the clients. Anyway, the port scanning of the clients would be so slow that probably by the time they found the correct port, the initial port would be free again (depending on OS, though). Then I think your way of testing might expose bugs that you wouldn't experience in real life. I mean that is good for bug reporting, but it isn't really meaningful to connect many clients in the same instance of Pd, even less so to connect them all in zero logical time. If you would run your clients in different instances of Pd (what I think is your plan anyway), you wouldn't have to find workarounds for issues exposed by running clients in the same instance. I figured you can only properly disconnect all clients with [net/tcpserver]. Unlike [iemnet/tcpserver], it has no 'disconnect' method. So you have to disconnect each client separately. [iemnet/tcpserver] is supposed to disconnect all clients with 'disconnect', but actually the [iemnet/tcpclient]s seem to ignore it and keep their connected state (outlet 3 doesn't switch to 0). Finishing sessions seems generally to be still a problem with iemnet. Also when a client disconnects, the server often doesn't change the number of connected clients. The net library seems much more robust in this regard. I simplified your patches a lot and as simple as they are, they seem to work fine here. Note about the server: you need to disconnect all clients before you close the server patch. This will the leave the network socket in a clean state and you will be able to occupy that port immediately again. The client tries to connect at a rate of one try per second. In case it gets disconnected by the server, it waits 5 seconds before trying to connect again. This gives you time to disconnect all clients and shutdown the server properly. all of this is quite complex for a not so difficult task : I want to make several instance of Pd talking to each other on one machine I started with udpsend/udpreceive but I found no way to do broadcast on localhost You mean broadcast on all ports? I don't think that something like that exists. and I do need to have lots of clients multicasting is not a solution because it need to have a working networking interface If you want to run many instances of Pd on localhost, then multicast can't help you. It is about sending data to many clients on different IPs at the same time. You want to send to many clients on localhost. then I tried udpserver which doesn't work (at least the version in the pd's SVN) then I switch to tcpserver and I got a lots of troubles... Things look good as long as you think you only need streams. As soon as you figure out that you need to send packets between your instances of Pd, things get really complex wit [tcpserver]. I will try out iohannes version of iemnet and I'll also investigate those bugs As of today, both libraries net and iemnet have known bugs. My examples use net because [iemnet/tcpserver] can't disconnect clients. but other (simpler) solutions are welcome :-) See attachment. I tested them with Pd-extended 0.43.3 on Ubuntu 12.04 Roman #N canvas 520 44 387 366 10; #X msg 104 205 connect localhost 1; #X obj 108 284 s \$0.connection; #X obj 104 24 r \$0.connection; #X obj 104 51 == 0; #X obj 104 176 metro 1000; #X obj 58 25 inlet; #X obj 58 329 outlet; #X obj 257 34 loadbang; #X msg 257 56 0; #X obj 257 79 s \$0.connection; #X obj 104 139 delay 5000; #X obj 104 75 sel 1; #X obj 131 96 t b a; #X msg 131 118 stop; #X obj 58 233 net/tcpclient; #X connect 0 0 14 0; #X connect 2 0 3 0; #X connect 3 0 11 0; #X connect 4 0 0 0; #X connect 5 0 14 0; #X connect 7 0 8 0; #X connect 8 0 9 0; #X connect 10 0 4 0; #X connect 11 0 10 0; #X connect 11 1 12 0; #X connect 12 0 13 0; #X connect 12 1 4 0; #X connect 13 0 10 0; #X connect
Re: [PD-dev] SIGPIPE on iemnet's tcpserver
On Mit, 2013-07-03 at 00:16 +0200, Roman Haefeli wrote: then I tried udpserver which doesn't work (at least the version in the pd's SVN) then I switch to tcpserver and I got a lots of troubles... Things look good as long as you think you only need streams. As soon as you figure out that you need to send packets between your instances of Pd, things get really complex wit [tcpserver]. Have you also tried [maxlib/netserver]? It is the oldest of all and does only send FUDI. For connecting many Pds together this might be the easiest to use. It's actually quite stable since it isn't threaded, but it blocks Pd if its buffer is full. This might not be an issue under normal circumstances as the buffer should get emptied quickly for connections on localhost. Problems arise when a client disappear without disconnecting properly. [maxlib/netserver] wants to still send data to such a vanished client and so the buffer is filled until it blocks Pd and also the traffic for all other clients. If that happens, there is nothing much you can do about but closing Pd. This means your clients must not crash and must not be closed without closing the connection. Networking doesn't come for free in Pd. Roman ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
