Re: [zeromq-dev] Visibility into pipes of a socket
Thanks for the response. Comments inline On 9/21/12 12:05 AM, Pieter Hintjens wrote: On Thu, Sep 20, 2012 at 9:40 PM, Stuart Brandt s...@compuserve.com wrote: All thoughts welcome! Very roughly, trying to do this kind of thing is why it takes people months to write even basic protocols over TCP. No layering. I'm a little puzzled on this statement. The API for 3.x has peer address transiting both into and out of the 0MQ library layer. Exposing the transport origin of a received message using the same philosophy as zmq_ctx_set_monitor (an after-the-fact thing intended for monitoring and operational concerns) doesn't strike me as a layering violation given 0MQ's existing APIs. Logging IP addresses in the server? Absolutely! Operational auditing, monitoring, and troubleshooting is a real PITA without it. Peer address info, or at least peer IP, is pretty much standard monitoring info in most distributed apps I've seen. Measuring latency of servers? Yes. Again this is one of those operational kinds of things. Measuring server latency is a valuable tool for identifying issues that need to be raised to operations staff. Most of what you want to achieve can be done easily above 0MQ by ignoring the physical network and talking application to application. Heartbeats from servers to clients. Application-level identifiers. Agreed...but only if you assume a well behaved app-level that never runs into version mismatch problems, or questionable/rogue peers, or word boundary alignment problems, or any of a variety of application problems that could pollute an app-level identifier. There are a lot of great lessons in The Guide, and in this case the part on reliability that cites application code as the worst offender when it comes to reliability seems to apply. I'm not looking to guard against a hacker working at the TCP or even ZMTP layer here. Protection against that is well beyond the scope of 0MQ. I am, however, looking to make sure that when some new client code makes it into production with a buffer overwrite bug that my server can log/alarm with enough specifics to be useful. Given the choice between 2012-09-24 14:25:51,644 ERROR - Unrecognized request from peer 192.0.0.1:12234 and 2012-09-24 14:25:51,644 ERROR - Unrecognized request from peer #^2ffwi23r098vnasdf0 which would you prefer to use for quickly finding the client instance that went bad? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Visibility into pipes of a socket
After a fair amount of time looking at ZeroMQ, I'm still at a loss to understand how to cover certain requirements without much visibility into the connections/pipes that comprise a socket. Given that the introduction of zmq_ctx_set_monitor seems to be significant positive step in that direction, I thought I'd throw out a couple scenarios to see if anyone has any good solutions on how to handle them. 1) Let's say I have an X/REQ socket through which requests are sent to a set of 'n' servers. My code detects that the standard deviation for response latency has been trending up, and in fact it seems like 1/nth of the responses have 4x the normal latency. In an ideal world, my code would be able to do some analysis of the issue to include correlation of latency to server/path and throw an alarm to an OPS type with information about network path and/or server that is out significantly out of norm. 2) Flipping scenario 1, let's say I have a server that notices 1/nth of the requests it's been getting in the past hour are bad' -- either malformed or in violation of some app-level authentication or authorization check. I'd really like my server to log these as an abnormality, incl. originating IP + request data, so that I could troubleshoot further. Taking it a step further, it would also be good if I could correlate these bad requests to a particular client/IP and force-drop the connection from the bind side. And yet another step further, it would be good if I could effectively RBL the IP so that my server could minimize the impact the rogue client might have should it repeatedly try to reconnect. RBLing aside, I think both of these scenarios would be significantly helped by either: 1) a call that retrieves the peer address info of the pipe associated with the most recent 'recv' oriented call completion 2) the addition of a parameter on the 'recv' oriented calls to return the peer address info 3) a new property of a message conveying peer address info that could be retrieved via zmq_msg_get All thoughts welcome! - Stuart ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Visibility into pipes of a socket
On Thu, Sep 20, 2012 at 9:40 PM, Stuart Brandt s...@compuserve.com wrote: All thoughts welcome! Very roughly, trying to do this kind of thing is why it takes people months to write even basic protocols over TCP. No layering. Logging IP addresses in the server? Measuring latency of servers? Most of what you want to achieve can be done easily above 0MQ by ignoring the physical network and talking application to application. Heartbeats from servers to clients. Application-level identifiers. If you'd used 0MQ in any depth you'd know this. So my conclusion is you've not internalized it, and this is the real issue. Write a thousand lines of 0MQ code, throw that away, and you'll see your perspective shifting, and these problems disappearing. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
