This ties into the heart of some related discussions we've been having at work recently. We have a bunch of new hires who are trying to come to grips with a messaging protocol that's evolved over the past year or so as we've gotten more familiar with 0mq and adapted it to our changing needs.
It's really just meandering commentary about the way I perceive the proposal as it relates to the way we're currently using the library (in C++, C#, python, and clojure via jzmq). It may be a complete waste of time, or it may be a validation of the proposed changes. Our current incarnation looks something like: Frame 1: address header (originally for pub/sub) Frame 2: bundle of serialized data (with a bunch of HTTP-style metadata, like Verb, URI, headers, the "actual" message body etc). Normal router/dealer identity frame sequences. New team members would very much like to split that apart into 7 or 8 more frames, just so the "actual" message body is totally isolated. We could probably accomplish something similar using zproto, though I'd much rather just cram everything into one single frame. We can't do either, mainly due to time constraints around changing the C++ layer. Pieter Hintjens <[email protected]> wrote: > I think we can either get the routing id as a message property, and/or > add a "reply to message" semantic that does this implicitly. The > second is easier for simple servers, the first is better for realistic > servers. > > Since it only applies to server sockets, I think we would benefit from > a zserver_reply () method (or similar). For our app, we have "clients" that have to establish the initial connection (they use dealer sockets. The central messaging hub has the router). After that, those "clients" really switch to being "servers." We're attaching the identity frame(s) to each incoming message so we can use them to send the response. We're also tracking them on the "server" side when a "client" first connects so the "server" (which will usually act more like a client) can send off its requests. We decided on Friday to just start to thinking of everything as a "peer" and just think in terms of "message events" instead of thinking in terms of "client/server" and "request/response". We haven't run into scenarios where peers drop connections, but we're also barely at the PoC stage. Arnaud Kapp <[email protected]> wrote: > > I think thread-safe socket would be cool. As in great to have in some > particular situation, but most of the case it shouldn't be needed. This would be a huge win for us. We have a ridiculously complicated pair of event loops in our central message hub(s) to make the multi-threaded/asynchronous pieces work correctly. I wrote it, and I'm absolutely positive that I'll regret it in a couple of months. That's on the clojure side, where multi-threading is about as simple as it can get. We can get away with single-threading in the C++ parts. The .NET parts are going to be really ugly. The python pieces...well, I keep being forced to say "I told you so" about them. > IMO framing is very good because it allows you to send (and receive) > various kind of data *easily*. >From our perspective: We really want to keep a single messaging protocol at every layer. The details of that protocol keep evolving as our understanding grows, but we've managed to maintain backwards compatibility despite my best efforts to break it. And that's been a very good thing. We started with this approach because we needed to have a string that subscribers could use. If that had just been a standard part of our main message body (which, at the time, was JSON), there wouldn't have been any good/standard way to put it at the front of the message. Yes, there are obviously ways to make that work. They aren't approaches that we've ever had time to put into practice. It's an ugly truth about startups: we never have time to do "internal" things correctly because we're always too busy working on the next set of must-have features. > Date: Tue, 3 Feb 2015 12:00:52 -0800 MinRK <[email protected]> wrote: > As far as I can tell, I also use the multihop case that Pieter's blog post > suggests "does not seem to happen in practice" every day, so maybe my use > of zeromq is not typical. We aren't using it yet, but it's a major part of our road-map. That doesn't mean we'll ever actually implement this, but it seems pretty likely that we will. I can definitely see the point that the blog post makes about this. We'd be better off doing our own routing/messaging-frame/custom message pieces in the middle. At the same time...when it becomes time to make this happen, I can almost guarantee that it will be a desperate last-minute feature that I have to implement over the course of one weekend. I'll be thrilled to use the library that I've already written to use the current architecture. (Thanks for never breaking backwards compatibility!!) Date: Tue, 3 Feb 2015 23:50:54 +0100 Arnaud Kapp <[email protected]> wrote: > However, a good read through the guide really > help understanding how things work. It does. My experience has been that convincing people to read that guide is painful. It's a travesty of injustice. The Guide is one of the best, most readable tech manuals that I've ever read. But half the people I work with absolutely refuse to read it. Instead they treat the whole thing like voodoo and spend their days recompiling, hoping the stack traces will go away. Or maybe that wireshark will magically show them that their sockets have started working when they're doing it wrong and can't find the time to RTFM. I don't believe my current job is atypical. Except that possibly I work with people who are far smarter than average. We live (and work) in a society that has forgotten how to read. I'd normally rather dumb things down at the app level than at the low-level socket level (and I consider 0mq the low-level interface these days). But I'm also tired of telling people that they just need to suck it up and cope with the NULL message frame to/from the Dealer socket because I'm not going to make my Router socket cope with their "just this once" special magical case that should ignore that particular convention. > I don't have much experience teaching, and none teaching ZMQ. I wouldn't say that I have "much" experience in either area. But this sort of thing is definitely a pain point. It's been more painful than convincing the C++ guy that he shouldn't NULL-terminate his strings. We've spent enough time coping with many of these issues that I got twitchy when I first read the announcement. I probably laughed a little hysterically when I mentioned it to my colleagues. This is going to totally invalidate an incredible volume of work. After I got over the twitches, I realized that this is a Very Good Thing (TM). Deleted source code is thoroughly debugged. Now I just have to convince our C++ guy... Regards, James _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
