On Fri, 2015-02-06 at 08:19 -0500, Rafael Schloming wrote: > A couple of questions/comments inline, but first off, any reason this > isn't on the list? (I don't mind, just curious.)
No good reason, including list now. Extremely useful discussion, thanks! Taking it all on board. New question: For dispatch I only care about the server side. JavaScript is the only websocket client I know of and as per discussion below we would actually need to *exclude* proton client-side websocket support from the JS client, so maybe we don't need it at all. Are there use cases for a websocket client transport? > On Fri, Feb 6, 2015 at 6:20 AM, Fraser Adams > <[email protected]> wrote: > Hi Again Alan, > Sorry this has turned out to be massively TL;DR I kind of got > on a roll...... > > > On the WebSocket thing it's worth pointing out that I've > mainly been in the game of *using* them and have never had the > pleasure of writing an implementation from scratch, but OTOH > I've done a reasonable amount of"hacking around" - but that's > mostly been in JavaScript and Python implementations so for C > things are more awkward. > > I've copied Rafael so he's in the loop I've also copied > Gordon, 'cause I'd really quite like WebSocket support in > qpidd at some point and Rob because he implemented the > WebSocket transport in the Java Broker and I suspect he knows > way more than me - especially about the trades around > different implementation choices, he also knows the AMQP > WebSocket spec intricacies way better than me so will be > better placed to advise gotchas - and also I guess an approach > that makes is easier for proton-j to introduce WebSocket > support too. > > > I guess first off I should say that when I suggested it might > be a car crash if not thought through I was perhaps being a > little parochial. As you know the JavaScript binding uses > emscripten to compile proton-c to JavaScript - the way that > works is the core grammar is compiled to LLVM bitcode and ends > up as asm.js but library/system calls either wind up calling > other compiled code or they break out into some pure > JavaScript, that's the case for the network code and basically > all the posix network calls get trapped and "faked" in a > useful way; library.js and library_sockfs.js are the main > places to look if you're interested. > > I guess that the key point is that all the *socket* calls in > proton-c wind up as *WebSocket* stuff in the compiled > JavaScript, for browsers it uses native WebSockets and for > Node.js it uses the ws WebSocket library, which is I think the > most popular one. > > The gnarly bit that I can think of is that clearly adding > WebSocket support to proton-c risks accidentally causing the > JavaScript stuff to implement WebSocket over WebSocket, which > would be quite funny but probably not terribly useful :-D > > > It sounds like we quite probably would want to keep any web socket > stuff compile-time optional and simply not build it at all with > emscripten. That would a) avoid the possibility of nested websocketry, > and b) keep useless code out of the already large proton.js file. > > > > Another thing I worry a little about is how one implements the > server side, there are two choices > 1. Have a separate WebSocket transport > 2. Note that WebSockets are actually a frame based protocol > layered on top of TCP and initiated via an HTTP UPGRADE > command. The spec is here: https://tools.ietf.org/html/rfc6455 > > The Java Broker (currently) takes the former approach, it's a > sensible and convenient approach because it uses Jetty as the > Web Server for its UI and Jetty comes complete with a > WebSocket implementation, but there's a down side - you have > to stand up a separate port so basically AMQP over TCP > connections connect to one port and AMQP over WebSocket > connections connect over a different port. > > But as I say WebSockets are actually layered over TCP so it's > possible if you have more control over the server to look at > the data on the TCP buffers and detect that it's actually > WebSocket, so from a user perspective it would be *really > nice* and technically possible to have a single listen port. > > > When I suggested implementing it in proton I meant the latter, simply > autodetecting and upgrading. Andrew has done some nice work around > this sort of thing already with detecting SSL vs plain and detecting > SASL vs just core AMQP, so autodetecting the HTTP header and doing the > upgrade would be a natural and hopefully straightforward way to go. At > that point you just configure on the transport whether you want to > allow it or not. > > > > That's kind of a nice segway into another point > 1. Use a library > 2. Roll your own/copy paste reuse > > I'm torn on that, using a library is clearly good from a > support perspective but the counter to that is illustrated by > the point I make about using Jetty in the Java Broker. Of > course you might *want* to force TCP and WebSocket connections > to different ports but do we care more that it's AMQP or what > the transport is (that's an open question BTW, I personally > like the idea of just connecting to a single port but there > may be good counter arguments). > > > On a slightly library related note it's worth mentioning > something on my TODO list. As you probably know the JavaScript > binding is currently limited to using a WebSocket transport, > that makes sense as a starter for ten 'cause obviously > browsers have a number of constraints but for Node.js there's > no real reason for that to be the case. You might have seen > the TCP->WebSocket proxy stuff I wrote, if you have you'll see > that it's actually technically pretty easy to implement at TCP > version using the Node.js built in net library. The main > reasons I've not done it yet are: > 1. Time - I only get to code at weekends and I'm juggling a > couple of O/S projects at the moment > #ifonlyididntneedtoworktolive :-) > 2. The emscripten network code really isn't as modular as it > should be - I've done quite a lot of it, but I didn't > originate it and TBH I'd quite like to pick it apart and make > it a bit more layered, lack of time again. > 3. The biggest reason though is actually what approach to > take. As above I'd really like to have a Node.js AMQP > application that can just receive either a TCP or a WebSocket > connection and *just work*. That's a bit awkward because I use > ws as the WebSocket server. As it's so well established it's a > bit daft to roll my own but I don't know, it's probably > possible to Monkey Patch or otherwise expose the native TCP > socket if a WebSocket connection fails, though probably the > best approach would be to add a feature to ws and hope that > the pull request gets accepted. > > > Hopefully point three above illustrates one potentially > awkward pinch-point, you'll be adding WebSocket server code in > pure C and I'll be making the backend of the TCP path all > WebSockety too :-) > > It *should* work out OK because the emscripten stuff will > establish a listener and ws will have removed all of the > WebSockety stuff and just exposes the AMQP frames in the octet > array that comes off the recv call, so the proton-c code > *should* just interpret that as AMQP off a TCP socket and not > send it to any of the proton-c WebSocket code, but depending > on how its done it could go horribly wrong, which is why I > figured I'd give a bit of background. > > > All of the above has been from the perspective of a server > receiving a WebSocket connection, in some respects though > establishing a connection is actually more awkward.... > > So it's not like it's technically hard to write the code to > have Node.js establish an AMQP over TCP connection - the net > library is pretty straightforward. The bit that I'm actually > struggling with is how best to let applications choose between > a TCP and WebSocket transport. > > As I say the default position is that the proton-c network > code maps to emscripten library_sockfs.js calls that push the > data over WebSockets - so what API to use to tell it to use > TCP :-) It's easy to set a compile time option but that sucks, > I'd like an application to be able to connect using either TCP > or WebSockets. > > I was wondering if the getprotoent /etc/protocols stuff might > be the best place, that's certainly where you'd get the > protocol number for UDP, TCP or SCTP to the sockets API but > that's used pretty much for the underlying Operating System > networking and WebSockets currently tend to be implemented as > user level libraries over TCP and I'm not aware of an official > protocol entry for WebSockets. > > That's the only place in the socket API that I can think of > that's even close to being able to specify that sort of thing > though :-/ > > > There are certainly other ways to pass that sort of info down > to emscripten, but as I say I'm struggling for a clean > approach. > > > So that conundrum is kind of relevant to the native proton-c > implementation too 'cause if you want to allow the ability to > connect using either TCP or WebSockets there needs to be > something in the API that allows users to choose. Is there > anything in any of the AMQP specs that have actually covered > this? I know that Qpid C++ has support for rdma but I *think* > that's specified in the Connection config which offers a > slightly richer set of connection options than the sort of > amqp connection available in Proton (certainly from what I'm > aware of with Messenger). > > > I think this should be fairly straightforward in proton-c. We already > have a similar set of scenarios there with SSL and SASL autodetect. > When you configure a Transport, you basically construct a stack of > protocols. In server mode you can it autodetect whether each layer is > present and fall through to the next. In client mode the stack is just > what you have specified. > > > So I've covered some of the thought processes going through my > mind as I've been thinking around WebSockets in native > proton-c and most of them aren't really about the technical > implementation of the WebSocket protocol, but I think are > worth bearing in mind. > > > In terms of actually implementing this stuff as Rafael said I > think that options in C are fairly limited. As well as the > library issues I mentioned earlier there's also the little > matter of worrying about license incompatibilities, so my gut > feeling is that the best approach might to get "inspiration" > from other places and to grasp the nettle and implement > something that integrates nicely with Proton. > > If it were me implementing this the first place I'd look would > be here: > https://github.com/kanaka/websockify > > WebSockify is actually a TCP->WebSocket proxy and TBH bits of > it irritate me, but it seems to be used quite a lot. The most > significant things though is that: > 1. It implements a pretty complete WebSocket stack in Python, > which you know so should help you figure things out > 2. It actually also has an implementation in C (under the > "other" directory) so you might want to look at that > 3. I *think* that it accepts TCP or WebSocket connections on > its listen port and just passes through TCP (I can't swear to > that but it rings a bell) > 4. It's well used and well maintained so is likely to be a > reliable starting point > > I'd also take a look at ws: > https://github.com/websockets/ws > > This is clearly JavaScript, but it's one of the most commonly > used and well maintained WebSocket libraries in any language, > so again as a means to figure out how it actually works it > might be a good place to look. > > > BTW Rafael mentioned the annoying thing about there being lots > of different WebSocket versions, that's certainly true, but is > much less of an issue now. That stuff was generally a pain as > the WebSocket protocol was evolving, but it has now been > standardised in the IETF RFC 6455 I linked earlier and all of > the main browsers primarily support that. Looking on the > websockify page there is a comment "Starting with websockify > 0.5.0, only the HyBi / IETF 6455 WebSocket protocol is > supported". Unless there's a compelling customer reason to > support pre-standard versions I don't think it's unreasonable > to only support RFC 6455. It's also worth bearing in mind that > the emscripten based JavaScript binding requires binary > support anyway and I think that's only supported on the final > versions of the spec. so there's probably little actual value > in supporting anything else. > > Sorry for the essay, hope it's useful. > > I'm off to focus on my other project now - I'm currently > trying to write an implementation of webcl using webgl, cause > I need to do some GPU programming and my platform doesn't have > opencl support....... > > Cheers, > Frase > > On 04/02/15 19:30, Alan Conway wrote: > On Wed, 2015-02-04 at 17:55 +0000, Fraser Adams wrote: > Hi Alan, very quick placeholder response. I've > a couple of family > commitments so won't be able to give a decent > response for a day or so - > don't want you to think I'm rude by not > getting back. > > It's a topic that deserves a bit of thought > 'cause it could be a bit of > a car-crash if not thought through. > Thanks for the warning! Andrew Stitcher is making > changes to the > transport layer that will affect this (make it easier) > so I'm not moving > on proton till that is done. When I do I will consult > carefully with > yourself and Andrew to avoid crashing any cars. > > I may try some short term hackery to tide me over but > if I do it will be > in the dispatch codebase and strictly limited to > supporting dispatch's > JS management console. Longer term proton is the right > place to do it > and I will avail of all advice so I do it right ;) > > Frase. > > On 04/02/15 14:46, Alan Conway wrote: > Hi Fraser, > > I need a websocket listener in > dispatch. Rafi points out (rightly) > that > it would be of wider use to do it in > proton-c than in dispatch. Quick > JIRA search reveals the word websocket > next to your name - is that > Java-only? I know nada about > websockets, and not much more about JS > if > you remember our prev. conversation ;) > Actually on that - I now realize > that (duh!) the java build does the > same barf-in-the-source-dir thing so > I was razzing you for nothing. > git-ignore is my friend. > > Hoping to pick your brains for > anything useful on websockets. I'm > guessing my options are > - find a C websockets library (looks > like there are some but none > packaged with popular OS distros (at > least the ones popular with me) > - implement the websocket spec in C. > Not sure how big a job that is yet. > The spec is full of non-normative > waffle so I'm not sure just how much > real action there is. > - grab a python ws library and just > stick with dispatch since it has a > nice C+python implementation. That > wouldn't help proton though. > > Any thoughts? > > Cheers > Alan. > > > >
