Thanks for the detailed answer. I have a few questions and comments inline.
On Thu, May 30, 2013 at 9:01 AM, Joakim Erdfelt <[email protected]> wrote: > That's an awfully wide open topic. ;-) > > A quick breakdown of everything from the start (obviously from a Jetty > 9.0.x perspective)... > > In order to have a websocket, you must first create the universe. > skipping a few obvious steps... > > The incoming connection arrives to Jetty as a standard HTTP (plain or SSL) > connection, which hits the normal flow of internal Jetty handlers and > eventually makes its way to the > WebSocketServlet<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/servlet/WebSocketServlet.html>implementation. > > *The WebSocket Upgrade Handshake:* > At this point a few tests are done against the request headers to > determine if its a HTTP upgrade, is it a WebSocket upgrade, does the origin > match, etc. > If this all checks out then the > WebSocketCreator<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/servlet/WebSocketCreator.html>is > asked for a websocket instance. The default creator will create a new > instance of the WSE (WebSocket Endpoint) each time, but you can provide > your own logic and reuse instances or even go stateless with a single > instance for all WSEs if you so desire. > If the WebSocketServlet gets a WSE at this point it proceeds to respond on > the HTTP that the upgrade is successful, sending HTTP Response 101 > Switching Protocols. > > *The WebSocket Initialization:* > In the setup of the > WebSocketConnection<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/common/io/AbstractWebSocketConnection.html>, > the following is initialized as well.. > * WebSocket > Session<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/Session.html>(this > represents the connected session) > * WebSocket > Parser<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/common/Parser.html>(frame > parser) > * WebSocket > Generator<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/common/Generator.html>(frame > generator) > * > EventDriver<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/common/events/EventDriver.html>(this > is the common abstraction for talking to a WSE instance using any of > the WSE definition techniques) > * WebSocket Extension > Stack<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/common/extensions/ExtensionStack.html>(using > any of the negotiated extensions during the WebSocket upgrade > handshake) > > The plumbing from the physical connection to the WSE is done by chaining > up > IncomingFrames<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/extensions/IncomingFrames.html>and > OutgoingFrames<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/extensions/OutgoingFrames.html>implementations. > It looks sort of like this ... > > [image: Inline image 1] > > Now, internally to Jetty, the WebSocketServlet has notified the HTTP > handling that the HTTP connection handling is now a WebSocketConnection. > The WebSocketConnection is setup with the WebSocket Endpoint > > The Open WebSocket: > At this point the RFC6455 state transitions from CONNECTING to OPEN. > The local WSE is notified that the Session is now open. > The JIOE (Jetty I/O Endpoint) now does its Async/NIO handling of read and > write interests and notifying the WebSocketConnection when the physical > connection is capable of doing a read or a write. > Is this a single thread, multiple, or configurable? Meaning the async selector? > If a read, it fills up an input buffer and parses what it can. > The Parser notifies its IncomingFrames on each frame encountered, which > goes through the extension stack and eventually hits the WSE. > If its a write, the Connection's outgoing queue of frames is used to > create a output buffer via the Generator. > > At this point, the Thread that is handling the JIOE side is the same > thread that Jetty pulled from the ThreadPool for the HTTP handling. > When the WebSocket Connection gets its read/write events it will continue > to use that same thread to parse, send the frame through the extension > stack, and hit the EventDriver, and eventually the WSE. > When the WSE gets the event for say a Text message, then its actually > operating on the same Thread that the JIOE is using. > When a JIOE event to write is detected, the same Thread is used to > generate and write to the network. > > Memory consumption is not constant with WebSocket, due to the nature of > the message based (not streaming) WebSocket protocol. > Buffering? Is this handled per connection or system wide, e.g. a buffer pool? > > Everything you do is based on a WebSocket message. reading and writing. > Per RFC6455, a WebSocket message has no upper limit in size, only the > individual frames do. > A WebSocket message is 0..n frames. a frame can be 0 > to 9,223,372,036,854,775,807 bytes in size (63-bit value btw). > > In Jetty, we obviously can't support that with modern hardware (who has 9 > exabytes of memory on their computer??) > So we provide a means to limit the per-message size via the > WebSocketPolicy.setMaxMessageSize()<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/WebSocketPolicy.html#setMaxMessageSize(long)> > which > will be checked on incoming frames, if in violation it results in a RFC6455 > close code 1009 > MESSAGE_TOO_LARGE<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/StatusCode.html#MESSAGE_TOO_LARGE> > . > Outgoing messages are not checked, as it should be obvious to you if you > can handle it or not. > > *Dealing with Extensions:* > The need to support RFC6455 WebSocket extensions means we have a layer > between the WSE and the physical connection called the ExtensionStack. > This complicates matters as the extensions can change the framing, the > number of frames, etc, making the traditional approach of read/write to the > physical connection from the WSE impossible. > > [image: Inline image 2] > > *Performance with Jetty 9.0.3:* > We've done some simple (really simple) load testing on WebSockets. > On a simple test of 1000 active WebSockets we got an average rate of > 200,000 messages a second per socket. > But we need MUCH more testing here, along with paying attention to memory > and CPU, not just I/O. > The cometd project has already embraced Jetty 9's WebSocket and have been > testing the performance from their point of view, while they have some > concerns with memory (the jetty websocket implementation seems to have > frequent, but small, GC behavior), overall they are happy with what they > see. > Have you looked into direct memory buffers? They seem ideal for this sort of thing. > > *The future of WebSocket on Jetty 9.* > We are close to completion on our implementation of JSR-356 (the > javax.websocket standard API). > It will likely show up in Jetty 9.1 (but this isn't fully decided yet) > What's the alternative? 9.0 or 9.2? > > It brings a few nice things to the Jetty side API as well. > * The WebSocketPolicy.setMaxMessage() is split up between text and binary > now. > * Support for streaming messages has been added > - you can write a message with an OutputStream or Writer now. > - you can receive InputStream or Readers (one for each message) on your > annotated WSE's now > * The RemoteEndpoint (how you write to a remote WSE) has some guards to > notify you if you abuse the message writing in a multi-threaded environment. > - Note: the RemoteEndpoint operates in a similar way, threading wise, > as a OutputStream (in other words, it is not advisable to attempt to write > from multiple threads) > > Once we get a stable build with JSR-356 out into the hands of our users, > we can start to tackle various performance testing and concerns. > > Hope this rambling post helped... > Very much, yes. Thanks. > > > -- > Joakim Erdfelt <[email protected]> > webtide.com <http://www.webtide.com/> > Developer advice, services and support > > from the Jetty & CometD experts > eclipse.org/jetty - cometd.org > > > On Thu, May 30, 2013 at 5:51 AM, Nils Kilden-Pedersen <[email protected]>wrote: > >> Trying once more. Presumably this is documented somewhere I haven't been >> able to find. >> >> >> On Mon, May 27, 2013 at 8:34 AM, Nils Kilden-Pedersen >> <[email protected]>wrote: >> >>> Could someone comment on how Jetty9 handles websockets, from a design >>> perspective? I'm thinking primarily about resource consumption, such as >>> memory usage per connection, number of threads and any (non-) blocking >>> behavior, etc. >>> >>> Thanks, >>> Nils >>> >> >> >> _______________________________________________ >> jetty-users mailing list >> [email protected] >> https://dev.eclipse.org/mailman/listinfo/jetty-users >> >> > > _______________________________________________ > jetty-users mailing list > [email protected] > https://dev.eclipse.org/mailman/listinfo/jetty-users > >
_______________________________________________ jetty-users mailing list [email protected] https://dev.eclipse.org/mailman/listinfo/jetty-users
