Joakim, any chance you could answer my additional questions? On Thu, May 30, 2013 at 10:06 AM, Nils Kilden-Pedersen <[email protected]>wrote:
> Thanks for the detailed answer. I have a few questions and comments inline. > > On Thu, May 30, 2013 at 9:01 AM, Joakim Erdfelt <[email protected]>wrote: > >> That's an awfully wide open topic. ;-) >> >> A quick breakdown of everything from the start (obviously from a Jetty >> 9.0.x perspective)... >> >> In order to have a websocket, you must first create the universe. >> skipping a few obvious steps... >> >> The incoming connection arrives to Jetty as a standard HTTP (plain or >> SSL) connection, which hits the normal flow of internal Jetty handlers and >> eventually makes its way to the >> WebSocketServlet<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/servlet/WebSocketServlet.html>implementation. >> >> *The WebSocket Upgrade Handshake:* >> At this point a few tests are done against the request headers to >> determine if its a HTTP upgrade, is it a WebSocket upgrade, does the origin >> match, etc. >> If this all checks out then the >> WebSocketCreator<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/servlet/WebSocketCreator.html>is >> asked for a websocket instance. The default creator will create a new >> instance of the WSE (WebSocket Endpoint) each time, but you can provide >> your own logic and reuse instances or even go stateless with a single >> instance for all WSEs if you so desire. >> If the WebSocketServlet gets a WSE at this point it proceeds to respond >> on the HTTP that the upgrade is successful, sending HTTP Response 101 >> Switching Protocols. >> >> *The WebSocket Initialization:* >> In the setup of the >> WebSocketConnection<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/common/io/AbstractWebSocketConnection.html>, >> the following is initialized as well.. >> * WebSocket >> Session<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/Session.html>(this >> represents the connected session) >> * WebSocket >> Parser<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/common/Parser.html>(frame >> parser) >> * WebSocket >> Generator<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/common/Generator.html>(frame >> generator) >> * >> EventDriver<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/common/events/EventDriver.html>(this >> is the common abstraction for talking to a WSE instance using any of >> the WSE definition techniques) >> * WebSocket Extension >> Stack<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/common/extensions/ExtensionStack.html>(using >> any of the negotiated extensions during the WebSocket upgrade >> handshake) >> >> The plumbing from the physical connection to the WSE is done by chaining >> up >> IncomingFrames<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/extensions/IncomingFrames.html>and >> OutgoingFrames<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/extensions/OutgoingFrames.html>implementations. >> It looks sort of like this ... >> >> [image: Inline image 1] >> >> Now, internally to Jetty, the WebSocketServlet has notified the HTTP >> handling that the HTTP connection handling is now a WebSocketConnection. >> The WebSocketConnection is setup with the WebSocket Endpoint >> >> The Open WebSocket: >> At this point the RFC6455 state transitions from CONNECTING to OPEN. >> The local WSE is notified that the Session is now open. >> The JIOE (Jetty I/O Endpoint) now does its Async/NIO handling of read and >> write interests and notifying the WebSocketConnection when the physical >> connection is capable of doing a read or a write. >> > > Is this a single thread, multiple, or configurable? Meaning the async selector? > > >> If a read, it fills up an input buffer and parses what it can. >> The Parser notifies its IncomingFrames on each frame encountered, which >> goes through the extension stack and eventually hits the WSE. >> If its a write, the Connection's outgoing queue of frames is used to >> create a output buffer via the Generator. >> >> At this point, the Thread that is handling the JIOE side is the same >> thread that Jetty pulled from the ThreadPool for the HTTP handling. >> When the WebSocket Connection gets its read/write events it will continue >> to use that same thread to parse, send the frame through the extension >> stack, and hit the EventDriver, and eventually the WSE. >> When the WSE gets the event for say a Text message, then its actually >> operating on the same Thread that the JIOE is using. >> When a JIOE event to write is detected, the same Thread is used to >> generate and write to the network. >> >> Memory consumption is not constant with WebSocket, due to the nature of >> the message based (not streaming) WebSocket protocol. >> > > Buffering? Is this handled per connection or system wide, e.g. a buffer pool? > > >> >> Everything you do is based on a WebSocket message. reading and writing. >> Per RFC6455, a WebSocket message has no upper limit in size, only the >> individual frames do. >> A WebSocket message is 0..n frames. a frame can be 0 >> to 9,223,372,036,854,775,807 bytes in size (63-bit value btw). >> >> In Jetty, we obviously can't support that with modern hardware (who has 9 >> exabytes of memory on their computer??) >> So we provide a means to limit the per-message size via the >> WebSocketPolicy.setMaxMessageSize()<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/WebSocketPolicy.html#setMaxMessageSize(long)> >> which >> will be checked on incoming frames, if in violation it results in a RFC6455 >> close code 1009 >> MESSAGE_TOO_LARGE<http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/StatusCode.html#MESSAGE_TOO_LARGE> >> . >> Outgoing messages are not checked, as it should be obvious to you if you >> can handle it or not. >> >> *Dealing with Extensions:* >> The need to support RFC6455 WebSocket extensions means we have a layer >> between the WSE and the physical connection called the ExtensionStack. >> This complicates matters as the extensions can change the framing, the >> number of frames, etc, making the traditional approach of read/write to the >> physical connection from the WSE impossible. >> >> [image: Inline image 2] >> >> *Performance with Jetty 9.0.3:* >> We've done some simple (really simple) load testing on WebSockets. >> On a simple test of 1000 active WebSockets we got an average rate of >> 200,000 messages a second per socket. >> But we need MUCH more testing here, along with paying attention to memory >> and CPU, not just I/O. >> The cometd project has already embraced Jetty 9's WebSocket and have been >> testing the performance from their point of view, while they have some >> concerns with memory (the jetty websocket implementation seems to have >> frequent, but small, GC behavior), overall they are happy with what they >> see. >> > > Have you considered direct memory buffers? They seem ideal for this sort of thing. > > >> >> *The future of WebSocket on Jetty 9.* >> We are close to completion on our implementation of JSR-356 (the >> javax.websocket standard API). >> It will likely show up in Jetty 9.1 (but this isn't fully decided yet) >> > > What's the alternative? 9.0 or 9.2? > > >> >> It brings a few nice things to the Jetty side API as well. >> * The WebSocketPolicy.setMaxMessage() is split up between text and >> binary now. >> * Support for streaming messages has been added >> - you can write a message with an OutputStream or Writer now. >> - you can receive InputStream or Readers (one for each message) on >> your annotated WSE's now >> * The RemoteEndpoint (how you write to a remote WSE) has some guards to >> notify you if you abuse the message writing in a multi-threaded environment. >> - Note: the RemoteEndpoint operates in a similar way, threading wise, >> as a OutputStream (in other words, it is not advisable to attempt to write >> from multiple threads) >> >> Once we get a stable build with JSR-356 out into the hands of our users, >> we can start to tackle various performance testing and concerns. >> >> Hope this rambling post helped... >> > > Very much, yes. Thanks. > > >> >> >> -- >> Joakim Erdfelt <[email protected]> >> webtide.com <http://www.webtide.com/> >> Developer advice, services and support >> >> from the Jetty & CometD experts >> eclipse.org/jetty - cometd.org >> >> >> On Thu, May 30, 2013 at 5:51 AM, Nils Kilden-Pedersen >> <[email protected]>wrote: >> >>> Trying once more. Presumably this is documented somewhere I haven't been >>> able to find. >>> >>> >>> On Mon, May 27, 2013 at 8:34 AM, Nils Kilden-Pedersen >>> <[email protected]>wrote: >>> >>>> Could someone comment on how Jetty9 handles websockets, from a design >>>> perspective? I'm thinking primarily about resource consumption, such as >>>> memory usage per connection, number of threads and any (non-) blocking >>>> behavior, etc. >>>> >>>> Thanks, >>>> Nils >>>> >>> >>> >>> _______________________________________________ >>> jetty-users mailing list >>> [email protected] >>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>> >>> >> >> _______________________________________________ >> jetty-users mailing list >> [email protected] >> https://dev.eclipse.org/mailman/listinfo/jetty-users >> >> >
_______________________________________________ jetty-users mailing list [email protected] https://dev.eclipse.org/mailman/listinfo/jetty-users
