Re: Tomcat 9, websocket server, threading
On 17/06/2023 16:54, Nikolai Zhubr wrote: Hi, On 6/14/23 19:43, Mark Thomas wrote: [...] There is no multi-threading within a single WebSocket connection. It is explicitly not allowed by the Jakarta WebSocket specification and Tomcat follows that rule. Could you please point out where such guarantee is implemented exactly? I'll turn that around. Can you point to a code path where that specification required guarantee isn't honored? Hint: work your way up the call stack and the answer will be obvious. From what I can see in 9.0.36 source code the call chain down to onMessage is apparently as follows: WsFrameServer.notifyDataAvailable() --> WsFrameServer.doOnDataAvailable() --> WsFrameServer.onDataAvailable() --> WsFrameBase.processInputBuffer() --> WsFrameBase.processData() --> WsFrameBase.processDataBinary() --> WsFrameBase.sendMessageBinary(...) --> binaryMsgHandler.onMessage(...) In notifyDataAvailable one can see some changeReadState(ReadState.WAITING, ReadState.PROCESSING) before calling onDataAvailable() and changeReadState(ReadState.PROCESSING, ReadState.WAITING) after. Is it the thing that is supposed to protect against concurrent (and/or out-of-order) calling of onMessage? Tomcat is open source. That means you can look at the commit history and (hopefully) find out why code was added. GitHub even provide a nice web GUI allowing you to step back through multiple changes one at a time. If you want to find out what that code does, start here: https://github.com/apache/tomcat/blame/main/java/org/apache/tomcat/websocket/server/WsFrameServer.java#L159 Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9, websocket server, threading
Hi, On 6/14/23 19:43, Mark Thomas wrote: [...] There is no multi-threading within a single WebSocket connection. It is explicitly not allowed by the Jakarta WebSocket specification and Tomcat follows that rule. Could you please point out where such guarantee is implemented exactly? From what I can see in 9.0.36 source code the call chain down to onMessage is apparently as follows: WsFrameServer.notifyDataAvailable() --> WsFrameServer.doOnDataAvailable() --> WsFrameServer.onDataAvailable() --> WsFrameBase.processInputBuffer() --> WsFrameBase.processData() --> WsFrameBase.processDataBinary() --> WsFrameBase.sendMessageBinary(...) --> binaryMsgHandler.onMessage(...) In notifyDataAvailable one can see some changeReadState(ReadState.WAITING, ReadState.PROCESSING) before calling onDataAvailable() and changeReadState(ReadState.PROCESSING, ReadState.WAITING) after. Is it the thing that is supposed to protect against concurrent (and/or out-of-order) calling of onMessage? Because other than that, I could not find anything relevant. Thank you, Regards, Nikolai - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9, websocket server, threading
Hi Mark, On 6/14/23 19:43, Mark Thomas wrote: There are multiple things that do not make sense in the above paragraph. It well might be. Tomcat is quite large and complicated, I have not dig it through really thoroughly yet, that's why I asked here. Servlets play no role in processing WebSocket data. Hmm, they consume it! Incoming WebSocket frames are processed synchronously on a single thread for as long as there is data to read. Processing may switch to a different thread between messages but that won't change the order in which messages are received on onMessage() is called. The Jakarta WebSocket specification explicitly states that "In all cases, the implementation must not invoke an endpoint instance with more than one thread per peer at a time" Does this strictly mean that any onMessage call may only happen after previous onMessage (of that same connection) returned? Not sure if it'd make much sense to redo my testing of now obsolete Tomcat 7, but it definitely showed something different than that (in default configuration). This all was observed ages ago and for Tomcat 7 a simple fix was found: replace "Protocol=HTTP/1.1" with "Protocol=org.apache.coyote.http11.http11protocol" Java class names and XML attribute names are case sensitive. And the quotes are in the wrong place. The above would never have worked. I assume you meant: protocol="org.apache.coyote.http11.Http11Protocol" Sure. That was quick hand-typing to show the idea, not exact copy-paste. Same as below. I wouldn't expect anybody to insert something from emails into their server.xml blindly anyway :) which is the deprecated (and removed in 8.5.x) BIO connector. Yes, exactly. There is no multi-threading within a single WebSocket connection. It is explicitly not allowed by the Jakarta WebSocket specification and Tomcat follows that rule. Ok. Maybe I'll need to still dig through the code to understand what is going on better. In fact, I have not observed any actual reordering with Tomcat 9 yet, just changing threads, so it might be that some relevant bug was fixed in these last 12 years or so :) Thank you, Regards, Nikolai How do I configure that? N/A. You need to fix whatever it is in your application that is causing messages to be processed out of order. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9, websocket server, threading
On 14/06/2023 15:21, Nikolai Zhubr wrote: Hi all, I'm trying to migrate my servlet previously running on Tomcat 7 for ages, to Tomcat 9.0.36 as per openSuse 15.4, and facing some problem. The servlet in question is using websocket, basically as a security-enhanced http-friendly replacement of plain-old TCP socket. That is, as a bi-directional binary stream with guaranteed data integrity and ordering. By default Tomcat usually tries to spread the load over multiple threads as much as reasonably possible. With normal http(s) requests, this all works quite well out-of-the-box in most cases with no substantial drawbacks. Now, websocket is somewhat different. Trying to automatically split incoming websocket data over multiple threads is both useless and harmfull, at least in some specific simple use-cases, because (1) the servlet is not prepared to parallelize incoming commands from the same client connection (it would make no sense due to application logic) so it only adds unnecessary locking burden, and (2) more importantly, it sometimes breaks the ordering of data in incoming websocket stream, because OS thread scheduling is non-deterministic, so some portions might arrive to the servlet in wrong order, from time to time. There are multiple things that do not make sense in the above paragraph. Servlets play no role in processing WebSocket data. Incoming WebSocket frames are processed synchronously on a single thread for as long as there is data to read. Processing may switch to a different thread between messages but that won't change the order in which messages are received on onMessage() is called. The Jakarta WebSocket specification explicitly states that "In all cases, the implementation must not invoke an endpoint instance with more than one thread per peer at a time" This all was observed ages ago and for Tomcat 7 a simple fix was found: replace "Protocol=HTTP/1.1" with "Protocol=org.apache.coyote.http11.http11protocol" Java class names and XML attribute names are case sensitive. And the quotes are in the wrong place. The above would never have worked. I assume you meant: protocol="org.apache.coyote.http11.Http11Protocol" which is the deprecated (and removed in 8.5.x) BIO connector. in respective item. This made Tomcat 7 not spread one connection between multiple threads and all worked just fine then. Now in Tomcat 9, the "http11protocol" option no longer exists. Instead, there is say "http11NioProtocol". But, it does not disable multithreaded websocket. Again, case is important. The above values are not correct. So, my question is, is it possible to somehow prevent multithreading in handling one connection in Tomcat 9? There is no multi-threading within a single WebSocket connection. It is explicitly not allowed by the Jakarta WebSocket specification and Tomcat follows that rule. How do I configure that? N/A. You need to fix whatever it is in your application that is causing messages to be processed out of order. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
