[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 Mark Thomas changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #17 from Mark Thomas --- Thanks for testing. Fixed in: - 10.1.x for 10.1.0-M16 onwards - 10.0.x for 10.0.22 onwards - 9.0.x for 9.0.64 onwards - 8.5.x for 8.5.80 onwards -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #16 from Phil Clay --- Good news! The test has been running with no failures continuously since my last comment. I'll keep letting it run today. But I'm pretty confident that the problem is fixed since the test would normally fail after a few of hours. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #15 from Phil Clay --- Got it. Testing... -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #14 from Mark Thomas --- Updated dev build: https://people.apache.org/~markt/dev/v9.0.64-dev-680db44/ Same caveats as before. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #13 from Mark Thomas --- I have a proposed fix. Just running the test suite. Assuming it passes, I'll commit the fix to 9.0.x and do another dev build. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #12 from Mark Thomas --- (In reply to Remy Maucherat from comment #10) > Well, here it would simply need the same change since there's nothing in > that other code path that checks the network output buffer: > https://github.com/apache/tomcat/blob/main/java/org/apache/tomcat/util/net/ > NioEndpoint.java#L1591 I think it is going to be a bit more complicated. For the case where all the application bytes have been written and it is just the netOutBuffer that has data left to write, I can't see a clean way to ensure the write isn't completed at https://github.com/apache/tomcat/blob/main/java/org/apache/tomcat/util/net/NioEndpoint.java#L1628 -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #11 from Remy Maucherat --- NIO2 should not be affected by this and does not need any patching, since SecureNio2Channel simply uses more async operations to fully empty the network output buffer. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #10 from Remy Maucherat --- Well, here it would simply need the same change since there's nothing in that otehr code path that checks the network output buffer: https://github.com/apache/tomcat/blob/main/java/org/apache/tomcat/util/net/NioEndpoint.java#L1591 -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #9 from Phil Clay --- The test using the dev build unfortunately failed after about 3 hours with the same symptom. After it failed, a re-ran it with a breakpoint in NioEndpoint.flushNonBlocking to see the new code in action, but my breakpoint unfortunately never gets hit when sending outbound websocket messages. (It does get hit when sending a normal HTTP response, so I can confirm the new code is active and the breakpoint is working.) Here's the partial stacktrace when SecureNioEndpoint.flush gets called (when it doesn't write everything out). Perhaps that provides some visibility into why NioEndpoint.flushNonBlocking is never called ? Breakpoint reached at org.apache.tomcat.util.net.SecureNioChannel.flush(SecureNioChannel.java:123) at org.apache.tomcat.util.net.SecureNioChannel.write(SecureNioChannel.java:863) at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper$NioOperationState.run(NioEndpoint.java:1674) at org.apache.tomcat.util.net.SocketWrapperBase$OperationState.start(SocketWrapperBase.java:1063) at org.apache.tomcat.util.net.SocketWrapperBase.vectoredOperation(SocketWrapperBase.java:1482) at org.apache.tomcat.util.net.SocketWrapperBase.write(SocketWrapperBase.java:1408) at org.apache.tomcat.util.net.SocketWrapperBase.write(SocketWrapperBase.java:1379) at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.doWrite(WsRemoteEndpointImplServer.java:93) at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.writeMessagePart(WsRemoteEndpointImplBase.java:512) at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.startMessage(WsRemoteEndpointImplBase.java:398) at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendBytesByCompletion(WsRemoteEndpointImplBase.java:155) at org.apache.tomcat.websocket.WsRemoteEndpointAsync.sendBinary(WsRemoteEndpointAsync.java:65) at org.springframework.web.reactive.socket.adapter.StandardWebSocketSession.sendMessage(StandardWebSocketSession.java:92) Should something in that stack be triggering NioEndpoint.flushNonBlocking (either directly or indirectly via flushing the socket)? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #8 from Phil Clay --- Thanks Mark! I'll get the test started today. The test will need to run for a while before I'm confident in the fix. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #7 from Mark Thomas --- A 9.0.x dev build is available here: https://people.apache.org/~markt/dev/v9.0.64-dev-b311c0a/ Usual caveats apply. This isn't an official release. Use at your own risk. It is the standard release layout so the JARs you want will be in bin/embed/apache-tomcat-9.0.64-dev-b311c0a-embed.tar.gz -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #6 from Phil Clay --- Perfect. Also note, my project uses embedded tomcat. Specifically these three artifacts: org.apache.tomcat.embed:tomcat-embed-core org.apache.tomcat.embed:tomcat-embed-el org.apache.tomcat.embed:tomcat-embed-websocket I assume the build will produce everything, but just wanted to be clear what I'm using. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #5 from Mark Thomas --- Great. I'll do a dev build for you. It isn't that tricky but I am all set up for it. I think I know what the problem is. I should have something for you to test on Monday. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #4 from Phil Clay --- > Once we have a potential fix, are you in a position to either builds Tomcat > 9.0.x yourself from source and/or test a 9.0.x dev build I provide for you? Yes, Definitely! I'd prefer a dev build to be provided if possible, but I'm sure I could spend some time to build it myself if needed. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #3 from Mark Thomas --- Thanks for the analysis. What you describe makes sense to me. It looks like the non-blocking variant of bug 65448. My current thinking is that SocketWrapper.flushNonBlocking() needs an NIO specific implementation that can take account of the status of netOutBuffer in NioSecureChannel. Once we have a potential fix, are you in a position to either builds Tomcat 9.0.x yourself from source and/or test a 9.0.x dev build I provide for you? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #2 from Phil Clay --- After a bit of investigation, my rudimentary understanding is that 1) a buffer of unencrypted data makes it down to SecureNioChannel.write https://github.com/apache/tomcat/blob/93108de1127fb228e343a3f3304554bfe7177583/java/org/apache/tomcat/util/net/SecureNioChannel.java#L786 2) SecureNioChannel.write encrypts it into a new buffer (netOutBuffer) https://github.com/apache/tomcat/blob/93108de1127fb228e343a3f3304554bfe7177583/java/org/apache/tomcat/util/net/SecureNioChannel.java#L810 3) And then flushes that encrypted buffer https://github.com/apache/tomcat/blob/93108de1127fb228e343a3f3304554bfe7177583/java/org/apache/tomcat/util/net/SecureNioChannel.java#L824 4) But flush is not guaranteed to write all the encrypted bytes to the channel, and returns false if the buffer was not completely emptied... https://github.com/apache/tomcat/blob/93108de1127fb228e343a3f3304554bfe7177583/java/org/apache/tomcat/util/net/SecureNioChannel.java#L120-L127 But the return value is ignored, and there's no subsequent re-flush until the next write. Therefore, the unwritten encrypted data remains in the netOutBuffer. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 Phil Clay changed: What|Removed |Added Version|9.0.60 |9.0.63 -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 66076] Final block of encrypted response payload not being sent occasionally over TLS
https://bz.apache.org/bugzilla/show_bug.cgi?id=66076 --- Comment #1 from Phil Clay --- Created attachment 38292 --> https://bz.apache.org/bugzilla/attachment.cgi?id=38292=edit server-side application log -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
