Re: WebSocket Handshake returns Unexpected Response Code 500

[Igal @ Lucee.org]

OK, apparently HandshakeRequest's getRequestURI().getAuthority() returns 
null and my code didn't account for that.


I'm surprised that nothing was logged anywhere though, even with a FINE 
log level.


Igal Sapir
Lucee Core Developer
Lucee.org 


On 12/6/2016 5:18 PM, Igal @ Lucee.org wrote:

On 12/6/2016 4:25 PM, Igal @ Lucee.org wrote:


This is a cross post with 
http://stackoverflow.com/questions/41007216/websocket-handshake-on-tomcat-returns-unexpected-response-code-500 
:


I have a WebSocket application that registers the endpoint 
programmatically, without the use of Annotations.


The relevant code is:

|ServerEndpointConfig.Configurator configurator = 
(ServerEndpointConfig.Configurator)configuratorClass.newInstance(); 
ServerEndpointConfig serverEndpointConfig = 
ServerEndpointConfig.Builder .create(endpointClass, endpointPath) 
.configurator(configurator) .build(); ServerContainer serverContainer 
= (ServerContainer)app .getServletContext() 
.getAttribute("javax.websocket.server.ServerContainer"); 
serverContainer.addEndpoint(serverEndpointConfig); |


Everything works fine when I run it inside an IDE with an embedded 
Jetty container, where the dependency is on tomcat-websocket and 
tomcat-websocket-api libraries via Maven, but when I deploy it in 
Tomcat (8.5.5), and then try to establish a WebSocket connection, I 
get the following error in the browser's console:


WebSocket connection to 'ws://127.0.0.1:8080/echo' failed: Error
during WebSocket handshake: Unexpected response code: 500

I do not get any error or other information logged in the Catalina 
output, and am not sure how to go about debugging this issue.


I need to configure the endpoint programmatically because much of the 
information is not known at compile time, so please do not point me 
to an Annotation solution.


Any help would be greatly appreciated, thanks!


I have set the log level to FINE, and here is the relevant snippet:

T.550 FINE [http-nio-IP-Acceptor-0] 
o.a.tomcat.util.threads.LimitLatch.countUpOrAwait Counting 
up[http-nio-IP-Acceptor-0] latch=2
T.551 FINE [http-nio-IP-exec-3] 
o.a.coyote.AbstractProtocol$ConnectionHandler.process Processing 
socket 
[o.a.tomcat.util.net.NioChannel@1e87b268:java.nio.channels.SocketChannel[connected 
local=/192.168.1.17:8080 remote=/192.168.1.17:51259]] with status 
[OPEN_READ]
T.551 FINE [http-nio-IP-exec-3] 
o.a.coyote.http11.Http11InputBuffer.parseRequestLine Received [GET 
/echo HTTP/1.1

Host: 192.168.1.17:8080
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: http://192.168.1.17:8080
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36

Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Cookie: LUCEE_ADMIN_LANG=en; 
cfid=de1b0016-ad0c-4601-b8ce-ecb47771b9c8; cftoken=0; 
LUCEE_ADMIN_LASTPAGE=debugging.logs

Sec-WebSocket-Key: Ogg0VPp9O3XZlR8rrgz4Gg==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits

]
T.553 FINE [http-nio-IP-exec-3] 
o.a.tomcat.util.http.Rfc6265CookieProcessor.parseCookieHeader Cookies: 
Parsing b[]: LUCEE_ADMIN_LANG=en; 
cfid=de1b0016-ad0c-4601-b8ce-ecb47771b9c8; cftoken=0; 
LUCEE_ADMIN_LASTPAGE=debugging.logs
T.554 FINE [http-nio-IP-exec-3] 
o.a.catalina.authenticator.AuthenticatorBase.invoke Security checking 
request GET /echo
T.554 FINE [http-nio-IP-exec-3] 
o.a.catalina.realm.RealmBase.findSecurityConstraints   No applicable 
constraints defined
T.554 FINE [http-nio-IP-exec-3] 
o.a.catalina.authenticator.AuthenticatorBase.invoke  Not subject to 
any constraint
T.561 FINE [http-nio-IP-exec-3] 
o.a.tomcat.util.http.Parameters.setEncoding Set encoding to ISO-8859-1
T.563 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.loadClass 
loadClass(o.a.tomcat.util.collections.LocalStrings, false)
T.563 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.loadClass   Delegating to 
parent classloader1 java.net.URLClassLoader@31befd9f
T.563 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.loadClass   Searching local 
repositories
T.563 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.findClass 
findClass(o.a.tomcat.util.collections.LocalStrings)
T.564 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.findClass --> Returning 
ClassNotFoundException
T.564 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.getResourceAsStream 
getResourceAsStream(org/apache/tomcat/util/collections/LocalStrings.properties)
T.564 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.getResourceAsStream 
Delegating to parent classloader java.net.URLClassLoader@31befd9f
T.565 FINE [http-nio-IP-exec-3] 

Re: WebSocket Handshake returns Unexpected Response Code 500

[Igal @ Lucee.org]

On 12/6/2016 4:25 PM, Igal @ Lucee.org wrote:


This is a cross post with 
http://stackoverflow.com/questions/41007216/websocket-handshake-on-tomcat-returns-unexpected-response-code-500 
:


I have a WebSocket application that registers the endpoint 
programmatically, without the use of Annotations.


The relevant code is:

|ServerEndpointConfig.Configurator configurator = 
(ServerEndpointConfig.Configurator)configuratorClass.newInstance(); 
ServerEndpointConfig serverEndpointConfig = 
ServerEndpointConfig.Builder .create(endpointClass, endpointPath) 
.configurator(configurator) .build(); ServerContainer serverContainer 
= (ServerContainer)app .getServletContext() 
.getAttribute("javax.websocket.server.ServerContainer"); 
serverContainer.addEndpoint(serverEndpointConfig); |


Everything works fine when I run it inside an IDE with an embedded 
Jetty container, where the dependency is on tomcat-websocket and 
tomcat-websocket-api libraries via Maven, but when I deploy it in 
Tomcat (8.5.5), and then try to establish a WebSocket connection, I 
get the following error in the browser's console:


WebSocket connection to 'ws://127.0.0.1:8080/echo' failed: Error
during WebSocket handshake: Unexpected response code: 500

I do not get any error or other information logged in the Catalina 
output, and am not sure how to go about debugging this issue.


I need to configure the endpoint programmatically because much of the 
information is not known at compile time, so please do not point me to 
an Annotation solution.


Any help would be greatly appreciated, thanks!


I have set the log level to FINE, and here is the relevant snippet:

T.550 FINE [http-nio-IP-Acceptor-0] 
o.a.tomcat.util.threads.LimitLatch.countUpOrAwait Counting 
up[http-nio-IP-Acceptor-0] latch=2
T.551 FINE [http-nio-IP-exec-3] 
o.a.coyote.AbstractProtocol$ConnectionHandler.process Processing socket 
[o.a.tomcat.util.net.NioChannel@1e87b268:java.nio.channels.SocketChannel[connected 
local=/192.168.1.17:8080 remote=/192.168.1.17:51259]] with status 
[OPEN_READ]
T.551 FINE [http-nio-IP-exec-3] 
o.a.coyote.http11.Http11InputBuffer.parseRequestLine Received [GET /echo 
HTTP/1.1

Host: 192.168.1.17:8080
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: http://192.168.1.17:8080
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36

Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Cookie: LUCEE_ADMIN_LANG=en; 
cfid=de1b0016-ad0c-4601-b8ce-ecb47771b9c8; cftoken=0; 
LUCEE_ADMIN_LASTPAGE=debugging.logs

Sec-WebSocket-Key: Ogg0VPp9O3XZlR8rrgz4Gg==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits

]
T.553 FINE [http-nio-IP-exec-3] 
o.a.tomcat.util.http.Rfc6265CookieProcessor.parseCookieHeader Cookies: 
Parsing b[]: LUCEE_ADMIN_LANG=en; 
cfid=de1b0016-ad0c-4601-b8ce-ecb47771b9c8; cftoken=0; 
LUCEE_ADMIN_LASTPAGE=debugging.logs
T.554 FINE [http-nio-IP-exec-3] 
o.a.catalina.authenticator.AuthenticatorBase.invoke Security checking 
request GET /echo
T.554 FINE [http-nio-IP-exec-3] 
o.a.catalina.realm.RealmBase.findSecurityConstraints   No applicable 
constraints defined
T.554 FINE [http-nio-IP-exec-3] 
o.a.catalina.authenticator.AuthenticatorBase.invoke  Not subject to any 
constraint
T.561 FINE [http-nio-IP-exec-3] 
o.a.tomcat.util.http.Parameters.setEncoding Set encoding to ISO-8859-1
T.563 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.loadClass 
loadClass(o.a.tomcat.util.collections.LocalStrings, false)
T.563 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.loadClass   Delegating to 
parent classloader1 java.net.URLClassLoader@31befd9f
T.563 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.loadClass   Searching local 
repositories
T.563 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.findClass 
findClass(o.a.tomcat.util.collections.LocalStrings)
T.564 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.findClass --> Returning 
ClassNotFoundException
T.564 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.getResourceAsStream 
getResourceAsStream(org/apache/tomcat/util/collections/LocalStrings.properties)
T.564 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.getResourceAsStream Delegating 
to parent classloader java.net.URLClassLoader@31befd9f
T.565 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.getResourceAsStream Searching 
local repositories
T.565 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.getResourceAsStream --> 
Resource not found, returning null
T.566 FINE [http-nio-IP-exec-3] 
o.a.catalina.loader.WebappClassLoaderBase.loadClass 

WebSocket Handshake returns Unexpected Response Code 500

[Igal @ Lucee.org]

This is a cross post with 
http://stackoverflow.com/questions/41007216/websocket-handshake-on-tomcat-returns-unexpected-response-code-500 
:


I have a WebSocket application that registers the endpoint 
programmatically, without the use of Annotations.


The relevant code is:

|ServerEndpointConfig.Configurator configurator = 
(ServerEndpointConfig.Configurator)configuratorClass.newInstance(); 
ServerEndpointConfig serverEndpointConfig = ServerEndpointConfig.Builder 
.create(endpointClass, endpointPath) .configurator(configurator) 
.build(); ServerContainer serverContainer = (ServerContainer)app 
.getServletContext() 
.getAttribute("javax.websocket.server.ServerContainer"); 
serverContainer.addEndpoint(serverEndpointConfig); |


Everything works fine when I run it inside an IDE with an embedded Jetty 
container, where the dependency is on tomcat-websocket and 
tomcat-websocket-api libraries via Maven, but when I deploy it in Tomcat 
(8.5.5), and then try to establish a WebSocket connection, I get the 
following error in the browser's console:


   WebSocket connection to 'ws://127.0.0.1:8080/echo' failed: Error
   during WebSocket handshake: Unexpected response code: 500

I do not get any error or other information logged in the Catalina 
output, and am not sure how to go about debugging this issue.


I need to configure the endpoint programmatically because much of the 
information is not known at compile time, so please do not point me to 
an Annotation solution.


Any help would be greatly appreciated, thanks!


--

Igal Sapir
Lucee Core Developer
Lucee.org 

Re: Connector bindOnInit=false not behaving as expected

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 12/6/16 4:17 AM, Mark Thomas wrote:
> On 05/12/2016 23:03, Christopher Schultz wrote:
>> Mark,
>> 
>> On 12/4/16 3:24 PM, Mark Thomas wrote:
>> It looks like the ProtocolHandler is really the place
>> where the TLS configuration is taking effect, and not
>> the Connector, so I'm largely ignoring the Connector for
>> now. Is that the right choice to make, here?
>> 
>>> It depends. What you actually need to do is swap out the 
>>> SSLContext. This is in the SSLHostConfigCertificate.
>> 
>> It looks like the only place the SSLContext is actually set is
>> from Nio(2?)Endpoint.bind(). So that means that, with the current
>> Tomcat implementation, changing the SSLContext means an unbind()
>> followed by a bind(). Do I have that correct?
> 
> Yes.

With changes to the Tomcat code, that answer can obviously change.

>> This would be for JSSE only... for OpenSSL/APR, I suspect it will
>> be the same thing, but any change would have to take both into
>> account.
>> 
>> What I'd really like to be able to do is replace the SSLContext 
>> without dropping any in-flight requests, while new requests wait
>> to be serviced until the new configuration was available.
>> Something like this:
>> 
>> 1. Stop processing incoming connections (e.g. still accept(),
>> but don't handshake, yet... or at least stay bound to the port,
>> but don't yet accept() and allow the TCP stack backlog to queue
>> incoming connections)
>> 
>> 2. Load the new TLS configuration
>> 
>> 3. Resume accepting connections with the updated TLS
>> configuration
>> 
>> 4. Requests accepted before step #1 continue to use the
>> configuration effective at the time
>> 
>> I'm not sure how all that squares with that JSSE is willing to
>> do.
> 
> It should be as simple as:
> 
> 1. Construct a new SSLContext
> 
> 2. Replace the old SSLContext with the new one.

So if there were a reloadTLSConfiguration method on the Connector (or
similar component), it could simply call setSSLContext() on the
SSLHostConfigCertificate and wait for the next request to come in?

> It gets slightly trickier with APR/native as you need to
> explicitly clean up the old one and you can't do that until all the
> connections using it have closed. JSSE doesn't have that
> requirement.

Understood. I'm focusing on JSSE for the moment.

I also understand that e.g. websocket makes this even more fun, since
the connections are expected to be relatively long-lived.

>> Once the TLS handshake occurs, presumably it means that the
>> connection will continue to be valid until it's closed. If a
>> connection as mentioned in #4 above is long-running, the old
>> SSLContext etc. will be GC'd after the connection finally closes.
>> If that assumption is not correct, I don't think any of the above
>> is even possible.
>> 
>> What probably is possible is pausing all incoming connections,
>> waiting X ms for them to complete, then unbind(), then bind().
>> Client observation would be that the service seems to stall, then
>> fail. Presumably, immediately re-trying would connect and get the
>> new configuration.
>> 
>> How difficult do you think it would be to implement such a
>> "graceful" shutdown of the connector? It would be the same as
>> "stop" except that it would first allow all in-flight requests to
>> complete (with some reasonable timeout... say, as specified by a
>> parameter to a gracefulShutdown(int) call?) and then, in the case
>> of the bindOnInit="false", it would unbind().
> 
> I think that is the wrong approach. Just leave everything running
> and just replace the SSLContext on the fly. With the caveat that
> APR/native is a little trickier. We might need to add usage
> counting or similar.

Cool.

>> That would allow a JMX client to say "shut down gracefully",
>> waiting for an OK response (which would indicate that the
>> connector had in fact shut down), then immediately request a
>> "start" and the connector would come back up. At this point, I'd
>> probably argue that we should add a "restart gracefully" to the
>> list of JMX-callable actions which just does both of those calls
>> on the server without a second HTTP roundtrip from the JMX
>> client.
> 
> I think what we need is a refreshTLSConfig() method.

Perfect.

If such a method were to be on the Connector (at least for the JMX
bean representing the Connector), it looks like a fairly simple call
(or calls.. for each SSLHostConfig) to
AbstractEndpoint.createSSLContext would do the trick. That method
seems to not only create the SSLContext but also set that new context
on the certificate (SSLHostConfigCertificate) object.

I think there might be some thread-safety issues though with the
changing configuration of e.g. sslHostConfig.enabledProtocols and
sslHostConfig.enabledCiphers before the certificate is updated. As it
stands, the code seems to expect that, once initialized, no later
re-initializations occur. We'd need to make sure that we swap-out 

Re: Connector bindOnInit=false not behaving as expected

[Mark Thomas]

On 05/12/2016 23:03, Christopher Schultz wrote:
> Mark,
> 
> On 12/4/16 3:24 PM, Mark Thomas wrote:
> It looks like the ProtocolHandler is really the place where
> the TLS configuration is taking effect, and not the
> Connector, so I'm largely ignoring the Connector for now. Is
> that the right choice to make, here?
> 
>> It depends. What you actually need to do is swap out the
>> SSLContext. This is in the SSLHostConfigCertificate.
> 
> It looks like the only place the SSLContext is actually set is from
> Nio(2?)Endpoint.bind(). So that means that, with the current Tomcat
> implementation, changing the SSLContext means an unbind() followed by
> a bind(). Do I have that correct?

Yes.

> This would be for JSSE only... for OpenSSL/APR, I suspect it will be
> the same thing, but any change would have to take both into account.
> 
> What I'd really like to be able to do is replace the SSLContext
> without dropping any in-flight requests, while new requests wait to be
> serviced until the new configuration was available. Something like this:
> 
> 1. Stop processing incoming connections (e.g. still accept(), but
> don't handshake, yet... or at least stay bound to the port, but don't
> yet accept() and allow the TCP stack backlog to queue incoming
> connections)
> 
> 2. Load the new TLS configuration
> 
> 3. Resume accepting connections with the updated TLS configuration
> 
> 4. Requests accepted before step #1 continue to use the configuration
> effective at the time
> 
> I'm not sure how all that squares with that JSSE is willing to do.

It should be as simple as:

1. Construct a new SSLContext

2. Replace the old SSLContext with the new one.

It gets slightly trickier with APR/native as you need to explicitly
clean up the old one and you can't do that until all the connections
using it have closed. JSSE doesn't have that requirement.

> Once the TLS handshake occurs, presumably it means that the connection
> will continue to be valid until it's closed. If a connection as
> mentioned in #4 above is long-running, the old SSLContext etc. will be
> GC'd after the connection finally closes. If that assumption is not
> correct, I don't think any of the above is even possible.
> 
> What probably is possible is pausing all incoming connections, waiting
> X ms for them to complete, then unbind(), then bind(). Client
> observation would be that the service seems to stall, then fail.
> Presumably, immediately re-trying would connect and get the new
> configuration.
> 
> How difficult do you think it would be to implement such a "graceful"
> shutdown of the connector? It would be the same as "stop" except that
> it would first allow all in-flight requests to complete (with some
> reasonable timeout... say, as specified by a parameter to a
> gracefulShutdown(int) call?) and then, in the case of the
> bindOnInit="false", it would unbind().

I think that is the wrong approach. Just leave everything running and
just replace the SSLContext on the fly. With the caveat that APR/native
is a little trickier. We might need to add usage counting or similar.

> That would allow a JMX client to say "shut down gracefully", waiting
> for an OK response (which would indicate that the connector had in
> fact shut down), then immediately request a "start" and the connector
> would come back up. At this point, I'd probably argue that we should
> add a "restart gracefully" to the list of JMX-callable actions which
> just does both of those calls on the server without a second HTTP
> roundtrip from the JMX client.

I think what we need is a refreshTLSConfig() method.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: SSDLC Compliance

[Mark Thomas]

On 06/12/2016 07:21, GAN Kok Leong, Adrian wrote:
> Hi,
> 
> We have a cybersecurity requirement for all software. We would like
> to know whether Tomcat version 7.0.25 is developed and comply with
> Secure Software Development Life Cycle (SSDLC)?

This sounds suspiciously like a security box ticking exercise. I'm sure
someone could make the case that Tomcat development does use a SSDLC
just as easily as someone could make that case that it does not.

A focus on actual security rather than box ticking might ask "Are there
known security vulnerabilities in a piece of software released 4, almost
5, years ago and if there are, why are we still using it?" The answer to
the first part of that question may be found here:

http://tomcat.apache.org/security-7.html

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org