Hi Chris,
Thanks again for replying. I agree it doesn't look like a Tomcat issue and
would be happy to close the issue. I've included responses to your points
below prefixed with [DMC] for my initials, but think I need to focus on the
Java side of things and perhaps open an Oracle ticket.
Best regards,
Diarmuid
On 28 Aug 2015 19:08, "Christopher Schultz" <ch...@christopherschultz.net>
wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Diarmuid,
>
> I'm marking this as off-topic because it seems like Tomcat really has
> nothing to do with the situation.
>
> On 8/27/15 4:31 PM, dmccrthy wrote:
> > Thanks for responding so quickly. My apologies, I should have been
> > clearer on the topology. We have a Tomcat instance with a 3rd party
> > web app deployed on it (the "Tomcat client") running on Windows
> > 2008 . This connects via HTTPS to a 3rd party service running
> > behind IBM Http Server on Solaris 10.
>
> Okay, so, just to confirm:
>
> Tomcat (app) -> HTTPS -> 3rd party service
>
> Configurations and results:
>
> 0. prod Tomcat -> prod 3rd-party service
> FAILS with TLS_RSA_WITH_AES_128_CBC_SHA
>
[DMC] Correct
> (do other ciphers work? or are you completely dead in the water?)
[DMC] No, this is the next step. Our production environment is completely
locked down and we have to exhaust all other avenues before making
speculative changes.
> 2. prod Tomcat -> prod 3rd-party service
> with lower version of Tomcat and/or Java
> WORKS with TLS_RSA_WITH_AES_128_CBC_SHA
[DMC] Correct
> 1. test Tocmat -> test 3rd-party service
> WORKS with TLS_RSA_WITH_AES_128_CBC_SHA
[DMC] Correct
> 2. test client (command-line?) on production host -> prod 3rd parts
> service
> WORKS with TLS_RSA_WITH_AES_128_CBC_SHA
[DMC] Correct. The command line test client is a simple Java class that
just opens an InputStream to the 3rd party service
> No changes are known/expected, other than Tomcat and Java version.
[DMC] It's a greenfield app and no changes are planned bar functional
patches. The same version of operating system, java, tomcat, and the
application is in our non-prod environment and works ok
> > So the tool we're using is our 3rd party client Web app (the
> > "Tomcat client"). It shows the same negotiation from TLSv1.2 down
> > to TLSv1. My reading of your comments is that the
> > truststore/keystore has no bearing here because it's an outgoing
> > client connection and there are no inbound connections at play.
>
> Exactly. Neither the keystore nor the truststore have any bearing,
> here. If the client library itself uses (or supports) a
> truststore/keystore, then you'll likely know about it because you'd
> have to make a call like client.setTruststore("3rdparty.jks") or whateve
> r.
>
> > * I don't know what the highest supported cipher suite is on either
> > client or server hosts. I do know that TLSv1 is the highest
> > supported by the 3rd party server.
>
> Good to know, but not terribly relevant. The client and server should
> negotiate a common cipher suite based upon those available on either
> end. If they can't agree, the handshake fails.
>
> > * we're not using JCE Unlimited Security because it hasn't been an
> > issue and frankly I didn't think of it when we were doing the
> > installations :)
>
> Okay.
>
> > Two additional pieces of analysis that came out of our
> > investigations today are:
> >
> > 1) we are using NextGen Cisco devices. These are application and
> > protocol aware. This means that the firewall might allow allow our
> > standalone Java client to connect using HTTPS, but interfere with
> > the HTTPS SOAP requests from our Tomcat/3rd party app "Tomcat
> > client".
>
> How can the device be "application aware"? How does the applicaiton
> identify itself to the device? How is that authenticated? What is
> stopping EvilApp.exe from claiming to be InnocentApp.exe?
[DMC] the firewall devices check the tcp connections. I don't know how. I
do know we had endless trouble from overzealous firewall guys putting
restrictive rules in place. The impact of that was that, for example, we
could telnet through the firewall but not establish a Sybase connection. In
any case, we compared all the non-prod and prod firewall rules and they are
configured the same way. We also had several debug sessions with the
firewall guys and no traffic is being blocked. So at least we've ruled that
out
> > 2) there are some anecdotes of Java 7 clients failing to downgrade
> > from TLSv1.2 to TLSv1. For example
> > http://stackoverflow.com/questions/31579587/java-tls-1-2-downgrade-to-
> tls-1-0-sometimes-error.
>
> This
> >
> is not a "piece of analysis". This is something you found online
> that might be related. But you haven't collected any information to
> support the idea that it's related. I note that the experience is
> different in that SO article: the user is getting a different error at
> a different part of the handshake.
[DMC] I agree. But in the spirit of trying different cipher suites, this is
something that may (or may not) be relevant
> > Some takeaway actions for me to explore are below. I'll welcome any
> > other thoughts that you may have.
> >
> > 1) compare the firewall rules on our working non-production client
> > with the rules governing the failing production client
>
> Firewalls are usually pretty dumb. I'd be surprised if the firewall is
> breaking anything, here.
>
> > 2) use openssl from our client server and see what it shows
>
> Yes, please do that. Why wasn't that the first thing you tried?
[DMC] capacity issues, to be honest. Analysing this type of issue in my
organisation involves numerous teams across multiple timezones and the
communication overheads are enormous. Plus our production servers are
strictly controlled so I can't install it myself. It's taken me a week to
get this far and it's been on my list all that time.
> > 3) consider forcing the Tomcat/3rd party Tomcat client to use
> > TLSv1
>
> This is a possibility, but you shouldn't have to hamper your own
> application because there are some servers out there that are so old
> they don't support TLSv1.2: you should always offer the best
> encryption you can, and only fall-back when necessary.
[DMC] The 3rd party service only supports TLSv1. So as a diagnostic step we
have nothing to lose.
> > 4) check other logs on the server side to see if other clients are
> > failing TLS handshakes
>
> You didn't do this already?
[DMC] our server-side partners are under the same production limitations
that we are and insisted on discounting all possible infrastructure and
client possibilities before increasing diagnostics. They did that today and
found that the connections are being received but are being terminated by a
generic SSL handshake error.
> > 5) investigate the supported ciphers on client and server
>
> There is code available in the archives of this mailing list to do
> that. Search for my name and "ssl" and poke-around.
[DMC] Thanks, I will
> > 6) consider upgrading Tomcat (as a last resort)
>
> I thought the initial problem was that an upgrade to Tomcat "broke
> something". I think it's more likely to be the Java upgrade that did
> something, and not Tomcat itself.
[DMC] No, sorry for not being clear. This production instance is a
greenfield installation and has never successfully connected (browser and
standalone tests excepted)
> Are you sure that the standalone test client is using the exact same
> version of Java as the Tomcat server is?
[DMC] 100%. There's only one version of java installed
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
>
> iQIcBAEBCAAGBQJV4KNUAAoJEBzwKT+lPKRYPVMQAILQm/n33Mxh7cbYF3u0VfNg
> eodcwvrXxWynhDX61ZjUGNTAPbEAsReLIuCzZjJpmB+LIaBsg8qsxpNL4zu+W4YA
> ZYB2wiORHpgqXVXG3JuX2tlzKGmAaF8YnlWnsjaw+taZNWOwVvFvJN4esdp8eYYw
> 8xaLU1R0RLMZspIk73VigYhReRSrVpfOxhGomLE9uJle4tH5KoK+uicmWqk9rwaj
> yRgM7fxrk5tctolnuzasKl/17JtWkdSpEHHhkGZiu+pgdBYkYmzAzkM9PmBIzkra
> MLZ/6kG2R6UPg5HLuxfad+IYGZP7zppZAbIPfxw5zTQm7edfCSBl+2OoKnOdqFxS
> dV/Tj5Jg1HkyReDDBH4A7+aYRCnXwurrM6XaLj++MxeNN94FmBiWo/Xo7KjH2S8j
> EYWz/l8L1SheLm0/sGDO59giCFc32WNW9CFvHfL/qL7pmsX5AluE5BydesSOqias
> AxyKMBci98zMgjOYw7SzOfr9fyag7NwW5hBTCfhf55tZSGnMWVGCWS6ihzDTDwl9
> j+xWA04tQqNU2Qfd/7gmGtHSNEbJbBYKZ9U3EzjDWZYHEwDGoYe2EJAAQ61v2OsF
> m7SWwlfAXQbGK85DO/n88beQ0w5QegfwFUNuYFfBvUVeK+cfJMIfbb05+wpme8g+
> cUZATik9jXqZNIIJMMdm
> =j1cd
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
