Okay, guys, please accept my most sincere apology on this thing. I just figured 1 thing out.
browser -> f5:443 -> httpd (23270, https) -> tomcat (http) using the /static works! (see below) If /static/index.html wasnt specified, then it hangs, then comes back as https://loadbalancer.domain.com:23270/static But if you go https://loadbalancer.domain.com/static/index.html it works!!! DirectoryIndex index.html is present. because if you just go to https://webserver1:23270/static it works fine, no need for the extra /index.html not sure why that is different there. I only found this out using curl and just happened to think of it. So you see the difference, from the tomcat access log, in the two variations. 172.18.160.66 - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1" 302 - 172.18.160.66 - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html HTTP/1.1" 200 32 Still /SelfService wont work though I cant think of a way to do /SelfService/something... Since it is a web app and i am no developer and its all from the vendor (BMC Software) and it never shows naything other than /SelfService in the URL. I dont think its a frame, but once again i dont know these things. I know they told me it is HTML5, as it was using Silverlight back in the older release. On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <aaronmg...@gmail.com> wrote: > Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were > already open to the VIP on the F5, we simply turned of 80 -> 443 redirect > on the F5, and then configured the F5 to use the non-HTTP port in Apache > HTTP Server (the two backend servers). So its HTTP the entire way through, > and the same issue persisted. > > browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl) > > So i think i can rule out any possibility that the fact I was terminating > SSL in Apache HTTP Server then proxying clear-text back as a reason. It > worked fine like I said if you just go direct to the HTTP Server and slip > the F5. And F5 works landing the /healthcheck.html (HTTP KeepAlive in the > root, which isnt proxied), so all good there. Just cant /SelfService > (configured w/ Proxy) > > thanks for your support here though. > > On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aaronmg...@gmail.com> wrote: > >> SSLProxyEngine On >> Was already turned on this entire time inside the ssl.conf (I include it) >> VirtualHost section. >> >> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app >> server, and then HTTPS the whole way through and see if that makes any >> difference. I may need to request a new firewall to be opened, which is >> not able to open until 3/2. Gonna see. >> >> >> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <cofi...@gmail.com> >> wrote: >> >>> >>> On 23/02/17 12:43, André Warnier (tomcat) wrote: >>> >>>> On 22.02.2017 19:22, Aaron Gray wrote: >>>> >>>>> So this is interesting. >>>>> >>>>> So from HTTP server #1 (172.1.1.1 example) I hit: >>>>> https://172.1.1.1:23270/static and I see this in the HTTP log: >>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200 >>>>> 32 >>>>> I see this in the Tomcat log: >>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200 >>>>> 32 >>>>> ((( Yes I know they look identical, but the logformat is the same, so >>>>> it >>>>> comes out looking the same, but they are two diff log files totally >>>>> ))) >>>>> >>>>> I then hit the https://loadbalancer.domain.com/static from my Win10 >>>>> laptop >>>>> (10.1.1.2 example) >>>>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat >>>>> access >>>>> log. >>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 - >>>>> >>>>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from http >>>>> -> >>>>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i >>>>> dont >>>>> know yet. >>>>> >>>> >>>> It looks as if, in that case, httpd is trying to connect to tomcat, but >>>> - either it /is/ connecting, and sending the request, but tomcat never >>>> answers >>>> - or it is connecting to something else, which isn't tomcat >>>> (but which accepts the request and doesn't answer either) >>>> >>>> I am not *sure*, but I believe, that if httpd was trying to connect to >>>> something that isn't tomcat, and is not listening on port 18080, it would >>>> get a different error, and a different logfile message. So it would seem >>>> that at least the TCP connection works. >>>> >>> >>> >>> Is the F5 loadbalancer configured to make reverse proxy to a SSL http? >>> The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled. >>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine >>> >>> If the loadbalancer it's delegating the SSL work to the server1 >>> server1 https listen = 172.1.1.1:23270 >>> >>> then the F5 could be connecting as http regular TCP connection to a SSL >>> TCP one and therefor the SSL handshake it's not happening. >>> >>> Antonio >>> >>> >>> >>> >>> >>> >>>> Now there is still a puzzle there : when you connect as >>>> browser --> httpd --> tomcat >>>> then everything works : httpd connects to tomcat, sends a request, gets >>>> a response, and returns it to the browser. >>>> >>>> But when you do the same as : >>>> browser --> F5 --> httpd --> tomcat >>>> it "does not work", and the problem seems to be at httpd --> tomat >>>> Why would it make a difference in httpd --> tomcat, if it is either the >>>> browser or the F5 which sends the request to httpd ? >>>> the httpd --> tomcat connection should be the same, no matter what. >>>> >>>> Hmm, for now as puzzled as you are. >>>> Maybe we have a quantum phenomenon here, where the mere fact of you >>>> looking at the logfile, changes what happened beforehand. Just kidding. >>>> >>>> Informational note : the Apache access log line, is written when Apache >>>> *has finished* processing the request (including the possible round-trip >>>> through tomcat) and has sent the response to the client. (That's because, >>>> for example, it can also write the time spent, and the size of the >>>> response; so it has to know those before it can write the log line). >>>> I presume that it is the same for tomcat. >>>> Just mentioning this, so that you would maybe have a second look at the >>>> tomcat access log, after a few minutes, to make sure that there really is >>>> no corresponding line there. >>>> It could be that tomcat actually /is/ receiving the request, and >>>> processing it, but that it takes a long time, and that httpd tires of >>>> waiting and already times out in the meantime. In such a case, you would >>>> probably see an error some time later in the tomcat error log, when it is >>>> trying to eventually return the response, and finds a closed connection >>>> instead. >>>> Now again, why it would do that in one case and not the other, is still >>>> a mystery. >>>> >>>> Another note : in the "timeout" log line above, httpd is not showing >>>> the request URL which triggered this timeout. >>>> a) it may be showing some additional information in the Apache error log >>>> b) these exists an httpd add-on module (mod_log_forensic ?) which could >>>> provide more details in the log, about what happens precisely >>>> >>>> >>>> >>>>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <a...@ice-sa.com >>>>> > >>>>> wrote: >>>>> >>>>> On 21.02.2017 23:28, Aaron Gray wrote: >>>>>> >>>>>> Antonio: The Tomcat server has no knowledge of the F5, or that it is >>>>>>> being >>>>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache >>>>>>> HTTP >>>>>>> Server, and clear-text from HTTP to Tomcat. >>>>>>> My redirect port for the normal HTTP listen in Tomcat is commented >>>>>>> out. >>>>>>> <Connector port="18080" protocol="HTTP/1.1" >>>>>>> connectionTimeout="20000" /> >>>>>>> <!-- A "Connector" using the shared thread pool--> >>>>>>> <!-- >>>>>>> <Connector executor="tomcatThreadPool" >>>>>>> port="8080" protocol="HTTP/1.1" >>>>>>> connectionTimeout="20000" >>>>>>> redirectPort="8443" /> >>>>>>> --> >>>>>>> >>>>>>> Andre: >>>>>>> The URL I am using is https://loadbalancer.domain.com >>>>>>> It is listening on port 80 and 443, if you hit 80, internally it >>>>>>> redirects >>>>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends >>>>>>> the >>>>>>> traffic to one of the two HTTP servers (round-robin, also tried >>>>>>> persistence, no difference). The HTTP server is listening only for >>>>>>> HTTPS >>>>>>> on 23270/tcp. >>>>>>> When I access these DMZ webservers via the F5 load balancer (to >>>>>>> which I >>>>>>> dont have access to, but the network folks configure for me), it >>>>>>> hangs. >>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270 >>>>>>> >>>>>>> https://loadbalancer.domain.com:23270/SelfService >>>>>>> cant load. >>>>>>> >>>>>>> No idea why the URL is being re-written with the ":23270". >>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to >>>>>>> test: >>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" /> >>>>>>> Then put a simple index.html in there. Accessing via the Apache Web >>>>>>> Servers works fine, but if you hit it with the Load Balancer it once >>>>>>> again >>>>>>> adds the https://loadbalancer.domain.com:23270/static >>>>>>> >>>>>>> Hitting https://loadbalancer.domain.com >>>>>>> I see my "Hello world!" which is all that is in index.html. This is >>>>>>> the >>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time. >>>>>>> >>>>>>> >>>>>> So in this case, there is no delay, and you get the Apache >>>>>> httpd-hosted >>>>>> "index.html" containing "Hello World. Right ? >>>>>> >>>>>> Only >>>>>> >>>>>> /SelfService and /static are proxied >>>>>>> /static just being my test of static content, but still served up by >>>>>>> Tomcat.. >>>>>>> >>>>>>> It's exactly 30 seconds before the page cannot be loaded when trying >>>>>>> anything proxied to Tomcat, but also accessed via the F5 load >>>>>>> balancer. >>>>>>> Not sure where the 30 seconds comes from; perhaps a load balancer >>>>>>> time >>>>>>> out, >>>>>>> as I dont see a "30" in my httpd configurations or my tomcat >>>>>>> server.xml >>>>>>> >>>>>>> >>>>>>> You can certainly look at the Apache httpd logs, and the tomcat >>>>>> logs, to >>>>>> see if you get a request or not. >>>>>> In Apache httpd, you can set the loglevel individually for mod_proxy >>>>>> (if >>>>>> you are running v 2.4), and it should show something if it gets this >>>>>> request and forwards it to tomcat. >>>>>> In tomcat, you can either enable an access log (which will show if it >>>>>> receives this request), or you could temporarily remove/rename the >>>>>> /static >>>>>> webapp. This way, it should trigger an error "not found" which you >>>>>> would >>>>>> also see in the error log. >>>>>> >>>>>> There should be nothing between them to hinder it. We have many load >>>>>> >>>>>>> balancers and this one specifically you dont need to open any >>>>>>> firewall >>>>>>> requests for the specific networks the HTTP servers are on. I did >>>>>>> have to >>>>>>> get the firewall opened up to allow me to hit >>>>>>> https://loadbalancer.domain.com because the VIP for " >>>>>>> loadbalancer.domain.com" >>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443 >>>>>>> without opening holes. But beyond that, any connection from the F5 >>>>>>> to the >>>>>>> HTTP Server should be 100% open bi-directional, since same subnet. >>>>>>> >>>>>>> >>>>>>> But something isn't working, otherwise you would not be asking. >>>>>> So, >>>>>> >>>>>> a) hitting the tomcat webapps through httpd seems to be working fine >>>>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static) >>>>>> >>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work >>>>>> fine >>>>>> too, even through the F5 >>>>>> (browser -> F5:443 -> httpd:23270 -> html page) >>>>>> >>>>>> c) it is only when you do : >>>>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or >>>>>> static) >>>>>> that you see this issue >>>>>> >>>>>> It would really help if you looked in the logs of both httpd and >>>>>> tomcat, >>>>>> and checked for differences betweens cases a, b and c above. >>>>>> >>>>>> I believe that the F5 message with the port 23270 is a minor issue, of >>>>>> information disclosure by the F5, that it should not disclose. >>>>>> >>>>>> But the reason why it returns this error is obviously that in that >>>>>> case, >>>>>> it does not get a response from his request to httpd. >>>>>> The reason for this response not coming back to the F5 (in case c >>>>>> only), >>>>>> can be due to either httpd or tomcat. But F5 doesn't know about >>>>>> tomcat. So >>>>>> for the F5, it is httpd which is not responding. Thus, >>>>>> - either httpd is never getting the request from the F5 (unlikely, >>>>>> because >>>>>> in b above it gets it and responds) >>>>>> - or httpd is getting the request from the F5, but not forwarding it >>>>>> to >>>>>> tomcat, but also not returning an immediate error response to F5 >>>>>> (which >>>>>> seems also unlikely, because of a and b) >>>>>> - or httpd is getting the request, forwarding it to tomcat, but not >>>>>> getting a response from tomcat. So >>>>>> - either tomcat is never getting the request from httpd (but in a, >>>>>> it >>>>>> gets it) >>>>>> - or tomcat is getting the request from httpd, but not responding >>>>>> (but >>>>>> in a, it does) >>>>>> - or tomcat is getting the request and responding, but the response >>>>>> never gets back to httpd (but in a, it does) >>>>>> >>>>>> So if a and b and c are all accurate, there is something apparently >>>>>> illogical happening. >>>>>> This would lead to the conclusion that a and b and c cannot all be >>>>>> accurate. >>>>>> >>>>>> The logs.. ? >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <cofi...@gmail.com >>>>>> > >>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> Aaron, on tomcat instances change the redirectPort attributte on the >>>>>>> http >>>>>>> >>>>>>>> conectó to the loabbalancer's port 443 >>>>>>>> >>>>>>>> My guess is that your webapp has restriction rule requesting SSL con >>>>>>>> fidntial channel. Therefore the non-confidential to the 18080 port >>>>>>>> from >>>>>>>> the >>>>>>>> balancer are redirected to the 23270 port, but it should be 443. >>>>>>>> >>>>>>>> Antonio >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aaronmg...@gmail.com> escribió: >>>>>>>> >>>>>>>> I have an application server from a vendor that comes bundled with >>>>>>>> an >>>>>>>> additional Apache Tomcat server. The webapp SelfService.war is >>>>>>>> vendor >>>>>>>> supplied too. >>>>>>>> >>>>>>>> Here's my problem (IP's replaced to protect the innocent): >>>>>>>> >>>>>>>> networks: >>>>>>>> DMZ=172.x.x.x >>>>>>>> INTERNAL=10.x.x.x >>>>>>>> >>>>>>>> server1 https listen = 172.1.1.1:23270 >>>>>>>> server2 https listen = 172.1.1.2:23270 >>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443 >>>>>>>> backend tomcat server = 10.1.1.1:18080 >>>>>>>> >>>>>>>> mod_proxy configuration: >>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService >>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService >>>>>>>> >>>>>>>> When I access these DMZ webservers which mod_proxy back to Apache >>>>>>>> Tomcat >>>>>>>> as: >>>>>>>> https://172.1.1.1:23270/SelfService >>>>>>>> and >>>>>>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfS >>>>>>>> ervice >>>>>>>> >>>>>>>>> >>>>>>>>> They load properly. Perfectly, every time! >>>>>>>> >>>>>>>> When I access these DMZ webservers via the F5 load balancer (to >>>>>>>> which I >>>>>>>> dont have access to, but the network folks configure for me), it >>>>>>>> hangs. >>>>>>>> Eventually returns: >>>>>>>> https://loadbalancer.domain.com:23270/SelfService >>>>>>>> cant load. >>>>>>>> >>>>>>>> No idea why the URL is being re-written with the ":23270". >>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to >>>>>>>> test: >>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" /> >>>>>>>> Then put a simple index.html in there. Accessing via the Apache Web >>>>>>>> Servers works fine, but if you hit it with the Load Balancer it once >>>>>>>> again >>>>>>>> adds the https://loadbalancer.domain.com:23270/static >>>>>>>> >>>>>>>> Do you have any thoughts? Thanks so much, I have been working with >>>>>>>> this >>>>>>>> for weeks now with no success >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>>>> For additional commands, e-mail: users-h...@tomcat.apache.org >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>> For additional commands, e-mail: users-h...@tomcat.apache.org >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >>> >> >
