subject:"HTTP Response Rewriting to Replace Internal IP with FQDN"

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-14 Thread Susheel Jalali


Dear Aleks,

Your insights are thorough and your knowledge seems to be vast---from 
HAProxy to Tomcat to Red5 server…and perhaps many more.  It is a 
pleasure to learn from you.  Below we have provided the information you 
requested.  Your further guidance will help solve this issue that we are 
facing to access products through HAProxy.


Yes, Tomcat proxies Red5 server.  Diff with red5.properties (you 
referred in your link) is given below.


Product1 can be accessed

(1) directly, i.e., without HAProxy

(2) via Apache mod_proxy, reverse proxy and mod_ssl (HTTPS).  SSL 
certificate is authentic as Apache HTTP server and nginx are using it to 
serve other products.


The only problem:  In HAProxy case, we are unable to access Product1.  
Product1 is talking http (no SSL) with Tomcat.  SSL termination happens 
at HAProxy.


++

Red5.properties:  Diff with 
(https://github.com/Red5/red5-server/blob/master/src/main/server/conf/red5.properties)


++

http.host=

http.port=5080 # for dev, 5081 for testing

…

rtmp.port=1935 # for dev, 1937 for testing

…

rtmp.deadlockguard.sheduler.pool_size=16

# message executor configs (per application) - adjust these as needed if 
you get tasks rejected


…

rtmp.channel.initial.capacity=3

rtmp.channel.concurrency.level=1

rtmp.stream.initial.capacity=1

rtmp.stream.concurrency.level=1

rtmp.pending.calls.initial.capacity=3

rtmp.pending.calls.concurrency.level=1

rtmp.reserved.streams.initial.capacity=1

rtmp.reserved.streams.concurrency.level=1

Thank you.

Sincerely,

 --
Susheel Jalali

Coscend Communications Solutions
Web site: www(DOT)Coscend(DOT)com
--
CONFIDENTIALITY NOTICE: See 'Confidentiality Notice Regarding E-mail
Messages from Coscend Communications Solutions' posted at:
www(DOT)Coscend(DOT)com/Terms_and_Conditions.html
--



    Original Message 
   Subject: Re: HTTP Response Rewriting to Replace Internal IP with FQDN
   From: Aleksandar Lazic <al-hapr...@none.at>
   Date: Tue, October 13, 2015 2:44 pm
   To: Susheel Jalali <susheel.jal...@coscend.com>
   Cc: haproxy@formilux.org, "i...@coscend.com" <i...@coscend.com>



   Am 13-10-2015 23:36, schrieb Aleksandar Lazic:
> Dear Susheel Jalali.
>
> Am 13-10-2015 22:20, schrieb Susheel Jalali:
>> Dear Aleks,
>
> [snipp]
>
>> ++
>> Tomcat’sweb.xml <http://web.xml/>
>> 
>> Inweb.xml <http://web.xml/>, the context parameter is:
>>
>> 
>> globalScope
>> default
>> 
>>
>> and the filter mapping for the application is:
>> 
>> Product1Application
>> /*
>> 
>>
>> ++
>> Tomcat’sconfig.xml <http://config.xml/>
>> ++
>> 
>> 
>> 1937
>> 8443
>> no
>> 5081
>> http
>> none
>> Product1
>> Product1
>> /Product1/
>
> Sorry but this could not be the fullweb.xml <http://web.xml/>and
   server.xml!
>
> Tomcat will not be able to start with this files!
>
> Du you try to proxy the RED5 server?!
>
> https://github.com/Red5/red5-server

   If yes what's in your red5.properties ?

   
https://github.com/Red5/red5-server/blob/master/src/main/server/conf/red5.properties

From my point of view it looks like you need to set the http.*
   variables
   to the right values.

   
https://github.com/Red5/red5-server/blob/3a6885433218ce2070b9064e195fc2ffde031c88/src/main/java/org/red5/server/net/servlet/RedirectHTTPServlet.java#L42

   e. g.: https.port=1443 or what you want

> BR
> Aleks

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-14 Thread Aleksandar Lazic


Dear Susheel Jalali,

Am 14-10-2015 08:32, schrieb Susheel Jalali:

Dear Aleks,

Your insights are thorough and your knowledge seems to be vast---from
HAProxy to Tomcat to Red5 server…and perhaps many more.  It is a
pleasure to learn from you.  Below we have provided the information you
requested.  Your further guidance will help solve this issue that we 
are

facing to access products through HAProxy.

Yes, Tomcat proxies Red5 server.  Diff with red5.properties (you
referred in your link) is given below.


What's the opinion about this case from the Red5 Community?
https://red5.github.io/
http://stackoverflow.com/questions/tagged/red5

Mybe this could you also help.
https://stackoverflow.com/questions/17101374/how-do-i-tunnel-red5-through-http-without-getting-netconnection-connect-closed-e/17104742#17104742


Product1 can be accessed

(1) directly, i.e., without HAProxy

(2) via Apache mod_proxy, reverse proxy and mod_ssl (HTTPS).  SSL
certificate is authentic as Apache HTTP server and nginx are using it 
to

serve other products.

The only problem:  In HAProxy case, we are unable to access Product1.
Product1 is talking http (no SSL) with Tomcat.  SSL termination happens
at HAProxy.

++
Red5.properties:  Diff with
(https://github.com/Red5/red5-server/blob/master/src/main/server/conf/red5.properties)
++

http.host=


What happen when you change the line abouve to

http.host=coscend.com

To keep on topic ;-)

Have you tried to change the rspirep line with the following?

rspirep ^Location:\ (https?://)([^/]*)(:[0-9]+)(/.*)$ Location:\ 
\1coscend.com:14443\4 if hdr_location



http.port=5080 # for dev, 5081 for testing

rtmp.port=1935 # for dev, 1937 for testing


Which of them are returned as  in this response?

+++
Location: http:///Product1/
+++

I assume 5080.


rtmp.deadlockguard.sheduler.pool_size=16

# message executor configs (per application) - adjust these as needed 
if

you get tasks rejected

…

rtmp.channel.initial.capacity=3

rtmp.channel.concurrency.level=1

rtmp.stream.initial.capacity=1

rtmp.stream.concurrency.level=1

rtmp.pending.calls.initial.capacity=3

rtmp.pending.calls.concurrency.level=1

rtmp.reserved.streams.initial.capacity=1

rtmp.reserved.streams.concurrency.level=1

Thank you.

Sincerely,

 --
Susheel Jalali

Coscend Communications Solutions
Web site: www(DOT)Coscend(DOT)com
--
CONFIDENTIALITY NOTICE: See 'Confidentiality Notice Regarding E-mail
Messages from Coscend Communications Solutions' posted at:
www(DOT)Coscend(DOT)com/Terms_and_Conditions.html
--


 Original Message 
Subject: Re: HTTP Response Rewriting to Replace Internal IP with FQDN
From: Aleksandar Lazic <al-hapr...@none.at>
Date: Tue, October 13, 2015 2:44 pm
To: Susheel Jalali <susheel.jal...@coscend.com>
Cc: haproxy@formilux.org, "i...@coscend.com" <i...@coscend.com>

Am 13-10-2015 23:36, schrieb Aleksandar Lazic:

Dear Susheel Jalali.

Am 13-10-2015 22:20, schrieb Susheel Jalali:

Dear Aleks,


[snipp]


++
Tomcat’s web.xml [1]

In web.xml [1], the context parameter is:


globalScope
default


and the filter mapping for the application is:

Product1Application
/*


++
Tomcat’s config.xml [2]
++


1937
8443
no
5081
http
none
Product1
Product1
/Product1/


Sorry but this could not be the full web.xml [1] and server.xml!

Tomcat will not be able to start with this files!

Du you try to proxy the RED5 server?!

https://github.com/Red5/red5-server


If yes what's in your red5.properties ?



https://github.com/Red5/red5-server/blob/master/src/main/server/conf/red5.properties


From my point of view it looks like you need to set the http.*
variables
to the right values.



https://github.com/Red5/red5-server/blob/3a6885433218ce2070b9064e195fc2ffde031c88/src/main/java/org/red5/server/net/servlet/RedirectHTTPServlet.java#L42


e. g.: https.port=1443 or what you want


BR
Aleks




Links:
--
[1] http://web.xml/
[2] http://config.xml/

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-13 Thread Susheel Jalali


Dear Aleks,

Thank you for your continued help.  As you advised, we did the 
following.  We would appreciate any guidance you could give to solve 
this issue.


(1) We ran haproxy (1.5.14) in debug mode in two use cases for 
Product1.  The debug output for each case is posted below.
(2) Also, as requested, posted are the relevant parts of Tomcat web.xml 
and config.xml.
(3) We are able to access another product (Product2) with similar 
configuration, but the debug output does not have Location header.


Product1 Debug
Scenario 1:  Right external URL shows on the address bar, but the page 
gives error
When we use in backend:   rspirep ^Location:\ 
(https?://)([^:]*)(:[0-9]+)(/.*) Location:\ \4 if hdr_location


Result:
Google Chrome:  ERR_TOO_MANY_REDIRECTS.   The webpage at 
https://coscend.com:8443/Product1/ has resulted in too many redirects. 
Clearing your cookies for this site or allowing third-party cookies may 
fix the problem. If not, it is possibly a server configuration issue and 
not a problem with your computer.


Firefox:  Firefox has detected that the server is redirecting the 
request for this address in a way that will never complete.  This 
problem can sometimes be caused by disabling or refusing to accept cookies.


Scenario 2: Right Page is served, but shows http:// 
on the address bar


When we use in backend:  rspirep ^Location:\ 
(https?://)([^/]*)(:[0-9]+)(.*)$ Location:\ \1coscend.com\3\4 if 
hdr_location

Result:  It should be https://External_URL/Product1/signin?xyz

==
Debug output

Scenario 1:  Right external URL shows on the address bar, but the page 
gives error



Available polling systems :
  epoll : pref=300,  test result OK
  poll : pref=200,  test result OK
 select : pref=150,  test result FAILED
Total: 3 (2 usable), will use epoll.

:webapps-frontend.accept(0006)=0009 from [192.168.100.153:57322]
:webapps-frontend.clicls[0009:]
:webapps-frontend.closed[0009:]
0001:webapps-frontend.accept(0006)=0009 from [192.168.100.153:57323]
0001:webapps-frontend.clireq[0009:]: GET /Product1/ HTTP/1.1
0001:webapps-frontend.clihdr[0009:]: Host: coscend.com:8443
0001:webapps-frontend.clihdr[0009:]: Connection: keep-alive
0001:webapps-frontend.clihdr[0009:]: Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
0001:webapps-frontend.clihdr[0009:]: 
Upgrade-Insecure-Requests: 1
0001:webapps-frontend.clihdr[0009:]: User-Agent: Mozilla/5.0 
(Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) 
Chrome/45.100.2444.201 Safari/537.36
0001:webapps-frontend.clihdr[0009:]: Accept-Encoding: gzip, 
deflate, sdch
0001:webapps-frontend.clihdr[0009:]: Accept-Language: 
en-US,en;q=0.8
0001:webapps-frontend.clihdr[0009:]: Cookie: 
JSESSIONID=6CD9160AEB4B2FC16AC392BE6479F3D9; 
express_sid=s%3A4wgQy_swupi3Nwxj8PS-Ly-ylGRU92iX.02vFzzzMQz51sLlz1A2A3Ecob28KL6mT79o0xS3Idmg

0001:subdomain_p1.srvrep[0009:000a]: HTTP/1.1 302 Found
0001:subdomain_p1.srvhdr[0009:000a]: Server: Apache-Coyote/1.1
0001:subdomain_p1.srvhdr[0009:000a]: Location: 
http:///Product1/

0001:subdomain_p1.srvhdr[0009:000a]: Transfer-Encoding: chunked
0001:subdomain_p1.srvhdr[0009:000a]: Date: Tue, 13 Oct 2015 19:10:19 GMT
0001:subdomain_p1.srvhdr[0009:000a]: Connection: close

+++
Scenario 2:
+++
Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result FAILED
Total: 3 (2 usable), will use epoll.

0002:webapps-frontend.accept(0006)=000b from [192.168.100.153:57304]
:webapps-frontend.accept(0006)=0009 from [192.168.100.153:57302]
0002:webapps-frontend.clicls[000b:]
0002:webapps-frontend.closed[000b:]
0001:webapps-frontend.accept(0006)=000a from [192.168.100.153:57303]
:webapps-frontend.clicls[0009:]
:webapps-frontend.closed[0009:]
0001:webapps-frontend.clicls[000a:]
0001:webapps-frontend.closed[000a:]
0003:webapps-frontend.accept(0006)=0009 from [192.168.100.153:57305]
0003:webapps-frontend.clireq[0009:]: GET /Product1/ HTTP/1.1
0003:webapps-frontend.clihdr[0009:]: Host: coscend.com:8443
0003:webapps-frontend.clihdr[0009:]: Connection: keep-alive
0003:webapps-frontend.clihdr[0009:]: Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
0003:webapps-frontend.clihdr[0009:]: 
Upgrade-Insecure-Requests: 1
0003:webapps-frontend.clihdr[0009:]: User-Agent: Mozilla/5.0 
(Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) 
Chrome/45.100.2444.201 Safari/537.36
0003:webapps-frontend.clihdr[0009:]: Accept-Encoding: gzip, 
deflate, sdch
0003:webapps-frontend.clihdr[0009:]:

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-13 Thread Aleksandar Lazic


Dear Susheel Jalali.

Am 13-10-2015 22:20, schrieb Susheel Jalali:

Dear Aleks,


[snipp]


++
Tomcat’s web.xml

In web.xml, the context parameter is:


globalScope
default


and the filter mapping for the application is:

Product1Application
/*


++
Tomcat’s config.xml
++


1937
8443
no
5081
http
none
Product1
Product1
/Product1/


Sorry but this could not be the full web.xml and server.xml!

Tomcat will not be able to start with this files!

Du you try to proxy the RED5 server?!

https://github.com/Red5/red5-server

BR
Aleks

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-13 Thread Aleksandar Lazic




Am 13-10-2015 23:36, schrieb Aleksandar Lazic:

Dear Susheel Jalali.

Am 13-10-2015 22:20, schrieb Susheel Jalali:

Dear Aleks,


[snipp]


++
Tomcat’s web.xml

In web.xml, the context parameter is:


globalScope
default


and the filter mapping for the application is:

Product1Application
/*


++
Tomcat’s config.xml
++


1937
8443
no
5081
http
none
Product1
Product1
/Product1/


Sorry but this could not be the full web.xml and server.xml!

Tomcat will not be able to start with this files!

Du you try to proxy the RED5 server?!

https://github.com/Red5/red5-server


If yes what's in your red5.properties ?

https://github.com/Red5/red5-server/blob/master/src/main/server/conf/red5.properties

From my point of view it looks like you need to set the http.* variables 
to the right values.


https://github.com/Red5/red5-server/blob/3a6885433218ce2070b9064e195fc2ffde031c88/src/main/java/org/red5/server/net/servlet/RedirectHTTPServlet.java#L42

e. g.: https.port=1443 or what you want


BR
Aleks

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-12 Thread Susheel Jalali


Dear HAProxy Developer community,

We are seeking your inputs in the following issue we are facing:

We would like to access Product1 via URL: 
https://coscend.com:14443/Product1/ , wherein 14443 is port forwarded by 
router to 443 inside the server.


Output URL from the Product1 server should be: 
https://coscend.com:14443/Product1/signin?xyz


Current set up is giving: 
http://coscend.com:/Product1/signin?xyz


(1)  instead of 14443 (forwarded port).

(2) http, instead of https


@Aleks

Thank you for your insights.  It helped make some progress, but are 
short of conquering the final frontier.  Your guidance will be appreciated.



Setup you advised:  Client -> haproxy -> Product1 app server (Tomcat), 
which is talking HTTP in its standard deployment 
(http://IP:port/appname) and has no custom rewriting or SSL or other 
proxy (Apache or nginx).


As the content is dynamic (real-time video), we are not using Varnish.

Logs and config deployed for HAProxy 1.5.14 (stable)

We are not getting any errors in the logs, as the right Web page is 
being displayed.



frontend webapps-frontend
bind  *:80 name http
bind  *:443 name https ssl crt /path/to/server.pem

log   global
optionforwardfor
optionhttplog clf

reqadd X-Forwarded-Proto:\ https if { ssl_fc }
reqadd X-Forwarded-Proto:\ http if !{ ssl_fc }

acl host_httpsreq.hdr(Host) coscend.com:14443  # 14443 is due 
to port forwarding deployment

acl path_subdomain_p1 path_beg -i /Product1

use_backend subdomain_p1-backend if host_https path_subdomain_p1


backend subdomain_p1-backend
http-request set-header Host 
reqirep ^([^\ ]*)\ /Product1/*([^\ ]*)\ (.*)$   \1\ /Product1\2\ \3

acl hdr_location res.hdr(Location) -m found
rspirep ^Location:\ (https?://)([^/]*)(:[0-9]+)(/.*)$ Location:\ 
\1coscend.com\3\4 if hdr_location


server Product1.VM0  
cookie c check



Output of haproxy -f … -db -V
Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result FAILED
Total: 3 (2 usable), will use epoll.

Thank you.

Sincerely,

--
Susheel Jalali

Coscend Communications Solutions
Web site: www(DOT)Coscend(DOT)com
--
CONFIDENTIALITY NOTICE: See 'Confidentiality Notice Regarding E-mail 
Messages from Coscend Communications Solutions' posted at:

www(DOT)Coscend(DOT)com/Terms_and_Conditions.html


On 10/08/15 03:50, Aleksandar Lazic wrote:

Dear Susheel Jalali.

Am 07-10-2015 23:24, schrieb Susheel Jalali:

Dear Igor and Aleks,

Thank you for your insights.  Very useful to us, as we are implementing
HAProxy for the first time.  Below we have described how we have
implemented your advise and the result. Output of "haproxy -vv" is
given at end.

We have also provided the configuration file and relevant logs. We
would appreciate any insights to replace the internal IP address
occurring in server-response URL with the externally valid domain name
either by using the rewriting of Location and Host headers or the
complete URL, using %HP.

We would like to access Product1 via URL:
https://coscend.com:14443/Product1/
Output URL from the Product1 server should be:
https://coscend.com:14443/Product1/signin?xyz

what we are getting: http://Internal_IP:14443/Product1/signin?xyz

--
Responses to your insights / questions

@ Aleks:  Yes, Tomcat has a reverse proxy setting for our Product1.  
Can we

not have two reverse proxies to Product1?


Sorry I do not understand what you mean.

Have your read and understood the proxy-howto?

https://tomcat.apache.org/tomcat-8.0-doc/proxy-howto.html

I assume your setup is like this.

Client->haproxy->apache-mod_proxy_ajp->tomcat-X

Is this right?

Maybe you can omit apache-mod_proxy_ajp and talk http with tomcat.
Client->haproxy->tomcat-X HTTP Connector

In case that you want to deliver static content you should consider to 
use such a setup.


Client->haproxy->varnish->tomcat-X HTTP Connector
Client->haproxy->nginx+cache->tomcat-X HTTP Connector

Take care that you setup one of the *NIO* or *Apr* protocol handler
https://tomcat.apache.org/tomcat-8.0-doc/config/http.html



@Igor,

(1) As you rightly pointed out, we are getting http, not https

(2) As you advised, we moved these two lines from backend to frontend,
but did not find any change.

acl hdr_location res.hdr(Location) -m found
rspirep ^(Location:)\ (https?://([^/]*))/(.*)$\1\
http://\3/Product1/\4 if hdr_location



Maybe you can offer (gist,download,...) some debug output from

haproxy -f ... ... -db -V'


(3) Configuration file

global
log 127.0.0.1 local2
log-tag haproxy
chroot  /var/haproxy/lib
pidfile /var/run/haproxy.pid
userhaproxy
group   haproxy
nbproc  1
maxconn 5000
spread-checks 5
daemon
#debug
stats socket

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-12 Thread Aleksandar Lazic


Dear Susheel Jalali.

Am 12-10-2015 22:06, schrieb Susheel Jalali:

Dear HAProxy Developer community,

We are seeking your inputs in the following issue we are facing:

We would like to access Product1 via URL:
https://coscend.com:14443/Product1/ , wherein 14443 is port forwarded 
by

router to 443 inside the server.

Output URL from the Product1 server should be:
https://coscend.com:14443/Product1/signin?xyz

Current set up is giving:
http://coscend.com:/Product1/signin?xyz

(1)  instead of 14443 (forwarded port).

(2) http, instead of https

@Aleks

Thank you for your insights.  It helped make some progress, but are
short of conquering the final frontier.  Your guidance will be
appreciated.

Setup you advised:  Client -> haproxy -> Product1 app server (Tomcat),
which is talking HTTP in its standard deployment
(http://IP:port/appname) and has no custom rewriting or SSL or other
proxy (Apache or nginx).


Please can you also add the tomcat config and web xml.


As the content is dynamic (real-time video), we are not using Varnish.

Logs and config deployed for HAProxy 1.5.14 (stable)

We are not getting any errors in the logs, as the right Web page is
being displayed.

frontend webapps-frontend
bind  *:80 name http
bind  *:443 name https ssl crt /path/to/server.pem

log   global
optionforwardfor
optionhttplog clf

reqadd X-Forwarded-Proto:\ https if { ssl_fc }
reqadd X-Forwarded-Proto:\ http if !{ ssl_fc }

acl host_httpsreq.hdr(Host) coscend.com:14443  # 14443 is due 
to

port forwarding deployment
acl path_subdomain_p1 path_beg -i /Product1

use_backend subdomain_p1-backend if host_https path_subdomain_p1

backend subdomain_p1-backend
http-request set-header Host

reqirep ^([^\ ]*)\ /Product1/*([^\ ]*)\ (.*)$   \1\ 
/Product1\2\

\3


shouldn't be here the  in the replacment?


acl hdr_location res.hdr(Location) -m found
rspirep ^Location:\ (https?://)([^/]*)(:[0-9]+)(/.*)$ Location:\
\1coscend.com\3\4 if hdr_location


Maybe I misunderstood your request but shouldn't be in the replacement 
the port 14443?


rspirep ^Location:\ (https?://)([^/]*)(:[0-9]+)(/.*)$ Location:\ 
\1coscend.com:14443\4 if hdr_location


But still I think you should use

https://tomcat.apache.org/tomcat-8.0-doc/config/filter.html#Remote_IP_Filter
https://tomcat.apache.org/tomcat-8.0-doc/proxy-howto.html
https://tomcat.apache.org/tomcat-8.0-doc/config/http.html

proxyName
proxyPort
redirectPort


server Product1.VM0  
cookie

c check

Output of haproxy -f … -db -V
Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result FAILED
Total: 3 (2 usable), will use epoll.


Please can you post some output from the -db thanks.

BR Aleks


Thank you.

Sincerely,

 --
Susheel Jalali

Coscend Communications Solutions
Web site: www(DOT)Coscend(DOT)com
--
CONFIDENTIALITY NOTICE: See 'Confidentiality Notice Regarding E-mail
Messages from Coscend Communications Solutions' posted at:
www(DOT)Coscend(DOT)com/Terms_and_Conditions.html

On 10/08/15 03:50, Aleksandar Lazic wrote:


Dear Susheel Jalali.

Am 07-10-2015 23:24, schrieb Susheel Jalali:


Dear Igor and Aleks,

Thank you for your insights.  Very useful to us, as we are
implementing
HAProxy for the first time.  Below we have described how we have
implemented your advise and the result. Output of "haproxy -vv" is
given at end.

We have also provided the configuration file and relevant logs.  We
would appreciate any insights to replace the internal IP address
occurring in server-response URL with the externally valid domain
name
either by using the rewriting of Location and Host headers or the
complete URL, using %HP.

We would like to access Product1 via URL:
https://coscend.com:14443/Product1/
Output URL from the Product1 server should be:
https://coscend.com:14443/Product1/signin?xyz

what we are getting:   http://Internal_IP:14443/Product1/signin?xyz

--
Responses to your insights / questions

@ Aleks:  Yes, Tomcat has a reverse proxy setting for our Product1.
Can we
not have two reverse proxies to Product1?


Sorry I do not understand what you mean.

Have your read and understood the proxy-howto?

https://tomcat.apache.org/tomcat-8.0-doc/proxy-howto.html

I assume your setup is like this.

Client->haproxy->apache-mod_proxy_ajp->tomcat-X

Is this right?

Maybe you can omit apache-mod_proxy_ajp and talk http with tomcat.
Client->haproxy->tomcat-X HTTP Connector

In case that you want to deliver static content you should consider to
use such a setup.

Client->haproxy->varnish->tomcat-X HTTP Connector
Client->haproxy->nginx+cache->tomcat-X HTTP Connector

Take care that you setup one of the *NIO* or *Apr* protocol handler
https://tomcat.apache.org/tomcat-8.0-doc/config/http.html


@Igor,

(1) As you rightly pointed out, we are

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-07 Thread Aleksandar Lazic


Dear Susheel Jalali.

Am 07-10-2015 23:24, schrieb Susheel Jalali:

Dear Igor and Aleks,

Thank you for your insights.  Very useful to us, as we are implementing
HAProxy for the first time.  Below we have described how we have
implemented your advise and the result. Output of "haproxy -vv" is
given at end.

We have also provided the configuration file and relevant logs.  We
would appreciate any insights to replace the internal IP address
occurring in server-response URL with the externally valid domain name
either by using the rewriting of Location and Host headers or the
complete URL, using %HP.

We would like to access Product1 via URL:
https://coscend.com:14443/Product1/
Output URL from the Product1 server should be:
https://coscend.com:14443/Product1/signin?xyz

what we are getting:   http://Internal_IP:14443/Product1/signin?xyz

--
Responses to your insights / questions

@ Aleks:  Yes, Tomcat has a reverse proxy setting for our Product1.  
Can we

not have two reverse proxies to Product1?


Sorry I do not understand what you mean.

Have your read and understood the proxy-howto?

https://tomcat.apache.org/tomcat-8.0-doc/proxy-howto.html

I assume your setup is like this.

Client->haproxy->apache-mod_proxy_ajp->tomcat-X

Is this right?

Maybe you can omit apache-mod_proxy_ajp and talk http with tomcat.
Client->haproxy->tomcat-X HTTP Connector

In case that you want to deliver static content you should consider to 
use such a setup.


Client->haproxy->varnish->tomcat-X HTTP Connector
Client->haproxy->nginx+cache->tomcat-X HTTP Connector

Take care that you setup one of the *NIO* or *Apr* protocol handler
https://tomcat.apache.org/tomcat-8.0-doc/config/http.html



@Igor,

(1) As you rightly pointed out, we are getting http, not https

(2) As you advised, we moved these two lines from backend to frontend,
but did not find any change.

acl hdr_location res.hdr(Location) -m found
rspirep ^(Location:)\ (https?://([^/]*))/(.*)$\1\
http://\3/Product1/\4 if hdr_location



Maybe you can offer (gist,download,...) some debug output from

haproxy -f ... ... -db -V'


(3) Configuration file

global
log 127.0.0.1 local2
log-tag haproxy
chroot  /var/haproxy/lib
pidfile /var/run/haproxy.pid
userhaproxy
group   haproxy
nbproc  1
maxconn 5000
spread-checks 5
daemon
#debug
stats socket  /var/haproxy/lib/stats

##
#   SSL section
##
maxsslconn 256
tune.ssl.default-dh-param 4096
ca-base /path/to/directory/of/server.pem


#-
# Defaults
#-
defaults
modehttp
log global
option  httplog
option  forwardfor
option  abortonclose
option  http-server-close
option  redispatch
retries 3
timeout queue   10s
timeout client  5ms
timeout server  5ms
timeout connect 5000ms
timeout http-keep-alive 10s
timeout http-request5s
timeout check   10s
maxconn 5

frontend webapps-frontend
bind  *:80 name http
bind  *:443 name https ssl crt /path/to/server.pem


How about to change to two frontends

###
frontend http-frontend
 bind  *:80 name http
 ... other frontend settings

frontend https-frontend
bind  *:443 name https ssl crt /path/to/server.pem
... other frontend settings
###

Then you can setup the tomcat connector for https to

secure="true"

Please take a look at this howto.
https://tomcat.apache.org/tomcat-8.0-doc/ssl-howto.html

I prefer to setup the appserver to work as expected, when it's possible, 
and not to do some 'magic' with the rewrites on any proxy ;-)


[snipp]


===

Output of "haproxy -vv":

HA-Proxy version 1.5.14 2015/07/02
Copyright 2000-2015 Willy Tarreau 

Build options :
  TARGET  = linux2628
  CPU = native
  CC  = gcc
  CFLAGS  = -m64 -march=x86-64 -O2 -march=native -g 
-fno-strict-aliasing

  OPTIONS = USE_CTTPROXY=1 USE_LIBCRYPT=1 USE_CRYPT_H=1
USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 USE_PCRE=1
USE_PCRE_JIT=1 USE_TFO=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 
200


Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.7
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Built with transparent proxy support using: CTTPROXY IP_TRANSPARENT

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-07 Thread Susheel Jalali


Dear Igor and Aleks,

Thank you for your insights.  Very useful to us, as we are implementing
HAProxy for the first time.  Below we have described how we have 
implemented your advise and the result. Output of "haproxy -vv" is given 
at end.


We have also provided the configuration file and relevant logs.  We 
would appreciate any insights to replace the internal IP address 
occurring in server-response URL with the externally valid domain name 
either by using the rewriting of Location and Host headers or the 
complete URL, using %HP.


We would like to access Product1 via URL:
https://coscend.com:14443/Product1/
Output URL from the Product1 server should be:
https://coscend.com:14443/Product1/signin?xyz

what we are getting:   http://Internal_IP:14443/Product1/signin?xyz

--
Responses to your insights / questions

@ Aleks:  Yes, Tomcat has a reverse proxy setting for our Product1.  Can we
not have two reverse proxies to Product1?

@Igor,

(1) As you rightly pointed out, we are getting http, not https

(2) As you advised, we moved these two lines from backend to frontend, 
but did not find any change.


acl hdr_location res.hdr(Location) -m found
rspirep ^(Location:)\ (https?://([^/]*))/(.*)$\1\
http://\3/Product1/\4 if hdr_location

(3) Configuration file

global
log 127.0.0.1 local2
log-tag haproxy
chroot  /var/haproxy/lib
pidfile /var/run/haproxy.pid
userhaproxy
group   haproxy
nbproc  1
maxconn 5000
spread-checks 5
daemon
#debug
stats socket  /var/haproxy/lib/stats

##
#   SSL section
##
maxsslconn 256
tune.ssl.default-dh-param 4096
ca-base /path/to/directory/of/server.pem


#-
# Defaults
#-
defaults
modehttp
log global
option  httplog
option  forwardfor
option  abortonclose
option  http-server-close
option  redispatch
retries 3
timeout queue   10s
timeout client  5ms
timeout server  5ms
timeout connect 5000ms
timeout http-keep-alive 10s
timeout http-request5s
timeout check   10s
maxconn 5

frontend webapps-frontend
bind  *:80 name http
bind  *:443 name https ssl crt /path/to/server.pem

log   global
optionforwardfor
optionhttplog clf

reqadd X-Forwarded-Proto:\ https if { ssl_fc }
reqadd X-Forwarded-Proto:\ http if !{ ssl_fc }
#http-request add-header X-Forwarded-Proto:\ https if { ssl_fc }  #
Don't know how to use it instead of reqadd
#http-request add-header X-Forwarded-Proto:\ http if !{ ssl_fc }   #
Don't know how to use it instead of reqadd

acl host_httpsreq.hdr(Host) coscend.com:14443  # 14443 is due to
port forwarding deployment
acl path_subdomain_p1 path_beg -i /Product1

use_backend subdomain_p1-backend if host_https path_subdomain_p1

backend subdomain_p1-backend
http-request set-header Host 
reqirep ^([^\ ]*)\ /Product1/?([^\ ]*)\ (.*)$   \1\ /Product1\2\ \3

acl hdr_location res.hdr(Location) -m found
#http-response replace-header Host (.*) %%HP if hdr_location   # 
This is

not working
rspirep ^(Location:)\ (https?://([^/]*))/(.*)$\1\
http://\3/Product1/\4 if hdr_location

server Product1.VM0  cookie c check

listen stats 10.10.10.51:8885
stats enable
stats auth[username]:[password]
stats hide-version
stats show-node
stats uri /stats
stats realm Haproxy\ Statistics
monitor-uri /monitor
stats refresh 10s
stats show-legends

Notice.log
Oct  7 15:42:34 localhost haproxy[12886]: Proxy webapps-frontend started.
Oct  7 15:42:34 localhost haproxy[12886]: Proxy webapps-frontend started.
Oct  7 15:42:34 localhost haproxy[12886]: Proxy webapps-backend started.
Oct  7 15:42:34 localhost haproxy[12886]: Proxy webapps-backend started..
Oct  7 15:42:34 localhost haproxy[12886]: Proxy subdomain_p1-backend
started.
Oct  7 15:42:34 localhost haproxy[12886]: Proxy Test-stats started.


Info.log
Oct  7 15:42:44 localhost haproxy[12887]: 192.168.100.153:58163
[07/Oct/2015:15:42:44.455] Test-stats Test-stats/ 12/0/0/0/12 200
29869 - - LR-- 1/1/0/0/0 0/0 "GET /stats HTTP/1.1"
Oct  7 15:42:44 localhost haproxy[12887]: 192.168.100.153:58163
[07/Oct/2015:15:42:44.467] Test-stats Test-stats/ 154/-1/-1/-1/154
503 213 - - SC-- 0/0/0/0/0 0/0 "GET /favicon.ico HTTP/1.1"
Oct  7 15:42:54 localhost haproxy[12887]: 192.168.100.153:58164
[07/Oct/2015:15:42:54.571] Test-stats Test-stats/ 7/0/0/0/7 200 29930
- - LR-- 1/1/0/0/0 0/0 "GET /stats HTTP/1.1"
Oct  7 15:42:54 localhost haproxy[12887]: 192.168.100.153:58164
[07/Oct/2015:15:42:54.579] Test-stats Test-stats/

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-07 Thread Aleksandar Lazic


Hi Susheel Jalali.

please can you show us the out put of haproxy -vv

Am 06-10-2015 22:06, schrieb Susheel Jalali:

Dear HAProxy Developers,

After incorporating insights from Bryan Talbot and articles from
Baptiste Assman on HAProxy Web site, we have been able to get the basic
configuration of HAProxy going.  Now we are adding configuration to
access specific products in our LAN.

We would like to access Product1 via URL:
https://coscend.com:14443/Product1/

Output URL from the Product1 server should be:
https://coscend.com:14443/Product1/signin?xyz

What we are getting:   https://Internal_IP:14443/Product1/signin?xyz


Have the product a setting for 'proxy' setting like tomcat?

http://tomcat.apache.org/tomcat-8.0-doc/proxy-howto.html

or something like 'public' or 'external' URL Option or Setting?



The server presents the right page, but with internal IP address of the
server.  Hence, the product can only be accessed from internal LAN, not
from WAN.  What are we missing?


[snipp]

As Igor mentioned in the his mail do you have any Warning Messages at 
Startup?


Cheers Aleks

HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-06 Thread Susheel Jalali


Dear HAProxy Developers,

After incorporating insights from Bryan Talbot and articles from 
Baptiste Assman on HAProxy Web site, we have been able to get the basic 
configuration of HAProxy going.  Now we are adding configuration to 
access specific products in our LAN.


We would like to access Product1 via URL: 
https://coscend.com:14443/Product1/


Output URL from the Product1 server should be: 
https://coscend.com:14443/Product1/signin?xyz


What we are getting:  https://Internal_IP:14443/Product1/signin?xyz

The server presents the right page, but with internal IP address of the 
server.  Hence, the product can only be accessed from internal LAN, not 
from WAN.  What are we missing?


Below is the configuration deployed.

global

[…]

default

[…]

frontend webapps-frontend

bind  *:80 name http

bind *:443 name https ssl crt /path/to/server.pem

log   global

optionforwardfor

option httplog clf

reqadd X-Forwarded-Proto:\ https if { ssl_fc }

reqadd X-Forwarded-Proto:\ http if !{ ssl_fc }

#http-request add-header X-Forwarded-Proto:\ https if { ssl_fc }  
#  Don't know how to use it instead of reqadd


#http-request add-header X-Forwarded-Proto:\ http if !{ ssl_fc }   #  
Don't know how to use it instead of reqadd


acl host_httpsreq.hdr(Host) coscend.com:14443  # 14443 is due to 
port forwarding deployment


acl path_subdomain_p1 path_beg -i /Product1

use_backend subdomain_p1-backend if host_https path_subdomain_p1

backend subdomain_p1-backend

http-request set-header Host 

reqirep ^([^\ ]*)\ /Product1/?([^\ ]*)\ (.*)$   \1\ /Product1\2\ \3

acl hdr_location res.hdr(Location) -m found

#http-response replace-header Host (.*) %%HP if hdr_location   # This is 
not working


rspirep ^(Location:)\ (https?://([^/]*))/(.*)$\1\ 
http://\3/Product1/\4 if hdr_location


server Product1.VM0  cookie c check


Thank you.

--

Sincerely,

Susheel Jalali

Coscend Communications Solutions

Elite Premio Complex Suite 200,  Pune 411045 Maharashtra India
susheel.jal...@coscend.com

Web site: www.Coscend.com
--

CONFIDENTIALITY NOTICE: See 'Confidentiality Notice Regarding E-mail 
Messages from Coscend Communications Solutions' posted at: 
http://www.Coscend.com/Terms_and_Conditions.html

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

2015-10-06 Thread Igor Cicimov

On Wed, Oct 7, 2015 at 7:06 AM, Susheel Jalali 
wrote:

> Dear HAProxy Developers,
>
> After incorporating insights from Bryan Talbot and articles from Baptiste
> Assman on HAProxy Web site, we have been able to get the basic
> configuration of HAProxy going.  Now we are adding configuration to access
> specific products in our LAN.
>
> We would like to access Product1 via URL:
> https://coscend.com:14443/Product1/
>
> Output URL from the Product1 server should be:
> https://coscend.com:14443/Product1/signin?xyz
>
> What we are getting:   https://Internal_IP:14443/Product1/signin?xyz
>
> The server presents the right page, but with internal IP address of the
> server.  Hence, the product can only be accessed from internal LAN, not
> from WAN.  What are we missing?
>
> Below is the configuration deployed.
>
> global
>
> […]
>
> default
>
> […]
>
>
>
> frontend webapps-frontend
>
> bind  *:80 name http
>
> bind  *:443 name https ssl crt /path/to/server.pem
>
>
>
> log   global
>
> optionforwardfor
>
> optionhttplog clf
>
>
>
> reqadd X-Forwarded-Proto:\ https if { ssl_fc }
>
> reqadd X-Forwarded-Proto:\ http if !{ ssl_fc }
>
> #http-request add-header X-Forwarded-Proto:\ https if { ssl_fc }  #
> Don't know how to use it instead of reqadd
>
> #http-request add-header X-Forwarded-Proto:\ http if !{ ssl_fc }   #
> Don't know how to use it instead of reqadd
>
>
>
> acl host_httpsreq.hdr(Host) coscend.com:14443  # 14443 is due to
> port forwarding deployment
>
> acl path_subdomain_p1 path_beg -i /Product1
>
>
>
> use_backend subdomain_p1-backend if host_https path_subdomain_p1
>
>
>
> backend subdomain_p1-backend
>
> http-request set-header Host 
>
> reqirep ^([^\ ]*)\ /Product1/?([^\ ]*)\ (.*)$   \1\ /Product1\2\ \3
>
>
>
> acl hdr_location res.hdr(Location) -m found
>
> #http-response replace-header Host (.*) %%HP if hdr_location   # This
> is not working
>
> rspirep ^(Location:)\ (https?://([^/]*))/(.*)$\1\
> http://\3/Product1/\4 if hdr_location
>


What happens if you move these two from the backend into the frontend
section (I believe that's where they belong):

acl hdr_location res.hdr(Location) -m found
rspirep ^(Location:)\ (https?://([^/]*))/(.*)$\1\
http://\3/Product1/\4 if hdr_location

Also in the rspirep you are rewriting https to http but you say the
response you are seeing is still with https:
https://Internal_IP:14443/Product1/signin?xyz
which most probably means that condition is not working for sure.

In case you are serving a single domain maybe simplifying this to begin
with may help:

rspirep ^(Location:)\ https?://[^/]*/(.*)$\1\
http://coscend.com/Product1/\2  if hdr_location

Also any messages during haproxy startup or in the haproxy log indicating
possible issues? Something along the lines of "this and this statement will
never match due to bla bla".


>
>
> server Product1.VM0  cookie c check
>
>
>
> Thank you.
>
> --
>
> Sincerely,
>
> Susheel Jalali
>
> Coscend Communications Solutions
>
> Elite Premio Complex Suite 200,  Pune 411045 Maharashtra India
> susheel.jal...@coscend.com
>
> Web site: www.Coscend.com
> --
>
> CONFIDENTIALITY NOTICE: See 'Confidentiality Notice Regarding E-mail
> Messages from Coscend Communications Solutions' posted at:
> http://www.Coscend.com/Terms_and_Conditions.html
>
>


-- 
Igor Cicimov | DevOps


p. +61 (0) 433 078 728
e. ig...@encompasscorporation.com 
w*.* encompasscorporation.com
a. Level 4, 65 York Street, Sydney 2000

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

HTTP Response Rewriting to Replace Internal IP with FQDN

Re: HTTP Response Rewriting to Replace Internal IP with FQDN

12 matches

Site Navigation

Mail list logo

Footer information