Re: ssl offloading and send-proxy-v2-ssl

[Arnall]

Le 27/12/2016 à 00:35, Patrick Hemmer a écrit :



On 2016/12/23 09:28, Arnall wrote:

Hi everyone,

i'm using a nbproc > 1 configuration for ssl offloading :

listen web_tls
mode http
bind *:443 ssl crt whatever.pem process 2
bind *:443 ssl crt whatever.pem process 3

../..
server web_plain u...@plain.sock send-proxy-v2-ssl

frontend web_plain
bind*:80 process 1
bind u...@plain.sock process 1 accept-proxy

../..

And i'm looking for a secure solution in the web_plain frontend to 
know if the request come from web_tls or not ( in fact i want to know 
if the connection was initially made via SSL/TLS transport ).


I though that send-proxy-v2-ssl could help but i have no idea how ... 
src and src_port are OK with the proxy protocol but ssl_fc in 
web_plain keeps answering false  ( 0 ) even the request come from 
web_tls.


I could set and forward a secret header set in web_tls but i don't 
like the idea ... (have to change the header each time an admin sys 
leave the enterprise... )


Thanks.





This use case has come up a few times: 
https://www.mail-archive.com/haproxy@formilux.org/msg23882.html
My crude solution is an ACL check on the port the client connected to 
(dst_port eq 443).


-Patrick


Thanks for the answer and happy new year !

Re: ssl offloading and send-proxy-v2-ssl

[Arnall]

Hi,

thanks for your answer, didn't know the src_is_local feature as it's a 
1.7 feature, we're still in 1.6.


the dst_port seems ok to me, will use it !

Happy new year !

Le 27/12/2016 à 08:29, Elias Abacioglu a écrit :

Sorry just realized,
src_is_local won't work when using proxy protocol.
Proxy protocol will preserve initial source information.

You can probably use dst_port like this instead:

acl secure dst_port  443
 if is secure

On Mon, Dec 26, 2016 at 11:09 PM, Elias Abacioglu 
<elias.abacio...@deltaprojects.com 
<mailto:elias.abacio...@deltaprojects.com>> wrote:


Perhaps you could use src_is_local.

Something like this

frontend web_plain

acl is_local src_is_local
http-response add-header X-External-Protocol https if is_local


/Elias

On Fri, Dec 23, 2016 at 3:28 PM, Arnall <arnall2...@gmail.com
<mailto:arnall2...@gmail.com>> wrote:

Hi everyone,

i'm using a nbproc > 1 configuration for ssl offloading :

listen web_tls
mode http
bind *:443 ssl crt whatever.pem process 2
bind *:443 ssl crt whatever.pem process 3

../..
server web_plain u...@plain.sock send-proxy-v2-ssl

frontend web_plain
bind*:80 process 1
bind u...@plain.sock process 1 accept-proxy

../..

And i'm looking for a secure solution in the web_plain
frontend to know if the request come from web_tls or not ( in
fact i want to know if the connection was initially made via
SSL/TLS transport ).

I though that send-proxy-v2-ssl could help but i have no idea
how ... src and src_port are OK with the proxy protocol but
ssl_fc in web_plain keeps answering false  ( 0 ) even the
request come from web_tls.

I could set and forward a secret header set in web_tls but i
don't like the idea ... (have to change the header each time
an admin sys leave the enterprise... )

Thanks.

Re: ssl offloading and send-proxy-v2-ssl

[Willy Tarreau]

Hi Patrick,

On Mon, Dec 26, 2016 at 11:35:51PM +, Patrick Hemmer wrote:
> On 2016/12/23 09:28, Arnall wrote:
> > I though that send-proxy-v2-ssl could help but i have no idea how ...
> > src and src_port are OK with the proxy protocol but ssl_fc in
> > web_plain keeps answering false  ( 0 ) even the request come from
> > web_tls.
> 
> This use case has come up a few times:
> https://www.mail-archive.com/haproxy@formilux.org/msg23882.html
> My crude solution is an ACL check on the port the client connected to
> (dst_port eq 443).

I think for next version we need to work a bit more on how we deal with
connections received using the proxy protocol. While we can emit *some*
information, we only use the family/address/port of what we receive and
that's a bit limited regarding the amount of information we can extract
from a regular connection.

I think we'll need to be able to keep at least :
  - the transport layer protocol (SSL/TLS version)
  - the transport layer ciphers
  - the transport layer authority (ie SNI for TLS)
  - the application layer protocol (NPN/ALPN)

Probably that we'll have to store more info (cf Bertrand's proxy-addr
patch set) and that we'll want to have a dynamic proxy-protocol part in
the connection to store all this. But anyway that's definitely something
we need to think about so that someone can work on it.

Cheers,
Willy

Re: ssl offloading and send-proxy-v2-ssl

[Elias Abacioglu]

Sorry just realized,
src_is_local won't work when using proxy protocol.
Proxy protocol will preserve initial source information.

You can probably use dst_port like this instead:

acl secure dst_port  443
 if is secure

On Mon, Dec 26, 2016 at 11:09 PM, Elias Abacioglu <
elias.abacio...@deltaprojects.com> wrote:

> Perhaps you could use src_is_local.
>
> Something like this
>
> frontend web_plain
> 
> acl is_local src_is_local
> http-response add-header X-External-Protocol https if is_local
>
>
> /Elias
>
> On Fri, Dec 23, 2016 at 3:28 PM, Arnall <arnall2...@gmail.com> wrote:
>
>> Hi everyone,
>>
>> i'm using a nbproc > 1 configuration for ssl offloading :
>>
>> listen web_tls
>> mode http
>> bind *:443 ssl crt whatever.pem process 2
>> bind *:443 ssl crt whatever.pem process 3
>>
>> ../..
>> server web_plain u...@plain.sock send-proxy-v2-ssl
>>
>> frontend web_plain
>> bind*:80 process 1
>> bind u...@plain.sock process 1 accept-proxy
>>
>> ../..
>>
>> And i'm looking for a secure solution in the web_plain frontend to know
>> if the request come from web_tls or not ( in fact i want to know if the
>> connection was initially made via SSL/TLS transport ).
>>
>> I though that send-proxy-v2-ssl could help but i have no idea how ... src
>> and src_port are OK with the proxy protocol but ssl_fc in web_plain keeps
>> answering false  ( 0 ) even the request come from web_tls.
>>
>> I could set and forward a secret header set in web_tls but i don't like
>> the idea ... (have to change the header each time an admin sys leave the
>> enterprise... )
>>
>> Thanks.
>>
>>
>>
>>
>

Re: ssl offloading and send-proxy-v2-ssl

[Patrick Hemmer]

On 2016/12/23 09:28, Arnall wrote:
> Hi everyone,
>
> i'm using a nbproc > 1 configuration for ssl offloading :
>
> listen web_tls
> mode http
> bind *:443 ssl crt whatever.pem process 2
> bind *:443 ssl crt whatever.pem process 3
>
> ../..
> server web_plain u...@plain.sock send-proxy-v2-ssl
>
> frontend web_plain
> bind*:80 process 1
> bind u...@plain.sock process 1 accept-proxy
>
> ../..
>
> And i'm looking for a secure solution in the web_plain frontend to
> know if the request come from web_tls or not ( in fact i want to know
> if the connection was initially made via SSL/TLS transport ).
>
> I though that send-proxy-v2-ssl could help but i have no idea how ...
> src and src_port are OK with the proxy protocol but ssl_fc in
> web_plain keeps answering false  ( 0 ) even the request come from
> web_tls.
>
> I could set and forward a secret header set in web_tls but i don't
> like the idea ... (have to change the header each time an admin sys
> leave the enterprise... )
>
> Thanks.
>
>
>

This use case has come up a few times:
https://www.mail-archive.com/haproxy@formilux.org/msg23882.html
My crude solution is an ACL check on the port the client connected to
(dst_port eq 443).

-Patrick

Re: ssl offloading and send-proxy-v2-ssl

[Elias Abacioglu]

Perhaps you could use src_is_local.

Something like this

frontend web_plain

acl is_local src_is_local
http-response add-header X-External-Protocol https if is_local


/Elias

On Fri, Dec 23, 2016 at 3:28 PM, Arnall <arnall2...@gmail.com> wrote:

> Hi everyone,
>
> i'm using a nbproc > 1 configuration for ssl offloading :
>
> listen web_tls
> mode http
> bind *:443 ssl crt whatever.pem process 2
> bind *:443 ssl crt whatever.pem process 3
>
> ../..
> server web_plain u...@plain.sock send-proxy-v2-ssl
>
> frontend web_plain
> bind*:80 process 1
> bind u...@plain.sock process 1 accept-proxy
>
> ../..
>
> And i'm looking for a secure solution in the web_plain frontend to know if
> the request come from web_tls or not ( in fact i want to know if the
> connection was initially made via SSL/TLS transport ).
>
> I though that send-proxy-v2-ssl could help but i have no idea how ... src
> and src_port are OK with the proxy protocol but ssl_fc in web_plain keeps
> answering false  ( 0 ) even the request come from web_tls.
>
> I could set and forward a secret header set in web_tls but i don't like
> the idea ... (have to change the header each time an admin sys leave the
> enterprise... )
>
> Thanks.
>
>
>
>

ssl offloading and send-proxy-v2-ssl

[Arnall]

Hi everyone,

i'm using a nbproc > 1 configuration for ssl offloading :

listen web_tls
mode http
bind *:443 ssl crt whatever.pem process 2
bind *:443 ssl crt whatever.pem process 3

../..
server web_plain u...@plain.sock send-proxy-v2-ssl

frontend web_plain
bind*:80 process 1
bind u...@plain.sock process 1 accept-proxy

../..

And i'm looking for a secure solution in the web_plain frontend to know 
if the request come from web_tls or not ( in fact i want to know if the 
connection was initially made via SSL/TLS transport ).


I though that send-proxy-v2-ssl could help but i have no idea how ... 
src and src_port are OK with the proxy protocol but ssl_fc in web_plain 
keeps answering false  ( 0 ) even the request come from web_tls.


I could set and forward a secret header set in web_tls but i don't like 
the idea ... (have to change the header each time an admin sys leave the 
enterprise... )


Thanks.

RE: tcpdump and Haproxy SSL Offloading

[mlist]

Hi,

thanks very much, I went a little deeper about Cipher Suite.

Now I changed the ciphers list supported by our haproxy LBs and

increased the security level (always trying to keep ciphers that support old

clients that still use our services - like XP/IE8).



I tried successfully Decrypting a trace file with Wireshark, taken with 
tcpdump. Setting NOT-Ephemeral

Ciphers on haproxy LBs (temporary disabling DHE, EDH chippers on haproxy LBs).

At least for the most part the trace file was Decrypted an shown correctly, but 
we see some parts that appear

as TCP or TLSv1.2 or SSL type “XXX Segment of a reassembled PDU”:



[cid:image002.png@01D1C1A9.2BA67CA0]



apparently of only few bytes, but if we point on the SSL/TLS layer field 
“Encrypted Application Data” we see lot of data.



[cid:image003.png@01D1C1AB.04BE6510]

We can see also 3 tabs for that line:  Frame, Reasembled TCP and Decrypted SSL 
Data, so we can see Decrypted Data,

and also show as HTTP if we Follow TCP traffic.





[cid:image001.png@01D1C1AC.AE248660]



I discovered that in Wireshark TCP protocol preference “Allow Subdissector to 
reassemble TCP streams” allow me to see

Decrypted Traffic, do you know about some other Wireshark configuration about 
these SSL / TCP reassembled PDU so

these can be seen differently in the Wireshark ?





As soon as possible I'll try the Ephemeral case Decryption using client session 
keys (as Igor suggested),

also if it is more difficult as I see there are different ssl handshake so it 
is not so clear

if Browser append or overwrite session key files and if it is simple to put in 
relationship

and to correctly analyze with wireshark these different SSL/HTTP stream, I 
think.



Probably in HAproxy it could be implemented a kind of virtual network interface 
mechanism that tcpdump could connect

to retrieve the Decrypted Traffic, so achieve less dependency from external 
tools and less original environment “contamination / manipulation”.



For haproxy "option  http-ignore-probes" I think this is a solution to test 
before, to evaluate what client see.

In my experience for Browsers that have problems correctly managing pre-connect 
and graceful TCP session closing,

not emitting 408 can totally hide problems that need to be analyzer and 
explained to avoid strange and hidden behavior.

Also if 408 is not an infrastructure problem, the customer perception can be 
different... I know also that these 408/400

can distort the statistics, but is not a simple choice.



Roberto



-Original Message-
From: Lukas Tribus [mailto:lu...@gmx.net]
Sent: domenica 5 giugno 2016 12.16
To: Igor Cicimov <ig...@encompasscorporation.com>; mlist <ml...@apsystems.it>
Cc: HAProxy <haproxy@formilux.org>
Subject: Re: tcpdump and Haproxy SSL Offloading



Hi,





Am 05.06.2016 um 02:19 schrieb Igor Cicimov:

>

> > In haproxy.cfg I used these cipher I found recommended:

> > ciphers ECDHE-RSA-AES256-SHA:RC4-SHA:RC4:HIGH:!MD5:!aNULL:!EDH:!AESGCM

>



I would not recommend this. Check [1] and [2] for some uptodate

recommendations.



Yes, removing ECDHE-RSA-AES256-SHA will force the server to use the

non-FS RC4 cipher.



Regarding the 408 problem, please have a look at the http-ignore-probes

option [3].







Regards,



Lukas







[1] https://wiki.mozilla.org/Security/Server_Side_TLS

[2]

https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy-1.6=1.0.2=no=intermediate

[3]

http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#4-option%20http-ignore-probes







--

Il messaggio e' stato analizzato alla ricerca di virus o

contenuti pericolosi da MailScanner, ed e'

risultato non infetto.

Haproxy with mixed modes: SSL offloading and passthrough

[DjamOlsky]

Hi,
I would like to implement the following simple scenario, all the tests I
have done unfortunately have failed (bad SSL requests, error 502...):
- Using SSL passthrough for the top level domain (mydomain.com) as the web
application is already presenting a wildcard SSL certificate for this domain.
- Offloading SSL on haproxy just for the subdomains (a SSL certificate
matching with *.sub01.mydomain.com for example).

Here is a simple configuration example showing the case:

frontend FE-https
bind 1.1.1.1:443 ssl crt /etc/ssl/certs/sub01.mydomain.com.pem
modehttp
acl is_sub01 hdr_end(host) -i sub01.mydomain.com
use_backend BE-sub01 if is_sub01
default_backend BE-default

# SSL Offloading for *.sub01.mydomain.com
backend BE-sub01
modehttp
server  srv1 10.10.10.10:443 

# SSL passthrough for *.mydomain.com
backend BE-default
modehttp
server  srv1 10.10.10.10:443 ssl verify none

To summarize, I would like to provide or not the right SSL certificate per
backend (depending on the url).
Thanks for your help :)

Re: tcpdump and Haproxy SSL Offloading

[Lukas Tribus]

Hi,


Am 05.06.2016 um 02:19 schrieb Igor Cicimov:


> In haproxy.cfg I used these cipher I found recommended:
> ciphers ECDHE-RSA-AES256-SHA:RC4-SHA:RC4:HIGH:!MD5:!aNULL:!EDH:!AESGCM



I would not recommend this. Check [1] and [2] for some uptodate 
recommendations.


Yes, removing ECDHE-RSA-AES256-SHA will force the server to use the 
non-FS RC4 cipher.


Regarding the 408 problem, please have a look at the http-ignore-probes 
option [3].




Regards,

Lukas



[1] https://wiki.mozilla.org/Security/Server_Side_TLS
[2] 
https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy-1.6=1.0.2=no=intermediate
[3] 
http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#4-option%20http-ignore-probes

RE: tcpdump and Haproxy SSL Offloading

[Igor Cicimov]

;TLS_RSA_WITH_3DES_EDE_CBC_SHA
(0xa)
112
>
>
>
> In haproxy.cfg I used these cipher I found recommended:
>
>
>
> ciphers ECDHE-RSA-AES256-SHA:RC4-SHA:RC4:HIGH:!MD5:!aNULL:!EDH:!AESGCM
>

Just remove ECDHE-RSA-AES256-SHA. Note that in your case this will leave
you with rc4 cyphers only which are not secure so dont run this setup for
long time.

>
>
> but I do not understand well the mapping with those recognized by test
(FS = I think are Forward Secrecy) an these configured and how to find if
they support or not PFS. Your help will be appreciated, so I can change
haproxy configuration to quickly disable/enable cipher so I can use
certificates private keys that means a simple trace analysis, I need to
have more control on chiper management.
>
>
>
>
>
>
>
>
>
> Roberto
>
>
>
>
>
> From: Igor Cicimov [mailto:ig...@encompasscorporation.com]
> Sent: sabato 4 giugno 2016 02.14
> To: Lukas Tribus <lu...@gmx.net>
> Cc: mlist <ml...@apsystems.it>; haproxy@formilux.org
>
> Subject: Re: tcpdump and Haproxy SSL Offloading
>
>
>
> Hi Lukas,
>
>
>
> On Sat, Jun 4, 2016 at 3:03 AM, Lukas Tribus <lu...@gmx.net> wrote:
>>
>> Hello,
>>
>>
>> you can dump the symmetric keys from the browser and import them in
wireshark to decrypt PFS protected TLS sessions [1]
>
>
>
> Yes in case you want to troubleshoot something generic this is a good
approach but if you want to troubleshoot sessions not initiated by your
self, ie particular client connection, this is practically impossible.
>
>>
>> or downgrade your ciphers settings to non-PF ciphers. Properly
decrypting the TLS session is the only way to really make sure you see what
happens, even if there is a TLS related bug in the client or server
(haproxy).
>>
>>
>>
>> Some other idea's are:
>>
>> - if your backend traffic is unencrypted, you may want to capture the
traffic there.
>
>
>
> Not practical though if you have tens of backend servers. Much better if
you have to troubleshoot on 2 instead of dozen of servers. At least that's
how I understand the question related to tcpdump usage on the haproxy
servers them self.
>
>>
>>
>> - if haproxy is rejecting the request, check "show errors" on the admin
socket.
>>
>>
>>
>> As you said, the best solution is to not depend on haproxy specific
features, as you don't want to modify existing infrastructure in a
troubleshooting case.Maybe something
>
>
>
> Outside haproxy maybe something like mitmproxy or sslstrip might help.
Not sure though have never used them myself.
>
>
>>
>>
>> Another proxy layer means that you decrypt TLS on the front-end proxy,
while you sniff the plaintext traffic between the front-end and the second
tier proxy. You can probably do this with a single haproxy instance
recirculating the traffic through a unix socket and capture the traffic on
it, but it would require some trial and error and definitely some testing.
>
>
>
> This will probably be faster but can't use tcpdump in that case.
>
>>
>>
>> I believe the SSLKEYLOGFILE approach [1] to be the most efficient and
simplest approach.
>>
>>
>> cheers,
>>
>> lukas
>>
>>
>> [1]
https://jimshaver.net/2015/02/11/decrypting-tls-browser-traffic-with-wireshark-the-easy-way/
>>
>
>
>
> --
> Il messaggio e' stato analizzato alla ricerca di virus o
> contenuti pericolosi da MailScanner, ed e'
> risultato non infetto.

RE: tcpdump and Haproxy SSL Offloading

[mlist]

)  
 112

In haproxy.cfg I used these cipher I found recommended:

ciphers ECDHE-RSA-AES256-SHA:RC4-SHA:RC4:HIGH:!MD5:!aNULL:!EDH:!AESGCM

but I do not understand well the mapping with those recognized by test (FS = I 
think are Forward Secrecy) an these configured and how to find if they support 
or not PFS. Your help will be appreciated, so I can change haproxy 
configuration to quickly disable/enable cipher so I can use certificates 
private keys that means a simple trace analysis, I need to have more control on 
chiper management.




Roberto


From: Igor Cicimov [mailto:ig...@encompasscorporation.com]
Sent: sabato 4 giugno 2016 02.14
To: Lukas Tribus <lu...@gmx.net>
Cc: mlist <ml...@apsystems.it>; haproxy@formilux.org
Subject: Re: tcpdump and Haproxy SSL Offloading

Hi Lukas,

On Sat, Jun 4, 2016 at 3:03 AM, Lukas Tribus 
<lu...@gmx.net<mailto:lu...@gmx.net>> wrote:
Hello,


you can dump the symmetric keys from the browser and import them in wireshark 
to decrypt PFS protected TLS sessions [1]

Yes in case you want to troubleshoot something generic this is a good approach 
but if you want to troubleshoot sessions not initiated by your self, ie 
particular client connection, this is practically impossible.

or downgrade your ciphers settings to non-PF ciphers. Properly decrypting the 
TLS session is the only way to really make sure you see what happens, even if 
there is a TLS related bug in the client or server (haproxy).


Some other idea's are:

- if your backend traffic is unencrypted, you may want to capture the traffic 
there.

Not practical though if you have tens of backend servers. Much better if you 
have to troubleshoot on 2 instead of dozen of servers. At least that's how I 
understand the question related to tcpdump usage on the haproxy servers them 
self.


- if haproxy is rejecting the request, check "show errors" on the admin socket.


As you said, the best solution is to not depend on haproxy specific features, 
as you don't want to modify existing infrastructure in a troubleshooting 
case.Maybe something

Outside haproxy maybe something like mitmproxy or sslstrip might help. Not sure 
though have never used them myself.


Another proxy layer means that you decrypt TLS on the front-end proxy, while 
you sniff the plaintext traffic between the front-end and the second tier 
proxy. You can probably do this with a single haproxy instance recirculating 
the traffic through a unix socket and capture the traffic on it, but it would 
require some trial and error and definitely some testing.

This will probably be faster but can't use tcpdump in that case.


I believe the SSLKEYLOGFILE approach [1] to be the most efficient and simplest 
approach.


cheers,

lukas


[1] 
https://jimshaver.net/2015/02/11/decrypting-tls-browser-traffic-with-wireshark-the-easy-way/



--
Il messaggio e' stato analizzato alla ricerca di virus o
contenuti pericolosi da MailScanner<http://www.mailscanner.info/>, ed e'
risultato non infetto.

Re: tcpdump and Haproxy SSL Offloading

[Lukas Tribus]

Hi,


Am 04.06.2016 um 02:14 schrieb Igor Cicimov:


you can dump the symmetric keys from the browser and import them
in wireshark to decrypt PFS protected TLS sessions [1] 



Yes in case you want to troubleshoot something generic this is a good 
approach but if you want to troubleshoot sessions not initiated by 
your self, ie particular client connection, this is practically 
impossible.


Temporarily disabling PFS ciphers is the only way then. Extracting the 
symmetric key from certain sessions on the haproxy side would be an 
interesting feature though.






Another proxy layer means that you decrypt TLS on the front-end
proxy, while you sniff the plaintext traffic between the front-end
and the second tier proxy. You can probably do this with a single
haproxy instance recirculating the traffic through a unix socket
and capture the traffic on it, but it would require some trial and
error and definitely some testing.


This will probably be faster but can't use tcpdump in that case.


Using the loopback interface instead of a unix socket will fix this.



Regards,

Lukas

Re: tcpdump and Haproxy SSL Offloading

[Igor Cicimov]

Hi Lukas,

On Sat, Jun 4, 2016 at 3:03 AM, Lukas Tribus  wrote:

> Hello,
>
>
> you can dump the symmetric keys from the browser and import them in
> wireshark to decrypt PFS protected TLS sessions [1]


Yes in case you want to troubleshoot something generic this is a good
approach but if you want to troubleshoot sessions not initiated by your
self, ie particular client connection, this is practically impossible.


> or downgrade your ciphers settings to non-PF ciphers. Properly decrypting
> the TLS session is the only way to really make sure you see what happens,
> even if there is a TLS related bug in the client or server (haproxy).
>
>
> Some other idea's are:
>
> - if your backend traffic is unencrypted, you may want to capture the
> traffic there.
>

Not practical though if you have tens of backend servers. Much better if
you have to troubleshoot on 2 instead of dozen of servers. At least that's
how I understand the question related to tcpdump usage on the haproxy
servers them self.


>
> - if haproxy is rejecting the request, check "show errors" on the admin
> socket.
>
>

As you said, the best solution is to not depend on haproxy specific
> features, as you don't want to modify existing infrastructure in a
> troubleshooting case.Maybe something
>
>
Outside haproxy maybe something like mitmproxy or sslstrip might help. Not
sure though have never used them myself.


>
> Another proxy layer means that you decrypt TLS on the front-end proxy,
> while you sniff the plaintext traffic between the front-end and the second
> tier proxy. You can probably do this with a single haproxy instance
> recirculating the traffic through a unix socket and capture the traffic on
> it, but it would require some trial and error and definitely some testing.
>
>
This will probably be faster but can't use tcpdump in that case.


>
> I believe the SSLKEYLOGFILE approach [1] to be the most efficient and
> simplest approach.
>
>
> cheers,
>
> lukas
>
>
> [1]
> https://jimshaver.net/2015/02/11/decrypting-tls-browser-traffic-with-wireshark-the-easy-way/
>
>
>

Re: tcpdump and Haproxy SSL Offloading

[Lukas Tribus]

Hello,


you can dump the symmetric keys from the browser and import them in 
wireshark to decrypt PFS protected TLS sessions [1] or downgrade your 
ciphers settings to non-PF ciphers. Properly decrypting the TLS session 
is the only way to really make sure you see what happens, even if there 
is a TLS related bug in the client or server (haproxy).



Some other idea's are:

- if your backend traffic is unencrypted, you may want to capture the 
traffic there.


- if haproxy is rejecting the request, check "show errors" on the admin 
socket.



As you said, the best solution is to not depend on haproxy specific 
features, as you don't want to modify existing infrastructure in a 
troubleshooting case.



Another proxy layer means that you decrypt TLS on the front-end proxy, 
while you sniff the plaintext traffic between the front-end and the 
second tier proxy. You can probably do this with a single haproxy 
instance recirculating the traffic through a unix socket and capture the 
traffic on it, but it would require some trial and error and definitely 
some testing.



I believe the SSLKEYLOGFILE approach [1] to be the most efficient and 
simplest approach.



cheers,

lukas


[1] 
https://jimshaver.net/2015/02/11/decrypting-tls-browser-traffic-with-wireshark-the-easy-way/

Re: tcpdump and Haproxy SSL Offloading

[Igor Cicimov]

On Fri, Jun 3, 2016 at 3:14 AM, mlist  wrote:

> Often I need to take tcpdump to analyze haproxy communication to clients
> and to backend servers.
>
> As we use haproxy as SSL termination point (haproxy SSL ofloading), at low
> levels (so tcpdump level)
>
> we see communication with client encrypted.
>

If you are not using DHE cyphers (but you should) then you can try ssldump.
In case of Diffie-Hellman though new encryption key is generated for each
ssl session so you are out of luck here.


> There are simple solution so I can do a tcpdump having
>
> unencrypted communication ? Has haproxy some mechanism ?
>

Not that I'm aware of but you can try chaining a local proxy where you can
see the traffic in clear text before you send the traffic to the backend .


>
>
> I have 3 haproxy LBs with 2 L4 LBs balancing on haproxy LBs so I want to
> avoid if possible to make more
>
> complex infrastructure introducing some other intermediate proxy to do
> that, so I make the communication
>
> path as simple and equal to normal request path as possible.
>
>
>
> Roberto
>
>
>
>
>

tcpdump and Haproxy SSL Offloading

[mlist]

Often I need to take tcpdump to analyze haproxy communication to clients and to 
backend servers.
As we use haproxy as SSL termination point (haproxy SSL ofloading), at low 
levels (so tcpdump level)
we see communication with client encrypted. There are simple solution so I can 
do a tcpdump having
unencrypted communication ? Has haproxy some mechanism ?

I have 3 haproxy LBs with 2 L4 LBs balancing on haproxy LBs so I want to avoid 
if possible to make more
complex infrastructure introducing some other intermediate proxy to do that, so 
I make the communication
path as simple and equal to normal request path as possible.

Roberto

Re: ssl offloading

[Gerd Mueller]

wow!

Thanks, again

Gerd


 Weitergeleitete Nachricht 
Von: Pavlos Parissis <pavlos.paris...@gmail.com>
An: Andrew Hayworth <andrew.haywo...@getbraintree.com>, Gerd Mueller 
Kopie: haproxy@formilux.org <haproxy@formilux.org>
Betreff: Re: ssl offloading
Datum: Sun, 3 Apr 2016 22:37:41 +0200

On 01/04/2016 04:20 μμ, Andrew Hayworth wrote:
> 
> Hi there -
> 
> Have you considered HAProxy in multiprocess mode? You could have a
> frontend spread across multiple threads that terminates SSL. We're
> experimenting with such a design here.
> 
It has been mentioned before that you can increase capacity[1] by
using:
- latest Intel CPUs
- Openssl 1.0.2g version or higher
- enable multiprocess mode
- PIN HAProxy to CPU processes
- stop irqbalancer and PIN network interrupt queues to CPUs, using
Intel
10GbE cards makes this very easy
- tune HAProxy and OS
- Enable RFC5077 TLS Session Resumption, tricky in distributed setup
- Deploy ECC certificates and enable ECC ciphers.

I managed to achieve 450K HTTPS/sec with object size 1K using 22 cores
out of 24. Disabling HT and use 10cores gave me 370K HTTPS/sec. HT is
disabled for now in our systems.


I wouldn't offload ssl to hardware as they are like blackbox. You don't
know what they do and how vulnerable they are.

Cheers,
Pavlos


[1] number of https/sec while CPU utilization(user+sys level) <=70%
why 70%? because you want to have some room to handle attacks, failures
on other nodes/DCs, mistakes by devs( a typo somewhere can increase
number of requests and cause a DDOS...)



signature.asc
Description: This is a digitally signed message part

Re: ssl offloading

[Pavlos Parissis]

On 01/04/2016 04:20 μμ, Andrew Hayworth wrote:
> Hi there -
> 
> Have you considered HAProxy in multiprocess mode? You could have a
> frontend spread across multiple threads that terminates SSL. We're
> experimenting with such a design here.
> 

It has been mentioned before that you can increase capacity[1] by using:
- latest Intel CPUs
- Openssl 1.0.2g version or higher
- enable multiprocess mode
- PIN HAProxy to CPU processes
- stop irqbalancer and PIN network interrupt queues to CPUs, using Intel
10GbE cards makes this very easy
- tune HAProxy and OS
- Enable RFC5077 TLS Session Resumption, tricky in distributed setup
- Deploy ECC certificates and enable ECC ciphers.

I managed to achieve 450K HTTPS/sec with object size 1K using 22 cores
out of 24. Disabling HT and use 10cores gave me 370K HTTPS/sec. HT is
disabled for now in our systems.


I wouldn't offload ssl to hardware as they are like blackbox. You don't
know what they do and how vulnerable they are.

Cheers,
Pavlos


[1] number of https/sec while CPU utilization(user+sys level) <=70%
why 70%? because you want to have some room to handle attacks, failures
on other nodes/DCs, mistakes by devs( a typo somewhere can increase
number of requests and cause a DDOS...)




signature.asc
Description: OpenPGP digital signature

Re: ssl offloading

[Andrew Hayworth]

Hi there -

Have you considered HAProxy in multiprocess mode? You could have a frontend
spread across multiple threads that terminates SSL. We're experimenting
with such a design here.

On Fri, Apr 1, 2016 at 5:31 AM, Gerd Mueller <gerd.muel...@mikatiming.de>
wrote:

> Ok sounds good. Thanks for the input.
>
> Gerd
>
>  Weitergeleitete Nachricht 
> Von: Vincent Bernat <ber...@luffy.cx>
> An: Conrad Hoffmann <con...@soundcloud.com>
> Kopie: Gerd Mueller <gerd.muel...@mikatiming.de>, haproxy@formilux.org
> <haproxy@formilux.org>
> Betreff: Re: ssl offloading
> Datum: Fri, 1 Apr 2016 11:29:16 +0200
>
>  ❦  1 avril 2016 11:11 +0200, Conrad Hoffmann <con...@soundcloud.com> :
>
> >
> > I can't really back this up with reliable numbers, but a company I
> > once
> > worked for experimented with such hardware. The outcome was, and I
> > would
> > still always recommend this today, to rather throw more regular
> > hardware at
> > the problem. Modern processors have a lot special instructions
> > specifically
> > for cryptographic operations (maybe make sure you are making full use
> > of
> > that) and are way cheaper than specialized SSL hardware. Stuff like
> > SSL
> > changes a lot and often needs immediate security fixes, so going with
> > general purpose hardware where you are not dependent on some vendor
> > support
> > will likely make your life easier at some point.
> >
> > That's just an opinion after all, of course.
> I agree with you. x86 hardware is far less expensive and performant
> than
> dedicated hardware. Dedicated hardware is only useful if your team
> don't
> want to handle software (but in this case, you can also look at the
> Aloha appliance). Go for the maximum number of GHz and as many cores as
> you want since the performance scales almost linearly.
>



-- 
- Andrew Hayworth

Re: ssl offloading

[Gerd Mueller]

Ok sounds good. Thanks for the input.

Gerd

 Weitergeleitete Nachricht 
Von: Vincent Bernat <ber...@luffy.cx>
An: Conrad Hoffmann <con...@soundcloud.com>
Kopie: Gerd Mueller <gerd.muel...@mikatiming.de>, haproxy@formilux.org
<haproxy@formilux.org>
Betreff: Re: ssl offloading
Datum: Fri, 1 Apr 2016 11:29:16 +0200

 ❦  1 avril 2016 11:11 +0200, Conrad Hoffmann <con...@soundcloud.com> :

> 
> I can't really back this up with reliable numbers, but a company I
> once
> worked for experimented with such hardware. The outcome was, and I
> would
> still always recommend this today, to rather throw more regular
> hardware at
> the problem. Modern processors have a lot special instructions
> specifically
> for cryptographic operations (maybe make sure you are making full use
> of
> that) and are way cheaper than specialized SSL hardware. Stuff like
> SSL
> changes a lot and often needs immediate security fixes, so going with
> general purpose hardware where you are not dependent on some vendor
> support
> will likely make your life easier at some point.
> 
> That's just an opinion after all, of course.
I agree with you. x86 hardware is far less expensive and performant
than
dedicated hardware. Dedicated hardware is only useful if your team
don't
want to handle software (but in this case, you can also look at the
Aloha appliance). Go for the maximum number of GHz and as many cores as
you want since the performance scales almost linearly.

Re: ssl offloading

[Vincent Bernat]

 ❦  1 avril 2016 11:11 +0200, Conrad Hoffmann  :

> I can't really back this up with reliable numbers, but a company I once
> worked for experimented with such hardware. The outcome was, and I would
> still always recommend this today, to rather throw more regular hardware at
> the problem. Modern processors have a lot special instructions specifically
> for cryptographic operations (maybe make sure you are making full use of
> that) and are way cheaper than specialized SSL hardware. Stuff like SSL
> changes a lot and often needs immediate security fixes, so going with
> general purpose hardware where you are not dependent on some vendor support
> will likely make your life easier at some point.
>
> That's just an opinion after all, of course.

I agree with you. x86 hardware is far less expensive and performant than
dedicated hardware. Dedicated hardware is only useful if your team don't
want to handle software (but in this case, you can also look at the
Aloha appliance). Go for the maximum number of GHz and as many cores as
you want since the performance scales almost linearly.
-- 
Wrinkles should merely indicate where smiles have been.
-- Mark Twain

Re: ssl offloading

[Conrad Hoffmann]

I can't really back this up with reliable numbers, but a company I once
worked for experimented with such hardware. The outcome was, and I would
still always recommend this today, to rather throw more regular hardware at
the problem. Modern processors have a lot special instructions specifically
for cryptographic operations (maybe make sure you are making full use of
that) and are way cheaper than specialized SSL hardware. Stuff like SSL
changes a lot and often needs immediate security fixes, so going with
general purpose hardware where you are not dependent on some vendor support
will likely make your life easier at some point.

That's just an opinion after all, of course.

Cheers,
Conrad

On 04/01/2016 10:06 AM, Gerd Mueller wrote:
> We are experiencing 100% cpu load by this specific haproxy thread during huge 
> ssl load. With haproxy .4 we first used stunnel, than apache with mod_ssl. I 
> think haproxy with ssl performance much better than the other 2 but I am 
> thinking about offloading to a specific ssl device. Does anybody know 
> anything about dedicated hardware?
> 
> Thanks Gerd
> 
>  Weitergeleitete Nachricht 
> Von: Nathan Williams 
> <nath.e.w...@gmail.com<mailto:nathan%20williams%20%3cnath.e.w...@gmail.com%3e>>
> An: Lukas Tribus 
> <luky...@hotmail.com<mailto:lukas%20tribus%20%3cluky...@hotmail.com%3e>>, 
> Gerd Mueller 
> <gerd.muel...@mikatiming.de<mailto:gerd%20mueller%20%3cgerd.muel...@mikatiming.de%3e>>,
>  haproxy@formilux.org 
> <haproxy@formilux.org<mailto:%22hapr...@formilux.org%22%20%3chapr...@formilux.org%3e>>
> Betreff: Re: ssl offloading
> Datum: Fri, 1 Apr 2016 01:54:29 +0000
> 
> 
> stunnel's what we used before Haproxy had it built in, which worked fine, but 
> SSL offloading in Haproxy's been excellent in our experience, so my guess 
> would be that you could make it work with some config tuning.
> 
> On Thu, Mar 31, 2016, 12:45 PM Lukas Tribus 
> <luky...@hotmail.com<mailto:luky...@hotmail.com>> wrote:
>> Hi list,
>>
>> what are your ideas about offloading of ssl? ssl inside haproxy is nice
>> but is very expensive.
> 
> Why would you think that?
> 
> 
> Lukas
> 
> 
> 

-- 
Conrad Hoffmann
Traffic Engineer

SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany

Managing Director: Alexander Ljung | Incorporated in England & Wales
with Company No. 6343600 | Local Branch Office | AG Charlottenburg |
HRB 110657B

Re: ssl offloading

[Nathan Williams]

stunnel's what we used before Haproxy had it built in, which worked fine,
but SSL offloading in Haproxy's been excellent in our experience, so my
guess would be that you could make it work with some config tuning.

On Thu, Mar 31, 2016, 12:45 PM Lukas Tribus <luky...@hotmail.com> wrote:

> > Hi list,
> >
> > what are your ideas about offloading of ssl? ssl inside haproxy is nice
> > but is very expensive.
>
> Why would you think that?
>
>
> Lukas
>
>
>

RE: ssl offloading

[Lukas Tribus]

> Hi list,
>
> what are your ideas about offloading of ssl? ssl inside haproxy is nice
> but is very expensive.

Why would you think that?


Lukas

  

ssl offloading

[Gerd Mueller]

Hi list,

what are your ideas about offloading of ssl? ssl inside haproxy is nice
but is very expensive. So I would like to offload the ssl to something
else. 

Any ideas?

Thanks,

Gerd

Re: FW: SSL offloading in HAProxy

[Baptiste]

Hi,

SSL offloading in front of IMAPs (port 993) is supported.
If you try to do STARTTLS over IMAP, it is not supported.

Baptiste



On Wed, Jul 15, 2015 at 10:38 AM, Cohen Galit galit.co...@comverse.com wrote:
 Hello HAProxy team,



 I see that the SSL offloading for http protocol is already supported (
 http://blog.haproxy.com/2012/09/10/how-to-get-ssl-with-haproxy-getting-rid-of-stunnel-stud-nginx-or-pound/
 )

 I would like to know if there is an option of SSL offloading for IMAP
 protocol.



 Thanks,

 Galit



 From: Avrahami David
 Sent: Wednesday, July 01, 2015 3:50 PM
 To: Cohen Galit
 Cc: Sabban Gili; Meltser Tiran
 Subject: SSL offloading in HAProxy



 Hi Galit,



 Can you please post the below question to HAProxy forum?



 I see that the SSL offloading for http protocol is already supported (
 http://blog.haproxy.com/2012/09/10/how-to-get-ssl-with-haproxy-getting-rid-of-stunnel-stud-nginx-or-pound/
 )

 I would like to know if there is an option of SSL offloading for IMAP
 protocol.





 Best Regards,

 David Avrahami

 Security SE

 Tel: +972-3-6452374

 Mobile: +972-544382374

 Email: david.avrah...@comverse.com



 
 “This e-mail message may contain confidential, commercial or privileged
 information that constitutes proprietary information of Comverse Inc. or its
 subsidiaries. If you are not the intended recipient of this message, you are
 hereby notified that any review, use or distribution of this information is
 absolutely prohibited and we request that you delete all copies and contact
 us by e-mailing to: secur...@comverse.com. Thank You.”

FW: SSL offloading in HAProxy

[Cohen Galit]

Hello HAProxy team,

I see that the SSL offloading for http protocol is already supported ( 
http://blog.haproxy.com/2012/09/10/how-to-get-ssl-with-haproxy-getting-rid-of-stunnel-stud-nginx-or-pound/
 )
I would like to know if there is an option of SSL offloading for IMAP protocol.

Thanks,
Galit

From: Avrahami David
Sent: Wednesday, July 01, 2015 3:50 PM
To: Cohen Galit
Cc: Sabban Gili; Meltser Tiran
Subject: SSL offloading in HAProxy

Hi Galit,

Can you please post the below question to HAProxy forum?

I see that the SSL offloading for http protocol is already supported ( 
http://blog.haproxy.com/2012/09/10/how-to-get-ssl-with-haproxy-getting-rid-of-stunnel-stud-nginx-or-pound/
 )
I would like to know if there is an option of SSL offloading for IMAP protocol.


Best Regards,
David Avrahami
Security SE
Tel: +972-3-6452374
Mobile: +972-544382374
Email: david.avrah...@comverse.commailto:david.avrah...@comverse.com


This e-mail message may contain confidential, commercial or privileged 
information that constitutes proprietary information of Comverse Inc. or its 
subsidiaries. If you are not the intended recipient of this message, you are 
hereby notified that any review, use or distribution of this information is 
absolutely prohibited and we request that you delete all copies and contact us 
by e-mailing to: secur...@comverse.com. Thank You.

is it possible to SSL offloading and then re-pack as SSL?

[Robert Chen]

I want to do haproxy 1.5 SSL offloading, filter/processing some headers and
then re-send as SSL?

thanks,
Robert

Re: is it possible to SSL offloading and then re-pack as SSL?

[Thomas Heil]

Hi,

On 16.10.2014 22:36, Lukas Tribus wrote:
 I want to do haproxy 1.5 SSL offloading, filter/processing some headers  
 and then re-send as SSL? 
 Yes, it is.


 Lukas
Have a look at
http://blog.haproxy.com/2013/02/26/ssl-offloading-impact-on-web-applications/
 


cheers
thomas

Re: Haproxy SSL offloading to nginx backend web servers - need persistent connections

[Baptiste]

Hi Chris,

How can you point HAProxy in such problem?
I mean, you have a single server in your farm, so if the browser can't
download objects, it may come from an other point.
And since you have a single server, you don't need persistence ;)

Can you confirm whether:
- the application works well in HTTP (clear)
- the application works well without HAProxy

Note that SSL offloading may have some impacts on web application:
http://blog.exceliance.fr/2013/02/26/ssl-offloading-impact-on-web-applications/

Can you enable logs as described in the link above and check the result?

Baptiste


On Mon, Nov 11, 2013 at 8:49 PM, Chris bludge...@gmail.com wrote:
 Hello everyone,

 I have a question concerning session persistence with offloading SSL via
 haproxy to backend nginx web servers running a PHP website.

 Basically, I have a configuration that is performing the SSL offloading
 successfully, however, it seems that session persistence is not working
 properly as some of the images from the nginx web server are not showing up,
 however the log in fields for this web server are showing up.

 What needs to happen is that an http request comes into haproxy, http calls
 get redirected to https, SSL is offloaded and the connection is handed over
 to the relevant back-end - with session persistence in-tact via ACLs within
 haproxy.

 I have read that http-server-close will preserve the connection and keep
 it persistent but I have not been able to get this to work.  I have also
 read that the type of balance used  (such as round robin) can affect the
 persistence of connections.  I have also tried to use

 cookie PHPSESSID insert nocache indirect  but I just am having no luck as
 there are a slew of configurations that can be used to do this task and I
 have gone over the haproxy documentation again and again and I just am not
 sure of the correct way of doing this.  The haproxy documentation is very
 thorough, but it is also very complex.

 In an effort to try to get this working, below is the configuration for my
 haproxy setup, I have stripped out all of the testing configurations that
 I've been using to try to get it to work.  I am hoping that someone might be
 able to assist me with properly getting this configured to make sessions
 persistent.

 Your expertise and advice are greatly welcomed and very appreciated - I
 thank you for your time.

 ---
 global
   log 127.0.0.1 local0
   log 127.0.0.1 local1
   user haproxy
   group haproxy
   daemon

 defaults
   log global
   mode http
   option httplog
   option dontlognull
   option redispatch
   stats enable
   maxconn 512
   retries 3
   contimeout 6
   clitimeout 6
   srvtimeout 6


 ## http frontend to redirect to https frontend

  frontend https_frontend
 bind 0.0.0.0:80
 redirect scheme https if !{ ssl_fc }


 ## https frontend to offload SSL to the backends

 frontend haproxy_https
   mode http
   option http-server-close
   bind 0.0.0.0:443 ssl crt /etc/haproxy/psl-wildcard/wildcard.pem ca-file
 /etc/haproxy/psl-wildcard/wildcard.ca-bundle

   acl is_psl_https  hdr_end(host)   -i www.test-site.com
   acl is_broker_psl_https   hdr_end(host)   -i broker.test-site.com
   acl is_eclose_psl_https   hdr_end(host)   -i eclose.test-site.com

   use_backend   is_psl_https_backendif  is_psl_https
   use_backend   is_broker_https_backend if  is_broker_psl_https
   use_backend   is_eclose_https_backend if  is_eclose_psl_https

   default_backend   is_psl_https_backend



 ## backends

 backend is_psl_https_backend
   mode http
   balance source
   option http-server-close
   server server1 10.10.221.171:80

 backend is_broker_https_backend
   mode http
   balance source
   option http-server-close
   server server1 10.10.221.172:80

 backend is_eclose_https_backend
   mode http
   balance source
   option http-server-close
   server server1 10.10.221.173:80


 listen admin 0.0.0.0:22002
   mode http
   stats uri /

 

 Again, thank you very much.

 Sincerely,

 Chris

Re: Haproxy SSL offloading to nginx backend web servers - need persistent connections

[Chris]

Baptiste,

Thanks again for taking the time to read through and respond.  I had done a
lot of troubleshooting and tried many different configuration options, to
the point that I am now confused.  You are correct, with only one web
server, persistence is not necessary, I should have realized this.  I tried
testing the web server by bypassing haproxy and I was getting the same
result, no image showing up so it is a web server problem that needs to be
resolved first.

With that said, the end plan is to indeed put 2 or 3 web servers behind
haproxy to be load balanced too - when that time comes, may I ask you what
the best way to implement session persistence would be, ensuring that when
a client connects to the web servers that they continue to stay on that web
server during their entire session?

Would this be the - option  http-server-close ?  Or, would implementing
some sort of PHPSESSID cookie be needed?

I think for true testing then, they need to get their web servers working
first, and then I need to get another one set up so that I can actually
test out session persistence.

Last question, when I get this into place, and to verify that sessions are
staying persistent, what is the best way to verify this?  Will haproxy logs
show this, or do I need to perform some sort of strace on the haproxy PID
to watch it?

Thanks a million Baptiste, you are a life saver - not only to me, to but
many people on this amazing list

Sincerely,

Chris


-Chris


On Tue, Nov 12, 2013 at 12:12 AM, Baptiste bed...@gmail.com wrote:

 Hi Chris,

 How can you point HAProxy in such problem?
 I mean, you have a single server in your farm, so if the browser can't
 download objects, it may come from an other point.
 And since you have a single server, you don't need persistence ;)

 Can you confirm whether:
 - the application works well in HTTP (clear)
 - the application works well without HAProxy

 Note that SSL offloading may have some impacts on web application:

 http://blog.exceliance.fr/2013/02/26/ssl-offloading-impact-on-web-applications/

 Can you enable logs as described in the link above and check the result?

 Baptiste


 On Mon, Nov 11, 2013 at 8:49 PM, Chris bludge...@gmail.com wrote:
  Hello everyone,
 
  I have a question concerning session persistence with offloading SSL via
  haproxy to backend nginx web servers running a PHP website.
 
  Basically, I have a configuration that is performing the SSL offloading
  successfully, however, it seems that session persistence is not working
  properly as some of the images from the nginx web server are not showing
 up,
  however the log in fields for this web server are showing up.
 
  What needs to happen is that an http request comes into haproxy, http
 calls
  get redirected to https, SSL is offloaded and the connection is handed
 over
  to the relevant back-end - with session persistence in-tact via ACLs
 within
  haproxy.
 
  I have read that http-server-close will preserve the connection and
 keep
  it persistent but I have not been able to get this to work.  I have also
  read that the type of balance used  (such as round robin) can affect the
  persistence of connections.  I have also tried to use
 
  cookie PHPSESSID insert nocache indirect  but I just am having no luck
 as
  there are a slew of configurations that can be used to do this task and I
  have gone over the haproxy documentation again and again and I just am
 not
  sure of the correct way of doing this.  The haproxy documentation is very
  thorough, but it is also very complex.
 
  In an effort to try to get this working, below is the configuration for
 my
  haproxy setup, I have stripped out all of the testing configurations that
  I've been using to try to get it to work.  I am hoping that someone
 might be
  able to assist me with properly getting this configured to make sessions
  persistent.
 
  Your expertise and advice are greatly welcomed and very appreciated - I
  thank you for your time.
 
  ---
  global
log 127.0.0.1 local0
log 127.0.0.1 local1
user haproxy
group haproxy
daemon
 
  defaults
log global
mode http
option httplog
option dontlognull
option redispatch
stats enable
maxconn 512
retries 3
contimeout 6
clitimeout 6
srvtimeout 6
 
 
  ## http frontend to redirect to https frontend
 
   frontend https_frontend
  bind 0.0.0.0:80
  redirect scheme https if !{ ssl_fc }
 
 
  ## https frontend to offload SSL to the backends
 
  frontend haproxy_https
mode http
option http-server-close
bind 0.0.0.0:443 ssl crt /etc/haproxy/psl-wildcard/wildcard.pem
 ca-file
  /etc/haproxy/psl-wildcard/wildcard.ca-bundle
 
acl is_psl_https  hdr_end(host)   -i www.test-site.com
acl is_broker_psl_https   hdr_end(host)   -i broker.test-site.com
acl is_eclose_psl_https   hdr_end(host)   -i eclose.test-site.com
 
use_backend   is_psl_https_backend

Re: Haproxy SSL offloading to nginx backend web servers - need persistent connections

[Baptiste]

Hi Chris,

It is recommended to use option http-server-close, unless for specific
use (like NTLM). So please turn it on.

You have basically 3 main choices:
- cookie SERVERID insert = HAProxy will setup a cookie called SERVERID
- cookie PHPSESSID prefix = HAProxy will use the application cookie
setup by your web servers
- stick table + stick store + stick match = HAProxy will learn
cookies, store them in a table and use it for persistence.

For option #1 and #2 don't forget the cookie parameter on the server directive.

Baptiste


On Tue, Nov 12, 2013 at 5:14 PM, Chris bludge...@gmail.com wrote:
 Baptiste,

 Thanks again for taking the time to read through and respond.  I had done a
 lot of troubleshooting and tried many different configuration options, to
 the point that I am now confused.  You are correct, with only one web
 server, persistence is not necessary, I should have realized this.  I tried
 testing the web server by bypassing haproxy and I was getting the same
 result, no image showing up so it is a web server problem that needs to be
 resolved first.

 With that said, the end plan is to indeed put 2 or 3 web servers behind
 haproxy to be load balanced too - when that time comes, may I ask you what
 the best way to implement session persistence would be, ensuring that when a
 client connects to the web servers that they continue to stay on that web
 server during their entire session?

 Would this be the - option  http-server-close ?  Or, would implementing some
 sort of PHPSESSID cookie be needed?

 I think for true testing then, they need to get their web servers working
 first, and then I need to get another one set up so that I can actually test
 out session persistence.

 Last question, when I get this into place, and to verify that sessions are
 staying persistent, what is the best way to verify this?  Will haproxy logs
 show this, or do I need to perform some sort of strace on the haproxy PID to
 watch it?

 Thanks a million Baptiste, you are a life saver - not only to me, to but
 many people on this amazing list

 Sincerely,

 Chris


 -Chris


 On Tue, Nov 12, 2013 at 12:12 AM, Baptiste bed...@gmail.com wrote:

 Hi Chris,

 How can you point HAProxy in such problem?
 I mean, you have a single server in your farm, so if the browser can't
 download objects, it may come from an other point.
 And since you have a single server, you don't need persistence ;)

 Can you confirm whether:
 - the application works well in HTTP (clear)
 - the application works well without HAProxy

 Note that SSL offloading may have some impacts on web application:

 http://blog.exceliance.fr/2013/02/26/ssl-offloading-impact-on-web-applications/

 Can you enable logs as described in the link above and check the result?

 Baptiste


 On Mon, Nov 11, 2013 at 8:49 PM, Chris bludge...@gmail.com wrote:
  Hello everyone,
 
  I have a question concerning session persistence with offloading SSL via
  haproxy to backend nginx web servers running a PHP website.
 
  Basically, I have a configuration that is performing the SSL offloading
  successfully, however, it seems that session persistence is not working
  properly as some of the images from the nginx web server are not showing
  up,
  however the log in fields for this web server are showing up.
 
  What needs to happen is that an http request comes into haproxy, http
  calls
  get redirected to https, SSL is offloaded and the connection is handed
  over
  to the relevant back-end - with session persistence in-tact via ACLs
  within
  haproxy.
 
  I have read that http-server-close will preserve the connection and
  keep
  it persistent but I have not been able to get this to work.  I have also
  read that the type of balance used  (such as round robin) can affect the
  persistence of connections.  I have also tried to use
 
  cookie PHPSESSID insert nocache indirect  but I just am having no luck
  as
  there are a slew of configurations that can be used to do this task and
  I
  have gone over the haproxy documentation again and again and I just am
  not
  sure of the correct way of doing this.  The haproxy documentation is
  very
  thorough, but it is also very complex.
 
  In an effort to try to get this working, below is the configuration for
  my
  haproxy setup, I have stripped out all of the testing configurations
  that
  I've been using to try to get it to work.  I am hoping that someone
  might be
  able to assist me with properly getting this configured to make sessions
  persistent.
 
  Your expertise and advice are greatly welcomed and very appreciated - I
  thank you for your time.
 
  ---
  global
log 127.0.0.1 local0
log 127.0.0.1 local1
user haproxy
group haproxy
daemon
 
  defaults
log global
mode http
option httplog
option dontlognull
option redispatch
stats enable
maxconn 512
retries 3
contimeout 6
clitimeout 6
srvtimeout 6
 
 
  ## http frontend to redirect to https

Haproxy SSL offloading to nginx backend web servers - need persistent connections

[Chris]

Hello everyone,

I have a question concerning session persistence with offloading SSL via
haproxy to backend nginx web servers running a PHP website.

Basically, I have a configuration that is performing the SSL offloading
successfully, however, it seems that session persistence is not working
properly as some of the images from the nginx web server are not showing
up, however the log in fields for this web server are showing up.

What needs to happen is that an http request comes into haproxy, http calls
get redirected to https, SSL is offloaded and the connection is handed over
to the relevant back-end - with session persistence in-tact via ACLs within
haproxy.

I have read that http-server-close will preserve the connection and keep
it persistent but I have not been able to get this to work.  I have also
read that the type of balance used  (such as round robin) can affect the
persistence of connections.  I have also tried to use

cookie PHPSESSID insert nocache indirect  but I just am having no luck as
there are a slew of configurations that can be used to do this task and I
have gone over the haproxy documentation again and again and I just am not
sure of the correct way of doing this.  The haproxy documentation is very
thorough, but it is also very complex.

In an effort to try to get this working, below is the configuration for my
haproxy setup, I have stripped out all of the testing configurations that
I've been using to try to get it to work.  I am hoping that someone might
be able to assist me with properly getting this configured to make sessions
persistent.

Your expertise and advice are greatly welcomed and very appreciated - I
thank you for your time.

---
global
  log 127.0.0.1 local0
  log 127.0.0.1 local1
  user haproxy
  group haproxy
  daemon

defaults
  log global
  mode http
  option httplog
  option dontlognull
  option redispatch
  stats enable
  maxconn 512
  retries 3
  contimeout 6
  clitimeout 6
  srvtimeout 6


## http frontend to redirect to https frontend

 frontend https_frontend
bind 0.0.0.0:80
redirect scheme https if !{ ssl_fc }


## https frontend to offload SSL to the backends

frontend haproxy_https
  mode http
  option http-server-close
  bind 0.0.0.0:443 ssl crt /etc/haproxy/psl-wildcard/wildcard.pem ca-file
/etc/haproxy/psl-wildcard/wildcard.ca-bundle

  acl is_psl_https  hdr_end(host)   -i www.test-site.com
  acl is_broker_psl_https   hdr_end(host)   -i broker.test-site.com
  acl is_eclose_psl_https   hdr_end(host)   -i eclose.test-site.com

  use_backend   is_psl_https_backendif  is_psl_https
  use_backend   is_broker_https_backend if  is_broker_psl_https
  use_backend   is_eclose_https_backend if  is_eclose_psl_https

  default_backend   is_psl_https_backend



## backends

backend is_psl_https_backend
  mode http
  balance source
  option http-server-close
  server server1 10.10.221.171:80

backend is_broker_https_backend
  mode http
  balance source
  option http-server-close
  server server1 10.10.221.172:80

backend is_eclose_https_backend
  mode http
  balance source
  option http-server-close
  server server1 10.10.221.173:80


listen admin 0.0.0.0:22002
  mode http
  stats uri /



Again, thank you very much.

Sincerely,

Chris

Re: I'm so close (I Think) - need a little help finishing this - Haproxy SSL offloading with ACLs and a wildcard cert

[Baptiste]

Hi Chris,

the option you mentioned keeps alive TCP connection on the client
side, if the client is compatible with HTTP keep alive.
The connection is kept alive as long as it needs to be, it can be
driven by HAProxy through the timeout http-keepalive parameter.

It does not mean all the requests from a client session will be
redirected to a single server.
For this purpose, you can use the cookie keyword, IE: cookie
SERVERID insert + cookie parameter on the server line.
This cookie will timeout when the browser windows will be closed. This
is a session cookie.
You can force an idle timeout on such cookie if you wish.

I recommend you to read the configuration manual of HAProxy about the
options above.

Baptiste


On Tue, Oct 29, 2013 at 12:31 AM, Chris bludge...@gmail.com wrote:
 Hello again Baptiste,

 Thank you for replying back to my questions, they are very helpful answers
 and I really do appreciate your time.

 Concerning your question back about persistence or stickiness:


 - Sessions need to stay alive after being offloaded to the backend web
 servers to port 80 on them.  (Session keepalive or stickiness?)

 do you mean persistence or connection keepalives ???


 I meant for session persistence, and does the following allow persistence to
 happen correctly?:

  option http-server-close


 If it does, one final question would be how does one control the length of
 persistence, is that something controlled with a specific timeout variable?

 Thank you so much again, I really appreciate your help a lot.

 Chris


 -Chris


 On Tue, Oct 22, 2013 at 2:13 AM, Baptiste bed...@gmail.com wrote:

 Hi Chris,

 My answers inline.

 On Mon, Oct 21, 2013 at 10:57 PM, Chris bludge...@gmail.com wrote:
  - If any of these domains and subdomains are called with http://, they
  need
  to be redirected to https://

 # redirect http to https when connection is not ciphered
 http-request redirect scheme https if !{ ssl_fc } { hdr(host)
 www.test.com broker.test.com eclose.test.com images.test.com }


  - If a call comes in just to test.com, it needs to be redirected to
  https
  www.test.com:443

 # redirect text.com:80 to https://www.test.com:443
 http-request redirect prefix https://www.test.com if !{ ssl_fc } {
 hdr(host) test.com }


  - Sessions need to stay alive after being offloaded to the backend web
  servers to port 80 on them.  (Session keepalive or stickiness?)

 do you mean persistence or connection keepalives ???


  - images.test.com does not need to have SSL offloaded and I'm wondering
  if I
  should incorporate this into haproxy at all, maybe for load balancing at
  some point, this seems like a good idea)

 well, this is not what you asked in your first question, you wanted to
 offload SSL on all your domains.
 That said, I agree, ciphering images (usually) is useless.

 # redirect http to https when connection is not ciphered
 http-request redirect scheme https if !{ ssl_fc } { hdr(host)
 www.test.com broker.test.com eclose.test.com }

 since images.test.com is not in the list, the traffic will be accepted
 on port 80.

  defaults
  mode http
  maxconn 512
  option dontlognull
  option http-server-close
  retries 3
  contimeout 6
  clitimeout 6
  srvtimeout 6
 
  frontend haproxy_http
  bind 0.0.0.0:80
  timeout client 8640
  acl is_test hdr_end(host) -i www.test.com
  acl is_broker_test hdr_end(host) -i broker.test.com
  acl is_eclose_test  hdr_end(host) -i eclose.test.com
  acl is_images_test  hdr_end(host)  -i images.test.com
 
 
   # redirect http to https when connection is not ciphered
   http-request redirect scheme https if !{ ssl_fc } { hdr(host)
 www.test.com broker.test.com eclose.test.com images.test.com }
# redirect text.com:80 to https://www.test.com:443
   http-request redirect prefix https://www.test.com if !{ ssl_fc }
 { hdr(host) test.com }

  use_backend is_images_test if { hdr(host) images.test.com }

 
 
 
  frontend haproxy_https
  bind 0.0.0.0:443 ssl crt /etc/haproxy/certs/wildcard.pem
  log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\
  %CC\
  %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\
  {%[ssl_c_verify],%{+Q}[ssl_c_s_dn],{+Q[ssl_c_i_dn]}\ %{+Q}r
  acl is_test_https hdr_end(host) -i www.test.com
  acl is_broker_test_https hdr_end(host) -i broker.test.com
  acl is_eclose_test_https hdr_end(host) -i eclose.test.com
  use_backend is_test_https_backend if  is_test_https
  use_backend is_broker_https_backendif
  is_broker_test_https
  use_backend is_eclose_https_backend  if is_eclose_test_https
 
 
 
  # backends
 
  backend is_test_https_backend
  mode http
  balance source
  option http-server-close
  server server1 192.168.33.70
  server server2 192.168.33.170
 
  backend is_broker_https_backend
  mode http
  balance source
  option http-server-close
  server 

Re: I'm so close (I Think) - need a little help finishing this - Haproxy SSL offloading with ACLs and a wildcard cert

[Chris]

Hello again Baptiste,

Thank you for replying back to my questions, they are very helpful answers
and I really do appreciate your time.

Concerning your question back about persistence or stickiness:


 - Sessions need to stay alive after being offloaded to the backend web
 servers to port 80 on them.  (Session keepalive or stickiness?)

do you mean persistence or connection keepalives ???


I meant for session persistence, and does the following allow persistence
to happen correctly?:

 option http-server-close


If it does, one final question would be how does one control the length of
persistence, is that something controlled with a specific timeout variable?

Thank you so much again, I really appreciate your help a lot.

Chris


-Chris


On Tue, Oct 22, 2013 at 2:13 AM, Baptiste bed...@gmail.com wrote:

 Hi Chris,

 My answers inline.

 On Mon, Oct 21, 2013 at 10:57 PM, Chris bludge...@gmail.com wrote:
  - If any of these domains and subdomains are called with http://, they
 need
  to be redirected to https://

 # redirect http to https when connection is not ciphered
 http-request redirect scheme https if !{ ssl_fc } { hdr(host)
 www.test.com broker.test.com eclose.test.com images.test.com }


  - If a call comes in just to test.com, it needs to be redirected to
 https
  www.test.com:443

 # redirect text.com:80 to https://www.test.com:443
 http-request redirect prefix https://www.test.com if !{ ssl_fc } {
 hdr(host) test.com }


  - Sessions need to stay alive after being offloaded to the backend web
  servers to port 80 on them.  (Session keepalive or stickiness?)

 do you mean persistence or connection keepalives ???


  - images.test.com does not need to have SSL offloaded and I'm wondering
 if I
  should incorporate this into haproxy at all, maybe for load balancing at
  some point, this seems like a good idea)

 well, this is not what you asked in your first question, you wanted to
 offload SSL on all your domains.
 That said, I agree, ciphering images (usually) is useless.

 # redirect http to https when connection is not ciphered
 http-request redirect scheme https if !{ ssl_fc } { hdr(host)
 www.test.com broker.test.com eclose.test.com }

 since images.test.com is not in the list, the traffic will be accepted
 on port 80.

  defaults
  mode http
  maxconn 512
  option dontlognull
  option http-server-close
  retries 3
  contimeout 6
  clitimeout 6
  srvtimeout 6
 
  frontend haproxy_http
  bind 0.0.0.0:80
  timeout client 8640
  acl is_test hdr_end(host) -i www.test.com
  acl is_broker_test hdr_end(host) -i broker.test.com
  acl is_eclose_test  hdr_end(host) -i eclose.test.com
  acl is_images_test  hdr_end(host)  -i images.test.com
 
 
   # redirect http to https when connection is not ciphered
   http-request redirect scheme https if !{ ssl_fc } { hdr(host)
 www.test.com broker.test.com eclose.test.com images.test.com }
# redirect text.com:80 to https://www.test.com:443
   http-request redirect prefix https://www.test.com if !{ ssl_fc }
 { hdr(host) test.com }

  use_backend is_images_test if { hdr(host) images.test.com }

 
 
 
  frontend haproxy_https
  bind 0.0.0.0:443 ssl crt /etc/haproxy/certs/wildcard.pem
  log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\
 %CC\
  %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\
  {%[ssl_c_verify],%{+Q}[ssl_c_s_dn],{+Q[ssl_c_i_dn]}\ %{+Q}r
  acl is_test_https hdr_end(host) -i www.test.com
  acl is_broker_test_https hdr_end(host) -i broker.test.com
  acl is_eclose_test_https hdr_end(host) -i eclose.test.com
  use_backend is_test_https_backend if  is_test_https
  use_backend is_broker_https_backendif
  is_broker_test_https
  use_backend is_eclose_https_backend  if is_eclose_test_https
 
 
 
  # backends
 
  backend is_test_https_backend
  mode http
  balance source
  option http-server-close
  server server1 192.168.33.70
  server server2 192.168.33.170
 
  backend is_broker_https_backend
  mode http
  balance source
  option http-server-close
  server server1 192.168.33.71
  server server2 192.168.33.171
 
  backend is_eclose_https_backend
  mode http
  balance source
  option http-server-close
  server server1 192.168.33.72
  server server2 192.168.33.172
 
  backend is_images_test
  mode http
  balance source
  option http-server-close
  server server1 192.168.33.73
  server server2 192.168.33.13
 
 
  I think I'm close, but just not sure if I'm sanely doing things.  I've
 tried
  to put piece of information together from several different posts around
 the
  Internet, but I have found nothing that is concise enough to really make
 me
  understand what I'm doing wrong.
 
  Thank you SO much,
 
  Chris


 configuration out of my head, with no testing, but should work.

 Baptiste

Re: I'm so close (I Think) - need a little help finishing this - Haproxy SSL offloading with ACLs and a wildcard cert

[Baptiste]

Hi Chris,

My answers inline.

On Mon, Oct 21, 2013 at 10:57 PM, Chris bludge...@gmail.com wrote:
 - If any of these domains and subdomains are called with http://, they need
 to be redirected to https://

# redirect http to https when connection is not ciphered
http-request redirect scheme https if !{ ssl_fc } { hdr(host)
www.test.com broker.test.com eclose.test.com images.test.com }


 - If a call comes in just to test.com, it needs to be redirected to https
 www.test.com:443

# redirect text.com:80 to https://www.test.com:443
http-request redirect prefix https://www.test.com if !{ ssl_fc } {
hdr(host) test.com }


 - Sessions need to stay alive after being offloaded to the backend web
 servers to port 80 on them.  (Session keepalive or stickiness?)

do you mean persistence or connection keepalives ???


 - images.test.com does not need to have SSL offloaded and I'm wondering if I
 should incorporate this into haproxy at all, maybe for load balancing at
 some point, this seems like a good idea)

well, this is not what you asked in your first question, you wanted to
offload SSL on all your domains.
That said, I agree, ciphering images (usually) is useless.

# redirect http to https when connection is not ciphered
http-request redirect scheme https if !{ ssl_fc } { hdr(host)
www.test.com broker.test.com eclose.test.com }

since images.test.com is not in the list, the traffic will be accepted
on port 80.

 defaults
 mode http
 maxconn 512
 option dontlognull
 option http-server-close
 retries 3
 contimeout 6
 clitimeout 6
 srvtimeout 6

 frontend haproxy_http
 bind 0.0.0.0:80
 timeout client 8640
 acl is_test hdr_end(host) -i www.test.com
 acl is_broker_test hdr_end(host) -i broker.test.com
 acl is_eclose_test  hdr_end(host) -i eclose.test.com
 acl is_images_test  hdr_end(host)  -i images.test.com


  # redirect http to https when connection is not ciphered
  http-request redirect scheme https if !{ ssl_fc } { hdr(host)
www.test.com broker.test.com eclose.test.com images.test.com }
   # redirect text.com:80 to https://www.test.com:443
  http-request redirect prefix https://www.test.com if !{ ssl_fc }
{ hdr(host) test.com }

 use_backend is_images_test if { hdr(host) images.test.com }




 frontend haproxy_https
 bind 0.0.0.0:443 ssl crt /etc/haproxy/certs/wildcard.pem
 log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %CC\
 %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\
 {%[ssl_c_verify],%{+Q}[ssl_c_s_dn],{+Q[ssl_c_i_dn]}\ %{+Q}r
 acl is_test_https hdr_end(host) -i www.test.com
 acl is_broker_test_https hdr_end(host) -i broker.test.com
 acl is_eclose_test_https hdr_end(host) -i eclose.test.com
 use_backend is_test_https_backend if  is_test_https
 use_backend is_broker_https_backendif  is_broker_test_https
 use_backend is_eclose_https_backend  if is_eclose_test_https



 # backends

 backend is_test_https_backend
 mode http
 balance source
 option http-server-close
 server server1 192.168.33.70
 server server2 192.168.33.170

 backend is_broker_https_backend
 mode http
 balance source
 option http-server-close
 server server1 192.168.33.71
 server server2 192.168.33.171

 backend is_eclose_https_backend
 mode http
 balance source
 option http-server-close
 server server1 192.168.33.72
 server server2 192.168.33.172

 backend is_images_test
 mode http
 balance source
 option http-server-close
 server server1 192.168.33.73
 server server2 192.168.33.13


 I think I'm close, but just not sure if I'm sanely doing things.  I've tried
 to put piece of information together from several different posts around the
 Internet, but I have found nothing that is concise enough to really make me
 understand what I'm doing wrong.

 Thank you SO much,

 Chris


configuration out of my head, with no testing, but should work.

Baptiste

I'm so close (I Think) - need a little help finishing this - Haproxy SSL offloading with ACLs and a wildcard cert

[Chris]

Hello,

I was hoping someone could help me out here, I'm fairly new to haproxy and
what I think I need should be fairly simple to do for someone that has some
experience with it, but for me,  I am not having the best of luck.

*Here is my scenario:*

I have an haproxy server set up that I want to use for SSL offloading, I
have compiled it with 1.5-dev19 and added the SSL options into the build
out, this part is done.


I simply need this haproxy to redirect port 80 traffic to 443 for 3 of the
4 sub-domains listed below, and then the haproxy server to offload SSL to
some backend web servers s via a wildcard cert.  I have generated a
wildcard cert for testing already and compiled together the PEM file that
haproxy currently sees as OK.



*Said domain:  *  test.com

*Sub-domains: *   www.test.com
 broker.test.com
eclose.test.com
 images.test.com  (globally open to everyone on port 80, no ssl offloading
needed)

*Requirements:   *

- If any of these domains and subdomains are called with http://, they need
to be redirected to https://
 - If a call comes in just to test.com, it needs to be redirected to https
www.test.com:443
 - Sessions need to stay alive after being offloaded to the backend web
servers to port 80 on them.  (Session keepalive or stickiness?)
 - images.test.com does not need to have SSL offloaded and I'm wondering if
I should incorporate this into haproxy at all, maybe for load balancing at
some point, this seems like a good idea)


I want to make it known that I have tried giving this a shot using the ACLs
for haproxy and
attempting to use the hdr_end(host) statements, but I am not certain if
this is correct, maybe I should be using hdr_begin(host)

As humbling as this might be, I will provide my best attempt at getting
this to work, maybe I'm way off, maybe I'm close, but I'm at my wit's end
on this and was hoping the community of professional might be able to steer
me in the right direction, below is my attempt at getting the above
scenario to work.

I thank you very much for your time and help.


*--- haproxy.conf ---*



defaults
mode http
maxconn 512
option dontlognull
option http-server-close  # session stickiness/persistence?
retries 3
contimeout 6
clitimeout 6
srvtimeout 6

frontend haproxy_http
bind 0.0.0.0:80 http://0.0.0.0/
timeout client 8640
acl is_test hdr_end(host) -i www.test.com
acl is_broker_test hdr_end(host) -i broker.test.com
acl is_eclose_test  hdr_end(host) -i eclose.test.com
acl is_images_test  hdr_end(host)  -i images.test.com


redirect location https ://www.test.com   if is_test
redirect location   https ://broker.test.com if is_broker_test
redirect location https ://eclose.test.com if is_eclose_test
use_backend is_images_test if is_images_test



frontend haproxy_https
bind 0.0.0.0:443 ssl crt /etc/haproxy/certs/wildcard.pem
log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\
%CC\ %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\
{%[ssl_c_verify],%{+Q}[ssl_c_s_dn],{+Q[ssl_c_i_dn]}\ %{+Q}r
acl is_test_https hdr_end(host) -i www.test.com
acl is_broker_test_https hdr_end(host) -i broker.test.com
acl is_eclose_test_https hdr_end(host) -i eclose.test.com
use_backend is_test_https_backend if  is_test_https
use_backend is_broker_https_backendif  is_broker_test_https
use_backend is_eclose_https_backend  if is_eclose_test_https



# backends

backend is_test_https_backend
mode http
balance source
option http-server-close
server server1 192.168.33.70
server server2 192.168.33.170

backend is_broker_https_backend
mode http
balance source
option http-server-close
server server1 192.168.33.71
server server2 192.168.33.171

backend is_eclose_https_backend
mode http
balance source
option http-server-close
server server1 192.168.33.72
server server2 192.168.33.172

backend is_images_test
mode http
balance source
option http-server-close
server server1 192.168.33.73
server server2 192.168.33.13


 I think I'm close, but just not sure if I'm sanely doing things.  I've
tried to put piece of information together from several different posts
around the Internet, but I have found nothing that is concise enough to
really make me understand what I'm doing wrong.

Thank you SO much,

Chris

SSL offloading configuration

[Chris Sarginson]

Hi,

Are there any plans to allow HAProxy to take the traffic that it can now 
SSL offload, perform header analysis, and then use an SSL encrypted 
connection to the backend server?


I have a situation where I need to be able to use ACLs against SSL 
encrypted traffic, but then continue passing the traffic to the backend 
over an encrypted connection.  This is specifically a security concern, 
rather than an issue with poor code.


Cheers
Chris

Re: SSL offloading configuration

[PiBa-NL]

Hi Chriss,

That seams possible already.?.
If you have the configuration for SSL offloading configured already all 
you need to add is the ssl option to your backend servers.


-- 
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5.2 
--

*ssl http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5-ssl*

This option enables SSL ciphering on outgoing connections to the server. At
the moment, server certificates are not checked, so this is prone to man in
the middle attacks. The real intended use is to permit SSL communication
with software which cannot work in other modes over networks that would
otherwise be considered safe enough for clear text communications. When this
option is used, health checks are automatically sent in SSL too unless there
is a port  http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#port or 
anaddr  http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#addr directive 
indicating the check should be sent to a
different location. See the check-ssl  
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#check-ssl optino to 
force SSL health checks.

--

Op 30-4-2013 14:47, Chris Sarginson schreef:

Hi,

Are there any plans to allow HAProxy to take the traffic that it can 
now SSL offload, perform header analysis, and then use an SSL 
encrypted connection to the backend server?


I have a situation where I need to be able to use ACLs against SSL 
encrypted traffic, but then continue passing the traffic to the 
backend over an encrypted connection.  This is specifically a security 
concern, rather than an issue with poor code.


Cheers
Chris

Re: SSL offloading configuration

[David Coulson]

Haproxy 1.5-Dev can do this already

Sent from my iPhone

On Apr 30, 2013, at 8:47 AM, Chris Sarginson ch...@sargy.co.uk wrote:

 Hi,
 
 Are there any plans to allow HAProxy to take the traffic that it can now SSL 
 offload, perform header analysis, and then use an SSL encrypted connection to 
 the backend server?
 
 I have a situation where I need to be able to use ACLs against SSL encrypted 
 traffic, but then continue passing the traffic to the backend over an 
 encrypted connection.  This is specifically a security concern, rather than 
 an issue with poor code.
 
 Cheers
 Chris
 

Re: SSL offloading configuration

[Chris Sarginson]

That's AWESOME!  Can't believe I didn't think of that, thanks a lot 
guys :)


Chris

On 30/04/2013 13:53, PiBa-NL wrote:

Hi Chriss,

That seams possible already.?.
If you have the configuration for SSL offloading configured already 
all you need to add is the ssl option to your backend servers.


-- 
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5.2 
--

*ssl http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5-ssl*
This option enables SSL ciphering on outgoing connections to the server. At
the moment, server certificates are not checked, so this is prone to man in
the middle attacks. The real intended use is to permit SSL communication
with software which cannot work in other modes over networks that would
otherwise be considered safe enough for clear text communications. When this
option is used, health checks are automatically sent in SSL too unless there
is a port  http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#port or 
anaddr  http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#addr directive 
indicating the check should be sent to a
different location. See the check-ssl  
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#check-ssl optino to 
force SSL health checks.
--

Op 30-4-2013 14:47, Chris Sarginson schreef:

Hi,

Are there any plans to allow HAProxy to take the traffic that it can 
now SSL offload, perform header analysis, and then use an SSL 
encrypted connection to the backend server?


I have a situation where I need to be able to use ACLs against SSL 
encrypted traffic, but then continue passing the traffic to the 
backend over an encrypted connection.  This is specifically a 
security concern, rather than an issue with poor code.


Cheers
Chris

Re: SSL offloading with NTLM auth

[Baptiste]

Could you please remove this pretent keepalive option from your
configuration and give it a try?
HAProxy may close the connection because of it.
And yes, a tcpdump between haproxy and the CAS server may help as well.

cheers


On Fri, Feb 1, 2013 at 7:11 AM, Roland r...@bayreuth.tk wrote:
 Hi Baptiste,

 thanks a lot!

 If I connect the same computer with the same account and unchanged settings
 (except the URL of webaccess) directly to the CAS it works without any
 problems. Connection is established immediately.

 I also verified with Microsoft Remote Connectivity Analyzer. It stops with
 an error:

 =
 Attempting to ping RPC proxy mc.nkd.com.
 RPC Proxy can't be pinged.

 Additional Details
 An HTTP 401 Unauthorized response was received from the remote
 Unknown server. This is usually the result of an incorrect username or
 password. If you are attempting to log onto an Office 365 service, ensure
 you are using your full User Principal Name (UPN).
 =

 All tests before (HTTP authentication methods, IIS configuration, SSL
 credentials a.s.o.) are running fine.

 I'm absoultely clueless.
 I think I'll try to narrow down the problem with tcpdump- maybe the
 connection is forcably closed on some side.

 Cheers,
 Roland




 On Thu, 31 Jan 2013, Baptiste wrote:

 Hi,

 401 is absolutely normal in NTLM.
 There are 2 or 3 request/response before the user is really
 authenticated when using NTLM.
 When HAProxy load-balances NTLM based services, the only log line
 you'll see will be 401 errors.
 Even if the connection works properly.
 This is due to the tunnel mode, which seems to be properly
 configured in your conf, as far as I can see.
 In tunnel mode, haproxy analyzes the first request, logs the first
 response, (hence the 401) and creates a tunnel between the client and
 the server. From now, on this connection, HAProxy will only transmit
 payload, even if that's HTTP, nothing will be analyzed anymore.
 The tunnel mode is mandatory for NTLM, because if you change TCP
 source port during the connection, it brakes the authentication.

 Could you confirm your outlook session works? I mean that your client
 is well connected to your exchange server?

 I can confirm HAProxy works properly with Exchange 2010 and with 2013 as
 well.

 Cheers



 On Thu, Jan 31, 2013 at 4:13 PM, Roland r...@bayreuth.tk wrote:

 Hi!

 I'm using haproxy 1.5dev17 and try to balance traffic destined for MS
 Exchange 2010 CAS servers.
 OWA and ActiveSync are working without any problems- but Outlook Anywhere
 (RPC over HTTP with NTLM auth) produces an error 401 even with Microsofts
 Remote Connectivity Analyzer.

 HAProxy runs in SSL offload mode. The cert is an officialy signed one.
 My haproxy.conf is (partially):
 ...
 defaults
 modehttp
 maxconn 5
 contimeout  4000
 clitimeout  5
 srvtimeout  5
 balance roundrobin
 log global
 option  tcplog
 option  redispatch
 option contstats
 option dontlognull
 timeout connect 5s
 timeout http-keep-alive 5s
 timeout http-request 15s
 timeout queue 30s
 timeout client 300s
 timeout server 300s
 default-server inter 3s rise 2 fall 3
 backlog 1
 option http-pretend-keepalive

 frontendWebAccess
 maxconn 5
 bind172.17.336.433:666 ssl crt
 /usr/local/etc/haproxy-certs/mc.dom.com.pem
 modehttp
 option httplog
 log global
 no option httpclose
 acl ACLRPC  path_beg -i /rpc/rpcproxy.dll
 use_backend OutlookAnywhere if ACLRPC
 ...

 backend OutlookAnywhere
 stick-table type ip size 10240k expire 60m
 stick on src
 cookie SRV insert nocache
 balance roundrobin
 option redispatch
 server juno 172.17.336.433:80 cookie oasrv1 weight 1 check
 ...

 The one active CAS server used for testing purposes (juno) is configured
 for
 SSL offloading for RPC. All other Exchange directories in IIS are set to
 not
 require SSL on this system.

 When running HAProxy in debug mode an Outlook Anywhere session looks
 like:
 0005:WebAccess.clireq[000d:]: RPC_IN_DATA
 /Rpc/RpcProxy.dll?lips.dom.intl:6001 HTTP/1.1
 0005:WebAccess.clihdr[000d:]: Accept: application/rpc
 0005:WebAccess.clihdr[000d:]: User-Agent: MSRPC
 0005:WebAccess.clihdr[000d:]: Authorization: NTLM
 TlRMTVNTUAABB4IIogAGAbEdDw==
 0005:WebAccess.clihdr[000d:]: Host: mc.dom.com
 0005:WebAccess.clihdr[000d:]: Content-Length: 0
 0005:OutlookAnywhere.srvrep[000d:000e]: HTTP/1.1 401 Unauthorized
 0005:OutlookAnywhere.srvhdr[000d:000e]: Content-Type: text/html
 0005:OutlookAnywhere.srvhdr[000d:000e]: Server: Microsoft-IIS/7.5
 0005:OutlookAnywhere.srvhdr[000d:000e]: WWW-Authenticate: NTLM

SSL offloading with NTLM auth

[Roland]

Hi!

I'm using haproxy 1.5dev17 and try to balance traffic destined for MS 
Exchange 2010 CAS servers.
OWA and ActiveSync are working without any problems- but Outlook Anywhere 
(RPC over HTTP with NTLM auth) produces an error 401 even with Microsofts 
Remote Connectivity Analyzer.


HAProxy runs in SSL offload mode. The cert is an officialy signed one.
My haproxy.conf is (partially):
...
defaults
modehttp
maxconn 5
contimeout  4000
clitimeout  5
srvtimeout  5
balance roundrobin
log global
option  tcplog
option  redispatch
option contstats
option dontlognull
timeout connect 5s
timeout http-keep-alive 5s
timeout http-request 15s
timeout queue 30s
timeout client 300s
timeout server 300s
default-server inter 3s rise 2 fall 3
backlog 1
option http-pretend-keepalive

frontendWebAccess
maxconn 5
bind172.17.336.433:666 ssl crt 
/usr/local/etc/haproxy-certs/mc.dom.com.pem

modehttp
option httplog
log global
no option httpclose
acl ACLRPC  path_beg -i /rpc/rpcproxy.dll
use_backend OutlookAnywhere if ACLRPC
...

backend OutlookAnywhere
stick-table type ip size 10240k expire 60m
stick on src
cookie SRV insert nocache
balance roundrobin
option redispatch
server juno 172.17.336.433:80 cookie oasrv1 weight 1 check
...

The one active CAS server used for testing purposes (juno) is configured 
for SSL offloading for RPC. All other Exchange directories in IIS are set 
to not require SSL on this system.


When running HAProxy in debug mode an Outlook Anywhere session looks like:
0005:WebAccess.clireq[000d:]: RPC_IN_DATA 
/Rpc/RpcProxy.dll?lips.dom.intl:6001 HTTP/1.1

0005:WebAccess.clihdr[000d:]: Accept: application/rpc
0005:WebAccess.clihdr[000d:]: User-Agent: MSRPC
0005:WebAccess.clihdr[000d:]: Authorization: NTLM 
TlRMTVNTUAABB4IIogAGAbEdDw==

0005:WebAccess.clihdr[000d:]: Host: mc.dom.com
0005:WebAccess.clihdr[000d:]: Content-Length: 0
0005:OutlookAnywhere.srvrep[000d:000e]: HTTP/1.1 401 Unauthorized
0005:OutlookAnywhere.srvhdr[000d:000e]: Content-Type: text/html
0005:OutlookAnywhere.srvhdr[000d:000e]: Server: Microsoft-IIS/7.5
0005:OutlookAnywhere.srvhdr[000d:000e]: WWW-Authenticate: NTLM 
TlRMTVNTUAACBgAGADgFgomiDMdfGnUDCJUAAGwAbAA+BgGxHQ9OAEsARAACAAYATgBLAEQAAQAIAEoAVQBOAE8ABAAQAG4AawBkAC4AaQBuAHQAbAADABoAagB1AG4AbwAuAG4AawBkAC4AaQBuAHQAbAAFABAAbgBrAGQALgBpAG4AdABsAAcACADUFzdfwP/NAQA=

0005:OutlookAnywhere.srvhdr[000d:000e]: WWW-Authenticate: Negotiate
0005:OutlookAnywhere.srvhdr[000d:000e]: X-Powered-By: ASP.NET
0005:OutlookAnywhere.srvhdr[000d:000e]: Date: Thu, 31 Jan 2013 
14:36:35 GMT

0005:OutlookAnywhere.srvhdr[000d:000e]: Content-Length: 58

I suspect NTLM to be the culprit.
Even after searching through all possible ressources I cannot find a 
solution for this problem. As you can see in the config above I already 
tried to implement some kind of keep-alive- with no success at all.


If I bypass HAProxy or change HAProxy config to mode tcp everything is 
fine.


Have anyone had this kind of problem already? Or maybe some similar?

Best regards,
Roland

Re: SSL offloading with NTLM auth

[Baptiste]

Hi,

401 is absolutely normal in NTLM.
There are 2 or 3 request/response before the user is really
authenticated when using NTLM.
When HAProxy load-balances NTLM based services, the only log line
you'll see will be 401 errors.
Even if the connection works properly.
This is due to the tunnel mode, which seems to be properly
configured in your conf, as far as I can see.
In tunnel mode, haproxy analyzes the first request, logs the first
response, (hence the 401) and creates a tunnel between the client and
the server. From now, on this connection, HAProxy will only transmit
payload, even if that's HTTP, nothing will be analyzed anymore.
The tunnel mode is mandatory for NTLM, because if you change TCP
source port during the connection, it brakes the authentication.

Could you confirm your outlook session works? I mean that your client
is well connected to your exchange server?

I can confirm HAProxy works properly with Exchange 2010 and with 2013 as well.

Cheers



On Thu, Jan 31, 2013 at 4:13 PM, Roland r...@bayreuth.tk wrote:
 Hi!

 I'm using haproxy 1.5dev17 and try to balance traffic destined for MS
 Exchange 2010 CAS servers.
 OWA and ActiveSync are working without any problems- but Outlook Anywhere
 (RPC over HTTP with NTLM auth) produces an error 401 even with Microsofts
 Remote Connectivity Analyzer.

 HAProxy runs in SSL offload mode. The cert is an officialy signed one.
 My haproxy.conf is (partially):
 ...
 defaults
 modehttp
 maxconn 5
 contimeout  4000
 clitimeout  5
 srvtimeout  5
 balance roundrobin
 log global
 option  tcplog
 option  redispatch
 option contstats
 option dontlognull
 timeout connect 5s
 timeout http-keep-alive 5s
 timeout http-request 15s
 timeout queue 30s
 timeout client 300s
 timeout server 300s
 default-server inter 3s rise 2 fall 3
 backlog 1
 option http-pretend-keepalive

 frontendWebAccess
 maxconn 5
 bind172.17.336.433:666 ssl crt
 /usr/local/etc/haproxy-certs/mc.dom.com.pem
 modehttp
 option httplog
 log global
 no option httpclose
 acl ACLRPC  path_beg -i /rpc/rpcproxy.dll
 use_backend OutlookAnywhere if ACLRPC
 ...

 backend OutlookAnywhere
 stick-table type ip size 10240k expire 60m
 stick on src
 cookie SRV insert nocache
 balance roundrobin
 option redispatch
 server juno 172.17.336.433:80 cookie oasrv1 weight 1 check
 ...

 The one active CAS server used for testing purposes (juno) is configured for
 SSL offloading for RPC. All other Exchange directories in IIS are set to not
 require SSL on this system.

 When running HAProxy in debug mode an Outlook Anywhere session looks like:
 0005:WebAccess.clireq[000d:]: RPC_IN_DATA
 /Rpc/RpcProxy.dll?lips.dom.intl:6001 HTTP/1.1
 0005:WebAccess.clihdr[000d:]: Accept: application/rpc
 0005:WebAccess.clihdr[000d:]: User-Agent: MSRPC
 0005:WebAccess.clihdr[000d:]: Authorization: NTLM
 TlRMTVNTUAABB4IIogAGAbEdDw==
 0005:WebAccess.clihdr[000d:]: Host: mc.dom.com
 0005:WebAccess.clihdr[000d:]: Content-Length: 0
 0005:OutlookAnywhere.srvrep[000d:000e]: HTTP/1.1 401 Unauthorized
 0005:OutlookAnywhere.srvhdr[000d:000e]: Content-Type: text/html
 0005:OutlookAnywhere.srvhdr[000d:000e]: Server: Microsoft-IIS/7.5
 0005:OutlookAnywhere.srvhdr[000d:000e]: WWW-Authenticate: NTLM
 TlRMTVNTUAACBgAGADgFgomiDMdfGnUDCJUAAGwAbAA+BgGxHQ9OAEsARAACAAYATgBLAEQAAQAIAEoAVQBOAE8ABAAQAG4AawBkAC4AaQBuAHQAbAADABoAagB1AG4AbwAuAG4AawBkAC4AaQBuAHQAbAAFABAAbgBrAGQALgBpAG4AdABsAAcACADUFzdfwP/NAQA=
 0005:OutlookAnywhere.srvhdr[000d:000e]: WWW-Authenticate: Negotiate
 0005:OutlookAnywhere.srvhdr[000d:000e]: X-Powered-By: ASP.NET
 0005:OutlookAnywhere.srvhdr[000d:000e]: Date: Thu, 31 Jan 2013 14:36:35
 GMT
 0005:OutlookAnywhere.srvhdr[000d:000e]: Content-Length: 58

 I suspect NTLM to be the culprit.
 Even after searching through all possible ressources I cannot find a
 solution for this problem. As you can see in the config above I already
 tried to implement some kind of keep-alive- with no success at all.

 If I bypass HAProxy or change HAProxy config to mode tcp everything is
 fine.

 Have anyone had this kind of problem already? Or maybe some similar?

 Best regards,
 Roland

Re: SSL offloading with NTLM auth

[Roland]

Hi Baptiste,

thanks a lot!

If I connect the same computer with the same account and unchanged 
settings (except the URL of webaccess) directly to the CAS it works 
without any problems. Connection is established immediately.


I also verified with Microsoft Remote Connectivity Analyzer. It stops with 
an error:


=
Attempting to ping RPC proxy mc.nkd.com.
RPC Proxy can't be pinged.

Additional Details
 	An HTTP 401 Unauthorized response was received from the remote 
Unknown server. This is usually the result of an incorrect username or 
password. If you are attempting to log onto an Office 365 service, ensure 
you are using your full User Principal Name (UPN).

=

All tests before (HTTP authentication methods, IIS configuration, SSL 
credentials a.s.o.) are running fine.


I'm absoultely clueless.
I think I'll try to narrow down the problem with tcpdump- maybe the 
connection is forcably closed on some side.


Cheers,
Roland



On Thu, 31 Jan 2013, Baptiste wrote:


Hi,

401 is absolutely normal in NTLM.
There are 2 or 3 request/response before the user is really
authenticated when using NTLM.
When HAProxy load-balances NTLM based services, the only log line
you'll see will be 401 errors.
Even if the connection works properly.
This is due to the tunnel mode, which seems to be properly
configured in your conf, as far as I can see.
In tunnel mode, haproxy analyzes the first request, logs the first
response, (hence the 401) and creates a tunnel between the client and
the server. From now, on this connection, HAProxy will only transmit
payload, even if that's HTTP, nothing will be analyzed anymore.
The tunnel mode is mandatory for NTLM, because if you change TCP
source port during the connection, it brakes the authentication.

Could you confirm your outlook session works? I mean that your client
is well connected to your exchange server?

I can confirm HAProxy works properly with Exchange 2010 and with 2013 as well.

Cheers



On Thu, Jan 31, 2013 at 4:13 PM, Roland r...@bayreuth.tk wrote:

Hi!

I'm using haproxy 1.5dev17 and try to balance traffic destined for MS
Exchange 2010 CAS servers.
OWA and ActiveSync are working without any problems- but Outlook Anywhere
(RPC over HTTP with NTLM auth) produces an error 401 even with Microsofts
Remote Connectivity Analyzer.

HAProxy runs in SSL offload mode. The cert is an officialy signed one.
My haproxy.conf is (partially):
...
defaults
modehttp
maxconn 5
contimeout  4000
clitimeout  5
srvtimeout  5
balance roundrobin
log global
option  tcplog
option  redispatch
option contstats
option dontlognull
timeout connect 5s
timeout http-keep-alive 5s
timeout http-request 15s
timeout queue 30s
timeout client 300s
timeout server 300s
default-server inter 3s rise 2 fall 3
backlog 1
option http-pretend-keepalive

frontendWebAccess
maxconn 5
bind172.17.336.433:666 ssl crt
/usr/local/etc/haproxy-certs/mc.dom.com.pem
modehttp
option httplog
log global
no option httpclose
acl ACLRPC  path_beg -i /rpc/rpcproxy.dll
use_backend OutlookAnywhere if ACLRPC
...

backend OutlookAnywhere
stick-table type ip size 10240k expire 60m
stick on src
cookie SRV insert nocache
balance roundrobin
option redispatch
server juno 172.17.336.433:80 cookie oasrv1 weight 1 check
...

The one active CAS server used for testing purposes (juno) is configured for
SSL offloading for RPC. All other Exchange directories in IIS are set to not
require SSL on this system.

When running HAProxy in debug mode an Outlook Anywhere session looks like:
0005:WebAccess.clireq[000d:]: RPC_IN_DATA
/Rpc/RpcProxy.dll?lips.dom.intl:6001 HTTP/1.1
0005:WebAccess.clihdr[000d:]: Accept: application/rpc
0005:WebAccess.clihdr[000d:]: User-Agent: MSRPC
0005:WebAccess.clihdr[000d:]: Authorization: NTLM
TlRMTVNTUAABB4IIogAGAbEdDw==
0005:WebAccess.clihdr[000d:]: Host: mc.dom.com
0005:WebAccess.clihdr[000d:]: Content-Length: 0
0005:OutlookAnywhere.srvrep[000d:000e]: HTTP/1.1 401 Unauthorized
0005:OutlookAnywhere.srvhdr[000d:000e]: Content-Type: text/html
0005:OutlookAnywhere.srvhdr[000d:000e]: Server: Microsoft-IIS/7.5
0005:OutlookAnywhere.srvhdr[000d:000e]: WWW-Authenticate: NTLM
TlRMTVNTUAACBgAGADgFgomiDMdfGnUDCJUAAGwAbAA+BgGxHQ9OAEsARAACAAYATgBLAEQAAQAIAEoAVQBOAE8ABAAQAG4AawBkAC4AaQBuAHQAbAADABoAagB1AG4AbwAuAG4AawBkAC4AaQBuAHQAbAAFABAAbgBrAGQALgBpAG4AdABsAAcACADUFzdfwP/NAQA=
0005:OutlookAnywhere.srvhdr[000d:000e]: WWW-Authenticate: Negotiate
0005:OutlookAnywhere.srvhdr[000d:000e]: X-Powered-By: ASP.NET
0005:OutlookAnywhere.srvhdr