Re: How to forward HTTP / HTTPS to different backend proxy servers

[Michael Ezzell]

On Jul 2, 2017 8:41 PM, "Daren Sefcik"  wrote:

yep, pretty much..I just need some help to figure out how to make it
work

example log entries for https and http, you can see how the "443" goes to
one backenad and the regular http "GET" request goes to another..but this
is not consistent and I know there has to be a better way..


use_backend HTPL_WEB_PROXY_http_ipvANY   if { meth_connect }

Or maybe...

use_backend HTPL_WEB_PROXY_http_ipvANY  if { meth_connect } !{ path_end :80
}

That should be all you need.

HTTPS through an HTTP proxy via HAProxy isn't an SSL session that HAProxy
can see.  It's an opaque tunnel, requested over HTTP, using CONNECT.

If the browser asks for a tunnel, it should be because it's wanting to
speak HTTPS once the target is connected.

Re: How to forward HTTP / HTTPS to different backend proxy servers

[Daren Sefcik]

On Sun, Jul 2, 2017 at 6:10 PM, Igor Cicimov  wrote:

>
>
> On Mon, Jul 3, 2017 at 10:38 AM, Daren Sefcik 
> wrote:
>
>>
>> On Sun, Jul 2, 2017 at 4:44 PM, Michael Ezzell 
>> wrote:
>>
>>>
>>>
>>> On Jul 2, 2017 19:15, "Daren Sefcik"  wrote:
>>>
>>>
>>> Most of the traffic is ssl, for example gmail, facebook, pandora all
>>> force https.
>>>
>>>
>>> I'm going to go out on a limb and suggest that *none* of the traffic is
>>> SSL in any sense that is meaningful from HAProxy's perspective.
>>>
>>> What do the HTTPS requests look like in the HAProxy logs?  Aren't they
>>> CONNECT requests?
>>>
>>>
>> yep, pretty much..I just need some help to figure out how to make it
>> work
>>
>> example log entries for https and http, you can see how the "443" goes to
>> one backenad and the regular http "GET" request goes to another..but this
>> is not consistent and I know there has to be a better way..
>>
>> HTPL_PROXY HTPL_SSL_PROXY_http_ipvANY/HTPL-PROXY-03_10.1.4.180
>> 0/0/0/22/10075 200 525 - - cD-- 124/124/103/103/0 0/0 "CONNECT
>> caltopo.com:443 HTTP/1.1"
>>
>> HTPL_PROXY HTPL_WEB_PROXY_http_ipvANY/HTPL-PROXY-04_10.1.4.181
>> 92/0/0/1/93 403 4309 - -  126/126/10/11/0 0/0 "GET
>> http://i2.wp.com/n4.nabble.com/images/avatar100.png HTTP/1.1"
>>
>>
>> TIA for any help with this..!
>>
>
> ​Is it possible that *some* of the clients have issues talking to the
> haproxy over ssl? You say in case of ssl it is not 100% successful but what
> does that mean? How does this manifest? Can you track the ssl request from
> particular client ending up on the http backend?
>

All clients talk to haproxy 100% fine. What I mean by not 100% is that
using that particular acl to try and determine if it is ssl traffic or not
is not 100% in haproxy. Maybe 80% percent of the time haproxy sends the
traffic to the intended backend and 20% of the time to the otherbut
100% of all traffic goes to one backend or the other.


TIA...

Re: How to forward HTTP / HTTPS to different backend proxy servers

[Igor Cicimov]

On Mon, Jul 3, 2017 at 9:44 AM, Michael Ezzell  wrote:

>
>
> On Jul 2, 2017 19:15, "Daren Sefcik"  wrote:
>
>
> Most of the traffic is ssl, for example gmail, facebook, pandora all force
> https.
>
>
> I'm going to go out on a limb and suggest that *none* of the traffic is
> SSL in any sense that is meaningful from HAProxy's perspective.
>
> What do the HTTPS requests look like in the HAProxy logs?  Aren't they
> CONNECT requests?
>

​I was going to go even further and ask for tcpdump capture on the haproxy
port :-)
 ​

Re: How to forward HTTP / HTTPS to different backend proxy servers

[Andrew Smalley]

I would like to ask why you have non ssl and ssl traffic on the same port?
while it seems it is possible it is not the right way to do it.


On 2 Jul 2017 23:37, "Igor Cicimov"  wrote:



On 3 Jul 2017 8:35 am, "Igor Cicimov" 
wrote:



On 3 Jul 2017 6:47 am, "Daren Sefcik"  wrote:

On Sat, Jul 1, 2017 at 4:39 PM, Igor Cicimov  wrote:

>
>
> On 29 Jun 2017 2:46 am, "Daren Sefcik"  wrote:
>
> On Wed, Jun 28, 2017 at 8:12 AM, Olivier Doucet 
> wrote:
>
>> Hi,
>>
>>
>> 2017-06-28 16:47 GMT+02:00 Daren Sefcik :
>>
>>> Hi, I have searched for an answer to this and tried several things but
>>> cannot seem to figure it out so am hoping someone can point me in the right
>>> direction. I have different backend proxy servers (squid) setup to handle
>>> specifically HTTP and HTTPS traffic but cannot figure out how to tell
>>> haproxy to tell the difference and send appropriately.
>>>
>>> For example, I have
>>>
>>> frontend proxy_servers
>>> backend http_proxies
>>> backend https_proxies
>>>
>>> how can I tell frontend to send all http traffic to backend http_proxies
>>> and all https traffic to https_backend? I have tried using dst_port 443 and
>>> the acl https ssl_fc but nothing seems to distinguish https traffic.
>>>
>>
>> Well, it should work. Send a copy of your config to see what's wrong in
>> it.
>>
>> Olivier
>>
>>
>>
>>>
>>> TIA...
>>>
>>
>>
> Here is an example, it continues to direct all https traffic to the web
> proxy and not the streaming media one.
>
> frontend HTPL_PROXY
>   bind10.1.4.105:8181 name 10.1.4.105:8181
>   modehttp
>   log global
>   option  http-server-close
>   option  forwardfor
>   acl https ssl_fc
>   http-request set-header X-Forwarded-Proto http if !https
>   http-request set-header X-Forwarded-Proto https if https
>   maxconn 9
>   timeout client  1
>   option tcp-smart-accept
>   acl is_youtube  hdr_sub(host) -i youtube.com
>   acl is_netflix  hdr_sub(host) -i netflix.com
>   acl is_nflixvideo   hdr_sub(host) -i nflxvideo.net
>   acl is_googlevideo  hdr_sub(host) -i googlevideo.com
>   acl is_google   hdr_sub(host) -i google.com
>   acl is_pandora  hdr_sub(host) -i pandora.com
>   acl is_httpsdst_port eq 443
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_youtube
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_netflix
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_nflixvideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_googlevideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_pandora
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_https
>   default_backend HTPL_WEB_PROXY_http_ipvANY
>
> Obviously dst_port 443 method can not work since you are listening on port
> 8181. Since both protocols are on same port you can try in tcp mode:
>
> mode tcp
> option tcplog
> bind *:8181
>
> tcp-request inspect-delay 5s
> acl is_ssl req.ssl_hello_type 1
>
>

Thank you, I have tried that with the below config and it still sends all
traffic to the default backend instead of my ssl backend, any other ideas?

frontend HTPL_PROXY
bind10.1.4.105:8181 name 10.1.4.105:8181

modetcp
log global
maxconn 9
timeout client  1
option tcp-smart-accept
tcp-request inspect-delay 5s
acl is_ssl  req.ssl_hello_type 1
use_backend HTPL_SSL_PROXY_tcp_ipvANY  if  is_ssl
default_backend HTPL_WEB_PROXY_tcp_ipvANY

Only explenation i can see is that no ssl traffik is hitting haproxy at
least not on port 8181

Or the ip it is bind to

Re: How to forward HTTP / HTTPS to different backend proxy servers

[Igor Cicimov]

On 3 Jul 2017 8:35 am, "Igor Cicimov" 
wrote:



On 3 Jul 2017 6:47 am, "Daren Sefcik"  wrote:

On Sat, Jul 1, 2017 at 4:39 PM, Igor Cicimov  wrote:

>
>
> On 29 Jun 2017 2:46 am, "Daren Sefcik"  wrote:
>
> On Wed, Jun 28, 2017 at 8:12 AM, Olivier Doucet 
> wrote:
>
>> Hi,
>>
>>
>> 2017-06-28 16:47 GMT+02:00 Daren Sefcik :
>>
>>> Hi, I have searched for an answer to this and tried several things but
>>> cannot seem to figure it out so am hoping someone can point me in the right
>>> direction. I have different backend proxy servers (squid) setup to handle
>>> specifically HTTP and HTTPS traffic but cannot figure out how to tell
>>> haproxy to tell the difference and send appropriately.
>>>
>>> For example, I have
>>>
>>> frontend proxy_servers
>>> backend http_proxies
>>> backend https_proxies
>>>
>>> how can I tell frontend to send all http traffic to backend http_proxies
>>> and all https traffic to https_backend? I have tried using dst_port 443 and
>>> the acl https ssl_fc but nothing seems to distinguish https traffic.
>>>
>>
>> Well, it should work. Send a copy of your config to see what's wrong in
>> it.
>>
>> Olivier
>>
>>
>>
>>>
>>> TIA...
>>>
>>
>>
> Here is an example, it continues to direct all https traffic to the web
> proxy and not the streaming media one.
>
> frontend HTPL_PROXY
>   bind10.1.4.105:8181 name 10.1.4.105:8181
>   modehttp
>   log global
>   option  http-server-close
>   option  forwardfor
>   acl https ssl_fc
>   http-request set-header X-Forwarded-Proto http if !https
>   http-request set-header X-Forwarded-Proto https if https
>   maxconn 9
>   timeout client  1
>   option tcp-smart-accept
>   acl is_youtube  hdr_sub(host) -i youtube.com
>   acl is_netflix  hdr_sub(host) -i netflix.com
>   acl is_nflixvideo   hdr_sub(host) -i nflxvideo.net
>   acl is_googlevideo  hdr_sub(host) -i googlevideo.com
>   acl is_google   hdr_sub(host) -i google.com
>   acl is_pandora  hdr_sub(host) -i pandora.com
>   acl is_httpsdst_port eq 443
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_youtube
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_netflix
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_nflixvideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_googlevideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_pandora
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_https
>   default_backend HTPL_WEB_PROXY_http_ipvANY
>
> Obviously dst_port 443 method can not work since you are listening on port
> 8181. Since both protocols are on same port you can try in tcp mode:
>
> mode tcp
> option tcplog
> bind *:8181
>
> tcp-request inspect-delay 5s
> acl is_ssl req.ssl_hello_type 1
>
>

Thank you, I have tried that with the below config and it still sends all
traffic to the default backend instead of my ssl backend, any other ideas?

frontend HTPL_PROXY
bind10.1.4.105:8181 name 10.1.4.105:8181

modetcp
log global
maxconn 9
timeout client  1
option tcp-smart-accept
tcp-request inspect-delay 5s
acl is_ssl  req.ssl_hello_type 1
use_backend HTPL_SSL_PROXY_tcp_ipvANY  if  is_ssl
default_backend HTPL_WEB_PROXY_tcp_ipvANY

Only explenation i can see is that no ssl traffik is hitting haproxy at
least not on port 8181

Or the ip it is bind to

Re: How to forward HTTP / HTTPS to different backend proxy servers

[Igor Cicimov]

On 3 Jul 2017 6:47 am, "Daren Sefcik"  wrote:

On Sat, Jul 1, 2017 at 4:39 PM, Igor Cicimov  wrote:

>
>
> On 29 Jun 2017 2:46 am, "Daren Sefcik"  wrote:
>
> On Wed, Jun 28, 2017 at 8:12 AM, Olivier Doucet 
> wrote:
>
>> Hi,
>>
>>
>> 2017-06-28 16:47 GMT+02:00 Daren Sefcik :
>>
>>> Hi, I have searched for an answer to this and tried several things but
>>> cannot seem to figure it out so am hoping someone can point me in the right
>>> direction. I have different backend proxy servers (squid) setup to handle
>>> specifically HTTP and HTTPS traffic but cannot figure out how to tell
>>> haproxy to tell the difference and send appropriately.
>>>
>>> For example, I have
>>>
>>> frontend proxy_servers
>>> backend http_proxies
>>> backend https_proxies
>>>
>>> how can I tell frontend to send all http traffic to backend http_proxies
>>> and all https traffic to https_backend? I have tried using dst_port 443 and
>>> the acl https ssl_fc but nothing seems to distinguish https traffic.
>>>
>>
>> Well, it should work. Send a copy of your config to see what's wrong in
>> it.
>>
>> Olivier
>>
>>
>>
>>>
>>> TIA...
>>>
>>
>>
> Here is an example, it continues to direct all https traffic to the web
> proxy and not the streaming media one.
>
> frontend HTPL_PROXY
>   bind10.1.4.105:8181 name 10.1.4.105:8181
>   modehttp
>   log global
>   option  http-server-close
>   option  forwardfor
>   acl https ssl_fc
>   http-request set-header X-Forwarded-Proto http if !https
>   http-request set-header X-Forwarded-Proto https if https
>   maxconn 9
>   timeout client  1
>   option tcp-smart-accept
>   acl is_youtube  hdr_sub(host) -i youtube.com
>   acl is_netflix  hdr_sub(host) -i netflix.com
>   acl is_nflixvideo   hdr_sub(host) -i nflxvideo.net
>   acl is_googlevideo  hdr_sub(host) -i googlevideo.com
>   acl is_google   hdr_sub(host) -i google.com
>   acl is_pandora  hdr_sub(host) -i pandora.com
>   acl is_httpsdst_port eq 443
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_youtube
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_netflix
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_nflixvideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_googlevideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_pandora
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_https
>   default_backend HTPL_WEB_PROXY_http_ipvANY
>
> Obviously dst_port 443 method can not work since you are listening on port
> 8181. Since both protocols are on same port you can try in tcp mode:
>
> mode tcp
> option tcplog
> bind *:8181
>
> tcp-request inspect-delay 5s
> acl is_ssl req.ssl_hello_type 1
>
>

Thank you, I have tried that with the below config and it still sends all
traffic to the default backend instead of my ssl backend, any other ideas?

frontend HTPL_PROXY
bind10.1.4.105:8181 name 10.1.4.105:8181

modetcp
log global
maxconn 9
timeout client  1
option tcp-smart-accept
tcp-request inspect-delay 5s
acl is_ssl  req.ssl_hello_type 1
use_backend HTPL_SSL_PROXY_tcp_ipvANY  if  is_ssl
default_backend HTPL_WEB_PROXY_tcp_ipvANY

Only explenation i can see is that no ssl traffik is hitting haproxy at
least not on port 8181

Fix building haproxy with recent LibreSSL

[Dmitry Sivachenko]

Hello,

can you please take a look at proposed patch to fix build of haproxy with 
recent version of LibreSSL?

https://www.mail-archive.com/haproxy@formilux.org/msg25819.html

Thanks.

Re: How to forward HTTP / HTTPS to different backend proxy servers

[Daren Sefcik]

On Sat, Jul 1, 2017 at 4:39 PM, Igor Cicimov  wrote:

>
>
> On 29 Jun 2017 2:46 am, "Daren Sefcik"  wrote:
>
> On Wed, Jun 28, 2017 at 8:12 AM, Olivier Doucet 
> wrote:
>
>> Hi,
>>
>>
>> 2017-06-28 16:47 GMT+02:00 Daren Sefcik :
>>
>>> Hi, I have searched for an answer to this and tried several things but
>>> cannot seem to figure it out so am hoping someone can point me in the right
>>> direction. I have different backend proxy servers (squid) setup to handle
>>> specifically HTTP and HTTPS traffic but cannot figure out how to tell
>>> haproxy to tell the difference and send appropriately.
>>>
>>> For example, I have
>>>
>>> frontend proxy_servers
>>> backend http_proxies
>>> backend https_proxies
>>>
>>> how can I tell frontend to send all http traffic to backend http_proxies
>>> and all https traffic to https_backend? I have tried using dst_port 443 and
>>> the acl https ssl_fc but nothing seems to distinguish https traffic.
>>>
>>
>> Well, it should work. Send a copy of your config to see what's wrong in
>> it.
>>
>> Olivier
>>
>>
>>
>>>
>>> TIA...
>>>
>>
>>
> Here is an example, it continues to direct all https traffic to the web
> proxy and not the streaming media one.
>
> frontend HTPL_PROXY
>   bind10.1.4.105:8181 name 10.1.4.105:8181
>   modehttp
>   log global
>   option  http-server-close
>   option  forwardfor
>   acl https ssl_fc
>   http-request set-header X-Forwarded-Proto http if !https
>   http-request set-header X-Forwarded-Proto https if https
>   maxconn 9
>   timeout client  1
>   option tcp-smart-accept
>   acl is_youtube  hdr_sub(host) -i youtube.com
>   acl is_netflix  hdr_sub(host) -i netflix.com
>   acl is_nflixvideo   hdr_sub(host) -i nflxvideo.net
>   acl is_googlevideo  hdr_sub(host) -i googlevideo.com
>   acl is_google   hdr_sub(host) -i google.com
>   acl is_pandora  hdr_sub(host) -i pandora.com
>   acl is_httpsdst_port eq 443
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_youtube
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_netflix
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_nflixvideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_googlevideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_pandora
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_https
>   default_backend HTPL_WEB_PROXY_http_ipvANY
>
> Obviously dst_port 443 method can not work since you are listening on port
> 8181. Since both protocols are on same port you can try in tcp mode:
>
> mode tcp
> option tcplog
> bind *:8181
>
> tcp-request inspect-delay 5s
> acl is_ssl req.ssl_hello_type 1
>
>

Thank you, I have tried that with the below config and it still sends all
traffic to the default backend instead of my ssl backend, any other ideas?

frontend HTPL_PROXY
bind10.1.4.105:8181 name 10.1.4.105:8181
modetcp
log global
maxconn 9
timeout client  1
option tcp-smart-accept
tcp-request inspect-delay 5s
acl is_ssl  req.ssl_hello_type 1
use_backend HTPL_SSL_PROXY_tcp_ipvANY  if  is_ssl
default_backend HTPL_WEB_PROXY_tcp_ipvANY

Re: How can we start haproxy in Linux

[Andrew Smalley]

​Hello ​
S
​abeer

You will find that information in the link I provided in my previous reply.

Also could you please keep your reply's to the mailing list so all can see
please.​


On 2 Jul 2017 4:39 p.m., "Sabeer Basheer"  wrote:

> Hi Andrew Smalley,
>
> Thanks for your fast support.
> Sorry to disturb you again :), It will be a great help, if you give an
> exact command to start and stop haproxy as referred to my attachment
> (currently running in our server).
>
>
> Regards,
> SABEER
>
>
> On Friday, June 30, 2017, Andrew Smalley 
> wrote:
>
>> Sabeer
>>
>> The command will kill haproxy by pid number
>>
>> kill $(cat /var/run/haproxy.pid)
>>
>> However a more basic understanding of haproxy will help
>>
>> https://www.haproxy.com/doc/hapee/1.5/administration/init.html
>>
>>
>> Andrew Smalley
>>
>> Loadbalancer.org Ltd.
>> www.loadbalancer.org 
>>
>> 
>> 
>> 
>> 
>> 
>> +1 888 867 9504 / +44 (0)330 380 1064
>> asmal...@loadbalancer.org
>>
>> Leave a Review
>>  | Deployment
>> Guides
>> 
>> | Blog 
>>
>> On 30 June 2017 at 15:47, Sabeer Basheer 
>> wrote:
>>
>>> Hi team,
>>>
>>> I am new to haproxy and linux, could you please help me by telling how
>>> can we stop/start haroxy in Linux server ( Service haproxy start or haproxy
>>> with -f conf file option). I am here by attaching the screenshot of
>>> currently running process of haproxy in my machine.
>>> Please give me a exact command to stop/start the haproxy from my
>>> attachment.
>>>
>>> Regards,
>>> Sabeer
>>>
>>>
>>> --
>>> With Regards...
>>> •´ ¸.•*´¨)  ¸.•*¨)
>>> (¸.•´(¸.•*   .♪♪♪Sabeer K Basheer
>>> 8089111246
>>>
>>
>>
>
> --
> With Regards...
> •´ ¸.•*´¨)  ¸.•*¨)
> (¸.•´(¸.•*   .♪♪♪Sabeer K Basheer
> 8089111246
>

Re: Proposal: modify stat socket command feedback

[Willy Tarreau]

Hi Dennis,

On Sat, Jul 01, 2017 at 01:37:52AM +0200, Dennis Jacobfeuerborn wrote:
> I had to deal with this issue recently as well but I'd rather like to
> see a more explicit and well defined response in all cases rather than
> the workaround proposed above.
> Its not clear why a warning should automatically mean the command was
> unsuccessful and identifying a specific error still requires string
> matches which breaks all clients the moment the wording is changed or a
> typo is fixed.
> 
> I'd rather see something along the lines of error codes i.e. 0 for "Ok"
> and anything else for specific errors followed by the message e.g.:
> 
> 0:Ok
> 1:No such backend
> 2:No such server in backend
> etc.
> 
> This would allow tools working with the socket to always get consistent
> responses and enable them to only parse the response code to identify
> the response regardless of the message.

This would do the opposite and force all tools to be constantly updated
to learn new codes. Even worse, you'd end up with multiple parallel
registries because most of the time contributed patches have been used
for some time internally, and multiple commands would end up using the
same status codes.

We could use a number of the severity instead of using a word, but in
general it seems important to me that the codification is made once for
all so that tools don't have to be modified in the future.

Also what I like with the severity/error levels is that some commands
currently return confusing output, which some people would understand
as an error and others as a warning or a notice that the operation was
performed but differently. With an error level indication it becomes
much clearer if that partially succeeded or failed.

We've already had some demands to log cli actions, and I tend to think
that using syslog-compatible levels could make this more generic for
all current and future contributions.

> This could either be introduced only in the current development version
> because of the compatibility breakage or a parameter could be introduced
> to configure the socket with the new "protocol". If that parameter is
> not set then the code could simply not be sent in the response and
> messages would be returned just as they are now.

This is a good idea. We could for example have a CLI command to enable
error code reporting in front of messages. It could become the default
in a future version, keeping an option on the stats socket config line
to go back to the current output.

Cheers,
Willy