Re: How to forward HTTP / HTTPS to different backend proxy servers

[Daren Sefcik]

On Sun, Jul 2, 2017 at 7:30 PM, Michael Ezzell <mich...@ezzell.net> wrote:

>
>
> On Jul 2, 2017 8:41 PM, "Daren Sefcik" <dsef...@hightechhigh.org> wrote:
>
> yep, pretty much..I just need some help to figure out how to make it
> work
>
> example log entries for https and http, you can see how the "443" goes to
> one backenad and the regular http "GET" request goes to another..but this
> is not consistent and I know there has to be a better way..
>
>
> use_backend HTPL_WEB_PROXY_http_ipvANY   if { meth_connect }
>
> Or maybe...
>
> use_backend HTPL_WEB_PROXY_http_ipvANY  if { meth_connect } !{ path_end
> :80 }
>
> That should be all you need.
>
> HTTPS through an HTTP proxy via HAProxy isn't an SSL session that HAProxy
> can see.  It's an opaque tunnel, requested over HTTP, using CONNECT.
>
> If the browser asks for a tunnel, it should be because it's wanting to
> speak HTTPS once the target is connected.
>


I think that was all I needed, thank you very much...!!

Re: How to forward HTTP / HTTPS to different backend proxy servers

[Daren Sefcik]

On Sun, Jul 2, 2017 at 6:10 PM, Igor Cicimov <ig...@encompasscorporation.com
> wrote:

>
>
> On Mon, Jul 3, 2017 at 10:38 AM, Daren Sefcik <dsef...@hightechhigh.org>
> wrote:
>
>>
>> On Sun, Jul 2, 2017 at 4:44 PM, Michael Ezzell <mich...@ezzell.net>
>> wrote:
>>
>>>
>>>
>>> On Jul 2, 2017 19:15, "Daren Sefcik" <dsef...@hightechhigh.org> wrote:
>>>
>>>
>>> Most of the traffic is ssl, for example gmail, facebook, pandora all
>>> force https.
>>>
>>>
>>> I'm going to go out on a limb and suggest that *none* of the traffic is
>>> SSL in any sense that is meaningful from HAProxy's perspective.
>>>
>>> What do the HTTPS requests look like in the HAProxy logs?  Aren't they
>>> CONNECT requests?
>>>
>>>
>> yep, pretty much..I just need some help to figure out how to make it
>> work
>>
>> example log entries for https and http, you can see how the "443" goes to
>> one backenad and the regular http "GET" request goes to another..but this
>> is not consistent and I know there has to be a better way..
>>
>> HTPL_PROXY HTPL_SSL_PROXY_http_ipvANY/HTPL-PROXY-03_10.1.4.180
>> 0/0/0/22/10075 200 525 - - cD-- 124/124/103/103/0 0/0 "CONNECT
>> caltopo.com:443 HTTP/1.1"
>>
>> HTPL_PROXY HTPL_WEB_PROXY_http_ipvANY/HTPL-PROXY-04_10.1.4.181
>> 92/0/0/1/93 403 4309 - -  126/126/10/11/0 0/0 "GET
>> http://i2.wp.com/n4.nabble.com/images/avatar100.png HTTP/1.1"
>>
>>
>> TIA for any help with this..!
>>
>
> ​Is it possible that *some* of the clients have issues talking to the
> haproxy over ssl? You say in case of ssl it is not 100% successful but what
> does that mean? How does this manifest? Can you track the ssl request from
> particular client ending up on the http backend?
>

All clients talk to haproxy 100% fine. What I mean by not 100% is that
using that particular acl to try and determine if it is ssl traffic or not
is not 100% in haproxy. Maybe 80% percent of the time haproxy sends the
traffic to the intended backend and 20% of the time to the otherbut
100% of all traffic goes to one backend or the other.


TIA...

Re: How to forward HTTP / HTTPS to different backend proxy servers

[Daren Sefcik]

On Sat, Jul 1, 2017 at 4:39 PM, Igor Cicimov <ig...@encompasscorporation.com
> wrote:

>
>
> On 29 Jun 2017 2:46 am, "Daren Sefcik" <dsef...@hightechhigh.org> wrote:
>
> On Wed, Jun 28, 2017 at 8:12 AM, Olivier Doucet <webmas...@ajeux.com>
> wrote:
>
>> Hi,
>>
>>
>> 2017-06-28 16:47 GMT+02:00 Daren Sefcik <dsef...@hightechhigh.org>:
>>
>>> Hi, I have searched for an answer to this and tried several things but
>>> cannot seem to figure it out so am hoping someone can point me in the right
>>> direction. I have different backend proxy servers (squid) setup to handle
>>> specifically HTTP and HTTPS traffic but cannot figure out how to tell
>>> haproxy to tell the difference and send appropriately.
>>>
>>> For example, I have
>>>
>>> frontend proxy_servers
>>> backend http_proxies
>>> backend https_proxies
>>>
>>> how can I tell frontend to send all http traffic to backend http_proxies
>>> and all https traffic to https_backend? I have tried using dst_port 443 and
>>> the acl https ssl_fc but nothing seems to distinguish https traffic.
>>>
>>
>> Well, it should work. Send a copy of your config to see what's wrong in
>> it.
>>
>> Olivier
>>
>>
>>
>>>
>>> TIA...
>>>
>>
>>
> Here is an example, it continues to direct all https traffic to the web
> proxy and not the streaming media one.
>
> frontend HTPL_PROXY
>   bind10.1.4.105:8181 name 10.1.4.105:8181
>   modehttp
>   log global
>   option  http-server-close
>   option  forwardfor
>   acl https ssl_fc
>   http-request set-header X-Forwarded-Proto http if !https
>   http-request set-header X-Forwarded-Proto https if https
>   maxconn 9
>   timeout client  1
>   option tcp-smart-accept
>   acl is_youtube  hdr_sub(host) -i youtube.com
>   acl is_netflix  hdr_sub(host) -i netflix.com
>   acl is_nflixvideo   hdr_sub(host) -i nflxvideo.net
>   acl is_googlevideo  hdr_sub(host) -i googlevideo.com
>   acl is_google   hdr_sub(host) -i google.com
>   acl is_pandora  hdr_sub(host) -i pandora.com
>   acl is_httpsdst_port eq 443
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_youtube
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_netflix
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_nflixvideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_googlevideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_pandora
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_https
>   default_backend HTPL_WEB_PROXY_http_ipvANY
>
> Obviously dst_port 443 method can not work since you are listening on port
> 8181. Since both protocols are on same port you can try in tcp mode:
>
> mode tcp
> option tcplog
> bind *:8181
>
> tcp-request inspect-delay 5s
> acl is_ssl req.ssl_hello_type 1
>
>

Thank you, I have tried that with the below config and it still sends all
traffic to the default backend instead of my ssl backend, any other ideas?

frontend HTPL_PROXY
bind10.1.4.105:8181 name 10.1.4.105:8181
modetcp
log global
maxconn 9
timeout client  1
option tcp-smart-accept
tcp-request inspect-delay 5s
acl is_ssl  req.ssl_hello_type 1
use_backend HTPL_SSL_PROXY_tcp_ipvANY  if  is_ssl
default_backend HTPL_WEB_PROXY_tcp_ipvANY

Re: How to forward HTTP / HTTPS to different backend proxy servers

[Daren Sefcik]

Would anybody here on the list be able to offer some pointers too how I can
make this work?

TIA...

On Wed, Jun 28, 2017 at 9:43 AM, Daren Sefcik <dsef...@hightechhigh.org>
wrote:

> On Wed, Jun 28, 2017 at 8:12 AM, Olivier Doucet <webmas...@ajeux.com>
> wrote:
>
>> Hi,
>>
>>
>> 2017-06-28 16:47 GMT+02:00 Daren Sefcik <dsef...@hightechhigh.org>:
>>
>>> Hi, I have searched for an answer to this and tried several things but
>>> cannot seem to figure it out so am hoping someone can point me in the right
>>> direction. I have different backend proxy servers (squid) setup to handle
>>> specifically HTTP and HTTPS traffic but cannot figure out how to tell
>>> haproxy to tell the difference and send appropriately.
>>>
>>> For example, I have
>>>
>>> frontend proxy_servers
>>> backend http_proxies
>>> backend https_proxies
>>>
>>> how can I tell frontend to send all http traffic to backend http_proxies
>>> and all https traffic to https_backend? I have tried using dst_port 443 and
>>> the acl https ssl_fc but nothing seems to distinguish https traffic.
>>>
>>
>> Well, it should work. Send a copy of your config to see what's wrong in
>> it.
>>
>> Olivier
>>
>>
>>
>>>
>>> TIA...
>>>
>>
>>
> Here is an example, it continues to direct all https traffic to the web
> proxy and not the streaming media one.
>
> frontend HTPL_PROXY
>   bind10.1.4.105:8181 name 10.1.4.105:8181
>   modehttp
>   log global
>   option  http-server-close
>   option  forwardfor
>   acl https ssl_fc
>   http-request set-header X-Forwarded-Proto http if !https
>   http-request set-header X-Forwarded-Proto https if https
>   maxconn 9
>   timeout client  1
>   option tcp-smart-accept
>   acl is_youtube  hdr_sub(host) -i youtube.com
>   acl is_netflix  hdr_sub(host) -i netflix.com
>   acl is_nflixvideo   hdr_sub(host) -i nflxvideo.net
>   acl is_googlevideo  hdr_sub(host) -i googlevideo.com
>   acl is_google   hdr_sub(host) -i google.com
>   acl is_pandora  hdr_sub(host) -i pandora.com
>   acl is_httpsdst_port eq 443
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_youtube
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_netflix
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_nflixvideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_googlevideo
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_pandora
>   use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_https
>   default_backend HTPL_WEB_PROXY_http_ipvANY
>
>

Re: How to forward HTTP / HTTPS to different backend proxy servers

[Daren Sefcik]

On Wed, Jun 28, 2017 at 8:12 AM, Olivier Doucet <webmas...@ajeux.com> wrote:

> Hi,
>
>
> 2017-06-28 16:47 GMT+02:00 Daren Sefcik <dsef...@hightechhigh.org>:
>
>> Hi, I have searched for an answer to this and tried several things but
>> cannot seem to figure it out so am hoping someone can point me in the right
>> direction. I have different backend proxy servers (squid) setup to handle
>> specifically HTTP and HTTPS traffic but cannot figure out how to tell
>> haproxy to tell the difference and send appropriately.
>>
>> For example, I have
>>
>> frontend proxy_servers
>> backend http_proxies
>> backend https_proxies
>>
>> how can I tell frontend to send all http traffic to backend http_proxies
>> and all https traffic to https_backend? I have tried using dst_port 443 and
>> the acl https ssl_fc but nothing seems to distinguish https traffic.
>>
>
> Well, it should work. Send a copy of your config to see what's wrong in
> it.
>
> Olivier
>
>
>
>>
>> TIA...
>>
>
>
Here is an example, it continues to direct all https traffic to the web
proxy and not the streaming media one.

frontend HTPL_PROXY
bind10.1.4.105:8181 name 10.1.4.105:8181
modehttp
log global
option  http-server-close
option  forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
maxconn 9
timeout client  1
option tcp-smart-accept
acl is_youtube  hdr_sub(host) -i youtube.com
acl is_netflix  hdr_sub(host) -i netflix.com
acl is_nflixvideo   hdr_sub(host) -i nflxvideo.net
acl is_googlevideo  hdr_sub(host) -i googlevideo.com
acl is_google   hdr_sub(host) -i google.com
acl is_pandora  hdr_sub(host) -i pandora.com
acl is_httpsdst_port eq 443
use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_youtube
use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_netflix
use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_nflixvideo
use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_googlevideo
use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_pandora
use_backend HTPL_STREAMING_MEDIA_PROXY_http_ipvANY  if  is_https
default_backend HTPL_WEB_PROXY_http_ipvANY

How to forward HTTP / HTTPS to different backend proxy servers

[Daren Sefcik]

Hi, I have searched for an answer to this and tried several things but
cannot seem to figure it out so am hoping someone can point me in the right
direction. I have different backend proxy servers (squid) setup to handle
specifically HTTP and HTTPS traffic but cannot figure out how to tell
haproxy to tell the difference and send appropriately.

For example, I have

frontend proxy_servers
backend http_proxies
backend https_proxies

how can I tell frontend to send all http traffic to backend http_proxies
and all https traffic to https_backend? I have tried using dst_port 443 and
the acl https ssl_fc but nothing seems to distinguish https traffic.

TIA...

Re: High Availability for haproxy itself

[Daren Sefcik]

We use PfSense with CARP & HaProxy, works great.

On Fri, Jun 2, 2017 at 1:34 AM, Jiafan Zhou 
wrote:

> Hi,
>
> Haproxy ensures the HA for real servers such as httpd. However, in the
> case of haproxy itself, if it fails, then it requires another instance of
> haproxy to be ready. Is there any High Availability solution for haproxy
> itself?
>
> Regards,
> Jiafan
>
>
>

[SOLVED] Re: Need some help configuring backend health checks

[Daren Sefcik]

On Fri, Oct 30, 2015 at 12:43 PM, Igor Cicimov <
ig...@encompasscorporation.com> wrote:

>
> On 31/10/2015 3:14 AM, "Daren Sefcik" <dsef...@hightechhigh.org> wrote:
> >
> >
> >
> > On Thu, Oct 29, 2015 at 11:15 PM, Igor Cicimov <
> ig...@encompasscorporation.com> wrote:
> >>
> >>
> >> On 30/10/2015 4:48 PM, "Daren Sefcik" <dsef...@hightechhigh.org> wrote:
> >> >
> >> > So I think those links were the right idea and I have been trying
> different configurations but am not quite there and am hoping somebody can
> offer a bit more guidance.
> >> >
> >> > So when I telnet to the icap server I type in the OPTIONS line
> followed by (2) return key presses and then it returns the ICAP text, below
> is my telent session output
> >> >
> >> > ===
> >> >
> >> > $ telnet 10.1.4.153 1344
> >> > Trying 10.1.4.153...
> >> > Connected to 10.1.4.153.
> >> > Escape character is '^]'.
> >> > OPTIONS icap://127.0.0.1:1344/respmod ICAP/1.0
> >> >
> >> > ICAP/1.0 200 OK
> >> > ISTAG: "5BDEEEA9-12E4-2"
> >> > Service: Diladele Web Safety 4.2.0.CBF4
> >> > Service-ID: qlproxy
> >> > Methods: RESPMOD
> >> > Options-TTL: 3600
> >> > Max-Connections: 15000
> >> > Allow: 204
> >> > Preview: 4096
> >> > Transfer-Preview: *
> >> > Encapsulated: null-body=0
> >> > Connection: close
> >> >
> >> > 
> >> >
> >> >
> >> > Here is what I have tried in the backend configurations
> >> >
> >> > option tcp-check
> >> > tcp-check send OPTIONS\ icap\:\/\/127\.0\.0\.1\:1344\/respmod\
> ICAP\/1\.0\r\n\
> >> > tcp-check send \r\n
> >> > tcp-check expect string ICAP\/1\.0\ 200\ OK
> >> >
> >> >
> >> > but it is still not working, I suspect I need to use some type of
> regex or such. Hoping somebody can help me along with this.
> >> >
> >> > TIA..
> >> >
> >> >
> >> > On Mon, Oct 19, 2015 at 7:42 AM, Daren Sefcik <
> dsef...@hightechhigh.org> wrote:
> >> >>
> >> >> Thanks Jarno, I am still not sure how I can apply this to each
> server using a different port but will poke around at it and see if I can
> figure it out.
> >> >>
> >> >> On Mon, Oct 19, 2015 at 1:04 AM, Jarno Huuskonen <
> jarno.huusko...@uef.fi> wrote:
> >> >>>
> >> >>> Hi,
> >> >>>
> >> >>> On Sun, Oct 18, Daren Sefcik wrote:
> >> >>> > I have an ICAP server backend with servers that each listen on
> different
> >> >>> > ports, can anyone offer some advice on how to configure health
> checks for
> >> >>> > it? I am currently using basic but that really doesn't help if
> the service
> >> >>> > is not responding.
> >> >>> >
> >> >>> > Here is my haproxy config for the backend:
> >> >>> >
> >> >>> > backend HTPL_CONT_FILTER_tcp_ipvANY
> >> >>> > mode tcp
> >> >>> > balance roundrobin
> >> >>> > timeout connect 5
> >> >>> > timeout server 5
> >> >>> > retries 3
> >> >>> > server HTPL-WEB-01_10.1.4.153 10.1.4.153:1344 check inter 5000
> weight 200
> >> >>> > maxconn 200 fastinter 1000 fall 5
> >> >>> > server HTPL-WEB-02_10.1.4.154 10.1.4.154:1344 check inter 5000
> weight 200
> >> >>> > maxconn 200 fastinter 1000 fall 5
> >> >>> > server HTPL-WEB-02_10.1.4.155_01 10.1.4.155:8102 check inter
> 5000  weight
> >> >>> > 200 maxconn 200 fastinter 1000 fall 5
> >> >>> > server HTPL-WEB-02_10.1.4.155_02 10.1.4.155:8202 check inter
> 5000  weight
> >> >>> > 200 maxconn 200 fastinter 1000 fall 5
> >> >>>
> >> >>> Do the icap servers (squid+diladele?) respond to something like
> this:
> >> >>> https://support.symantec.com/en_US/article.TECH220980.html
> >> >>> or
> https://exchange.icinga.org/oldmonex/1733-check_icap.pl/check_icap.pl
> >> >>>
> >> >>> Maybe you can use tcp-check to send icap request a

Re: Need some help configuring backend health checks

[Daren Sefcik]

On Thu, Oct 29, 2015 at 11:15 PM, Igor Cicimov <
ig...@encompasscorporation.com> wrote:

>
> On 30/10/2015 4:48 PM, "Daren Sefcik" <dsef...@hightechhigh.org> wrote:
> >
> > So I think those links were the right idea and I have been trying
> different configurations but am not quite there and am hoping somebody can
> offer a bit more guidance.
> >
> > So when I telnet to the icap server I type in the OPTIONS line followed
> by (2) return key presses and then it returns the ICAP text, below is my
> telent session output
> >
> > ===
> >
> > $ telnet 10.1.4.153 1344
> > Trying 10.1.4.153...
> > Connected to 10.1.4.153.
> > Escape character is '^]'.
> > OPTIONS icap://127.0.0.1:1344/respmod ICAP/1.0
> >
> > ICAP/1.0 200 OK
> > ISTAG: "5BDEEEA9-12E4-2"
> > Service: Diladele Web Safety 4.2.0.CBF4
> > Service-ID: qlproxy
> > Methods: RESPMOD
> > Options-TTL: 3600
> > Max-Connections: 15000
> > Allow: 204
> > Preview: 4096
> > Transfer-Preview: *
> > Encapsulated: null-body=0
> > Connection: close
> >
> > 
> >
> >
> > Here is what I have tried in the backend configurations
> >
> > option tcp-check
> > tcp-check send OPTIONS\ icap\:\/\/127\.0\.0\.1\:1344\/respmod\
> ICAP\/1\.0\r\n\
> > tcp-check send \r\n
> > tcp-check expect string ICAP\/1\.0\ 200\ OK
> >
> >
> > but it is still not working, I suspect I need to use some type of regex
> or such. Hoping somebody can help me along with this.
> >
> > TIA..
> >
> >
> > On Mon, Oct 19, 2015 at 7:42 AM, Daren Sefcik <dsef...@hightechhigh.org>
> wrote:
> >>
> >> Thanks Jarno, I am still not sure how I can apply this to each server
> using a different port but will poke around at it and see if I can figure
> it out.
> >>
> >> On Mon, Oct 19, 2015 at 1:04 AM, Jarno Huuskonen <
> jarno.huusko...@uef.fi> wrote:
> >>>
> >>> Hi,
> >>>
> >>> On Sun, Oct 18, Daren Sefcik wrote:
> >>> > I have an ICAP server backend with servers that each listen on
> different
> >>> > ports, can anyone offer some advice on how to configure health
> checks for
> >>> > it? I am currently using basic but that really doesn't help if the
> service
> >>> > is not responding.
> >>> >
> >>> > Here is my haproxy config for the backend:
> >>> >
> >>> > backend HTPL_CONT_FILTER_tcp_ipvANY
> >>> > mode tcp
> >>> > balance roundrobin
> >>> > timeout connect 5
> >>> > timeout server 5
> >>> > retries 3
> >>> > server HTPL-WEB-01_10.1.4.153 10.1.4.153:1344 check inter 5000
> weight 200
> >>> > maxconn 200 fastinter 1000 fall 5
> >>> > server HTPL-WEB-02_10.1.4.154 10.1.4.154:1344 check inter 5000
> weight 200
> >>> > maxconn 200 fastinter 1000 fall 5
> >>> > server HTPL-WEB-02_10.1.4.155_01 10.1.4.155:8102 check inter 5000
> weight
> >>> > 200 maxconn 200 fastinter 1000 fall 5
> >>> > server HTPL-WEB-02_10.1.4.155_02 10.1.4.155:8202 check inter 5000
> weight
> >>> > 200 maxconn 200 fastinter 1000 fall 5
> >>>
> >>> Do the icap servers (squid+diladele?) respond to something like this:
> >>> https://support.symantec.com/en_US/article.TECH220980.html
> >>> or
> https://exchange.icinga.org/oldmonex/1733-check_icap.pl/check_icap.pl
> >>>
> >>> Maybe you can use tcp-check to send icap request and look for
> >>> "ICAP/1.0 200" response:
> >>>
> https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#tcp-check%20connect
> >>>
> http://blog.haproxy.com/2014/01/02/haproxy-advanced-redis-health-check/
> >>>
> >>> -Jarno
> >>>
> >>> --
> >>> Jarno Huuskonen
> >>
> >>
> >
> Since your telnet session is on port 1344, maybe
>
> tcp-check connect port 1344
>
> before the send command.
>
Thank you but each backend server has a different port configured, that is
just one example.


server HTPL-WEB-01_10.1.4.153 10.1.4.153:1344 check inter 5000  weight 200
maxconn 200 fastinter 1000 rise 1 fall 5
server HTPL-WEB-02_10.1.4.154 10.1.4.154:1344 check inter 5000  weight 200
maxconn 200 fastinter 1000 rise 1 fall 5
server HTPL-WEB-02-DOCK-02_10.1.4.155_01 10.1.4.155:8102 check inter 5000
 weight 200 maxconn 200 fastinter 1000 rise 1 fall 5
server HTPL-WEB-02-DOCK-02_10.1.4.155_02 10.1.4.155:8202 check inter 5000
 weight 200 maxconn 200 fastinter 1000 rise 1 fall 5

Re: Need some help configuring backend health checks

[Daren Sefcik]

So I think those links were the right idea and I have been trying different
configurations but am not quite there and am hoping somebody can offer a
bit more guidance.

So when I telnet to the icap server I type in the OPTIONS line followed by
(2) return key presses and then it returns the ICAP text, below is my
telent session output

===

$ telnet 10.1.4.153 1344
Trying 10.1.4.153...
Connected to 10.1.4.153.
Escape character is '^]'.
OPTIONS icap://127.0.0.1:1344/respmod ICAP/1.0

ICAP/1.0 200 OK
ISTAG: "5BDEEEA9-12E4-2"
Service: Diladele Web Safety 4.2.0.CBF4
Service-ID: qlproxy
Methods: RESPMOD
Options-TTL: 3600
Max-Connections: 15000
Allow: 204
Preview: 4096
Transfer-Preview: *
Encapsulated: null-body=0
Connection: close




Here is what I have tried in the backend configurations

option tcp-check
tcp-check send OPTIONS\ icap\:\/\/127\.0\.0\.1\:1344\/respmod\
ICAP\/1\.0\r\n\
tcp-check send \r\n
tcp-check expect string ICAP\/1\.0\ 200\ OK


but it is still not working, I suspect I need to use some type of regex or
such. Hoping somebody can help me along with this.

TIA..


On Mon, Oct 19, 2015 at 7:42 AM, Daren Sefcik <dsef...@hightechhigh.org>
wrote:

> Thanks Jarno, I am still not sure how I can apply this to each server
> using a different port but will poke around at it and see if I can figure
> it out.
>
> On Mon, Oct 19, 2015 at 1:04 AM, Jarno Huuskonen <jarno.huusko...@uef.fi>
> wrote:
>
>> Hi,
>>
>> On Sun, Oct 18, Daren Sefcik wrote:
>> > I have an ICAP server backend with servers that each listen on different
>> > ports, can anyone offer some advice on how to configure health checks
>> for
>> > it? I am currently using basic but that really doesn't help if the
>> service
>> > is not responding.
>> >
>> > Here is my haproxy config for the backend:
>> >
>> > backend HTPL_CONT_FILTER_tcp_ipvANY
>> > mode tcp
>> > balance roundrobin
>> > timeout connect 5
>> > timeout server 5
>> > retries 3
>> > server HTPL-WEB-01_10.1.4.153 10.1.4.153:1344 check inter 5000  weight
>> 200
>> > maxconn 200 fastinter 1000 fall 5
>> > server HTPL-WEB-02_10.1.4.154 10.1.4.154:1344 check inter 5000  weight
>> 200
>> > maxconn 200 fastinter 1000 fall 5
>> > server HTPL-WEB-02_10.1.4.155_01 10.1.4.155:8102 check inter 5000
>> weight
>> > 200 maxconn 200 fastinter 1000 fall 5
>> > server HTPL-WEB-02_10.1.4.155_02 10.1.4.155:8202 check inter 5000
>> weight
>> > 200 maxconn 200 fastinter 1000 fall 5
>>
>> Do the icap servers (squid+diladele?) respond to something like this:
>> https://support.symantec.com/en_US/article.TECH220980.html
>> or https://exchange.icinga.org/oldmonex/1733-check_icap.pl/check_icap.pl
>>
>> Maybe you can use tcp-check to send icap request and look for
>> "ICAP/1.0 200" response:
>>
>> https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#tcp-check%20connect
>> http://blog.haproxy.com/2014/01/02/haproxy-advanced-redis-health-check/
>>
>> -Jarno
>>
>> --
>> Jarno Huuskonen
>>
>
>

Re: Need some help configuring backend health checks

[Daren Sefcik]

Thanks Jarno, I am still not sure how I can apply this to each server using
a different port but will poke around at it and see if I can figure it out.

On Mon, Oct 19, 2015 at 1:04 AM, Jarno Huuskonen <jarno.huusko...@uef.fi>
wrote:

> Hi,
>
> On Sun, Oct 18, Daren Sefcik wrote:
> > I have an ICAP server backend with servers that each listen on different
> > ports, can anyone offer some advice on how to configure health checks for
> > it? I am currently using basic but that really doesn't help if the
> service
> > is not responding.
> >
> > Here is my haproxy config for the backend:
> >
> > backend HTPL_CONT_FILTER_tcp_ipvANY
> > mode tcp
> > balance roundrobin
> > timeout connect 5
> > timeout server 5
> > retries 3
> > server HTPL-WEB-01_10.1.4.153 10.1.4.153:1344 check inter 5000  weight
> 200
> > maxconn 200 fastinter 1000 fall 5
> > server HTPL-WEB-02_10.1.4.154 10.1.4.154:1344 check inter 5000  weight
> 200
> > maxconn 200 fastinter 1000 fall 5
> > server HTPL-WEB-02_10.1.4.155_01 10.1.4.155:8102 check inter 5000
> weight
> > 200 maxconn 200 fastinter 1000 fall 5
> > server HTPL-WEB-02_10.1.4.155_02 10.1.4.155:8202 check inter 5000
> weight
> > 200 maxconn 200 fastinter 1000 fall 5
>
> Do the icap servers (squid+diladele?) respond to something like this:
> https://support.symantec.com/en_US/article.TECH220980.html
> or https://exchange.icinga.org/oldmonex/1733-check_icap.pl/check_icap.pl
>
> Maybe you can use tcp-check to send icap request and look for
> "ICAP/1.0 200" response:
>
> https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#tcp-check%20connect
> http://blog.haproxy.com/2014/01/02/haproxy-advanced-redis-health-check/
>
> -Jarno
>
> --
> Jarno Huuskonen
>

Re: Dynamically change server maxconn possible?

[Daren Sefcik]

Thanks, this will be helpful to find a good load balance as the systems are
running.

On Mon, Oct 19, 2015 at 1:12 PM, Willy Tarreau  wrote:

> On Mon, Oct 19, 2015 at 02:19:52PM -0500, Andrew Hayworth wrote:
> > I was just thinking about how useful this would be, and will submit a
> patch
> > for it.
>
> Thank you Andrew.
>
> Willy
>
>

Need some help configuring backend health checks

[Daren Sefcik]

I have an ICAP server backend with servers that each listen on different
ports, can anyone offer some advice on how to configure health checks for
it? I am currently using basic but that really doesn't help if the service
is not responding.

Here is my haproxy config for the backend:

backend HTPL_CONT_FILTER_tcp_ipvANY
mode tcp
balance roundrobin
timeout connect 5
timeout server 5
retries 3
server HTPL-WEB-01_10.1.4.153 10.1.4.153:1344 check inter 5000  weight 200
maxconn 200 fastinter 1000 fall 5
server HTPL-WEB-02_10.1.4.154 10.1.4.154:1344 check inter 5000  weight 200
maxconn 200 fastinter 1000 fall 5
server HTPL-WEB-02_10.1.4.155_01 10.1.4.155:8102 check inter 5000  weight
200 maxconn 200 fastinter 1000 fall 5
server HTPL-WEB-02_10.1.4.155_02 10.1.4.155:8202 check inter 5000  weight
200 maxconn 200 fastinter 1000 fall 5

If I use curl and get the headers from a healthy response here is what is
returned:

curl -v http://10.1.4.155:8202/

* Hostname was NOT found in DNS cache
*   Trying 10.1.4.105...
* Connected to 10.1.4.105 (10.1.4.105) port 8181 (#0)
> GET http://10.1.4.155:8202/ HTTP/1.1
> User-Agent: curl/7.35.0
> Host: 10.1.4.155:8202
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
* Server squid/3.4.10 is not blacklisted
< Server: squid/3.4.10
< Mime-Version: 1.0
< Date: Sun, 18 Oct 2015 23:49:22 GMT
< X-Transformed-From: HTTP/0.9
< X-Cache: MISS from HTPL-PROXY-02
< X-Cache-Lookup: MISS from HTPL-PROXY-02:3128
< Transfer-Encoding: chunked
< Via: 1.1 HTPL-PROXY-02 (squid/3.4.10)
<
ICAP/1.0 400 Protocol Error
ISTAG: "5BDEEEA9-12E4-2"
Service: Diladele Web Safety 4.2.0.CBF4
Connection: close
Encapsulated: resp-body=0

ICAP Protocol error: icap_header - unknown ICAP method found
* Connection #0 to host 10.1.4.105 left intact


TIA...

Dynamically change server maxconn possible?

[Daren Sefcik]

I am thinking the answer is no but figured I would ask just to make
sure...basically can I change individual server maxconn numbers on-the-fly
while haproxy is running or do I need to do a full restart to have them
take effect?

TIA...

Re: Squid Backend Health Checks

[Daren Sefcik]

The only thing I can look at is the stats page which report failed health
checks. I have tried to configure pfsense to log things but have been
unsuccessful. Any insight how to configure logging in pfsense would be
really appreciated.

On Wed, Oct 14, 2015 at 7:08 AM, Baptiste <bed...@gmail.com> wrote:

> Hi Daren,
>
> What type of errors are reported?
>
> Baptiste
>
> On Wed, Oct 14, 2015 at 8:19 AM, Daren Sefcik <dsef...@hightechhigh.org>
> wrote:
> > I followed Willy's advice from this post
> >
> > http://www.mail-archive.com/haproxy@formilux.org/msg05171.html
> >
> > but seem to get a lot of health check errors and (false) Dwntme. Is
> there a
> > newer or better way to do health checks or am I missing something?
> >
> > TIA..
> >
> > BTW, nice work on 1.6, am looking forward to trying it out soon...
> >
> >
> > Here is my relevant code, http://10.1.4.105:9090 is the CARP address my
> > clienst also use as the proxy ip to use. I tried using the local IP and
> had
> > the same problems.
> >
> > listen check-responder
> > bind *:9090
> > mode http
> > monitor-uri /
> > timeout client 5000
> > timeout connect 5000
> > timeout server 5000
> >
> >
> > backend HTPL_WEB_PROXY_http_ipvANY
> > mode http
> > stick-table type ip size 50k expire 5m
> > stick on src
> > balance roundrobin
> > timeout connect 5
> > timeout server 5
> > retries 3
> > option httpchk GET http://10.1.4.105:9090/ HTTP/1.0
> > server HTPL-PROXY-01_10.1.4.103 10.1.4.103:3128 cookie HTPLPROXY01 check
> > inter 3  weight 175 maxconn 1500 fastinter 1000 fall 5
> > server HTPL-PROXY-02_10.1.4.104 10.1.4.104:3128 cookie HTPLPROXY02 check
> > inter 3  weight 175 maxconn 1500 fastinter 1000 fall 5
> > server HTPL-PROXY-03_10.1.4.107 10.1.4.107:3128 cookie HTPLPROXY03 check
> > inter 3  weight 100 maxconn 1500 fastinter 1000 fall 5
> > server HTPL-PROXY-04_10.1.4.108 10.1.4.108:3128 cookie HTPLPROXY04 check
> > inter 3  weight 200 maxconn 1500 fastinter 1000 fall 5
> > server HTHPL-PROXY-02_10.1.4.101 10.1.4.101:3128 cookie HTHPLPROXY02
> check
> > inter 3  weight 150 maxconn 1500 fastinter 1000 fall 5
> > server HTHPL-PROXY-03_10.1.4.102 10.1.4.102:3128 cookie HTHPLPROXY03
> check
> > inter 3  weight 125 maxconn 1000 fastinter 1000 fall 5
> >
> >
>

Re: Squid Backend Health Checks

[Daren Sefcik]

Using HATop I see lots of ERSP errors which I guess is response
errors...not sure if that is helpful.

On Wed, Oct 14, 2015 at 7:40 AM, Daren Sefcik <dsef...@hightechhigh.org>
wrote:

> The only thing I can look at is the stats page which report failed health
> checks. I have tried to configure pfsense to log things but have been
> unsuccessful. Any insight how to configure logging in pfsense would be
> really appreciated.
>
> On Wed, Oct 14, 2015 at 7:08 AM, Baptiste <bed...@gmail.com> wrote:
>
>> Hi Daren,
>>
>> What type of errors are reported?
>>
>> Baptiste
>>
>> On Wed, Oct 14, 2015 at 8:19 AM, Daren Sefcik <dsef...@hightechhigh.org>
>> wrote:
>> > I followed Willy's advice from this post
>> >
>> > http://www.mail-archive.com/haproxy@formilux.org/msg05171.html
>> >
>> > but seem to get a lot of health check errors and (false) Dwntme. Is
>> there a
>> > newer or better way to do health checks or am I missing something?
>> >
>> > TIA..
>> >
>> > BTW, nice work on 1.6, am looking forward to trying it out soon...
>> >
>> >
>> > Here is my relevant code, http://10.1.4.105:9090 is the CARP address my
>> > clienst also use as the proxy ip to use. I tried using the local IP and
>> had
>> > the same problems.
>> >
>> > listen check-responder
>> > bind *:9090
>> > mode http
>> > monitor-uri /
>> > timeout client 5000
>> > timeout connect 5000
>> > timeout server 5000
>> >
>> >
>> > backend HTPL_WEB_PROXY_http_ipvANY
>> > mode http
>> > stick-table type ip size 50k expire 5m
>> > stick on src
>> > balance roundrobin
>> > timeout connect 5
>> > timeout server 5
>> > retries 3
>> > option httpchk GET http://10.1.4.105:9090/ HTTP/1.0
>> > server HTPL-PROXY-01_10.1.4.103 10.1.4.103:3128 cookie HTPLPROXY01
>> check
>> > inter 3  weight 175 maxconn 1500 fastinter 1000 fall 5
>> > server HTPL-PROXY-02_10.1.4.104 10.1.4.104:3128 cookie HTPLPROXY02
>> check
>> > inter 3  weight 175 maxconn 1500 fastinter 1000 fall 5
>> > server HTPL-PROXY-03_10.1.4.107 10.1.4.107:3128 cookie HTPLPROXY03
>> check
>> > inter 3  weight 100 maxconn 1500 fastinter 1000 fall 5
>> > server HTPL-PROXY-04_10.1.4.108 10.1.4.108:3128 cookie HTPLPROXY04
>> check
>> > inter 3  weight 200 maxconn 1500 fastinter 1000 fall 5
>> > server HTHPL-PROXY-02_10.1.4.101 10.1.4.101:3128 cookie HTHPLPROXY02
>> check
>> > inter 3  weight 150 maxconn 1500 fastinter 1000 fall 5
>> > server HTHPL-PROXY-03_10.1.4.102 10.1.4.102:3128 cookie HTHPLPROXY03
>> check
>> > inter 3  weight 125 maxconn 1000 fastinter 1000 fall 5
>> >
>> >
>>
>
>

Squid Backend Health Checks

[Daren Sefcik]

I followed Willy's advice from this post

http://www.mail-archive.com/haproxy@formilux.org/msg05171.html

but seem to get a lot of health check errors and (false) Dwntme. Is there a
newer or better way to do health checks or am I missing something?

TIA..

BTW, nice work on 1.6, am looking forward to trying it out soon...


Here is my relevant code, http://10.1.4.105:9090 is the CARP address my
clienst also use as the proxy ip to use. I tried using the local IP and had
the same problems.

listen check-responder
bind *:9090
mode http
monitor-uri /
timeout client 5000
timeout connect 5000
timeout server 5000


backend HTPL_WEB_PROXY_http_ipvANY
mode http
stick-table type ip size 50k expire 5m
stick on src
balance roundrobin
timeout connect 5
timeout server 5
retries 3
option httpchk GET http://10.1.4.105:9090/ HTTP/1.0
server HTPL-PROXY-01_10.1.4.103 10.1.4.103:3128 cookie HTPLPROXY01 check
inter 3  weight 175 maxconn 1500 fastinter 1000 fall 5
server HTPL-PROXY-02_10.1.4.104 10.1.4.104:3128 cookie HTPLPROXY02 check
inter 3  weight 175 maxconn 1500 fastinter 1000 fall 5
server HTPL-PROXY-03_10.1.4.107 10.1.4.107:3128 cookie HTPLPROXY03 check
inter 3  weight 100 maxconn 1500 fastinter 1000 fall 5
server HTPL-PROXY-04_10.1.4.108 10.1.4.108:3128 cookie HTPLPROXY04 check
inter 3  weight 200 maxconn 1500 fastinter 1000 fall 5
server HTHPL-PROXY-02_10.1.4.101 10.1.4.101:3128 cookie HTHPLPROXY02 check
inter 3  weight 150 maxconn 1500 fastinter 1000 fall 5
server HTHPL-PROXY-03_10.1.4.102 10.1.4.102:3128 cookie HTHPLPROXY03 check
inter 3  weight 125 maxconn 1000 fastinter 1000 fall 5

Re: How to configure frontend/backend for SSL OR Non SSL Backend?

[Daren Sefcik]

I will need to look at your suggestion some more to understand how it will
pass 443 traffic to the backend server, I only see definitions for port 80.
Since I am using pfsense and the haproxy package I will also have to figure
out how to define peers as that does not seem to be available in the web
gui.

thanks

On Mon, Oct 12, 2015 at 8:22 AM, Baptiste <bed...@gmail.com> wrote:

> So basically, here is what you want to do:
> peers mypeers
>  # read the doc for the info to store here
>
> frontend ftapp
>  bind :80
>  bind :443
>  mode tcp
>  default_backend bkapp
>
> backend bkapp
>  mode tcp
>  stick-table type ip size 10k peers mypeers
>  stick on src
>  server s1 a.b.c.d check port 80
>  server s2 a.b.c.e check port 80
>
>
> Baptiste
>
>
> On Mon, Oct 12, 2015 at 4:40 PM, Daren Sefcik <dsef...@hightechhigh.org>
> wrote:
> > humm...not sure I know how to answer that...we have servers that require
> SSL
> > for some requests and not for others. I am not needing to do anything
> other
> > than pass the traffic along, not doing any inspection or verifying of
> cert
> > or anything. I tried to setup a frontend with 2 servers in the backend,
> one
> > with 443 and the other with 80 but that didn't seem to work, like it
> would
> > pick the wrong one to send to.
> >
> > On Mon, Oct 12, 2015 at 7:29 AM, Baptiste <bed...@gmail.com> wrote:
> >>
> >> Hi Daren,
> >>
> >> Do you want/need to decipher the traffic when using SSL?
> >>
> >> Baptiste
> >>
> >> On Mon, Oct 12, 2015 at 4:24 PM, Daren Sefcik <dsef...@hightechhigh.org
> >
> >> wrote:
> >> > I am probably totally overlooking something but how do I configure a
> >> > frontend/backend to pass to the same server for both SSL and Non SSL
> >> > requests?  We have server that require ssl for some applications but
> >> > most of
> >> > the time not.
> >> >
> >
> >
>

Re: How to configure frontend/backend for SSL OR Non SSL Backend?

[Daren Sefcik]

humm...not sure I know how to answer that...we have servers that require
SSL for some requests and not for others. I am not needing to do anything
other than pass the traffic along, not doing any inspection or verifying of
cert or anything. I tried to setup a frontend with 2 servers in the
backend, one with 443 and the other with 80 but that didn't seem to work,
like it would pick the wrong one to send to.

On Mon, Oct 12, 2015 at 7:29 AM, Baptiste <bed...@gmail.com> wrote:

> Hi Daren,
>
> Do you want/need to decipher the traffic when using SSL?
>
> Baptiste
>
> On Mon, Oct 12, 2015 at 4:24 PM, Daren Sefcik <dsef...@hightechhigh.org>
> wrote:
> > I am probably totally overlooking something but how do I configure a
> > frontend/backend to pass to the same server for both SSL and Non SSL
> > requests?  We have server that require ssl for some applications but
> most of
> > the time not.
> >
>

Optimal Configuration examples for Squid or other cache backends?

[Daren Sefcik]

Can anyone point me to any references that explain optimal squid/web cache
specific examples of configuring haproxy? Everything I am finding so far
seems specific to web servers being the backend. Things like should I be
using stick tables or cookies or best client time out, etc

TIA

How to configure frontend/backend for SSL OR Non SSL Backend?

[Daren Sefcik]

I am probably totally overlooking something but how do I configure a
frontend/backend to pass to the same server for both SSL and Non SSL
requests?  We have server that require ssl for some applications but most
of the time not.

Re: About maxconn and minconn

[Daren Sefcik]

How about kern.ipc.somaxconn

On Thu, Oct 8, 2015 at 5:22 AM, Dmitry Sivachenko 
wrote:

>
> > On 8 окт. 2015 г., at 3:51, Igor Cicimov 
> wrote:
> >
> >
> > The only thing I can think of is you have left net.core.somaxconn = 128,
> try increasing it to 4096 lets say to match your planned capacity of 3000
> >
>
>
> I forgot to mention that I am using FreeBSD, I don't think it has similar
> sysctl.
>

Re: Frontend ACL rewrites URL incorrectly to backend

[Daren Sefcik]

Hey Joris, I appreciate the help...I am not sure I quite understand though,
is there something I can configure in haproxy to resolve this? It is not
just a Wordpress problem, I have other sites also that do not behave
correctly when I put haproxy in front of them.

On Mon, Oct 5, 2015 at 8:22 AM, joris dedieu <joris.ded...@gmail.com> wrote:

> Hi,
>
> 2015-10-04 23:33 GMT+02:00 Daren Sefcik <dsef...@hightechhigh.org>:
> > I am trying to make some requests go to specific backends but am finding
> > that in certain backends that the url gets doubled up or otherwise
> mangled,
> > ie:
> >
> > request to frontend = http://my.company.com
> > what the backend server ends up with =
> > http://my.company.comhttp://my.company.com
> >
> > This does not happen in all of the backends, only a few...a wordpress
> site
>
> This is typically what append when wordpress is invoked with a wrong
> Host header.
> It must match WP_SITEURL and WP_HOME
>
> Regards
> Joris
>
> > comes to mind as a specific example. Since this does not happen on every
> > single backend server I suspect it is instead something happening on the
> > receiving server but since it only happens when I put haproxy in front
> of it
> > there is some connection between them.
> >
> > Can someone help me understand what haproxy is doing or how to fix this
> from
> > happening?
> > Before anyone says it is varnish doing it I should say several of the
> other
> > backends using varnish work fine, it is only a few that get the url
> messed
> > up.
> >
> > TIA
> >
> > example ACL:
> >
> > acl   acl_my.company.com hdr(host) -i my.company.com
> > use_backend  VARNISH_BKEND if acl_my.company.com
>

Re: x-forwarded-for help

[Daren Sefcik]

On Mon, Oct 5, 2015 at 5:29 AM, Travis Fitch  wrote:

> My issue is, if I bypass the hardware LB, I see the X-Forwarded-For
> header, if I go via the the Hardware LB to haproxy and onto Apache, I don't
> see any x-forwarded-for headers in Apaches log files.


I am new to haproxy but it sounds like maybe haproxy does not know how to
extract the x-forwarded-for header and pass it on, it is only looking at
the x-client header and adding it to the XFF header.

Re: HAProxy Slows At 1500+ connections Really Need some help to figure out why

[Daren Sefcik]

On Sun, Oct 4, 2015 at 7:30 AM, joris dedieu  wrote:

> Broadcom NICs : you should check man 4 bge and
> https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards
>

While I had already looked at this early in my troubleshooting process I
totally screwed it up, I have bge cards and instead of changing the "bce"
configurations to "bge" I left them at bce. I changed them just now and was
able to easily achieve these numbers without a wink:

*pid = *50054 (process #1, nbproc = 1)
*uptime = *0d 0h03m25s
*system limits:* memmax = unlimited; ulimit-n = 100047
*maxsock = *100047; *maxconn = *5; *maxpipes = *0
current conns = 5562; current pipes = 0/0; conn rate = 64/sec
Running tasks: 1/5587; idle = 97 %

I am going to load some real clients on to the system tomorrow when school
is back in session and see how that works, hopefully that is all I needed.
With Apache Bench I am getting good numbers on the stats page but my client
is still sluggish so I may have a bottleneck somewhere else now.

I will report back with real world results, thanks for everyone's
suggestions so far, it is much appreciated. I do have some other questions
though I will post in a separate topic.

Daren

Re: HAProxy Slows At 1500+ connections Really Need some help to figure out why

[Daren Sefcik]

>
>
> Interesting, so maybe in fact you're running on cards with an old horrible
> firmware like what I described in the previous e-mail.
>

Possible I suppose but these are brand new Dell systems, just got them a
few a weeks ago. Any advice on how to check and I will do so.


> Maybe your client is experiencing the same issue if it runs on similar
> hardware. Do not hesitate to do "pfctl -d" as suggested by someone (maybe
> Joris, I don't remember). It used to help a lot on openbsd in the past for
> similar reasons as conntrack on linux (session table full). At least you'll
> know.
>

My ab tests are run from a separate server on the lan, they are not from my
client. When the ab tests are finished my client responds quickly again.
BTW, my client is a Mac PB running Linux Mint.

Disabling the firewall has no effect, the response times stay the same.

Re: HAProxy Slows At 1500+ connections Really Need some help to figure out why

[Daren Sefcik]

On Sun, Oct 4, 2015 at 1:03 PM, Willy Tarreau  wrote:

> I don't know how you can do on FreeBSD. On Linux you check with ethtool -i.
> The worst versions of the chips reported firmware 1.9.6, though several
> more recent ones still had problems but not *that* important.
>

Does this help?

/root: pciconf -alV bge0
bge0@pci0:2:0:0: class=0x02 card=0x1f5b1028 chip=0x165f14e4 rev=0x00
hdr=0x00
VPD ident  = 'Broadcom NetXtreme Gigabit Ethernet'
VPD ro PN  = 'BCM95720'
VPD ro MN  = '1028'
VPD ro V0  = 'FFV7.10.59'
VPD ro V1  = 'DSV1028VPDR.VER1.0'
VPD ro V2  = 'NPY2'
VPD ro V3  = 'PMT1'
VPD ro V4  = 'NMVBroadcom Corp'
VPD ro V5  = 'DTINIC'
VPD ro V6  = 'DCM1001008d452101008d45'

Frontend ACL rewrites URL incorrectly to backend

[Daren Sefcik]

I am trying to make some requests go to specific backends but am finding
that in certain backends that the url gets doubled up or otherwise mangled,
ie:

request to frontend = http://my.company.com
what the backend server ends up with = http://my.company.comhttp://
my.company.com

This does not happen in all of the backends, only a few...a wordpress site
comes to mind as a specific example. Since this does not happen on every
single backend server I suspect it is instead something happening on the
receiving server but since it only happens when I put haproxy in front of
it there is some connection between them.

Can someone help me understand what haproxy is doing or how to fix this
from happening?
Before anyone says it is varnish doing it I should say several of the other
backends using varnish work fine, it is only a few that get the url messed
up.

TIA

example ACL:

acl   acl_my.company.com hdr(host) -i my.company.com
use_backend  VARNISH_BKEND if acl_my.company.com

Re: HAProxy Slows At 1500+ connections Really Need some help to figure out why

[Daren Sefcik]

On Sat, Oct 3, 2015 at 12:09 AM, joris dedieu 
wrote:

> > Running tasks: 1/2252; idle = 85 %
>
> How is the system during stress ?
>
mostly idle according to top, haproxy stats and other utils in the web gui.
here is what top reports during some of the apache bench tests:

last pid: 88988;  load averages:  0.02,  0.02,  0.00   up
0+23:32:10  00:40:54
40 processes:  1 running, 39 sleeping
CPU:  0.1% user,  0.0% nice,  0.3% system,  0.4% interrupt, 99.2% idle
Mem: 95M Active, 850M Inact, 802M Wired, 176K Cache, 1505M Buf, 60G Free
Swap: 128G Total, 128G Free

  PID USERNAME  THR PRI NICE   SIZERES STATE   C   TIMEWCPU COMMAND
19560 www 1  250 94392K 48156K kqread  1   0:36  10.69% haproxy
  951 proxy  17  200  1574M   713M uwait  12   6:39   1.37% squid
28926 root1  200 16804K  2800K bpf22   5:44   0.29%
filterlog
84768 unbound32  200   290M   104M kqread 23   0:29   0.00% unbound
45071 root1  200   126M 11896K kqread 20   0:09   0.00% lighttpd

Is there an accept filter
>
Not sure I understand this question, can you explain?


> Is there some kernel messages
> Load, swap usage, disk space
>
again, according to my limited know how, top and other built in utilities
all report the system is barely doing anything and there is tons of memory
and disk space

> During stress :
> Is there more sys/interrupt than user cpu usage
> Link saturation
> Packet lost
>
I am not sure how to check this, I will try and figure this out but if you
have any advice that would be appreciated.
The LAN interface is a bonded interface with (3) 1000mb NIC cards so I am
doubtful it is being saturated from this simple apache bench test. Here is
what the Interfaces status shows me:

*Status up*
MTU 1500
Media autoselect
LAGG Protocol lacp lagghash l2,l3,l4
LAGG Ports bge3 flags=1c
bge2 flags=1c
bge1 flags=1c
In/out packets 248989670/305051696 (77.73 GB/88.68 GB)
In/out packets (pass) 248989670/305051696 (77.73 GB/88.68 GB)
In/out packets (block) 4130394/147 (4.75 GB/70 KB)
In/out errors 0/608
Collisions 0

Suboptimal firewall rules : replay stress packet filter unloaded.
>
There are only two simple allow firewall rules for LAN access, nothing
complicated at all.

I am really stumped by this problem and am hoping you guys can help me get
this figured out. If there are any commands I can run to get info that
would be helpful please let me know.

Thanks..

Re: HAProxy Slows At 1500+ connections Really Need some help to figure out why

[Daren Sefcik]

Thanks Bryan/Cyril for trying to help me outI am not super familiar
with dealing with systems at that level so I may need a little hand
holding...

Here is what the system currently tells me:

[2.2.4-RELEASE][root@HTPL-PROXY-03]/root:* pfctl -si | grep current*
  current entries 6788
[2.2.4-RELEASE][root@HTPL-PROXY-03]/root: *pfctl -sm*
stateshard limit  654
src-nodes hard limit  654
frags hard limit 5000
table-entries hard limit   20

and haproxy stats shows this: (i have offloaded my clients for now but no
limits are reached when the slow down happens, not even close)

*maxsock = *100043; *maxconn = *5; *maxpipes = *0
current conns = 292; current pipes = 0/0; conn rate = 22/sec
Running tasks: 1/311; idle = 99 %

Based on the comments from Cyril I made the following changes (I did have
the maxconn numbers set before for the frontend(s) when the slowness
occurred but I took them out trying to solve the problem which probably
made it worse)

*/*var/etc/haproxy.cfg file contents:

global
maxconn 5
log /var/run/log local0 info
stats socket /tmp/haproxy.socket level admin
uid 80
gid 80
nbproc 1
chroot /tmp/haproxy_chroot
daemon
spread-checks 5

listen HAProxyLocalStats
bind 127.0.0.1:2200 name localstats
mode http
stats enable
stats admin if TRUE
stats uri /haproxy_stats.php?haproxystats=1
timeout client 5000
timeout connect 5000
timeout server 5000

frontend HTPL_PROXY
bind 10.1.4.105:8181 name 10.1.4.105:8181
mode http
log global
option http-server-close
option forwardfor
acl https ssl_fc
reqadd X-Forwarded-Proto:\ http if !https
reqadd X-Forwarded-Proto:\ https if https
maxconn 4
timeout client 5000
default_backend HTPL_WEB_PROXY_http_ipvANY

frontend HTPL_CONTENT_FILTER
bind 10.1.4.106:8182 name 10.1.4.106:8182
mode tcp
log global
maxconn 1
timeout client 5000
default_backend HTPL_CONT_FILTER_tcp_ipvANY

backend HTPL_WEB_PROXY_http_ipvANY
mode http
cookie SERVERID insert indirect
balance roundrobin
timeout connect 5
timeout server 5
retries 3
server HTPL-PROXY-01 10.1.4.103:3128 cookie HTPLPROXY01 check inter 6
 weight 150 fastinter 1000 fall 5
server HTPL-PROXY-02 10.1.4.104:3128 cookie HTPLPROXY02 check inter 6
 weight 100 fastinter 1000 fall 5
server HTPL-PROXY-03 10.1.4.107:3128 cookie HTPLPROXY03 check inter 6
 weight 50 fastinter 1000 fall 5
server HTPL-PROXY-04 10.1.4.108:3128 cookie HTPLPROXY04 check inter 6
 weight 200 fastinter 1000 fall 5
server HTHPL-PROXY-01 10.1.4.101:3128 cookie HTHPLPROXY01 check inter 6
disabled weight 150 fastinter 1000 fall 5
server HTHPL-PROXY-02 10.1.4.102:3128 cookie HTHPLPROXY02 check inter 6
disabled weight 100 fastinter 1000 fall 5

backend HTPL_CONT_FILTER_tcp_ipvANY
mode tcp
balance roundrobin
timeout connect 5
timeout server 5
retries 3
server HTHPL-PROXY-01 10.1.4.101:1344 check inter 6 disabled weight 100
fastinter 1000 fall 5
server HTHPL-PROXY-02 10.1.4.102:1344 check inter 6 disabled weight 100
fastinter 1000 fall 5
server HTPL-WEB-01 10.1.4.153:1344 check inter 6  weight 200 fastinter
1000 fall 5
server HTPL-WEB-02 10.1.4.154:1344 check inter 6  weight 200 fastinter
1000 fall 5



On Fri, Oct 2, 2015 at 2:17 PM, Bryan Talbot <bryan.tal...@ijji.com> wrote:

> On Fri, Oct 2, 2015 at 1:48 PM, Daren Sefcik <dsef...@hightechhigh.org>
> wrote:
>
>> I Hope this is the right place to ask for help..if not please flame me
>> and send me on my way
>>
>> So I had haproxy 1.5 installed (as a front end for a cluster of squid
>> proxies) on a low end Dell server with pfsense(PFS) 2.1.5 and was
>> experiencing slow down with 1500+ connections so I  built up a new PFS
>> 2.2.4 machine on a brand new Dell R630  with 64gb RAM, Dual CPU,  bad ass
>> raid disks etcloaded and configured haproxy with several squid backends
>> and some ICAP  backends. Things work great until I hit about 1500 or more
>> connections and then everything just slows to a crawl. Restarting haproxy
>> helps momentarily but it will slow back down again very quickly. If I
>> offload clients to the point of only 300-400 connections it will become
>> responsive again. In the haproxy stats page it will show 97% idle or
>> similar and the output from top will show maybe 5% cpu for haproxy. If I
>> configure the browser client to use one of the squid backends directly it
>> works fast but as soon as I put the broswer proxy config back to use the
>> haproxy frontend IP it will slow down.
>>
>
>
> The problem seems consistent with your connection tracking tables filling
> up. You don't say if the 1500 concurrent connections creates a lot of new
> connections or if they are 1500 connections that last for a long time. If
> your connection lifetime is short then the connection tracking tables
> probably need to be tuned.
>
> I don't recall what the conntrack controls are for FreeBSD but it's
> probably something in the pfctl utility, right?
>
> -Bryan
>
>

Re: HAProxy Slows At 1500+ connections Really Need some help to figure out why

[Daren Sefcik]

So after making the changes (somewhat implied by Cyril) I ran apache bench
with 2 concurrent instances of "-n 1 -c 500 -w -k" and the result on
haproxy stats page is:

pid = 18093 (process #1, nbproc = 1)
uptime = 0d 2h55m08s
system limits: memmax = unlimited; ulimit-n = 100043
maxsock = 100043; maxconn = 5; maxpipes = 0
current conns = 2235; current pipes = 0/0; conn rate = 39/sec
Running tasks: 1/2252; idle = 85 %

and response times from the client are unacceptable, 15-20 seconds or
longer. once the apache bench tests finish and concurrent conns go down to
a few hundred or less the client response times are normal and quick. Not
scientific but during the long wait on the client the browser reports down
in the bottom browser bar "waiting for socket..." or "waiting for proxy
tunnel..."

TIA for any further help anyone can provide, I really would like to get
this figured out.