Re: Help tracking "connection refused" under pressure on v2.9

[Ricardo Nabinger Sanchez]

Hi Willy,

On Fri, 29 Mar 2024 07:17:56 +0100
Willy Tarreau  wrote:

> > These "connection refused" is from our watchdog; but the effects are as
> > perceptible from the outside.  When our watchdog hits this situation,
> > it will forcefully restart HAProxy (we have 2 instances) because there
> > will be a considerable service degradation.  If you remember, there's
> > https://github.com/haproxy/haproxy/issues/1895 and we talked briefly
> > about this in person, at HAProxyConf.  
> 
> Thanks for the context. I didn't remember about the issue. I remembered
> we discussed for a while but didn't remember about the issue in question
> obviously, given the number of issues I'm dealing with :-/
> 
> In the issue above I'm seeing an element from Felipe saying that telnet
> to port 80 can take between 3 seconds to accept. That really makes me
> think about either the SYN queue being full, causing drops and retransmits,
> or a lack of socket memory to accept packets. That one could possibly be
> caused by tcp_mem not being large enough due to some transfers with high
> latency fast clients taking a lot of RAM, but it should not affect the
> local UNIX socket. Also, killing the process means killing all the
> associated connections and will definitely result in freeing a huge
> amount of network buffers, so it could fuel that direction. If you have
> two instances, did you notice if the two start to behave badly at the
> same time ? If that's the case, it would definitely indicate a possible
> resource-based cause like socket memory etc.

Of our 2 HAProxy instances, it is usually one (mostly the frontend one)
that exhibits this behavior.  And as it is imperative that the
corrective action be as swift as possible, all instances are terminated
(which can include older instances, from graceful reloads), and new
instances are started.  Very harsh, but at >50 Gbps, each full second
of downtime adds up considerably to network pressure.

So for context, our least capable machine has 256 GB RAM.  We have not
seen any spikes over the metrics we monitor, and this issue tends to
happen at a very stable steady-state, albeit a loaded one.  While it is
currently outside of our range for detailed data, we didn't notice
anything unusual, especially regarding memory usage, on these traps we
reported.

But of course, there could be a metric that we're not yet aware that
correlates.  Anyone from the dustier, darkest corners that you know of?
:-)


> 
> > But this is incredibly elusive to reproduce; it comes and goes.  It
> > might happen every few minutes, or not happen at all for months.  Not
> > tied to a specific setup: different versions, kernels, machines.  In
> > fact, we do not have better ways to detect the situation, at least not
> > as fast, reactive, and resilient.  
> 
> It might be useful to take periodic snapshots of /proc/slabinfo and
> see if something jumps during such incidents (grep for TCP, net, skbuff
> there). I guess you have not noticed any "out of socket memory" nor such
> indications in your kernel logs, of course :-/

We have no indications of memory pressure related to network.  At the
peak, we usually see something like 15~22% overall active memory (it
fails me, but it might take >70% of active memory for these machines to
actually degrade, maybe more).  As for TCP stuff, around 16~30k active
sockets, plus some 50~100k in timewait, and still not creating any
problems.


> 
> Another one that could make sense to monitor is "PoolFailed" in
> "show info". It should always remain zero.

We collect this (all available actually); I don't remember this one
ever measuring more than zero.  But we'll keep an eye on it.

In time, could this be somewhat unrelated to HAProxy?  I.e., maybe
kernel?

Cheers,

-- 
Ricardo Nabinger Sanchez https://www.taghos.com.br/

Re: Help tracking "connection refused" under pressure on v2.9

[Willy Tarreau]

Hi Ricardo,

On Thu, Mar 28, 2024 at 06:21:16PM -0300, Ricardo Nabinger Sanchez wrote:
> Hi Willy,
> 
> On Thu, 28 Mar 2024 04:37:11 +0100
> Willy Tarreau  wrote:
> 
> > Thanks guys! So there seems to be an annoying bug. However I'm not sure
> > how this is related to your "connection refused", except if you try to
> > connect at the moment the process crashes and restarts, of course. I'm
> > seeing that the bug here is stktable_requeue_exp() calling task_queue()
> > with an invalid task expiration. I'm having a look now. I'll respond in
> > the issue with what I can find, thanks for your report.
> 
> These "connection refused" is from our watchdog; but the effects are as
> perceptible from the outside.  When our watchdog hits this situation,
> it will forcefully restart HAProxy (we have 2 instances) because there
> will be a considerable service degradation.  If you remember, there's
> https://github.com/haproxy/haproxy/issues/1895 and we talked briefly
> about this in person, at HAProxyConf.

Thanks for the context. I didn't remember about the issue. I remembered
we discussed for a while but didn't remember about the issue in question
obviously, given the number of issues I'm dealing with :-/

In the issue above I'm seeing an element from Felipe saying that telnet
to port 80 can take between 3 seconds to accept. That really makes me
think about either the SYN queue being full, causing drops and retransmits,
or a lack of socket memory to accept packets. That one could possibly be
caused by tcp_mem not being large enough due to some transfers with high
latency fast clients taking a lot of RAM, but it should not affect the
local UNIX socket. Also, killing the process means killing all the
associated connections and will definitely result in freeing a huge
amount of network buffers, so it could fuel that direction. If you have
two instances, did you notice if the two start to behave badly at the
same time ? If that's the case, it would definitely indicate a possible
resource-based cause like socket memory etc.

> But this is incredibly elusive to reproduce; it comes and goes.  It
> might happen every few minutes, or not happen at all for months.  Not
> tied to a specific setup: different versions, kernels, machines.  In
> fact, we do not have better ways to detect the situation, at least not
> as fast, reactive, and resilient.

It might be useful to take periodic snapshots of /proc/slabinfo and
see if something jumps during such incidents (grep for TCP, net, skbuff
there). I guess you have not noticed any "out of socket memory" nor such
indications in your kernel logs, of course :-/

Another one that could make sense to monitor is "PoolFailed" in
"show info". It should always remain zero.

> > Since you were speaking about FD count and maxconn at 900k, please let
> > me take this opportunity for a few extra sanity checks. By default we
> > assign up to about 50% of the FD to pipes (i.e. up to 25% pipes compared
> > to connections), so if maxconn is 900k you can reach 1800 + 900 = 2700k
> > FD. One thing to keep in mind is that /proc/sys/fs/nr_open sets a
> > per-process hard limit and usually is set to 1M, and that
> > /proc/sys/fs/file-max sets a system-wide limit and depends on the amount
> > of RAM, so both may interact with such a large setting. We could for
> > example imagine that at ~256k connections with as many pipes you're
> > reaching around 1M FDs and that the connection from socat to the CLI
> > socket cannot be accepted and is rejected. Since you recently updated
> > your kernel, it might be worth checking if the default values are still
> > in line with your usage.
> 
> We set our defaults pretty high in anticipation:
> 
>   /proc/sys/fs/file-max = 5M;
>   /proc/sys/fs/nr_open = 3M;

OK!

> Even with our software stack, we do not reach the limits.  A long time
> ago we did hit (lower limits back then) and the effects are devastating.

Yes, that's always the problem with certain limits, they hit you at the
worst ever moments, when the most users are counting on you to work fine
and when it's the hardest to spot anomalies.

Willy

Re: Help tracking "connection refused" under pressure on v2.9

[Ricardo Nabinger Sanchez]

Hi Willy,

On Thu, 28 Mar 2024 04:37:11 +0100
Willy Tarreau  wrote:

> Thanks guys! So there seems to be an annoying bug. However I'm not sure
> how this is related to your "connection refused", except if you try to
> connect at the moment the process crashes and restarts, of course. I'm
> seeing that the bug here is stktable_requeue_exp() calling task_queue()
> with an invalid task expiration. I'm having a look now. I'll respond in
> the issue with what I can find, thanks for your report.

These "connection refused" is from our watchdog; but the effects are as
perceptible from the outside.  When our watchdog hits this situation,
it will forcefully restart HAProxy (we have 2 instances) because there
will be a considerable service degradation.  If you remember, there's
https://github.com/haproxy/haproxy/issues/1895 and we talked briefly
about this in person, at HAProxyConf.

But this is incredibly elusive to reproduce; it comes and goes.  It
might happen every few minutes, or not happen at all for months.  Not
tied to a specific setup: different versions, kernels, machines.  In
fact, we do not have better ways to detect the situation, at least not
as fast, reactive, and resilient.


> 
> Since you were speaking about FD count and maxconn at 900k, please let
> me take this opportunity for a few extra sanity checks. By default we
> assign up to about 50% of the FD to pipes (i.e. up to 25% pipes compared
> to connections), so if maxconn is 900k you can reach 1800 + 900 = 2700k
> FD. One thing to keep in mind is that /proc/sys/fs/nr_open sets a
> per-process hard limit and usually is set to 1M, and that
> /proc/sys/fs/file-max sets a system-wide limit and depends on the amount
> of RAM, so both may interact with such a large setting. We could for
> example imagine that at ~256k connections with as many pipes you're
> reaching around 1M FDs and that the connection from socat to the CLI
> socket cannot be accepted and is rejected. Since you recently updated
> your kernel, it might be worth checking if the default values are still
> in line with your usage.

We set our defaults pretty high in anticipation:

/proc/sys/fs/file-max = 5M;
/proc/sys/fs/nr_open = 3M;

Even with our software stack, we do not reach the limits.  A long time
ago we did hit (lower limits back then) and the effects are devastating.

Cheers,

-- 
Ricardo Nabinger Sanchez https://www.taghos.com.br/

Re: Help tracking "connection refused" under pressure on v2.9

[Willy Tarreau]

On Wed, Mar 27, 2024 at 02:26:47PM -0300, Ricardo Nabinger Sanchez wrote:
> On Wed, 27 Mar 2024 11:06:39 -0300
> Felipe Wilhelms Damasio  wrote:
> 
> > kernel: traps: haproxy[2057993] trap invalid opcode ip:5b3e26
> > sp:7fd7c002f100 error:0 in haproxy[42c000+1f7000]
> 
> We managed to get a core file, and so created ticket #2508
> (https://github.com/haproxy/haproxy/issues/2508) with more details.

Thanks guys! So there seems to be an annoying bug. However I'm not sure
how this is related to your "connection refused", except if you try to
connect at the moment the process crashes and restarts, of course. I'm
seeing that the bug here is stktable_requeue_exp() calling task_queue()
with an invalid task expiration. I'm having a look now. I'll respond in
the issue with what I can find, thanks for your report.

Since you were speaking about FD count and maxconn at 900k, please let
me take this opportunity for a few extra sanity checks. By default we
assign up to about 50% of the FD to pipes (i.e. up to 25% pipes compared
to connections), so if maxconn is 900k you can reach 1800 + 900 = 2700k
FD. One thing to keep in mind is that /proc/sys/fs/nr_open sets a
per-process hard limit and usually is set to 1M, and that
/proc/sys/fs/file-max sets a system-wide limit and depends on the amount
of RAM, so both may interact with such a large setting. We could for
example imagine that at ~256k connections with as many pipes you're
reaching around 1M FDs and that the connection from socat to the CLI
socket cannot be accepted and is rejected. Since you recently updated
your kernel, it might be worth checking if the default values are still
in line with your usage.

Cheers,
Willy

Re: Help tracking "connection refused" under pressure on v2.9

[Ricardo Nabinger Sanchez]

On Wed, 27 Mar 2024 11:06:39 -0300
Felipe Wilhelms Damasio  wrote:

> kernel: traps: haproxy[2057993] trap invalid opcode ip:5b3e26
> sp:7fd7c002f100 error:0 in haproxy[42c000+1f7000]

We managed to get a core file, and so created ticket #2508
(https://github.com/haproxy/haproxy/issues/2508) with more details.

Cheers,

-- 
Ricardo Nabinger Sanchez https://www.taghos.com.br/

Re: Help tracking "connection refused" under pressure on v2.9

[Ricardo Nabinger Sanchez]

On Wed, 27 Mar 2024 11:06:39 -0300
Felipe Wilhelms Damasio  wrote:

> kernel: traps: haproxy[2057993] trap invalid opcode ip:5b3e26 sp:7fd7c002f100 
> error:0 in haproxy[42c000+1f7000]

In our build, this would be where instruction pointer was:

(gdb) list *0x5b10e6
0x5b10e6 is in __task_queue (src/task.c:285).
280(wq != _ctx->timers && wq != _ctx->timers));
281 #endif
282 /* if this happens the process is doomed anyway, so better 
catch it now
283  * so that we have the caller in the stack.
284  */
285 BUG_ON(task->expire == TICK_ETERNITY);
286
287 if (likely(task_in_wq(task)))
288 __task_unlink_wq(task);
289

However, we can't produce a stack trace from only the instruction
pointer; at least I don't know how (but would love to learn if it is
possible).

We are trying to get a core dump, too.

Cheers,

-- 
Ricardo Nabinger Sanchez https://www.taghos.com.br/

Re: Help tracking "connection refused" under pressure on v2.9

[Felipe Wilhelms Damasio]

Hi,

We've confirmed a few findings after we poured ~75-80Gbps of traffic
on purpose on a single machine:
- haproxy does indeed crashes;
- hence, we have no stats socket to collect a few things;

It seems that under pressure (not sure which conditions yet) the
kernel seems to be killing it. dmesg shows:

kernel: traps: haproxy[2057993] trap invalid opcode ip:5b3e26
sp:7fd7c002f100 error:0 in haproxy[42c000+1f7000]

This is a relatively new kernel:

Linux ndt-spo-12 6.1.60 #1 SMP PREEMPT_DYNAMIC Wed Oct 25 19:17:36 UTC
2023 x86_64 Intel(R) Xeon(R) Gold 6338N CPU @ 2.20GHz GenuineIntel
GNU/Linux

And it seems to happen on different kernels.

Does anyone have any tips on how to proceed to track this down?

Before the crash, "show info" showed only around 27,000 CurConn, so
not a great deal for maxconn 90.

Thanks!

On Tue, Mar 26, 2024 at 11:33 PM Felipe Wilhelms Damasio
 wrote:
>
> Hi,
>
> Since we don't really know how to track this one, we thought it might
> be better to reach out here to get feedback.
>
> We're using haproxy to deliver streaming files under pressure
> (80-90Gbps per machine). When using h1/http, splice-response is a
> great help to keep load under control. We use branch v2.9 at the
> moment.
>
> However, we've hit a bug with splice-response (Github issue created)
> and we had to use all day our haproxies without splicing.
>
> When we reach a certain load, a "connection refused" alarm starting
> buzzing like crazy (2-3 times every 30 minutes). This alarm is simply
> a connect to localhost with 500ms timeout:
>
> socat /dev/null  tcp4-connect:127.0.0.1:80,connect-timeout=0.5
>
> The log file indicates the port is virtually closed:
>
> 2024/03/27 01:06:04 socat[984480] E read(6, 0xe98000, 8192): Connection 
> refused
>
> The thing is haproxy process is very much alive...so we just restart
> it everytime this happens.
>
> What data do you suggest we collect to help track this down? Not sure
> if the stats socket is available, but we can definitely try and get
> some information.
>
> We're not running out of fds, or even connections with/without backlog
> (we have a global maxconn of 90 with ~30,000 streaming sessions
> active and we have tcp_max_syn_backlog set to 262144), we checked. But
> it seems to correlate with heavy traffic.
>
> Thanks!
>
> --
> Felipe Damasio



-- 
Felipe Damasio

Re: Help with peer setup and "srv_conn(bk_customer/haproxy-dest1)"

[Aleksandar Lazic]

Hi.

Anyone which can help to protect the backen with backend states?

Regards
Alex

On 05.12.21 11:42, Aleksandar Lazic wrote:


Hi.

I try to protect an backend server against a overload within a master/master 
setup.
The test setup looks like this

lb1: 8081 \
    -hap-dest: 8080
lb2: 8082 /

When I now call lb1 with curl the "tracker/quota1" gpc is increased and the 
second request is denied.
The problem is that the peer on lb2 does not get the counter data to protect 
the backend on lb2 too.

Please can anybody help me to fix my mistake and find a proper solution.


```
curl -v http://127.0.0.1:8081/; curl -v http://127.0.0.1:8081
* Connected to 127.0.0.1 (127.0.0.1) port 8081 (#0)
 > GET / HTTP/1.1
 > Host: 127.0.0.1:8081
 > User-Agent: curl/7.68.0
 > Accept: */*
 >
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< dest_dst_conn: 1
< content-length: 0
<
* Connection #0 to host 127.0.0.1 left intact


* Connected to 127.0.0.1 (127.0.0.1) port 8081 (#0)
 > GET / HTTP/1.1
 > Host: 127.0.0.1:8081

< HTTP/1.1 403 Forbidden
< content-length: 93
< cache-control: no-cache
< content-type: text/html

```

``` lb1
echo "show peers;show table tracker/quota1;show table tracker/quota2"|socat - 
tcp4-connect:127.0.0.1:9990

0x55bb71554dc0: [05/Dec/2021:10:27:17] id=tracker disabled=0 flags=0x33 
resync_timeout= task_calls=5
   0x55bb71558350: id=tracker(remote,inactive) addr=127.0.0.1:20001 
last_status=NAME last_hdshk=5m36s
     reconnect= heartbeat= confirm=0 tx_hbt=0 rx_hbt=0 
no_hbt=0 new_conn=1 proto_err=0 coll=0
     flags=0x0
     shared tables:
   0x55bb7156f1e0 local_id=2 remote_id=0 flags=0x0 remote_data=0x0
   last_acked=0 last_pushed=0 last_get=0 teaching_origin=0 update=0
   table:0x55bb71556a50 id=tracker/quota1 update=3 localupdate=3 
commitupdate=0 refcnt=1
     Dictionary cache not dumped (use "show peers dict")
   0x55bb7156f090 local_id=1 remote_id=0 flags=0x0 remote_data=0x0
   last_acked=0 last_pushed=0 last_get=0 teaching_origin=0 update=0
   table:0x55bb71556c60 id=tracker/quota2 update=2 localupdate=2 
commitupdate=0 refcnt=1
     Dictionary cache not dumped (use "show peers dict")
   0x55bb71557300: id=h1(local,inactive) addr=127.0.0.1:2 last_status=NONE 
last_hdshk=
     reconnect= heartbeat= confirm=0 tx_hbt=0 rx_hbt=0 
no_hbt=0 new_conn=0 proto_err=0 coll=0
     flags=0x0
     shared tables:
   0x55bb7156f230 local_id=2 remote_id=0 flags=0x0 remote_data=0x0
   last_acked=0 last_pushed=0 last_get=0 teaching_origin=0 update=0
   table:0x55bb71556a50 id=tracker/quota1 update=3 localupdate=3 
commitupdate=0 refcnt=1
     Dictionary cache not dumped (use "show peers dict")
   0x55bb7156f0e0 local_id=1 remote_id=0 flags=0x0 remote_data=0x0
   last_acked=0 last_pushed=0 last_get=0 teaching_origin=0 update=0
   table:0x55bb71556c60 id=tracker/quota2 update=2 localupdate=2 
commitupdate=0 refcnt=1
     Dictionary cache not dumped (use "show peers dict")

# table: tracker/quota1, type: string, size:100, used:1
0x55bb71772888: key=0 use=0 exp=53297 server_id=0 gpc0=1

# table: tracker/quota2, type: string, size:100, used:1
0x55bb71772958: key=0 use=0 exp=53297 server_id=0 gpc1=0

```

``` lb2
echo "show peers;show table tracker/quota1;show table tracker/quota2"|socat - 
tcp4-connect:127.0.0.1:9991

0x5618ae836dc0: [05/Dec/2021:10:27:12] id=tracker disabled=0 flags=0x33 
resync_timeout= task_calls=5
   0x5618ae83a350: id=tracker(remote,inactive) addr=127.0.0.1:2 
last_status=NAME last_hdshk=5m31s
     reconnect= heartbeat= confirm=0 tx_hbt=0 rx_hbt=0 
no_hbt=0 new_conn=2 proto_err=0 coll=0
     flags=0x0
     shared tables:
   0x5618ae8511e0 local_id=2 remote_id=0 flags=0x0 remote_data=0x0
   last_acked=0 last_pushed=0 last_get=0 teaching_origin=0 update=0
   table:0x5618ae838a50 id=tracker/quota1 update=0 localupdate=0 
commitupdate=0 refcnt=1
     Dictionary cache not dumped (use "show peers dict")
   0x5618ae851090 local_id=1 remote_id=0 flags=0x0 remote_data=0x0
   last_acked=0 last_pushed=0 last_get=0 teaching_origin=0 update=0
   table:0x5618ae838c60 id=tracker/quota2 update=0 localupdate=0 
commitupdate=0 refcnt=1
     Dictionary cache not dumped (use "show peers dict")
   0x5618ae839300: id=h2(local,inactive) addr=127.0.0.1:20001 last_status=NONE 
last_hdshk=
     reconnect= heartbeat= confirm=0 tx_hbt=0 rx_hbt=0 
no_hbt=0 new_conn=0 proto_err=0 coll=0
     flags=0x0
     shared tables:
   0x5618ae851230 local_id=2 remote_id=0 flags=0x0 remote_data=0x0
   last_acked=0 last_pushed=0 last_get=0 teaching_origin=0 update=0
   table:0x5618ae838a50 id=tracker/quota1 update=0 localupdate=0 
commitupdate=0 refcnt=1
     Dictionary cache not dumped (use "show peers dict")
  

Re: Help

[Aleksandar Lazic]

Hi.

On 16.07.21 14:34, Anilton Silva Fernandes wrote:

Hi there…

Can I get another HELP:

This time, I want to receive a request, and check for URL to know which backend 
should be call.

This is my config:

frontend web_accounts
     mode tcp
     bind 10.15.1.12:443
     default_backend accounts_servers

frontend web_apimanager
     mode tcp
     bind 10.15.1.13:443

     use_backend apiservices if   { path_beg /api/ }    
# IF THERE’S API ON THE URL SEND TO APISERVICES
     use_backend apimanager  unless   { path_beg /api }  # IF 
THERE’S NOT API, SEND IT TO APIMANAGER


This is not possible with TCP mode.
You have to switch to HTTP mode.

In this Blog post is such a example documented and more about HAProxy acls.

https://www.haproxy.com/blog/introduction-to-haproxy-acls/


backend accounts_servers
    mode tcp
    balance roundrobin
    server  accounts1 10.16.18.128:443 check

backend apimanager
    mode tcp
    balance roundrobin
    server  apimanager1 10.16.18.129:9445 check

backend apiservices
    mode tcp
    balance roundrobin
    server  apimanagerqa.cvt.cv 10.16.18.129:8245 check

Thank you

*From:*Emerson Gomes [mailto:emerson.go...@gmail.com]
*Sent:* 7 de julho de 2021 12:34
*To:* Anilton Silva Fernandes 
*Cc:* haproxy@formilux.org
*Subject:* Re: Help

Hello Anilton,

In the "bind *:443" line, do not specify a PEM file directly, but only the 
directory where your PEM file(s) resides.

Also, make sure that both the certificate and private key are contained within 
the same PEM file.

It should look like this:

-BEGIN CERTIFICATE-
    xxx
-END CERTIFICATE-
-BEGIN PRIVATE KEY-
   xxx
-END PRIVATE KEY-

BR.,

Emerson

Em qua., 7 de jul. de 2021 às 14:47, Anilton Silva Fernandes mailto:anilton.fernan...@cvt.cv>> escreveu:

Hi there.

Can I get some help from you.

I’m configuring HAProxy as a frontend on HTTPS with centified and I want 
clients to be redirect to BACKEND on HTTPS as well (443) but I want clients to 
see only HAProxy certificate, as the backend one is not valid.

Bellow the schematic of my design:

So, on

This is the configuration file I’m using:




frontend haproxy mode http bind *:80 bind *:443 ssl crt 
/etc/ssl/cvt.cv/accounts_cvt.pem default_backend wso2 backend wso2 mode http 
option forwardfor redirect scheme https if !{ ssl_fc } server my-api 
10.16.18.128:443 check ssl verify none http-request set-header X-Forwarded-Port 
%[dst_port] http-request add-header X-Forwarded-Proto https if { ssl_fc }



frontend web_accounts mode tcp bind 192.168.1.214:443 default_backend 
accounts_servers frontend web_apimanager mode tcp bind 192.168.1.215:443 
default_backend apimanager_servers backend accounts_servers balance roundrobin 
server accounts1 10.16.18.128:443 check server accounts2 10.16.18.128:443 check 
backend apimanager_servers balance roundrobin server accounts1 10.16.18.128:443 
check server accounts2 10.16.18.128:443 check





The first one is what works but we got SSL problems due to invalid 
certificates on Backend;

The second one is what we would like, but does not work and says some erros:

[ALERT] 187/114337 (7823) : parsing [/etc/haproxy/haproxy.cfg:85] : 'bind *:443' 
: unable to load SSL private key from PEM file '/etc/ssl/cvt.cv/accounts_cvt.pem 
<http://cvt.cv/accounts_cvt.pem>'.

[ALERT] 187/114337 (7823) : Error(s) found in configuration file : 
/etc/haproxy/haproxy.cfg

[ALERT] 187/114337 (7823) : Proxy 'haproxy': no SSL certificate specified 
for bind '*:443' at [/etc/haproxy/haproxy.cfg:85] (use 'crt').

[ALERT] 187/114337 (7823) : Fatal errors found in configuration.

Errors in configuration file, check with haproxy check.

This is on CentOS 6

Thank you

Melhores Cumprimentos

**

*Anilton Fernandes | Plataformas, Sistemas e Infraestruturas*

Cabo Verde Telecom, SA

Group Cabo Verde Telecom

Rua Cabo Verde Telecom, 1, Edificio CVT

198, Praia, Santiago, República de Cabo Verde

Phone: +238 3503934 | Mobile: +238 9589123 | Email – anilton.fernan...@cvt.cv 
<mailto:anilton.fernan...@cvt.cv>

cid:image001.jpg@01D5997A.B9848FB0

RE: Help

[Anilton Silva Fernandes]

Hi there…

Can I get another HELP:

This time, I want to receive a request, and check for URL to know which backend 
should be call.

This is my config:

frontend web_accounts
mode tcp
bind 10.15.1.12:443
default_backend accounts_servers

frontend web_apimanager
mode tcp
bind 10.15.1.13:443
use_backend apiservices if   { path_beg /api/ }
# IF THERE’S API ON THE URL SEND TO APISERVICES
use_backend apimanager  unless   { path_beg /api }  # IF 
THERE’S NOT API, SEND IT TO APIMANAGER


backend accounts_servers
   mode tcp
   balance roundrobin
   server  accounts1 10.16.18.128:443 check

backend apimanager
   mode tcp
   balance roundrobin
   server  apimanager1 10.16.18.129:9445 check


backend apiservices
   mode tcp
   balance roundrobin
   server  apimanagerqa.cvt.cv 10.16.18.129:8245 check


Thank you

From: Emerson Gomes [mailto:emerson.go...@gmail.com]
Sent: 7 de julho de 2021 12:34
To: Anilton Silva Fernandes 
Cc: haproxy@formilux.org
Subject: Re: Help

Hello Anilton,

In the "bind *:443" line, do not specify a PEM file directly, but only the 
directory where your PEM file(s) resides.
Also, make sure that both the certificate and private key are contained within 
the same PEM file.

It should look like this:

-BEGIN CERTIFICATE-
   xxx
-END CERTIFICATE-
-BEGIN PRIVATE KEY-
  xxx
-END PRIVATE KEY-

BR.,
Emerson

Em qua., 7 de jul. de 2021 às 14:47, Anilton Silva Fernandes 
mailto:anilton.fernan...@cvt.cv>> escreveu:
Hi there.

Can I get some help from you.

I’m configuring HAProxy as a frontend on HTTPS with centified and I want 
clients to be redirect to BACKEND on HTTPS as well (443) but I want clients to 
see only HAProxy certificate, as the backend one is not valid.

Bellow the schematic of my design:

[cid:image001.png@01D77A35.28392CD0]


So, on

This is the configuration file I’m using:



[frontend haproxy mode http bind *:80 bind *:443 ssl crt 
/etc/ssl/cvt.cv/accounts_cvt.pem default_backend wso2backend wso2 
mode http option forwardfor redirect scheme https if !{ ssl_fc } 
server my-api 10.16.18.128:443 check ssl verify none http-request 
set-header X-Forwarded-Port %[dst_port] http-request add-header 
X-Forwarded-Proto https if { ssl_fc }]


[frontend web_accounts  mode tcp  bind 192.168.1.214:443  
default_backend accounts_serversfrontend web_apimanager  mode tcp  
bind 192.168.1.215:443  default_backend apimanager_serversbackend 
accounts_servers  balance roundrobin  server  accounts1 
10.16.18.128:443 check  server  accounts2 10.16.18.128:443 checkbackend 
apimanager_servers  balance roundrobin  server  accounts1 
10.16.18.128:443 check  server  accounts2 10.16.18.128:443 check]




























The first one is what works but we got SSL problems due to invalid certificates 
on Backend;

The second one is what we would like, but does not work and says some erros:
[ALERT] 187/114337 (7823) : parsing [/etc/haproxy/haproxy.cfg:85] : 'bind 
*:443' : unable to load SSL private key from PEM file 
'/etc/ssl/cvt.cv/accounts_cvt.pem<http://cvt.cv/accounts_cvt.pem>'.
[ALERT] 187/114337 (7823) : Error(s) found in configuration file : 
/etc/haproxy/haproxy.cfg
[ALERT] 187/114337 (7823) : Proxy 'haproxy': no SSL certificate specified for 
bind '*:443' at [/etc/haproxy/haproxy.cfg:85] (use 'crt').
[ALERT] 187/114337 (7823) : Fatal errors found in configuration.
Errors in configuration file, check with haproxy check.


This is on CentOS 6

Thank you




Melhores Cumprimentos

Anilton Fernandes | Plataformas, Sistemas e Infraestruturas
Cabo Verde Telecom, SA
Group Cabo Verde Telecom
Rua Cabo Verde Telecom, 1, Edificio CVT
198, Praia, Santiago, República de Cabo Verde
Phone: +238 3503934 | Mobile: +238 9589123 | Email – 
anilton.fernan...@cvt.cv<mailto:anilton.fernan...@cvt.cv>

[cid:image001.jpg@01D5997A.B9848FB0]

Re: Help

[Emerson Gomes]

Hello Anilton,

In the "bind *:443" line, do not specify a PEM file directly, but only the
directory where your PEM file(s) resides.
Also, make sure that both the certificate and private key are contained
within the same PEM file.

It should look like this:

-BEGIN CERTIFICATE-
   xxx
-END CERTIFICATE-
-BEGIN PRIVATE KEY-
  xxx
-END PRIVATE KEY-

BR.,
Emerson

Em qua., 7 de jul. de 2021 às 14:47, Anilton Silva Fernandes <
anilton.fernan...@cvt.cv> escreveu:

> Hi there.
>
>
>
> Can I get some help from you.
>
>
>
> I’m configuring HAProxy as a frontend on HTTPS with centified and I want
> clients to be redirect to BACKEND on HTTPS as well (443) but I want clients
> to see only HAProxy certificate, as the backend one is not valid.
>
>
>
> Bellow the schematic of my design:
>
>
>
>
>
>
>
> So, on
>
>
>
> This is the configuration file I’m using:
>
>
>
> [image: frontend haproxy mode http bind *:80 bind *:443 ssl crt
> /etc/ssl/cvt.cv/accounts_cvt.pem default_backend wso2 backend wso2 mode
> http option forwardfor redirect scheme https if !{ ssl_fc } server my-api
> 10.16.18.128:443 check ssl verify none http-request set-header
> X-Forwarded-Port %[dst_port] http-request add-header X-Forwarded-Proto
> https if { ssl_fc }]
> [image: frontend web_accounts mode tcp bind 192.168.1.214:443
> default_backend accounts_servers frontend web_apimanager mode tcp bind
> 192.168.1.215:443 default_backend apimanager_servers backend
> accounts_servers balance roundrobin server accounts1 10.16.18.128:443 check
> server accounts2 10.16.18.128:443 check backend apimanager_servers balance
> roundrobin server accounts1 10.16.18.128:443 check server accounts2
> 10.16.18.128:443 check]
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> The first one is what works but we got SSL problems due to invalid
> certificates on Backend;
>
>
>
> The second one is what we would like, but does not work and says some
> erros:
>
> [ALERT] 187/114337 (7823) : parsing [/etc/haproxy/haproxy.cfg:85] : 'bind
> *:443' : unable to load SSL private key from PEM file '/etc/ssl/
> cvt.cv/accounts_cvt.pem'.
>
> [ALERT] 187/114337 (7823) : Error(s) found in configuration file :
> /etc/haproxy/haproxy.cfg
>
> [ALERT] 187/114337 (7823) : Proxy 'haproxy': no SSL certificate specified
> for bind '*:443' at [/etc/haproxy/haproxy.cfg:85] (use 'crt').
>
> [ALERT] 187/114337 (7823) : Fatal errors found in configuration.
>
> Errors in configuration file, check with haproxy check.
>
>
>
>
>
> This is on CentOS 6
>
>
>
> Thank you
>
>
>
>
>
>
>
>
>
> Melhores Cumprimentos
>
>
>
> *Anilton Fernandes | Plataformas, Sistemas e Infraestruturas*
>
> Cabo Verde Telecom, SA
>
> Group Cabo Verde Telecom
>
> Rua Cabo Verde Telecom, 1, Edificio CVT
>
> 198, Praia, Santiago, República de Cabo Verde
>
> Phone: +238 3503934 | Mobile: +238 9589123 | Email –
> anilton.fernan...@cvt.cv
>
>
>
> [image: cid:image001.jpg@01D5997A.B9848FB0]
>
>
>
>
>

Re: Help

[Shawn Heisey]

On 7/7/2021 6:45 AM, Anilton Silva Fernandes wrote:


Hi there.

Can I get some help from you.

I’m configuring HAProxy as a frontend on HTTPS with centified and I 
want clients to be redirect to BACKEND on HTTPS as well (443) but I 
want clients to see only HAProxy certificate, as the backend one is 
not valid.





The second one is what we would like, but does not work and says some 
erros:


[ALERT] 187/114337 (7823) : parsing [/etc/haproxy/haproxy.cfg:85] : 
'bind *:443' : unable to load SSL private key from PEM file 
'/etc/ssl/cvt.cv/accounts_cvt.pem'.



The error is shown clearly.  It's telling you that the private key is 
not contained in the file you mentioned for your certificate.


For haproxy, the certificate file must contain three things:  1) The 
server certificate.  2) Any intermediate certificates.  3) The private 
key for the server certificate.  Order is not important for 2 and 3, but 
I'm pretty sure the first certificate in the file must be the server 
cert.  The root certificate is usually not required -- the end-user's 
browser should already contain that.  I am not aware of any way to 
specify the private key in a separate file, but one might exist that I 
have never seen.


My certificate files also contain a fourth item - "DH PARAMETERS", 
generated with "openssl dhparam 2048".  Each certificate gets its own 
dhparam, and it is regenerated each time I renew the cert.


Thanks,
Shawn

Re: help for implementation of first fetch function "sample_fetch_json_string"

[Aleksandar Lazic]

Tim,

you are great ;-)

On 08.04.21 18:14, Tim Düsterhus wrote:

Aleks,

On 4/8/21 5:07 PM, Aleksandar Lazic wrote:
http-request set-var(sess.json) %[req.hdr(Authorization),b64dec,json_string("\$.kubernetes\\.io/serviceaccount/namespace")] 


http-request set-var() does not expect the %[] syntax, because it always takes 
a sample. Even the following returns the same error message:


http-request set-var(sess.json) %[req.hdr(Authorization)]

I expect that the decoded json string is in args[0] and the 
"\$.kubernetes\\.io/serviceaccount/namespace" is in smp, is this assumption 
right?


The assumption is not correct, because you are not searching for a fetch. You 
want a converter, because you are converting an existing sample. I suggest you 
take a look at the "digest" converter. You can find it in sample.c.


Best regards
Tim Düsterhus


Thanks ,this was the hint I needed ;-)

Regards
Alex

Re: help for implementation of first fetch function "sample_fetch_json_string"

[Tim Düsterhus]

Aleks,

On 4/8/21 5:07 PM, Aleksandar Lazic wrote:
http-request set-var(sess.json) 
%[req.hdr(Authorization),b64dec,json_string("\$.kubernetes\\.io/serviceaccount/namespace")] 


http-request set-var() does not expect the %[] syntax, because it always 
takes a sample. Even the following returns the same error message:


http-request set-var(sess.json) %[req.hdr(Authorization)]

I expect that the decoded json string is in args[0] and the 
"\$.kubernetes\\.io/serviceaccount/namespace" is in smp, is this 
assumption right?


The assumption is not correct, because you are not searching for a 
fetch. You want a converter, because you are converting an existing 
sample. I suggest you take a look at the "digest" converter. You can 
find it in sample.c.


Best regards
Tim Düsterhus

Re: [HELP] slim is being set to 1

[Aleksandar Lazic]

Hi.

On 07.01.20 07:27, Shah, Aman wrote:

Hello all,

We have an haproxy server and we are getting some odd values. Value of slim is 
coming as 1 for most of the domains. The value is nowhere being set and maxconn 
is being set to 2000. Can you please help me to figure out what could be the 
actual reason behind the slim value coming as 1. Do we have to manually enforce 
it? How slim value is actually being set ?


Here's the full output from haproxy -vv:

root@haproxy01:~# haproxy -vv
HA-Proxy version 1.8.8-1ubuntu0.1 2018/05/29
Copyright 2000-2018 Willy Tarreau 

Build options :
   TARGET  = linux2628
   CPU = generic
   CC  = gcc
   CFLAGS  = -g -O2 -fdebug-prefix-map=/build/haproxy-VmwZ9X/haproxy-1.8.8=. 
-fstack-protector-strong -Wformat -Werror=format-security -Wdate-time 
-D_FORTIFY_SOURCE=2
   OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 USE_LUA=1 
USE_SYSTEMD=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_NS=1


Default settings :
   maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
Running on OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.3
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT 
IP_FREEBIND

Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.39 2016-06-14
Running on PCRE version : 8.39 2016-06-14
PCRE library supports JIT : yes
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"), 
raw-deflate("deflate"), gzip("gzip")

Built with network namespace support.

Available polling systems :
   epoll : pref=300,  test result OK
    poll : pref=200,  test result OK
  select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
     [SPOE] spoe
     [COMP] compression
     [TRACE] trace


And also one of the sample output of haproxy stats for the reference:


# haproxyctl show stat
# 
pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,comp_in,comp_out,comp_byp,comp_rsp,lastsess,last_chk,last_agt,qtime,ctime,rtime,ttime,agent_status,agent_code,agent_duration,check_desc,agent_desc,check_rise,check_fall,check_health,agent_rise,agent_fall,agent_health,addr,cookie,mode,algo,conn_rate,conn_rate_max,conn_tot,intercepted,dcon,dses, 



abc,BACKEND,0,0,0,1,1,1,216,1407,0,0,,0,0,0,0,UP,1,1,0,,0,6783,0,,1,5,0,,1,,1,0,,10,1,0,0,0,01,0,0,0,0,0,0,3086,,,0,0,1,1,,http,roundrobin,,, 


Have you set anywhere maxconn or fullconn because this are the parameters which 
are used for slim.


http://git.haproxy.org/?p=haproxy-1.8.git=search=HEAD=grep=ST_F_SLIM

What's your output of `show info`?
Can you share a minimal config.


Thanks,
Aman


Regards
Aleks

Re: Re: Help, URL does not work with CHINESE charactor?

[Willy Tarreau]

Hi!

On Tue, Dec 24, 2019 at 07:59:25PM +0800, JWD wrote:
> It works with "option accept-invalid-http-request".
> Thanks a lot.
> 
> Yes, it does not work with IE only.
> Other web browser is fine.

I already noticed this behaviour with very old versions of IE in the past,
it would do this on redirects: if the server sends binary characters in a
redirect, then it would follow them without re-encoding them. I didn't know
that more recent versions would still do this mistake.

I strongly suspect that your application is the one producing such wrong
links and that IE blindly follows them. You must really have a look there
and fix the application. Each time you enable "option accept-invalid-something"
it must be understood as a final warning for something deemed to fail sooner
or later.

Willy

Re: Re: Help, URL does not work with CHINESE charactor?

[JWD]

It works with "option accept-invalid-http-request".
Thanks a lot.

Yes, it does not work with IE only.
Other web browser is fine.




JWD

From: Lukas Tribus
Date: 2019-12-24 19:31
To: JWD
CC: Aleksandar Lazic; haproxy
Subject: Re: Re: Help, URL does not work with CHINESE charactor?
On Tue, 24 Dec 2019 at 11:46, JWD  wrote:
>
> I have tried version 1.7,1.8,2.0,2.1, all the same.
>
> Config:
> frontend www
> acl acl-app hdr(host) -i sharepoint.domain.com
> use_backend app if acl-app
> backend
> cookie HA-Server insert indirect nocache
> server app 192.168.129.66:80 cookie app check inter 30s
>
> Log:
> Dec 24 18:37:01 localhost haproxy[20108]: 192.168.134.81 - - 
> [24/Dec/2019:10:37:01 +] "" 400 0 "" "" 2423 066 "www" "www" 
> "" -1 -1 -1 -1 0 CR-- 2 2 0 0 0 0 0 "" "" "" ""
>
> # echo "show errors" | socat unix-connect:/etc/haproxy/hastats stdio
> Total events captured on [24/Dec/2019:10:13:18.909] : 3
>
> [24/Dec/2019:10:07:53.573] frontend www (#2): invalid request
>   backend  (#-1), server  (#-1), event #2
>   src 192.168.134.81:3400, session #103, session flags 0x0080
>   HTTP msg state MSG_RQURI(4), msg flags 0x, tx flags 0x
>   HTTP chunk len 0 bytes, HTTP body len 0 bytes
>   buffer flags 0x20808002, out 0 bytes, total 566 bytes
>   pending 566 bytes, wrapping at 16384, error at position 109:
>
>   0  GET 
> /CorWork/_layouts/15/TD.ECM.DoucmentDepartment/DepartmentFileDefau
>   00070+ lt.aspx?destLink=/CorWork/ProjectShare/\xB8\xC4\xBD\xF8\xCF\xEE\xC4
>   00116+ \xBF/ECM\xD0\xC2\xB9\xA6\xC4\xDC HTTP/1.1\r\n

Those are invalid requests, the URL must be encoded. Does IE really
still sends this crap after all those years?

You can try ignoring this with:
option accept-invalid-http-request

But it does not ignore everything. See:

https://cbonte.github.io/haproxy-dconv/2.0/configuration.html#4.2-option%20accept-invalid-http-request



Lukas

Re: Re: Help, URL does not work with CHINESE charactor?

[Lukas Tribus]

On Tue, 24 Dec 2019 at 11:46, JWD  wrote:
>
> I have tried version 1.7,1.8,2.0,2.1, all the same.
>
> Config:
> frontend www
> acl acl-app hdr(host) -i sharepoint.domain.com
> use_backend app if acl-app
> backend
> cookie HA-Server insert indirect nocache
> server app 192.168.129.66:80 cookie app check inter 30s
>
> Log:
> Dec 24 18:37:01 localhost haproxy[20108]: 192.168.134.81 - - 
> [24/Dec/2019:10:37:01 +] "" 400 0 "" "" 2423 066 "www" "www" 
> "" -1 -1 -1 -1 0 CR-- 2 2 0 0 0 0 0 "" "" "" ""
>
> # echo "show errors" | socat unix-connect:/etc/haproxy/hastats stdio
> Total events captured on [24/Dec/2019:10:13:18.909] : 3
>
> [24/Dec/2019:10:07:53.573] frontend www (#2): invalid request
>   backend  (#-1), server  (#-1), event #2
>   src 192.168.134.81:3400, session #103, session flags 0x0080
>   HTTP msg state MSG_RQURI(4), msg flags 0x, tx flags 0x
>   HTTP chunk len 0 bytes, HTTP body len 0 bytes
>   buffer flags 0x20808002, out 0 bytes, total 566 bytes
>   pending 566 bytes, wrapping at 16384, error at position 109:
>
>   0  GET 
> /CorWork/_layouts/15/TD.ECM.DoucmentDepartment/DepartmentFileDefau
>   00070+ lt.aspx?destLink=/CorWork/ProjectShare/\xB8\xC4\xBD\xF8\xCF\xEE\xC4
>   00116+ \xBF/ECM\xD0\xC2\xB9\xA6\xC4\xDC HTTP/1.1\r\n

Those are invalid requests, the URL must be encoded. Does IE really
still sends this crap after all those years?

You can try ignoring this with:
option accept-invalid-http-request

But it does not ignore everything. See:

https://cbonte.github.io/haproxy-dconv/2.0/configuration.html#4.2-option%20accept-invalid-http-request



Lukas

Re: Re: Help, URL does not work with CHINESE charactor?

[JWD]

I have tried version 1.7,1.8,2.0,2.1, all the same.

Config:
frontend www 
acl acl-app hdr(host) -i sharepoint.domain.com
use_backend app if acl-app 
backend
cookie HA-Server insert indirect nocache
server app 192.168.129.66:80 cookie app check inter 30s

Log:
Dec 24 18:37:01 localhost haproxy[20108]: 192.168.134.81 - - 
[24/Dec/2019:10:37:01 +] "" 400 0 "" "" 2423 066 "www" "www" 
"" -1 -1 -1 -1 0 CR-- 2 2 0 0 0 0 0 "" "" "" "" 

# echo "show errors" | socat unix-connect:/etc/haproxy/hastats stdio
Total events captured on [24/Dec/2019:10:13:18.909] : 3

[24/Dec/2019:10:07:53.573] frontend www (#2): invalid request
  backend  (#-1), server  (#-1), event #2
  src 192.168.134.81:3400, session #103, session flags 0x0080
  HTTP msg state MSG_RQURI(4), msg flags 0x, tx flags 0x
  HTTP chunk len 0 bytes, HTTP body len 0 bytes
  buffer flags 0x20808002, out 0 bytes, total 566 bytes
  pending 566 bytes, wrapping at 16384, error at position 109:

  0  GET /CorWork/_layouts/15/TD.ECM.DoucmentDepartment/DepartmentFileDefau
  00070+ lt.aspx?destLink=/CorWork/ProjectShare/\xB8\xC4\xBD\xF8\xCF\xEE\xC4
  00116+ \xBF/ECM\xD0\xC2\xB9\xA6\xC4\xDC HTTP/1.1\r\n
  00138  Accept: text/html, application/xhtml+xml, image/jxr, */*\r\n
  00196  Accept-Language: zh-CN\r\n
  00220  User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; Touch; r
  00290+ v:11.0) like Gecko\r\n
  00310  Accept-Encoding: gzip, deflate\r\n
  00342  Host: app.td-tech.com\r\n
  00365  Connection: Keep-Alive\r\n
  00389  Cookie: HA-Server=app; WSS_FullScreenMode=false; stsSyncAppName=Outloo
  00459+ k; stsSyncIconPath=%2F%5Flayouts%2F15%2Fimages%2Fmenuoutl%2Egif; Ribbo
  00529+ n.Document=973830|-1|0|-533254637\r\n
  00564  \r\n





JWD

From: Aleksandar Lazic
Date: 2019-12-24 17:08
To: JWD; haproxy
Subject: Re: Help, URL does not work with CHINESE charactor?
Hi JWD.

On 24.12.19 02:53, JWD wrote:
> Hi,all
> I have a backend, which is sharepoint website.
> If URL include CHINESE charactor, it return HTTP 400 ERROR from IE 11 with 
> haproxy.
> But it is ok without haproxy.
> Can anyone help me?
> Thanks.
> This can not access, return HTTP 400 ERROR:
> http://sharepoint.domain.com/CorWork/_layouts/15/TD.ECM.DoucmentDepartment 
> /DepartmentFileDefault.aspx?destLink=/CorWork/ProjectShare/改进项目/知识管理一期 
> 项目/年度评审会
> This is ok, if encode URL:
> http://sharepoint.domain.com/CorWork/_layouts/15/TD.ECM.DoucmentDepartment/DepartmentFileDefault.aspx?destLink=/CorWork/ProjectShare/%e6%94%b9%e8%bf%9b%e9%a1%b9%e7%9b%ae/%e7%9f%a5%e8%af%86%e7%ae%a1%e7%90%86%e4%b8%80%e6%9c%9f%e9%a1%b9%e7%9b%ae/%e5%b9%b4%e5%ba%a6%e8%af%84%e5%ae%a1%e4%bc%9a
>  
> 

Which haproxy version do you use?
haproxy -vv

What's in your haproxy log?
What's your haproxy config, shorten for the use case.

My assumption is that you try to use something like this

`option httpchk GET \r\nHost:\ sharepoint.domain.com\r\rn... `

This option sends the URL 1:1 as written in the config, no conversion will be 
done.

Maybe in the future there will be a funtion `url_enc` similar to the url_dec 
command but for now you will need to encode the URL as you have done.

I have created a feature request for url_enc function.

https://www.mail-archive.com/haproxy@formilux.org/msg35783.html

> JWD

Regards
Aleks

Re: Help, URL does not work with CHINESE charactor?

[Aleksandar Lazic]

Hi JWD.

On 24.12.19 02:53, JWD wrote:

Hi,all
I have a backend, which is sharepoint website.
If URL include CHINESE charactor, it return HTTP 400 ERROR from IE 11 with 
haproxy.
But it is ok without haproxy.
Can anyone help me?
Thanks.
This can not access, return HTTP 400 ERROR:
http://sharepoint.domain.com/CorWork/_layouts/15/TD.ECM.DoucmentDepartment 
/DepartmentFileDefault.aspx?destLink=/CorWork/ProjectShare/改进项目/知识管理一期 
项目/年度评审会

This is ok, if encode URL:
http://sharepoint.domain.com/CorWork/_layouts/15/TD.ECM.DoucmentDepartment/DepartmentFileDefault.aspx?destLink=/CorWork/ProjectShare/%e6%94%b9%e8%bf%9b%e9%a1%b9%e7%9b%ae/%e7%9f%a5%e8%af%86%e7%ae%a1%e7%90%86%e4%b8%80%e6%9c%9f%e9%a1%b9%e7%9b%ae/%e5%b9%b4%e5%ba%a6%e8%af%84%e5%ae%a1%e4%bc%9a 



Which haproxy version do you use?
haproxy -vv

What's in your haproxy log?
What's your haproxy config, shorten for the use case.

My assumption is that you try to use something like this

`option httpchk GET \r\nHost:\ sharepoint.domain.com\r\rn... `

This option sends the URL 1:1 as written in the config, no conversion will be 
done.

Maybe in the future there will be a funtion `url_enc` similar to the url_dec 
command but for now you will need to encode the URL as you have done.


I have created a feature request for url_enc function.

https://www.mail-archive.com/haproxy@formilux.org/msg35783.html


JWD


Regards
Aleks

RE: Help required ehhe

[Andy.ANTHOINE.ext]

Thanks a lot man, i'll take a look at that ! really appreciate it :) i'll come 
back to you if needed, but the help was great! :)

-Message d'origine-
De : Aleksandar Lazic [mailto:al-hapr...@none.at]
Envoyé : vendredi 23 août 2019 11:14
À : ANTHOINE Andy (EXT) 
Cc : haproxy@formilux.org
Objet : Re: Help required ehhe



Am 23-08-2019 02:04, schrieb andy.anthoine@opt.nc:
> REALLY SORRY ABOUT THAT lo
>
> I'll copy paste my bad :D
>
> admin@sld-loadb-01-prd-cit:~$ curl -v -o /dev/null --max-time 5
> --http1.0 http://10.154.2.29:8080/iws/ curl -v -o /dev/null --max-time
> 5 --http1.0 http://10.154.2.29:8080/eoc/login
> -bash: curl: command not found
>
> Ok, i'll separate the commands next time :)
>
> The lb i m working on is the production one, i'll check, but can't
> really do anything like installing on it.

Okay.

> Telnet works :) if it's enough for you ?

should work.

> admin@sld-loadb-01-prd-cit:~$ telnet 10.154.2.29 8080 Trying
> 10.154.2.29...
> Connected to 10.154.2.29.
> Escape character is '^]'.
>
>
>> So on the back ends are the check ports reachable and you get a 200
>> back.
>
> server sli-ecmapp-01-prd-cit 10.154.2.29:8443 server
> sli-ecmapp-02-prd-cit 10.154.2.31:8443
>
> admin@sld-loadb-01-prd-cit:~$ telnet 10.154.2.29 8443 Trying
> 10.154.2.29...
> telnet: Unable to connect to remote host: No route to host

That's the problem!

The test was wrong. you make a telnet to 8443 but the check port is 8080 Try 
this.

echo -e 'GET /iws/ HTTP/1.0\n\r\n\r'|telnet 10.154.2.29 8080

> admin@sld-loadb-01-prd-cit:~$ telnet 10.154.2.31 8443 Trying
> 10.154.2.31...
> Connected to 10.154.2.31.
> Escape character is '^]'.
> ^CConnection closed by foreign host.

That's another one.

echo -e 'GET /iws/ HTTP/1.0\n\r\n\r'|telnet 10.154.2.31 8080

> Seems like i got one of my answer

Yes looks like.

>> Is there any firewall in between the LB & BE?
> Yes there is one

Are both ports (8080 & 8443) open from LB top both BE?

Looks like not a haproxy issue.

Regards
Aleks


> -Message d'origine-
> De : Aleksandar Lazic [mailto:al-hapr...@none.at] Envoyé : vendredi 23
> août 2019 10:57 À : ANTHOINE Andy (EXT)  Cc
> : haproxy@formilux.org Objet : Re: Help required ehhe
>
> Hi.
>
> Am 23-08-2019 01:43, schrieb andy.anthoine@opt.nc:
>
>> Hi,
>>
>> I can't launch the command from the LB
>
> I love screenshots! It's so easy to copy paste from them 8-/
>
> do you have any other tool which you can use to check if the
> connection is possible from LB to Backend?
>
> nc?
> telnet?
> ...?
>
>> But from the server he is what i get
>>
>> [root@sli-ecmapp2-prd ~]# curl -v -o /dev/null --max-time 5 --http1.0
>> http://10.154.2.29:8080/iws/  curl -v -o /dev/null --max-time 5
>> --http1.0 http://10.154.2.29:8080/eoc/login
>
> Please one curl AFTER the other, the next time, just separate the
> commands with ;.
>
> curl -v -o /dev/null --max-time 5 --http1.0
> http://10.154.2.29:8080/iws/ ; curl -v -o /dev/null --max-time 5
> --http1.0 http://10.154.2.29:8080/eoc/login
>
>> * About to connect() to 10.154.2.29 port 8080 (#0)
>> *   Trying 10.154.2.29...
> ...
>> 0 00 00 0  0  0 --:--:-- --:--:-- --:--:--
>>0* Connected to 10.154.2.29 (10.154.2.29) port 8080 (#0)
>>
>>> GET /iws/ HTTP/1.0
>>> User-Agent: curl/7.29.0
>>> Host: 10.154.2.29:8080
>>> Accept: */*
>>>
>>
>> < HTTP/1.1 200 OK
>> < Server: Apache-Coyote/1.1
>
> [snipp]
>
>> curl: (28) Resolving timed out after 5515 milliseconds
>
> That looks strange, but maybe not the issue for now.
>
>> * About to connect() to 10.154.2.29 port 8080 (#2)
>> *   Trying 10.154.2.29...
>> * Connected to 10.154.2.29 (10.154.2.29) port 8080 (#2)
>>
>>> GET /eoc/login HTTP/1.0
>>> User-Agent: curl/7.29.0
>>> Host: 10.154.2.29:8080
>>> Accept: */*
>>>
>>
>> < HTTP/1.1 200 OK
>> < Server: Apache-Coyote/1.1
>
> [snipp]
>
>> * Closing connection 2
>
> So on the back ends are the check ports reachable and you get a 200
> back.
> Is there any firewall in between the LB & BE?
>
> What's your haproxy version?
>
> haproxy -vv
>
>
>> Best regards
>>
>> Andy
>>
>> -Message d'origine-
>> De : Aleksandar Lazic [mailto:al-hapr...@none.at] Envoyé : vendredi
>> 23 août 2019 10:33 À : ANTHOINE Andy (EXT) 
>> Cc
>> : haproxy@formilux.org Objet : Re: Help required ehhe
>>
>> Am 23-08-2019 00:49, schrieb andy.anthoine@opt.nc:
>>
>>> Hi,

Re: Help required ehhe

[Aleksandar Lazic]

Am 23-08-2019 02:04, schrieb andy.anthoine@opt.nc:

REALLY SORRY ABOUT THAT lo

I'll copy paste my bad :D

admin@sld-loadb-01-prd-cit:~$ curl -v -o /dev/null --max-time 5
--http1.0 http://10.154.2.29:8080/iws/ curl -v -o /dev/null --max-time
5 --http1.0 http://10.154.2.29:8080/eoc/login
-bash: curl: command not found

Ok, i'll separate the commands next time :)

The lb i m working on is the production one, i'll check, but can't
really do anything like installing on it.


Okay.


Telnet works :) if it's enough for you ?


should work.


admin@sld-loadb-01-prd-cit:~$ telnet 10.154.2.29 8080
Trying 10.154.2.29...
Connected to 10.154.2.29.
Escape character is '^]'.


So on the back ends are the check ports reachable and you get a 200 
back.


server sli-ecmapp-01-prd-cit 10.154.2.29:8443
server sli-ecmapp-02-prd-cit 10.154.2.31:8443

admin@sld-loadb-01-prd-cit:~$ telnet 10.154.2.29 8443
Trying 10.154.2.29...
telnet: Unable to connect to remote host: No route to host


That's the problem!

The test was wrong. you make a telnet to 8443 but the check port is 8080
Try this.

echo -e 'GET /iws/ HTTP/1.0\n\r\n\r'|telnet 10.154.2.29 8080


admin@sld-loadb-01-prd-cit:~$ telnet 10.154.2.31 8443
Trying 10.154.2.31...
Connected to 10.154.2.31.
Escape character is '^]'.
^CConnection closed by foreign host.


That's another one.

echo -e 'GET /iws/ HTTP/1.0\n\r\n\r'|telnet 10.154.2.31 8080


Seems like i got one of my answer


Yes looks like.


Is there any firewall in between the LB & BE?

Yes there is one


Are both ports (8080 & 8443) open from LB top both BE?

Looks like not a haproxy issue.

Regards
Aleks



-Message d'origine-
De : Aleksandar Lazic [mailto:al-hapr...@none.at]
Envoyé : vendredi 23 août 2019 10:57
À : ANTHOINE Andy (EXT) 
Cc : haproxy@formilux.org
Objet : Re: Help required ehhe

Hi.

Am 23-08-2019 01:43, schrieb andy.anthoine@opt.nc:


Hi,

I can't launch the command from the LB


I love screenshots! It's so easy to copy paste from them 8-/

do you have any other tool which you can use to check if the
connection is possible from LB to Backend?

nc?
telnet?
...?


But from the server he is what i get

[root@sli-ecmapp2-prd ~]# curl -v -o /dev/null --max-time 5 --http1.0
http://10.154.2.29:8080/iws/  curl -v -o /dev/null --max-time 5
--http1.0 http://10.154.2.29:8080/eoc/login


Please one curl AFTER the other, the next time, just separate the
commands with ;.

curl -v -o /dev/null --max-time 5 --http1.0
http://10.154.2.29:8080/iws/ ; curl -v -o /dev/null --max-time 5
--http1.0 http://10.154.2.29:8080/eoc/login


* About to connect() to 10.154.2.29 port 8080 (#0)
*   Trying 10.154.2.29...

...

0 00 00 0  0  0 --:--:-- --:--:-- --:--:--
   0* Connected to 10.154.2.29 (10.154.2.29) port 8080 (#0)


GET /iws/ HTTP/1.0
User-Agent: curl/7.29.0
Host: 10.154.2.29:8080
Accept: */*



< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1


[snipp]


curl: (28) Resolving timed out after 5515 milliseconds


That looks strange, but maybe not the issue for now.


* About to connect() to 10.154.2.29 port 8080 (#2)
*   Trying 10.154.2.29...
* Connected to 10.154.2.29 (10.154.2.29) port 8080 (#2)


GET /eoc/login HTTP/1.0
User-Agent: curl/7.29.0
Host: 10.154.2.29:8080
Accept: */*



< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1


[snipp]


* Closing connection 2


So on the back ends are the check ports reachable and you get a 200 
back.

Is there any firewall in between the LB & BE?

What's your haproxy version?

haproxy -vv



Best regards

Andy

-Message d'origine-
De : Aleksandar Lazic [mailto:al-hapr...@none.at] Envoyé : vendredi 23
août 2019 10:33 À : ANTHOINE Andy (EXT)  Cc
: haproxy@formilux.org Objet : Re: Help required ehhe

Am 23-08-2019 00:49, schrieb andy.anthoine@opt.nc:


Hi,







Ehhe not an external ip don't worry, or i would have deleted it hehe







No change, the problem seems to be there since before i m here, and



they now need it to be fixed







I don't see anything in particular in the logs, beside that kind of



thing which is normal since the server is rebooted at this time ;)







Aug 22 05:00:23 sld-loadb-01-prd-cit local1.alert haproxy[2244]:



Server ecmapp-prd-be-8443/sli-ecmapp-01-prd-cit is DOWN, reason:



Layer4 connection problem, info: "Connection refused at step 1 of



tcp-check (connect port 8080)", check duration: 0ms. 1 active and 0



backup servers left. 62 sessions active, 0 requeued, 0 remaining in



queue.


this looks to me that the loadbalancer can't connect to the backend
check port, is something listen on the backend server on port 8080?

Please can you try the following command from the loadbalancer.

curl -v -o /dev/null --max-time 5 --http1.0
http://10.154.2.29:8080/iws/ curl -v -o /dev/null --max-time 5
--http1.0 http://10.154.2.29:8080/eoc/login

The same for 10.154.2.31


What do you mean by that ?


I have shorten the config to reduce the size of 

RE: Help required ehhe

[Andy.ANTHOINE.ext]

REALLY SORRY ABOUT THAT lo

I'll copy paste my bad :D

admin@sld-loadb-01-prd-cit:~$ curl -v -o /dev/null --max-time 5 --http1.0 
http://10.154.2.29:8080/iws/ curl -v -o /dev/null --max-time 5 --http1.0 
http://10.154.2.29:8080/eoc/login
-bash: curl: command not found

Ok, i'll separate the commands next time :)

The lb i m working on is the production one, i'll check, but can't really do 
anything like installing on it.

Telnet works :) if it's enough for you ?

admin@sld-loadb-01-prd-cit:~$ telnet 10.154.2.29 8080
Trying 10.154.2.29...
Connected to 10.154.2.29.
Escape character is '^]'.


So on the back ends are the check ports reachable and you get a 200 back.

server sli-ecmapp-01-prd-cit 10.154.2.29:8443
server sli-ecmapp-02-prd-cit 10.154.2.31:8443

admin@sld-loadb-01-prd-cit:~$ telnet 10.154.2.29 8443
Trying 10.154.2.29...
telnet: Unable to connect to remote host: No route to host
admin@sld-loadb-01-prd-cit:~$ telnet 10.154.2.31 8443
Trying 10.154.2.31...
Connected to 10.154.2.31.
Escape character is '^]'.
^CConnection closed by foreign host.

Seems like i got one of my answer


Is there any firewall in between the LB & BE? Yes there is one




-Message d'origine-
De : Aleksandar Lazic [mailto:al-hapr...@none.at]
Envoyé : vendredi 23 août 2019 10:57
À : ANTHOINE Andy (EXT) 
Cc : haproxy@formilux.org
Objet : Re: Help required ehhe

Hi.

Am 23-08-2019 01:43, schrieb andy.anthoine@opt.nc:

> Hi,
>
> I can't launch the command from the LB

I love screenshots! It's so easy to copy paste from them 8-/

do you have any other tool which you can use to check if the connection is 
possible from LB to Backend?

nc?
telnet?
...?

> But from the server he is what i get
>
> [root@sli-ecmapp2-prd ~]# curl -v -o /dev/null --max-time 5 --http1.0
> http://10.154.2.29:8080/iws/  curl -v -o /dev/null --max-time 5
> --http1.0 http://10.154.2.29:8080/eoc/login

Please one curl AFTER the other, the next time, just separate the commands with 
;.

curl -v -o /dev/null --max-time 5 --http1.0 http://10.154.2.29:8080/iws/ ; curl 
-v -o /dev/null --max-time 5 --http1.0 http://10.154.2.29:8080/eoc/login

> * About to connect() to 10.154.2.29 port 8080 (#0)
> *   Trying 10.154.2.29...
...
> 0 00 00 0  0  0 --:--:-- --:--:-- --:--:--
>0* Connected to 10.154.2.29 (10.154.2.29) port 8080 (#0)
>
>> GET /iws/ HTTP/1.0
>> User-Agent: curl/7.29.0
>> Host: 10.154.2.29:8080
>> Accept: */*
>>
>
> < HTTP/1.1 200 OK
> < Server: Apache-Coyote/1.1

[snipp]

> curl: (28) Resolving timed out after 5515 milliseconds

That looks strange, but maybe not the issue for now.

> * About to connect() to 10.154.2.29 port 8080 (#2)
> *   Trying 10.154.2.29...
> * Connected to 10.154.2.29 (10.154.2.29) port 8080 (#2)
>
>> GET /eoc/login HTTP/1.0
>> User-Agent: curl/7.29.0
>> Host: 10.154.2.29:8080
>> Accept: */*
>>
>
> < HTTP/1.1 200 OK
> < Server: Apache-Coyote/1.1

[snipp]

> * Closing connection 2

So on the back ends are the check ports reachable and you get a 200 back.
Is there any firewall in between the LB & BE?

What's your haproxy version?

haproxy -vv


> Best regards
>
> Andy
>
> -Message d'origine-
> De : Aleksandar Lazic [mailto:al-hapr...@none.at] Envoyé : vendredi 23
> août 2019 10:33 À : ANTHOINE Andy (EXT)  Cc
> : haproxy@formilux.org Objet : Re: Help required ehhe
>
> Am 23-08-2019 00:49, schrieb andy.anthoine@opt.nc:
>
>> Hi,
>
>>
>
>> Ehhe not an external ip don't worry, or i would have deleted it hehe
>
>>
>
>> No change, the problem seems to be there since before i m here, and
>
>> they now need it to be fixed
>
>>
>
>> I don't see anything in particular in the logs, beside that kind of
>
>> thing which is normal since the server is rebooted at this time ;)
>
>>
>
>> Aug 22 05:00:23 sld-loadb-01-prd-cit local1.alert haproxy[2244]:
>
>> Server ecmapp-prd-be-8443/sli-ecmapp-01-prd-cit is DOWN, reason:
>
>> Layer4 connection problem, info: "Connection refused at step 1 of
>
>> tcp-check (connect port 8080)", check duration: 0ms. 1 active and 0
>
>> backup servers left. 62 sessions active, 0 requeued, 0 remaining in
>
>> queue.
>
> this looks to me that the loadbalancer can't connect to the backend
> check port, is something listen on the backend server on port 8080?
>
> Please can you try the following command from the loadbalancer.
>
> curl -v -o /dev/null --max-time 5 --http1.0
> http://10.154.2.29:8080/iws/ curl -v -o /dev/null --max-time 5
> --http1.0 http://10.154.2.29:8080/eoc/login
>
> The same for 10.154.2.31
>
>> What do you mean by that ?
>
> I have shorten the config to redu

Re: Help required ehhe

[Aleksandar Lazic]

Hi.

Am 23-08-2019 01:43, schrieb andy.anthoine@opt.nc:


Hi,

I can't launch the command from the LB


I love screenshots! It's so easy to copy paste from them 8-/

do you have any other tool which you can use to check if the
connection is possible from LB to Backend?

nc?
telnet?
...?


But from the server he is what i get

[root@sli-ecmapp2-prd ~]# curl -v -o /dev/null --max-time 5 --http1.0
http://10.154.2.29:8080/iws/  curl -v -o /dev/null --max-time 5
--http1.0 http://10.154.2.29:8080/eoc/login


Please one curl AFTER the other, the next time, just separate the
commands with ;.

curl -v -o /dev/null --max-time 5 --http1.0 http://10.154.2.29:8080/iws/
; curl -v -o /dev/null --max-time 5 --http1.0
http://10.154.2.29:8080/eoc/login


* About to connect() to 10.154.2.29 port 8080 (#0)
*   Trying 10.154.2.29...

...

0 00 00 0  0  0 --:--:-- --:--:-- --:--:--
   0* Connected to 10.154.2.29 (10.154.2.29) port 8080 (#0)


GET /iws/ HTTP/1.0
User-Agent: curl/7.29.0
Host: 10.154.2.29:8080
Accept: */*



< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1


[snipp]


curl: (28) Resolving timed out after 5515 milliseconds


That looks strange, but maybe not the issue for now.


* About to connect() to 10.154.2.29 port 8080 (#2)
*   Trying 10.154.2.29...
* Connected to 10.154.2.29 (10.154.2.29) port 8080 (#2)


GET /eoc/login HTTP/1.0
User-Agent: curl/7.29.0
Host: 10.154.2.29:8080
Accept: */*



< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1


[snipp]


* Closing connection 2


So on the back ends are the check ports reachable and you get a 200
back.
Is there any firewall in between the LB & BE?

What's your haproxy version?

haproxy -vv



Best regards

Andy

-Message d'origine-
De : Aleksandar Lazic [mailto:al-hapr...@none.at]
Envoyé : vendredi 23 août 2019 10:33
À : ANTHOINE Andy (EXT) 
Cc : haproxy@formilux.org
Objet : Re: Help required ehhe

Am 23-08-2019 00:49, schrieb andy.anthoine@opt.nc:


Hi,







Ehhe not an external ip don't worry, or i would have deleted it hehe







No change, the problem seems to be there since before i m here, and



they now need it to be fixed







I don't see anything in particular in the logs, beside that kind of



thing which is normal since the server is rebooted at this time ;)







Aug 22 05:00:23 sld-loadb-01-prd-cit local1.alert haproxy[2244]:



Server ecmapp-prd-be-8443/sli-ecmapp-01-prd-cit is DOWN, reason:



Layer4 connection problem, info: "Connection refused at step 1 of



tcp-check (connect port 8080)", check duration: 0ms. 1 active and 0



backup servers left. 62 sessions active, 0 requeued, 0 remaining in



queue.


this looks to me that the loadbalancer can't connect to the backend
check port, is something listen on the backend server on port 8080?

Please can you try the following command from the loadbalancer.

curl -v -o /dev/null --max-time 5 --http1.0
http://10.154.2.29:8080/iws/ curl -v -o /dev/null --max-time 5
--http1.0 http://10.154.2.29:8080/eoc/login

The same for 10.154.2.31


What do you mean by that ?


I have shorten the config to reduce the size of the mail.


mode http







[snip more config]







-Message d'origine-



De : Aleksandar Lazic [mailto:al-hapr...@none.at] Envoyé : vendredi 23



août 2019 09:41 À : ANTHOINE Andy (EXT)  Cc



: haproxy@formilux.org Objet : Re: Help required ehhe







Hi.







Am 23-08-2019 00:28, schrieb andy.anthoine@opt.nc:







Hi,







I got that email on this site, not sure if it's still working etc



https://www.slideshare.net/haproxytech/haproxy-best-practice


[snipp]


What's your haproxy version?







haproxy -vv







Application load balancing & high availability v8.5.7 (8546)


That's not the full output of the command line call haproxy -vv .


Thanks a lot for the answer man !















Thanks !







Andy


Regards

Aleks

Ce
message et toutes les pièces jointes (ci-après le « message ») sont à
l'attention exclusive des destinataires désignés. Il peut contenir des
informations confidentielles. Si vous le recevez par erreur, merci d'en
informer immédiatement l'émetteur et de le détruire. Toute utilisation,
diffusion ou toute publication, totale ou partielle, est interdite,
sauf autorisation. Tout message électronique étant susceptible
d'altération, l'OPT NC décline toute responsabilité au titre de ce
message dans l'hypothèse où il aurait été modifié.
This message and any attachments (the «
message ») are intended solely for the addresses. It may contain
privileged information. If you receive this message in error, please
immediately notify the sender and delete it. Any use, dissemination or
disclosure, either whole or partial, is prohibited unless formal
approval. Emails are susceptible to alteration; OPT NC shall not
therefore be liable for the message if modified.

Pensez à l'environnement, n'imprimez que si nécessaire.

RE: Help required ehhe

[Andy.ANTHOINE.ext]

Hi,



I can't launch the command from the LB



[cid:image001.png@01D5599F.9D2D6E10]

But from the server he is what i get



[root@sli-ecmapp2-prd ~]# curl -v -o /dev/null --max-time 5 --http1.0 
http://10.154.2.29:8080/iws/ curl -v -o /dev/null --max-time 5 --http1.0 
http://10.154.2.29:8080/eoc/login

* About to connect() to 10.154.2.29 port 8080 (#0)

*   Trying 10.154.2.29...

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current

 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0* 
Connected to 10.154.2.29 (10.154.2.29) port 8080 (#0)

> GET /iws/ HTTP/1.0

> User-Agent: curl/7.29.0

> Host: 10.154.2.29:8080

> Accept: */*

>

< HTTP/1.1 200 OK

< Server: Apache-Coyote/1.1

< Accept-Ranges: bytes

< ETag: W/"1283-1552374458000"

< Last-Modified: Tue, 12 Mar 2019 07:07:38 GMT

< Content-Type: text/html

< Content-Length: 1283

< Date: Thu, 22 Aug 2019 23:42:28 GMT

< Connection: close

<

{ [data not shown]

100  1283  100  12830 0   632k  0 --:--:-- --:--:-- --:--:-- 1252k

* Closing connection 0

  0 00 00 0  0  0 --:--:--  0:00:04 --:--:-- 0* 
Resolving timed out after 5515 milliseconds

  0 00 00 0  0  0 --:--:--  0:00:06 --:--:-- 0

* Closing connection 1

curl: (28) Resolving timed out after 5515 milliseconds

* About to connect() to 10.154.2.29 port 8080 (#2)

*   Trying 10.154.2.29...

* Connected to 10.154.2.29 (10.154.2.29) port 8080 (#2)

> GET /eoc/login HTTP/1.0

> User-Agent: curl/7.29.0

> Host: 10.154.2.29:8080

> Accept: */*

>

< HTTP/1.1 200 OK

< Server: Apache-Coyote/1.1

< Date: Thu, 22 Aug 2019 23:42:34 GMT

< Cache-Control: no-cache

< Cache-Control: no-store

< Cache-Control: max-age=0

< X-Content-Type-Options: nosniff

< X-XSS-Protection: 1

< Set-Cookie: JSESSIONID=hr-UQa5y1CpyiQvYjONLkNyf; Path=/eoc

< Content-Type: text/html;charset=UTF-8

< Content-Length: 4398

< Connection: close

<







Login



var cwWebappPrefix='/eoc';var cwComma=",";var 
cwDecimal=".";var cwSessionTimeout=3000;var cwSessionTimeoutWarning=0;var 
cwHtEsc=false;var cwEnumHtEsc=true;var cwSkinnedWarnDialog = false;var 
formCount=0;var startLoading=false;function finishLoading(id){if(window[id] != 
null) {startLoading = false; return true;} else return false;}function 
loading(){return startLoading;}function 
increaseFormCount(){formCount++;}function decreaseFormCount(){formCount--;}var 
cwDefaultDialogMaskOpacity = 0;;var cwUE0311='Invalid date value. Valid format 
is ';

var cwUE0328='Start date must occur BEFORE End Date. Please correct 
the\n\t\t\tcriteria.';

var cwUE0116='This document contains errors and will not be saved.';

var cwUU0313='Warning';

var cwUU0315=' could not be matched to the display format: ';

var cwUU0316=' row(s) fetched (more exist but are not shown).';

var cwUU0317=' row(s) fetched.';

var cwUU0059='Please wait, operation in progress...';

var cwUU0318='Cancel';

var cwUU0319='Save';

var cwUU0322='No items found.';

var cwUU0324=' row(s) fetched out of ';

var cwUU0325=' row(s) found in total ';

var cwUU0326=' page(s).';





var isomorphicDir="/eoc/isomorphic/";var 
cwDir="../";









if(window.Browser.isIE){window.isc.Canvas.addClassProperties({$42a: 
"margin:0px;border:0px;padding:0px;background-color:transparent;background-image:none;",neverUsePNGWorkaround
 : true});window.isc.Canvas.addProperties({$r9:false});}

















if(window.Browser.isIE){isc.defineClass("MyResultSet", 
"ResultSet").addProperties({fetchRemoteDataReply : function (dsResponse, data, 
request) {if (dsResponse.httpResponseCode == 12152) 
{isc.RPCManager.handleError(dsResponse, request);} else {return 
this.Super("fetchRemoteDataReply", 
arguments);}}});isc.DataSource.addProperties({resultSetClass : 
"MyResultSet"});window.isc.RPCManager.addClassProperties({handleError : 
function (response, request) {Log.logWarn("httpResponseCode:  " + 
response.httpResponseCode);if (response.httpResponseCode == 12152) 
{window.isc.RPCManager.suspendTransaction(response.transactionNum);window.isc.Timer.setTimeout(function
 () 
{window.isc.RPCManager.resendTransaction(response.transactionNum);});}}});}







var $cwCurrentCss = '/css/ui_common.css'



isc.setAutoDraw(false);





* Closing connection 2



Best regards



Andy



-Message d'origine-
De : Aleksandar Lazic [mailto:al-hapr...@none.at]
Envoyé : vendredi 23 août 2019 10:33
À : ANTHOINE Andy (EXT) 
Cc : haproxy@formilux.org
Objet : Re: Help required ehhe







Am 23-08-2019 00:49, schrieb 
andy.anthoine@opt.nc<mailto:andy.anthoine@opt.nc>:

> Hi,

>

> Ehhe not an external ip don't worry, or

Re: Help required ehhe

[Aleksandar Lazic]

Am 23-08-2019 00:49, schrieb andy.anthoine@opt.nc:

Hi,

Ehhe not an external ip don't worry, or i would have deleted it hehe

No change, the problem seems to be there since before i m here, and
they now need it to be fixed

I don't see anything in particular in the logs, beside that kind of
thing which is normal since the server is rebooted at this time ;)

Aug 22 05:00:23 sld-loadb-01-prd-cit local1.alert haproxy[2244]:
Server ecmapp-prd-be-8443/sli-ecmapp-01-prd-cit is DOWN, reason:
Layer4 connection problem, info: "Connection refused at step 1 of
tcp-check (connect port 8080)", check duration: 0ms. 1 active and 0
backup servers left. 62 sessions active, 0 requeued, 0 remaining in
queue.


this looks to me that the loadbalancer can't connect to the backend
check port, is something listen on the backend server on port 8080?

Please can you try the following command from the loadbalancer.

curl -v -o /dev/null --max-time 5 --http1.0 http://10.154.2.29:8080/iws/
curl -v -o /dev/null --max-time 5 --http1.0 
http://10.154.2.29:8080/eoc/login


The same for 10.154.2.31


What do you mean by that ?


I have shorten the config to reduce the size of the mail.


mode http


[snip more config]

-Message d'origine-
De : Aleksandar Lazic [mailto:al-hapr...@none.at]
Envoyé : vendredi 23 août 2019 09:41
À : ANTHOINE Andy (EXT) 
Cc : haproxy@formilux.org
Objet : Re: Help required ehhe

Hi.

Am 23-08-2019 00:28, schrieb andy.anthoine@opt.nc:


Hi,

I got that email on this site, not sure if it's still working etc
https://www.slideshare.net/haproxytech/haproxy-best-practice


[snipp]


What's your haproxy version?

haproxy -vv

  Application load balancing & high availability v8.5.7 (8546)


That's not the full output of the command line call haproxy -vv .



Thanks a lot for the answer man !




Thanks !

Andy


Regards
Aleks

RE: Help required ehhe

[Andy.ANTHOINE.ext]

Hi,

Ehhe not an external ip don't worry, or i would have deleted it hehe

No change, the problem seems to be there since before i m here, and they now 
need it to be fixed

I don't see anything in particular in the logs, beside that kind of thing which 
is normal since the server is rebooted at this time ;)

Aug 22 05:00:23 sld-loadb-01-prd-cit local1.alert haproxy[2244]: Server 
ecmapp-prd-be-8443/sli-ecmapp-01-prd-cit is DOWN, reason: Layer4 connection 
problem, info: "Connection refused at step 1 of tcp-check (connect port 8080)", 
check duration: 0ms. 1 active and 0 backup servers left. 62 sessions active, 0 
requeued, 0 remaining in queue.

What do you mean by that ?

> mode http

[snip more config]




-Message d'origine-
De : Aleksandar Lazic [mailto:al-hapr...@none.at]
Envoyé : vendredi 23 août 2019 09:41
À : ANTHOINE Andy (EXT) 
Cc : haproxy@formilux.org
Objet : Re: Help required ehhe

Hi.

Am 23-08-2019 00:28, schrieb andy.anthoine@opt.nc:

> Hi,
>
> I got that email on this site, not sure if it's still working etc
> https://www.slideshare.net/haproxytech/haproxy-best-practice
>
> I'm having some issue with a service (white page sometimes for a
> webservice), and i m not totally sure it has anything to do with issue
> on my load balancer (i'm not the one who configured it, just having to
> work on it, guy gone, no documentation left etc…)
>
> When i started looking, i noticed that only this service had so many
> warnings and errors, so i thought « maybe » it can have something to
> do with it…

Was any changes in the setup which cloud be the reason for the the behaviour or 
you just recognizes it and it's "normal"?
What's in the logs?

> backend ecmapp-prd-be-8443
> acl kerberos hdr_beg(Authorization) -m beg Negotiate http-request
> replace-header Authorization (.*) "Basic " if kerberos

Ähm I hope this is not an U:P which is external visible

> mode http

[snip more config]

> Here is the configuration of the service, i m able to read half of it,
> chechking the documentation to find more information
>
> Thanks a lot of you can help, if not have a great day J

What's your haproxy version?

haproxy -vv

  Application load balancing & high availability v8.5.7 (8546)


Thanks a lot for the answer man !



> Thanks !
>
> Andy
>
> Ce
> message et toutes les pièces jointes (ci-après le « message ») sont à
> l'attention exclusive des destinataires désignés. Il peut contenir des
> informations confidentielles. Si vous le recevez par erreur, merci
> d'en informer immédiatement l'émetteur et de le détruire. Toute
> utilisation, diffusion ou toute publication, totale ou partielle, est
> interdite, sauf autorisation. Tout message électronique étant
> susceptible d'altération, l'OPT NC décline toute responsabilité au
> titre de ce message dans l'hypothèse où il aurait été modifié.
> This message and any attachments (the «
> message ») are intended solely for the addresses. It may contain
> privileged information. If you receive this message in error, please
> immediately notify the sender and delete it. Any use, dissemination or
> disclosure, either whole or partial, is prohibited unless formal
> approval. Emails are susceptible to alteration; OPT NC shall not
> therefore be liable for the message if modified.
>
> Pensez à l'environnement, n'imprimez que si nécessaire.

Always the same crap footer from the "enterprise" Companies to public mailing 
lists.

Regards
Aleks



Ce message et toutes les pièces jointes (ci-après le « message ») sont à 
l'attention exclusive des destinataires désignés. Il peut contenir des 
informations confidentielles. Si vous le recevez par erreur, merci d'en 
informer immédiatement l'émetteur et de le détruire. Toute utilisation, 
diffusion ou toute publication, totale ou partielle, est interdite, sauf 
autorisation. Tout message électronique étant susceptible d'altération, l'OPT 
NC décline toute responsabilité au titre de ce message dans l'hypothèse où il 
aurait été modifié.

This message and any attachments (the « message ») are intended solely for the 
addresses. It may contain privileged information. If you receive this message 
in error, please immediately notify the sender and delete it. Any use, 
dissemination or disclosure, either whole or partial, is prohibited unless 
formal approval. Emails are susceptible to alteration; OPT NC shall not 
therefore be liable for the message if modified.

Pensez à l'environnement, n'imprimez que si nécessaire.

Re: Help required ehhe

[Aleksandar Lazic]

Hi.

Am 23-08-2019 00:28, schrieb andy.anthoine@opt.nc:


Hi,

I got that email on this site, not sure if it's still working etc
https://www.slideshare.net/haproxytech/haproxy-best-practice

I'm having some issue with a service (white page sometimes for a
webservice), and i m not totally sure it has anything to do with issue
on my load balancer (i'm not the one who configured it, just having to
work on it, guy gone, no documentation left etc…)

When i started looking, i noticed that only this service had so many
warnings and errors, so i thought « maybe » it can have something to do
with it…


Was any changes in the setup which cloud be the reason for the the
behaviour or you just recognizes it and it's "normal"?
What's in the logs?


backend ecmapp-prd-be-8443
acl kerberos hdr_beg(Authorization) -m beg Negotiate
http-request replace-header Authorization (.*) "Basic " if
kerberos


Ähm I hope this is not an U:P which is external visible


mode http


[snip more config]


Here is the configuration of the service, i m able to read half of it,
chechking the documentation to find more information

Thanks a lot of you can help, if not have a great day J


What's your haproxy version?

haproxy -vv


Thanks !

Andy

Ce
message et toutes les pièces jointes (ci-après le « message ») sont à
l'attention exclusive des destinataires désignés. Il peut contenir des
informations confidentielles. Si vous le recevez par erreur, merci d'en
informer immédiatement l'émetteur et de le détruire. Toute utilisation,
diffusion ou toute publication, totale ou partielle, est interdite,
sauf autorisation. Tout message électronique étant susceptible
d'altération, l'OPT NC décline toute responsabilité au titre de ce
message dans l'hypothèse où il aurait été modifié.
This message and any attachments (the «
message ») are intended solely for the addresses. It may contain
privileged information. If you receive this message in error, please
immediately notify the sender and delete it. Any use, dissemination or
disclosure, either whole or partial, is prohibited unless formal
approval. Emails are susceptible to alteration; OPT NC shall not
therefore be liable for the message if modified.

Pensez à l'environnement, n'imprimez que si nécessaire.


Always the same crap footer from the "enterprise" Companies to public
mailing lists.

Regards
Aleks

Re: Help with 1.8.1/4 and spoa_server/spoa_example

[Aleksandar Lazic]

Hi Christopher.

Am 05.07.2019 um 16:29 schrieb Christopher Faulet:
> Le 03/07/2019 à 16:16, Aleksandar Lazic a écrit :
>> I know this is a old haproxy version but I have not the option to update as 
>> it's
>> part of a vendor product.
>>
>> I need to bring the `[haproxy-2.0.git]/ contrib/spoa_server/` up and runnig 
>> or
>> `[haproxy-1.8.git]/contrib/spoa_example/` from 1.8.4 to be able to run a 
>> python
>> script.
>>
>> Any help is welcome and I can offer some money for the help. It's urgent so I
>> will need the help asap.
>>
>> You can contact me also off the list.
> 
> 
> Hi Aleks,
> 
> The spoa_example cannot run python script. But I take a look to the 
> spoa_server
> and it seems to usable with HAProxy 1.8 with a small patch. You must downgrade
> the SPOE version and change the encoding of the frame's flags. The example
> configuration must also be adapted because there is no debug converter in
> HAProxy 1.8.
> 
> But, as you said, HAProxy 1.8.4 is old, and many fixes was pushed on the SPOE
> since then. So you would experience some bugs. Be careful.
> 
> And I noticed a bug with the spoe_server, It seems to accept and process only
> one connection per worker because of a while loop to read frames. I'm cc'ing
> Thierry.
> 
> I attached a quick-and-dirty patch to downgrade SPOP version of the 
> spoa_server
> to 1.0.

Thank you very much.

Best regards
Aleks

Re: Help with 1.8.1/4 and spoa_server/spoa_example

[Christopher Faulet]

Le 03/07/2019 à 16:16, Aleksandar Lazic a écrit :

I know this is a old haproxy version but I have not the option to update as it's
part of a vendor product.

I need to bring the `[haproxy-2.0.git]/ contrib/spoa_server/` up and runnig or
`[haproxy-1.8.git]/contrib/spoa_example/` from 1.8.4 to be able to run a python
script.

Any help is welcome and I can offer some money for the help. It's urgent so I
will need the help asap.

You can contact me also off the list.



Hi Aleks,

The spoa_example cannot run python script. But I take a look to the spoa_server 
and it seems to usable with HAProxy 1.8 with a small patch. You must downgrade 
the SPOE version and change the encoding of the frame's flags. The example 
configuration must also be adapted because there is no debug converter in 
HAProxy 1.8.


But, as you said, HAProxy 1.8.4 is old, and many fixes was pushed on the SPOE 
since then. So you would experience some bugs. Be careful.


And I noticed a bug with the spoe_server, It seems to accept and process only 
one connection per worker because of a while loop to read frames. I'm cc'ing 
Thierry.


I attached a quick-and-dirty patch to downgrade SPOP version of the spoa_server 
to 1.0.



--
Christopher Faulet
>From 955c47ce6e8bf8a1ed644ffd353b52b54516c2fa Mon Sep 17 00:00:00 2001
From: Christopher Faulet 
Date: Fri, 5 Jul 2019 16:25:34 +0200
Subject: [PATCH] WIP: spoa_server: Downgrade SPOP version to 1.0

---
 contrib/spoa_server/spoa.c | 6 +++---
 contrib/spoa_server/spoa.h | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/contrib/spoa_server/spoa.c b/contrib/spoa_server/spoa.c
index f36c3db90..933eb79c8 100644
--- a/contrib/spoa_server/spoa.c
+++ b/contrib/spoa_server/spoa.c
@@ -687,7 +687,7 @@ static void prepare_agentack(struct worker *w)
 	w->ack[w->ack_len++] = SPOE_FRM_T_AGENT_ACK;
 
 	/* Set flags */
-	flags |= htonl(SPOE_FRM_FL_FIN);
+	flags |= SPOE_FRM_FL_FIN;
 	memcpy(w->ack + w->ack_len, , 4);
 	w->ack_len += 4;
 
@@ -949,7 +949,7 @@ prepare_agenthello(struct worker *w)
 	w->buf[idx++] = SPOE_FRM_T_AGENT_HELLO;
 
 	/* Set flags */
-	flags |= htonl(SPOE_FRM_FL_FIN);
+	flags |= SPOE_FRM_FL_FIN;
 	memcpy(w->buf+idx, , 4);
 	idx += 4;
 
@@ -994,7 +994,7 @@ prepare_agentdicon(struct worker *w)
 	w->buf[idx++] = SPOE_FRM_T_AGENT_DISCON;
 
 	/* Set flags */
-	flags |= htonl(SPOE_FRM_FL_FIN);
+	flags |= SPOE_FRM_FL_FIN;
 	memcpy(w->buf+idx, , 4);
 	idx += 4;
 
diff --git a/contrib/spoa_server/spoa.h b/contrib/spoa_server/spoa.h
index 8f912e435..0166c48c4 100644
--- a/contrib/spoa_server/spoa.h
+++ b/contrib/spoa_server/spoa.h
@@ -18,7 +18,7 @@
 #include 
 
 #define MAX_FRAME_SIZE16384
-#define SPOP_VERSION  "2.0"
+#define SPOP_VERSION  "1.0"
 #define SPOA_CAPABILITIES ""
 
 /* Flags set on the SPOE frame */
-- 
2.20.1

Re: Help you generate more revenue for your haproxy.com.

[Olivier Houchard]

On Wed, Sep 05, 2018 at 10:14:55PM +1000, Rob Thomas wrote:
> You gotta wonder how this guy got this mailing list.  He must have actually
> LOOKED at the website, right?
> 
> Sigh. Spammers.
> 
> For anyone who cares, I don't think it's possible for haproxy to get MORE
> exposure on google.
> 
> [image: image.png]
> 

Notice how he mentionned haproxy.com, which is only 2nd in your google search.

I think we can trust somebody named FREDDIE KIRK, after all he is SEO
Strategist AND Business Development Manager.

Regards,

Olivier

> On Wed, 5 Sep 2018 at 22:07, FREDDIE KIRK 
> wrote:
> 
> > Hi *haproxy.com ,*
> >
> > *Do you need to know how your website currently ranks on search engine
> > result pages and how you can start beating your competitors right now?*
> >
> > Today, I went through your website *haproxy.com *;
> > you seem to have a great website, but only the thing is People are already
> > searching for your products and services, but if you don't use the Right
> > Keywords they're searching for on your site, it will be difficult for them
> > to find you.
> >
> > We will deliver you a huge ROI, high ranking, more traffic, clicks, page
> > views and most importantly converting those visitors into paying customers.
> >
> > Let me know if I should share a *Plan of Action* for your website
> >
> > Kind Regards
> >
> > SEO Strategist
> > Business Development Manager
> >

Re: Help with backend server sni setup

[Aleksandar Lazic]

Hi.

On 30/07/2018 16:39, Lukas Tribus wrote:

On Mon, 30 Jul 2018 at 13:30, Aleksandar Lazic  wrote:


Hi.

I have the following Setup.

APP -> Internal Haproxy -(HTTPS)-> external HAProxy -> APP

The external HAProxy is configured with multiple TLS Vhost.


Never use SNI for Vhosting. It should work with the host header only.
SNI should only be used for certificate selection, otherwise
overlapping certificates will cause wrong forwarding decisions.


The openshift router, based on haproxy 1.8, looks for the sni hostname
for routing.

https://github.com/openshift/origin/blob/master/images/router/haproxy/conf/haproxy-config.template#L198-L209

Due to this fact we *must* set the ssl hostname


I assume that when I add `server  sni appinternal.domain.com` to
the server line will be set the hostname field in the TLS session to
this value.


No, the sni keyword expects a fetch expression.

Set it to the host header for example:
sni req.hdr(host)

Or to a static string:
sni str(www.example.com)


When I take a look into the code I see this line.

http://git.haproxy.org/?p=haproxy-1.8.git;a=blob;f=src/backend.c;hb=ada31afbc1e9095d494973cad91a4e507c4c1d9b#l1255
ssl_sock_set_servername(srv_conn, smp->data.u.str.str);

and the implementation of this function is here
http://git.haproxy.org/?p=haproxy-1.8.git;a=blob;f=src/ssl_sock.c;hb=ada31afbc1e9095d494973cad91a4e507c4c1d9b#l5922

The blocks begins here.
http://git.haproxy.org/?p=haproxy-1.8.git;a=blob;f=src/backend.c;hb=ada31afbc1e9095d494973cad91a4e507c4c1d9b#l1236

As far as I understood this block and I'm not sure that I have it
understood right the fetch sample checks for the string, as you have
written, AND SET the hostname into the SSL/TLS header for SNI.

Now after I looked into the code and read the doc again it's clear now
for me.

This options set's

 cite from doc
the host name sent in the SNI TLS extension to the server.


Please apologise for the rush and my stupidity.


cheers,
lukas


Best greetings
aleks

Re: Help with backend server sni setup

[Lukas Tribus]

On Mon, 30 Jul 2018 at 13:30, Aleksandar Lazic  wrote:
>
> Hi.
>
> I have the following Setup.
>
> APP -> Internal Haproxy -(HTTPS)-> external HAProxy -> APP
>
> The external HAProxy is configured with multiple TLS Vhost.

Never use SNI for Vhosting. It should work with the host header only.
SNI should only be used for certificate selection, otherwise
overlapping certificates will cause wrong forwarding decisions.



> I assume that when I add `server  sni appinternal.domain.com` to the
> server line will be set the hostname field in the TLS session to this
> value.

No, the sni keyword expects a fetch expression.

Set it to the host header for example:
sni req.hdr(host)

Or to a static string:
sni str(www.example.com)


cheers,
lukas

Re: Help with environment variables in config

[Willy Tarreau]

On Sat, Jul 21, 2018 at 05:44:44PM +0100, Jonathan Matthews wrote:
> No. Sudo doesn't pass envvars through to its children by default:
> https://stackoverflow.com/questions/8633461/how-to-keep-environment-variables-when-using-sudo
> 
> Read that page *and* the comments - in particular be aware that you have to
> request (at the CLI) that sudo preserve envvars, and you also have to have
> been granted permission to do this, via the sudoers config file.
> 
> If this is all sounding a bit complicated, that's because it is.
> 
> You've chosen a relatively uncommon way of running haproxy - directly, via
> sudo. Consider running via an init script or systemd unit (?) or, failing
> that, just a script which is itself the sudo target, which sets the envvars
> in the privileged environment.

Also, something that people don't necessarily know is that haproxy can
set its own variables using "setenv" in the global sections. Some will
say this is stupid since you're going to use them in the same config,
but when you add to this the fact that we can load multiple files, it
becomes more obvious that it easily allows to parse an environment file
first made of a global section with a few variables, then the regular
(possibly shared) config. Typically it would do something like this :

$ cat /etc/haproxy/env.cfg
global
setenv GRAPH_ADDRESS graph.server.com
setenv GRAPH_PORT 8182

$ sudo haproxy -D -f /etc/haproxy/env.cfg -f /etc/haproxy/haproxy.cfg

Hoping this helps,
Willy

Re: Help with environment variables in config

[Jonathan Matthews]

No. Sudo doesn't pass envvars through to its children by default:
https://stackoverflow.com/questions/8633461/how-to-keep-environment-variables-when-using-sudo

Read that page *and* the comments - in particular be aware that you have to
request (at the CLI) that sudo preserve envvars, and you also have to have
been granted permission to do this, via the sudoers config file.

If this is all sounding a bit complicated, that's because it is.

You've chosen a relatively uncommon way of running haproxy - directly, via
sudo. Consider running via an init script or systemd unit (?) or, failing
that, just a script which is itself the sudo target, which sets the envvars
in the privileged environment.

J

On Sat, 21 Jul 2018 at 17:31, jdtommy  wrote:

> would this chain of calls not work?
>
> ubuntu@ip-172-31-30-4:~$ export GRAPH_ADDRESS=graph.server.com
> ubuntu@ip-172-31-30-4:~$ export GRAPH_PORT=8182
> ubuntu@ip-172-31-30-4:~$ sudo haproxy -d -V -f /etc/haproxy/haproxy.cfg
>
> On Sat, Jul 21, 2018 at 3:26 AM Igor Cicimov <
> ig...@encompasscorporation.com> wrote:
>
>> On Sat, Jul 21, 2018 at 7:12 PM, Jonathan Matthews <
>> cont...@jpluscplusm.com> wrote:
>>
>>> On Sat, 21 Jul 2018 at 09:12, jdtommy  wrote:
>>>
 I am setting them before I start haproxy in the terminal. I tried both
 starting it as a service and starting directly, but neither worked. It
 still would not forward it along.

>>>
>>> Make sure that, as well as setting them, you're *exporting* the envvars
>>> before asking a child process (i.e. haproxy) to use them.
>>>
>>> J
>>> --
>>> Jonathan Matthews
>>> London, UK
>>> http://www.jpluscplusm.com/contact.html
>>>
>> ​
>> As Jonathan said, plus make sure they are included/exported in the init
>> script or systemd file for the service.
>>
>>
>
> --
> Jarad Duersch
>
-- 
Jonathan Matthews
London, UK
http://www.jpluscplusm.com/contact.html

Re: Help with environment variables in config

[jdtommy]

actually, looking at this now I realize my sudo is messing it up.  I need
to set the env variables for the su environment. Yeah it works now.
Thanks for helping me get to that conclusion.

On Sat, Jul 21, 2018 at 10:31 AM jdtommy  wrote:

> would this chain of calls not work?
>
> ubuntu@ip-172-31-30-4:~$ export GRAPH_ADDRESS=graph.server.com
> ubuntu@ip-172-31-30-4:~$ export GRAPH_PORT=8182
> ubuntu@ip-172-31-30-4:~$ sudo haproxy -d -V -f /etc/haproxy/haproxy.cfg
>
> On Sat, Jul 21, 2018 at 3:26 AM Igor Cicimov <
> ig...@encompasscorporation.com> wrote:
>
>> On Sat, Jul 21, 2018 at 7:12 PM, Jonathan Matthews <
>> cont...@jpluscplusm.com> wrote:
>>
>>> On Sat, 21 Jul 2018 at 09:12, jdtommy  wrote:
>>>
 I am setting them before I start haproxy in the terminal. I tried both
 starting it as a service and starting directly, but neither worked. It
 still would not forward it along.

>>>
>>> Make sure that, as well as setting them, you're *exporting* the envvars
>>> before asking a child process (i.e. haproxy) to use them.
>>>
>>> J
>>> --
>>> Jonathan Matthews
>>> London, UK
>>> http://www.jpluscplusm.com/contact.html
>>>
>> ​
>> As Jonathan said, plus make sure they are included/exported in the init
>> script or systemd file for the service.
>>
>>
>
> --
> Jarad Duersch
>


-- 
Jarad Duersch

Re: Help with environment variables in config

[jdtommy]

would this chain of calls not work?

ubuntu@ip-172-31-30-4:~$ export GRAPH_ADDRESS=graph.server.com
ubuntu@ip-172-31-30-4:~$ export GRAPH_PORT=8182
ubuntu@ip-172-31-30-4:~$ sudo haproxy -d -V -f /etc/haproxy/haproxy.cfg

On Sat, Jul 21, 2018 at 3:26 AM Igor Cicimov 
wrote:

> On Sat, Jul 21, 2018 at 7:12 PM, Jonathan Matthews <
> cont...@jpluscplusm.com> wrote:
>
>> On Sat, 21 Jul 2018 at 09:12, jdtommy  wrote:
>>
>>> I am setting them before I start haproxy in the terminal. I tried both
>>> starting it as a service and starting directly, but neither worked. It
>>> still would not forward it along.
>>>
>>
>> Make sure that, as well as setting them, you're *exporting* the envvars
>> before asking a child process (i.e. haproxy) to use them.
>>
>> J
>> --
>> Jonathan Matthews
>> London, UK
>> http://www.jpluscplusm.com/contact.html
>>
> ​
> As Jonathan said, plus make sure they are included/exported in the init
> script or systemd file for the service.
>
>

-- 
Jarad Duersch

Re: Help with environment variables in config

[Igor Cicimov]

On Sat, Jul 21, 2018 at 7:12 PM, Jonathan Matthews 
wrote:

> On Sat, 21 Jul 2018 at 09:12, jdtommy  wrote:
>
>> I am setting them before I start haproxy in the terminal. I tried both
>> starting it as a service and starting directly, but neither worked. It
>> still would not forward it along.
>>
>
> Make sure that, as well as setting them, you're *exporting* the envvars
> before asking a child process (i.e. haproxy) to use them.
>
> J
> --
> Jonathan Matthews
> London, UK
> http://www.jpluscplusm.com/contact.html
>
​
As Jonathan said, plus make sure they are included/exported in the init
script or systemd file for the service.

Re: Help with environment variables in config

[Jonathan Matthews]

On Sat, 21 Jul 2018 at 09:12, jdtommy  wrote:

> I am setting them before I start haproxy in the terminal. I tried both
> starting it as a service and starting directly, but neither worked. It
> still would not forward it along.
>

Make sure that, as well as setting them, you're *exporting* the envvars
before asking a child process (i.e. haproxy) to use them.

J

> --
Jonathan Matthews
London, UK
http://www.jpluscplusm.com/contact.html

Re: Help with environment variables in config

[jdtommy]

I am setting them before I start haproxy in the terminal. I tried both
starting it as a service and starting directly, but neither worked. It
still would not forward it along.

On Sat, Jul 21, 2018, 2:06 AM Igor Cicimov 
wrote:

>
>
> On Sat, Jul 21, 2018 at 4:49 PM, jdtommy  wrote:
>
>> here is my simple `listen` section of the haproxy config file:
>>
>> listen graph_front
>>bind *:8182
>>mode tcp
>>server graph_server graph.server.com:8182
>>
>> this works just fine, but I need the address and port to be a environment
>> variable. So I changed it to this:
>>
>> listen graph_front
>>bind *:8182
>>mode tcp
>>server graph_server "$GRAPH_ADDRESS":"$GRAPH_PORT"
>>
>> the haproxy server restarts just fine, but it no longer forwards to the
>> address in the environment variables.
>>
>> any ideas?
>>
>
> ​Where are you setting the variables?​
>
>

Re: Help with environment variables in config

[Igor Cicimov]

On Sat, Jul 21, 2018 at 4:49 PM, jdtommy  wrote:

> here is my simple `listen` section of the haproxy config file:
>
> listen graph_front
>bind *:8182
>mode tcp
>server graph_server graph.server.com:8182
>
> this works just fine, but I need the address and port to be a environment
> variable. So I changed it to this:
>
> listen graph_front
>bind *:8182
>mode tcp
>server graph_server "$GRAPH_ADDRESS":"$GRAPH_PORT"
>
> the haproxy server restarts just fine, but it no longer forwards to the
> address in the environment variables.
>
> any ideas?
>

​Where are you setting the variables?​

Re: help for setting up haproxy using Linux system proxy

[Vivek Malik]

I had faced the same issue and have documented my solution at
https://www.mail-archive.com/haproxy@formilux.org/msg26387.html

Regards,
Vivek

On Sat, Jul 8, 2017 at 5:44 AM, Aaron West  wrote:
> On 8 July 2017 at 11:25, Aaron West  wrote:
>> for me at least, it would
>> boggle the mind that you have a reverse proxy using a forward proxy.
>
> I think I I should clarify the above... I don't mean using forward
> proxies as a real server(Think webfilters) because that makes sense I
> just mean routing to client or server via a forward proxy seems
> confusing to me...
>
> Aaron West
>
> Loadbalancer.org
>
> www.loadbalancer.org
> +1 888 867 9504 / +44 (0)330 380 1064
> aa...@loadbalancer.org
>
> LEAVE A REVIEW | DEPLOYMENT GUIDES | BLOG
>

Re: help for configuration between http and tcp mode

[M]

Hi,

> 
> Because your Host header is certainly "influxdb-drp.example.net:8086", not 
> "influxdb-drp.example.net". You can verify this with this acl instead :
>  acl host_influxdb-drp.example.net hdr(host) -i influxdb-drp.example.net:8086
> 
> Or you can even capture the header in your logs, it's quite useful to debug 
> acls ;-)
> 

You are right. The port was missing from acl :(

Using "capture request header Host len 150"  was effectively showing it in log:

Jul  9 18:54:59 kalinga haproxy[46185]: 192.168.246.17:59204 
[09/Jul/2017:18:54:59.275] https_influxdb~ influxdb-drp.example.net/https_8086 
0/0/1/0/1 200 332 - -  1/1/0/0/0 0/0 {influxdb-drp.example.net:8086} "GET 
/query?q=SHOW%20DATABASES HTTP/1.1"

Thank you.

Regards,
M.

Re: help for configuration between http and tcp mode

[Cyril Bonté]

Hi,

Le 09/07/2017 à 17:58, M a écrit :

Hi,

It seems the error is related to acl and I don’t yet understand why > [...]
frontend https_influxdb
   bind 192.168.246.17:8086 ssl crt /data/ssl_certs no-sslv3 ciphers 
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:DES-CBC3-SHA:!NULL:!aNULL:!RC4:!RC2:!MEDIUM:!LOW:!EXPORT:!DES:!MD5:!PSK:!3DES
   mode http
[...]
   acl host_influxdb-drp.example.net hdr(host) -i influxdb-drp.example.net
   use_backend influxdb-drp.example.net if host_influxdb-drp.example.net
[...]
#curl -G https://influxdb-drp.example.net:8086/query -u admin:'xxx' --data-urlencode 
"q=SHOW DATABASES"
503 Service Unavailable
No server is available to handle this request.


Jul  9 15:46:16 kalinga haproxy[50375]: 192.168.246.17:57242 
[09/Jul/2017:15:46:16.665] https_influxdb~ https_influxdb/ -1/-1/135 212 
SC 4/0/0/0/0 0/0

The acl is not matching under this frontend :-(
[...] 
Why acl is matching only on frontend https and not on frontend https_influxdb?


Because your Host header is certainly "influxdb-drp.example.net:8086", 
not "influxdb-drp.example.net". You can verify this with this acl instead :
  acl host_influxdb-drp.example.net hdr(host) -i 
influxdb-drp.example.net:8086


Or you can even capture the header in your logs, it's quite useful to 
debug acls ;-)



--
Cyril Bonté

Re: help for configuration between http and tcp mode

[M]

Hi,

It seems the error is related to acl and I don’t yet understand why.

I have done additional tests with configuration: 

———

frontend https
  bind 0.0.0.0:443 ssl crt /data/ssl_certs no-sslv3 ciphers 
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:DES-CBC3-SHA:!NULL:!aNULL:!RC4:!RC2:!MEDIUM:!LOW:!EXPORT:!DES:!MD5:!PSK:!3DES
  mode http
  option dontlognull

  reqadd X-Forwarded-Proto:\ https

  option http-server-close
  option forwardfor

  acl host_piwigo.example.org hdr(host) -i piwigo.example.org
  use_backend piwigo.example.org if host_piwigo.example.org

  acl host_wordpress.example.com hdr(host) -i wordpress.example.com
  use_backend wordpress.example.com if host_wordpress.example.com

  # adding redirection on https tcp/443 fronted:
  acl host_influxdb-drp.example.net hdr(host) -i influxdb-drp.example.net
  use_backend influxdb-drp.example.net if host_influxdb-drp.example.net


frontend https_influxdb
  bind 192.168.246.17:8086 ssl crt /data/ssl_certs no-sslv3 ciphers 
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:DES-CBC3-SHA:!NULL:!aNULL:!RC4:!RC2:!MEDIUM:!LOW:!EXPORT:!DES:!MD5:!PSK:!3DES
  mode http
  option dontlognull

  reqadd X-Forwarded-Proto:\ https

  option http-server-close
  option forwardfor

  #default_backend influxdb-drp.example.net # to test later with a default 
backend as acl seems not working 

  acl host_influxdb-drp.example.net hdr(host) -i influxdb-drp.example.net
  use_backend influxdb-drp.example.net if host_influxdb-drp.example.net

backend influxdb-drp.example.net
  mode http
  server https_8086 127.0.0.1:8086 check ssl verify none

———

Backend is answering on https on tcp/8086. 

Testing against frontend https on tcp/443, it is working:

#curl -G https://influxdb-drp.example.net:443/query -u admin:'xxx' 
--data-urlencode "q=SHOW DATABASES"
{"results":[{"statement_id":0,"series":[{"name":"databases","columns":["name"],"values":[["_internal"]]}]}]}

Jul  9 15:46:32 kalinga haproxy[50375]: 192.168.246.17:59154 
[09/Jul/2017:15:46:31.900] https~ influxdb-drp.example.net/https_8086 0/1/132 
332 -- 5/5/0/0/0 0/0

The acl has matched to forward to the expected backend.

Testing against frontend https_influxdb on tcp/8086, it is failing:

#curl -G https://influxdb-drp.example.net:8086/query -u admin:'xxx' 
--data-urlencode "q=SHOW DATABASES"
503 Service Unavailable
No server is available to handle this request.


Jul  9 15:46:16 kalinga haproxy[50375]: 192.168.246.17:57242 
[09/Jul/2017:15:46:16.665] https_influxdb~ https_influxdb/ -1/-1/135 212 
SC 4/0/0/0/0 0/0

The acl is not matching under this frontend :-(

When testing by uncommenting following line:
  #default_backend influxdb-drp.example.net
in order to add a default backend, it is working:

#curl -G https://influxdb-drp.example.net:8086/query -u admin:'xxx' 
--data-urlencode "q=SHOW DATABASES"
{"results":[{"statement_id":0,"series":[{"name":"databases","columns":["name"],"values":[["_internal"]]}]}]}

Jul  9 15:46:32 kalinga haproxy[50375]: 192.168.246.17:59154 
[09/Jul/2017:15:46:31.900] https~ influxdb-drp.example.net/https_8086 0/1/132 
332 -- 5/5/0/0/0 0/0

Why acl is matching only on frontend https and not on frontend https_influxdb?

Haproxy version is the latest stable 1.7.8.


Regards,
M.

> Le 8 juil. 2017 à 22:19, M  a écrit :
> 
> Hi,
> 
> I don't understand why http mode is no able provide a backend server whereas 
> at same time tcp mode is able to do it. 
> 
> I am trying to setup Haproxy in front of an Influxdb database running on 
> HTTPS.
> 
> When frontend is configured on http mode, requests are failing with NOSRV 
> error.
> When fronted is configure on tcp mod, requests are working. 
> 
> Example below with http mode on tcp/8086 port and tcp mode on tcp/8087 with 
> same backend:
> 
> #curl -G https://influxdb-drp.example.net:8086/query -u admin:'' 
> --data-urlencode "q=SHOW DATABASES"
> 503 Service Unavailable
> No server is available to handle this request.
> 
> 
> Haproxy log showing NOSRV and SC:
> Jul  8 19:59:44 kalinga haproxy[26228]: 192.168.246.17:52946 
> [08/Jul/2017:19:59:44.661] https_influxdb~ https_influxdb/ -1/-1/136 
> 212 SC 0/0/0/0/0 0/0
> 
> 
> #curl -G https://influxdb-drp.example.net:8087/query -u admin:'' 
> --data-urlencode "q=SHOW DATABASES"
> {"results":[{"statement_id":0,"series":[{"name":"databases","columns":["name"],"values":[["_internal"]]}]}]}
> 
> Haproxy log:
> Jul  8 20:00:16 kalinga haproxy[26228]: 192.168.246.17:37142 
> [08/Jul/2017:20:00:16.672] https_influxdb_tcp~ 
> influxdb-drp.example.net/https_8086 0/2/123 332 -- 1/1/0/0/0 0/0
> 
> Haproxy configuration file:
> 
> 

Re: help for configuration between http and tcp mode

[Andrew Smalley]

Hello M

Looking at your configuration you are trying to use HTTPS/TCP/SSL
Connection for HTTP

frontend https_influxdb
  bind 192.168.246.17:8086 ssl crt /data/ssl_certs no-sslv3 ciphers
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:
ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-
RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:
DHE-RSA-AES128-SHA:DES-CBC3-SHA:!NULL:!aNULL:!RC4:!RC2:!
MEDIUM:!LOW:!EXPORT:!DES:!MD5:!PSK:!3DES
  mode http
  option dontlognull

I would make this assuming port 8086 is a HTTP Connection

frontend https_influxdb
  bind 192.168.246.17:8086
  mode http
  option dontlognull


TCP Mode will work with any connection however HTTP will only work with
unencrypted HTTP Type traffic as it is application aware.

Also TCP Mode is really Layer4 and non application aware.

Andrew Smalley

Loadbalancer.org Ltd.
www.loadbalancer.org 






+1 888 867 9504 / +44 (0)330 380 1064
asmal...@loadbalancer.org

Leave a Review
 | Deployment
Guides

| Blog 

On 8 July 2017 at 21:19, M  wrote:

> Hi,
>
> I don't understand why http mode is no able provide a backend server
> whereas at same time tcp mode is able to do it.
>
> I am trying to setup Haproxy in front of an Influxdb database running on
> HTTPS.
>
> When frontend is configured on http mode, requests are failing with NOSRV
> error.
> When fronted is configure on tcp mod, requests are working.
>
> Example below with http mode on tcp/8086 port and tcp mode on tcp/8087
> with same backend:
>
> #curl -G https://influxdb-drp.example.net:8086/query -u admin:''
> --data-urlencode "q=SHOW DATABASES"
> 503 Service Unavailable
> No server is available to handle this request.
> 
>
> Haproxy log showing NOSRV and SC:
> Jul  8 19:59:44 kalinga haproxy[26228]: 192.168.246.17:52946
> [08/Jul/2017:19:59:44.661] https_influxdb~ https_influxdb/ -1/-1/136
> 212 SC 0/0/0/0/0 0/0
>
>
> #curl -G https://influxdb-drp.example.net:8087/query -u admin:''
> --data-urlencode "q=SHOW DATABASES"
> {"results":[{"statement_id":0,"series":[{"name":"databases",
> "columns":["name"],"values":[["_internal"]]}]}]}
>
> Haproxy log:
> Jul  8 20:00:16 kalinga haproxy[26228]: 192.168.246.17:37142
> [08/Jul/2017:20:00:16.672] https_influxdb_tcp~ influxdb-drp.example.net/
> https_8086 0/2/123 332 -- 1/1/0/0/0 0/0
>
> Haproxy configuration file:
>
> 
> ---
>
> global
> log /dev/log local1 debug
> maxconn 4096
> debug
> tune.ssl.default-dh-param 4096
>
> defaults
> log global
> modetcp
> option  tcplog
> retries 3
> option redispatch
> maxconn 4000
> timeout connect 5000ms
> timeout client 5ms
> timeout server 5ms
>
> frontend https_influxdb
>   bind 192.168.246.17:8086 ssl crt /data/ssl_certs no-sslv3 ciphers
> ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:
> ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-
> RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:
> DHE-RSA-AES128-SHA:DES-CBC3-SHA:!NULL:!aNULL:!RC4:!RC2:!
> MEDIUM:!LOW:!EXPORT:!DES:!MD5:!PSK:!3DES
>   mode http
>   option dontlognull
>
>   reqadd X-Forwarded-Proto:\ https
>
>   option http-server-close
>   option forwardfor
>
>   acl host_influxdb-drp.example.net hdr(host) -i influxdb-drp.example.net
>   use_backend influxdb-drp.example.net if host_influxdb-drp.example.net
>
> frontend https_influxdb_tcp
>   bind 192.168.246.17:8087 ssl crt /data/ssl_certs no-sslv3 ciphers
> ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:
> ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-
> RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:
> DHE-RSA-AES128-SHA:DES-CBC3-SHA:!NULL:!aNULL:!RC4:!RC2:!
> MEDIUM:!LOW:!EXPORT:!DES:!MD5:!PSK:!3DES
>   mode tcp
>   option dontlognull
>   option http-server-close
>   option tcplog
>
>   default_backend influxdb-drp.example.net
>
> backend influxdb-drp.example.net
>   mode http
>   server https_8086 127.0.0.1:8086 check ssl verify none
>
> 
> ---
>
> How can I configure Haproxy to work on http mode?
>
> M.
>

Re: help for setting up haproxy using Linux system proxy

[Aaron West]

On 8 July 2017 at 11:25, Aaron West  wrote:
> for me at least, it would
> boggle the mind that you have a reverse proxy using a forward proxy.

I think I I should clarify the above... I don't mean using forward
proxies as a real server(Think webfilters) because that makes sense I
just mean routing to client or server via a forward proxy seems
confusing to me...

Aaron West

Loadbalancer.org

www.loadbalancer.org
+1 888 867 9504 / +44 (0)330 380 1064
aa...@loadbalancer.org

LEAVE A REVIEW | DEPLOYMENT GUIDES | BLOG

Re: help for setting up haproxy using Linux system proxy

[Aaron West]

I'm going to go out on a limb here and say that you can't(Or shouldn't
just in case somehow you can...), first, for me at least, it would
boggle the mind that you have a reverse proxy using a forward proxy.

I guess you could use a transparent proxy although I'm still not sure
why but HAproxy would then just be routing to it's destination unaware
there's a proxy in the way...

However, the question really has to be why? Only thing I can think of
is that you are trying to implement a DMZ so have decided to restrict
all outbound traffic, why not just allow the specific traffic through
the firewall? You can cherry pick exactly what to allow so only allow
the destination or sources that you actually require, it would be much
much easier.

Sorry if that's not what you wanted to hear or if I'm wrong in some way...

Aaron West

Loadbalancer.org

www.loadbalancer.org
+1 888 867 9504 / +44 (0)330 380 1064
aa...@loadbalancer.org

LEAVE A REVIEW | DEPLOYMENT GUIDES | BLOG


On 8 July 2017 at 08:56, founder  wrote:
>
> i have linux sever setting up with haproxy but this sever can't access 
> directly to Internet . we have set up a proxy at linux server but it seems 
> haproxy has not used system proxy. how can we set haproxy to use system 
> proxy. thanks in adance.
> 发自我的iPhone

Re: Help Needed || haproxy limiting the connection rate per user

[Chad Lavoie]

Greetings,


On 08/30/2016 01:10 PM, Samrat Roy wrote:

Thank you sir for your quick reply.

I am now able to give custom error code for my HAproxy configuration. 
However I am facing one more issue .


With the above approach HAproxy is rejecting each and every calls once 
the limit has crossed. It is behaving as a circuit breaker . But my 
requirement is to have a throttling for example every 10 second I 
should allow 200 request and anything more than 200 will be rejected.


There are two ways I can think to interpret your question:
1) You want to have a tick every 10 seconds which resets the counter to zero
2) You want to not count requests over the limit (which get blocked) to 
count to the blocking


For 1 you would need a script to talk to the socket, and I'd not advise 
doing that unless you know what you are doing and why there is no 
cleaner alternative.
For 2 I'd add gpc0,gpc0_rate(10s) to the stick table in place of 
conn_rate, then use something like the following:

http-request allow if { sc_inc_gpc0(0) }
After the use_backend statement.  Then instead of checking conn_rate 
check sc_gpc0_rate(0) per 
http://cbonte.github.io/haproxy-dconv/1.7/configuration.html#7.3.3-sc_gpc0_rate.


Because in that case gpc0 will only be incremented if the request 
doesn't end up at the custom backend/blocked/etc that should fill your 
needs there.


Thanks,
- Chad


Is there any way I can achieve this .Please help me to configure the same.

Thanks in advance
Samrat


On Fri, Aug 26, 2016 at 10:16 PM, Chad Lavoie > wrote:


Greetings,


On 08/26/2016 09:14 AM, Samrat Roy wrote:

Hello Sir,



down votefavorite





I am trying to achieve rate limiting using HAProxy. I am trying
to follow the "Limiting the connection rate per user" approach. I
am able to achieve this by the below configuration. But facing
one problem, that is, i am not able to send a custom error code
once the rate limit is reached. For example if i reached the rate
limit i want to send HTTP error code 429. In this case the proxy
is simply rejecting the incoming call and users are getting http
status code as 0.



"tcp-request connection reject" rejects the connection, so there
is no status code in this case.  If you want to send a 403 replace
it with "http-request deny if ..." instead.

If you want to respond with HTTP 429 make a backend with no
backend servers (so that all requests will get a 503) and set a
custom 503 error page, editing the headers at the top of the file
so that the response code is 429 (or whatever other
code/message/etc you desire).

- Chad


Please let me know how can i do this

frontend localnodes

|bind *:80 mode http default_backend nodes stick-table type ip
size 100k expire 30s store conn_rate(5s) tcp-request connection
reject if { src_conn_rate ge 60 } tcp-request connection
track-sc1 src |

backend nodes

|cookie MYSRV insert indirect nocache server srv1 :80
check cookie srv1 maxconn 500 |


Thanks
Samrat

Re: Help Needed || haproxy limiting the connection rate per user

[Samrat Roy]

Thank you sir for your quick reply.

I am now able to give custom error code for my HAproxy configuration.
However I am facing one more issue .

With the above approach HAproxy is rejecting each and every calls once the
limit has crossed. It is behaving as a circuit breaker . But my requirement
is to have a throttling for example every 10 second I should allow 200
request and anything more than 200 will be rejected.

Is there any way I can achieve this .Please help me to configure the same.

Thanks in advance
Samrat


On Fri, Aug 26, 2016 at 10:16 PM, Chad Lavoie  wrote:

> Greetings,
>
> On 08/26/2016 09:14 AM, Samrat Roy wrote:
>
> Hello Sir,
>
>
>
> down votefavorite
> 
>
> I am trying to achieve rate limiting using HAProxy. I am trying to follow
> the "Limiting the connection rate per user" approach. I am able to achieve
> this by the below configuration. But facing one problem, that is, i am not
> able to send a custom error code once the rate limit is reached. For
> example if i reached the rate limit i want to send HTTP error code 429. In
> this case the proxy is simply rejecting the incoming call and users are
> getting http status code as 0.
>
>
> "tcp-request connection reject" rejects the connection, so there is no
> status code in this case.  If you want to send a 403 replace it with
> "http-request deny if ..." instead.
>
> If you want to respond with HTTP 429 make a backend with no backend
> servers (so that all requests will get a 503) and set a custom 503 error
> page, editing the headers at the top of the file so that the response code
> is 429 (or whatever other code/message/etc you desire).
>
> - Chad
>
> Please let me know how can i do this
>
> frontend localnodes
>
> bind *:80
> mode http
> default_backend nodes
>
> stick-table type ip size 100k expire 30s store conn_rate(5s)
> tcp-request connection reject if { src_conn_rate ge 60 }
> tcp-request connection track-sc1 src
>
> backend nodes
>
> cookie MYSRV insert indirect nocache
> server srv1 :80 check cookie srv1 maxconn 500
>
>
> Thanks
> Samrat
>
>
>

Re: Help Needed || haproxy limiting the connection rate per user

[Chad Lavoie]

Greetings,


On 08/26/2016 09:14 AM, Samrat Roy wrote:

Hello Sir,



down votefavorite 
 




I am trying to achieve rate limiting using HAProxy. I am trying to 
follow the "Limiting the connection rate per user" approach. I am able 
to achieve this by the below configuration. But facing one problem, 
that is, i am not able to send a custom error code once the rate limit 
is reached. For example if i reached the rate limit i want to send 
HTTP error code 429. In this case the proxy is simply rejecting the 
incoming call and users are getting http status code as 0.




"tcp-request connection reject" rejects the connection, so there is no 
status code in this case.  If you want to send a 403 replace it with 
"http-request deny if ..." instead.


If you want to respond with HTTP 429 make a backend with no backend 
servers (so that all requests will get a 503) and set a custom 503 error 
page, editing the headers at the top of the file so that the response 
code is 429 (or whatever other code/message/etc you desire).


- Chad


Please let me know how can i do this

frontend localnodes

|bind *:80 mode http default_backend nodes stick-table type ip size 
100k expire 30s store conn_rate(5s) tcp-request connection reject if { 
src_conn_rate ge 60 } tcp-request connection track-sc1 src |


backend nodes

|cookie MYSRV insert indirect nocache server srv1 :80 check 
cookie srv1 maxconn 500 |



Thanks
Samrat

Re: Help

[Baptiste]

Hi,

It supports it, just need the right version:
http://blog.haproxy.com/haproxy/proxy-protocol/

Baptiste

On Tue, Aug 2, 2016 at 11:59 PM, Jeff Palmer  wrote:

> OK,  then you'll need to enable the PROXY protocol in exim assuming it
> supports the protocol.
>
>
>
> On Tue, Aug 2, 2016 at 5:39 PM, Grégory MARCASSIN - Magic <
> gmarcas...@magic.fr> wrote:
>
>> It s true but i want the real IP of the client so i need this option.
>>
>>
>>
>>
>> Le 02/08/2016 à 23:37, Jeff Palmer a écrit :
>>
>> I'm not familiar with exim itself,  but unless it supports the PROXY
>> protocol,  you should probably remove the "send-proxy" part of your
>> bk_postfix backend.
>>
>>
>>
>> On Tue, Aug 2, 2016 at 5:21 PM, Grégory MARCASSIN - Magic <
>> gmarcas...@magic.fr> wrote:
>>
>>> Hi,
>>>
>>> Sorry to ask you something that I can probably find on the internet. :$
>>> But I do not find clearly what I wish.
>>> Thank you in advance for your help.
>>>
>>> I wish to inform exim to support haproxy
>>>
>>> check:
>>>
>>> $ Telnet VIP 25
>>> Trying 192.168.56.50 ...
>>> Connected to vip.
>>> Escape character is '^]'.
>>> 554 SMTP synchronization error
>>> Connection closed by foreign host.
>>>
>>> log:
>>> 2016-08-02 8:37:37 p.m. SMTP protocol synchronization error (input feels
>>> without waiting for greeting) rejected connection from H lb1web
>>> [192.168.56.51] input = "PROXY TCP4 192.168.56.1 192.168.56.50 64036 25 \ r
>>> \ not"
>>>
>>>
>>> Currently here is my conf
>>>
>>>  global
>>> chroot  /var/lib/haproxy
>>> pidfile /var/run/haproxy.pid
>>> maxconn 8000
>>> userhaproxy
>>> group   haproxy
>>> daemon
>>> tune.ssl.default-dh-param 2048
>>> tune.bufsize 72000
>>>
>>> #nbproc 4
>>>
>>> # turn on stats unix socket
>>> stats socket /var/lib/haproxy/stats
>>>
>>> defaults
>>> modehttp
>>> log global
>>> option  httplog
>>> option  dontlognull
>>> option  http-server-close
>>> option forwardfor
>>> option forwardfor   header X-ClientIP
>>> option  redispatch
>>> option  httpchk
>>> retries 3
>>> timeout http-request5s
>>> timeout queue   1m
>>> timeout connect 5s
>>> timeout client  30s
>>> timeout server  10s
>>> timeout http-keep-alive 10s
>>> timeout check   10s
>>> maxconn 3000
>>>
>>> # HAProxy web ui
>>> listen stats 192.168.56.50:8080
>>> mode http
>>> stats hide-version
>>> stats enable
>>> stats uri /stats
>>> stats realm HAProxy\ Statistics
>>> stats auth admin:admin
>>> stats admin if TRUE
>>>
>>> frontend smtp_proxy
>>> mode tcp
>>> bind 192.168.56.50:25 transparent
>>> default_backend bk_postfix
>>>
>>> backend bk_postfix
>>> mode tcp
>>> option smtpchk
>>> option forwardfor
>>> #source 0.0.0.0 usesrc clientip
>>> server postfix2 192.168.56.53:25 send-proxy check
>>> server postfix3 192.168.56.54:25 send-proxy check
>>>
>>>
>>> Exim with défault config
>>>
>>> Le 02/08/2016 à 23:12, Jeff Palmer a écrit :
>>>
>>> What specifically are you having issues with,  and what are the
>>> configurations you have already tried?
>>>
>>>
>>> Just asking someone to do your job for you isn't likely to get a lot of 
>>> replies.
>>>
>>>
>>>
>>> On Tue, Aug 2, 2016 at 4:36 PM, Grégory MARCASSIN - 
>>> Magic  wrote:
>>>
>>> Hi,
>>> Sorry to ask ...
>>>
>>> Can you send me a url or a conf for the support EXIM4 ?
>>>
>>>
>>> Thx a lot
>>>
>>> Best regards
>>> --
>>>
>>>
>>>
>>>
>>> --
>>> 130-134 Avenue du Président Wilson
>>> 93512 Montreuil Cedex
>>> www.magic.fr
>>>
>>> Grégory MARCASSIN
>>> Administrateur Système
>>> Tél : +33 (0)1 41 58 22 81
>>> Fax : +33 (0)1 56 72 93 30 <%2B33%20%280%291%2056%2072%2093%2030>
>>> SAV : +33 (0)1 41 58 22 50 <%2B33%20%280%291%2041%2058%2022%2050>
>>>
>>
>>
>>
>> --
>> Jeff Palmer
>> https://PalmerIT.net
>>
>>
>> --
>> 130-134 Avenue du Président Wilson
>> 93512 Montreuil Cedex
>> www.magic.fr
>>
>> Grégory MARCASSIN
>> Administrateur Système
>> Tél : +33 (0)1 41 58 22 81
>> Fax : +33 (0)1 56 72 93 30 <%2B33%20%280%291%2056%2072%2093%2030>
>> SAV : +33 (0)1 41 58 22 50 <%2B33%20%280%291%2041%2058%2022%2050>
>>
>
>
>
> --
> Jeff Palmer
> https://PalmerIT.net
>

Re: Help

[Cyril Bonté]

Hi,

Le 02/08/2016 à 23:21, Grégory MARCASSIN - Magic a écrit :

Hi,

Sorry to ask you something that I can probably find on the internet. :$
But I do not find clearly what I wish.
Thank you in advance for your help.

I wish to inform exim to support haproxy

check:

$ Telnet VIP 25
Trying 192.168.56.50 ...
Connected to vip.
Escape character is '^]'.
554 SMTP synchronization error
Connection closed by foreign host.

log:
2016-08-02 8:37:37 p.m. SMTP protocol synchronization error (input feels
without waiting for greeting) rejected connection from H lb1web
[192.168.56.51] input = "PROXY TCP4 192.168.56.1 192.168.56.50 64036 25
\ r \ not"


Currently here is my conf

 global
chroot  /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 8000
userhaproxy
group   haproxy
daemon
tune.ssl.default-dh-param 2048
tune.bufsize 72000

#nbproc 4

# turn on stats unix socket
stats socket /var/lib/haproxy/stats

defaults
modehttp
log global
option  httplog
option  dontlognull
option  http-server-close
option forwardfor
option forwardfor   header X-ClientIP
option  redispatch
option  httpchk
retries 3
timeout http-request5s
timeout queue   1m
timeout connect 5s
timeout client  30s
timeout server  10s
timeout http-keep-alive 10s
timeout check   10s
maxconn 3000

# HAProxy web ui
listen stats 192.168.56.50:8080
mode http
stats hide-version
stats enable
stats uri /stats
stats realm HAProxy\ Statistics
stats auth admin:admin
stats admin if TRUE

frontend smtp_proxy
mode tcp
bind 192.168.56.50:25 transparent
default_backend bk_postfix

backend bk_postfix
mode tcp
option smtpchk
option forwardfor
#source 0.0.0.0 usesrc clientip
server postfix2 192.168.56.53:25 send-proxy check
server postfix3 192.168.56.54:25 send-proxy check


Here, you are enabling the Proxy Protocol when connecting to Exim 
(really a "good" idea to name those servers "postfix2" and "postfix3"...)



Exim with défault config


You have to enable the Proxy Protocol on the exim side, then.
Please read the Exim documentation to see how to do that (look at the 
keyword "hosts_proxy").





Le 02/08/2016 à 23:12, Jeff Palmer a écrit :

What specifically are you having issues with,  and what are the
configurations you have already tried?


Just asking someone to do your job for you isn't likely to get a lot of replies.



On Tue, Aug 2, 2016 at 4:36 PM, Grégory MARCASSIN - Magic
 wrote:

Hi,
Sorry to ask ...

Can you send me a url or a conf for the support EXIM4 ?


Thx a lot

Best regards
--







--
130-134 Avenue du Président Wilson
93512 Montreuil Cedex
www.magic.fr 


Grégory MARCASSIN
Administrateur Système
Tél : +33 (0)1 41 58 2281
Fax : +33 (0)1 56 72 93 30 
SAV : +33 (0)1 41 58 22 50 




--
Cyril Bonté

Re: Help

[Grégory MARCASSIN - Magic]

It s true but i want the real IP of the client so i need this option.




Le 02/08/2016 à 23:37, Jeff Palmer a écrit :
I'm not familiar with exim itself,  but unless it supports the PROXY 
protocol,  you should probably remove the "send-proxy" part of your 
bk_postfix backend.




On Tue, Aug 2, 2016 at 5:21 PM, Grégory MARCASSIN - Magic 
> wrote:


Hi,

Sorry to ask you something that I can probably find on the
internet. :$
But I do not find clearly what I wish.
Thank you in advance for your help.

I wish to inform exim to support haproxy

check:

$ Telnet VIP 25
Trying 192.168.56.50 ...
Connected to vip.
Escape character is '^]'.
554 SMTP synchronization error
Connection closed by foreign host.

log:
2016-08-02 8:37:37 p.m. SMTP protocol synchronization error (input
feels without waiting for greeting) rejected connection from H
lb1web [192.168.56.51] input = "PROXY TCP4 192.168.56.1
192.168.56.50 64036 25 \ r \ not"


Currently here is my conf

 global
chroot  /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 8000
userhaproxy
group   haproxy
daemon
tune.ssl.default-dh-param 2048
tune.bufsize 72000

#nbproc 4

# turn on stats unix socket
stats socket /var/lib/haproxy/stats

defaults
modehttp
log global
option  httplog
option  dontlognull
option  http-server-close
option forwardfor
option forwardfor   header X-ClientIP
option  redispatch
option  httpchk
retries 3
timeout http-request5s
timeout queue   1m
timeout connect 5s
timeout client  30s
timeout server  10s
timeout http-keep-alive 10s
timeout check   10s
maxconn 3000

# HAProxy web ui
listen stats 192.168.56.50:8080 
mode http
stats hide-version
stats enable
stats uri /stats
stats realm HAProxy\ Statistics
stats auth admin:admin
stats admin if TRUE

frontend smtp_proxy
mode tcp
bind 192.168.56.50:25  transparent
default_backend bk_postfix

backend bk_postfix
mode tcp
option smtpchk
option forwardfor
#source 0.0.0.0 usesrc clientip
server postfix2 192.168.56.53:25 
send-proxy check
server postfix3 192.168.56.54:25 
send-proxy check


Exim with défault config

Le 02/08/2016 à 23:12, Jeff Palmer a écrit :

What specifically are you having issues with,  and what are the
configurations you have already tried?


Just asking someone to do your job for you isn't likely to get a lot of 
replies.



On Tue, Aug 2, 2016 at 4:36 PM, Grégory MARCASSIN - Magic
   wrote:

Hi,
Sorry to ask ...

Can you send me a url or a conf for the support EXIM4 ?


Thx a lot

Best regards
--




-- 
130-134 Avenue du Président Wilson

93512 Montreuil Cedex
www.magic.fr 

Grégory MARCASSIN
Administrateur Système
Tél : +33 (0)1 41 58 2281
Fax : +33 (0)1 56 72 93 30 
SAV : +33 (0)1 41 58 22 50 




--
Jeff Palmer
https://PalmerIT.net


--
130-134 Avenue du Président Wilson
93512 Montreuil Cedex
www.magic.fr 

Grégory MARCASSIN
Administrateur Système
Tél : +33 (0)1 41 58 2281
Fax : +33 (0)1 56 72 93 30 
SAV : +33 (0)1 41 58 22 50 

Re: Help

[Jeff Palmer]

I'm not familiar with exim itself,  but unless it supports the PROXY
protocol,  you should probably remove the "send-proxy" part of your
bk_postfix backend.



On Tue, Aug 2, 2016 at 5:21 PM, Grégory MARCASSIN - Magic <
gmarcas...@magic.fr> wrote:

> Hi,
>
> Sorry to ask you something that I can probably find on the internet. :$
> But I do not find clearly what I wish.
> Thank you in advance for your help.
>
> I wish to inform exim to support haproxy
>
> check:
>
> $ Telnet VIP 25
> Trying 192.168.56.50 ...
> Connected to vip.
> Escape character is '^]'.
> 554 SMTP synchronization error
> Connection closed by foreign host.
>
> log:
> 2016-08-02 8:37:37 p.m. SMTP protocol synchronization error (input feels
> without waiting for greeting) rejected connection from H lb1web
> [192.168.56.51] input = "PROXY TCP4 192.168.56.1 192.168.56.50 64036 25 \ r
> \ not"
>
>
> Currently here is my conf
>
>  global
> chroot  /var/lib/haproxy
> pidfile /var/run/haproxy.pid
> maxconn 8000
> userhaproxy
> group   haproxy
> daemon
> tune.ssl.default-dh-param 2048
> tune.bufsize 72000
>
> #nbproc 4
>
> # turn on stats unix socket
> stats socket /var/lib/haproxy/stats
>
> defaults
> modehttp
> log global
> option  httplog
> option  dontlognull
> option  http-server-close
> option forwardfor
> option forwardfor   header X-ClientIP
> option  redispatch
> option  httpchk
> retries 3
> timeout http-request5s
> timeout queue   1m
> timeout connect 5s
> timeout client  30s
> timeout server  10s
> timeout http-keep-alive 10s
> timeout check   10s
> maxconn 3000
>
> # HAProxy web ui
> listen stats 192.168.56.50:8080
> mode http
> stats hide-version
> stats enable
> stats uri /stats
> stats realm HAProxy\ Statistics
> stats auth admin:admin
> stats admin if TRUE
>
> frontend smtp_proxy
> mode tcp
> bind 192.168.56.50:25 transparent
> default_backend bk_postfix
>
> backend bk_postfix
> mode tcp
> option smtpchk
> option forwardfor
> #source 0.0.0.0 usesrc clientip
> server postfix2 192.168.56.53:25 send-proxy check
> server postfix3 192.168.56.54:25 send-proxy check
>
>
> Exim with défault config
>
> Le 02/08/2016 à 23:12, Jeff Palmer a écrit :
>
> What specifically are you having issues with,  and what are the
> configurations you have already tried?
>
>
> Just asking someone to do your job for you isn't likely to get a lot of 
> replies.
>
>
>
> On Tue, Aug 2, 2016 at 4:36 PM, Grégory MARCASSIN - 
> Magic  wrote:
>
> Hi,
> Sorry to ask ...
>
> Can you send me a url or a conf for the support EXIM4 ?
>
>
> Thx a lot
>
> Best regards
> --
>
>
>
>
>
> --
> 130-134 Avenue du Président Wilson
> 93512 Montreuil Cedex
> www.magic.fr
>
> Grégory MARCASSIN
> Administrateur Système
> Tél : +33 (0)1 41 58 22 81
> Fax : +33 (0)1 56 72 93 30 <%2B33%20%280%291%2056%2072%2093%2030>
> SAV : +33 (0)1 41 58 22 50 <%2B33%20%280%291%2041%2058%2022%2050>
>



-- 
Jeff Palmer
https://PalmerIT.net

Re: Help

[Grégory MARCASSIN - Magic]

Hi,

Sorry to ask you something that I can probably find on the internet. :$
But I do not find clearly what I wish.
Thank you in advance for your help.

I wish to inform exim to support haproxy

check:

$ Telnet VIP 25
Trying 192.168.56.50 ...
Connected to vip.
Escape character is '^]'.
554 SMTP synchronization error
Connection closed by foreign host.

log:
2016-08-02 8:37:37 p.m. SMTP protocol synchronization error (input feels 
without waiting for greeting) rejected connection from H lb1web 
[192.168.56.51] input = "PROXY TCP4 192.168.56.1 192.168.56.50 64036 25 
\ r \ not"



Currently here is my conf

 global
chroot  /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 8000
userhaproxy
group   haproxy
daemon
tune.ssl.default-dh-param 2048
tune.bufsize 72000

#nbproc 4

# turn on stats unix socket
stats socket /var/lib/haproxy/stats

defaults
modehttp
log global
option  httplog
option  dontlognull
option  http-server-close
option forwardfor
option forwardfor   header X-ClientIP
option  redispatch
option  httpchk
retries 3
timeout http-request5s
timeout queue   1m
timeout connect 5s
timeout client  30s
timeout server  10s
timeout http-keep-alive 10s
timeout check   10s
maxconn 3000

# HAProxy web ui
listen stats 192.168.56.50:8080
mode http
stats hide-version
stats enable
stats uri /stats
stats realm HAProxy\ Statistics
stats auth admin:admin
stats admin if TRUE

frontend smtp_proxy
mode tcp
bind 192.168.56.50:25 transparent
default_backend bk_postfix

backend bk_postfix
mode tcp
option smtpchk
option forwardfor
#source 0.0.0.0 usesrc clientip
server postfix2 192.168.56.53:25 send-proxy check
server postfix3 192.168.56.54:25 send-proxy check


Exim with défault config

Le 02/08/2016 à 23:12, Jeff Palmer a écrit :

What specifically are you having issues with,  and what are the
configurations you have already tried?


Just asking someone to do your job for you isn't likely to get a lot of replies.



On Tue, Aug 2, 2016 at 4:36 PM, Grégory MARCASSIN - Magic
 wrote:

Hi,
Sorry to ask ...

Can you send me a url or a conf for the support EXIM4 ?


Thx a lot

Best regards
--







--
130-134 Avenue du Président Wilson
93512 Montreuil Cedex
www.magic.fr 

Grégory MARCASSIN
Administrateur Système
Tél : +33 (0)1 41 58 2281
Fax : +33 (0)1 56 72 93 30 
SAV : +33 (0)1 41 58 22 50 

Re: Help

[Jeff Palmer]

What specifically are you having issues with,  and what are the
configurations you have already tried?


Just asking someone to do your job for you isn't likely to get a lot of replies.



On Tue, Aug 2, 2016 at 4:36 PM, Grégory MARCASSIN - Magic
 wrote:
> Hi,
> Sorry to ask ...
>
> Can you send me a url or a conf for the support EXIM4 ?
>
>
> Thx a lot
>
> Best regards
> --
>
>



-- 
Jeff Palmer
https://PalmerIT.net

Re: Help! HAProxy randomly failing health checks!

[Zachary Punches]

Hey guys, just following up. Still running into the issue.


From: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Date: Friday, March 18, 2016 at 6:07 PM
To: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!

Ok! Here is a bunch of info that might better assist with the issue:


Each of our clients has an HAProxy install that forwards requests for 80 and 
443 to 1025 and 1026 respectively. These requests are forwarded over TCP using 
proxy protocol to our HAP instances.
Our HAP instances then SSL term the request and forward them off to our backend 
on port 80.

See attached diagram which should better explain the entire flow.

During an outage due to the SSL handshakes failing, I was running TCPDump so I 
could look through and discover what was causing the failure, I was able to 
discover that we are receiving connection resets on some SSL connections. We 
then tested all the SSL certs from our client side to our side to verify that 
there is not a mismatched cert. This test was completed with no issues.

Here is a connection reset packet I found in that TCP Dump

29525 158.09621710.1.4.11954.239.21.251TCP5438740 → 443 [RST] Seq=3533 Win=0 
Len=0

Frame 29523: 54 bytes on wire (432 bits), 54 bytes captured (432 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Mar 17, 2016 14:58:07.34584 PDT
[Time shift for this packet: 0.0 seconds]
Epoch Time: 1458251887.34584 seconds
[Time delta from previous captured frame: 0.2 seconds]
[Time delta from previous displayed frame: 0.021655000 seconds]
[Time since reference or first frame: 158.096184000 seconds]
Frame Number: 29523
Frame Length: 54 bytes (432 bits)
Capture Length: 54 bytes (432 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:tcp]
[Coloring Rule Name: TCP RST]
[Coloring Rule String: tcp.flags.reset eq 1]
Ethernet II, Src: 12:8d:18:05:0f:91 (12:8d:18:05:0f:91), Dst: 1e:8f:a6:6b:52:58 
(1e:8f:a6:6b:52:58)
Destination: 1e:8f:a6:6b:52:58 (1e:8f:a6:6b:52:58)
Address: 1e:8f:a6:6b:52:58 (1e:8f:a6:6b:52:58)
 ..1.     = LG bit: Locally administered address 
(this is NOT the factory default)
 ...0     = IG bit: Individual address (unicast)
Source: 12:8d:18:05:0f:91 (12:8d:18:05:0f:91)
Address: 12:8d:18:05:0f:91 (12:8d:18:05:0f:91)
 ..1.     = LG bit: Locally administered address 
(this is NOT the factory default)
 ...0     = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: $SRC IP Dst: $DST IP
0100  = Version: 4
 0101 = Header Length: 20 bytes
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
 00.. = Differentiated Services Codepoint: Default (0)
 ..00 = Explicit Congestion Notification: Not ECN-Capable Transport 
(0)
Total Length: 40
Identification: 0x5f56 (24406)
Flags: 0x02 (Don't Fragment)
0...  = Reserved bit: Not set
.1..  = Don't fragment: Set
..0.  = More fragments: Not set
Fragment offset: 0
Time to live: 64
Protocol: TCP (6)
Header checksum: 0x8018 [validation disabled]
[Good: False]
[Bad: False]
Source: $SourceIP
Destination: $DestinationIP
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
Transmission Control Protocol, Src Port: 38740 (38740), Dst Port: 443 (443), 
Seq: 3533, Len: 0
Source Port: 38740
Destination Port: 443
[Stream index: 2799]
[TCP Segment Len: 0]
Sequence number: 3533(relative sequence number)
Acknowledgment number: 0
Header Length: 20 bytes
Flags: 0x004 (RST)
000.   = Reserved: Not set
...0   = Nonce: Not set
 0...  = Congestion Window Reduced (CWR): Not set
 .0..  = ECN-Echo: Not set
 ..0.  = Urgent: Not set
 ...0  = Acknowledgment: Not set
  0... = Push: Not set
  .1.. = Reset: Set
[Expert Info (Warn/Sequence): Connection reset (RST)]
[Connection reset (RST)]
[Severity level: Warn]
[Group: Sequence]
  ..0. = Syn: Not set
  ...0 = Fin: Not set
[TCP Flags: *R**]
Window size value: 0
[Calculated window size: 0]
[Window size scaling factor: 128]
Checksum: 0x5c2f [validation disabled]
[Good Checksum: False]
[Bad Che

Re: Help! HAProxy randomly failing health checks!

[Zachary Punches]

hake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60376 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 68.116.153.225:57824 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60365 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60364 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60362 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:37490 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 108.59.8.48:43566 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:59763 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:59760 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60319 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60299 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60293 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60292 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60284 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60282 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:38664 
[17/Mar/2016:18:37:45.736] shared_incoming/2: SSL handshake failure
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60270 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:33270 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:33273 
[17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL 
handshake
Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60089 
[17/Mar/2016:18:37:06.938] shared_incoming~ shared_incoming/ 
-1/-1/-1/-1/0 400 187 - - CR-- 314/314/0/0/0 0/0 ""
Mar 17 18:37:45 localhost haproxy[28703]: 109.154.74.227:53964 
[17/Mar/2016:18:37:06.938] shared_incoming shared_incoming/ 
-1/-1/-1/-1/0 400 0 - - CR-- 313/313/0/0/0 0/0 ""
Mar 17 18:37:45 localhost haproxy[28703]: 66.87.151.25:3325 
[17/Mar/2016:18:37:06.938] shared_incoming shared_incoming/ 
-1/-1/-1/-1/0 400 0 - - CR-- 312/312/0/0/0 0/0 ""
Mar 17 18:37:45 localhost haproxy[28703]: 108.59.8.48:33611 
[17/Mar/2016:18:36:55.938] shared_incoming provedmedia/provedmedia_http 
279/0/0/-1/279 -1 0 - - CD-- 311/311/91/91/0 0/0 "GET 
/?a=61=22008=7346_0_1=1_0_0_0_0_2102824_0_571_61811_0_0 HTTP/1.1"

From: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Date: Wednesday, March 16, 2016 at 5:01 PM
To: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!



On Thu, Mar 17, 2016 at 10:55 AM, Zachary Punches 
<zpunc...@getcake.com<mailto:zpunc...@getcake.com>> wrote:
Thanks for the reply!

Ok so based on what you saw in my config, does it look like we’re misconfigured 
enough to cause this to happen?

If we were misconfigured, one would assume we would go down all the time yeah?

From: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Date: Wednesday, March 16, 2016 at 4:50 PM
To: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!



On Thu, Mar 17, 2016 at 10:47 AM, Igor Ci

Re: Help! HAProxy randomly failing health checks!

[Igor Cicimov]

On Thu, Mar 17, 2016 at 10:47 AM, Igor Cicimov <
ig...@encompasscorporation.com> wrote:

>
>
> On Thu, Mar 17, 2016 at 5:29 AM, Zachary Punches 
> wrote:
>
>> I’m not, these guys aren’t sitting behind an ELB. They sit behind route53
>> routing. If one of the proxy boxes fails 3 checks in 30 seconds (with 4
>> checks done a second) then Route53 changes its routing from the first proxy
>> box to the second
>>
>>
>>
>>
>> On 3/15/16, 9:46 PM, "Baptiste"  wrote:
>>
>> >Maybe you're checking a third party VM :)
>> >
>>
>
> AFAIK the Route53 health checks come from different points around the
> globe and it is possible that at some time of the day AWS has scheduled
> some specific end points to perform the HC. And it is possible that those
> ones have different SSL settings from the ones performing the HC during
> your day time. I would suggest you bring up this issue with AWS support,
> let them know your SSL cypher settings in HAP and ask if they are
> compatible with ALL their servers performing SSL health checks.
>
> I personally haven't seen any issues with failed SSL handshakes coming
> from AWS servers and have HAP's running in AU and UK regions.
>
> Igor
>

That is if you are absolutely sure that the failed handshakes are not
caused by overload or misconfigured (system) settings on HAP

Re: Help! HAProxy randomly failing health checks!

[Igor Cicimov]

On Thu, Mar 17, 2016 at 10:55 AM, Zachary Punches <zpunc...@getcake.com>
wrote:

> Thanks for the reply!
>
> Ok so based on what you saw in my config, does it look like we’re
> misconfigured enough to cause this to happen?
>
> If we were misconfigured, one would assume we would go down all the time
> yeah?
>
> From: Igor Cicimov <ig...@encompasscorporation.com>
> Date: Wednesday, March 16, 2016 at 4:50 PM
> To: Zachary Punches <zpunc...@getcake.com>
> Cc: Baptiste <bed...@gmail.com>, "haproxy@formilux.org" <
> haproxy@formilux.org>
> Subject: Re: Help! HAProxy randomly failing health checks!
>
>
>
> On Thu, Mar 17, 2016 at 10:47 AM, Igor Cicimov <
> ig...@encompasscorporation.com> wrote:
>
>>
>>
>> On Thu, Mar 17, 2016 at 5:29 AM, Zachary Punches <zpunc...@getcake.com>
>> wrote:
>>
>>> I’m not, these guys aren’t sitting behind an ELB. They sit behind
>>> route53 routing. If one of the proxy boxes fails 3 checks in 30 seconds
>>> (with 4 checks done a second) then Route53 changes its routing from the
>>> first proxy box to the second
>>>
>>>
>>>
>>>
>>> On 3/15/16, 9:46 PM, "Baptiste" <bed...@gmail.com> wrote:
>>>
>>> >Maybe you're checking a third party VM :)
>>> >
>>>
>>
>> AFAIK the Route53 health checks come from different points around the
>> globe and it is possible that at some time of the day AWS has scheduled
>> some specific end points to perform the HC. And it is possible that those
>> ones have different SSL settings from the ones performing the HC during
>> your day time. I would suggest you bring up this issue with AWS support,
>> let them know your SSL cypher settings in HAP and ask if they are
>> compatible with ALL their servers performing SSL health checks.
>>
>> I personally haven't seen any issues with failed SSL handshakes coming
>> from AWS servers and have HAP's running in AU and UK regions.
>>
>> Igor
>>
>
> That is if you are absolutely sure that the failed handshakes are not
> caused by overload or misconfigured (system) settings on HAP
>
>
I was saying this in regards to system (kernel) settings. For example,
assuming Unix/Linux is your net.core.somaxconn actually set *higher* than
your maxconn which is set to 3 and 15000 in HAP? Any other kernel
settings you might have changed? (output of "sysctl -p" command)

What is your pick hour load, how many connections/sessions are you seeing
on each HAP?

Another suggestion is maybe set tune.ssl.default-dh-param to 1024 and see
if that helps.

Re: Help! HAProxy randomly failing health checks!

[Zachary Punches]

I went ahead and added the performance tuning you recommended (changing the 
maxconn to 1024). Hopefully this adds some stability

As for the port, we’re using 1027 for our SSL traffic vs 443. We are currently 
getting SSL traffic that isn’t always failing on handshake.

As for what is in front of our HAP:
Our clients have HAP servers that are forwarding requests off to our HAP setup

From: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Date: Wednesday, March 16, 2016 at 6:51 PM
To: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!



On Thu, Mar 17, 2016 at 12:46 PM, Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>> wrote:


On Thu, Mar 17, 2016 at 11:14 AM, Zachary Punches 
<zpunc...@getcake.com<mailto:zpunc...@getcake.com>> wrote:
I wanna say average is like 4-6 connections a second? Super minimal

From what I’ve seen in the logs during the SSL errors, the log hangs then 
outputs a bunch of SSL errors all at once.

Here it the output from sysctl –p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key
error: "net.bridge.bridge-nf-call-iptables" is an unknown key
error: "net.bridge.bridge-nf-call-arptables" is an unknown key
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296

What would lowering the tune.ssl.default-dh-param to 1024 do?
From: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Date: Wednesday, March 16, 2016 at 5:01 PM
To: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!



On Thu, Mar 17, 2016 at 10:55 AM, Zachary Punches 
<zpunc...@getcake.com<mailto:zpunc...@getcake.com>> wrote:
Thanks for the reply!

Ok so based on what you saw in my config, does it look like we’re misconfigured 
enough to cause this to happen?

If we were misconfigured, one would assume we would go down all the time yeah?

From: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Date: Wednesday, March 16, 2016 at 4:50 PM
To: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!



On Thu, Mar 17, 2016 at 10:47 AM, Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>> wrote:


On Thu, Mar 17, 2016 at 5:29 AM, Zachary Punches 
<zpunc...@getcake.com<mailto:zpunc...@getcake.com>> wrote:
I’m not, these guys aren’t sitting behind an ELB. They sit behind route53 
routing. If one of the proxy boxes fails 3 checks in 30 seconds (with 4 checks 
done a second) then Route53 changes its routing from the first proxy box to the 
second




On 3/15/16, 9:46 PM, "Baptiste" <bed...@gmail.com<mailto:bed...@gmail.com>> 
wrote:

>Maybe you're checking a third party VM :)
>

AFAIK the Route53 health checks come from different points around the globe and 
it is possible that at some time of the day AWS has scheduled some specific end 
points to perform the HC. And it is possible that those ones have different SSL 
settings from the ones performing the HC during your day time. I would suggest 
you bring up this issue with AWS support, let them know your SSL cypher 
settings in HAP and ask if they are compatible with ALL their servers 
performing SSL health checks.

I personally haven't seen any issues with failed SSL handshakes coming from AWS 
servers and have HAP's running in AU and UK regions.

Igor

That is if you are absolutely sure that the failed handshakes are not caused by 
overload or misconfigured (system) settings on HAP


I was saying this in regards to system (kernel) settings. For example, assuming 
Unix/Linux is your net.core.somaxconn actually set *higher* than your maxconn 
which is set to 3 and 15000 in HAP? Any other kernel settings you might 
have changed? (output of "sysctl -p" com

Re: Help! HAProxy randomly failing health checks!

[Igor Cicimov]

ng/2: Connection closed during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:59222
> [17/Mar/2016:18:37:45.736] shared_incoming/2: SSL handshake failure
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60388
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60379
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60376
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 68.116.153.225:57824
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60365
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60364
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60362
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:37490
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 108.59.8.48:43566
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:59763
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:59760
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60319
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60299
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60293
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60292
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60284
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60282
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:38664
> [17/Mar/2016:18:37:45.736] shared_incoming/2: SSL handshake failure
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60270
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:33270
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 208.100.26.237:33273
> [17/Mar/2016:18:37:45.736] shared_incoming/2: Connection error during SSL
> handshake
> Mar 17 18:37:45 localhost haproxy[28703]: 54.218.28.138:60089
> [17/Mar/2016:18:37:06.938] shared_incoming~ shared_incoming/
> -1/-1/-1/-1/0 400 187 - - CR-- 314/314/0/0/0 0/0 ""
> Mar 17 18:37:45 localhost haproxy[28703]: 109.154.74.227:53964
> [17/Mar/2016:18:37:06.938] shared_incoming shared_incoming/
> -1/-1/-1/-1/0 400 0 - - CR-- 313/313/0/0/0 0/0 ""
> Mar 17 18:37:45 localhost haproxy[28703]: 66.87.151.25:3325
> [17/Mar/2016:18:37:06.938] shared_incoming shared_incoming/
> -1/-1/-1/-1/0 400 0 - - CR-- 312/312/0/0/0 0/0 ""
> Mar 17 18:37:45 localhost haproxy[28703]: 108.59.8.48:33611
> [17/Mar/2016:18:36:55.938] shared_incoming provedmedia/provedmedia_http
> 279/0/0/-1/279 -1 0 - - CD-- 311/311/91/91/0 0/0 "GET
> /?a=61=22008=7346_0_1=1_0_0_0_0_2102824_0_571_61811_0_0 HTTP/1.1"
>
> From: Igor Cicimov <ig...@encompasscorporation.com>
> Date: Wednesday, March 16, 2016 at 5:01 PM
> To: Zachary Punches <zpunc...@getcake.com>
> Cc: Baptiste <bed...@gmail.com>, "haproxy@formilux.org" <
> haproxy@formilux.org>
> Subject: Re: Help! HAProxy randomly failing health checks!
>
>
>
> On Thu, Mar 17, 2016 at 10:55 AM, Zachary Punches <zpunc...@getcake.com>
> wrote:
>
>> Thanks for the reply!
>>
>> Ok so based on what you saw in my config, does it loo

Re: Help! HAProxy randomly failing health checks!

[Igor Cicimov]

On Thu, Mar 17, 2016 at 12:46 PM, Igor Cicimov <
ig...@encompasscorporation.com> wrote:

>
>
> On Thu, Mar 17, 2016 at 11:14 AM, Zachary Punches <zpunc...@getcake.com>
> wrote:
>
>> I wanna say average is like 4-6 connections a second? Super minimal
>>
>> From what I’ve seen in the logs during the SSL errors, the log hangs then
>> outputs a bunch of SSL errors all at once.
>>
>> Here it the output from sysctl –p
>> net.ipv4.ip_forward = 0
>> net.ipv4.conf.default.rp_filter = 1
>> net.ipv4.conf.default.accept_source_route = 0
>> kernel.sysrq = 0
>> kernel.core_uses_pid = 1
>> net.ipv4.tcp_syncookies = 1
>> error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key
>> error: "net.bridge.bridge-nf-call-iptables" is an unknown key
>> error: "net.bridge.bridge-nf-call-arptables" is an unknown key
>> kernel.msgmnb = 65536
>> kernel.msgmax = 65536
>> kernel.shmmax = 68719476736
>> kernel.shmall = 4294967296
>>
>> What would lowering the tune.ssl.default-dh-param to 1024 do?
>> From: Igor Cicimov <ig...@encompasscorporation.com>
>> Date: Wednesday, March 16, 2016 at 5:01 PM
>> To: Zachary Punches <zpunc...@getcake.com>
>> Cc: Baptiste <bed...@gmail.com>, "haproxy@formilux.org" <
>> haproxy@formilux.org>
>> Subject: Re: Help! HAProxy randomly failing health checks!
>>
>>
>>
>> On Thu, Mar 17, 2016 at 10:55 AM, Zachary Punches <zpunc...@getcake.com>
>> wrote:
>>
>>> Thanks for the reply!
>>>
>>> Ok so based on what you saw in my config, does it look like we’re
>>> misconfigured enough to cause this to happen?
>>>
>>> If we were misconfigured, one would assume we would go down all the time
>>> yeah?
>>>
>>> From: Igor Cicimov <ig...@encompasscorporation.com>
>>> Date: Wednesday, March 16, 2016 at 4:50 PM
>>> To: Zachary Punches <zpunc...@getcake.com>
>>> Cc: Baptiste <bed...@gmail.com>, "haproxy@formilux.org" <
>>> haproxy@formilux.org>
>>> Subject: Re: Help! HAProxy randomly failing health checks!
>>>
>>>
>>>
>>> On Thu, Mar 17, 2016 at 10:47 AM, Igor Cicimov <
>>> ig...@encompasscorporation.com> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Mar 17, 2016 at 5:29 AM, Zachary Punches <zpunc...@getcake.com>
>>>> wrote:
>>>>
>>>>> I’m not, these guys aren’t sitting behind an ELB. They sit behind
>>>>> route53 routing. If one of the proxy boxes fails 3 checks in 30 seconds
>>>>> (with 4 checks done a second) then Route53 changes its routing from the
>>>>> first proxy box to the second
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 3/15/16, 9:46 PM, "Baptiste" <bed...@gmail.com> wrote:
>>>>>
>>>>> >Maybe you're checking a third party VM :)
>>>>> >
>>>>>
>>>>
>>>> AFAIK the Route53 health checks come from different points around the
>>>> globe and it is possible that at some time of the day AWS has scheduled
>>>> some specific end points to perform the HC. And it is possible that those
>>>> ones have different SSL settings from the ones performing the HC during
>>>> your day time. I would suggest you bring up this issue with AWS support,
>>>> let them know your SSL cypher settings in HAP and ask if they are
>>>> compatible with ALL their servers performing SSL health checks.
>>>>
>>>> I personally haven't seen any issues with failed SSL handshakes coming
>>>> from AWS servers and have HAP's running in AU and UK regions.
>>>>
>>>> Igor
>>>>
>>>
>>> That is if you are absolutely sure that the failed handshakes are not
>>> caused by overload or misconfigured (system) settings on HAP
>>>
>>>
>> I was saying this in regards to system (kernel) settings. For example,
>> assuming Unix/Linux is your net.core.somaxconn actually set *higher* than
>> your maxconn which is set to 3 and 15000 in HAP? Any other kernel
>> settings you might have changed? (output of "sysctl -p" command)
>>
>> What is your pick hour load, how many connections/sessions are you seeing
>> on each HAP?
>>
>> Another suggestion is maybe set tune.ssl.default-dh-param to 1024 and see
>> if that helps.
>>
>
>
> Ok, so on default ubuntu cloud instance this is what we have:
>
> # sysctl -a | grep maxconn
> net.core.somaxconn = 128
>
> which is too low for production server. Check your value and adjust it to
> your needs.
>
> By the way, what is accept-proxy doing there in your setup? Is there
> anything else in front of HAP using PROXY protocol?
>
>
Wait a minute:

bind *:1027 # Health checking port

are you maybe using https health check on a non SSL port?

Re: Help! HAProxy randomly failing health checks!

[Igor Cicimov]

On Fri, Mar 18, 2016 at 1:38 PM, Igor Cicimov <
ig...@encompasscorporation.com> wrote:

>
>
> On Fri, Mar 18, 2016 at 12:04 PM, Zachary Punches <zpunc...@getcake.com>
> wrote:
>
>> Yeah port 1027 is used for health checks over SSL.
>>
>> This HAP forwards requests off to our databases. The databases have a
>> string in a table that indicates that the HAP instance can move all the way
>> through the entire process before it lights as green.
>>
>> Our health checks in route 53 are setup to ping 1027 as the SSL port
>>
>> From: Igor Cicimov <ig...@encompasscorporation.com>
>> Date: Thursday, March 17, 2016 at 4:18 PM
>> To: Zachary Punches <zpunc...@getcake.com>
>> Cc: Baptiste <bed...@gmail.com>, "haproxy@formilux.org" <
>> haproxy@formilux.org>
>> Subject: Re: Help! HAProxy randomly failing health checks!
>>
>> So is port 1027 used for health checks over SSL or not? I don't see any
>> ssl settings on that port.
>>
>
>
> I see. In that case, since you are doing ssl pass-through in http mode,
> you need to add ssl to your server line:
>
> backend server0  ## added to allow gs ssl meta tag verification
> reqrep ^GET\ /.*\ (HTTP/.*)GET\ /GlobalSignVerification\ \1
> server server0_http server0.domain.com:80/GlobalSignVerification/
>
> so it becomes:
>
> server server0_http server0.domain.com:80/GlobalSignVerification/ ssl
>
> if I understood your intention correctly and server0 is ssl enabled db.
>

# Route traffic based on domain
use_backend gs_verify if gs_texthtml or gs_user_agent## allow gs
meta tag verification

Where is this backend in your config? Which domain is used fore the health
checks?

Re: Help! HAProxy randomly failing health checks!

[Zachary Punches]

Yeah port 1027 is used for health checks over SSL.

This HAP forwards requests off to our databases. The databases have a string in 
a table that indicates that the HAP instance can move all the way through the 
entire process before it lights as green.

Our health checks in route 53 are setup to ping 1027 as the SSL port

From: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Date: Thursday, March 17, 2016 at 4:18 PM
To: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!

So is port 1027 used for health checks over SSL or not? I don't see any ssl 
settings on that port.

Re: Help! HAProxy randomly failing health checks!

[Zachary Punches]

I’m not, these guys aren’t sitting behind an ELB. They sit behind route53 
routing. If one of the proxy boxes fails 3 checks in 30 seconds (with 4 checks 
done a second) then Route53 changes its routing from the first proxy box to the 
second




On 3/15/16, 9:46 PM, "Baptiste"  wrote:

>Maybe you're checking a third party VM :)
>

Re: Help! HAProxy randomly failing health checks!

[Igor Cicimov]

On Fri, Mar 18, 2016 at 12:04 PM, Zachary Punches <zpunc...@getcake.com>
wrote:

> Yeah port 1027 is used for health checks over SSL.
>
> This HAP forwards requests off to our databases. The databases have a
> string in a table that indicates that the HAP instance can move all the way
> through the entire process before it lights as green.
>
> Our health checks in route 53 are setup to ping 1027 as the SSL port
>
> From: Igor Cicimov <ig...@encompasscorporation.com>
> Date: Thursday, March 17, 2016 at 4:18 PM
> To: Zachary Punches <zpunc...@getcake.com>
> Cc: Baptiste <bed...@gmail.com>, "haproxy@formilux.org" <
> haproxy@formilux.org>
> Subject: Re: Help! HAProxy randomly failing health checks!
>
> So is port 1027 used for health checks over SSL or not? I don't see any
> ssl settings on that port.
>


I see. In that case, since you are doing ssl pass-through in http mode, you
need to add ssl to your server line:

backend server0  ## added to allow gs ssl meta tag verification
reqrep ^GET\ /.*\ (HTTP/.*)GET\ /GlobalSignVerification\ \1
server server0_http server0.domain.com:80/GlobalSignVerification/

so it becomes:

server server0_http server0.domain.com:80/GlobalSignVerification/ ssl

if I understood your intention correctly and server0 is ssl enabled db.

Re: Help! HAProxy randomly failing health checks!

[Zachary Punches]

I wanna say average is like 4-6 connections a second? Super minimal

From what I’ve seen in the logs during the SSL errors, the log hangs then 
outputs a bunch of SSL errors all at once.

Here it the output from sysctl –p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key
error: "net.bridge.bridge-nf-call-iptables" is an unknown key
error: "net.bridge.bridge-nf-call-arptables" is an unknown key
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296

What would lowering the tune.ssl.default-dh-param to 1024 do?
From: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Date: Wednesday, March 16, 2016 at 5:01 PM
To: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!



On Thu, Mar 17, 2016 at 10:55 AM, Zachary Punches 
<zpunc...@getcake.com<mailto:zpunc...@getcake.com>> wrote:
Thanks for the reply!

Ok so based on what you saw in my config, does it look like we’re misconfigured 
enough to cause this to happen?

If we were misconfigured, one would assume we would go down all the time yeah?

From: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Date: Wednesday, March 16, 2016 at 4:50 PM
To: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!



On Thu, Mar 17, 2016 at 10:47 AM, Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>> wrote:


On Thu, Mar 17, 2016 at 5:29 AM, Zachary Punches 
<zpunc...@getcake.com<mailto:zpunc...@getcake.com>> wrote:
I’m not, these guys aren’t sitting behind an ELB. They sit behind route53 
routing. If one of the proxy boxes fails 3 checks in 30 seconds (with 4 checks 
done a second) then Route53 changes its routing from the first proxy box to the 
second




On 3/15/16, 9:46 PM, "Baptiste" <bed...@gmail.com<mailto:bed...@gmail.com>> 
wrote:

>Maybe you're checking a third party VM :)
>

AFAIK the Route53 health checks come from different points around the globe and 
it is possible that at some time of the day AWS has scheduled some specific end 
points to perform the HC. And it is possible that those ones have different SSL 
settings from the ones performing the HC during your day time. I would suggest 
you bring up this issue with AWS support, let them know your SSL cypher 
settings in HAP and ask if they are compatible with ALL their servers 
performing SSL health checks.

I personally haven't seen any issues with failed SSL handshakes coming from AWS 
servers and have HAP's running in AU and UK regions.

Igor

That is if you are absolutely sure that the failed handshakes are not caused by 
overload or misconfigured (system) settings on HAP


I was saying this in regards to system (kernel) settings. For example, assuming 
Unix/Linux is your net.core.somaxconn actually set *higher* than your maxconn 
which is set to 3 and 15000 in HAP? Any other kernel settings you might 
have changed? (output of "sysctl -p" command)

What is your pick hour load, how many connections/sessions are you seeing on 
each HAP?

Another suggestion is maybe set tune.ssl.default-dh-param to 1024 and see if 
that helps.

Re: Help! HAProxy randomly failing health checks!

[Zachary Punches]

Thanks for the reply!

Ok so based on what you saw in my config, does it look like we’re misconfigured 
enough to cause this to happen?

If we were misconfigured, one would assume we would go down all the time yeah?

From: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Date: Wednesday, March 16, 2016 at 4:50 PM
To: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!



On Thu, Mar 17, 2016 at 10:47 AM, Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>> wrote:


On Thu, Mar 17, 2016 at 5:29 AM, Zachary Punches 
<zpunc...@getcake.com<mailto:zpunc...@getcake.com>> wrote:
I’m not, these guys aren’t sitting behind an ELB. They sit behind route53 
routing. If one of the proxy boxes fails 3 checks in 30 seconds (with 4 checks 
done a second) then Route53 changes its routing from the first proxy box to the 
second




On 3/15/16, 9:46 PM, "Baptiste" <bed...@gmail.com<mailto:bed...@gmail.com>> 
wrote:

>Maybe you're checking a third party VM :)
>

AFAIK the Route53 health checks come from different points around the globe and 
it is possible that at some time of the day AWS has scheduled some specific end 
points to perform the HC. And it is possible that those ones have different SSL 
settings from the ones performing the HC during your day time. I would suggest 
you bring up this issue with AWS support, let them know your SSL cypher 
settings in HAP and ask if they are compatible with ALL their servers 
performing SSL health checks.

I personally haven't seen any issues with failed SSL handshakes coming from AWS 
servers and have HAP's running in AU and UK regions.

Igor

That is if you are absolutely sure that the failed handshakes are not caused by 
overload or misconfigured (system) settings on HAP

Re: Help! HAProxy randomly failing health checks!

[Igor Cicimov]

On Thu, Mar 17, 2016 at 11:14 AM, Zachary Punches <zpunc...@getcake.com>
wrote:

> I wanna say average is like 4-6 connections a second? Super minimal
>
> From what I’ve seen in the logs during the SSL errors, the log hangs then
> outputs a bunch of SSL errors all at once.
>
> Here it the output from sysctl –p
> net.ipv4.ip_forward = 0
> net.ipv4.conf.default.rp_filter = 1
> net.ipv4.conf.default.accept_source_route = 0
> kernel.sysrq = 0
> kernel.core_uses_pid = 1
> net.ipv4.tcp_syncookies = 1
> error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key
> error: "net.bridge.bridge-nf-call-iptables" is an unknown key
> error: "net.bridge.bridge-nf-call-arptables" is an unknown key
> kernel.msgmnb = 65536
> kernel.msgmax = 65536
> kernel.shmmax = 68719476736
> kernel.shmall = 4294967296
>
> What would lowering the tune.ssl.default-dh-param to 1024 do?
> From: Igor Cicimov <ig...@encompasscorporation.com>
> Date: Wednesday, March 16, 2016 at 5:01 PM
> To: Zachary Punches <zpunc...@getcake.com>
> Cc: Baptiste <bed...@gmail.com>, "haproxy@formilux.org" <
> haproxy@formilux.org>
> Subject: Re: Help! HAProxy randomly failing health checks!
>
>
>
> On Thu, Mar 17, 2016 at 10:55 AM, Zachary Punches <zpunc...@getcake.com>
> wrote:
>
>> Thanks for the reply!
>>
>> Ok so based on what you saw in my config, does it look like we’re
>> misconfigured enough to cause this to happen?
>>
>> If we were misconfigured, one would assume we would go down all the time
>> yeah?
>>
>> From: Igor Cicimov <ig...@encompasscorporation.com>
>> Date: Wednesday, March 16, 2016 at 4:50 PM
>> To: Zachary Punches <zpunc...@getcake.com>
>> Cc: Baptiste <bed...@gmail.com>, "haproxy@formilux.org" <
>> haproxy@formilux.org>
>> Subject: Re: Help! HAProxy randomly failing health checks!
>>
>>
>>
>> On Thu, Mar 17, 2016 at 10:47 AM, Igor Cicimov <
>> ig...@encompasscorporation.com> wrote:
>>
>>>
>>>
>>> On Thu, Mar 17, 2016 at 5:29 AM, Zachary Punches <zpunc...@getcake.com>
>>> wrote:
>>>
>>>> I’m not, these guys aren’t sitting behind an ELB. They sit behind
>>>> route53 routing. If one of the proxy boxes fails 3 checks in 30 seconds
>>>> (with 4 checks done a second) then Route53 changes its routing from the
>>>> first proxy box to the second
>>>>
>>>>
>>>>
>>>>
>>>> On 3/15/16, 9:46 PM, "Baptiste" <bed...@gmail.com> wrote:
>>>>
>>>> >Maybe you're checking a third party VM :)
>>>> >
>>>>
>>>
>>> AFAIK the Route53 health checks come from different points around the
>>> globe and it is possible that at some time of the day AWS has scheduled
>>> some specific end points to perform the HC. And it is possible that those
>>> ones have different SSL settings from the ones performing the HC during
>>> your day time. I would suggest you bring up this issue with AWS support,
>>> let them know your SSL cypher settings in HAP and ask if they are
>>> compatible with ALL their servers performing SSL health checks.
>>>
>>> I personally haven't seen any issues with failed SSL handshakes coming
>>> from AWS servers and have HAP's running in AU and UK regions.
>>>
>>> Igor
>>>
>>
>> That is if you are absolutely sure that the failed handshakes are not
>> caused by overload or misconfigured (system) settings on HAP
>>
>>
> I was saying this in regards to system (kernel) settings. For example,
> assuming Unix/Linux is your net.core.somaxconn actually set *higher* than
> your maxconn which is set to 3 and 15000 in HAP? Any other kernel
> settings you might have changed? (output of "sysctl -p" command)
>
> What is your pick hour load, how many connections/sessions are you seeing
> on each HAP?
>
> Another suggestion is maybe set tune.ssl.default-dh-param to 1024 and see
> if that helps.
>


Ok, so on default ubuntu cloud instance this is what we have:

# sysctl -a | grep maxconn
net.core.somaxconn = 128

which is too low for production server. Check your value and adjust it to
your needs.

By the way, what is accept-proxy doing there in your setup? Is there
anything else in front of HAP using PROXY protocol?

Re: Help! HAProxy randomly failing health checks!

[Igor Cicimov]

On Thu, Mar 17, 2016 at 5:29 AM, Zachary Punches 
wrote:

> I’m not, these guys aren’t sitting behind an ELB. They sit behind route53
> routing. If one of the proxy boxes fails 3 checks in 30 seconds (with 4
> checks done a second) then Route53 changes its routing from the first proxy
> box to the second
>
>
>
>
> On 3/15/16, 9:46 PM, "Baptiste"  wrote:
>
> >Maybe you're checking a third party VM :)
> >
>

AFAIK the Route53 health checks come from different points around the globe
and it is possible that at some time of the day AWS has scheduled some
specific end points to perform the HC. And it is possible that those ones
have different SSL settings from the ones performing the HC during your day
time. I would suggest you bring up this issue with AWS support, let them
know your SSL cypher settings in HAP and ask if they are compatible with
ALL their servers performing SSL health checks.

I personally haven't seen any issues with failed SSL handshakes coming from
AWS servers and have HAP's running in AU and UK regions.

Igor

Re: Help! HAProxy randomly failing health checks!

[Zachary Punches]

Ok! Here is a bunch of info that might better assist with the issue:


Each of our clients has an HAProxy install that forwards requests for 80 and 
443 to 1025 and 1026 respectively. These requests are forwarded over TCP using 
proxy protocol to our HAP instances.
Our HAP instances then SSL term the request and forward them off to our backend 
on port 80.

See attached diagram which should better explain the entire flow.

During an outage due to the SSL handshakes failing, I was running TCPDump so I 
could look through and discover what was causing the failure, I was able to 
discover that we are receiving connection resets on some SSL connections. We 
then tested all the SSL certs from our client side to our side to verify that 
there is not a mismatched cert. This test was completed with no issues.

Here is a connection reset packet I found in that TCP Dump

29525 158.09621710.1.4.119 54.239.21.251TCP 5438740 → 443 [RST] Seq=3533 Win=0 
Len=0

Frame 29523: 54 bytes on wire (432 bits), 54 bytes captured (432 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Mar 17, 2016 14:58:07.34584 PDT
[Time shift for this packet: 0.0 seconds]
Epoch Time: 1458251887.34584 seconds
[Time delta from previous captured frame: 0.2 seconds]
[Time delta from previous displayed frame: 0.021655000 seconds]
[Time since reference or first frame: 158.096184000 seconds]
Frame Number: 29523
Frame Length: 54 bytes (432 bits)
Capture Length: 54 bytes (432 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:tcp]
[Coloring Rule Name: TCP RST]
[Coloring Rule String: tcp.flags.reset eq 1]
Ethernet II, Src: 12:8d:18:05:0f:91 (12:8d:18:05:0f:91), Dst: 1e:8f:a6:6b:52:58 
(1e:8f:a6:6b:52:58)
Destination: 1e:8f:a6:6b:52:58 (1e:8f:a6:6b:52:58)
Address: 1e:8f:a6:6b:52:58 (1e:8f:a6:6b:52:58)
 ..1.     = LG bit: Locally administered address 
(this is NOT the factory default)
 ...0     = IG bit: Individual address (unicast)
Source: 12:8d:18:05:0f:91 (12:8d:18:05:0f:91)
Address: 12:8d:18:05:0f:91 (12:8d:18:05:0f:91)
 ..1.     = LG bit: Locally administered address 
(this is NOT the factory default)
 ...0     = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: $SRC IP Dst: $DST IP
0100  = Version: 4
 0101 = Header Length: 20 bytes
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
 00.. = Differentiated Services Codepoint: Default (0)
 ..00 = Explicit Congestion Notification: Not ECN-Capable Transport 
(0)
Total Length: 40
Identification: 0x5f56 (24406)
Flags: 0x02 (Don't Fragment)
0...  = Reserved bit: Not set
.1..  = Don't fragment: Set
..0.  = More fragments: Not set
Fragment offset: 0
Time to live: 64
Protocol: TCP (6)
Header checksum: 0x8018 [validation disabled]
[Good: False]
[Bad: False]
Source: $SourceIP
Destination: $DestinationIP
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
Transmission Control Protocol, Src Port: 38740 (38740), Dst Port: 443 (443), 
Seq: 3533, Len: 0
Source Port: 38740
Destination Port: 443
[Stream index: 2799]
[TCP Segment Len: 0]
Sequence number: 3533(relative sequence number)
Acknowledgment number: 0
Header Length: 20 bytes
Flags: 0x004 (RST)
000.   = Reserved: Not set
...0   = Nonce: Not set
 0...  = Congestion Window Reduced (CWR): Not set
 .0..  = ECN-Echo: Not set
 ..0.  = Urgent: Not set
 ...0  = Acknowledgment: Not set
  0... = Push: Not set
  .1.. = Reset: Set
[Expert Info (Warn/Sequence): Connection reset (RST)]
[Connection reset (RST)]
[Severity level: Warn]
[Group: Sequence]
  ..0. = Syn: Not set
  ...0 = Fin: Not set
[TCP Flags: *R**]
Window size value: 0
[Calculated window size: 0]
[Window size scaling factor: 128]
Checksum: 0x5c2f [validation disabled]
[Good Checksum: False]
[Bad Checksum: False]
Urgent pointer: 0





From: Igor Cicimov 
<ig...@encompasscorporation.com<mailto:ig...@encompasscorporation.com>>
Date: Thursday, March 17, 2016 at 8:40 PM
To: Zachary Punches <zpunc...@getcake.com<mailto:zpunc...@getcake.com>>
Cc: Baptiste <bed...@gmail.com<mailto:bed...@gmail.com>>, 
"haproxy@formilux.org<mailto:haproxy@formilux.org>" 
<haproxy@formilux.org<mailto:haproxy@formilux.org>>
Subject: Re: Help! HAProxy randomly failing health checks!



On Fri, Mar 18, 2016 at 

Re: Help! HAProxy randomly failing health checks!

[Baptiste]

Has ELB changed its IP address???
Maybe you're checking a third party VM :)

Baptiste

Re: Help! HAProxy randomly failing health checks!

[Igor Cicimov]

On Wed, Mar 16, 2016 at 5:54 AM, Zachary Punches 
wrote:

> Hello!
>
>
>
> My name is Zack, and I have been in the middle of an on going HAProxy
> issue that has me scratching my head.
>
>
>
> Here is the setup:
>
>
>
> Our setup is hosted by amazon, and our HAProxy (1.6.3) boxes are in each
> region in 3 regions. We have 2 HAProxy boxes per region for a total of 6
> proxy boxes.
>
>
>
> These boxes are routed information through route 53. Their entire job is
> to forward data from one of our clients to our database backend. It handles
> this absolutely fine, except between the hours of 7pm PST and 7am PST.
> During these hours, our route53 health checks time out thus causing the
> traffic to switch to the other HAProxy box inside of the same region.
>
>
>
> During the other 12 hours of the day, we receive 0 alerts from our health
> checks.
>
>
>
> I have noticed that we get a series of SSL handshake failures (though this
> happens throughout the entire day) that causes the server to hang for a
> second, thus causing the health checks to fail. During the day our SSL
> failures do not cause the server to hang long enough to go fail the checks,
> they only fail at night. I have attached my HAProxy config hoping that you
> guys have an answer for me. Lemme know if you need any more info.
>
>
>
> I have done a few tcpdump captures during the SSL handshake failures (not
> at night during it failing, but during the day when it still gets the SSL
> handshake failures, but doesn’t fail the health check) and it seems there
> is a d/c and a reconnect during the handshake.
>
>
>
> Here is my config, I will be running a tcpdump tonight to capture the
> packets during the failure and will attach it if you guys need more info.
>
>
>
>  #-
>
> # Example configuration for a possible web application.  See the
>
> # full configuration options online.
>
> #
>
> #   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
>
> #
>
> #-
>
>
>
> #-
>
> # Global settings
>
> #-
>
> global
>
> log 127.0.0.1 local2
>
>
>
> pidfile /var/run/haproxy.pid
>
> maxconn 3
>
> userhaproxy
>
> group   haproxy
>
> daemon
>
> ssl-default-bind-options no-sslv3 no-tls-tickets
>
> tune.ssl.default-dh-param 2048
>
>
>
>  # turn on stats unix socket
>
> #stats socket /var/lib/haproxy/stats`
>
>
>
> #-
>
> # common defaults that all the 'listen' and 'backend' sections will
>
> # use if not designated in their block
>
> #-
>
> defaults
>
> modehttp
>
> log global
>
> option  httplog
>
> retries 3
>
> timeout http-request5s
>
> timeout queue   1m
>
> timeout connect 31s
>
> timeout client  31s
>
> timeout server  31s
>
> maxconn 15000
>
>
>
> # Stats
>
> statsenable
>
> stats uri   /haproxy?stats
>
> stats realm  Strictly\ Private
>
> stats auth$StatsUser:$StatsPass
>
>
>
> #-
>
> # main frontend which proxys to the backends
>
> #-
>
>
>
> frontend shared_incoming
>
> maxconn 15000
>
> timeout http-request 5s
>
>
>
> #Bind ports of incoming traffic
>
> bind *:1025 accept-proxy # http
>
> bind *:1026 accept-proxy ssl crt /path/to/default/ssl/cert.pem ssl crt
> /path/to/cert/folder/ # https
>
> bind *:1027 # Health checking port
>
> acl gs_texthtml url_reg \/gstext\.html## allow gs to do meta tag
> verififcation
>
>  acl gs_user_agent hdr_sub(User-Agent) -i globalsign##
> allow gs to do meta tag verififcation
>
>
>
> #  Add headers
>
> http-request set-header $Proxy-Header-Ip %[src]
>
> http-request set-header $Proxy-Header-Proto http if !{ ssl_fc }
>
> http-request set-header $Proxy-Header-Proto https if { ssl_fc }
>
>
>
> # Route traffic based on domain
>
> use_backend gs_verify if gs_texthtml or gs_user_agent## allow gs
> meta tag verification
>
> use_backend
> %[req.hdr(host),lower,map_dom(/path/to/map/file.map,unknown_domain)]
>
>
>
> # Drop unrecognized traffic
>
> default_backend unknown_domain
>
>
>
> #-
>
> # Backends
>
> #-
>
>
>
> backend 

Re: Help! HAProxy randomly failing health checks!

[Chad Lavoie]

Greetings,

On 03/15/2016 02:54 PM, Zachary Punches wrote:


Hello!

My name is Zack, and I have been in the middle of an on going HAProxy 
issue that has me scratching my head.


Here is the setup:

Our setup is hosted by amazon, and our HAProxy (1.6.3) boxes are in 
each region in 3 regions. We have 2 HAProxy boxes per region for a 
total of 6 proxy boxes.


These boxes are routed information through route 53. Their entire job 
is to forward data from one of our clients to our database backend. It 
handles this absolutely fine, except between the hours of 7pm PST and 
7am PST. During these hours, our route53 health checks time out thus 
causing the traffic to switch to the other HAProxy box inside of the 
same region.


During the other 12 hours of the day, we receive 0 alerts from our 
health checks.


I have noticed that we get a series of SSL handshake failures (though 
this happens throughout the entire day) that causes the server to hang 
for a second, thus causing the health checks to fail. During the day 
our SSL failures do not cause the server to hang long enough to go 
fail the checks, they only fail at night. I have attached my HAProxy 
config hoping that you guys have an answer for me. Lemme know if you 
need any more info.


Before thinking about less obvious potential causes, the CPU of the 
instance isn't close to getting capped out during the time in question?
Also, are the connection counts under 15,000 (otherwise I could see it 
ending up with a timeout and trying again)?


- Chad


I have done a few tcpdump captures during the SSL handshake failures 
(not at night during it failing, but during the day when it still gets 
the SSL handshake failures, but doesn’t fail the health check) and it 
seems there is a d/c and a reconnect during the handshake.


Here is my config, I will be running a tcpdump tonight to capture the 
packets during the failure and will attach it if you guys need more info.


#-

# Example configuration for a possible web application.  See the

# full configuration options online.

#

# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt

#

#-

#-

# Global settings

#-

global

log 127.0.0.1 local2

pidfile /var/run/haproxy.pid

maxconn 3

userhaproxy

group   haproxy

daemon

ssl-default-bind-options no-sslv3 no-tls-tickets

tune.ssl.default-dh-param 2048

 # turn on stats unix socket

#stats socket /var/lib/haproxy/stats`

#-

# common defaults that all the 'listen' and 'backend' sections will

# use if not designated in their block

#-

defaults

modehttp

log global

option  httplog

retries 3

timeout http-request5s

timeout queue   1m

timeout connect 31s

timeout client  31s

timeout server  31s

maxconn 15000

# Stats

stats  enable

stats uri  /haproxy?stats

stats realm Strictly\ Private

stats auth  $StatsUser:$StatsPass

#-

# main frontend which proxys to the backends

#-

frontend shared_incoming

maxconn 15000

timeout http-request 5s

#Bind ports of incoming traffic

bind *:1025 accept-proxy # http

bind *:1026 accept-proxy ssl crt /path/to/default/ssl/cert.pem ssl 
crt /path/to/cert/folder/ # https


bind *:1027 # Health checking port

acl gs_texthtml url_reg \/gstext\.html## allow gs to do meta 
tag verififcation


 acl gs_user_agent hdr_sub(User-Agent) -i globalsign## allow gs to 
do meta tag verififcation


#  Add headers

http-request set-header $Proxy-Header-Ip %[src]

http-request set-header $Proxy-Header-Proto http if !{ ssl_fc }

http-request set-header $Proxy-Header-Proto https if { ssl_fc }

# Route traffic based on domain

use_backend gs_verify if gs_texthtml or gs_user_agent## allow 
gs meta tag verification


use_backend 
%[req.hdr(host),lower,map_dom(/path/to/map/file.map,unknown_domain)]


# Drop unrecognized traffic

default_backend unknown_domain

#-

# Backends

#-

backend server0  ## added to allow gs ssl meta tag verification

reqrep ^GET\ /.*\ (HTTP/.*)GET\ /GlobalSignVerification\ \1

server server0_http 

Re: Help haproxy

[Mathieu Sergent]

Yes, i have just the option that's make gone right.
It's just the option as you know.

Regards,

Mathieu

2015-02-05 10:03 GMT+01:00 Yuan Long yuan.l...@chinanetcloud.com:

 Do you have the words option forward for in your config.
 http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#option%20forwardfor

 Can you copy/paste your config (without sensitive info if needed).

 Regards,

 
 Long Wu Yuan 龙 武 缘
 Sr. Linux Engineer 高级工程师
 ChinaNetCloud 云络网络科技(上海)有限公司 | www.ChinaNetCloud.com1238 Xietu Lu, X2
 Space 1-601, Shanghai, China | 中国上海市徐汇区斜土路1238号X2空 间1-601室

 24x7 Support Hotline: +86-400-618-0024 | Office Tel: +86-(21)-6422-1946
 We are hiring! http://careers.chinanetcloud.com  | Customer Portal -
 https://customer-portal.service.chinanetcloud.com/

 


 On Mon, Feb 2, 2015 at 11:45 PM, Sander Klein roe...@roedie.nl wrote:

 On 02.02.2015 16:33, Mathieu Sergent wrote:

 Hi Sander,

 Yes i reloaded the haproxy and my web server too. But no change.
  And i'm not using proxy protocol.

 To give you more precisions, on my web server i used tcpdump functions
 which give me back the header of the requete http. And in this i found
 my client's address.
 But this is really strange that i can do it without the forwardfor.


 The only other thing that I can think of is that your client is behind a
 proxy server which adds the X-Forward-For header for you...

 Or you got something strange in your config...

 Sander

Re: Help haproxy

[Yuan Long]

Maybe that is why the clinet ipaddress is forwarded.
What is your webserver (ngix/apache) software and what is the logformat
configuration in it.

Regards,

Long Wu Yuan 龙 武 缘
Sr. Linux Engineer 高级工程师
ChinaNetCloud 云络网络科技(上海)有限公司 | www.ChinaNetCloud.com1238 Xietu Lu, X2 Space
1-601, Shanghai, China | 中国上海市徐汇区斜土路1238号X2空 间1-601室

24x7 Support Hotline: +86-400-618-0024 | Office Tel: +86-(21)-6422-1946
We are hiring! http://careers.chinanetcloud.com  | Customer Portal -
https://customer-portal.service.chinanetcloud.com/



On Mon, Feb 9, 2015 at 5:28 PM, Mathieu Sergent 
mathieu.sergent...@gmail.com wrote:


 Yes, i have just the option that's make gone right.
 It's just the option as you know.

 Regards,

 Mathieu


 2015-02-05 10:03 GMT+01:00 Yuan Long yuan.l...@chinanetcloud.com:

 Do you have the words option forward for in your config.
 http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#option%20forwardfor

 Can you copy/paste your config (without sensitive info if needed).

 Regards,

 
 Long Wu Yuan 龙 武 缘
 Sr. Linux Engineer 高级工程师
 ChinaNetCloud 云络网络科技(上海)有限公司 | www.ChinaNetCloud.com1238 Xietu Lu, X2
 Space 1-601, Shanghai, China | 中国上海市徐汇区斜土路1238号X2空 间1-601室

 24x7 Support Hotline: +86-400-618-0024 | Office Tel: +86-(21)-6422-1946
 We are hiring! http://careers.chinanetcloud.com  | Customer Portal -
 https://customer-portal.service.chinanetcloud.com/

 


 On Mon, Feb 2, 2015 at 11:45 PM, Sander Klein roe...@roedie.nl wrote:

 On 02.02.2015 16:33, Mathieu Sergent wrote:

 Hi Sander,

 Yes i reloaded the haproxy and my web server too. But no change.
  And i'm not using proxy protocol.

 To give you more precisions, on my web server i used tcpdump functions
 which give me back the header of the requete http. And in this i found
 my client's address.
 But this is really strange that i can do it without the forwardfor.


 The only other thing that I can think of is that your client is behind a
 proxy server which adds the X-Forward-For header for you...

 Or you got something strange in your config...

 Sander

Re: Help haproxy

[Yuan Long]

Do you have the words option forward for in your config.
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#option%20forwardfor

Can you copy/paste your config (without sensitive info if needed).

Regards,

Long Wu Yuan 龙 武 缘
Sr. Linux Engineer 高级工程师
ChinaNetCloud 云络网络科技(上海)有限公司 | www.ChinaNetCloud.com1238 Xietu Lu, X2 Space
1-601, Shanghai, China | 中国上海市徐汇区斜土路1238号X2空 间1-601室

24x7 Support Hotline: +86-400-618-0024 | Office Tel: +86-(21)-6422-1946
We are hiring! http://careers.chinanetcloud.com  | Customer Portal -
https://customer-portal.service.chinanetcloud.com/



On Mon, Feb 2, 2015 at 11:45 PM, Sander Klein roe...@roedie.nl wrote:

 On 02.02.2015 16:33, Mathieu Sergent wrote:

 Hi Sander,

 Yes i reloaded the haproxy and my web server too. But no change.
  And i'm not using proxy protocol.

 To give you more precisions, on my web server i used tcpdump functions
 which give me back the header of the requete http. And in this i found
 my client's address.
 But this is really strange that i can do it without the forwardfor.


 The only other thing that I can think of is that your client is behind a
 proxy server which adds the X-Forward-For header for you...

 Or you got something strange in your config...

 Sander

Re: Help haproxy

[Mathieu Sergent]

2015-02-02 16:45 GMT+01:00 Sander Klein roe...@roedie.nl:

The only other thing that I can think of is that your client is behind a
 proxy server which adds the X-Forward-For header for you...

 Or you got something strange in your config...

 Sander


You're totally right. My client have a proxy server.
I feel very sorry for this stupid mistake.


Thanks for your time and your help.

Regards,

Mathieu

Re: Help haproxy

[Sander Klein]

On 02.02.2015 12:09, Mathieu Sergent wrote:

Hi,

I try to set up a load balancing with HAProxy and 3 web servers.
I want to receive on my web servers the address' client.
I read that it is possible with the option  source ip usesrc   but
you need to be root.
If you want to not be root, you have to used  HAProxy with Tproxy.
But Tproxy demand too much system configuration.
There is an other solution ?
I hope that you have understood my problem.

Yours sincerely.

Mathieu Sergent

PS : Sorry for my English.


Your English is no problem. ;-)

You can add an X-Forwarded-For header using haproxy. If you then use 
mod_rpaf for apache or realip on nginx you can easily substitute the 
loadbalancer ip with the ip of the client.


Regards,

Sander

Re: Help haproxy

[Jarno Huuskonen]

Hi,

On Mon, Feb 02, Sander Klein wrote:
 On 02.02.2015 12:09, Mathieu Sergent wrote:
 Hi,
 
 I try to set up a load balancing with HAProxy and 3 web servers.
 I want to receive on my web servers the address' client.
 I read that it is possible with the option  source ip usesrc   but
 you need to be root.
 If you want to not be root, you have to used  HAProxy with Tproxy.
 But Tproxy demand too much system configuration.
 There is an other solution ?
 I hope that you have understood my problem.
 
 Yours sincerely.
 
 Mathieu Sergent
 
 PS : Sorry for my English.
 
 Your English is no problem. ;-)
 
 You can add an X-Forwarded-For header using haproxy. If you then use
 mod_rpaf for apache or realip on nginx you can easily substitute the
 loadbalancer ip with the ip of the client.

Or if you're running apache 2.4 then it should come with
mod_remoteip: http://httpd.apache.org/docs/current/mod/mod_remoteip.html

And for tomcat there's:
https://tomcat.apache.org/tomcat-7.0-doc/api/org/apache/catalina/valves/RemoteIpValve.html

-Jarno

-- 
Jarno Huuskonen

Re: Help haproxy

[Mathieu Sergent]

Hi Sander,

Yes i reloaded the haproxy and my web server too. But no change.
 And i'm not using proxy protocol.

To give you more precisions, on my web server i used tcpdump functions
which give me back the header of the requete http. And in this i found my
client's address.
But this is really strange that i can do it without the forwardfor.

Regards,

Mathieu


2015-02-02 16:15 GMT+01:00 Sander Klein roe...@roedie.nl:

 Hi Mathieu,

 Pleas keep the list in the CC.

 On 02.02.2015 15:26, Mathieu Sergent wrote:

 Thanks for your reply.

 I just used the option forwardfor in the haproxy configuration. And i
 can find client's address from my web server (with tcpdump).
 But if i don't use the option forwardfor, the web server still find
 the client's address. That's make any sense ?


 To be honest, that doesn't make any sense to me. Are you sure you have
 reloaded the haproxy process after you removed the forwardfor?

 Or, could it be you are using the proxy protocol (send-proxy)?

 Greets,

 Sander

Re: Help haproxy

[Sander Klein]

Hi Mathieu,

Pleas keep the list in the CC.

On 02.02.2015 15:26, Mathieu Sergent wrote:

Thanks for your reply.

I just used the option forwardfor in the haproxy configuration. And i
can find client's address from my web server (with tcpdump).
But if i don't use the option forwardfor, the web server still find
the client's address. That's make any sense ?


To be honest, that doesn't make any sense to me. Are you sure you have 
reloaded the haproxy process after you removed the forwardfor?


Or, could it be you are using the proxy protocol (send-proxy)?

Greets,

Sander

Re: Help haproxy

[Sander Klein]

On 02.02.2015 16:33, Mathieu Sergent wrote:

Hi Sander,

Yes i reloaded the haproxy and my web server too. But no change.
 And i'm not using proxy protocol.

To give you more precisions, on my web server i used tcpdump functions
which give me back the header of the requete http. And in this i found
my client's address.
But this is really strange that i can do it without the forwardfor.


The only other thing that I can think of is that your client is behind a 
proxy server which adds the X-Forward-For header for you...


Or you got something strange in your config...

Sander

RE: help!

[Lukas Tribus]

 Ok, I wll compose an email and write to haproxy@formilux.org ?

Yes, on every mail, including your responses to me (at least in CC).


And please drop your if you have received this email in error signature when 
your a posting, that just looks silly since your mail is public anyway.





Regards,

Lukas

  

RE: help!

[Lukas Tribus]

Hi Matthew,



 I have some basic queries on setting haproxy up for a novice! 

Just read the documentation and ask here.



 Are you aware of any good support forums out there? 

No forum, this mailing list is the proper way to ask for
help/advice/support haproxy.

There is also an irc channel, if you prefer that. See the
website for details.




Regards,

Lukas

  

Re: Help with dynamic backend selection

[Rajat Chopra]

- Original Message -
 From: Willy Tarreau w...@1wt.eu
 To: Rajat Chopra rcho...@redhat.com
 Cc: haproxy haproxy@formilux.org
 Sent: Saturday, May 10, 2014 12:01:14 AM
 Subject: Re: Help with dynamic backend selection
 
 On Sat, May 10, 2014 at 07:58:25AM +0200, Willy Tarreau wrote:
   May I ask about the ETA on this?
  
  It's too early for me to know, I need to go down deep into the ebtrees
  first to
  see if longest match is compatible with strings storage-wise, then I need
  to
  study how patterns are built as trees to see how to do that as well.
  Possibly
  it's just one or two days of work once I understand everything.
 
 OK in the end it was extremely easy :-)
 Thierry has done an amazing job at making the pattern management very
 modular,
 because I just changed the index and lookup to try in a tree first with a
 different algorithm and that works fine! So we don't care about the
 compatibility between regular string match and beginning.
 
 So that's pushed into git now if you want to give it a try.


Thanks a lot Willy. It works as intended. I am trying to spin a real large case 
and check the performance. Will report.

Thanks again.
Rajat

Re: Help with dynamic backend selection

[Willy Tarreau]

On Sat, May 10, 2014 at 07:58:25AM +0200, Willy Tarreau wrote:
  May I ask about the ETA on this?
 
 It's too early for me to know, I need to go down deep into the ebtrees first 
 to
 see if longest match is compatible with strings storage-wise, then I need to
 study how patterns are built as trees to see how to do that as well. Possibly
 it's just one or two days of work once I understand everything.

OK in the end it was extremely easy :-)
Thierry has done an amazing job at making the pattern management very modular,
because I just changed the index and lookup to try in a tree first with a
different algorithm and that works fine! So we don't care about the
compatibility between regular string match and beginning.

So that's pushed into git now if you want to give it a try.

Willy

Re: Help with dynamic backend selection

[Willy Tarreau]

Hi Rajat,

On Tue, May 06, 2014 at 07:34:34PM -0400, Rajat Chopra wrote:
 Hi!
 The new feature to dynamically select a backend has been going great for 
 me.
 I use it like this and it all works like a charm :
 
 use_backend bk_%[hdr(host),host_to_backend.mapfile] if TRUE
 
 
 Now I need help with a situation. Some of the servers meant for a particular
 hostname need path replacement. i.e. reqrep/resrep.
 e.g. I have a set of 5 servers. While three of them need to be balanced for
 any hostname/service1 requests, the other two need to be load balanced for
 hostname/service2 requests. How can I dynamically choose between two
 backends depending on hostname+path_beg?
 
 Can I use something like this?
 
 use_backend bk_%[hdr(host),host_to_backend.mapfile]_%[path] 
 
 where a request to hostname/service1 would mean look for a backend named 
 bk_hostname_service1. 
 Clearly it will not work like that because hostname/service1/login.php will 
 break the backend lookup.

Absolutely. And you'd have to have as many backends as possible paths, which
is not practical!

Why don't you run that based on ACLs if you're certain that service2 will
not cause any trouble ?

Eg:

use_backend bk_%[hdr(host),host_to_backend.mapfile]_%[path] if { path_beg 
/service2 }
use_backend bk_%[hdr(host),host_to_backend.mapfile]

But again, that requires that you know the exhaustive list of valid paths for
service2. I don't know if that's your case or not, but that could be the
solution.

Regards,
Willy

Re: Help with dynamic backend selection

[Rajat Chopra]

- Original Message -
 From: Willy Tarreau w...@1wt.eu
 To: Rajat Chopra rcho...@redhat.com
 Cc: haproxy haproxy@formilux.org
 Sent: Friday, May 9, 2014 10:43:32 AM
 Subject: Re: Help with dynamic backend selection
 
 Hi Rajat,
 
 On Tue, May 06, 2014 at 07:34:34PM -0400, Rajat Chopra wrote:
  Hi!
  The new feature to dynamically select a backend has been going great
  for me.
  I use it like this and it all works like a charm :
  
  use_backend bk_%[hdr(host),host_to_backend.mapfile] if TRUE
  
  
  Now I need help with a situation. Some of the servers meant for a
  particular
  hostname need path replacement. i.e. reqrep/resrep.
  e.g. I have a set of 5 servers. While three of them need to be balanced for
  any hostname/service1 requests, the other two need to be load balanced
  for
  hostname/service2 requests. How can I dynamically choose between two
  backends depending on hostname+path_beg?
  
  Can I use something like this?
  
  use_backend bk_%[hdr(host),host_to_backend.mapfile]_%[path]
  
  where a request to hostname/service1 would mean look for a backend named
  bk_hostname_service1.
  Clearly it will not work like that because hostname/service1/login.php
  will break the backend lookup.
 
 Absolutely. And you'd have to have as many backends as possible paths, which
 is not practical!
 
 Why don't you run that based on ACLs if you're certain that service2 will
 not cause any trouble ?
 
 Eg:
 
 use_backend bk_%[hdr(host),host_to_backend.mapfile]_%[path] if { path_beg
 /service2 }
 use_backend bk_%[hdr(host),host_to_backend.mapfile]
 
 But again, that requires that you know the exhaustive list of valid paths for
 service2. I don't know if that's your case or not, but that could be the
 solution.

Yes. Thats the only solution I could come up with. I would have the exhaustive 
list as the script generates the config file, but I worry that with 150k 
backends, what if each backend comes up with its own unique path_beg 
requirement. It would defeat the purpose of the map.

Also. In this statement (use_backend 
bk_%[hdr(host),host_to_backend.mapfile]_%[path] if { path_beg /service2 })
using _%[path] is not good, right? We need to use 
'bk_%[hdr(host),map(mapfile)]_service2', right?

Wonder if we could use some pluggable logic for map. e.g. User can provide a 
.so with the user defined map function. So my call will become :
use_backend bk_%[hdr(host),path,myown_func(path_to_dot_so)] if TRUE
And at config time, haproxy does a dlopen on path_to_dot_so, and expects that 
dlsym finds a myown_func method. One that returns a string when given the two 
arguments (host,path).
It may be useful for other dynamic user defined acls also.

Far fetched feature, eh? :)

Thanks for the help again.
Rajat 

Re: Help with dynamic backend selection

[Willy Tarreau]

Hi Rajat,

On Fri, May 09, 2014 at 02:13:47PM -0400, Rajat Chopra wrote:
  Why don't you run that based on ACLs if you're certain that service2 will
  not cause any trouble ?
  
  Eg:
  
  use_backend bk_%[hdr(host),host_to_backend.mapfile]_%[path] if { 
  path_beg
  /service2 }
  use_backend bk_%[hdr(host),host_to_backend.mapfile]
  
  But again, that requires that you know the exhaustive list of valid paths 
  for
  service2. I don't know if that's your case or not, but that could be the
  solution.
 
 Yes. Thats the only solution I could come up with. I would have the
 exhaustive list as the script generates the config file, but I worry that
 with 150k backends, what if each backend comes up with its own unique
 path_beg requirement. It would defeat the purpose of the map.

Yeah for sure.

 Also. In this statement (use_backend 
 bk_%[hdr(host),host_to_backend.mapfile]_%[path] if { path_beg /service2 })
 using _%[path] is not good, right? We need to use 
 'bk_%[hdr(host),map(mapfile)]_service2', right?

Indeed good point.

 Wonder if we could use some pluggable logic for map. e.g. User can provide a
 .so with the user defined map function. So my call will become :

It's still early to do this, there are strict types on the patterns and
their own alloc/release/insert/delete functions.

 use_backend bk_%[hdr(host),path,myown_func(path_to_dot_so)] if TRUE
 And at config time, haproxy does a dlopen on path_to_dot_so, and expects that
 dlsym finds a myown_func method. One that returns a string when given the two
 arguments (host,path).
 It may be useful for other dynamic user defined acls also.

In practice we should not need to define that many new ACLs since they're
supposed to rely on the sample fetch functions. However, sample fetch
functions can very easily be added.

 Far fetched feature, eh? :)

No that much. I'm thinking about something. Maps can retrieve the first
matching prefix using map_beg. So if you pass base to your map, you'll
have the host+path on input, and take the first matching prefix. Thus, it
will give you this :

   use_backend bk_%[base,map_beg(mapfile)]

The file would contain all the host+path sorted from the longest to the
shortest (basically just a sort -r) :

   domain1/path1/   backend1
   domain1/path2/   backend2
   domain1/ backend3
   domain2/path1/   backend1
   domain2/ backend4

The only problem is that the beg match needs to iterate over all entries,
so it will be slower than a tree based lookup, but much faster than running
150k rules!

And further, I think that with some tweeking, we could make the beg match
use the longest prefix match function of the ebtrees. I remember that there's
something tricky about it related to the string length, but if we need to pad
with a zero or something like this, it might not be a big deal. So maybe in
the end we could improve the map match to lookup host+uri prefixes faster
than we do today.

That would be great for URL or Location rewriting!

Regards,
Willy

Re: Help with dynamic backend selection

[Rajat Chopra]

 
  Wonder if we could use some pluggable logic for map. e.g. User can provide
  a
  .so with the user defined map function. So my call will become :
 
 It's still early to do this, there are strict types on the patterns and
 their own alloc/release/insert/delete functions.
 
  use_backend bk_%[hdr(host),path,myown_func(path_to_dot_so)] if TRUE
  And at config time, haproxy does a dlopen on path_to_dot_so, and expects
  that
  dlsym finds a myown_func method. One that returns a string when given the
  two
  arguments (host,path).
  It may be useful for other dynamic user defined acls also.
 
 In practice we should not need to define that many new ACLs since they're
 supposed to rely on the sample fetch functions. However, sample fetch
 functions can very easily be added.
 
  Far fetched feature, eh? :)
 
 No that much. I'm thinking about something. Maps can retrieve the first
 matching prefix using map_beg. So if you pass base to your map, you'll
 have the host+path on input, and take the first matching prefix. Thus, it
 will give you this :
 
use_backend bk_%[base,map_beg(mapfile)]
 
 The file would contain all the host+path sorted from the longest to the
 shortest (basically just a sort -r) :
 
domain1/path1/   backend1
domain1/path2/   backend2
domain1/ backend3
domain2/path1/   backend1
domain2/ backend4
 
 The only problem is that the beg match needs to iterate over all entries,
 so it will be slower than a tree based lookup, but much faster than running
 150k rules!
 

That looks great.
Humble opinion here, or just my discomfort maybe. The map_beg will certainly 
solve my current case, but it feels like that a generic user callable function 
would offload the myriad other cases that might crop up in the future. Pretty 
sure someone will ask for a map_reg down the line - a map with keys as compiled 
regexes is compelling.



 And further, I think that with some tweeking, we could make the beg match
 use the longest prefix match function of the ebtrees. I remember that there's
 something tricky about it related to the string length, but if we need to pad
 with a zero or something like this, it might not be a big deal. So maybe in
 the end we could improve the map match to lookup host+uri prefixes faster
 than we do today.
 
 That would be great for URL or Location rewriting!
 

May I ask about the ETA on this?


Greedily,
With sincere thanks.
Rajat

Re: Help with dynamic backend selection

[Willy Tarreau]

On Fri, May 09, 2014 at 07:54:33PM -0400, Rajat Chopra wrote:
  No that much. I'm thinking about something. Maps can retrieve the first
  matching prefix using map_beg. So if you pass base to your map, you'll
  have the host+path on input, and take the first matching prefix. Thus, it
  will give you this :
  
 use_backend bk_%[base,map_beg(mapfile)]
  
  The file would contain all the host+path sorted from the longest to the
  shortest (basically just a sort -r) :
  
 domain1/path1/   backend1
 domain1/path2/   backend2
 domain1/ backend3
 domain2/path1/   backend1
 domain2/ backend4
  
  The only problem is that the beg match needs to iterate over all entries,
  so it will be slower than a tree based lookup, but much faster than running
  150k rules!
  
 
 That looks great.
 Humble opinion here, or just my discomfort maybe. The map_beg will certainly
 solve my current case, but it feels like that a generic user callable
 function would offload the myriad other cases that might crop up in the
 future. Pretty sure someone will ask for a map_reg down the line - a map with
 keys as compiled regexes is compelling.

But it already exists as well. In fact, all the operators you have for the ACLs
also exist for the maps. Hmmm I just realized that for whatever reason, map_beg
was missing from the map_* table in the doc, which might explain why you didn't
find it. Just fixed it.

  And further, I think that with some tweeking, we could make the beg match
  use the longest prefix match function of the ebtrees. I remember that 
  there's
  something tricky about it related to the string length, but if we need to 
  pad
  with a zero or something like this, it might not be a big deal. So maybe in
  the end we could improve the map match to lookup host+uri prefixes faster
  than we do today.
  
  That would be great for URL or Location rewriting!
  
 
 May I ask about the ETA on this?

It's too early for me to know, I need to go down deep into the ebtrees first to
see if longest match is compatible with strings storage-wise, then I need to
study how patterns are built as trees to see how to do that as well. Possibly
it's just one or two days of work once I understand everything.

Regards,
Willy

RE: [Help] How can I use haproxy with ssl

[Lukas Tribus]

Hi,


 My Linux is
 [...]
 Description: CentOS release 5.10 (Final)
 [...]
 and Installed openssl-1.0.1e

Please decide now: would you like to use the openssl from your
repository or do you want to build it on your own? There is no
point in troubleshooting 2 different problems at the same time.

Please always respond to the mailing list.




Regards,

Lukas 

RE: [Help] How can I use haproxy with ssl

[Lukas Tribus]

Hi,

 but my web server use https (ssl)so load-balancer have to provide
 this. download the haproxy-1.5-dev21and make this.but... error...
 
 src/shctx.o: In function `atomic_dec':
 /root/haproxy-1.5-dev21/src/shctx.c:134: undefined reference to 
 `__sync_sub_and_fetch_4'
 src/shctx.o: In function `cmpxchg':
 /root/haproxy-1.5-dev21/src/shctx.c:129: undefined reference to 
 `__sync_val_compare_and_swap_4'

What linux distribution do you use? You need to install openssl-dev
packages, for example in Debian/Ubuntu with:
 apt-get install libssl-dev

Then you can just use the systems openssl version (which is the simpliest
thing to do; because you also benefit from the system updates).



Otherwise, if you want to go the static build approach:

 and re try  make TARGET=linux26 USE_OPENSSL=1
 ADDINC=-I/tmp/libsslbuild/include ADDLIB=-L/tmp/libsslbuild/lib -ldl
 
 but same error

Did you fully compile openssl and install it in those paths?
What does:
 ls -l /tmp/libsslbuild/include
 ls -l /tmp/libsslbuild/lib

say?


README contains step by step instructions from line 93 to 102.



Regards,

Lukas


[1] http://haproxy.1wt.eu/git?p=haproxy.git;a=blob;f=README 
  

RE: help with long connect times and timeout

[Lukas Tribus]

Hi Ricardo,


 I'm using the current defaults and added this to the config: 

Can you post the full configuration? The part about timeouts
is important in this case.

Also, please specify what release you are using.



 However once I startup haproxy, and I attempt to connect to the proxy 
 from the outside server, it usually (95% of the time) takes a few 
 seconds for the connect to establish and sometimes it even times out 
 and the connection is closed. 
 
 However, even though going from the outside server through haproxy I 
 get delays and timeouts, I can still go directly from the outside 
 server to any of the 3 servers above without any delays or timeouts. 
 The issue is only when going through the proxy 

That sounds like simple packet loss issue to me.

Can you tcpdump frontend and backend traffic of such a broken session?
Could you start haproxy in debug mode, reproduce the issue and post the
log?



Regards,

Lukas 

RE: Help with kQueue

[Lukas Tribus]

Hi Fred,


 #proxy/haproxy -V -d -f /proxy/lr.cfg -n 8192
 Available polling systems :
 select : pref=150, test result OK
 kqueue : disabled, test result OK
 poll : disabled, test result OK 
 Total: 3 (1 usable), will use select. 
 Using select() as the polling mechanism. 


Does it work if you omit -d?

Can you post the full output of haproxy -vv?
What HAproxy version are you using and how did you compile
it (what USE_FLAGs)?


Lukas 

RE: Help with kQueue

[Lukas Tribus]

 # /proxy/haproxy -vv -c -f l2cr.cfg
 Available polling systems :
 kqueue : pref=300,  test result OK 

 #proxy/haproxy -V -d -f /proxy/lr.cfg -n 8192
 Available polling systems :
 select : pref=150,  test result OK


You are comparing two different configurations, are you sure non of them
disables poll and kqueue? Can you run:

egrep -i queue|poll l2cr.cfg
egrep -i queue|poll /proxy/lr.cfg
diff -u l2cr.cfg /proxy/lr.cfg


 1.4.20

Can you grab the latest tarball at http://haproxy.1wt.eu/, this release
is outdated.



Lukas 

Re: Help with cleaning up our error log output and request error counts

[Brendon Colby]

On 12/27/2012 1:53 AM, Willy Tarreau wrote:

On Wed, Dec 26, 2012 at 07:03:02PM -0500, Brendon Colby wrote:

I was thinking that this is just standard browser behavior too. IE also does
this - it just seems to open fewer connections. This is why I was confused
and thought I was missing something, because it seems like normal browser
behavior even though the docs indicated that it should only happen during an
attack or some other anomalous event.

This stupid behaviour is something very recent. I did not know that IE was
doing this too, but after all I'm not surprized, with the browser war...


I'm really surprised no one else as seen and reported this behavior. I 
know one site in particular (trello.com) uses haproxy and that has to be 
higher volume (at least in terms of req/s) than our site. Reading their 
architecture posts is how I found out about haproxy. Maybe I will send 
them an e-mail to see if they've seen anything like this.



If you can reproduce the behaviour with your browser, I think that dontlognull
will be your only solution and that we'll have to update the doc to indicate
that browsers have adopted such an internet-unfriendly behaviour that it's
better to leave the option on. What I don't like with proactively opened
connections is that they're killing servers with 10-100 times the load they
would have to sustain and that even small sites might experience issues with
this. If you see 200 of them per second and they last 5s on average, it means
you're constantly having 5000 idle connections just because of this. Many
web servers can't handle this :-/

I can see smaller sites having a hard time with this! Before we added
timeout http-request we were seeing over 22K established connections to the
haproxy server. That's why we have such a high maxconn on our frontend - we
hit several limits once we went live (which reminds me to lower it) and had
to keep increasing it.

That's really disgusting. Products such as haproxy or nginx can easily deal
with that many concurrent connections, but many other legacy servers cannot.

Would you mind reporting this to the HTTP working group at the IETF ? The
HTTP/1.1 spec is currently being refined and is almost done, but we can
still add information there. Good and bad practices can be updated with
this experience. The address is ietf-http...@w3.org.


I'll see what I can do.


Once we added the timeout though, established connections plummeted to (as of
now, for example) about 5K. The total TCP connections did NOT go down,
however, because now most of them are now in TIME_WAIT (92K!).

TIME_WAIT are harmless on the server side. You can easily reach millions
without any issues.


OK. I didn't know it could reach millions. This is good to know.


The only thing
this seems to affect is our monitoring system which uses netstat to get TCP
stats. It sometimes takes almost a minute to run and uses 100% CPU, but
otherwise doesn't seem to affect anything so we've left it for now.

There are two commands that you must absolutely never use in a monitoring
system :
   - netstat -a
   - ipcs -a

Both of them will saturate the system and considerably slow it down when
something starts to go wrong. For the sockets you should use what's in
/proc/net/sockstat. You have all the numbers you want. If you need more
details, use ss -a instead of netstat -a, it uses the netlink interface
and is several orders of magnitude faster.


Awesome - ss -a runs much faster than netstat. I will update our 
monitoring system to use that instead. This is good to know.



So if connections are terminated because of timeouts that I have explicitly
set, is there any reason to log and count that as a request error? That to me
seems like something that could be logged as info for troubleshooting and not
counted as an error at all. Just a thought - it's nothing major.

This is a real error. It's not because some browsers decided to do stupid
things that it's not an error. When you don't count this case not attacks,
the first reason for not getting a request is that it is blocked by too
short an MTU in some VPNs. It's very important to know that a browser
could not send a POST or a request with a large cookie due to a short MTU
somewhere.


OK - makes sense. So if a browser proactively opens several connections 
to the server, doesn't send any HTTP requests, and then shuts down the 
connection after so many seconds, you're saying this is basically the 
same behavior you'd see from clients with too short an MTU in some types 
of VPNs?



With dontlognulls we still see this type of error at a rate of several per 
second:

Dec 26 18:38:17 localhost haproxy[32259]: n.n.n.n:33685
[26/Dec/2012:18:38:17.155] http-in webservers-ngfiles/ngwebNN 10/0/0/1/649
200 31671 - - CD-- 4166/4159/139/31/0 0/0 GET
/images/14/o-eternal-o_the-vampire-dragon.jpg HTTP/1.1

This one happens more frequently and is also an error as the transfer was not
complete, as seen from haproxy.


We see this a lot in our