date:20191119

yep, 3.0 stands for openssl master branch.
the point is to catch incompatibilities before it is released.

вт, 19 нояб. 2019 г. в 22:51, Gibson, Brian (IMS) :

> Maybe after they stop security fixes we can drop 1.1.0.  I know there are
> many distributions still in support that use this branch.  3.0 doesn’t
> exist yet, and won’t until later in 2020 which is unfortunate since that
> means there will be no FIPS validated branch for several months.
>
>
>
> *From:* Илья Шипицин [mailto:chipits...@gmail.com]
> *Sent:* Tuesday, November 19, 2019 12:48 PM
> *To:* HAProxy 
> *Subject:* Re: travis-ci: should we drop openssl-1.1.0 and replace it
> with 3.0 ?
>
>
>
> well, we can actually build bigger matrix by adding builds. I just want to
> save some electricity on non needed builds.
>
>
>
> вт, 19 нояб. 2019 г. в 22:41, Илья Шипицин :
>
> hello,
>
>
>
> https://www.openssl.org/source/ says "The 1.1.0 series is currently only
> receiving security fixes and will go out of support on 11th September 2019"
>
>
>
>
>
> what if we drop it ? and replace with 3.0 ?
>
>
>
> cheers,
>
> Ilya Shipitcin
>
>
> --
>
> Information in this e-mail may be confidential. It is intended only for
> the addressee(s) identified above. If you are not the addressee(s), or an
> employee or agent of the addressee(s), please note that any dissemination,
> distribution, or copying of this communication is strictly prohibited. If
> you have received this e-mail in error, please notify the sender of the
> error.
>

RE: travis-ci: should we drop openssl-1.1.0 and replace it with 3.0 ?

2019-11-19 Thread Gibson, Brian (IMS)

Maybe after they stop security fixes we can drop 1.1.0.  I know there are many 
distributions still in support that use this branch.  3.0 doesn’t exist yet, 
and won’t until later in 2020 which is unfortunate since that means there will 
be no FIPS validated branch for several months.

From: Илья Шипицин [mailto:chipits...@gmail.com]
Sent: Tuesday, November 19, 2019 12:48 PM
To: HAProxy 
Subject: Re: travis-ci: should we drop openssl-1.1.0 and replace it with 3.0 ?

well, we can actually build bigger matrix by adding builds. I just want to save 
some electricity on non needed builds.

вт, 19 нояб. 2019 г. в 22:41, Илья Шипицин 
mailto:chipits...@gmail.com>>:
hello,

https://www.openssl.org/source/ says "The 1.1.0 series is currently only 
receiving security fixes and will go out of support on 11th September 2019"

what if we drop it ? and replace with 3.0 ?

cheers,
Ilya Shipitcin

Information in this e-mail may be confidential. It is intended only for the 
addressee(s) identified above. If you are not the addressee(s), or an employee 
or agent of the addressee(s), please note that any dissemination, distribution, 
or copying of this communication is strictly prohibited. If you have received 
this e-mail in error, please notify the sender of the error.

Re: travis-ci: should we drop openssl-1.1.0 and replace it with 3.0 ?

well, we can actually build bigger matrix by adding builds. I just want to
save some electricity on non needed builds.

вт, 19 нояб. 2019 г. в 22:41, Илья Шипицин :

> hello,
>
> https://www.openssl.org/source/ says "The 1.1.0 series is currently only
> receiving security fixes and will go out of support on 11th September 2019"
>
>
> what if we drop it ? and replace with 3.0 ?
>
> cheers,
> Ilya Shipitcin
>

travis-ci: should we drop openssl-1.1.0 and replace it with 3.0 ?

hello,

https://www.openssl.org/source/ says "The 1.1.0 series is currently only
receiving security fixes and will go out of support on 11th September 2019"


what if we drop it ? and replace with 3.0 ?

cheers,
Ilya Shipitcin

Re: master-worker no-exit-on-failure with SO_REUSEPORT and a port being already in use

2019-11-19 Thread William Lallemand

On Tue, Nov 19, 2019 at 04:19:26PM +0100, William Lallemand wrote:
> > I then add another bind for port 80, which is in use by squid already 
> > and try to reload HAProxy. It takes some time until it failes:
> > 
> > Nov 19 14:39:21 894a0f616fec haproxy[2978]: [WARNING] 322/143921 (2978) 
> > : Reexecuting Master process
> > ...
> > Nov 19 14:39:28 894a0f616fec haproxy[2978]: [ALERT] 322/143922 (2978) : 
> > Starting frontend somefrontend: cannot bind socket [0.0.0.0:80]
> > ...
> > Nov 19 14:39:28 894a0f616fec systemd[1]: haproxy.service: Main process 
> > exited, code=exited, status=1/FAILURE
> > 
> > The reload itself is still running (systemd) and will timeout after 
> > about 90s. After that, because of the Restart=always, I guess, it ends 
> > up in a restart loop.
> > 
> > So I would have expected that the master process will fallback to the 
> > old process and proceed with the old child until the problem has been 
> > fixed.
> > 

The patch in attachment fixes a bug where haproxy could reexecute itself in
waitpid mode with -sf -1.

I'm not sure this is your bug, but if this is the case you should see haproxy
in waitpid mode, then the master exiting with the usage message in your logs.

-- 
William Lallemand
>From 481a3c62a622974587c731b1bdc1478538fd6527 Mon Sep 17 00:00:00 2001
From: William Lallemand 
Date: Tue, 19 Nov 2019 17:04:18 +0100
Subject: [PATCH] BUG/MEDIUM: mworker: don't fill the -sf argument with -1
 during the reexec

Upon a reexec_on_failure, if the process tried to exit after the
initialization of the process structure but before it was filled with a
PID, the PID in the mworker_proc structure is set to -1.

In this particular case the -sf argument is filled with -1 and haproxy
will exit with the usage message because of that argument.

Should be backported in 2.0.
---
 src/haproxy.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/haproxy.c b/src/haproxy.c
index a0e630dfa..1d4771e64 100644
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -673,7 +673,7 @@ void mworker_reload()
 		next_argv[next_argc++] = "-sf";
 
 		list_for_each_entry(child, _list, list) {
-			if (!(child->options & (PROC_O_TYPE_WORKER|PROC_O_TYPE_PROG)))
+			if (!(child->options & (PROC_O_TYPE_WORKER|PROC_O_TYPE_PROG)) || child->pid <= -1 )
 continue;
 			next_argv[next_argc] = memprintf(, "%d", child->pid);
 			if (next_argv[next_argc] == NULL)
-- 
2.21.0

RE: native prometheus exporter: retrieving check_status

2019-11-19 Thread Pierre Cheynier

> Hi Pierre,

Hi!,

> I addressed this issue based on a William's idea. I also proposed to add a 
> filter to exclude all servers in maintenance from the export. Let me know if 
> you 
> see a better way to do so. For the moment, from the exporter point of view, 
> it 
> is not really hard to do such filtering.

Yes, that's a great addition, and should improve a lot, but I'm still not sure 
if it will be
sufficient (meaning that we could end up dropping all servers if the endpoint 
are still
too huge, as we used to do with the old exporter).

BTW, we also did that since we're using server-templates, and the naming in 
templates
make the server-name information useless (since we can't modify the name at 
runtime).
So we previously had a sufficient level of info at backend level, thanks to 
native
aggregations.

>> [ST_F_CHECK_STATUS]   = IST("untyped"),
>> What could be done to be able to retrieve them? (I thought about something 
>> similar to 
>> `HRSP_[1-5]XX`, where the different check status could be defined and 
>> counted).
>> 
>
> Hum, I can add the check status. Mapping all status on integers is possible. 
> However, having a metric per status is probably not the right solution, 
> because 
> it is not a counter but just a state (a boolean). If we do so, all status 
> would 
> be set to 0 except the current status. It is not really handy. But a mapping 
> is 
> possible. We already do this for the frontend/backend/server status 
> (ST_F_STATUS).

Yes, it would work perfectly. At the end the goal for us would be to be able to 
retrieve this
state. My idea about a counter was more about backend-level aggregations, if 
consistent
(I'm not sure it is actually, hence the feedback request).

>> * also for `check_status`, there is the case of L7STS and its associated 
>> values that are present
>> in another field. Most probably it could benefit from a better 
>> representation in a prometheus
>> output (thanks to labels)?
>>
> We can also export the metrics ST_F_CHECK_CODE. For the use of labels, I have 
> no 
> idea. For now, the labels are static in the exporter. And I don't know if it 
> is 
> pertinent to add dynamic info in labels. If so, what is your idea ? Add a 
> "code" 
> label associated to the check_status metric ?

Here again, my maybe-not-so-good idea was to keep the ability to retrieve all 
the
underlying details at backend level, such as:
* 100 servers are L7OK
* 1 server is L4TOUT
* 2 servers are L4CON
* 2 servers are L7STS
** 1 due to a HTTP 429
** 1 due to a HTTP 503

But this is maybe overkill in terms of complexity, we could maybe push more on
our ability to retrieve non-maint servers status.

> It is feasible. But only counters may be aggregated. It may be enabled using 
> a 
> parameter in the query-string. However, it is probably pertinent only when 
> the 
> server metrics are filtered out. Because otherwise, Prometheus can handle the 
> aggregation itself.

Sure, we should rely on this as much as possible.

--
Pierre

Re: [PATCH] MINOR: contrib/prometheus-exporter: allow to select the exported metrics

Hi Christopher,

On Tue, Nov 19, 2019 at 04:35:47PM +0100, Christopher Faulet wrote:
> Here is updated patches with the support for "scope" and "no-maint"
> parameters. If this solution is good enough for you (and if it works :), I
> will push it.

this looks good to me and the test was conclusive on our side.
For now we won't make use of no-maint as the server are completely
deactivated. We should however measure how much data does it represent
now, in order to see if we could simply use no-maint parameter.
I will push the no-maint patch on our production later today.

Thank you for your work,
-- 
William

Re: native prometheus exporter: retrieving check_status

On Tue, Nov 19, 2019 at 03:31:28PM +0100, Christopher Faulet wrote:
> > * also for `check_status`, there is the case of L7STS and its associated
> > values that are present in another field. Most probably it could benefit
> > from a better representation in a prometheus output (thanks to labels)?
> 
> We can also export the metrics ST_F_CHECK_CODE. For the use of labels, I
> have no idea. For now, the labels are static in the exporter. And I don't
> know if it is pertinent to add dynamic info in labels. If so, what is your
> idea ? Add a "code" label associated to the check_status metric ?

we need to be very careful here indeed. It's not very clear in my mind
how much values we are talking about, but labels trigger the creation of
a new metric of each key/value pair. So it can quickly explode your
memory on scrapping side.

> > * what about getting some backend-level aggregation of server metrics, such 
> > as the one that was previously mentioned, to avoid retrieving all the 
> > server metrics but still be able to get some insights?
> > I'm thinking about an aggregation of some fields at backend level, which 
> > was not previously done with the CSV output.
> 
> It is feasible. But only counters may be aggregated. It may be enabled using
> a parameter in the query-string. However, it is probably pertinent only when
> the server metrics are filtered out. Because otherwise, Prometheus can
> handle the aggregation itself.

My only fear for this point would be to make the code too complicated
and harder to maintain.

-- 
William

Re: http-buffer-request details

2019-11-19 Thread Tim Düsterhus

Christopher,

Am 19.11.19 um 16:39 schrieb Christopher Faulet:
> You're right Tim. But this one is small enough to be fixed immediately
> :). I will push the patch with other ones I'm working on. But it is
> already fixed.
> 

That's even better of course! Disregard my comment in *this specific*
case then :-)

Best regards
Tim Düsterhus

Re: http-buffer-request details


Le 19/11/2019 à 16:32, Tim Düsterhus a écrit :

Christopher,

Am 19.11.19 um 16:23 schrieb Christopher Faulet:

As mentioned in the documentation, HTTP processing is delayed waiting
the whole body is received or the request buffer is full. The condition
about the first chunk of a chunked request is only valid for the legacy
HTTP mode. It was removed in 2.1, so the documentation is a bit outdated
on this point.


Consider filing a short `type: doc` issue if you realize that the
documentation is outdated, but you don't have time to think about a
proper wording / creating a patch:
https://github.com/haproxy/haproxy/issues?q=is%3Aopen+is%3Aissue+label%3A%22type%3A+doc%22

That way one can come around later and make the necessary adjustments
when there's a bit more time.



You're right Tim. But this one is small enough to be fixed immediately :). I 
will push the patch with other ones I'm working on. But it is already fixed.


--
Christopher Faulet

Re: [PATCH] MINOR: contrib/prometheus-exporter: allow to select the exported metrics


Le 19/11/2019 à 14:51, Christopher Faulet a écrit :


Regarding the problem of servers in maintenance, since we parse the
query-string, it is possible to add more filters. I may add a parameter to
filter out servers in maintenance. For instance, by passing "no-maint" in the
query-string, all servers in maintenance could be ignored. This way, it would
still be possible to have servers metrics.



William,

Here is updated patches with the support for "scope" and "no-maint" parameters. 
If this solution is good enough for you (and if it works :), I will push it.


Thanks,
--
Christopher Faulet
>From 205c4775ccec914a2a501fe9499a77b96285315b Mon Sep 17 00:00:00 2001
From: Christopher Faulet 
Date: Mon, 18 Nov 2019 14:47:08 +0100
Subject: [PATCH 1/2] MINOR: contrib/prometheus-exporter: filter exported
 metrics by scope

Now, the prometheus exporter parses the HTTP query-string to filter or to adapt
the exported metrics. In this first version, it is only possible select the
scopes of metrics to export. To do so, one or more parameters with "scope" as
name must be passed in the query-string, with one of those values: global,
frontend, backend, server or '*' (means all). A scope parameter with no value
means to filter out all scopes (nothing is returned). The scope parameters are
parsed in their appearance order in the query-string. So an empty scope will
reset all scopes already parsed. But it can be overridden by following scope
parameters in the query-string. By default everything is exported.

The filtering can also be done on prometheus scraping configuration, but general
aim is to optimise the source of data to improve load and scraping time. This is
particularly true for huge configuration with thousands of backends and servers.
Also note that this configuration was possible on the previous official haproxy
exporter but with even more parameters to select the needed metrics. Here we
thought it was sufficient to simply avoid a given type of metric. However, more
filters are still possible.

Thanks to William Dauchy. This patch is based on his work.
---
 contrib/prometheus-exporter/README|  24 +++
 .../prometheus-exporter/service-prometheus.c  | 157 +++---
 2 files changed, 156 insertions(+), 25 deletions(-)

diff --git a/contrib/prometheus-exporter/README b/contrib/prometheus-exporter/README
index 915fc7f54..84ae8e27e 100644
--- a/contrib/prometheus-exporter/README
+++ b/contrib/prometheus-exporter/README
@@ -50,6 +50,30 @@ spend much more ressources to produce the Prometheus metrics than the CSV export
 through the stats page. To give a comparison order, quick benchmarks shown that
 a PROMEX dump is 5x slower and 20x more verbose than a CSV export.
 
+
+metrics filtering
+---
+
+It is possible to dynamically select the metrics to export if you don't use all
+of them passing parameters in the query-string.
+
+* Filtering on scopes
+
+The metrics may be filtered by scopes. Multiple parameters with "scope" as name
+may be passed in the query-string to filter exported metrics, with one of those
+values: global, frontend, backend, server or '*' (means all). A scope parameter
+with no value means to filter out all scopes (nothing is returned). The scope
+parameters are parsed in their appearance order in the query-string. So an empty
+scope will reset all scopes already parsed. But it can be overridden by
+following scope parameters in the query-string. By default everything is
+exported. Here are examples:
+
+  /metrics?scope=server # ==> server metrics will be exported
+  /metrics?scope=frontend=backend # ==> Frontend and backend metrics will be exported
+  /metrics?scope=*=   # ==> no metrics will be exported
+  /metrics?scope==global  # ==> global metrics will be exported
+
+
 Exported metrics
 --
 
diff --git a/contrib/prometheus-exporter/service-prometheus.c b/contrib/prometheus-exporter/service-prometheus.c
index 508e6b1fc..c70daa905 100644
--- a/contrib/prometheus-exporter/service-prometheus.c
+++ b/contrib/prometheus-exporter/service-prometheus.c
@@ -64,6 +64,12 @@ enum {
 #define PROMEX_FL_METRIC_HDR0x0001
 #define PROMEX_FL_INFO_METRIC   0x0002
 #define PROMEX_FL_STATS_METRIC  0x0004
+#define PROMEX_FL_SCOPE_GLOBAL  0x0008
+#define PROMEX_FL_SCOPE_FRONT   0x0010
+#define PROMEX_FL_SCOPE_BACK0x0020
+#define PROMEX_FL_SCOPE_SERVER  0x0040
+
+#define PROMEX_FL_SCOPE_ALL (PROMEX_FL_SCOPE_GLOBAL|PROMEX_FL_SCOPE_FRONT|PROMEX_FL_SCOPE_BACK|PROMEX_FL_SCOPE_SERVER)
 
 /* The max length for metrics name. It is a hard limit but it should be
  * enougth.
@@ -2102,72 +2108,81 @@ static int promex_dump_srv_metrics(struct appctx *appctx, struct htx *htx)
 static int promex_dump_metrics(struct appctx *appctx, struct stream_interface *si, struct htx *htx)
 {
 	int ret;
+	int flags = appctx->ctx.stats.flags;
 
 	switch (appctx->st1) {
 		case PROMEX_DUMPER_INIT:
 			appctx->ctx.stats.px =

Re: http-buffer-request details

2019-11-19 Thread Tim Düsterhus

Christopher,

Am 19.11.19 um 16:23 schrieb Christopher Faulet:
> As mentioned in the documentation, HTTP processing is delayed waiting
> the whole body is received or the request buffer is full. The condition
> about the first chunk of a chunked request is only valid for the legacy
> HTTP mode. It was removed in 2.1, so the documentation is a bit outdated
> on this point.

Consider filing a short `type: doc` issue if you realize that the
documentation is outdated, but you don't have time to think about a
proper wording / creating a patch:
https://github.com/haproxy/haproxy/issues?q=is%3Aopen+is%3Aissue+label%3A%22type%3A+doc%22

That way one can come around later and make the necessary adjustments
when there's a bit more time.

Best regards
Tim Düsterhus

Re: http-buffer-request details


Le 19/11/2019 à 13:20, Илья Шипицин a écrit :

hello,

how is that supposed to work ?

https://github.com/haproxy/haproxy/blob/master/doc/configuration.txt#L6225

does it buffer the entire body ? does it use memory / hdd for buffering ?

how are those buffers allocated ? what if I do not have a lot of RAM ?



Hi Ilya,

As mentioned in the documentation, HTTP processing is delayed waiting the whole 
body is received or the request buffer is full. The condition about the first 
chunk of a chunked request is only valid for the legacy HTTP mode. It was 
removed in 2.1, so the documentation is a bit outdated on this point.


BTW, this means that HAProxy waits for the entire request's body before 
processing it, if this one is smaller than a buffer. Otherwise, it waits the 
request buffer is full. In this case, only a part of the payload will be 
available. But there is no extra memory allocated to store the entire body.


About the buffers allocation, as for most of other objects dynamically allocated 
in HAProxy, it uses memory pools to do so. The buffer size if 16k by default and 
may be tuned via the "tune.bufsize" global directive.


Hope it helps,
--
Christopher Faulet

Re: master-worker no-exit-on-failure with SO_REUSEPORT and a port being already in use

2019-11-19 Thread William Lallemand

On Tue, Nov 19, 2019 at 03:45:09PM +0100, Christian Ruppert wrote:
> Hi list,
> 

Hello,

> I'm facing some issues with already in use ports and the fallback 
> feature, during a reload. SO_REUSEPORT already makes ist easier/better 
> but not perfect, as there are still cases were it fails.
> In my test case I've got a Squid running on port 80 and a HAProxy with 
> "master-worker no-exit-on-failure".

The "no-exit-on-failure" option is only useful when you don't want the master
to kill all the HAProxy processes when one of the workers was killed by
another thing that the master (segv, OOM, bug..). In this case you still need
another worker available to do the job. It's mostly used with a configuration
with nbproc > 1.

> I am using the shipped (2.0.8) 
> systemd unit file and startup HAProxy with some frontend and a bind on 
> like 1337 or something.
> I then add another bind for port 80, which is in use by squid already 
> and try to reload HAProxy. It takes some time until it failes:
> 
> Nov 19 14:39:21 894a0f616fec haproxy[2978]: [WARNING] 322/143921 (2978) 
> : Reexecuting Master process
> ...
> Nov 19 14:39:28 894a0f616fec haproxy[2978]: [ALERT] 322/143922 (2978) : 
> Starting frontend somefrontend: cannot bind socket [0.0.0.0:80]
> ...
> Nov 19 14:39:28 894a0f616fec systemd[1]: haproxy.service: Main process 
> exited, code=exited, status=1/FAILURE
> 
> The reload itself is still running (systemd) and will timeout after 
> about 90s. After that, because of the Restart=always, I guess, it ends 
> up in a restart loop.
> 
> So I would have expected that the master process will fallback to the 
> old process and proceed with the old child until the problem has been 
> fixed.
> 
> Can anybody confirm that? Is that intended?
> 
> https://cbonte.github.io/haproxy-dconv/2.0/management.html#4
> https://cbonte.github.io/haproxy-dconv/2.0/configuration.html#3.1-master-worker
>

Looks like a bug to me, the master should have fallback to the "waitpid mode" 
in this case.

Maybe we don't send the sd_notify OK when we are in waitpid mode and systemd
kills the process after the reload timeout.

I'll do some tests to check what's going on. 

-- 
William Lallemand

master-worker no-exit-on-failure with SO_REUSEPORT and a port being already in use

2019-11-19 Thread Christian Ruppert


Hi list,

I'm facing some issues with already in use ports and the fallback 
feature, during a reload. SO_REUSEPORT already makes ist easier/better 
but not perfect, as there are still cases were it fails.
In my test case I've got a Squid running on port 80 and a HAProxy with 
"master-worker no-exit-on-failure". I am using the shipped (2.0.8) 
systemd unit file and startup HAProxy with some frontend and a bind on 
like 1337 or something.
I then add another bind for port 80, which is in use by squid already 
and try to reload HAProxy. It takes some time until it failes:


Nov 19 14:39:21 894a0f616fec haproxy[2978]: [WARNING] 322/143921 (2978) 
: Reexecuting Master process

...
Nov 19 14:39:28 894a0f616fec haproxy[2978]: [ALERT] 322/143922 (2978) : 
Starting frontend somefrontend: cannot bind socket [0.0.0.0:80]

...
Nov 19 14:39:28 894a0f616fec systemd[1]: haproxy.service: Main process 
exited, code=exited, status=1/FAILURE


The reload itself is still running (systemd) and will timeout after 
about 90s. After that, because of the Restart=always, I guess, it ends 
up in a restart loop.


So I would have expected that the master process will fallback to the 
old process and proceed with the old child until the problem has been 
fixed.


Can anybody confirm that? Is that intended?

https://cbonte.github.io/haproxy-dconv/2.0/management.html#4
https://cbonte.github.io/haproxy-dconv/2.0/configuration.html#3.1-master-worker

--
Regards,
Christian Ruppert

Re: native prometheus exporter: retrieving check_status

Hi Pierre,

Sorry I missed you email. Thanks to William for the reminder.

Le 15/11/2019 à 15:55, Pierre Cheynier a écrit :

We've recently tried to switch to the native prometheus exporter, but went
quickly stopped in our initiative given the output on one of our preprod server:

$ wc -l metrics.out
1478543 metrics.out
$ ls -lh metrics.out
-rw-r--r-- 1 pierre pierre 130M nov. 15 15:33 metrics.out

This is not only due to a large setup, but essentially related to server lines,
since we extensively user server-templates for server addition/deletion at
runtime.

# backend & servers number
$ echo "show stat -1 2 -1" | sudo socat stdio /var/lib/haproxy/stats | wc -l
1309
$ echo "show stat -1 4 -1" | sudo socat stdio /var/lib/haproxy/stats | wc -l
36360
# But a lot of them are actually "waiting to be provisioned" (especially on
this preprod environment)
$ echo "show stat -1 4 -1" | sudo socat stdio /var/lib/haproxy/stats | grep
MAINT | wc -l
34113

We'll filter out the server metrics as a quick fix, and will hopefully submit
something to do it natively, but we would also like to get your feedbacks about
some use-cases we expected to solve with this native exporter.

I addressed this issue based on a William's idea. I also proposed to add a
filter to exclude all servers in maintenance from the export. Let me know if you
see a better way to do so. For the moment, from the exporter point of view, it
is not really hard to do such filtering.

Ultimately, one of them would be a great value-added for us: being able to
count check_status types (and their values in the L7STS case) per backend.

So, there are 3 associated points:
* it's great to have new metrics (such as
`haproxy_process_current_zlib_memory`), but we also noticed that some very
useful ones were not present due to their type, example:
[ST_F_CHECK_STATUS] = IST("untyped"),
What could be done to be able to retrieve them? (I thought about something
similar to `HRSP_[1-5]XX`, where the different check status could be defined
and counted).

Hum, I can add the check status. Mapping all status on integers is possible.
However, having a metric per status is probably not the right solution, because
it is not a counter but just a state (a boolean). If we do so, all status would
be set to 0 except the current status. It is not really handy. But a mapping is
possible. We already do this for the frontend/backend/server status (ST_F_STATUS).

* also for `check_status`, there is the case of L7STS and its associated values that are present in another field. Most probably it could benefit from a better representation in a prometheus output (thanks to labels)?

We can also export the metrics ST_F_CHECK_CODE. For the use of labels, I have no
idea. For now, the labels are static in the exporter. And I don't know if it is
pertinent to add dynamic info in labels. If so, what is your idea ? Add a "code"
label associated to the check_status metric ?

* what about getting some backend-level aggregation of server metrics, such as
the one that was previously mentioned, to avoid retrieving all the server
metrics but still be able to get some insights?
I'm thinking about an aggregation of some fields at backend level, which was
not previously done with the CSV output.

It is feasible. But only counters may be aggregated. It may be enabled using a
parameter in the query-string. However, it is probably pertinent only when the
server metrics are filtered out. Because otherwise, Prometheus can handle the
aggregation itself.

--
Christopher Faulet

Re: [PATCH] MINOR: contrib/prometheus-exporter: allow to select the exported metrics


Hi William,

I missed Pierre's email. I'm CCing him.

Le 18/11/2019 à 21:00, William Dauchy a écrit :

Thanks. Having a way to filter metrics in the Prometheus exporter was on my
todo-list :) Filtering on scopes is pretty simple and it is a good start to
solve performance issues for huge configs. It is a good idea.
However, IMHO, it is easier to use the query-string to select exported
metrics. I don't know if it is compatible with the way Prometheus is used.
For instance, based on your idea, to get only metrics about servers, the URI
could be "/metric?scope=server". And to get metrics about frontends and
backends, it could be "/metric?scope=frontend=backend". Of course, it
is extensible. We can imagine to add filters on frontend/backend/server
names.
Here is a quick patch based on this. What do you think about it ? If you
said me your way to select metrics is better from the Prometheus point of
view, I'm ok with that.


In fact I even did not thought about it because my state of mind was to
filter at startup like we were doing before. I like you propostion as it
is way more extensible than mine. I was not even thinking it would be
possible to access the URL, I now have a lot of ideas of the future ;)

example config would look like:

- job_name: 'lb-builtin-metrics'
 metrics_path: '/metrics?scope=backend'
 consul_sd_configs:
   - server: localhost:8500
 services:
   - lb-builtin-exporter
 relabel_configs:
   - source_labels: ['__meta_consul_dc']
 target_label: 'dc'
   - source_labels: ['__address__', '__meta_consul_service_id']
 target_label: 'instance'

If you have other things on your todo list regarding that feel free to
share so we might share the work around it.


Well, for now, there is nothing concrete. But I've planned to add a filter on 
frontend/backend names and, if possible, a way to list the metrics to export. 
The scope idea may be used too for this last one, by grouping metrics by 
categories (http, conn, timers, checks, rates...). In the end, it's more a 
question of time than anything else :)



Also have you seen the message from Pierre? They are a few things
we would like to discuss whether it would be possible to aggregate a few
things on backend side.
https://www.mail-archive.com/haproxy@formilux.org/msg35369.html
(we somehow forgot to put you in copy though)


Regarding the problem of servers in maintenance, since we parse the 
query-string, it is possible to add more filters. I may add a parameter to 
filter out servers in maintenance. For instance, by passing "no-maint" in the 
query-string, all servers in maintenance could be ignored. This way, it would 
still be possible to have servers metrics.


For others points, I will reply to the previous email.



Thank you for this patch. Do you think it could be acceptable to mark it
as a possible backport for v2.0.x? It's quite critical on our side as we
are dealing with ~130MB on some cases. If not we will do the backport on
our wide while waiting for the v2.1.



It can safely be backported to 2.0. It is not a critical component and it is 
independent of other parts.




I will push a test based on your patch tomorrow on our side.


Ok, let me know if you encounter any issues.

Thanks,
--
Christopher Faulet

http-buffer-request details

hello,

how is that supposed to work ?

https://github.com/haproxy/haproxy/blob/master/doc/configuration.txt#L6225

does it buffer the entire body ? does it use memory / hdd for buffering ?

how are those buffers allocated ? what if I do not have a lot of RAM ?

thanks,
Ilya Shipitcin

Re: Firewall and Haproxy

2019-11-19 Thread Baptiste

On Sun, Nov 17, 2019 at 2:41 PM TomK  wrote:

> Hey All,
>
> When adding hosts to a F/W behind a VIP (keepalived for example) to
> which Haproxy is bound, should just the VIP be added to the F/W or would
> all member hosts behind Haproxy need to be added as well?
>
> If all member hosts behind haproxy need to be added, why?
>
> Only reason I can think of adding individual host members is for
> troubleshooting purposes.  Other then that, can't think of a valid
> reason why each member host would connect separately.
>
> --
> Thx,
> TK.
>
>
Hi,

You should just open traffic to ports configured on the VIP in HAProxy.

Baptiste

Re: HAProxy question

2019-11-19 Thread Aleksandar Lazic



Hi.
 
Nov 19, 2019 11:05:34 AM Micael Gillet :
 
> Hello, As part of a project, I have some questions about HAProxy's abilities.
> Could you confirm if HAProxy is able to handle the following points?
> 
> * STP Protection (RSTP)
> * VLANs interfaces
 
This is to low level for HAProxy, IMHO.
 
> * HA Cluster in Active / Passive mode
 
Yes it's possible.
 
> * SNMP for monitoring
 
Not out of the box but with tools possible.
 
> * HealthCheck of LDAP services
> * Round robin and failover load balancing
> * Routing flows to a specific pool based on the source IP address
> * Filtering incoming flow by IP/port
 
Yes it's possible.
 
> * Oneconnect" type profile
 
Is this what you mean with that question?
 
https://support.f5.com/csp/article/K7208
 
It looks like you want to replace a F5 cluster.
I would recommend to get in touch with HAProxy Company for a proposal as I 
assume that the commercial product will fit in your requirements.
 
> Thanks for your support. Regards Micael Gillet
 
Regards aleks
 
> Courriel confidentiel: 
> Ce message est protégé par les règles relatives au secret des 
> correspondances. Il est donc établi à destination exclusive de son 
> destinataire. Celui-ci peut donc contenir des informations confidentielles. 
> La divulgation de ces informations est à ce titre rigoureusement interdite. 
> Si vous avez reçu ce message par erreur, merci de le renvoyer à l'expéditeur 
> dont l'adresse e-mail figure ci-dessus et de détruire le message ainsi que 
> toute pièce jointe. 
> This message is protected by the secrecy of correspondence rules. Therefore, 
> this message is intended solely for the attention of the addressee. This 
> message may contain privileged or confidential information, as such the 
> disclosure of these informations is strictly forbidden. If, by mistake, you 
> have received this message, please return this message to the addressser 
> whose e-mail address is written above and destroy this message and all files 
> attached. 
> 
> 
> [https://www.msa.fr/lfy/documents/98830/d420d3b1-9e7c-05ab-6765-009d1e6c1d1f?t=1572346078784]
>

Re: [PATCH] BUG/MINOR: init: fix set-dumpable when using uid/gid

2019-11-19 Thread Willy Tarreau

On Tue, Nov 19, 2019 at 10:11:36AM +0100, William Dauchy wrote:
> Here is the backport for haproxy-20 tree.

Now merged, thanks William.
Willy

HAProxy question

2019-11-19 Thread Micael Gillet

Hello,
As part of a project, I have some questions about HAProxy's abilities.
Could you confirm if HAProxy is able to handle the following points?

1. STP Protection (RSTP)
2. VLANs interfaces
3. HA Cluster in Active / Passive mode
4. SNMP for monitoring
5. HealthCheck of LDAP services
6. Round robin and failover load balancing
7. Routing flows to a specific pool based on the source IP address
8. Filtering incoming flow by IP/port
9. Oneconnect" type profile

Thanks for your support.
Regards
Micael Gillet

Courriel confidentiel:
Ce message est protégé par les règles relatives au secret des correspondances.
Il est donc établi à destination exclusive de son destinataire. Celui-ci peut
donc contenir des informations confidentielles. La divulgation de ces
informations est à ce titre rigoureusement interdite. Si vous avez reçu ce
message par erreur, merci de le renvoyer à l'expéditeur dont l'adresse e-mail
figure ci-dessus et de détruire le message ainsi que toute pièce jointe.
This message is protected by the secrecy of correspondence rules. Therefore,
this message is intended solely for the attention of the addressee. This
message may contain privileged or confidential information, as such the
disclosure of these informations is strictly forbidden. If, by mistake, you
have received this message, please return this message to the addressser whose
e-mail address is written above and destroy this message and all files attached.

[https://www.msa.fr/lfy/documents/98830/d420d3b1-9e7c-05ab-6765-009d1e6c1d1f?t=1572346078784]

Re: [PATCH] BUG/MINOR: init: fix set-dumpable when using uid/gid

Hi,

On Tue, Nov 19, 2019 at 02:42:23PM +0500, Илья Шипицин wrote:
> small question. 
> `/proc/sys/fs/suid_dumpable` is linux specific. will it work under freebsd,
> openbsd ? windows ?
> also, linux might not mount that filesystem. will it work ?

this code is protected around USE_PRCTL define, so I guess it is not
selected on other OS.

for the linux part, we do not use this path in proc fs, I was referring
to that to explain the behaviour of prctl.

hope it helps,
-- 
William

Re: [PATCH] BUG/MINOR: init: fix set-dumpable when using uid/gid

вт, 19 нояб. 2019 г. в 14:15, William Dauchy :

> in mworker mode used with uid/gid settings, it was not possible to get
> a coredump despite the set-dumpable option.
> indeed prctl(2) manual page specifies the dumpable attribute is reverted
> to `/proc/sys/fs/suid_dumpable` in a few conditions such as process
> effective user and group are changed.
>

small question.

`/proc/sys/fs/suid_dumpable` is linux specific. will it work under freebsd,
openbsd ? windows ?
also, linux might not mount that filesystem. will it work ?



>
> this patch moves the whole set-dumpable logic before the polling code in
> order to catch all possible cases where we could have changed the
> uid/gid. It however does not cover the possible segfault at startup.
>
> this patch should be backported in 2.0.
>
> Signed-off-by: William Dauchy 
>
> ---
>
> Hi Willy,
>
> Here is the backport for haproxy-20 tree.
>
> Thanks,
>
> William
>
> ---
>  src/haproxy.c | 50 +-
>  1 file changed, 25 insertions(+), 25 deletions(-)
>
> diff --git a/src/haproxy.c b/src/haproxy.c
> index fa3fbad4..a0e630df 100644
> --- a/src/haproxy.c
> +++ b/src/haproxy.c
> @@ -2946,31 +2946,6 @@ int main(int argc, char **argv)
> }
> }
>
> -   /* try our best to re-enable core dumps depending on system
> capabilities.
> -* What is addressed here :
> -*   - remove file size limits
> -*   - remove core size limits
> -*   - mark the process dumpable again if it lost it due to
> user/group
> -*/
> -   if (global.tune.options & GTUNE_SET_DUMPABLE) {
> -   limit.rlim_cur = limit.rlim_max = RLIM_INFINITY;
> -
> -#if defined(RLIMIT_FSIZE)
> -   if (setrlimit(RLIMIT_FSIZE, ) == -1)
> -   ha_warning("[%s.main()] Failed to set the raise
> the maximum file size.\n", argv[0]);
> -#endif
> -
> -#if defined(RLIMIT_CORE)
> -   if (setrlimit(RLIMIT_CORE, ) == -1)
> -   ha_warning("[%s.main()] Failed to set the raise
> the core dump size.\n", argv[0]);
> -#endif
> -
> -#if defined(USE_PRCTL)
> -   if (prctl(PR_SET_DUMPABLE, 1, 0, 0, 0) == -1)
> -   ha_warning("[%s.main()] Failed to set the dumpable
> flag, no core will be dumped.\n", argv[0]);
> -#endif
> -   }
> -
> /* check ulimits */
> limit.rlim_cur = limit.rlim_max = 0;
> getrlimit(RLIMIT_NOFILE, );
> @@ -3253,6 +3228,31 @@ int main(int argc, char **argv)
> fork_poller();
> }
>
> +   /* try our best to re-enable core dumps depending on system
> capabilities.
> +* What is addressed here :
> +*   - remove file size limits
> +*   - remove core size limits
> +*   - mark the process dumpable again if it lost it due to
> user/group
> +*/
> +   if (global.tune.options & GTUNE_SET_DUMPABLE) {
> +   limit.rlim_cur = limit.rlim_max = RLIM_INFINITY;
> +
> +#if defined(RLIMIT_FSIZE)
> +   if (setrlimit(RLIMIT_FSIZE, ) == -1)
> +   ha_warning("[%s.main()] Failed to set the raise
> the maximum file size.\n", argv[0]);
> +#endif
> +
> +#if defined(RLIMIT_CORE)
> +   if (setrlimit(RLIMIT_CORE, ) == -1)
> +   ha_warning("[%s.main()] Failed to set the raise
> the core dump size.\n", argv[0]);
> +#endif
> +
> +#if defined(USE_PRCTL)
> +   if (prctl(PR_SET_DUMPABLE, 1, 0, 0, 0) == -1)
> +   ha_warning("[%s.main()] Failed to set the dumpable
> flag, no core will be dumped.\n", argv[0]);
> +#endif
> +   }
> +
> global.mode &= ~MODE_STARTING;
> /*
>  * That's it : the central polling loop. Run until we stop.
> --
> 2.24.0
>
>
>

[PATCH] BUG/MINOR: init: fix set-dumpable when using uid/gid