Re: Question on stats socket values

[Abhijeet Rastogi]

Hi Jesse,

By Golang's client's runtime API library, I'm assuming you mean
https://github.com/haproxytech/client-native. However, whatever library you
end up using, your first source of knowledge should come from the HAproxy
documentation itself, and then work your way towards reading API
documentation of a library.

>For example, I know I can use the Golang client’s runtime api to access
the “Econ” value from the stats socket. Does this value only ever increase
for a backend server, or will it be reset at some point?

In this case, as you're concerned about *econ*, refer to the official
documentation here:
https://docs.haproxy.org/dev/management.html#9.3-show%20stat

In the doc, one of the arguments is "json". I typically use "show stat
json" and then look at the output to figure out the type of stat. In this
case, I can see this.

{
  "objType": "Backend",
  "proxyId": 6,
  "id": 0,
  "field": {
"pos": 13,
"name": "econ"
  },
  "processNum": 1,
  "tags": {
"origin": "Metric",
"nature": "Counter",
"scope": "Process"
  },
  "value": {
"type": "u64",
"value": 0
  }
},

The documentation also provides this explanation:

 13. econ [..BS]: number of requests that encountered an error trying to
 connect to a backend server. The backend stat is the sum of the stat
 for all servers of that backend, plus any connection errors not
 associated with a particular server (such as the backend having no
 active servers).

You can use that workflow to figure out the answer. I hope that answered
your question.

Thanks,
Abhijeet


On Thu, Jan 18, 2024 at 3:50 PM Jesse Brink  wrote:

> Hello,
>
> I have a question re: HAProxy’s stats socket. I would like to know how
> often / if their values are reset. Are they monotonically increasing, or
> will they be reset after some period of time if a backend server is not
> taking traffic?
>
> For example, I know I can use the Golang client’s runtime api to access
> the “Econ” value from the stats socket. Does this value only ever increase
> for a backend server, or will it be reset at some point? I am guessing the
> former, but wanted to try to confirm. Thank you.
>
> -Jesse
>


-- 
Cheers,
Abhijeet (https://abhi.host)

Question on stats socket values

[Jesse Brink]

Hello,

I have a question re: HAProxy’s stats socket. I would like to know how often / 
if their values are reset. Are they monotonically increasing, or will they be 
reset after some period of time if a backend server is not taking traffic? 

For example, I know I can use the Golang client’s runtime api to access the 
“Econ” value from the stats socket. Does this value only ever increase for a 
backend server, or will it be reset at some point? I am guessing the former, 
but wanted to try to confirm. Thank you.

-Jesse

Re: show errors from stats socket

[Willy Tarreau]

Hi Elias,

On Mon, Sep 28, 2020 at 12:51:18PM +0200, Elias Abacioglu wrote:
> Hi
> 
> I'm trying to get details about some errors in one of my backends.
> Looking at the stats, there are a bunch of errors per server.
> However, when I try to get details about the errors via the stats socket.
> 
> # echo "show errors -1" | socat stdio /var/lib/haproxy/stats.sock
> Total events captured on [28/Sep/2020:12:44:29.832] : 0
> 
> I get nothing.
> Anyone with a clue on what I might be doing wrong?

That's already a good thing! Errors captured there are unparsable messages
(requests or responses) indicating either attacks from a client, or a
severe bug on a server. What type of errors are you noticing ? It's
possible to have plenty of other types of errors, including closed
connections, resets, timeouts etc, which will not appear there.

Willy

show errors from stats socket

[Elias Abacioglu]

Hi

I'm trying to get details about some errors in one of my backends.
Looking at the stats, there are a bunch of errors per server.
However, when I try to get details about the errors via the stats socket.

# echo "show errors -1" | socat stdio /var/lib/haproxy/stats.sock
Total events captured on [28/Sep/2020:12:44:29.832] : 0

I get nothing.
Anyone with a clue on what I might be doing wrong?

The stats socket has operator level.

Thanks
Elias

Stats socket set map errors out if key does not exist

[Sachin Shetty]

Hi,

As per the documentation (
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-http-request)
stats socket set map should handle adding a new key as well as a updating
an existing key, but my tests show otherwise

echo "set map /somemap.txt abcKey abcValue" | socat stdio
/var/run/haproxy_socket
>>entry not found.

if the key is missing, set map fails.

The stats socket documentation in management section does not mention
anything about this?

Is there a way to check to do a single add Or setIfExists kind of operation
using stats socket?

Thanks
Sachin

Re: Is there a way to extract list of bound IPs via stats socket ?

[Mariusz Gronczewski]

On Fri, 1 Sep 2017 19:07:58 +0200, Willy Tarreau  wrote:

> On Fri, Sep 01, 2017 at 05:49:38PM +0200, Lukas Tribus wrote:
> > Hello,
> > 
> > 
> > Am 01.09.2017 um 15:46 schrieb Mariusz Gronczewski:  
> > > Hi,
> > >
> > > I've been working on a piece of code to announce IPs (via ExaBGP) only if:
> > >
> > > * HAProxy is running
> > > * HAProxy actually uses a given IP
> > > * a frontend with given IP is up for few seconds.
> > >
> > > I could do that via lsof but that's pretty processor-intensive.   
> > 
> > Not sure about the stats or admin socket, but why not use ss instead?
> > 
> > Something like:
> > sudo ss -tln  '( sport = :80 or sport = :443 )'
> > 
> > add "-p" if you need the PID.
> > 
> > Should perform well enough.  
> 
> I think it would not be too hard to add this feature to the CLI. We already
> have "show cli socket" which lists the listening stats sockets. We could
> reuse this code to list all listening sockets and not the just stats ones.
> Maybe "show listeners [optional frontend]" or something like this ?
> 
> Just my two cents,
> Willy

Anyway, if anyone's interested I have a version that just uses ss to get open 
sockets and iproute2 to verify "right" IP actually exists on server:

https://github.com/efigence/go-ha2bgp

It generates exabgp3-compatible announcements + few basic protections (delay 
annoucement to allow service to get up, do not immediately withdraw to allow 
for restarts, withdraw if it is flapping for too long). In theory it should be 
compatible with any service that listens on standard tcp socket (filter by 
default only looks for 80 or 443 ports)

We use it for ECMP setup with 4 boxes, upgraded from "just put some ip addr add 
inside haproxy init script and hope nobody ever stops it" setup


--
Mariusz Gronczewski, Administrator

Efigence S. A.
ul. Wołoska 9a, 02-583 Warszawa
T: [+48] 22 380 13 13
F: [+48] 22 380 13 14
E: mariusz.gronczew...@efigence.com 

Re: Is there a way to extract list of bound IPs via stats socket ?

[Mariusz Gronczewski]

On Fri, 1 Sep 2017 17:49:38 +0200, Lukas Tribus  wrote:

> Hello,
>
>
> Am 01.09.2017 um 15:46 schrieb Mariusz Gronczewski:
> > Hi,
> >
> > I've been working on a piece of code to announce IPs (via ExaBGP) only if:
> >
> > * HAProxy is running
> > * HAProxy actually uses a given IP
> > * a frontend with given IP is up for few seconds.
> >
> > I could do that via lsof but that's pretty processor-intensive.
>
> Not sure about the stats or admin socket, but why not use ss instead?
>
> Something like:
> sudo ss -tln  '( sport = :80 or sport = :443 )'
>
> add "-p" if you need the PID.
>
> Should perform well enough.
>
Huh, interesting.

I just assumed it will be similiar speed no matter which tool I use to get that 
info but ss does that < 100 ms while lsof and netstat take ages:

time lsof -iTCP -sTCP:LISTEN >/dev/null

real0m13.460s
user0m0.201s
sys 0m12.897s

time netstat -l -n -t >/dev/null

real0m43.439s
user0m0.190s
sys 0m42.395s

time  ss -tln  '( sport = :80 or sport = :443 )' >/dev/null

real0m0.032s
user0m0.000s
sys 0m0.032s


Now I know why netstat is getting replaced instead of "just" fixed... thanks.


--
Mariusz Gronczewski, Administrator

Efigence S. A.
ul. Wołoska 9a, 02-583 Warszawa
T: [+48] 22 380 13 13
F: [+48] 22 380 13 14
E: mariusz.gronczew...@efigence.com 

Re: Is there a way to extract list of bound IPs via stats socket ?

[Willy Tarreau]

On Fri, Sep 01, 2017 at 05:49:38PM +0200, Lukas Tribus wrote:
> Hello,
> 
> 
> Am 01.09.2017 um 15:46 schrieb Mariusz Gronczewski:
> > Hi,
> >
> > I've been working on a piece of code to announce IPs (via ExaBGP) only if:
> >
> > * HAProxy is running
> > * HAProxy actually uses a given IP
> > * a frontend with given IP is up for few seconds.
> >
> > I could do that via lsof but that's pretty processor-intensive. 
> 
> Not sure about the stats or admin socket, but why not use ss instead?
> 
> Something like:
> sudo ss -tln  '( sport = :80 or sport = :443 )'
> 
> add "-p" if you need the PID.
> 
> Should perform well enough.

I think it would not be too hard to add this feature to the CLI. We already
have "show cli socket" which lists the listening stats sockets. We could
reuse this code to list all listening sockets and not the just stats ones.
Maybe "show listeners [optional frontend]" or something like this ?

Just my two cents,
Willy

Re: Is there a way to extract list of bound IPs via stats socket ?

[Lukas Tribus]

Hello,


Am 01.09.2017 um 15:46 schrieb Mariusz Gronczewski:
> Hi,
>
> I've been working on a piece of code to announce IPs (via ExaBGP) only if:
>
> * HAProxy is running
> * HAProxy actually uses a given IP
> * a frontend with given IP is up for few seconds.
>
> I could do that via lsof but that's pretty processor-intensive. 

Not sure about the stats or admin socket, but why not use ss instead?

Something like:
sudo ss -tln  '( sport = :80 or sport = :443 )'

add "-p" if you need the PID.

Should perform well enough.



Regards,
Lukas

Is there a way to extract list of bound IPs via stats socket ?

[Mariusz Gronczewski]

Hi,

I've been working on a piece of code to announce IPs (via ExaBGP) only if:

* HAProxy is running
* HAProxy actually uses a given IP
* a frontend with given IP is up for few seconds.

I could do that via lsof but that's pretty processor-intensive. Is there a way 
to extract list of binded IPs (or, running config) via stats socket ? I found a 
way to do that with backend server IPs but I can't seem to find a way to do it 
for frontends.

Cheers, Mariusz
--
Mariusz Gronczewski, Administrator

Efigence S. A.
ul. Wołoska 9a, 02-583 Warszawa
T: [+48] 22 380 13 13
F: [+48] 22 380 13 14
E: mariusz.gronczew...@efigence.com <mailto:mariusz.gronczew...@efigence.com>

Re: in-house vulnerability scan vs. stats socket

[Michael Ezzell]

On Mon, Jun 19, 2017 at 3:34 PM, Jim Freeman <sovr...@gmail.com> wrote:

> FWIW / FYI -
>
> # haproxy -v
> HA-Proxy version 1.5.18 2016/05/10
>
> An in-house vulnerability scanner found our haproxy stats sockets and
> started probing, sending bogus requests, HTTP_* methods, etc.
>
> The many requests, even though the request paths were not valid at the
> stats socket, made for a DoS attack (with haproxy's CPU consumption
> often pegging at 100% generating stats pages).
>
> Since it looks like the only valid stats socket requests are GETs
> ​
>

​Possible point of clarification, it sounds like you're referring to a
listener/frontend with stats enabled.

The stats socket
<http://cbonte.github.io/haproxy-dconv/1.5/configuration.html#9.2> doesn't
speak http.​

RE: in-house vulnerability scan vs. stats socket

[Gibson, Brian (IMS)]

What scanner did you use?

-Original Message-
From: Jim Freeman [sovr...@gmail.com]
Received: Monday, 19 Jun 2017, 3:36PM
To: HAProxy [haproxy@formilux.org]
Subject: in-house vulnerability scan vs. stats socket

FWIW / FYI -

# haproxy -v
HA-Proxy version 1.5.18 2016/05/10

An in-house vulnerability scanner found our haproxy stats sockets and
started probing, sending bogus requests, HTTP_* methods, etc.

The many requests, even though the request paths were not valid at the
stats socket, made for a DoS attack (with haproxy's CPU consumption
often pegging at 100% generating stats pages).

Since it looks like the only valid stats socket requests are GETs to
'/' (with possible ';', '#', and '?' modifiers), we ameliorated the
in-house DoS using these 2 lines in the cfg for the stats socket :

  http-request tarpit unless { path_reg ^/($|\?|\#|\;) }
  http-request tarpit unless METH_GET # silent-drop > 1.5




Information in this e-mail may be confidential. It is intended only for the 
addressee(s) identified above. If you are not the addressee(s), or an employee 
or agent of the addressee(s), please note that any dissemination, distribution, 
or copying of this communication is strictly prohibited. If you have received 
this e-mail in error, please notify the sender of the error.

in-house vulnerability scan vs. stats socket

[Jim Freeman]

FWIW / FYI -

# haproxy -v
HA-Proxy version 1.5.18 2016/05/10

An in-house vulnerability scanner found our haproxy stats sockets and
started probing, sending bogus requests, HTTP_* methods, etc.

The many requests, even though the request paths were not valid at the
stats socket, made for a DoS attack (with haproxy's CPU consumption
often pegging at 100% generating stats pages).

Since it looks like the only valid stats socket requests are GETs to
'/' (with possible ';', '#', and '?' modifiers), we ameliorated the
in-house DoS using these 2 lines in the cfg for the stats socket :

  http-request tarpit unless { path_reg ^/($|\?|\#|\;) }
  http-request tarpit unless METH_GET # silent-drop > 1.5

[PATCH v3 4/9] MEDIUM: mworker: try to guess the next stats socket to use with -x

[William Lallemand]

In master worker mode, you can't specify the stats socket where you get
your listeners FDs on a reload, because the command line of the re-exec
is launched by the master.

To solve the problem, when -x is found on the command line, its
parameter is rewritten on a reexec with the first stats socket with the
capability to send sockets. It tries to reuse the original parameter if
it has this capability.
---
 src/haproxy.c | 73 +++
 1 file changed, 68 insertions(+), 5 deletions(-)

diff --git a/src/haproxy.c b/src/haproxy.c
index c196d51..4e892c0 100644
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -173,6 +173,8 @@ static int oldpids_sig; /* use USR1 or TERM */
 /* Path to the unix socket we use to retrieve listener sockets from the old 
process */
 static const char *old_unixsocket;
 
+static char *cur_unixsocket = NULL;
+
 /* this is used to drain data, and as a temporary buffer for sprintf()... */
 struct chunk trash = { };
 
@@ -512,6 +514,42 @@ int delete_oldpid(int pid)
return 0;
 }
 
+
+static void get_cur_unixsocket()
+{
+   /* if -x was used, try to update the stat socket if not available 
anymore */
+   if (global.stats_fe) {
+   struct bind_conf *bind_conf;
+
+   /* pass through all stats socket */
+   list_for_each_entry(bind_conf, _fe->conf.bind, 
by_fe) {
+   struct listener *l;
+
+   list_for_each_entry(l, _conf->listeners, by_bind) {
+
+   if (l->addr.ss_family == AF_UNIX &&
+   (bind_conf->level & ACCESS_FD_LISTENERS)) {
+   const struct sockaddr_un *un;
+
+   un = (struct sockaddr_un *)>addr;
+   /* priority to old_unixsocket */
+   if (!cur_unixsocket) {
+   cur_unixsocket = 
strdup(un->sun_path);
+   } else {
+   if (old_unixsocket && 
!strcmp(un->sun_path, old_unixsocket)) {
+   free(cur_unixsocket);
+   cur_unixsocket = 
strdup(old_unixsocket);
+   return;
+   }
+   }
+   }
+   }
+   }
+   }
+   if (!cur_unixsocket && old_unixsocket)
+   cur_unixsocket = strdup(old_unixsocket);
+}
+
 /*
  * When called, this function reexec haproxy with -sf followed by current
  * children PIDs and possibily old children PIDs if they didn't leave yet.
@@ -530,8 +568,8 @@ static void mworker_reload()
while (next_argv[next_argc])
next_argc++;
 
-   /* 1 for haproxy -sf */
-   next_argv = realloc(next_argv, (next_argc + 1 + global.nbproc + 
nb_oldpids + 1) * sizeof(char *));
+   /* 1 for haproxy -sf, 2 for -x /socket */
+   next_argv = realloc(next_argv, (next_argc + 1 + 2 + global.nbproc + 
nb_oldpids + 1) * sizeof(char *));
if (next_argv == NULL)
goto alloc_error;
 
@@ -555,6 +593,26 @@ static void mworker_reload()
msg = NULL;
}
next_argv[next_argc] = NULL;
+
+   /* if -x was used, try to update the stat socket if not available 
anymore */
+   if (cur_unixsocket) {
+
+   if (old_unixsocket) {
+
+   /* look for -x  */
+   for (j = 0; next_argv[j]; j++) {
+   if (!strcmp(next_argv[j], "-x"))
+   next_argv[j + 1] = (char 
*)cur_unixsocket;
+   }
+   } else {
+   /* if -x is not specified but we know the socket, add 
-x with it */
+   next_argv[next_argc++] = "-x";
+   next_argv[next_argc++] = (char *)cur_unixsocket;
+   next_argv[next_argc++] = NULL;
+
+   }
+   }
+
deinit(); /* we don't want to leak FD there */
Warning("Reexecuting Master process\n");
execv(next_argv[0], next_argv);
@@ -2216,11 +2274,16 @@ int main(int argc, char **argv)
}
 
if (old_unixsocket) {
-   if (get_old_sockets(old_unixsocket) != 0) {
-   Alert("Failed to get the sockets from the old 
process!\n");
-   exit(1);
+   if (strcmp("/dev/null", old_unixsocket) != 0) {
+   if (get_old_sockets(old_unixsocket) != 0) {
+   Alert("Failed to get 

[PATCH 4/9] MEDIUM: mworker: try to guess the next stats socket to use with -x

[William Lallemand]

In master worker mode, you can't specify the stats socket where you get
your listeners FDs on a reload, because the command line of the re-exec
is launched by the master.

To solve the problem, when -x is found on the command line, its
parameter is rewritten on a reexec with the first stats socket with the
capability to send sockets. It tries to reuse the original parameter if
it has this capability.
---
 src/haproxy.c | 74 +++
 1 file changed, 69 insertions(+), 5 deletions(-)

diff --git a/src/haproxy.c b/src/haproxy.c
index 61114b6..57a5db6 100644
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -175,6 +175,8 @@ static int oldpids_sig; /* use USR1 or TERM */
 /* Path to the unix socket we use to retrieve listener sockets from the old 
process */
 static const char *old_unixsocket;
 
+static char *cur_unixsocket = NULL;
+
 /* this is used to drain data, and as a temporary buffer for sprintf()... */
 struct chunk trash = { };
 
@@ -514,6 +516,42 @@ int delete_oldpid(int pid)
return 0;
 }
 
+
+static void get_cur_unixsocket()
+{
+   /* if -x was used, try to update the stat socket if not available 
anymore */
+   if (global.stats_fe) {
+   struct bind_conf *bind_conf;
+
+   /* pass through all stats socket */
+   list_for_each_entry(bind_conf, _fe->conf.bind, 
by_fe) {
+   struct listener *l;
+
+   list_for_each_entry(l, _conf->listeners, by_bind) {
+
+   if (l->addr.ss_family == AF_UNIX &&
+   (bind_conf->level & ACCESS_FD_LISTENERS)) {
+   const struct sockaddr_un *un;
+
+   un = (struct sockaddr_un *)>addr;
+   /* priority to old_unixsocket */
+   if (!cur_unixsocket) {
+   cur_unixsocket = 
strdup(un->sun_path);
+   } else {
+   if (old_unixsocket && 
!strcmp(un->sun_path, old_unixsocket)) {
+   free(cur_unixsocket);
+   cur_unixsocket = 
strdup(old_unixsocket);
+   return;
+   }
+   }
+   }
+   }
+   }
+   }
+   if (!cur_unixsocket && old_unixsocket)
+   cur_unixsocket = strdup(old_unixsocket);
+}
+
 /*
  * When called, this function reexec haproxy with -sf followed by current
  * children PIDs and possibily old children PIDs if they didn't leave yet.
@@ -532,8 +570,8 @@ static void mworker_reload()
while (next_argv[next_argc])
next_argc++;
 
-   /* 1 for haproxy -sf */
-   next_argv = realloc(next_argv, (next_argc + 1 + global.nbproc + 
nb_oldpids + 1) * sizeof(char *));
+   /* 1 for haproxy -sf, 2 for -x /socket */
+   next_argv = realloc(next_argv, (next_argc + 1 + 2 + global.nbproc + 
nb_oldpids + 1) * sizeof(char *));
if (next_argv == NULL)
goto alloc_error;
 
@@ -557,6 +595,26 @@ static void mworker_reload()
msg = NULL;
}
next_argv[next_argc] = NULL;
+
+   /* if -x was used, try to update the stat socket if not available 
anymore */
+   if (cur_unixsocket) {
+
+   if (old_unixsocket) {
+
+   /* look for -x  */
+   for (j = 0; next_argv[j]; j++) {
+   if (!strcmp(next_argv[j], "-x"))
+   next_argv[j + 1] = (char 
*)cur_unixsocket;
+   }
+   } else {
+   /* if -x is not specified but we know the socket, add 
-x with it */
+   next_argv[next_argc++] = "-x";
+   next_argv[next_argc++] = (char *)cur_unixsocket;
+   next_argv[next_argc++] = NULL;
+
+   }
+   }
+
deinit(); /* we don't want to leak FD there */
Warning("Reexecuting Master process\n");
execv(next_argv[0], next_argv);
@@ -2218,11 +2276,16 @@ int main(int argc, char **argv)
}
 
if (old_unixsocket) {
-   if (get_old_sockets(old_unixsocket) != 0) {
-   Alert("Failed to get the sockets from the old 
process!\n");
-   exit(1);
+   if (strcmp("/dev/null", old_unixsocket) != 0) {
+   if (get_old_sockets(old_unixsocket) != 0) {
+   Alert("Failed to get 

Re: server FQDN changes from stats socket + server-state file

[Willy Tarreau]

On Tue, May 02, 2017 at 04:49:50PM +0200, Frederic Lecaille wrote:
> Here is an update of my patch which integrates your fix.

Now applied, thanks to you both!

Willy

Re: server FQDN changes from stats socket + server-state file

[Frederic Lecaille]

On 05/02/2017 03:45 PM, Baptiste wrote:


Here is a new patch version which takes into an account Baptiste
remarks.

Thank you again Baptiste.


Hi Fred,

I gave a try to your code today and found a segfault at the next DNS
request following the fqdn change.
I attached a patch to this email to fix it, simply merge it into yours.


Here is an update of my patch which integrates your fix.

Fred.

>From 31429d2aefa16c6bd6c83ca2ab60d48cef666c60 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20L=C3=A9caille?= <flecai...@haproxy.com>
Date: Wed, 26 Apr 2017 11:24:02 +0200
Subject: [PATCH] MINOR: server: cli: Add server FQDNs to server-state file and
 stats socket.

This patch adds a new stats socket command to modify server
FQDNs at run time.
Its syntax:
  set server / fqdn 
This patch also adds FQDNs to server state file at the end
of each line for backward compatibility ("-" if not present).
---
 doc/management.txt |   7 +++
 include/types/server.h |  22 ++-
 src/cfgparse.c |   2 -
 src/proxy.c|   4 +-
 src/server.c   | 165 +
 5 files changed, 184 insertions(+), 16 deletions(-)

diff --git a/doc/management.txt b/doc/management.txt
index 3368277..d1ea1f6 100644
--- a/doc/management.txt
+++ b/doc/management.txt
@@ -1650,6 +1650,9 @@ set server / weight [%]
   Change a server's weight to the value passed in argument. This is the exact
   equivalent of the "set weight" command below.
 
+set server / fqdn 
+  Change a server's FQDN to the value passed in argument.
+
 set ssl ocsp-response 
   This command is used to update an OCSP Response for a certificate (see "crt"
   on "bind" lines). Same controls are performed as during the initial loading of
@@ -1962,6 +1965,9 @@ show servers state []
 0x20 = SRV_ADMF_RMAINT
   The server is in maintenance because of an
   IP address resolution failure.
+0x40 = SRV_ADMF_HMAINT
+      The server FQDN was set from stats socket.
+
  srv_uweight: User visible server's weight.
  srv_iweight: Server's initial weight.
  srv_time_since_last_change:  Time since last operational change.
@@ -2003,6 +2009,7 @@ show servers state []
   configuration.
  srv_f_forced_id: Flag to know if the server's ID is forced by
   configuration.
+ srv_fqdn:Server FQDN.
 
 show sess
   Dump all known sessions. Avoid doing this on slow connections as this can
diff --git a/include/types/server.h b/include/types/server.h
index 8d68dcb..83b1e80 100644
--- a/include/types/server.h
+++ b/include/types/server.h
@@ -82,6 +82,7 @@ enum srv_admin {
 	SRV_ADMF_IDRAIN= 0x10,/* the server has inherited the drain status from a tracked server */
 	SRV_ADMF_DRAIN = 0x18,/* mask to check if any drain flag is present */
 	SRV_ADMF_RMAINT= 0x20,/* the server is down because of an IP address resolution failure */
+	SRV_ADMF_HMAINT= 0x40,/* the server FQDN has been set from socket stats */
 };
 
 /* options for servers' "init-addr" parameter
@@ -103,7 +104,26 @@ enum srv_initaddr {
 #define SRV_STATE_FILE_VERSION 1
 #define SRV_STATE_FILE_VERSION_MIN 1
 #define SRV_STATE_FILE_VERSION_MAX 1
-#define SRV_STATE_FILE_FIELD_NAMES "be_id be_name srv_id srv_name srv_addr srv_op_state srv_admin_state srv_uweight srv_iweight srv_time_since_last_change srv_check_status srv_check_result srv_check_health srv_check_state srv_agent_state bk_f_forced_id srv_f_forced_id"
+#define SRV_STATE_FILE_FIELD_NAMES \
+"be_id "  \
+"be_name "\
+"srv_id " \
+"srv_name "   \
+"srv_addr "   \
+"srv_op_state "   \
+"srv_admin_state "\
+"srv_uweight "\
+"srv_iweight "\
+"srv_time_since_last_change " \
+"srv_check_status "   \
+"srv_check_result "   \
+"srv_check_health "   \
+"srv_check_state "\
+"srv_agent_state "\
+"bk_f_forced_id " \
+"srv_f_forced_id "\
+"srv_fqdn"
+
 #define SRV_STATE_FILE_MAX_FIELDS 18
 #define SRV_STATE_FILE_NB_FIELDS_VERSION_1 18
 #define SRV_STATE_LINE_MAXLEN 512
diff --git a/src/cfgparse.c b/src/cfgparse.c
index d44949a..f167321 100644
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -8492,8 +8492,6 @@ out_uri_auth_compat:

Re: server FQDN changes from stats socket + server-state file

[Baptiste]

Appart the bug, the new feature works smoothly!

Great job, Fred @haproxy.com :p

Baptiste

Re: server FQDN changes from stats socket + server-state file

[Frederic Lecaille]

On 05/02/2017 03:45 PM, Baptiste wrote:


Here is a new patch version which takes into an account Baptiste
remarks.

Thank you again Baptiste.


Hi Fred,


Hello Baptiste,


I gave a try to your code today and found a segfault at the next DNS
request following the fqdn change.
I attached a patch to this email to fix it, simply merge it into yours.



Ok I will have a look to this patch asap.


Note that my fix is temporary, since I'm reorganizing the DNS
structures. The ugly foreach loop will disapear once I'll get my patches
merged.

Baptiste



Thank you for reviewing/testing Baptiste.

Fred.

Re: server FQDN changes from stats socket + server-state file

[Baptiste]

>
>
>> Here is a new patch version which takes into an account Baptiste remarks.
>
> Thank you again Baptiste.
>
>
Hi Fred,

I gave a try to your code today and found a segfault at the next DNS
request following the fqdn change.
I attached a patch to this email to fix it, simply merge it into yours.

Note that my fix is temporary, since I'm reorganizing the DNS structures.
The ugly foreach loop will disapear once I'll get my patches merged.

Baptiste
From d44a8bfd8a1f0e49e53a71b0b69274579477fe76 Mon Sep 17 00:00:00 2001
From: Baptiste Assmann 
Date: Tue, 2 May 2017 15:40:37 +0200
Subject: [PATCH 2/2] FIX for fred

---
 src/cfgparse.c |  2 --
 src/server.c   | 17 +
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/src/cfgparse.c b/src/cfgparse.c
index d44949a..f167321 100644
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -8492,8 +8492,6 @@ out_uri_auth_compat:
 	newsrv->id, newsrv->resolvers_id);
 	cfgerr++;
 } else {
-	free(newsrv->resolvers_id);
-	newsrv->resolvers_id = NULL;
 	if (newsrv->resolution)
 		newsrv->resolution->resolvers = curr_resolvers;
 }
diff --git a/src/server.c b/src/server.c
index 536ab6b..85da083 100644
--- a/src/server.c
+++ b/src/server.c
@@ -1690,6 +1690,23 @@ static int srv_alloc_dns_resolution(struct server *srv, const char *hostname)
 	/* a first resolution has been done by the configuration parser */
 	srv->resolution->last_resolution = 0;
 
+	if (srv->resolvers_id) {
+		struct dns_resolvers *curr_resolvers;
+		int found = 0;
+
+		list_for_each_entry(curr_resolvers, _resolvers, list) {
+			if (!strcmp(curr_resolvers->id, srv->resolvers_id)) {
+found = 1;
+break;
+			}
+		}
+		if (!found)
+			goto err;
+		srv->resolution->resolvers = curr_resolvers;
+	}
+	else
+		goto err;
+
 	return 0;
 
  err:
-- 
2.7.4

Re: server FQDN changes from stats socket + server-state file

[Frederic Lecaille]

On 04/27/2017 03:20 PM, Frederic Lecaille wrote:

On 04/27/2017 02:56 PM, Baptiste wrote:



On Thu, Apr 27, 2017 at 2:44 PM, Frederic Lecaille
<flecai...@haproxy.com <mailto:flecai...@haproxy.com>> wrote:

On 04/27/2017 12:43 PM, Baptiste wrote:



On Thu, Apr 27, 2017 at 11:22 AM, Frederic Lecaille
<flecai...@haproxy.com <mailto:flecai...@haproxy.com>
<mailto:flecai...@haproxy.com <mailto:flecai...@haproxy.com>>>
wrote:

On 04/27/2017 10:47 AM, Frederic Lecaille wrote:

Hello HAProxy ML,

Please find attached to this mail a patch proposal which
allows
    server FQDNs changes from stats socket.

These FQDNs are also added to server-state file.

Regards,

Fred.


A new version of this patch which fixes a memleak (server
hostname
was strdup() both in srv_set_fqdn() and
srv_alloc_dns_resolution()).



Hi Fred,


Hi Baptiste,

I did a quick read of the patch and I noticed the following
points:

- in update_server_fqdn(), I don't think it's relevant to
perform the
name resolution validation (the call to str2ip2)
  First, str2ip2 uses libc resolver, which may be different
from the
runtime resolver of HAProxy and second, it's up to the
admin/devops/script which performs this change to ensure he is
not
messing up...


Ok but will the new FQDN be resolved in any case after any update
from stats socket to match with the server IP address or is this
something the operator must also take care of (provide new FQDNs
which may resolve to the current IP address???)?


From my point of view, if no runtime resolution is configured, then
changing the fqdn through the CLI should have no impact (since this is
HAProxy's behavior since the beginning). Unless I misunderstood the
purpose of this feature...
By the way, I think that the call to getaddrinfo() in str2ip2 is
blocking... Hence, if I'm not wrong about getaddrinfo, it's a definitive
no-go to call str2ip2 at runtime.


I do not know all the internals of haproxy but I agree that if str2ip2()
may make any haproxy internal processing block I have definitively to
remove this call to str2ip2(). ;)

And modify two lines which launch another call 2 str2ip2() (see the
comment: /* Reset lastaddr value. etc. */).

Thank you Baptiste.


Here is a new patch version which takes into an account Baptiste remarks.

Thank you again Baptiste.




>From 54ce2b25e74c4b95a86d3da87b5bea261a7183c5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20L=C3=A9caille?= <flecai...@haproxy.com>
Date: Wed, 26 Apr 2017 11:24:02 +0200
Subject: [PATCH] MINOR: server: cli: Add server FQDNs to server-state file and
 stats socket.

This patch adds a new stats socket command to modify server
FQDNs at run time.
Its syntax:
  set server / fqdn 
This patch also adds FQDNs to server state file at the end
of each line for backward compatibility ("-" if not present).
---
 doc/management.txt |   7 +++
 include/types/server.h |  22 +++-
 src/proxy.c|   4 +-
 src/server.c   | 148 +
 4 files changed, 167 insertions(+), 14 deletions(-)

diff --git a/doc/management.txt b/doc/management.txt
index 3368277..d1ea1f6 100644
--- a/doc/management.txt
+++ b/doc/management.txt
@@ -1650,6 +1650,9 @@ set server / weight [%]
   Change a server's weight to the value passed in argument. This is the exact
   equivalent of the "set weight" command below.
 
+set server / fqdn 
+  Change a server's FQDN to the value passed in argument.
+
 set ssl ocsp-response 
   This command is used to update an OCSP Response for a certificate (see "crt"
   on "bind" lines). Same controls are performed as during the initial loading of
@@ -1962,6 +1965,9 @@ show servers state []
 0x20 = SRV_ADMF_RMAINT
   The server is in maintenance because of an
   IP address resolution failure.
+0x40 = SRV_ADMF_HMAINT
+  The server FQDN was set from stats socket.
+
  srv_uweight: User visible server's weight.
  srv_iweight: Server's initial weight.
  srv_time_since_last_change:  Time since last operational change.
@@ -2003,6 +2009,7 @@ show servers state []
   configuration.
  srv_f_forced_id: Flag to know if the server's ID is forced by
   configuration.
+ srv_fqdn:Server FQDN.
 
 show sess
   Dump all known sessions. Avoid doing this on slow connections as this can
diff --git a/include/types/server.

Re: server FQDN changes from stats socket + server-state file

[Frederic Lecaille]

On 04/27/2017 02:56 PM, Baptiste wrote:



On Thu, Apr 27, 2017 at 2:44 PM, Frederic Lecaille
<flecai...@haproxy.com <mailto:flecai...@haproxy.com>> wrote:

On 04/27/2017 12:43 PM, Baptiste wrote:



On Thu, Apr 27, 2017 at 11:22 AM, Frederic Lecaille
<flecai...@haproxy.com <mailto:flecai...@haproxy.com>
<mailto:flecai...@haproxy.com <mailto:flecai...@haproxy.com>>>
wrote:

On 04/27/2017 10:47 AM, Frederic Lecaille wrote:

Hello HAProxy ML,

Please find attached to this mail a patch proposal which
allows
    server FQDNs changes from stats socket.

These FQDNs are also added to server-state file.

Regards,

Fred.


A new version of this patch which fixes a memleak (server
hostname
was strdup() both in srv_set_fqdn() and
srv_alloc_dns_resolution()).



Hi Fred,


Hi Baptiste,

I did a quick read of the patch and I noticed the following points:

- in update_server_fqdn(), I don't think it's relevant to
perform the
name resolution validation (the call to str2ip2)
  First, str2ip2 uses libc resolver, which may be different from the
runtime resolver of HAProxy and second, it's up to the
admin/devops/script which performs this change to ensure he is not
messing up...


Ok but will the new FQDN be resolved in any case after any update
from stats socket to match with the server IP address or is this
something the operator must also take care of (provide new FQDNs
which may resolve to the current IP address???)?


From my point of view, if no runtime resolution is configured, then
changing the fqdn through the CLI should have no impact (since this is
HAProxy's behavior since the beginning). Unless I misunderstood the
purpose of this feature...
By the way, I think that the call to getaddrinfo() in str2ip2 is
blocking... Hence, if I'm not wrong about getaddrinfo, it's a definitive
no-go to call str2ip2 at runtime.


I do not know all the internals of haproxy but I agree that if str2ip2() 
may make any haproxy internal processing block I have definitively to 
remove this call to str2ip2(). ;)


And modify two lines which launch another call 2 str2ip2() (see the 
comment: /* Reset lastaddr value. etc. */).


Thank you Baptiste.

Re: server FQDN changes from stats socket + server-state file

[Baptiste]

On Thu, Apr 27, 2017 at 2:44 PM, Frederic Lecaille <flecai...@haproxy.com>
wrote:

> On 04/27/2017 12:43 PM, Baptiste wrote:
>
>>
>>
>> On Thu, Apr 27, 2017 at 11:22 AM, Frederic Lecaille
>> <flecai...@haproxy.com <mailto:flecai...@haproxy.com>> wrote:
>>
>> On 04/27/2017 10:47 AM, Frederic Lecaille wrote:
>>
>> Hello HAProxy ML,
>>
>> Please find attached to this mail a patch proposal which allows
>> server FQDNs changes from stats socket.
>>
>> These FQDNs are also added to server-state file.
>>
>> Regards,
>>
>> Fred.
>>
>>
>> A new version of this patch which fixes a memleak (server hostname
>> was strdup() both in srv_set_fqdn() and srv_alloc_dns_resolution()).
>>
>>
>>
>> Hi Fred,
>>
>
> Hi Baptiste,
>
> I did a quick read of the patch and I noticed the following points:
>>
>> - in update_server_fqdn(), I don't think it's relevant to perform the
>> name resolution validation (the call to str2ip2)
>>   First, str2ip2 uses libc resolver, which may be different from the
>> runtime resolver of HAProxy and second, it's up to the
>> admin/devops/script which performs this change to ensure he is not
>> messing up...
>>
>
> Ok but will the new FQDN be resolved in any case after any update from
> stats socket to match with the server IP address or is this something the
> operator must also take care of (provide new FQDNs which may resolve to the
> current IP address???)?


>From my point of view, if no runtime resolution is configured, then
changing the fqdn through the CLI should have no impact (since this is
HAProxy's behavior since the beginning). Unless I misunderstood the purpose
of this feature...
By the way, I think that the call to getaddrinfo() in str2ip2 is
blocking... Hence, if I'm not wrong about getaddrinfo, it's a definitive
no-go to call str2ip2 at runtime.

Baptiste

Re: server FQDN changes from stats socket + server-state file

[Frederic Lecaille]

On 04/27/2017 12:43 PM, Baptiste wrote:



On Thu, Apr 27, 2017 at 11:22 AM, Frederic Lecaille
<flecai...@haproxy.com <mailto:flecai...@haproxy.com>> wrote:

On 04/27/2017 10:47 AM, Frederic Lecaille wrote:

Hello HAProxy ML,

Please find attached to this mail a patch proposal which allows
server FQDNs changes from stats socket.

These FQDNs are also added to server-state file.

Regards,

Fred.


A new version of this patch which fixes a memleak (server hostname
was strdup() both in srv_set_fqdn() and srv_alloc_dns_resolution()).



Hi Fred,


Hi Baptiste,


I did a quick read of the patch and I noticed the following points:

- in update_server_fqdn(), I don't think it's relevant to perform the
name resolution validation (the call to str2ip2)
  First, str2ip2 uses libc resolver, which may be different from the
runtime resolver of HAProxy and second, it's up to the
admin/devops/script which performs this change to ensure he is not
messing up...


Ok but will the new FQDN be resolved in any case after any update from 
stats socket to match with the server IP address or is this something 
the operator must also take care of (provide new FQDNs which may resolve 
to the current IP address???)?



- in srv_alloc_dns_resolution(), if strdup (or any alloc function)
fails, then we should report an error to the function caller and display
a message on the CLI.


Ok.


Baptiste

Re: server FQDN changes from stats socket + server-state file

[Baptiste]

On Thu, Apr 27, 2017 at 11:22 AM, Frederic Lecaille <flecai...@haproxy.com>
wrote:

> On 04/27/2017 10:47 AM, Frederic Lecaille wrote:
>
>> Hello HAProxy ML,
>>
>> Please find attached to this mail a patch proposal which allows
>> server FQDNs changes from stats socket.
>>
>> These FQDNs are also added to server-state file.
>>
>> Regards,
>>
>> Fred.
>>
>
> A new version of this patch which fixes a memleak (server hostname
> was strdup() both in srv_set_fqdn() and srv_alloc_dns_resolution()).
>
>

Hi Fred,

I did a quick read of the patch and I noticed the following points:

- in update_server_fqdn(), I don't think it's relevant to perform the name
resolution validation (the call to str2ip2)
  First, str2ip2 uses libc resolver, which may be different from the
runtime resolver of HAProxy and second, it's up to the admin/devops/script
which performs this change to ensure he is not messing up...

- in srv_alloc_dns_resolution(), if strdup (or any alloc function) fails,
then we should report an error to the function caller and display a message
on the CLI.

Baptiste

Re: server FQDN changes from stats socket + server-state file

[Frederic Lecaille]

On 04/27/2017 10:47 AM, Frederic Lecaille wrote:

Hello HAProxy ML,

Please find attached to this mail a patch proposal which allows
server FQDNs changes from stats socket.

These FQDNs are also added to server-state file.

Regards,

Fred.


A new version of this patch which fixes a memleak (server hostname
was strdup() both in srv_set_fqdn() and srv_alloc_dns_resolution()).




>From f6adc66337655cf81a29a519dff6c40d3d24013d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20L=C3=A9caille?= <flecai...@haproxy.com>
Date: Wed, 26 Apr 2017 11:24:02 +0200
Subject: [PATCH] MINOR: server: cli: Add server FQDNs to server-state file and
 stats socket.

This patch adds a new stats socket command to modify server
FQDNs at run time.
Its syntax:
  set server / fqdn 
This patch also adds FQDNs to server state file at the end
of each line for backward compatibility ("-" if not present).
---
 doc/management.txt |   7 +++
 include/types/server.h |  22 +++-
 src/proxy.c|   4 +-
 src/server.c   | 144 ++---
 4 files changed, 166 insertions(+), 11 deletions(-)

diff --git a/doc/management.txt b/doc/management.txt
index 3368277..d1ea1f6 100644
--- a/doc/management.txt
+++ b/doc/management.txt
@@ -1650,6 +1650,9 @@ set server / weight [%]
   Change a server's weight to the value passed in argument. This is the exact
   equivalent of the "set weight" command below.
 
+set server / fqdn 
+  Change a server's FQDN to the value passed in argument.
+
 set ssl ocsp-response 
   This command is used to update an OCSP Response for a certificate (see "crt"
   on "bind" lines). Same controls are performed as during the initial loading of
@@ -1962,6 +1965,9 @@ show servers state []
 0x20 = SRV_ADMF_RMAINT
   The server is in maintenance because of an
   IP address resolution failure.
+0x40 = SRV_ADMF_HMAINT
+      The server FQDN was set from stats socket.
+
  srv_uweight: User visible server's weight.
  srv_iweight: Server's initial weight.
  srv_time_since_last_change:  Time since last operational change.
@@ -2003,6 +2009,7 @@ show servers state []
   configuration.
  srv_f_forced_id: Flag to know if the server's ID is forced by
   configuration.
+ srv_fqdn:Server FQDN.
 
 show sess
   Dump all known sessions. Avoid doing this on slow connections as this can
diff --git a/include/types/server.h b/include/types/server.h
index 8d68dcb..83b1e80 100644
--- a/include/types/server.h
+++ b/include/types/server.h
@@ -82,6 +82,7 @@ enum srv_admin {
 	SRV_ADMF_IDRAIN= 0x10,/* the server has inherited the drain status from a tracked server */
 	SRV_ADMF_DRAIN = 0x18,/* mask to check if any drain flag is present */
 	SRV_ADMF_RMAINT= 0x20,/* the server is down because of an IP address resolution failure */
+	SRV_ADMF_HMAINT= 0x40,/* the server FQDN has been set from socket stats */
 };
 
 /* options for servers' "init-addr" parameter
@@ -103,7 +104,26 @@ enum srv_initaddr {
 #define SRV_STATE_FILE_VERSION 1
 #define SRV_STATE_FILE_VERSION_MIN 1
 #define SRV_STATE_FILE_VERSION_MAX 1
-#define SRV_STATE_FILE_FIELD_NAMES "be_id be_name srv_id srv_name srv_addr srv_op_state srv_admin_state srv_uweight srv_iweight srv_time_since_last_change srv_check_status srv_check_result srv_check_health srv_check_state srv_agent_state bk_f_forced_id srv_f_forced_id"
+#define SRV_STATE_FILE_FIELD_NAMES \
+"be_id "  \
+"be_name "\
+"srv_id " \
+"srv_name "   \
+"srv_addr "   \
+"srv_op_state "   \
+"srv_admin_state "\
+"srv_uweight "\
+"srv_iweight "\
+"srv_time_since_last_change " \
+"srv_check_status "   \
+"srv_check_result "   \
+"srv_check_health "   \
+"srv_check_state "\
+"srv_agent_state "\
+"bk_f_forced_id " \
+"srv_f_forced_id "\
+"srv_fqdn"
+
 #define SRV_STATE_FILE_MAX_FIELDS 18
 #define SRV_STATE_FILE_NB_FIELDS_VERSION_1 18
 #define SRV_STATE_LINE_MAXLEN 512
diff --git a/src/proxy.c b/src/proxy.c
index dc70213..f6a3214 100644
--- a/src/proxy.c
+++ b/src/proxy.c
@@ -1458,13 +1458,13 @@ static int dump_servers_state(struct stream_interface *si, struct chunk *buf)
 "%d %s

server FQDN changes from stats socket + server-state file

[Frederic Lecaille]

Hello HAProxy ML,

Please find attached to this mail a patch proposal which allows
server FQDNs changes from stats socket.

These FQDNs are also added to server-state file.

Regards,

Fred.
>From f9c1001175d406a15414e893f11d6120cf22 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20L=C3=A9caille?= <flecai...@haproxy.com>
Date: Wed, 26 Apr 2017 11:24:02 +0200
Subject: [PATCH] MINOR: server: cli: Add server FQDNs to server-state file and
 stats socket.

This patch adds a new stats socket command to modify server
FQDNs at run time.
Its syntax:
  set server / fqdn 
This patch also adds FQDNs to server state file at the end
of each line for backward compatibility ("-" if not present).
---
 doc/management.txt |   7 +++
 include/types/server.h |  22 +++-
 src/proxy.c|   4 +-
 src/server.c   | 145 ++---
 4 files changed, 167 insertions(+), 11 deletions(-)

diff --git a/doc/management.txt b/doc/management.txt
index 3368277..d1ea1f6 100644
--- a/doc/management.txt
+++ b/doc/management.txt
@@ -1650,6 +1650,9 @@ set server / weight [%]
   Change a server's weight to the value passed in argument. This is the exact
   equivalent of the "set weight" command below.
 
+set server / fqdn 
+  Change a server's FQDN to the value passed in argument.
+
 set ssl ocsp-response 
   This command is used to update an OCSP Response for a certificate (see "crt"
   on "bind" lines). Same controls are performed as during the initial loading of
@@ -1962,6 +1965,9 @@ show servers state []
 0x20 = SRV_ADMF_RMAINT
   The server is in maintenance because of an
   IP address resolution failure.
+0x40 = SRV_ADMF_HMAINT
+      The server FQDN was set from stats socket.
+
  srv_uweight: User visible server's weight.
  srv_iweight: Server's initial weight.
  srv_time_since_last_change:  Time since last operational change.
@@ -2003,6 +2009,7 @@ show servers state []
   configuration.
  srv_f_forced_id: Flag to know if the server's ID is forced by
   configuration.
+ srv_fqdn:Server FQDN.
 
 show sess
   Dump all known sessions. Avoid doing this on slow connections as this can
diff --git a/include/types/server.h b/include/types/server.h
index 8d68dcb..83b1e80 100644
--- a/include/types/server.h
+++ b/include/types/server.h
@@ -82,6 +82,7 @@ enum srv_admin {
 	SRV_ADMF_IDRAIN= 0x10,/* the server has inherited the drain status from a tracked server */
 	SRV_ADMF_DRAIN = 0x18,/* mask to check if any drain flag is present */
 	SRV_ADMF_RMAINT= 0x20,/* the server is down because of an IP address resolution failure */
+	SRV_ADMF_HMAINT= 0x40,/* the server FQDN has been set from socket stats */
 };
 
 /* options for servers' "init-addr" parameter
@@ -103,7 +104,26 @@ enum srv_initaddr {
 #define SRV_STATE_FILE_VERSION 1
 #define SRV_STATE_FILE_VERSION_MIN 1
 #define SRV_STATE_FILE_VERSION_MAX 1
-#define SRV_STATE_FILE_FIELD_NAMES "be_id be_name srv_id srv_name srv_addr srv_op_state srv_admin_state srv_uweight srv_iweight srv_time_since_last_change srv_check_status srv_check_result srv_check_health srv_check_state srv_agent_state bk_f_forced_id srv_f_forced_id"
+#define SRV_STATE_FILE_FIELD_NAMES \
+"be_id "  \
+"be_name "\
+"srv_id " \
+"srv_name "   \
+"srv_addr "   \
+"srv_op_state "   \
+"srv_admin_state "\
+"srv_uweight "\
+"srv_iweight "\
+"srv_time_since_last_change " \
+"srv_check_status "   \
+"srv_check_result "   \
+"srv_check_health "   \
+"srv_check_state "\
+"srv_agent_state "\
+"bk_f_forced_id " \
+"srv_f_forced_id "\
+"srv_fqdn"
+
 #define SRV_STATE_FILE_MAX_FIELDS 18
 #define SRV_STATE_FILE_NB_FIELDS_VERSION_1 18
 #define SRV_STATE_LINE_MAXLEN 512
diff --git a/src/proxy.c b/src/proxy.c
index dc70213..f6a3214 100644
--- a/src/proxy.c
+++ b/src/proxy.c
@@ -1458,13 +1458,13 @@ static int dump_servers_state(struct stream_interface *si, struct chunk *buf)
 "%d %s %s "
 "%d %d %d %d %ld "
 "%d %d %d %d %d "
-"%d %d"
+"%d %d %s"
 "\n",
 px->uuid, px->id,
 srv-

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

Hi,

On Mon, Nov 14, 2016 at 03:29:58PM +, Mirek Svoboda wrote:
> What if we have the descriptions in the source code, serving as a single
> source of truth, and generate the JSON schema file from the source code
> upon build?

... or on the fly. That's what I was thinking as well. Ie
"show stats json-schema" and use that output.

> There might be also another use case for the descriptions in the source
> code in the future, though cannot come with an example now.

Clearly the source code doesn't need the descriptions, however it's the
easiest place to ensure consistency. When you add a new field and you
only have to type 5 words in a 3rd column, you have no excuse for not
doing it. When you have to open a file you don't know exists, or to try
to remember what file this was because you remember being instructed to
do so in the past, it's quite different.

Regards,
Willy

Re: Getting JSON encoded data from the stats socket.

[Mirek Svoboda]

Hi

> > OK. So does this mean that a schema will have to be maintained by hand in
> > parallel or will it be deduced from the dump ? I'm starting to be worried
> > about something not being kept up to date if we have to maintain it, or
> > causing a slow down in adoption of new stats entries.
>
> I envisage the schema being maintained in the same way that documentation
> is. In the draft schema I posted it should not be necessary to update each
> time a new item is added to the output of show flow or show info. Rather,
> the schema would need to be updated if the format of the data changes some
> how: f.e. a new field is added which would be analagous to adding a new
> column to the output of typed output, or a new type of value, such as u16,
> was added.
>

What if we have the descriptions in the source code, serving as a single
source of truth, and generate the JSON schema file from the source code
upon build?
There might be also another use case for the descriptions in the source
code in the future, though cannot come with an example now.

Regards,
Mirek Svoboda

>

Re: Getting JSON encoded data from the stats socket.

[Simon Horman]

On Mon, Nov 14, 2016 at 08:50:54AM -0500, hapr...@stormcloud9.net wrote:
> Might help to see an example of what the results look like when using
> this schema, however I do have one comment below.

Yes, agreed. I plan to work on making that so.

> On 2016/11/14 03:09, Simon Horman wrote:
> > Hi Willy, Hi All,
> >
> > On Thu, Nov 10, 2016 at 04:52:56PM +0100, Willy Tarreau wrote:
> >> Hi Simon!
> >>
> >> On Thu, Nov 10, 2016 at 04:27:15PM +0100, Simon Horman wrote:
> >>> My preference is to take things calmly as TBH I am only just getting
> >>> started on this and I think the schema could take a little time to get
> >>> a consensus on.
> >> I totally agree with you. I think the most difficult thing is not to
> >> run over a few arrays and dump them but manage to make everyone agree
> >> on the schema. And that will take more than a few days I guess. Anyway
> >> I'm fine with being proven wrong :-)
> > I took a first pass at defining a schema.
> >
> > * The schema follows what is described on json-schema.org (or at least
> >   tries to). Is this a suitable approach?
> > * The schema only covers "show info" and "show stat" and the fields
> >   are based on the typed output variants of those commands.
> >   This leads me to several questions:
> >   - Is this field selection desirable? It seems to make sense to me
> > as presumably the intention of the JSON output is for it to
> > be machine readable.
> >   - Is such an approach appropriate for other show commands?
> >   - And more generally, which other show commands are desired to
> > support output in JSON (in the near term)?
> >
> > {
> > "$schema": "http://json-schema.org/draft-04/schema#;,
> > "oneOf": [
> > {
> > "title": "Info",
> > "description": "Info about HAProxy status",
> > "type": "array",
> > "items": {
> > "properties": {
> > "title": "Info Item",
> > "type": "object",
> > "field": { "$ref": "#/definitions/field" },
> > "processNum": { "$ref": "#/definitions/processNum" },
> > "tags": { "$ref": "#/definitions/tags" },
> > "value": { "$ref": "#/definitions/typedValue" }
> > },
> > "required": ["field", "processNum", "tags", "value"]
> > }
> > },
> > {
> > "title": "Stat",
> > "description": "HAProxy statistics",
> > "type": "array",
> > "items": {
> > "title": "Info Item",
> > "type": "object",
> > "properties": {
> > "objType": {
> > "enum": ["F", // Frontend
> >  "B", // Backend
> >  "L", // Listener
> >  "S"  // Server
> Do we really need to save a few bytes and abbreviate these? We're
> already far more chatty than the CSV output as you're outputting field
> names (e.g. "proxyId" and "processNum"), so abbreviating the values when
> you've got full field names seems rather contrary. And then as you've
> demonstrated, this requires defining a "sub-schema" for explaining what
> "F", "B", etc, are. Thus requiring anyone parsing the json to have to
> keep a mapping of the values (and do the translation) within their code.
> Ditto for all the other "enum" types down below.

Good point. I'm not sure why that didn't occur to me.
But it does seem like a good idea.

> > ]
> > },
> > "proxyId": {
> > "type": "integer",
> > "minimum": 0
> > },
> > "id": {
> > "description": "Unique identifyier of object within 
> > proxy",
> > "type": "integer",
> > "minimum": 0
> > },
> > "field": { "$ref": "#/definitions/field" },
> > "processNum": { "$ref": "#/definitions/processNum" },
> > "tags": { "$ref": "#/definitions/tags" },
> > "typedValue": { "$ref": "#/definitions/typedValue" }
> > },
> > "required": ["objType", "proxyId", "id", "field", 
> > "processNum",
> >  "tags", "value"]
> > }
> > }
> > ],
> > "definitions": {
> > "field": {
> > "type": "object",
> > "pos": {
> > "description": "Position of field",
> > "type": "integer",
> > "minimum": 0
> > },
> > "name": {
> > "description": "Name of field",
> > "type": "string"
> > },
> > "required": ["pos", "name"]
> > },
> > "processNum": {
> > 

Re: Getting JSON encoded data from the stats socket.

[Simon Horman]

Hi Willy,

On Mon, Nov 14, 2016 at 03:10:18PM +0100, Willy Tarreau wrote:
> On Mon, Nov 14, 2016 at 11:34:18AM +0100, Simon Horman wrote:
> > > Sometimes a description like above appears in your example, is it just
> > > for a few fields or do you intend to describe all of them ? I'm asking
> > > because we don't have such descriptions right now, and while I won't
> > > deny that forcing contributors to add one when adding new stats could be
> > > reasonable (it's like doc), I fear that it would significantly inflate
> > > the output.
> > 
> > My understanding is that the description is part of the schema but would
> > not be included in a JSON instance. Or on other words, would not
> > be included in the output of a show command.
> 
> OK. So does this mean that a schema will have to be maintained by hand in
> parallel or will it be deduced from the dump ? I'm starting to be worried
> about something not being kept up to date if we have to maintain it, or
> causing a slow down in adoption of new stats entries.

I envisage the schema being maintained in the same way that documentation
is. In the draft schema I posted it should not be necessary to update each
time a new item is added to the output of show flow or show info. Rather,
the schema would need to be updated if the format of the data changes some
how: f.e. a new field is added which would be analagous to adding a new
column to the output of typed output, or a new type of value, such as u16,
was added.

> > My intention was to add descriptions for all fields. But in many case
> > the field name seemed to be sufficiently descriptive or at least I couldn't
> > think of a better description. And in such cases I omitted the description
> > to avoid being repetitive.
> 
> OK that's a good point. So we can possibly have a first implementation reusing
> the field name everywhere, and later make these descriptions mandatory in the
> code for new fields so that the output description becomes more readable.
> 
> > I do not feel strongly about the descriptions. I'm happy to remove some or
> > all of them if they are deemed unnecessary or otherwise undesirable; to add
> > them to every field for consistency; or something in between.
> 
> I think dumping only known descriptions and falling back to the name (or
> simply suggesting that the consumer just uses the same when there's no desc)
> sounds reasonable to me for now.
> 
> > > Also, do you have an idea about the verbosity of the dump here ? For
> > > example let's say you have 100 listeners with 4 servers each (which is
> > > an average sized config). I'm just looking for a rought order of 
> > > magnitude,
> > > ie closer to 10-100k or to 1-10M. The typed output is already quite heavy
> > > for large configs so it should not be a big deal, but it's something we
> > > have to keep in mind.
> > 
> > I don't think the type, description, etc... should be included in such
> > output as they can be supplied by the schema out-of-band. But the field
> > name and values along with syntactic elements (brackets, quotes, etc...) do
> > need to be included.
> 
> OK.
> 
> > I can try and come up with an estimate if it is
> > important but my guess is the result would be several times the size of the
> > typed output (mainly owing to the size of the field names in the output).
> 
> No, don't worry, this rough estimate is enough.

-- 
Simon Horman  si...@horms.nl
Horms Solutions BV  www.horms.nl
Parnassusweg 819, 1082 LZ Amsterdam, Netherlands
Tel: +31 (0)20 800 6155Skype: horms7

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

Hi Simon,

On Mon, Nov 14, 2016 at 09:09:21AM +0100, Simon Horman wrote:
> I took a first pass at defining a schema.
> 
> * The schema follows what is described on json-schema.org (or at least
>   tries to). Is this a suitable approach?

I'll let others respond as I have no idea since I never need nor use JSON :-)

> * The schema only covers "show info" and "show stat" and the fields
>   are based on the typed output variants of those commands.
>   This leads me to several questions:
>   - Is this field selection desirable? It seems to make sense to me
> as presumably the intention of the JSON output is for it to
> be machine readable.

Yes in my opinion it's the goal. And these are the two only parts that
were converted to typed output for this reason.

>   - Is such an approach appropriate for other show commands?

At the moment I don't think so because the other ones are more related
to state management than statistics.

>   - And more generally, which other show commands are desired to
> support output in JSON (in the near term)?

I can't think of any right now.

However I have a question below :

> "id": {
> "description": "Unique identifyier of object within 
> proxy",
> "type": "integer",
> "minimum": 0
> },

Sometimes a description like above appears in your example, is it just
for a few fields or do you intend to describe all of them ? I'm asking
because we don't have such descriptions right now, and while I won't
deny that forcing contributors to add one when adding new stats could be
reasonable (it's like doc), I fear that it would significantly inflate
the output.

Also, do you have an idea about the verbosity of the dump here ? For
example let's say you have 100 listeners with 4 servers each (which is
an average sized config). I'm just looking for a rought order of magnitude,
ie closer to 10-100k or to 1-10M. The typed output is already quite heavy
for large configs so it should not be a big deal, but it's something we
have to keep in mind.

Oh BTW just to let you know, I'm working on a painful bug and possibly a
small regression which will force me to revert some recent fixes, so you
may still have a bit of time left :-)

Thanks,
Willy

Re: Getting JSON encoded data from the stats socket.

[Simon Horman]

Hi Willy, Hi All,

On Thu, Nov 10, 2016 at 04:52:56PM +0100, Willy Tarreau wrote:
> Hi Simon!
> 
> On Thu, Nov 10, 2016 at 04:27:15PM +0100, Simon Horman wrote:
> > My preference is to take things calmly as TBH I am only just getting
> > started on this and I think the schema could take a little time to get
> > a consensus on.
> 
> I totally agree with you. I think the most difficult thing is not to
> run over a few arrays and dump them but manage to make everyone agree
> on the schema. And that will take more than a few days I guess. Anyway
> I'm fine with being proven wrong :-)

I took a first pass at defining a schema.

* The schema follows what is described on json-schema.org (or at least
  tries to). Is this a suitable approach?
* The schema only covers "show info" and "show stat" and the fields
  are based on the typed output variants of those commands.
  This leads me to several questions:
  - Is this field selection desirable? It seems to make sense to me
as presumably the intention of the JSON output is for it to
be machine readable.
  - Is such an approach appropriate for other show commands?
  - And more generally, which other show commands are desired to
support output in JSON (in the near term)?

{
"$schema": "http://json-schema.org/draft-04/schema#;,
"oneOf": [
{
"title": "Info",
"description": "Info about HAProxy status",
"type": "array",
"items": {
"properties": {
"title": "Info Item",
"type": "object",
"field": { "$ref": "#/definitions/field" },
"processNum": { "$ref": "#/definitions/processNum" },
"tags": { "$ref": "#/definitions/tags" },
"value": { "$ref": "#/definitions/typedValue" }
},
"required": ["field", "processNum", "tags", "value"]
}
},
{
"title": "Stat",
"description": "HAProxy statistics",
"type": "array",
"items": {
"title": "Info Item",
"type": "object",
"properties": {
"objType": {
"enum": ["F", // Frontend
 "B", // Backend
 "L", // Listener
 "S"  // Server
]
},
"proxyId": {
"type": "integer",
"minimum": 0
},
"id": {
"description": "Unique identifyier of object within 
proxy",
"type": "integer",
"minimum": 0
},
"field": { "$ref": "#/definitions/field" },
"processNum": { "$ref": "#/definitions/processNum" },
"tags": { "$ref": "#/definitions/tags" },
"typedValue": { "$ref": "#/definitions/typedValue" }
},
"required": ["objType", "proxyId", "id", "field", "processNum",
 "tags", "value"]
}
}
],
"definitions": {
"field": {
"type": "object",
"pos": {
"description": "Position of field",
"type": "integer",
"minimum": 0
},
"name": {
"description": "Name of field",
"type": "string"
},
"required": ["pos", "name"]
},
"processNum": {
"description": "Relative process number",
"type": "integer",
"minimum": 1
},
"tags": {
"type": "object",
"origin": {
"description": "Origin value was extracted from",
"type": "string",
"enum": ["M", // Metric
 "S", // Status
 "K", // Sorting Key
 "C", // From Configuration
 "P"  // From Product
]
},
"nature": {
"description": "Nature of information carried by field",
"type": "string",
"enum": ["A", // Age since last event
 "a", // Averaged value
 "C", // Cumulative counter
 "D", // Duration for a status
 "G", // Gague - measure at one instant
 "L", // Limit
 "M", // Maximum
 "m", // Minimum
 "N", // Name
 "O", // Free text output
 "R", // Event rate - measure at one instant
 "T"  // Date or time
]
},
  

Re: Getting JSON encoded data from the stats socket.

[ge...@riseup.net]

Hi,

On 16-11-10 16:56:33, Willy Tarreau wrote:
> I removed you from the To in this response, but just as a hint we
> generally recommend to keep people CCed since most of us subscribed
> to lists have filters to automatically place them in the right box,
> and some people may participate without being subscribed. 

Yeah, I'm using filtering as well, but this doesn't deal with getting
the same mail(s) multiple times.

> On most lists, when people don't want to be automatically CCed on
> replies, they simply set their Reply-To header to the list's address.

Thanks, wasn't aware of this. I did so now.

> OK but just so that there's no misunderstanding, next release will be
> in approx one year. However if the patch is merged early, it will very
> likely apply well to the stable release meaning you can easily add it
> to your own packages.

Ah, I see, wasn't aware of this. Well then...this is fine as well.. :)

Cheers,
Georg


signature.asc
Description: Digital signature

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

On Thu, Nov 10, 2016 at 04:30:57PM +0100, ge...@riseup.net wrote:
> (Please don't Cc: me, I'm subscribed to the list.)

I removed you from the To in this response, but just as a hint we
generally recommend to keep people CCed since most of us subscribed
to lists have filters to automatically place them in the right box,
and some people may participate without being subscribed. On most
lists, when people don't want to be automatically CCed on replies,
they simply set their Reply-To header to the list's address.

> Even if I'm not Simon, I'll say a word, hope thats okay, because I've
> dug out this old thread: It's fine for me if it will go into 1.7 or
> 1.8. I don't need this within the next two weeks, but looking forward to
> use it. If it will take another four, six or eight weeks, this is
> completely fine with me.

OK but just so that there's no misunderstanding, next release will be in
approx one year. However if the patch is merged early, it will very likely
apply well to the stable release meaning you can easily add it to your own
packages.

Cheers,
Willy

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

Hi Simon!

On Thu, Nov 10, 2016 at 04:27:15PM +0100, Simon Horman wrote:
> My preference is to take things calmly as TBH I am only just getting
> started on this and I think the schema could take a little time to get
> a consensus on.

I totally agree with you. I think the most difficult thing is not to
run over a few arrays and dump them but manage to make everyone agree
on the schema. And that will take more than a few days I guess. Anyway
I'm fine with being proven wrong :-)

Cheers,
Willy

Re: Getting JSON encoded data from the stats socket.

[ge...@riseup.net]

(Please don't Cc: me, I'm subscribed to the list.)

On 16-11-10 16:12:31, Willy Tarreau wrote:
> That's cool!
> 
> The only thing is that I don't want to delay the release only for this,
> and at the same time I'm pretty sure it's possible to do something which
> will not impact existing code within a reasonable time frame. I just
> don't know how long it takes to make everyone agree on the schema. My
> intent is to release 1.7 by the end of next week *if we don't discover
> new scary bugs*. So if you think it's doable by then, that's fine. Or
> if you want to buy more time, you need to discover a big bug which will
> keep me busy and cause the release to be delayed ;-) Otherwise I think
> it will have to be in 1.8.
> 
> Note, to be clear, if many people insist on having this, we don't have an
> emergency to release by the end of next week, but it's just a policy we
> cannot pursue forever, at least by respect for those who were pressured
> to send their stuff in time. So I think that we can negociate one extra
> week if we're sure to have something completed, but only if people here
> insist on having it in 1.7.
> 
> Thus the first one who has a word to say is obviously Simon : if you
> think that even two weeks are not achievable, let's calmly postpone
> and avoid any stress.

Even if I'm not Simon, I'll say a word, hope thats okay, because I've
dug out this old thread: It's fine for me if it will go into 1.7 or
1.8. I don't need this within the next two weeks, but looking forward to
use it. If it will take another four, six or eight weeks, this is
completely fine with me.

All the best,
Georg


signature.asc
Description: Digital signature

Re: Getting JSON encoded data from the stats socket.

[Simon Horman]

On Thu, Nov 10, 2016 at 04:12:31PM +0100, Willy Tarreau wrote:
> Hi Malcolm,
> 
> On Thu, Nov 10, 2016 at 12:53:13PM +, Malcolm Turnbull wrote:
> > Georg,
> > 
> > That's a timely reminder thanks:
> > I just had another chat with Simon Horman who has kindly offered to
> > take a look at this again.
> 
> That's cool!
> 
> The only thing is that I don't want to delay the release only for this,
> and at the same time I'm pretty sure it's possible to do something which
> will not impact existing code within a reasonable time frame. I just
> don't know how long it takes to make everyone agree on the schema. My
> intent is to release 1.7 by the end of next week *if we don't discover
> new scary bugs*. So if you think it's doable by then, that's fine. Or
> if you want to buy more time, you need to discover a big bug which will
> keep me busy and cause the release to be delayed ;-) Otherwise I think
> it will have to be in 1.8.
> 
> Note, to be clear, if many people insist on having this, we don't have an
> emergency to release by the end of next week, but it's just a policy we
> cannot pursue forever, at least by respect for those who were pressured
> to send their stuff in time. So I think that we can negociate one extra
> week if we're sure to have something completed, but only if people here
> insist on having it in 1.7.
> 
> Thus the first one who has a word to say is obviously Simon : if you
> think that even two weeks are not achievable, let's calmly postpone and
> avoid any stress.

My preference is to take things calmly as TBH I am only just getting
started on this and I think the schema could take a little time to get
a consensus on.

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

Hi Malcolm,

On Thu, Nov 10, 2016 at 12:53:13PM +, Malcolm Turnbull wrote:
> Georg,
> 
> That's a timely reminder thanks:
> I just had another chat with Simon Horman who has kindly offered to
> take a look at this again.

That's cool!

The only thing is that I don't want to delay the release only for this,
and at the same time I'm pretty sure it's possible to do something which
will not impact existing code within a reasonable time frame. I just
don't know how long it takes to make everyone agree on the schema. My
intent is to release 1.7 by the end of next week *if we don't discover
new scary bugs*. So if you think it's doable by then, that's fine. Or
if you want to buy more time, you need to discover a big bug which will
keep me busy and cause the release to be delayed ;-) Otherwise I think
it will have to be in 1.8.

Note, to be clear, if many people insist on having this, we don't have an
emergency to release by the end of next week, but it's just a policy we
cannot pursue forever, at least by respect for those who were pressured
to send their stuff in time. So I think that we can negociate one extra
week if we're sure to have something completed, but only if people here
insist on having it in 1.7.

Thus the first one who has a word to say is obviously Simon : if you
think that even two weeks are not achievable, let's calmly postpone and
avoid any stress.

Thanks,
Willy

Re: Getting JSON encoded data from the stats socket.

[Dave Cottlehuber]

On Thu, 10 Nov 2016, at 13:53, Malcolm Turnbull wrote:
> Georg,
> 
> That's a timely reminder thanks:
> I just had another chat with Simon Horman who has kindly offered to
> take a look at this again.

Sounds great!

I'm very interested in logging this continually via chrooted unix
socket,
into both riemann & rsyslog and into graylog/splunk. I'm happy to help
test
and contribute documentation as well.

I was planning to use riemann-tools with csv format
 https://github.com/riemann/riemann-tools/blob/master/bin/riemann-haproxy 

A+
Dave

Re: Getting JSON encoded data from the stats socket.

[Malcolm Turnbull]

Georg,

That's a timely reminder thanks:
I just had another chat with Simon Horman who has kindly offered to
take a look at this again.




On 10 November 2016 at 10:54, ge...@riseup.net <ge...@riseup.net> wrote:
> Hi all,
>
> On 16-07-05 10:05:13, Mark Brookes wrote:
>> I wondered if we could start a discussion about the possibility of
>> having the stats socket return stats data in JSON format.
>
> After the discussion we had in July, I'm wondering what's the current
> status regarding this topic?
>
> Thanks and all the best,
> Georg



-- 
Regards,

Malcolm Turnbull.

Loadbalancer.org Ltd.
Phone: +44 (0)330 380 1064
http://www.loadbalancer.org/

Re: Getting JSON encoded data from the stats socket.

[ge...@riseup.net]

Hi all,

On 16-07-05 10:05:13, Mark Brookes wrote:
> I wondered if we could start a discussion about the possibility of
> having the stats socket return stats data in JSON format.

After the discussion we had in July, I'm wondering what's the current
status regarding this topic?

Thanks and all the best,
Georg


signature.asc
Description: Digital signature

Re: Getting JSON encoded data from the stats socket.

[ge...@riseup.net]

Hi,

On 16-07-26 21:47:55, Willy Tarreau wrote:
> I'd like to wait for other people to have the time to participate to
> this discussion, I know that some people are very careful about the
> relevance and accuracy of the stats, some people may want to report
> other suggestions.

I can't add that much, and have no specific suggestions, so just this:

(I'm a long time user of HAProxy, my setups aren't that big, mostly
around 50 backends, but I absolutely love the software. Thanks for this
great work!)

Regarding the topic: I absolutely support the proposal to dump the stats
into json. In my opinion, this is a much more easily parseable (and
modern) format, instead of csv. I think that grouping by process makes
sense, but to include "overall stats" as well. Additionally, I support
your view Willy, about the amount of the data to dump: I would speak in
favor of "dumping as much as possible", because, not sure if I got this
right, it's already possible to do so, it just needs support to dump to
json. Better safe then sorry, let's include all the data which _might_
be of interest, instead of data which _is now_ of interest. If some
"useless" (for now) data would be dumped...so what?

Thanks for the proposal Pavlos!

Cheers,
Georg


signature.asc
Description: Digital signature

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

On Tue, Jul 26, 2016 at 09:06:05PM +0200, Pavlos Parissis wrote:
> > You probably have not looked at the output of "show stats typed", it
> > gives you the nature of each value letting you know how to aggregate
> > them (min, max, avg, sum, pick any, etc).
> > 
> 
> I have seen it but it isn't available on 1.6. It could simplify my code, I
> should give a try.

Ah indeed you're right. Well it's not in 1.6 mainline but we backported
it to hapee-1.6 in case that's relevant to the machines you're interested
in.

> >> The stats are already aggregated and few metrics are excluded. For example 
> >> all status stuff.
> >> Each process performs healthchecking, so they act as little brains which 
> >> never agree on the
> >> status of a server as they run their checks on different interval.
> > 
> > Absolutely, but at least you want to see their stats. For example how many
> > times a server has switched state per process then in total (meaning a
> > proportional amount of possibly visible issues).
> > 
> 
> True, but in setups with ECMP in front of N HAProxy nodes which run in nbproc 
> mode you offload
> application healthchecking to a dedicated daemon which runs on servers(service
> discovery+service availability with consul/zookeeper stuff) and you only run 
> TCP checks
> from HAProxy.
> 
> In our setup we don't real care about how many times a server flapped, it 
> doesn't tell us
> something we don't know already, application is in broken state.

In such a case I agree.

> But, other people may find it useful.

Anyway that was just an example, what I meant by this is that we must
take care not to selectively pick some elements and not other ones. I
prefer that the output contains 10% of useless stuff and that we never
have anything special to do for the upcoming stuff to automatically
appear than to have to explicitly add new stuff all the time! When
you see the size of the csv dump function right now, it's a joke
and I really expect the JSON dump to follow the same philosophy.

> > My issue is that if the *format* doesn't support per-process stats, we'll 
> > have
> > to emit a new format 3 months later for all the people who want to process 
> > it.
> > We've reworked the stats dump to put an end to the problem where depending 
> > on
> > the output format you used to have different types of information, and there
> > was no single representation carrying them all at once. For me now it's
> > essential that if we prepare a new format it's not stripped down from the
> > info people need, otherwise it will automatically engender yet another 
> > format.
> > 
> 
> Agree. I am fine giving per process stats for servers/frontends/backends.
> Adding another top level key 'per_process' in my proposal should be a good 
> start:
> 
> {
> "per_process": {
> "proc1": {
> "frontend": {
> "www.haproxy.org": {
> "bin": "",
> "lbtot": "55",
> ...
(...)

Yes, I think so and that's also more or less similar to what Mark proposed.
Also I'm not much worried by the extra output size, if we dump this through
HTTP we'll have it gzipped.

Also, we want to have the values typed otherwise you're fucked as we used
to be with the CSV dump in the past. The current code supports this and
that's important. I don't know how it may impact the JSON output. Maybe
some parts will be just "numbers", but I remember that certain of them
have some properties (eg: max, limit, age, percentage, PID, SNMP ID, etc).
I'm less worried about the strings, we basically have identifiers,
descriptions and outputs from what I remember. But taking a look at this
will help refine the format.

I'd like to wait for other people to have the time to participate to this
discussion, I know that some people are very careful about the relevance
and accuracy of the stats, some people may want to report other suggestions.

Cheers,
Willy

Re: Getting JSON encoded data from the stats socket.

[Pavlos Parissis]

On 26/07/2016 06:56 μμ, Willy Tarreau wrote:
> On Tue, Jul 26, 2016 at 05:51:08PM +0200, Pavlos Parissis wrote:
>> In all my setups I have nbproc > 1 and after a lot of changes and on how I 
>> aggregate HAProxy
>> stats and what most people want to see on graphs, I came up that with 
>> something like the following:
>>
>> {
>> "frontend": {
>> "www.haproxy.org": {
>> "bin": "",
>> "lbtot": "55",
>> ...
>> },
>> "www.haproxy.com": {
>> "bin": "",
>> "lbtot": "55",
>> ...
>> },
>> },
>> "backend": {
>> "www.haproxy.org": {
>> "bin": "",
>> "lbtot": "55",
>> 
>> "server": {
>> "srv1": {
>> "bin": "",
>> "lbtot": "55",
>>
>> },
>> ...
>> },
>> },
>> },
>> "haproxy": {
>> "PipesFree": "555",
>> ...
>> ,
>> "per_process": {
>> "id1": {
>> "PipesFree": "555",
>> "Process_num": "1",
>> ...
>> },
>> "id2": {
>> "PipesFree": "555",
>> "Process_num": "2",
>> ...
>> },
>> ...
>> },
>> },
>> "server": {
>> "srv1": {
>> "bin": "",
>> "lbtot": "55",
>> ...
>> },
>> ...
>> },
>> }
>>
>>
>> Let me explain a bit:
>>
>> - It is very useful and handy to know stats for a server per backend but 
>> also across all
>> backends. Thus, I include a top level key 'server' which holds stats for 
>> each server across all
>> backends. Few server's stats has to be excluded as they are meaningless in 
>> this context.
>> For example, status, lastchg, check_duration, check_code and few others. For 
>> those which aren't
>> counters but fixed numbers you want to either sum them(slim) or get the 
>> average(weight). I
>> don't do the latter in my setup.
> 
> You probably have not looked at the output of "show stats typed", it
> gives you the nature of each value letting you know how to aggregate
> them (min, max, avg, sum, pick any, etc).
> 

I have seen it but it isn't available on 1.6. It could simplify my code, I 
should give a try.

>> - Aggregation across multiple processes for haproxy stats(show info output)
> 
> It's not only "show info", this one reports only the process health.
> 
>> As you can see I provide stats per process and across all processes.
>> It has been proven very useful to know the CPU utilization per process. We 
>> depend on the kernel
>> to do the distribution of incoming connects to all processes and so far it 
>> works very well, but
>> sometimes you see a single process to consume a lot of CPU and if you don't 
>> provide percentiles
>> or stats per process then you are going to miss it. The metrics about 
>> uptime, version,
>> description and few other can be excluded in the aggregation.
> 
> These last ones are in the "pick any" type of aggregation I was talking about.
> 
>> - nbproc > 1 and aggregation for frontend/backend/server
>> My proposal doesn't cover stats for frontend/backend/server per haproxy 
>> process.
> 
> But that's precisely the limitation I'm reporting :-)
> 
>> The stats are already aggregated and few metrics are excluded. For example 
>> all status stuff.
>> Each process performs healthchecking, so they act as little brains which 
>> never agree on the
>> status of a server as they run their checks on different interval.
> 
> Absolutely, but at least you want to see their stats. For example how many
> times a server has switched state per process then in total (meaning a
> proportional amount of possibly visible issues).
> 

True, but in setups with ECMP in front of N HAProxy nodes which run in nbproc 
mode you offload
application healthchecking to a dedicated daemon which runs on servers(service
discovery+service availability with consul/zookeeper stuff) and you only run 
TCP checks
from HAProxy.

In our setup we don't real care about how many times a server flapped, it 
doesn't tell us
something we don't know already, application is in broken state.

But, other people may find it useful.

> My issue is that if the *format* doesn't support per-process stats, we'll have
> to emit a new format 3 months later for all the people who want to process it.
> We've reworked the stats dump to put an end to the problem where depending on
> the output format you used to have different types of information, and there
> was no single representation carrying them all at once. For me now it's
> essential that if we prepare a new format it's not stripped down from the
> info people need, otherwise it will automatically engender yet another 

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

On Tue, Jul 26, 2016 at 05:51:08PM +0200, Pavlos Parissis wrote:
> In all my setups I have nbproc > 1 and after a lot of changes and on how I 
> aggregate HAProxy
> stats and what most people want to see on graphs, I came up that with 
> something like the following:
> 
> {
> "frontend": {
> "www.haproxy.org": {
> "bin": "",
> "lbtot": "55",
> ...
> },
> "www.haproxy.com": {
> "bin": "",
> "lbtot": "55",
> ...
> },
> },
> "backend": {
> "www.haproxy.org": {
> "bin": "",
> "lbtot": "55",
> 
> "server": {
> "srv1": {
> "bin": "",
> "lbtot": "55",
>
> },
> ...
> },
> },
> },
> "haproxy": {
> "PipesFree": "555",
> ...
> ,
> "per_process": {
> "id1": {
> "PipesFree": "555",
> "Process_num": "1",
> ...
> },
> "id2": {
> "PipesFree": "555",
> "Process_num": "2",
> ...
> },
> ...
> },
> },
> "server": {
> "srv1": {
> "bin": "",
> "lbtot": "55",
> ...
> },
> ...
> },
> }
> 
> 
> Let me explain a bit:
> 
> - It is very useful and handy to know stats for a server per backend but also 
> across all
> backends. Thus, I include a top level key 'server' which holds stats for each 
> server across all
> backends. Few server's stats has to be excluded as they are meaningless in 
> this context.
> For example, status, lastchg, check_duration, check_code and few others. For 
> those which aren't
> counters but fixed numbers you want to either sum them(slim) or get the 
> average(weight). I
> don't do the latter in my setup.

You probably have not looked at the output of "show stats typed", it
gives you the nature of each value letting you know how to aggregate
them (min, max, avg, sum, pick any, etc).

> - Aggregation across multiple processes for haproxy stats(show info output)

It's not only "show info", this one reports only the process health.

> As you can see I provide stats per process and across all processes.
> It has been proven very useful to know the CPU utilization per process. We 
> depend on the kernel
> to do the distribution of incoming connects to all processes and so far it 
> works very well, but
> sometimes you see a single process to consume a lot of CPU and if you don't 
> provide percentiles
> or stats per process then you are going to miss it. The metrics about uptime, 
> version,
> description and few other can be excluded in the aggregation.

These last ones are in the "pick any" type of aggregation I was talking about.

> - nbproc > 1 and aggregation for frontend/backend/server
> My proposal doesn't cover stats for frontend/backend/server per haproxy 
> process.

But that's precisely the limitation I'm reporting :-)

> The stats are already aggregated and few metrics are excluded. For example 
> all status stuff.
> Each process performs healthchecking, so they act as little brains which 
> never agree on the
> status of a server as they run their checks on different interval.

Absolutely, but at least you want to see their stats. For example how many
times a server has switched state per process then in total (meaning a
proportional amount of possibly visible issues).

My issue is that if the *format* doesn't support per-process stats, we'll have
to emit a new format 3 months later for all the people who want to process it.
We've reworked the stats dump to put an end to the problem where depending on
the output format you used to have different types of information, and there
was no single representation carrying them all at once. For me now it's
essential that if we prepare a new format it's not stripped down from the
info people need, otherwise it will automatically engender yet another format.

Thanks,
Willy

Re: Getting JSON encoded data from the stats socket.

[Pavlos Parissis]

On 26/07/2016 03:30 μμ, Willy Tarreau wrote:
> Hi Pavlos!
> 
> On Tue, Jul 26, 2016 at 03:23:01PM +0200, Pavlos Parissis wrote:
>> Here is a suggestion { "frontend": { "www.haproxy.org": { "bin": 
>> "", "lbtot":
>> "55", ... }, "www.haproxy.com": { "bin": "", "lbtot": 
>> "55", ... }, }, 
>> "backend": { "www.haproxy.org": { "bin": "", "lbtot": "55", 
>>  "server":
>> { "srv1": { "bin": "", "lbtot": "55",  }, ... }
>> 
>> }, }, "haproxy": { "id1": { "PipesFree": "555", "Process_num": "1", ... }, 
>> "id2": { 
>> "PipesFree": "555", "Process_num": "2", ... }, ... }, }
> 
> Thanks. How does it scale if we later want to aggregate these ones over 
> multiple processes
> and/or nodes ? The typed output already emits a process number for each 
> field. Also, we do
> have the information of how data need to be parsed and aggregated. I suspect 
> that we want to
> produce this with the JSON output as well so that we don't lose information 
> when dumping in
> JSON mode. I would not be surprized if people find JSON easier to process 
> than our current
> format to aggregate their stats, provided we have all the fields :-)
> 
> Cheers, Willy
> 

I am glad you asked about aggregation as I deliberately didn't include 
aggregation.
In all my setups I have nbproc > 1 and after a lot of changes and on how I 
aggregate HAProxy
stats and what most people want to see on graphs, I came up that with something 
like the following:

{
"frontend": {
"www.haproxy.org": {
"bin": "",
"lbtot": "55",
...
},
"www.haproxy.com": {
"bin": "",
"lbtot": "55",
...
},
},
"backend": {
"www.haproxy.org": {
"bin": "",
"lbtot": "55",

"server": {
"srv1": {
"bin": "",
"lbtot": "55",
   
},
...
},
},
},
"haproxy": {
"PipesFree": "555",
...
,
"per_process": {
"id1": {
"PipesFree": "555",
"Process_num": "1",
...
},
"id2": {
"PipesFree": "555",
"Process_num": "2",
...
},
...
},
},
"server": {
"srv1": {
"bin": "",
"lbtot": "55",
...
},
...
},
}


Let me explain a bit:

- It is very useful and handy to know stats for a server per backend but also 
across all
backends. Thus, I include a top level key 'server' which holds stats for each 
server across all
backends. Few server's stats has to be excluded as they are meaningless in this 
context.
For example, status, lastchg, check_duration, check_code and few others. For 
those which aren't
counters but fixed numbers you want to either sum them(slim) or get the 
average(weight). I
don't do the latter in my setup.

- Aggregation across multiple processes for haproxy stats(show info output)
As you can see I provide stats per process and across all processes.
It has been proven very useful to know the CPU utilization per process. We 
depend on the kernel
to do the distribution of incoming connects to all processes and so far it 
works very well, but
sometimes you see a single process to consume a lot of CPU and if you don't 
provide percentiles
or stats per process then you are going to miss it. The metrics about uptime, 
version,
description and few other can be excluded in the aggregation.


- nbproc > 1 and aggregation for frontend/backend/server
My proposal doesn't cover stats for frontend/backend/server per haproxy process.
The stats are already aggregated and few metrics are excluded. For example all 
status stuff.
Each process performs healthchecking, so they act as little brains which never 
agree on the
status of a server as they run their checks on different interval. But, if 
nbproc == 1 then
these metrics have to be included.


Cheers,
Pavlos







signature.asc
Description: OpenPGP digital signature

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

On Tue, Jul 26, 2016 at 03:06:35PM +0100, Mark Brookes wrote:
> Could we perhaps group by the node then process_num then?
> {nodename:value:
> {pid: pid1: {
> haproxy: {
> Uptime_sec:100,
> PoolFailed:1
> }
>   stats: { "frontend": {
> "www.haproxy.org": {
> "bin": "",
> "lbtot": "55",
> ...
(...)

Yes I think it's fine this way because in practice, clients will consult
a single process at a time so it's easier to have per-process dumps to
aggregate later.

Willy

Re: Getting JSON encoded data from the stats socket.

[Mark Brookes]

Could we perhaps group by the node then process_num then?
{nodename:value:
{pid: pid1: {
haproxy: {
Uptime_sec:100,
PoolFailed:1
}
  stats: { "frontend": {
"www.haproxy.org": {
"bin": "",
"lbtot": "55",
...
},
"www.haproxy.com": {
"bin": "",
"lbtot": "55",
...
},
},
"backend": {
"www.haproxy.org": {
"bin": "",
"lbtot": "55",

"server": {
"srv1": {
"bin": "",
"lbtot": "55",
   
},
...
}

},
{pid: pid2: { haproxy: {
Uptime_sec:100,
PoolFailed:1
}
  stats: { "frontend": {
"www.haproxy.org": {
"bin": "",
"lbtot": "55",
...
},
"www.haproxy.com": {
"bin": "",
"lbtot": "55",
...
},
},
"backend": {
"www.haproxy.org": {
"bin": "",
"lbtot": "55",

"server": {
"srv1": {
"bin": "",
"lbtot": "55",
   
},
...
}

},

ignore the close brackets im pretty sure they are wrong, but you get the idea.

On 26 July 2016 at 14:30, Willy Tarreau  wrote:
> Hi Pavlos!
>
> On Tue, Jul 26, 2016 at 03:23:01PM +0200, Pavlos Parissis wrote:
>> Here is a suggestion
>> {
>> "frontend": {
>> "www.haproxy.org": {
>> "bin": "",
>> "lbtot": "55",
>> ...
>> },
>> "www.haproxy.com": {
>> "bin": "",
>> "lbtot": "55",
>> ...
>> },
>> },
>> "backend": {
>> "www.haproxy.org": {
>> "bin": "",
>> "lbtot": "55",
>> 
>> "server": {
>> "srv1": {
>> "bin": "",
>> "lbtot": "55",
>>
>> },
>> ...
>> }
>>
>> },
>> },
>> "haproxy": {
>> "id1": {
>> "PipesFree": "555",
>> "Process_num": "1",
>> ...
>> },
>> "id2": {
>> "PipesFree": "555",
>> "Process_num": "2",
>> ...
>> },
>> ...
>> },
>> }
>
> Thanks. How does it scale if we later want to aggregate these ones over
> multiple processes and/or nodes ? The typed output already emits a
> process number for each field. Also, we do have the information of how
> data need to be parsed and aggregated. I suspect that we want to produce
> this with the JSON output as well so that we don't lose information when
> dumping in JSON mode. I would not be surprized if people find JSON easier
> to process than our current format to aggregate their stats, provided we
> have all the fields :-)
>
> Cheers,
> Willy

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

Hi Pavlos!

On Tue, Jul 26, 2016 at 03:23:01PM +0200, Pavlos Parissis wrote:
> Here is a suggestion
> {
> "frontend": {
> "www.haproxy.org": {
> "bin": "",
> "lbtot": "55",
> ...
> },
> "www.haproxy.com": {
> "bin": "",
> "lbtot": "55",
> ...
> },
> },
> "backend": {
> "www.haproxy.org": {
> "bin": "",
> "lbtot": "55",
> 
> "server": {
> "srv1": {
> "bin": "",
> "lbtot": "55",
>
> },
> ...
> }
> 
> },
> },
> "haproxy": {
> "id1": {
> "PipesFree": "555",
> "Process_num": "1",
> ...
> },
> "id2": {
> "PipesFree": "555",
> "Process_num": "2",
> ...
> },
> ...
> },
> }

Thanks. How does it scale if we later want to aggregate these ones over
multiple processes and/or nodes ? The typed output already emits a
process number for each field. Also, we do have the information of how
data need to be parsed and aggregated. I suspect that we want to produce
this with the JSON output as well so that we don't lose information when
dumping in JSON mode. I would not be surprized if people find JSON easier
to process than our current format to aggregate their stats, provided we
have all the fields :-)

Cheers,
Willy

Re: Getting JSON encoded data from the stats socket.

[Pavlos Parissis]

On 26/07/2016 03:08 μμ, Willy Tarreau wrote:
> On Tue, Jul 26, 2016 at 02:05:56PM +0100, Mark Brookes wrote:
>>> So for sure I definitely support this proposal :-)
>>
>> Thats great news. Do you have a JSON structure in mind?
>> Or would you like me to come up with something?
> 
> I'm probably the worst ever person to suggest a JSON structure. If you
> have any ideas, please bring them on the list. You know how it works,
> once nobody criticizes anymore, your design is fine. And you'll just
> have to ignore people who complain after the work is done :-)
> 
> Cheers,
> Willy
> 

Here is a suggestion
{
"frontend": {
"www.haproxy.org": {
"bin": "",
"lbtot": "55",
...
},
"www.haproxy.com": {
"bin": "",
"lbtot": "55",
...
},
},
"backend": {
"www.haproxy.org": {
"bin": "",
"lbtot": "55",

"server": {
"srv1": {
"bin": "",
"lbtot": "55",
   
},
...
}

},
},
"haproxy": {
"id1": {
"PipesFree": "555",
"Process_num": "1",
...
},
"id2": {
"PipesFree": "555",
"Process_num": "2",
...
},
...
},
}

Cheers,
Pavlos




signature.asc
Description: OpenPGP digital signature

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

On Tue, Jul 26, 2016 at 02:05:56PM +0100, Mark Brookes wrote:
> >So for sure I definitely support this proposal :-)
> 
> Thats great news. Do you have a JSON structure in mind?
> Or would you like me to come up with something?

I'm probably the worst ever person to suggest a JSON structure. If you
have any ideas, please bring them on the list. You know how it works,
once nobody criticizes anymore, your design is fine. And you'll just
have to ignore people who complain after the work is done :-)

Cheers,
Willy

Re: Getting JSON encoded data from the stats socket.

[Mark Brookes]

>So for sure I definitely support this proposal :-)

Thats great news. Do you have a JSON structure in mind?
Or would you like me to come up with something?

On 5 July 2016 at 18:04, Willy Tarreau <w...@1wt.eu> wrote:
> Hi Mark,
>
> On Tue, Jul 05, 2016 at 10:05:13AM +0100, Mark Brookes wrote:
>> Hi Willy/All
>>
>> I wondered if we could start a discussion about the possibility of
>> having the stats socket return stats data in JSON format.
>>
>> Im primarily interested in the data that is returned by issuing a
>> 'show stat' which is normally returned as a csv.
>>
>> I wont go into specifics as to how the data would be structured, we
>> can decide on that later (Assuming you are happy with this idea).
>>
>> Ive approached Simon Horman and hes happy to do the work for us.
>>
>> Please let me know your thoughts
>
> Well, I completely reworked the stats internals recently for two
> purposes :
>   1) bringing the ability to dump them in another format such as JSON ;
>   2) making it easier to aggregate them over multiple processes/nodes
>
> So for sure I definitely support this proposal :-)
>
> Best regards
> Willy

Re: Getting JSON encoded data from the stats socket.

[Willy Tarreau]

Hi Mark,

On Tue, Jul 05, 2016 at 10:05:13AM +0100, Mark Brookes wrote:
> Hi Willy/All
> 
> I wondered if we could start a discussion about the possibility of
> having the stats socket return stats data in JSON format.
> 
> Im primarily interested in the data that is returned by issuing a
> 'show stat' which is normally returned as a csv.
> 
> I wont go into specifics as to how the data would be structured, we
> can decide on that later (Assuming you are happy with this idea).
> 
> Ive approached Simon Horman and hes happy to do the work for us.
> 
> Please let me know your thoughts

Well, I completely reworked the stats internals recently for two
purposes :
  1) bringing the ability to dump them in another format such as JSON ;
  2) making it easier to aggregate them over multiple processes/nodes

So for sure I definitely support this proposal :-)

Best regards
Willy

Getting JSON encoded data from the stats socket.

[Mark Brookes]

Hi Willy/All

I wondered if we could start a discussion about the possibility of
having the stats socket return stats data in JSON format.

Im primarily interested in the data that is returned by issuing a
'show stat' which is normally returned as a csv.

I wont go into specifics as to how the data would be structured, we
can decide on that later (Assuming you are happy with this idea).

Ive approached Simon Horman and hes happy to do the work for us.

Please let me know your thoughts

Thanks

Mark

Re: Setting weight through stats socket, nbproc > 1

[Conrad Hoffmann]

Hi,

On 05/31/2016 11:55 AM, Unknown User wrote:
> I am running haproxy with nbproc 3. (I know its not supported, but i
> want to use more than 1 cpu with haproxy)

Don't worry, it still works really well :)

> The following is my config related to the stats socket
> 
> daemon
> nbproc 3
> pidfile /var/run/hapxy.pid
> # Does not work with nbproc  >1
> stats socket  /var/run/hapxy.sock  level admin
> # echo 'show stat' | nc -U /var/run/hapxy.sock
> # Logging

Note that the processes don't (really) know anything about each other. As
such, when talking to one admin socket, you are reaching exactly one
process. Which one depends on your config and some other stuff.

> When setting weight, it seems only one of the haproxy process get the
> updated weight. See below. The weight is increased to 20 from a
> default 10 for all processes, but subsequent show stat's may show
> either 10 or 20.
> Is there any way to make the weight change applicable for all processes?

Yes. Make sure each process has it's own stats socket, e.g.:

  nbproc 3

  # expose admin socket for each process
  stats socket /var/run/hapxy.1.sock level admin process 1
  stats socket /var/run/hapxy.2.sock level admin process 2
  stats socket /var/run/hapxy.3.sock level admin process 3

Now, the only thing that's slightly unfortunate, you have to send the
corresponding command to each admin socket. But if you do so, everything
will work as expected.

Hope that helps,
Conrad


> [r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
> grep host02.unknownservers.net
> pxy_noxff,host02.unknownservers.net,0,0,0,1,,1,984,221,,0,,0,0,0,0,UP,10,1,0,0,0,7,0,,2,7,25,,1,,2,0,,1,L7OK,200,0,0,0,1,0,0,0,00,0,
> pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,2,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
> [r...@unknownservers.net  run]# echo set weight
> pxy_noxff/host02.unknownservers.net 20 | nc -U hapxy.sock
> 
> [r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
> grep host02.unknownservers.net
> pxy_noxff,host02.unknownservers.net,0,0,0,1,,3,4574,3367,,0,,0,0,0,0,UP,20,1,0,0,0,18,0,,3,7,25,,3,,2,0,,1,L7OK,200,0,0,1,2,0,0,0,00,0,
> pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,3,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
> [r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
> grep host02.unknownservers.net
> pxy_noxff,host02.unknownservers.net,0,0,0,1,,4,5709,4877,,0,,0,0,0,0,UP,20,1,0,0,0,20,0,,3,7,25,,4,,2,1,,1,L7OK,200,0,0,2,2,0,0,0,00,0,
> pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,3,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
> [r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
> grep host02.unknownservers.net
> pxy_noxff,host02.unknownservers.net,0,0,1,1,,5,5709,4877,,0,,0,0,0,0,UP,20,1,0,0,0,24,0,,3,7,25,,5,,2,1,,1,L7OK,200,0,0,2,2,0,0,0,00,0,
> pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,3,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
> [r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
> grep host02.unknownservers.net
> pxy_noxff,host02.unknownservers.net,0,0,0,1,,2,3118,510,,0,,0,0,0,0,UP,10,1,0,0,0,27,0,,2,7,25,,2,,2,0,,1,L7OK,200,0,0,1,1,0,0,0,00,0,
> pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,2,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
> [r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
> grep host02.unknownservers.net
> pxy_noxff,host02.unknownservers.net,0,0,0,1,,2,3118,510,,0,,0,0,0,0,UP,10,1,0,0,0,30,0,,2,7,25,,2,,2,0,,1,L7OK,200,0,0,1,1,0,0,0,00,0,
> pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,2,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
> [r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
> grep host02.unknownservers.net
> pxy_noxff,host02.unknownservers.net,0,0,0,1,,9,13922,7429,,0,,0,0,0,0,UP,20,1,0,0,0,36,0,,3,7,25,,9,,2,1,,1,L7OK,200,0,0,4,5,0,0,0,00,0,
> pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,3,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
> [r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
> grep host02.unknownservers.net
> pxy_noxff,host02.unknownservers.net,0,0,0,1,,3,5232,799,,0,,0,0,0,0,UP,10,1,0,0,0,41,0,,2,7,25,,3,,2,1,,1,L7OK,200,0,0,2,1,0,0,0,00,0,
> pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,2,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
> [r...@unknownserve

Setting weight through stats socket, nbproc > 1

[Unknown User]

I am running haproxy with nbproc 3. (I know its not supported, but i
want to use more than 1 cpu with haproxy)

The following is my config related to the stats socket

daemon
nbproc 3
pidfile /var/run/hapxy.pid
# Does not work with nbproc  >1
    stats socket  /var/run/hapxy.sock  level admin
# echo 'show stat' | nc -U /var/run/hapxy.sock
# Logging


When setting weight, it seems only one of the haproxy process get the
updated weight. See below. The weight is increased to 20 from a
default 10 for all processes, but subsequent show stat's may show
either 10 or 20.
Is there any way to make the weight change applicable for all processes?



[r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
grep host02.unknownservers.net
pxy_noxff,host02.unknownservers.net,0,0,0,1,,1,984,221,,0,,0,0,0,0,UP,10,1,0,0,0,7,0,,2,7,25,,1,,2,0,,1,L7OK,200,0,0,0,1,0,0,0,00,0,
pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,2,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
[r...@unknownservers.net  run]# echo set weight
pxy_noxff/host02.unknownservers.net 20 | nc -U hapxy.sock

[r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
grep host02.unknownservers.net
pxy_noxff,host02.unknownservers.net,0,0,0,1,,3,4574,3367,,0,,0,0,0,0,UP,20,1,0,0,0,18,0,,3,7,25,,3,,2,0,,1,L7OK,200,0,0,1,2,0,0,0,00,0,
pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,3,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
[r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
grep host02.unknownservers.net
pxy_noxff,host02.unknownservers.net,0,0,0,1,,4,5709,4877,,0,,0,0,0,0,UP,20,1,0,0,0,20,0,,3,7,25,,4,,2,1,,1,L7OK,200,0,0,2,2,0,0,0,00,0,
pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,3,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
[r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
grep host02.unknownservers.net
pxy_noxff,host02.unknownservers.net,0,0,1,1,,5,5709,4877,,0,,0,0,0,0,UP,20,1,0,0,0,24,0,,3,7,25,,5,,2,1,,1,L7OK,200,0,0,2,2,0,0,0,00,0,
pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,3,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
[r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
grep host02.unknownservers.net
pxy_noxff,host02.unknownservers.net,0,0,0,1,,2,3118,510,,0,,0,0,0,0,UP,10,1,0,0,0,27,0,,2,7,25,,2,,2,0,,1,L7OK,200,0,0,1,1,0,0,0,00,0,
pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,2,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
[r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
grep host02.unknownservers.net
pxy_noxff,host02.unknownservers.net,0,0,0,1,,2,3118,510,,0,,0,0,0,0,UP,10,1,0,0,0,30,0,,2,7,25,,2,,2,0,,1,L7OK,200,0,0,1,1,0,0,0,00,0,
pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,2,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
[r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
grep host02.unknownservers.net
pxy_noxff,host02.unknownservers.net,0,0,0,1,,9,13922,7429,,0,,0,0,0,0,UP,20,1,0,0,0,36,0,,3,7,25,,9,,2,1,,1,L7OK,200,0,0,4,5,0,0,0,00,0,
pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,3,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
[r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
grep host02.unknownservers.net
pxy_noxff,host02.unknownservers.net,0,0,0,1,,3,5232,799,,0,,0,0,0,0,UP,10,1,0,0,0,41,0,,2,7,25,,3,,2,1,,1,L7OK,200,0,0,2,1,0,0,0,00,0,
pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,2,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,
[r...@unknownservers.net  run]# echo show stat | nc -U hapxy.sock  |
grep host02.unknownservers.net
pxy_noxff,host02.unknownservers.net,0,0,0,1,,11,17448,7938,,0,,0,0,0,0,UP,20,1,0,0,0,43,0,,3,7,25,,11,,2,0,,1,L7OK,200,0,0,5,6,0,0,0,00,0,
pxy_xff,host02.unknownservers.net,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,,3,8,25,,0,pxy_noxff/host02.unknownservers.net,2,0,,00,0,0,0,0,0,00,0,

[PATCH] DOC: specify that stats socket doc (section 9.2) is in management

[Kevin Decherf]

Commit 44aed90ce102c4136a5eda66d541f6fa79e141e8 moved the stats socket
documentation from config to management but the remaining references to
section 9.2 were not updated; improve it to be less confusing.

Signed-off-by: Kevin Decherf <ke...@kdecherf.com>
---
 doc/configuration.txt | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index b509238..be7e78f 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -859,7 +859,8 @@ stats socket [|] [param*]
   Binds a UNIX socket to  or a TCPv4/v6 address to .
   Connections to this socket will return various statistics outputs and even
   allow some commands to be issued to change some runtime settings. Please
-  consult section 9.2 "Unix Socket commands" for more details.
+  consult section 9.2 "Unix Socket commands" of Management Guide for more
+  details.
 
   All parameters supported by "bind" lines are supported, for instance to
   restrict access to some users or their access rights. Please consult
@@ -4233,7 +4234,7 @@ load-server-state-from-file { global | local | none }
 
   The format of the file is versionned and is very specific. To understand it,
   please read the documentation of the "show servers state" command (chapter
-  9.2).
+  9.2 of Management Guide).
 
   Arguments:
 global load the content of the file pointed by the global directive
-- 
2.5.1

Re: [PATCH] DOC: specify that stats socket doc (section 9.2) is in management

[Willy Tarreau]

On Tue, Oct 13, 2015 at 11:26:44PM +0200, Kevin Decherf wrote:
> Commit 44aed90ce102c4136a5eda66d541f6fa79e141e8 moved the stats socket
> documentation from config to management but the remaining references to
> section 9.2 were not updated; improve it to be less confusing.

Ah crap! I paid attention to this but still missed them :-(
Thanks Kevin, patch applied.

Willy

Re: Interactive stats socket broken on master

[Baptiste]

On Mon, Oct 12, 2015 at 12:06 AM, Willy Tarreau <w...@1wt.eu> wrote:
> On Sat, Oct 10, 2015 at 08:55:44PM -0500, Andrew Hayworth wrote:
>> Bump -
>>
>> I don't mind maintaining my own HAProxy package, but it seems bad to
>> release a major version with the interactive stats socket broken. Any
>> thoughts on the patch?
>
> Has anyone else tested it ? Since the beginning of the thread I must
> confess it's unclear to me as Jesse reported the issue, you said that
> your patch works for you then Jesse asks whether we should merge it.
> Jesse, have you tested it as well, so that we ensure you're facing
> the same issue ?
>
> Andrew BTW, your patch looks good and seems to do what you described
> in the message, I'm just asking to be sure that it addresses Jesse's
> bug as well.
>
> Last point guys, please keep in mind that not everybody reads all
> e-mails, so when you want to have a patch integrated, clearly mark
> it in the subject and don't leave it pending at the trail of a
> thread like this.
>
> Thanks,
> Willy
>


Hi all,

I confirm attached patch fix the issue: natively on my computer
(Willy, I was wrong this morning, I still had the bug) and when
HAProxy runs in docker too.

Baptiste

Re: Interactive stats socket broken on master

[Jesse Hathaway]

On Sun, Oct 11, 2015 at 5:06 PM, Willy Tarreau  wrote:
>
> Has anyone else tested it ? Since the beginning of the thread I must
> confess it's unclear to me as Jesse reported the issue, you said that
> your patch works for you then Jesse asks whether we should merge it.
> Jesse, have you tested it as well, so that we ensure you're facing
> the same issue ?

I can confirm the patch does indeed fix the issue for me, sorry for
any confusion.

Re: Interactive stats socket broken on master

[Willy Tarreau]

On Mon, Oct 12, 2015 at 12:33:39PM -0500, Jesse Hathaway wrote:
> On Sun, Oct 11, 2015 at 5:06 PM, Willy Tarreau  wrote:
> >
> > Has anyone else tested it ? Since the beginning of the thread I must
> > confess it's unclear to me as Jesse reported the issue, you said that
> > your patch works for you then Jesse asks whether we should merge it.
> > Jesse, have you tested it as well, so that we ensure you're facing
> > the same issue ?
> 
> I can confirm the patch does indeed fix the issue for me, sorry for
> any confusion.

Great, thanks for clearing this up, I've applied it now. Funnily, I just
faced the bug while writing the management doc and looking for an example
output from the CLI to copy-paste :-)

Willy

Re: Interactive stats socket broken on master

[Willy Tarreau]

On Sat, Oct 10, 2015 at 08:55:44PM -0500, Andrew Hayworth wrote:
> Bump -
> 
> I don't mind maintaining my own HAProxy package, but it seems bad to
> release a major version with the interactive stats socket broken. Any
> thoughts on the patch?

Has anyone else tested it ? Since the beginning of the thread I must
confess it's unclear to me as Jesse reported the issue, you said that
your patch works for you then Jesse asks whether we should merge it.
Jesse, have you tested it as well, so that we ensure you're facing
the same issue ?

Andrew BTW, your patch looks good and seems to do what you described
in the message, I'm just asking to be sure that it addresses Jesse's
bug as well.

Last point guys, please keep in mind that not everybody reads all
e-mails, so when you want to have a patch integrated, clearly mark
it in the subject and don't leave it pending at the trail of a
thread like this.

Thanks,
Willy

Re: Interactive stats socket broken on master

[Andrew Hayworth]

Bump -

I don't mind maintaining my own HAProxy package, but it seems bad to
release a major version with the interactive stats socket broken. Any
thoughts on the patch?

On Tue, Oct 6, 2015 at 9:29 AM, Jesse Hathaway <je...@mbuki-mvuki.org>
wrote:

> On Fri, Oct 2, 2015 at 10:24 AM, Andrew Hayworth
> <andrew.haywo...@getbraintree.com> wrote:>
> > Attached is a patch that fixes the issue for me.
>
> Willy, any thoughts on merging this patch into 1.6?
>



-- 
- Andrew Hayworth

Re: Interactive stats socket broken on master

[Jesse Hathaway]

On Fri, Oct 2, 2015 at 10:24 AM, Andrew Hayworth
 wrote:>
> Attached is a patch that fixes the issue for me.

Willy, any thoughts on merging this patch into 1.6?

Re: Interactive stats socket broken on master

[Andrew Hayworth]

Hi all -

On Thu, Oct 1, 2015 at 8:55 AM, Jesse Hathaway <je...@mbuki-mvuki.org> wrote:
> It appears the following commit broke the interactive stats socket.
> When this commit is applied the stats socket disconnects after typing
> prompt and hitting enter.

Attached is a patch that fixes the issue for me.

Thanks!

>From 9f785d7bc67c34ea441187c0e14c0ef573a71692 Mon Sep 17 00:00:00 2001
From: Andrew Hayworth <andrew.haywo...@getbraintree.com>
Date: Fri, 2 Oct 2015 15:08:10 +
Subject: [PATCH 1/1] BUG/MINOR: Handle interactive mode in cli handler

A previous commit broke the interactive stats cli prompt. Specifically,
it was not clear that we could be in STAT_CLI_PROMPT when we get to
the output functions for the cli handler, and the switch statement did
not handle this case. We would then fall through to the default
statement, which was recently changed to set error flags on the socket.
This in turn causes the socket to be closed, which is not what we wanted
in this specific case.

To fix, we add a case for STAT_CLI_PROMPT, and simply break out of the
switch statement.

Testing:
 - Connected to unix stats socket, issued 'prompt', observed that I
   could issue multiple consecutive commands.
 - Connected to unix stats socket, issued 'prompt', observed that socket
   timed out after inactivity expired.
 - Connected to unix stats socket, issued 'prompt' then 'set timeout cli
   5', observed that socket timed out after 5 seconds expired.
 - Connected to unix stats socket, issued invalid commands, received
   usage output.
 - Connected to unix stats socket, issued 'show info', received info
   output and socket disconnected.
 - Connected to unix stats socket, issued 'show stat', received stats
   output and socket disconnected.
 - Repeated above tests with TCP stats socket.
---
 src/dumpstats.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/dumpstats.c b/src/dumpstats.c
index bdfb7e3..1a39258 100644
--- a/src/dumpstats.c
+++ b/src/dumpstats.c
@@ -2484,6 +2484,8 @@ static void cli_io_handler(struct appctx *appctx)
}
else {  /* output functions */
switch (appctx->st0) {
+   case STAT_CLI_PROMPT:
+   break;
case STAT_CLI_PRINT:
if (bi_putstr(si_ic(si),
appctx->ctx.cli.msg) != -1)
appctx->st0 = STAT_CLI_PROMPT;
--
2.1.3


0001-BUG-MINOR-Handle-interactive-mode-in-cli-handler.patch
Description: Binary data

Interactive stats socket broken on master

[Jesse Hathaway]

It appears the following commit broke the interactive stats socket.
When this commit is applied the stats socket disconnects after typing
prompt and hitting enter.

This breaks applications such as hatop. We found the breaking commit using
git bisect.

Please let me know if I can help debug further.

commit 5cfa3bcc22708571b674a40f1d9cdfa0657ab33e
Author: Willy Tarreau <w...@1wt.eu>
Date:   Fri Sep 25 20:08:51 2015 +0200

MINOR: cli: do not call the release handler on internal error.

It's dangerous to call this on internal state error, because it risks
to perform a double-free. This can only happen when a state is not
handled. Note that the switch/case currently doesn't offer any option
for missed states since they're all declared. Better fix this anyway.
The fix was tested by commenting out some entries in the switch/case.

diff --git a/src/dumpstats.c b/src/dumpstats.c
index 87e62eb..bdfb7e3 100644
--- a/src/dumpstats.c
+++ b/src/dumpstats.c
@@ -2554,8 +2554,7 @@ static void cli_io_handler(struct appctx *appctx)
break;
 #endif
default: /* abnormal state */
-   cli_release_handler(appctx);
-   appctx->st0 = STAT_CLI_PROMPT;
+   si->flags |= SI_FL_ERR;
break;
}

Thanks, Jesse

Re: stats socket over TCP - dangerous?

[Willy Tarreau]

On Mon, Sep 21, 2015 at 10:34:17PM +, Thrawn wrote:
> > Does this mean that the TCP port is dangerous, or that the multiple
> > instances of the socket is dangerous??
> > What is the danger with it?
> 
> I believe the danger is that it may be externally visible, allowing outsiders
> to start and stop your servers, add new backends and redirect traffic to
> them, etc...
> 
> It should be safe enough if the socket only listens on the loopback address,
> but in that case, you may as well use Unix sockets. Otherwise, make sure that
> you have strong authentication on the socket, a good firewall, and a trusted
> network - and ask yourself whether you really need it.
> 
> In some cases, it may be feasible to listen on the loopback address and
> access it via an SSH tunnel.

Yes, that's *exactly* the idea. You can even use haproxy as a SSL frontend
and filter to forward the connection. But SSH is quite secure and handy here.

Willy

Re: stats socket over TCP - dangerous?

[Eduard Martinescu]

The loopback address is a good idea for allowing communication locally,
with software that can't use Unix sockets, like Java.

--
✉ Eduard Martinescu  | ✆ (585) 708-9685 | [image:
http://www.salsalabs.com]  - ignite action. fuel
change.

On Mon, Sep 21, 2015 at 6:34 PM, Thrawn 
wrote:

> > Does this mean that the TCP port is dangerous, or that the multiple
> > instances of the socket is dangerous??
> > What is the danger with it?
>
> I believe the danger is that it may be externally visible, allowing outsiders 
> to start and stop your servers, add new backends and redirect traffic to 
> them, etc...
>
> It should be safe enough if the socket only listens on the loopback address, 
> but in that case, you may as well use Unix sockets. Otherwise, make sure that 
> you have strong authentication on the socket, a good firewall, and a trusted 
> network - and ask yourself whether you really need it.
>
> In some cases, it may be feasible to listen on the loopback address and 
> access it via an SSH tunnel.
>
>

Re: stats socket over TCP - dangerous?

[Thrawn]

> software that can't use Unix sockets, like Java


Er...that's generally true (although there are libraries like junixsocket), and 
not to put you off, but why do you have Java talking to the HAProxy admin 
socket?

Re: stats socket over TCP - dangerous?

[Thrawn]

> Does this mean that the TCP port is dangerous, or that the multiple
> instances of the socket is dangerous??
> What is the danger with it?

I believe the danger is that it may be externally visible, allowing outsiders 
to start and stop your servers, add new backends and redirect traffic to them, 
etc...

It should be safe enough if the socket only listens on the loopback address, 
but in that case, you may as well use Unix sockets. Otherwise, make sure that 
you have strong authentication on the socket, a good firewall, and a trusted 
network - and ask yourself whether you really need it.

In some cases, it may be feasible to listen on the loopback address and access 
it via an SSH tunnel.

stats socket over TCP - dangerous?

[Larry O'Neill]

Hi,

I am looking at pulling stuff out of the stats socket, but the
documentation is unclear on this point:

It is also possible to add multiple instances of the stats socket by repeating
the line, and make them listen to a TCP port instead of a UNIX socket. This is
never done by default because this is dangerous, but can be handy in some
situations

Does this mean that the TCP port is dangerous, or that the multiple
instances of the socket is dangerous??
What is the danger with it?

Le meas,
Larry

-- 
*Confidentiality Notice:*  - This message and any attached files may 
contain information that is confidential and/or subject of legal privilege 
intended only for the use by the intended recipient. If you are not the 
intended recipient or the person responsible for delivering the message to 
the intended recipient, be advised that you have received this message in 
error and that any dissemination, copying or use of this message or 
attachment is strictly forbidden, as is the disclosure of the information 
therein. If you have received this message in error please notify the 
sender immediately and delete the message.

Session counter doesnt increase for stats socket

[Philipp Kolmann]

Hi,

I have several haproxy installations running on HAProxy version 1.5.4, 
released 2014/09/02 (debian packet) and I see no counting up on the 
stats socket session counter.


The Bytes-counter count up on every reload, but the sessions total stay 
and don't get updated on every reload (as I was used to have).


listen stats
bind 128.130.XX.XX:80 name IPv4
bind 2001:629:XX::XX:80 transparent name IPv6
mode http
stats enable
stats realm Haproxy\ Statistics
stats uri /

acl stats-access src 127.0.0.1
acl stats-access src 128.130.XX.XX
acl stats-access src 2001:629:XX::XX # wspk.zid
http-request deny if ! stats-access


thanks
Philipp

--
---
DI Mag. Philipp Kolmann  mail: kolm...@zid.tuwien.ac.at
Technische Universitaet Wien  web: www.zid.tuwien.ac.at
Zentraler Informatikdienst (ZID) tel: +43(1)58801-42011
Wiedner Hauptstr. 8-10, A-1040 WienDVR: 0005886
---

Re: Client certs on tcp and securing stats socket

[Baptiste]

On Sat, Aug 2, 2014 at 2:22 AM, Meyers, Donovan
donovan_mey...@playstation.sony.com wrote:

 Hi Conrad,

 Thanks for your excellent suggestion!

 I did a quick test and got the same results with https. I was able to
 connect with no client certificate, despite my config that should require
 one.

 I've moved on to other things, since it looks like I won't need this
 functionality after all. But I am curious about what's going on.

 D


 On 7/24/14 7:00 PM, Conrad Hoffmann con...@soundcloud.com wrote:

Hey Donovan,

I never used client certificates in haproxy, but have you tried the
setup with a regular HTTP backend instead of the stats socket? That is
something that should work and you could use to verify that your certs
are working correctly...

Cheers,
Conrad



Hi Donovan,

You should have find some answers here:
http://blog.haproxy.com/2012/10/03/ssl-client-certificate-management-at-application-level/

If it is not a question of configuration, then it could be related to
your certificates and the way you generated them.

Also, is there any reason why you're not setting up SSL directly on
the stats listen section?
(I mean why do you need to proxy the connection to a loopback interface)

Baptiste

Re: Client certs on tcp and securing stats socket

[Meyers, Donovan]

Hi Conrad,

Thanks for your excellent suggestion!

I did a quick test and got the same results with https. I was able to
connect with no client certificate, despite my config that should require
one.

I've moved on to other things, since it looks like I won't need this
functionality after all. But I am curious about what's going on.

D


On 7/24/14 7:00 PM, Conrad Hoffmann con...@soundcloud.com wrote:

Hey Donovan,

I never used client certificates in haproxy, but have you tried the
setup with a regular HTTP backend instead of the stats socket? That is
something that should work and you could use to verify that your certs
are working correctly...

Cheers,
Conrad

Re: Client certs on tcp and securing stats socket

[Conrad Hoffmann]

Hey Donovan,

I never used client certificates in haproxy, but have you tried the
setup with a regular HTTP backend instead of the stats socket? That is
something that should work and you could use to verify that your certs
are working correctly...

Cheers,
Conrad

On 07/24/2014 09:36 PM, Meyers, Donovan wrote:
 
 Any ideas why verify required is allowing connections with no client
 cert?
 
 Could this be an haproxy bug?
 
 I hate to even ask that, because it's so presumptuous and this software is
 so well-written. It's one thousand times more likely that I'm doing
 something wrong.
 
 But from where I sit it seems simple:
 - I used verify require which should require a client cert
 - I can connect without specifying a client cert
 
 
 
 So, am I
 
 
  - misusing/misunderstanding verify require?
  - screwing up some other piece of haproxy config?
  - using the wrong socat/openssl syntax to send (or not) the client cert?
  - thinking a client cert is valid with tcp when it isn't?
 
 I'm not sure what else to try, without spending money on a real client
 cert, which doesn't seem like it should matter. Hopefully someone can tell
 me what I'm doing wrong!
 
 
 Also, sorry about the HTML email for my original post.
 
 Best,
 Donovan
 
 
 From:  Meyers, Donovan Meyers donovan_mey...@playstation.sony.com
 Date:  Tuesday, July 22, 2014 3:33 PM
 To:  haproxy@formilux.org haproxy@formilux.org
 Subject:  Client certs on tcp and securing stats socket
 
 
 Hi all,
 
 First of all, thanks again and always for the great work and an excellent
 product.
 
 I'm looking for a secure way to communicate with the haproxy stats socket
 for remote management of haproxy (without ssh), preferably without running
 some extra API service or anything.
 
 I love that 1.5 can now listen on a socket instead of a file. My concern
 is securing it.
 
 My current best idea is to have the stats socket on 127.0.0.1:7828, and to
 have haproxy handle authentication before forwarding remote connections to
 it. The only authentication I could think of would be client SSL
 certificates.
 
 Having the socket on 127.0.0.1:7828 works great -- you can connect from
 locally on the box but not from elsewhere. Having a TCP-mode proxy
 listening and forwarding to there works great too, even with SSL. Haproxy
 is proxying connections to its own stats
  socket!
 
 Where I'm running into trouble is requiring a client cert -- even with
 verify required on my frontend (or listen) bind line, I can connect with
 no client cert.
 
 My intention is to use an ACL to check for specific information in the
 client cert (using ssl_c_s_dn, ssl_c_serial, and/or ssl_c_sha1). I also
 ran into trouble matching against that information, but then I discovered
 that I wasn't even requiring a client
  cert.
 
 I'm using all self-signed certs. Maybe there's an issue with the way I'm
 generating certs that's causing this? Here's how I generated them:
 
 FILENAME=haproxy-server
 openssl genrsa -out $FILENAME.key 1024
 openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt
 cat $FILENAME.key $FILENAME.crt $FILENAME.pem
 
 FILENAME=haproxy-client
 openssl genrsa -out $FILENAME.key 1024
 openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt
 cat $FILENAME.key $FILENAME.crt $FILENAME.pem
 
 At first I didn't enter any info for the certs (just pressed Enter on each
 line), but then I recreated the client cert with some valid info.
 
 Here's my config:
 
 global
   daemon
   pidfile  /var/run/haproxy.pid
   stats socket ipv4@127.0.0.1:7828
   log 127.0.0.1 local0 debug
 
 defaults
   log  global
   option  redispatch
   retries  3
   timeout  http-request 10s
   timeout  queue 1m
   timeout  connect 10s
   timeout  client 1m
   timeout  server 1m
   timeout  check 10s
 
 listen default_stats
   bind 0.0.0.0:10800
   mode  http
   stats  enable
 
 listen main-service
   bind 0.0.0.0:80
   mode  http
 
 frontend stats-socket
   bind 10.108.0.166:7828 ssl crt /root/haproxy-ssl/haproxy-server.pem
 verify required ca-file /root/haproxy-ssl/haproxy-client.crt
   mode tcp
   log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ %Tw/%Tc/%Tt\ %B\ %ts\
 %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %{+Q}[ssl_c_s_dn]
   default_backend local_socket
 
 backend local_socket
   server localhost 127.0.0.1:7828
 
 Here's my haproxy version and deets:
 
 HA-Proxy version 1.5.2 2014/07/12
 Copyright 2000-2014 Willy Tarreau w...@1wt.eu
 
 Build options :
   TARGET  = linux2628
   CPU = generic
   CC  = gcc
   CFLAGS  = -O2 -g -fno-strict-aliasing
   OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1
 
 Default settings :
   maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200
 
 Encrypted password support via crypt(3): yes
 Built with zlib version : 1.2.3
 Compression algorithms supported : identity, deflate, gzip
 Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
 Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
 OpenSSL library supports TLS extensions

Client certs on tcp and securing stats socket

[Meyers, Donovan]

Hi all,

First of all, thanks again and always for the great work and an excellent 
product.

I'm looking for a secure way to communicate with the haproxy stats socket for 
remote management of haproxy (without ssh), preferably without running some 
extra API service or anything.

I love that 1.5 can now listen on a socket instead of a file. My concern is 
securing it.

My current best idea is to have the stats socket on 127.0.0.1:7828, and to have 
haproxy handle authentication before forwarding remote connections to it. The 
only authentication I could think of would be client SSL certificates.

Having the socket on 127.0.0.1:7828 works great -- you can connect from locally 
on the box but not from elsewhere. Having a TCP-mode proxy listening and 
forwarding to there works great too, even with SSL. Haproxy is proxying 
connections to its own stats socket!

Where I'm running into trouble is requiring a client cert -- even with verify 
required on my frontend (or listen) bind line, I can connect with no client 
cert.

My intention is to use an ACL to check for specific information in the client 
cert (using ssl_c_s_dn, ssl_c_serial, and/or ssl_c_sha1). I also ran into 
trouble matching against that information, but then I discovered that I wasn't 
even requiring a client cert.

I'm using all self-signed certs. Maybe there's an issue with the way I'm 
generating certs that's causing this? Here's how I generated them:

FILENAME=haproxy-server
openssl genrsa -out $FILENAME.key 1024
openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt
cat $FILENAME.key $FILENAME.crt $FILENAME.pem

FILENAME=haproxy-client
openssl genrsa -out $FILENAME.key 1024
openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt
cat $FILENAME.key $FILENAME.crt $FILENAME.pem

At first I didn't enter any info for the certs (just pressed Enter on each 
line), but then I recreated the client cert with some valid info.

Here's my config:

global
  daemon
  pidfile  /var/run/haproxy.pid
  stats socket ipv4@127.0.0.1:7828
  log 127.0.0.1 local0 debug

defaults
  log  global
  option  redispatch
  retries  3
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s

listen default_stats
  bind 0.0.0.0:10800
  mode  http
  stats  enable

listen main-service
  bind 0.0.0.0:80
  mode  http

frontend stats-socket
  bind 10.108.0.166:7828 ssl crt /root/haproxy-ssl/haproxy-server.pem verify 
required ca-file /root/haproxy-ssl/haproxy-client.crt
  mode tcp
  log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ %Tw/%Tc/%Tt\ %B\ %ts\ 
%ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %{+Q}[ssl_c_s_dn]
  default_backend local_socket

backend local_socket
  server localhost 127.0.0.1:7828

Here's my haproxy version and deets:

HA-Proxy version 1.5.2 2014/07/12
Copyright 2000-2014 Willy Tarreau w...@1wt.eu

Build options :
  TARGET  = linux2628
  CPU = generic
  CC  = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing
  OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.3
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 7.8 2008-09-05
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT 
IP_FREEBIND

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Here's how I'm connecting with a client cert:

socat stdio 
openssl-connect:domeyers-haproxy-socket-test.gha.usw1.cld.scea.com:7828,cert=/root/haproxy-ssl/haproxy-client.pem,cafile=/root/haproxy-ssl/haproxy-server.crt
OR (less often - I tried openssl in case there was a problem with my socat)
openssl s_client -connect 
domeyers-haproxy-socket-test.gha.usw1.cld.scea.com:7828 -cert 
./haproxy-client.crt -key ./haproxy-client.key

And without:

socat stdio 
openssl-connect:domeyers-haproxy-socket-test.gha.usw1.cld.scea.com:7828,cafile=/root/haproxy-ssl/haproxy-server.crt
openssl s_client -connect 
domeyers-haproxy-socket-test.gha.usw1.cld.scea.com:7828

Here is it allowing my connection, without a client cert, despite the config 
above:

[root@domeyers-haproxy-socket-test haproxy-ssl]# socat stdio 
openssl-connect:domeyers-haproxy-socket-test.gha.usw1.cld.scea.com:7828,cafile=/root/haproxy-ssl/haproxy-server.crt
show stat
# 
pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid

Stats Socket

[Daniel Schultze]

Developers,

Is anyone working on on a feature to make the stats socket more
deterministic or sane. As is documented in 1.5 any haproxy instance
may respond on the stat socket file. I would like to see only the most
recent instance of haproxy respond to the stats socket file and an
option to make the stats socket file append its pid so that every
instance is able to be communicated with.

I would like to know: Is anyone is already working on this? and is
this a sane feature?

-Dan

Re: Stats Socket

[Nenad Merdanovic]

Hello Daniel,

On 07/03/2014 12:06 AM, Daniel Schultze wrote: Developers,

 Is anyone working on on a feature to make the stats socket more
 deterministic or sane. As is documented in 1.5 any haproxy instance
 may respond on the stat socket file. I would like to see only the most
 recent instance of haproxy respond to the stats socket file and an
 option to make the stats socket file append its pid so that every
 instance is able to be communicated with.

 I would like to know: Is anyone is already working on this? and is
 this a sane feature?

 -Dan


You can already select which HAproxy process will reply to which socket
by using the 'process' bind option [1] on the stats socket. This way you
can define socket-per-process and be able to communicate with all of them.

[1]
http://cbonte.github.io/haproxy-dconv/snapshot/configuration-1.5.html#5.1-process

Regards,
-- 
Nenad Merdanovic | PGP: 0x423edcb2 | Web: http://nimzo.info
Linkedin: http://www.linkedin.com/in/nenadmerdanovic

Re: Haproxy stats socket with nbproc1

[Russell Cohen]

So what is the current behavior? Does the stats socket function with nbproc  
1, or will it only modify one of the two processes if you drain / disable 
a server, etc.)

Russell

RE: Haproxy stats socket with nbproc1

[Lukas Tribus]

Hi!

 So what is the current behavior? Does the stats socket function with nbproc
 1

No.



 or will it only modify one of the two processes if you drain / disable
 a server, etc.)

Yes, it will only modify the process where the stats process runs on, so in
other words the behavior will be completely random.



Regards,

Lukas 

Re: about stats socket per process

[Willy Tarreau]

On Mon, Sep 30, 2013 at 04:07:00PM +0400, Avatar wrote:
 I would like to ask you about any perspective in implementation
 statistics socket at each process in nbproc. Do you have any plans to
 work on it?

It was started a few months ago but ended in a maelstorm where many other
things had to be changed (including the peers to stick table relations).
So for now the work was paused, in the hope to start again after we finish
the ongoing things.

Willy

about stats socket per process

[Avatar]

I would like to ask you about any perspective in implementation
statistics socket at each process in nbproc. Do you have any plans to
work on it?

Thanks.

-- 
The more you know, the less you need.
Rgrds, Pavel Morozov

RE: Remotely accessible stats socket and HATop

[Simon Green]

Hi Willy,

Ref: Since a recent enough version, maybe dev16 or 17, you can even have your 
stats socket listen to TCP directly :-)

How is that done? I'm struggling to find it in the docs. That would be much 
cleaner than using socat!

Simon

-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu] 
Sent: 24 June 2013 21:06
To: Simon Green
Cc: haproxy@formilux.org
Subject: Re: Remotely accessible stats socket and HATop

Hi Simon,

On Sun, Jun 23, 2013 at 10:17:29PM +, Simon Green wrote:
 Hi List,
 
 We had a requirement to be able to put servers in and out of maint 
 mode remotely from a script. To facilitate this we exposed the stats 
 socket using socat, and wrote an init script to do that for us:
 https://github.com/Wirehive/haproxy-remote

Since a recent enough version, maybe dev16 or 17, you can even have your stats 
socket listen to TCP directly :-)

 We then thought it would be nice, as we manage a lot of HAProxy 
 instances, to be able to use HATop (http://feurix.org/projects/hatop/) 
 remotely, so we've modified HATop to accept a host and port for TCP socket 
 connections:
 https://github.com/Wirehive/hatop
 
 I'm not sure how useful this will be for everyone else, but I thought 
 I'd share it just in case :)

Good idea, I think that can be useful especially for people who monitor special 
events (eg stores selling with high rebates for one day, etc...).
 
Regards,
Willy

Re: Remotely accessible stats socket and HATop

[Baptiste]

Hey,

Can you give a try to
stats socket ipv4@a.b.c.d:1234

You can even cipher the connection and even use a client certificate
to allow people using the socket.

Baptiste


Baptiste

On Tue, Jul 9, 2013 at 10:48 AM, Simon Green si...@wirehive.net wrote:
 Hi Willy,

 Ref: Since a recent enough version, maybe dev16 or 17, you can even have 
 your stats socket listen to TCP directly :-)

 How is that done? I'm struggling to find it in the docs. That would be much 
 cleaner than using socat!

 Simon

 -Original Message-
 From: Willy Tarreau [mailto:w...@1wt.eu]
 Sent: 24 June 2013 21:06
 To: Simon Green
 Cc: haproxy@formilux.org
 Subject: Re: Remotely accessible stats socket and HATop

 Hi Simon,

 On Sun, Jun 23, 2013 at 10:17:29PM +, Simon Green wrote:
 Hi List,

 We had a requirement to be able to put servers in and out of maint
 mode remotely from a script. To facilitate this we exposed the stats
 socket using socat, and wrote an init script to do that for us:
 https://github.com/Wirehive/haproxy-remote

 Since a recent enough version, maybe dev16 or 17, you can even have your 
 stats socket listen to TCP directly :-)

 We then thought it would be nice, as we manage a lot of HAProxy
 instances, to be able to use HATop (http://feurix.org/projects/hatop/)
 remotely, so we've modified HATop to accept a host and port for TCP socket 
 connections:
 https://github.com/Wirehive/hatop

 I'm not sure how useful this will be for everyone else, but I thought
 I'd share it just in case :)

 Good idea, I think that can be useful especially for people who monitor 
 special events (eg stores selling with high rebates for one day, etc...).

 Regards,
 Willy

Re: Remotely accessible stats socket and HATop

[Willy Tarreau]

Hi Simon,

On Sun, Jun 23, 2013 at 10:17:29PM +, Simon Green wrote:
 Hi List,
 
 We had a requirement to be able to put servers in and out of maint mode
 remotely from a script. To facilitate this we exposed the stats socket using
 socat, and wrote an init script to do that for us:
 https://github.com/Wirehive/haproxy-remote

Since a recent enough version, maybe dev16 or 17, you can even have your
stats socket listen to TCP directly :-)

 We then thought it would be nice, as we manage a lot of HAProxy instances, to
 be able to use HATop (http://feurix.org/projects/hatop/) remotely, so we've
 modified HATop to accept a host and port for TCP socket connections:
 https://github.com/Wirehive/hatop
 
 I'm not sure how useful this will be for everyone else, but I thought I'd
 share it just in case :)

Good idea, I think that can be useful especially for people who monitor
special events (eg stores selling with high rebates for one day, etc...).
 
Regards,
Willy

Per process stats socket

[Vincent Bernat]

Hi!

Actually, the Unix stats socket is global. It is possible to pin it to
some processes, but it is not possible to have one Unix socket per
process. Has someone already tried to add the appropriate code to
declare several Unix sockets?

Thanks.
-- 
printk(Entering UltraSMPenguin Mode...\n);
2.2.16 /usr/src/linux/arch/sparc64/kernel/smp.c

Re: Per process stats socket

[Willy Tarreau]

Hi Vincent,

On Tue, Feb 26, 2013 at 10:19:48AM +0100, Vincent Bernat wrote:
 Hi!
 
 Actually, the Unix stats socket is global. It is possible to pin it to
 some processes, but it is not possible to have one Unix socket per
 process. Has someone already tried to add the appropriate code to
 declare several Unix sockets?

There are ongoing changes about this. Right now, the bind_proc setting
is per-frontend and I'd like to move it per-listener, so that we can
start one stats socket per process. There are some complexities in doing
this caused by the fact that the frontend has to be bound to all processes
it has listeners on, which slightly changes the way the bind-process
setting is computed. But I have established an algorithm to solve that
that I just need time to implement. After that we should be able to
benefit from a per-bind bind-process setting, which will then also
be available for the stats socket statements.

Regards,
Willy

'stats socket' doesn't accept 'level' parameter

[Frederik Festi]

Hi,

I'm trying set up HAProxy with the UNIX socket and admin access to 
change the weights and add/remove servers. But when i include the 
'level' parameter HAProxy gives me an error on startup. Without the 
'level' parameter the 'show info' command works but when i try to to use 
'remove server' i get command not found, followed by a list of possible 
commands which only has 'list info' in it.


The error on startup when i include the 'level' parameter:
[ALERT] 302/195316 (16456) : parsing [/etc/haproxy.cfg:4] : 'stats 
socket' only supports 'user', 'uid', 'group', 'gid', and 'mode'
[ALERT] 302/195316 (16456) : Error(s) found in configuration file : 
/etc/haproxy.cfg

[ALERT] 302/195316 (16456) : Fatal errors found in configuration.

Here's the commad line i use to start HAProxy: ~/temp/haproxy-1.4.22$ 
sudo haproxy -f /etc/haproxy.cfg -p haproxyPID.txt



The config I'm using:

global
daemon
maxconn 256
stats socket /tmp/haproxy level admin

defaults
mode http
timeout connect 5000ms
timeout client 5ms
timeout server 5ms

frontend http-in
bind *:80
default_backend vmpool1

backend vmpool1
server s1 {some-ip-address}:80 maxconn 32 check
server s2 {another-ip-address}:80 maxconn 32 check

listen admin
bind *:8080
stats enable


The Server runs an Unbuntu distribution:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=10.04
DISTRIB_CODENAME=lucid
DISTRIB_DESCRIPTION=Ubuntu 10.04.4 LTS

Kernel Version: 2.6.32-25-hvmflexiant

I've build HAProxy 1.4.22 from source with 'linux24', 'linux26' and 
'default' as TARGET parameter. and i've tried version HAProxy 1.3.26 too 
same result. Can someone tell me what I'm doing wrong.


mfg Frederik

Re: Haproxy stats socket with nbproc1

[Willy Tarreau]

On Wed, Jul 25, 2012 at 05:14:38PM +0200, Stojan Rancic (Iprom) wrote:
 On 25.7.2012 15:12, Vincent Bernat wrote:
   ??? 25 juillet 2012 12:00 CEST, Stojan Rancic (Iprom) 
   sto...@iprom.si :
 
 is it possible to create a stats socket (or per-process stats socket)
 when haproxy is configured with 'nbproc=2' or higher ?
 
 Hi!
 
 I am unsure if this is possible with a socket. However, you can do it
 for the web interface.
 
 It looks like it's only possible to configure the socket in the global 
 configuration section. Thanks for the hint anyway :)

Indeed we can't do this with the global stats sock. I remember having
been thinking about doing something as simple as auto-numbering to have
multiple sockets (one per process) but in the process of doing so, I
faced some issues related to the reloading process, so in the end I
gave up.

Instead I would rather implement cross-process bounces, eg you connect
to a socket on any process, then type enter XX where XX is the ID
of another process, then haproxy will simply proxy the request to the
other process via an internal socket. Just needs a bit of time to be
done, as usual :-)

Regards,
Willy

Re: Haproxy stats socket with nbproc1

[Vincent Bernat]

 ❦ 25 juillet 2012 12:00 CEST, Stojan Rancic (Iprom) sto...@iprom.si :

 is it possible to create a stats socket (or per-process stats socket)
 when haproxy is configured with 'nbproc=2' or higher ?

Hi!

I am unsure if this is possible with a socket. However, you can do it
for the web interface.

listen monitoring-1
  bind 127.0.0.1:64001
  bind-process 1
  stats enable
  stats uri /stats?haproxy
  stats refresh 5s
listen monitoring-2
  bind 127.0.0.1:64002
  bind-process 2
  stats enable
  stats uri /stats?haproxy
  stats refresh 5s

-- 
Treat end of file conditions in a uniform manner.
- The Elements of Programming Style (Kernighan  Plauger)

Re: Haproxy stats socket with nbproc1

[Stojan Rancic (Iprom)]

On 25.7.2012 15:12, Vincent Bernat wrote:

  ❦ 25 juillet 2012 12:00 CEST, Stojan Rancic (Iprom) sto...@iprom.si :


is it possible to create a stats socket (or per-process stats socket)
when haproxy is configured with 'nbproc=2' or higher ?


Hi!

I am unsure if this is possible with a socket. However, you can do it
for the web interface.


It looks like it's only possible to configure the socket in the global 
configuration section. Thanks for the hint anyway :)


br, Stojan

Re: [PATCH] Avoid truncation of the path used for the stats socket

[Willy Tarreau]

On Thu, Jan 20, 2011 at 10:44:03AM +0100, Andreas Kohn wrote:
 On Wed, 2011-01-19 at 20:38 +0100, Andreas Kohn wrote:
  Hi,
  
  while working further on the changes to allow for dynamic
  adding/removing of backend servers we noticed a potential problem: the
  path given for the 'stats socket' global option may get truncated when
  copying it into the sockaddr_un.sun_path field.
  
  Attached patch checks the length, and reports an error if truncation
  would happen.
  
 
 After some more digging around in the code I found standard.c:str2sun(),
 which does check the length. So in the interest of avoiding duplication
 attached a slightly different patch with the same goal.

Patch applied to both 1.4 and 1.5.

Thanks Andreas!
Willy

Re: [PATCH] Avoid truncation of the path used for the stats socket

[Andreas Kohn]

On Wed, 2011-01-19 at 20:38 +0100, Andreas Kohn wrote:
 Hi,
 
 while working further on the changes to allow for dynamic
 adding/removing of backend servers we noticed a potential problem: the
 path given for the 'stats socket' global option may get truncated when
 copying it into the sockaddr_un.sun_path field.
 
 Attached patch checks the length, and reports an error if truncation
 would happen.
 

After some more digging around in the code I found standard.c:str2sun(),
which does check the length. So in the interest of avoiding duplication
attached a slightly different patch with the same goal.

--
Andreas

-- 
Never attribute to malice that which can be adequately explained by
stupidity.-- Hanlon's Razor
From b0b51f9d351cb99c43212e13c79496ab974fc3ef Mon Sep 17 00:00:00 2001
From: Andreas Kohn andre...@fredhopper.com
Date: Wed, 19 Jan 2011 20:29:32 +0100
Subject: [PATCH] Check whether the path given for the stats socket actually fits into the sockaddr_un structure to avoid truncation.

This uses str2sun() which already does the checking instead of rolling a duplicate variant here.
---
 src/dumpstats.c |   12 +++-
 1 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/src/dumpstats.c b/src/dumpstats.c
index a45d40a..980b284 100644
--- a/src/dumpstats.c
+++ b/src/dumpstats.c
@@ -151,7 +151,7 @@ static int stats_parse_global(char **args, int section_type, struct proxy *curpx
 {
 	args++;
 	if (!strcmp(args[0], socket)) {
-		struct sockaddr_un su;
+		struct sockaddr_un *su;
 		int cur_arg;
 
 		if (*args[1] == 0) {
@@ -164,10 +164,12 @@ static int stats_parse_global(char **args, int section_type, struct proxy *curpx
 			return -1;
 		}
 
-		su.sun_family = AF_UNIX;
-		strncpy(su.sun_path, args[1], sizeof(su.sun_path));
-		su.sun_path[sizeof(su.sun_path) - 1] = 0;
-		memcpy(global.stats_sock.addr, su, sizeof(su)); // guaranteed to fit
+		su = str2sun(args[1]);
+		if (!su) {
+			snprintf(err, errlen, 'stats socket' path would require truncation);
+			return -1;
+		}
+		memcpy(global.stats_sock.addr, su, sizeof(struct sockaddr_un)); // guaranteed to fit
 
 		if (!global.stats_fe) {
 			if ((global.stats_fe = alloc_stats_fe(GLOBAL)) == NULL) {
-- 
1.7.2.3



signature.asc
Description: This is a digitally signed message part

[PATCH] Avoid truncation of the path used for the stats socket

[Andreas Kohn]

Hi,

while working further on the changes to allow for dynamic
adding/removing of backend servers we noticed a potential problem: the
path given for the 'stats socket' global option may get truncated when
copying it into the sockaddr_un.sun_path field.

Attached patch checks the length, and reports an error if truncation
would happen.

This issue was noticed by Joerg Sonnenberger jo...@netbsd.org.


Regards,
--
Andreas
-- 
Never attribute to malice that which can be adequately explained by
stupidity.-- Hanlon's Razor
From 5cb754dc2e408e4eb1ca55792247cb449087f6c5 Mon Sep 17 00:00:00 2001
From: Andreas Kohn andre...@fredhopper.com
Date: Wed, 19 Jan 2011 20:29:32 +0100
Subject: [PATCH] Check whether the path given for the stats socket actually fits into the sockaddr_un structure to avoid truncation.

Noticed-by: Joerg Sonnenberger jo...@netbsd.org
---
 src/dumpstats.c |4 
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/src/dumpstats.c b/src/dumpstats.c
index a45d40a..d77812e 100644
--- a/src/dumpstats.c
+++ b/src/dumpstats.c
@@ -164,6 +164,10 @@ static int stats_parse_global(char **args, int section_type, struct proxy *curpx
 			return -1;
 		}
 
+		if (strlen(args[1]) = sizeof(su.sun_path)) {
+			snprintf(err, errlen, 'stats socket' path would require truncation, path must not be longer than %zu bytes, sizeof(su.sun_path));
+			return -1;
+		}
 		su.sun_family = AF_UNIX;
 		strncpy(su.sun_path, args[1], sizeof(su.sun_path));
 		su.sun_path[sizeof(su.sun_path) - 1] = 0;
-- 
1.7.2.3



signature.asc
Description: This is a digitally signed message part

HAProxy init and stats socket script

[carlo flores]

Hi folks.

For our company's convenience, we prefer to have our HAProxy unit
script also allow us to talk to the stats socket.  We also do a couple
other things, like show connections with a status argument. Perhaps
you can use it too.

Code and readme here: https://github.com/flores/haproxyctl

A copy of that readme with examples is below. Note you need to enclose
something like show sess (spaces) in quotes.   Suggestions or
requests for features is appreciated!

-

Simple wrapper to make life with HAProxy a little more convenient.  It talks to
or creates the stats socket and pid file, and doubles as an init script for
those folks who installed haproxy from source.

Here's a blog post about the inspiration for it:
http://borderstylo.com/posts/228-haproxy-the-quick-and-the-dirty

Examples of usage:

# ./haproxyctl
usage: ./haproxyctl argument
where argument can be:
  start  : start haproxy unless it is already running
  stop   : stop an existing haproxy
  restart: restart the process
  status : is haproxy running?  on what ports per lsof?
  configcheck: check /etc/haproxy/haproxy.cfg
  cloudkick  : cloudkick.com-friendly status and metric for connected users
  clear counters : clear max statistics counters (add 'all' for all counters)
  help   : this message
  prompt : toggle interactive mode with prompt
  quit   : disconnect
  show info  : report information about the running process
  show stat  : report counters for each proxy and server
  show errors: report last request and response errors for each proxy
  show sess [id] : report the list of current sessions or dump this session
  get weight : report a server's current weight
  set weight : change a server's weight
  set timeout: change a timeout setting
  disable server : set a server in maintenance mode
  enable server  : re-enable a server that was previously in maintenance mode

# ./haproxyctl status
haproxy is running.  these ports are used and guys are connected.
*:www (LISTEN)
*:49122

# ./haproxyctl show stat
# 
pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,
http,FRONTEND,,,3,3,2000,4,1641,4353,0,0,0,OPEN,1,1,00,0,0,20,2,2,0,0,0,,0,2,4,,,
ei,guinea,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,0,0,28,0,,1,2,1,,0,,2,0,,0,L4OK,,0,0,0,0,0,0,0,00,0,
ei,belem,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,0,1,0,0,28,0,,1,2,2,,0,,2,0,,0,L4OK,,42,0,0,0,0,0,0,00,0,
ei,BACKEND,0,0,0,0,0,0,0,0,0,0,,0,0,0,0,UP,1,1,1,,0,28,0,,1,2,0,,0,,1,0,,00,0,0,0,0,0,0,0,
apache,guinea,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,0,0,28,0,,1,3,1,,0,,2,0,,0,L4OK,,0,0,0,0,0,0,0,00,0,
apache,belem,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,0,1,0,0,28,0,,1,3,2,,0,,2,0,,0,L4OK,,42,0,0,0,0,0,0,00,0,
apache,BACKEND,0,0,0,0,0,0,0,0,0,0,,0,0,0,0,UP,1,1,1,,0,28,0,,1,3,0,,0,,1,0,,00,0,0,0,0,0,0,0,
sinatra,sinatra_downoi,0,0,0,1,,1,484,3641,,0,,0,0,0,0,UP,1,1,0,0,0,28,0,,1,4,1,,1,,2,0,,1,L4OK,,46,0,1,0,0,0,0,00,0,
sinatra,sinatra_rindica,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,0,0,28,0,,1,4,2,,0,,2,0,,0,L4OK,,45,0,0,0,0,0,0,00,0,
sinatra,sinatra_guinea,0,0,0,0,,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,2,2,,1,4,3,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,00,0,
sinatra,BACKEND,0,0,0,1,0,1,484,3641,0,0,,0,0,0,0,UP,2,2,0,,0,28,0,,1,4,0,,1,,1,0,,10,1,0,0,0,0,0,0,
static,ngnix_downoi,0,0,0,1,,1,153,264,,0,,0,0,0,0,UP
1/2,1,1,0,0,0,28,0,,1,5,1,,1,,2,0,,1,INI,,,0,1,0,0,0,0,00,0,
static,ngnix_petite,0,0,0,1,,1,522,230,,0,,0,0,0,0,UP
1/2,1,1,0,0,0,28,0,,1,5,2,,1,,2,0,,1,INI,,,0,0,1,0,0,0,00,0,
static,ngnix_rindica,0,0,0,1,,3,0,0,,0,,0,0,2,1,UP
1/2,1,1,0,0,0,28,0,,1,5,3,,1,,2,0,,1,INI,,,0,0,0,0,0,0,00,0,
static,nginx_stellatus,0,0,0,1,,1,482,218,,0,,0,0,0,0,UP
1/2,1,1,0,0,0,28,0,,1,5,4,,1,,2,0,,1,INI,,,0,0,1,0,0,0,00,0,
static,nginx_belem,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP
1/2,1,1,0,0,0,28,0,,1,5,5,,0,,2,0,,0,INI,,,0,0,0,0,0,0,00,0,
static,nginx_petite,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP
1/2,1,1,0,0,0,28,0,,1,5,6,,0,,2,0,,0,INI,,,0,0,0,0,0,0,00,0,
static,apache_guinea,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP
1/2,1,1,0,0,0,28,0,,1,5,7,,0,,2,0,,0,INI,,,0,0,0,0,0,0,00,0,
static,BACKEND,0,0,0,1,0,3,1157,712,0,0,,0,0,2,1,UP,7,7,0,,0,28,0,,1,5,0,,4,,1,0,,20,1,2,0,0,0,0,0,


The real magic in the script (talking to the stats socket) is under the else
case in the ARGV argument, or here: https://gist.github.com/632613

Thank you for HAProxy and a great list!
Carlo

Re: stats socket issues with concurrent accesses

[Cyril Bonté]

Le Dimanche 14 Mars 2010 19:58:08, Willy Tarreau a écrit :
 Here's the patch I'm going to merge. I did a bunch of tests on it
 (both unix and tcp) and it appears OK.

Oh nice ! It works well, I couldn't reproduce the problem with 10 loops running 
for more than 15 minutes.
Thanks.

-- 
Cyril Bonté

stats socket issues with concurrent accesses

[Cyril Bonté]

Hi Willy,

Our monitoring scripts use the unix socket to get haproxy's status. Sometimes 
they detect haproxy DOWN when it's not really the case.
After some tests, it appears that 2 concurrent accesses break things : one of 
the request receives an empty reply. I wonder if it can be a more severe issue 
when someone uses administration commands (set weight, disable/enable 
server,...)

At this step I can't give you much more information but I'll look at the code 
this week-end.

Something interesting, I could reproduce the issue with haproxy 1.4.1, 1.3.23, 
1.3.19 and 1.3.16 (only tested these ones) but there's no error with 1.3.15.x 
versions (tested with 1.3.15.12, 1.3.15.11, 1.3.15.10 and 1.3.15.7).
Maybe this can remind you some modifications of code.

To reproduce the issue :
- a simple configuration file :
global
stats socket /tmp/haproxy.socket

listen dummy :1
balance roundrobin

- short script (with 2 or 3 instances in parrallel) :
#!/bin/bash
while true;
do
hainfo=`echo show info| /usr/bin/socat /tmp/haproxy.socket stdio`
if [ $hainfo ==  ];
then
echo ### EMPTY REPLY ###
fi
done

-- 
Cyril Bonté

Re: stats socket issues with concurrent accesses

[Willy Tarreau]

Hi Cyril,

On Fri, Mar 12, 2010 at 05:03:15PM +0100, Cyril Bonté wrote:
 Hi Willy,
 
 Our monitoring scripts use the unix socket to get haproxy's status. Sometimes 
 they detect haproxy DOWN when it's not really the case.

Do you know how much time it takes to observe it ? I'm currently
running it on 1.4.0-3 here. It's been running for the last 10 minutes
with 2, then 3 and now 10 concurrent scripts. For the record in case
that matters, it's running on socat 1.6.0.0 :
  # /root/bin/socat -V
  socat by Gerhard Rieger - see www.dest-unreach.org
  socat version 1.6.0.0 on Oct 28 2007 21:29:34
 running on Linux version #1 Sun Jan 31 00:55:16 CET 2010, release 
2.4.37-wt3-fw, machine i686

 After some tests, it appears that 2 concurrent accesses break things : one of 
 the request receives an empty reply. I wonder if it can be a more severe 
 issue when someone uses administration commands (set weight, disable/enable 
 server,...)

I'm not worried for that because those operations are atomic, so either
they are processed or they are not, there is no risk of leaving them in
a half-processed state. However, if your scripts rely on those commands
to have been succesfully completed, then yes there's a risk that a
command is not considered (just the same as with a process being
restarted, BTW).

 At this step I can't give you much more information but I'll look at the code 
 this week-end.

OK.

 Something interesting, I could reproduce the issue with haproxy 1.4.1, 
 1.3.23, 1.3.19 and 1.3.16 (only tested these ones) but there's no error with 
 1.3.15.x versions (tested with 1.3.15.12, 1.3.15.11, 1.3.15.10 and 1.3.15.7).
 Maybe this can remind you some modifications of code.

Not much as most of the rework happened between 1.3 and 1.4. In fact,
some part of the work also happened between 1.3.15 and 1.3.16 but it
was the low-level I/O which is now common with TCP/HTTP. It would be
nice to try with strace socat instead of socat alone. I wonder
if it's just a scheduling issue sometimes causing socat to close its
output channel after sending the request and before receiving the
response (as we commonly have with netcat). But from my experience,
socat does not seem to abort any transfer after a unidirectional
close. So that would indicate that haproxy's stats output stops
if the input channel closes, which I don't think is the case at
all.

Regards,
Willy

Re: stats socket issues with concurrent accesses

[Cyril Bonté]

Le Vendredi 12 Mars 2010 20:17:14, Willy Tarreau a écrit :
 Hi Cyril,
 
 On Fri, Mar 12, 2010 at 05:03:15PM +0100, Cyril Bonté wrote:
  Hi Willy,
  
  Our monitoring scripts use the unix socket to get haproxy's status. 
  Sometimes they detect haproxy DOWN when it's not really the case.
 
 Do you know how much time it takes to observe it ? I'm currently

It's random but generally I don't wait more than 10 seconds (less than 1500 
loops at home).

 running it on 1.4.0-3 here. It's been running for the last 10 minutes
 with 2, then 3 and now 10 concurrent scripts. For the record in case
 that matters, it's running on socat 1.6.0.0 :
   # /root/bin/socat -V
   socat by Gerhard Rieger - see www.dest-unreach.org
   socat version 1.6.0.0 on Oct 28 2007 21:29:34

At work it should be version 1.6.0.1 (debian lenny package)
Tonight, my tests are done with the version 1.7.1.2.

  running on Linux version #1 Sun Jan 31 00:55:16 CET 2010, release 
 2.4.37-wt3-fw, machine i686

I don't think this makes big differences but my tests were done with 
2.6.{18,24,31,33} kernels.

 Not much as most of the rework happened between 1.3 and 1.4. In fact,
 some part of the work also happened between 1.3.15 and 1.3.16 but it
 was the low-level I/O which is now common with TCP/HTTP. It would be
 nice to try with strace socat instead of socat alone. I wonder
 if it's just a scheduling issue sometimes causing socat to close its
 output channel after sending the request and before receiving the
 response (as we commonly have with netcat).

This might be the case but then it's strange that it doesn't happen with non 
concurrent accesses.
Working trace :
...
stat(/tmp/haproxy.socket, {st_mode=S_IFSOCK|0755, st_size=0, ...}) = 0
socket(PF_FILE, SOCK_STREAM, 0) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)   = 0
connect(3, {sa_family=AF_FILE, path=/tmp/haproxy.socket}, 21) = 0
getsockname(3, {sa_family=AF_FILE, NULL}, [2]) = 0
getsockname(3, {sa_family=AF_FILE, NULL}, [2]) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff6290e590) = -1 EINVAL (Invalid 
argument)
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff6290e590) = -1 EINVAL (Invalid 
argument)
select(4, [0 3], [1 3], [], NULL)   = 3 (in [0], out [1 3])
read(0, show info\n, 8192)= 10
write(3, show info\n, 10) = 10
select(4, [0 3], [3], [], NULL) = 3 (in [0 3], out [3])
read(3, Name: HAProxy\nVersion: 1.3.16\nRe..., 8192) = 255
write(1, Name: HAProxy\nVersion: 1.3.16\nRe..., 255) = 255
read(0, , 8192)   = 0
shutdown(3, 1 /* send */)   = 0
select(4, [3], [1], [], {0, 50})= 2 (in [3], out [1], left {0, 48})
read(3, , 8192)   = 0
shutdown(3, 1 /* send */)   = 0
shutdown(3, 2 /* send and receive */)   = 0
exit_group(0)   = ?

Non working one :
stat(/tmp/haproxy.socket, {st_mode=S_IFSOCK|0755, st_size=0, ...}) = 0
socket(PF_FILE, SOCK_STREAM, 0) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)   = 0
connect(3, {sa_family=AF_FILE, path=/tmp/haproxy.socket}, 21) = 0
getsockname(3, {sa_family=AF_FILE, NULL}, [2]) = 0
getsockname(3, {sa_family=AF_FILE, NULL}, [2]) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff5716e590) = -1 EINVAL (Invalid 
argument)
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff5716e590) = -1 EINVAL (Invalid 
argument)
select(4, [0 3], [1 3], [], NULL)   = 3 (in [0], out [1 3])
read(0, show info\n, 8192)= 10
write(3, show info\n, 10) = 10
select(4, [0 3], [3], [], NULL) = 2 (in [0], out [3])
read(0, , 8192)   = 0
shutdown(3, 1 /* send */)   = 0
select(4, [3], [], [], {0, 50}) = 1 (in [3], left {0, 48})
read(3, , 8192)   = 0
shutdown(3, 1 /* send */)   = 0
shutdown(3, 2 /* send and receive */)   = 0
exit_group(0)   = ?

 But from my experience,
 socat does not seem to abort any transfer after a unidirectional
 close. So that would indicate that haproxy's stats output stops
 if the input channel closes, which I don't think is the case at
 all.
 
 Regards,
 Willy
 
 

-- 
Cyril Bonté