Re: HAProxy 2.2.2 CE issue report

[Илья Шипицин]

please open an issue at https://github.com/haproxy/haproxy/issues
also, do you have collected dump file (as your output says "core dumped"),
maybe coredumpctl or so ?
if you do, it would be great to provide gdb backtrace

вт, 25 авг. 2020 г. в 00:02, Milen Simeonov :

> I've got an issue after HAProxy 2.0.17 upgrade to 2.2.2
>
>
> ## Detailed description of the problem
>
> The following packages will be upgraded:
>   haproxy
> 1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
> Need to get 1,702 kB of archives.
> After this operation, 547 kB of additional disk space will be used.
> Do you want to continue? [Y/n]
> Get:1 http://ppa.launchpad.net/vbernat/haproxy-2.2/ubuntu focal/main
> amd64 haproxy amd64 2.2.2-1ppa1~focal [1,702 kB]
> Fetched 1,702 kB in 1s (1,652 kB/s)
> (Reading database ... 114932 files and directories currently installed.)
> Preparing to unpack .../haproxy_2.2.2-1ppa1~focal_amd64.deb ...
> Unpacking haproxy (2.2.2-1ppa1~focal) over (2.0.15-1ppa1~bionic) ...
> Setting up haproxy (2.2.2-1ppa1~focal) ...
> Job for haproxy.service failed.
> See "systemctl status haproxy.service" and "journalctl -xe" for details.
> invoke-rc.d: initscript haproxy, action "reload" failed.
> Processing triggers for man-db (2.9.1-1) ...
> Processing triggers for rsyslog (8.2001.0-1ubuntu1.1) ...
> Processing triggers for systemd (245.4-4ubuntu3.2) ...
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
>
> service haproxy restart
> Job for haproxy.service failed because a fatal signal was delivered
> causing the control process to dump core.
> See "systemctl status haproxy.service" and "journalctl -xe" for details.
> systemctl status haproxy.service
> ● haproxy.service - HAProxy Load Balancer
>  Loaded: loaded (/lib/systemd/system/haproxy.service; enabled; vendor
> preset: enabled)
>  Active: failed (Result: core-dump) since Mon 2020-08-24 05:20:21 CDT;
> 36s ago
>Docs: man:haproxy(1)
>  file:/usr/share/doc/haproxy/configuration.txt.gz
> Process: 17050 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q
> $EXTRAOPTS (code=dumped, signal=ABRT)
> Aug 24 05:20:21 k2 systemd[1]: haproxy.service: Scheduled restart job,
> restart counter is at 5.
> Aug 24 05:20:21 k2 systemd[1]: Stopped HAProxy Load Balancer.
> Aug 24 05:20:21 k2 systemd[1]: haproxy.service: Start request repeated too
> quickly.
> Aug 24 05:20:21 k2 systemd[1]: haproxy.service: Failed with result
> 'core-dump'.
> Aug 24 05:20:21 k2 systemd[1]: Failed to start HAProxy Load Balancer.
>
> haproxy -f /etc/haproxy/haproxy.cfg -c
> Configuration file is valid
> free(): invalid pointer
> Aborted (core dumped)
>
>
> ## Expected behavior
>
> HAproxy 2.2.2 to work
>
> ## Steps to reproduce the behavior
>
> The following configuration, derived from the one you posted above, is
> enough to reproduce the issue with haproxy 2.2-dev11:
> global
> log /dev/loglocal0
> log /dev/loglocal1 notice
> chroot /var/lib/haproxy
> stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd
> listeners
> stats timeout 30s
> user haproxy
> group haproxy
> daemon
> defaults
> log global
> mode http
> maxconn 5
> option httplog
> option dontlognull
> retries 3
> timeout connect 5s
> timeout client  50s
> timeout client-fin 50s
> timeout server  50s
> frontend fe_main
> bind 127.0.0.1:443 ssl crt-list
> /etc/haproxy/certs/websites.crt_list
> default_backend be_main
> backend be_main
> mode http
> server localhost 127.0.0.1:80
> haproxy -c -f will error out.
>
>
> ## Do you have any idea what may have caused this?
> No, I haven't
>
> ## Do you have an idea how to solve the issue?
> No, I haven't
>
>
>
>
> Kind Regards,
> --
> *Milen Simeonov*
> IT Team Leader
> Kukui Corporation 
>
>
> CONFIDENTIALITY NOTICE: This Email and any attachments may contain
> private, confidential, and privileged material is for the sole use of Kukui
> Corporation to the intended recipient. Any review, copying, or distribution
> of this email and any attachments by others is strictly prohibited. If you
> are not the intended recipient, please contact the sender immediately and
> permanently delete the original and any copies of this email and any
> attachments thereto.
>

[PATCH 6/6] BUG/MEDIUM: spoa/python: Fix ipv4_address used instead of ipv6_address

[gilchrist]

From: Gilchrist Dadaglo 

Typo leads to checking the wrong object for memory issues
---
 contrib/spoa_server/ps_python.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/contrib/spoa_server/ps_python.c b/contrib/spoa_server/ps_python.c
index 468ee43..b6f4ad1 100644
--- a/contrib/spoa_server/ps_python.c
+++ b/contrib/spoa_server/ps_python.c
@@ -410,7 +410,7 @@ static int ps_python_start_worker(struct worker *w)
}
 
ipv6_address = PyObject_GetAttrString(module_ipaddress, "IPv6Address");
-   if (ipv4_address == NULL) {
+   if (ipv6_address == NULL) {
PyErr_Print();
return 0;
}
-- 
2.24.3.AMZN

[PATCH 3/6] BUG/MINOR: spoa/python: Ensure ip address references are freed

[gilchrist]

From: Gilchrist Dadaglo 

IP addresses references passed in argument for ps_python are not freed after 
they have been used.
Leading to a small chance of mem leak if a lot of ip addresses are
passed around
---
 contrib/spoa_server/ps_python.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/contrib/spoa_server/ps_python.c b/contrib/spoa_server/ps_python.c
index 2cc97e3..c27387c 100644
--- a/contrib/spoa_server/ps_python.c
+++ b/contrib/spoa_server/ps_python.c
@@ -239,6 +239,8 @@ static PyObject *ps_python_set_var_ipv4(PyObject *self, 
PyObject *args)
PyErr_SetString(spoa_error, "No space left available");
return NULL;
}
+   /* Once we set the IP value in the worker, we don't need it anymore... 
*/
+   Py_XDECREF(value);
return Py_None;
 }
 
@@ -274,6 +276,8 @@ static PyObject *ps_python_set_var_ipv6(PyObject *self, 
PyObject *args)
PyErr_SetString(spoa_error, "No space left available");
return NULL;
}
+   /* Once we set the IP value in the worker, we don't need it anymore... 
*/
+   Py_XDECREF(value);
return Py_None;
 }
 
-- 
2.24.3.AMZN

[PATCH 5/6] BUG/MINOR: spoa/python: Updating references to free in case of failure

[gilchrist]

From: Gilchrist Dadaglo 

When we encounter a failure, all previously borrowed references should
be freed. Especially if the program is not failing immediately
---
 contrib/spoa_server/ps_python.c | 9 +
 1 file changed, 9 insertions(+)

diff --git a/contrib/spoa_server/ps_python.c b/contrib/spoa_server/ps_python.c
index 811014e..468ee43 100644
--- a/contrib/spoa_server/ps_python.c
+++ b/contrib/spoa_server/ps_python.c
@@ -562,6 +562,7 @@ static int ps_python_exec_message(struct worker *w, void 
*ref, int nargs, struct
key = PY_STRING_FROM_STRING("name");
if (key == NULL) {
Py_DECREF(kw_args);
+   Py_DECREF(ent);
PyErr_Print();
return 0;
}
@@ -627,6 +628,7 @@ static int ps_python_exec_message(struct worker *w, void 
*ref, int nargs, struct
if (func == NULL) {
Py_DECREF(kw_args);
Py_DECREF(ent);
+   Py_DECREF(key);
PyErr_Print();
return 0;
}
@@ -634,6 +636,7 @@ static int ps_python_exec_message(struct worker *w, void 
*ref, int nargs, struct
if (ip_dict == NULL) {
Py_DECREF(kw_args);
Py_DECREF(ent);
+   Py_DECREF(key);
Py_DECREF(func);
PyErr_Print();
return 0;
@@ -642,6 +645,7 @@ static int ps_python_exec_message(struct worker *w, void 
*ref, int nargs, struct
if (ip_name == NULL) {
Py_DECREF(kw_args);
Py_DECREF(ent);
+   Py_DECREF(key);
Py_DECREF(func);
Py_DECREF(ip_dict);
PyErr_Print();
@@ -651,6 +655,7 @@ static int ps_python_exec_message(struct worker *w, void 
*ref, int nargs, struct
if (ip_value == NULL) {
Py_DECREF(kw_args);
Py_DECREF(ent);
+   Py_DECREF(key);
Py_DECREF(func);
Py_DECREF(ip_dict);
Py_DECREF(ip_name);
@@ -661,6 +666,10 @@ static int ps_python_exec_message(struct worker *w, void 
*ref, int nargs, struct
Py_DECREF(ip_name);
Py_DECREF(ip_value);
if (ret == -1) {
+   Py_DECREF(kw_args);
+   Py_DECREF(ent);
+   Py_DECREF(key);
+   Py_DECREF(func);
Py_DECREF(ip_dict);
PyErr_Print();
return 0;
-- 
2.24.3.AMZN

[PATCH 4/6] BUG/MINOR: spoa/python: Do not free reference to NULL

[gilchrist]

From: Gilchrist Dadaglo 

As per https://docs.python.org/3/c-api/refcounting.html, Py_DECREF
should not be called on NULL objects.
---
 contrib/spoa_server/ps_python.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/contrib/spoa_server/ps_python.c b/contrib/spoa_server/ps_python.c
index c27387c..811014e 100644
--- a/contrib/spoa_server/ps_python.c
+++ b/contrib/spoa_server/ps_python.c
@@ -553,7 +553,6 @@ static int ps_python_exec_message(struct worker *w, void 
*ref, int nargs, struct
ent = PyDict_New();
if (ent == NULL) {
Py_DECREF(kw_args);
-   Py_DECREF(ent);
PyErr_Print();
return 0;
}
-- 
2.24.3.AMZN

[PATCH 2/6] BUG/MAJOR: spoa/python: Fix unhandled python call leading to memory leak

[gilchrist]

From: Gilchrist Dadaglo 

The result from spoa evaluation of the user provided python code is
never passed back to the main spoa process nor freed.
Same for the keyword list passed.
This results into the elements never freed by Python as reference count
never goes down.
https://docs.python.org/3/extending/extending.html#reference-counting-in-python
---
 contrib/spoa_server/ps_python.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/contrib/spoa_server/ps_python.c b/contrib/spoa_server/ps_python.c
index 8f002eb..2cc97e3 100644
--- a/contrib/spoa_server/ps_python.c
+++ b/contrib/spoa_server/ps_python.c
@@ -733,10 +733,14 @@ static int ps_python_exec_message(struct worker *w, void 
*ref, int nargs, struct
}
 
result = PyObject_Call(python_ref, empty_array, fkw);
+   Py_DECREF(fkw);
if (result == NULL) {
PyErr_Print();
return 0;
}
+   if (result != Py_None) {
+   Py_DECREF(result);
+   }
 
return 1;
 }
-- 
2.24.3.AMZN

[PATCH 1/6] MINOR: spoa: allow MAX_FRAME_SIZE override

[gilchrist]

From: Bertrand Jacquin 

MAX_FRAME_SIZE is forced to the default value of tune.bufsize, however
they don't necessarily have to be tight together.
---
 contrib/spoa_server/spoa.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/contrib/spoa_server/spoa.h b/contrib/spoa_server/spoa.h
index 8f912e4..8d6d4be 100644
--- a/contrib/spoa_server/spoa.h
+++ b/contrib/spoa_server/spoa.h
@@ -17,7 +17,10 @@
 #include 
 #include 
 
+#ifndef MAX_FRAME_SIZE
 #define MAX_FRAME_SIZE16384
+#endif
+
 #define SPOP_VERSION  "2.0"
 #define SPOA_CAPABILITIES ""
 
-- 
2.24.3.AMZN

HAProxy 2.2.2 CE issue report

[Milen Simeonov]

<<< text/html; charset=utf-8: Unrecognized >>>

RE: Logging using %HP (path) produce different results with H1 and H2

[Pierre Cheynier]

On Fri, Aug 21, 2020 at 8:11 PM William Dauchy  wrote:

So awesome to get the first response from your direct colleague :)

> I believe this is expected; this behaviour has changed since v2.1 though.

Indeed, we don't use this logging variable since a long time, so I'm not really 
able to confirm if this is so new.
Anyway, I understand this is related to handling h2 and its specifics, still I 
think there should be something to fix (in one way or the other) to get back to 
a consistent/deterministic meaning of %HP (and maybe in other places where this 
had an impact).

Willy, any thought about this?

Pierre