Re: Does HAProxy 1.8.0 need new param vs 1.7.9
On Tue, Nov 28, 2017 at 01:16:28PM -0600, Coscend@HAProxy wrote:
> Thank you for your prompt guidance. Result of your insights: SUCCESS in
> loading stats and other pages in both the use cases listed below:
>
> 1). Removed mailers section in 1.8.0.
>
> 2). Compiled 1.8-commit 8f42b19 ("BUG/MEDIUM: tcp-check: Don't lock the
> server in tcpcheck_main")
Great, thanks for confirming!
> Hope the verification helps other users and companies that contribute to the
> betterment of our economy and society.
Definitely :-) All bug reports help.
> Just to expand our understanding for the future, did you nail the issue from
> multi-character process state "Rs", i.e., "runnable on queue""is a session
> leader"?
I didn't even notice it, I just saw "99.5" in the %CPU column.
> Hats off to you for starting this project. HAProxy has empowered us to
> contribute. Should you like to see the impact of HAProxy's indirect
> contribution to common global denizens, please take a look at what HAProxy
> enables in remote disadvantaged areas:
> http://www.coscend.com/Anchor/SSF_Tour/RMCC_Tour/Overview/CoscendCC_Telemedi
> cine_anchor.html
Thanks for the link, you're welcome.
Willy
RE: Does HAProxy 1.8.0 need new param vs 1.7.9
Dear Willy,
Thank you for your prompt guidance. Result of your insights: SUCCESS in
loading stats and other pages in both the use cases listed below:
1). Removed mailers section in 1.8.0.
2). Compiled 1.8-commit 8f42b19 ("BUG/MEDIUM: tcp-check: Don't lock the
server in tcpcheck_main")
Hope the verification helps other users and companies that contribute to the
betterment of our economy and society.
Just to expand our understanding for the future, did you nail the issue from
multi-character process state "Rs", i.e., "runnable on queue""is a session
leader"?
Hats off to you for starting this project. HAProxy has empowered us to
contribute. Should you like to see the impact of HAProxy's indirect
contribution to common global denizens, please take a look at what HAProxy
enables in remote disadvantaged areas:
http://www.coscend.com/Anchor/SSF_Tour/RMCC_Tour/Overview/CoscendCC_Telemedi
cine_anchor.html
Dear Christopher,
Thank you for this timely commit 8f42b19 ("BUG/MEDIUM: tcp-check: Don't lock
the server in tcpcheck_main").
Sincerely,
Hemant K. Sabat
Coscend Communications Solutions
<http://www.coscend.com/> www.Coscend.com
--
Real-time, Interactive Video Collaboration, Tele-healthcare, Tele-education,
Telepresence Services, on the fly.
--
CONFIDENTIALITY NOTICE: See 'Confidentiality Notice Regarding E-mail
Messages from Coscend Communications Solutions' posted at:
<http://www.Coscend.com/Anchor/Common/Terms_and_Conditions.html>
http://www.Coscend.com/Anchor/Common/Terms_and_Conditions.html
-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu]
Sent: Tuesday, November 28, 2017 12:25 PM
To: Coscend@HAProxy
Cc: haproxy@formilux.org
Subject: Re: Does HAProxy 1.8.0 need new param vs 1.7.9
Hi Hemant,
On Tue, Nov 28, 2017 at 12:15:32PM -0600, Coscend@HAProxy wrote:
> Thank you for giving time after a large 1.8.0 release to provide us
> vectors on testing HAProxy work mode. This community's intensity and
> laser focus are a standout. Below is all the information you asked us
> to look at. By any chance, was there any critical change between
> 1.8-dev3 and 1.8-rc series/1.8.0?
Sure, a lot! Too many to enumerate in fact.
> New checks: Loads stats page and accesses all backends
>
> SUCCESS: 1.8-dev0/dev1/dev2/dev3
> FAILS: 1.8-rc1/rc2rc3/rc4 and 1.8.0
>
> Browser message:
>
> This site can't be reached
>
> coscend.com took too long to respond.
Ah OK so it's not a "connection refused" as I thought it was, but the fact
that it apparently hangs. That's a different case. I suspect the process
quickly goes up in a loop at 100% CPU due to a bug. There has been a locking
bug on the mailers section causing this, would you happen to have one such
setting in your config ?
> (1) check if the process is still present (ps aux)
>
> root 2801 0.0 0.1 74928 11560 ?S11:28 0:00
> /usr/local/sbin/haproxy -Ws -V -C /usr/local/haproxy-1.8.0/conf -f
> /usr/local/haproxy-1.8.0/conf -f -D -p
> /var/run/haproxy.pid
>
> haproxy 2802 99.5 0.1 75620 12408 ?Rs 11:28 10:50
^
Bingo, spinning process!
Thanks for all the elements. If you're having a "mailers" section, then the
next nighly snapshot will contain the fix, or you can pick it yourself, it's
commit 8f42b19 ("BUG/MEDIUM: tcp-check: Don't lock the server in
tcpcheck_main").
If you don't have such a section, that's embarrassing, it means there is
another locking bug. In this case I'll ask you for more info off-list.
Regards,
Willy
---
This email has been checked for viruses by AVG.
http://www.avg.com
Re: Does HAProxy 1.8.0 need new param vs 1.7.9
Hi Hemant,
On Tue, Nov 28, 2017 at 12:15:32PM -0600, Coscend@HAProxy wrote:
> Thank you for giving time after a large 1.8.0 release to provide us vectors
> on testing HAProxy work mode. This community's intensity and laser focus
> are a standout. Below is all the information you asked us to look at. By
> any chance, was there any critical change between 1.8-dev3 and 1.8-rc
> series/1.8.0?
Sure, a lot! Too many to enumerate in fact.
> New checks: Loads stats page and accesses all backends
>
> SUCCESS: 1.8-dev0/dev1/dev2/dev3
> FAILS: 1.8-rc1/rc2rc3/rc4 and 1.8.0
>
> Browser message:
>
> This site can't be reached
>
> coscend.com took too long to respond.
Ah OK so it's not a "connection refused" as I thought it was, but the
fact that it apparently hangs. That's a different case. I suspect the
process quickly goes up in a loop at 100% CPU due to a bug. There has
been a locking bug on the mailers section causing this, would you happen
to have one such setting in your config ?
> (1) check if the process is still present (ps aux)
>
> root 2801 0.0 0.1 74928 11560 ?S11:28 0:00
> /usr/local/sbin/haproxy -Ws -V -C /usr/local/haproxy-1.8.0/conf -f
> /usr/local/haproxy-1.8.0/conf -f -D -p
> /var/run/haproxy.pid
>
> haproxy 2802 99.5 0.1 75620 12408 ?Rs 11:28 10:50
^
Bingo, spinning process!
Thanks for all the elements. If you're having a "mailers" section,
then the next nighly snapshot will contain the fix, or you can pick
it yourself, it's commit 8f42b19 ("BUG/MEDIUM: tcp-check: Don't lock
the server in tcpcheck_main").
If you don't have such a section, that's embarrassing, it means there
is another locking bug. In this case I'll ask you for more info off-list.
Regards,
Willy
RE: Does HAProxy 1.8.0 need new param vs 1.7.9
on stats show-node stats uri /HAProxy.stats stats refresh 10s stats show-legends stats show-desc"Master load balancer, reverse proxy server and HTTP security server" -Original Message- From: Willy Tarreau [mailto:w...@1wt.eu] Sent: Tuesday, November 28, 2017 12:46 AM To: Coscend@Coscend Cc: haproxy@formilux.org Subject: Re: Does HAProxy 1.8.0 need new param vs 1.7.9 Hi Hemant, On Mon, Nov 27, 2017 at 12:11:25PM -0600, Coscend@Coscend wrote: > Dear HAProxy community, > > We have been successfully loading Stats page and other applications > via HAProxy 1.7.9. We successfully compiled, installed and ran 1.8.0 > as a systemd service. However, with 1.8.0, we are unable to access > the same stats page or any other application. We are using the same > set of multiple HAProxy configuration files for both 1.8.0 and 1.7.9. > Ports on firewalls are open and policies are enabled, as verified by a working v. 1.7.9. > > No log is being captured by HAProxy during access to these pages / > applications. Router log gives HAProxy is resetting the request. The > Web page on browser states "the connection was reset." > > Would you be kind enough to provide any vectors on what new > configuration parameter we should add / modify for 1.8.0 (different from 1.7.9)? Nothing in theory. Can you check if the process is still present (ps aux) ? If it is, are the ports properly listening (netstat -ltnp) ? > Below is haproxy -vv. Command to start HAProxy: > > CONFIG= > > ExecStart=/usr/local/sbin/haproxy -Ws -V -C $CONFIG -f $CONFIG -f > $ -f $ -D -p $PIDFILE Just out of curiosity, are you certain that the binary in this absolute path is the one you expect ? For example if you had installed it in another place still in your path, haproxy -vv would find the new one but the one above would be the old one and would fail on "-Ws". Do you have anything in systemd's logs related to the startup ? Regards, Willy --- This email has been checked for viruses by AVG. http://www.avg.com
Re: Does HAProxy 1.8.0 need new param vs 1.7.9
Hi Hemant, On Mon, Nov 27, 2017 at 12:11:25PM -0600, Coscend@Coscend wrote: > Dear HAProxy community, > > We have been successfully loading Stats page and other applications via > HAProxy 1.7.9. We successfully compiled, installed and ran 1.8.0 as a > systemd service. However, with 1.8.0, we are unable to access the same > stats page or any other application. We are using the same set of multiple > HAProxy configuration files for both 1.8.0 and 1.7.9. Ports on firewalls > are open and policies are enabled, as verified by a working v. 1.7.9. > > No log is being captured by HAProxy during access to these pages / > applications. Router log gives HAProxy is resetting the request. The Web > page on browser states "the connection was reset." > > Would you be kind enough to provide any vectors on what new configuration > parameter we should add / modify for 1.8.0 (different from 1.7.9)? Nothing in theory. Can you check if the process is still present (ps aux) ? If it is, are the ports properly listening (netstat -ltnp) ? > Below is haproxy -vv. Command to start HAProxy: > > CONFIG= > > ExecStart=/usr/local/sbin/haproxy -Ws -V -C $CONFIG -f $CONFIG -f > $ -f $ -D -p $PIDFILE Just out of curiosity, are you certain that the binary in this absolute path is the one you expect ? For example if you had installed it in another place still in your path, haproxy -vv would find the new one but the one above would be the old one and would fail on "-Ws". Do you have anything in systemd's logs related to the startup ? Regards, Willy
RE: Does HAProxy 1.8.0 need new param vs 1.7.9
Further, we have added the following new parameters in the configuration
before testing 1.8.0 [{I} -Baptiste Assman's article]. Still the pages /
applications are not accessible via 1.8.0. What obvious thing could we be
missing with 1.8.0 installation/configuration vs. 1.7.9? Your vectors to
help find a solution would be appreciated.
(i)Used systemd: ExecStart -Ws.
OR
global
master-worker
(ii) frontend webapps-frontend
bind *:443 name https ssl crt
"$PATH_TO_&_NAME_OF_SSL_CRT_PATH_FILE" alpn h2,http/1.1
The OS is CentOS 7.1 and has systemd and system-devel.
[I] https://www.haproxy.com/blog/whats-new-haproxy-1-8/
Thank you.
Sincerely,
Hemant K. Sabat
Coscend Communications Solutions
<http://www.coscend.com/> www.Coscend.com
--
Real-time, Interactive Video Collaboration, Tele-healthcare, Tele-education,
Telepresence Services, on the fly.
--
CONFIDENTIALITY NOTICE: See 'Confidentiality Notice Regarding E-mail
Messages from Coscend Communications Solutions' posted at:
http://www.Coscend.com/Anchor/Common/Terms_and_Conditions.html
From: Coscend@Coscend [mailto:haproxy.insig...@coscend.com]
Sent: Monday, November 27, 2017 12:11 PM
To: haproxy@formilux.org
Subject: Does HAProxy 1.8.0 need new param vs 1.7.9
Dear HAProxy community,
We have been successfully loading Stats page and other applications via
HAProxy 1.7.9. We successfully compiled, installed and ran 1.8.0 as a
systemd service. However, with 1.8.0, we are unable to access the same
stats page or any other application. We are using the same set of multiple
HAProxy configuration files for both 1.8.0 and 1.7.9. Ports on firewalls
are open and policies are enabled, as verified by a working v. 1.7.9.
No log is being captured by HAProxy during access to these pages /
applications. Router log gives HAProxy is resetting the request. The Web
page on browser states "the connection was reset."
Would you be kind enough to provide any vectors on what new configuration
parameter we should add / modify for 1.8.0 (different from 1.7.9)?
Below is haproxy -vv. Command to start HAProxy:
CONFIG=
ExecStart=/usr/local/sbin/haproxy -Ws -V -C $CONFIG -f $CONFIG -f
$ -f $ -D -p $PIDFILE
ExecReload=/usr/local/sbin/haproxy -C $CONFIG -f $CONFIG -f
$BACKENDS_DEFAULT -f $BACKENDS_SECURITY -f $BACKENDS_COSCENDCC -f
$BACKENDS_PRODUCTS -c -q
ExecReload=/bin/kill -USR2 $MAINPID
KillMode=mixed
Restart=always
Type=forking
WantedBy=multi-user.target
Thank you.
HA-Proxy version 1.8.0 2017/11/26
Copyright 2000-2017 Willy Tarreau mailto:wi...@haproxy.org> >
Build options :
TARGET = linux2628
CPU = native
CC = gcc
CFLAGS = -m64 -march=x86-64 -O2 -march=native -g -fno-strict-aliasing
-Wdeclaration-after-statement -fwrapv -Wno-unused-label
OPTIONS = USE_LIBCRYPT=1 USE_CRYPT_H=1 USE_GETADDRINFO=1 USE_ZLIB=1
USE_REGPARM=1 USE_OPENSSL=1 USE_LUA=1 USE_SYSTEMD=1 USE_PCRE=1
USE_PCRE_JIT=1 USE_TFO=1 USE_NS=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with OpenSSL version : OpenSSL 1.0.2k 26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k 26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.1
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"),
raw-deflate("deflate"), gzip("gzip")
Built with network namespace support.
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
Sincerely,
Hemant K. Sabat
Coscend Communications Solutions
<http://www.coscend.com/> www.Coscend.com
--
Real-time, Interactive Video Collaboration, Tele-healthcare, Tele-education,
Telepresence Services, on the fly.
--
CONFIDENTIALITY NOTICE: See 'Confidentiality Notice Regarding E-mail
Messages from Coscend Communications Solutions&#
Does HAProxy 1.8.0 need new param vs 1.7.9
Dear HAProxy community,
We have been successfully loading Stats page and other applications via
HAProxy 1.7.9. We successfully compiled, installed and ran 1.8.0 as a
systemd service. However, with 1.8.0, we are unable to access the same
stats page or any other application. We are using the same set of multiple
HAProxy configuration files for both 1.8.0 and 1.7.9. Ports on firewalls
are open and policies are enabled, as verified by a working v. 1.7.9.
No log is being captured by HAProxy during access to these pages /
applications. Router log gives HAProxy is resetting the request. The Web
page on browser states "the connection was reset."
Would you be kind enough to provide any vectors on what new configuration
parameter we should add / modify for 1.8.0 (different from 1.7.9)?
Below is haproxy -vv. Command to start HAProxy:
CONFIG=
ExecStart=/usr/local/sbin/haproxy -Ws -V -C $CONFIG -f $CONFIG -f
$ -f $ -D -p $PIDFILE
ExecReload=/usr/local/sbin/haproxy -C $CONFIG -f $CONFIG -f
$BACKENDS_DEFAULT -f $BACKENDS_SECURITY -f $BACKENDS_COSCENDCC -f
$BACKENDS_PRODUCTS -c -q
ExecReload=/bin/kill -USR2 $MAINPID
KillMode=mixed
Restart=always
Type=forking
WantedBy=multi-user.target
Thank you.
HA-Proxy version 1.8.0 2017/11/26
Copyright 2000-2017 Willy Tarreau
Build options :
TARGET = linux2628
CPU = native
CC = gcc
CFLAGS = -m64 -march=x86-64 -O2 -march=native -g -fno-strict-aliasing
-Wdeclaration-after-statement -fwrapv -Wno-unused-label
OPTIONS = USE_LIBCRYPT=1 USE_CRYPT_H=1 USE_GETADDRINFO=1 USE_ZLIB=1
USE_REGPARM=1 USE_OPENSSL=1 USE_LUA=1 USE_SYSTEMD=1 USE_PCRE=1
USE_PCRE_JIT=1 USE_TFO=1 USE_NS=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with OpenSSL version : OpenSSL 1.0.2k 26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k 26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.1
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"),
raw-deflate("deflate"), gzip("gzip")
Built with network namespace support.
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
Sincerely,
Hemant K. Sabat
Coscend Communications Solutions
http://www.Coscend.com/Anchor/Common/Terms_and_Conditions.html
---
This email has been checked for viruses by AVG.
http://www.avg.com
