Re: HAPROXY CAN NOT POINT IN TO PORT 5000 OF PATRONI
On 4/22/21 11:39 AM, Maciej Zdeb wrote: Hi, try removing those two lines from config: option httpchk http-check expect status 200 it is postgres (tcp backend), you should not expect http response on health check. As far as I can see, this is Patroni style HA setup, where Patroni services manage Postgres and detect the current master (i.e. writeable server). Hence the http checks (on different port) https://github.com/zalando/patroni/blob/master/haproxy.cfg It's probably a problem in general Patroni setup, but that's out of the scope of this list. Revert to the original (patroni) provided haproxy configuration and check if the backend servers are up. (i.e. open the "stats" frontend endpoint in your browser). Best regards, -- Adis Nezirovic Software Engineer HAProxy Technologies - Powering your uptime! 375 Totten Pond Road, Suite 302 | Waltham, MA 02451, US +1 (844) 222-4340 | https://www.haproxy.com
Re: HAPROXY CAN NOT POINT IN TO PORT 5000 OF PATRONI
Hi, try removing those two lines from config: option httpchk http-check expect status 200 it is postgres (tcp backend), you should not expect http response on health check. Kind regards, Maciej czw., 22 kwi 2021 o 02:38 thủy bùi napisał(a): > I have open all port by setting firewall rules, But still get the same > error > > Vào Th 4, 21 thg 4, 2021 lúc 23:01 thủy bùi đã > viết: > >> I have change my configuration to TCP mode as below and restart haproxy >> but still got the same error: >> global maxconn 100defaults log global mode tcp retries 2 timeout client >> 30m timeout connect 4s timeout server 30m timeout check 5slisten stats >> mode tcp bind *:8009 stats enable stats uri /listen postgres bind *:5000 >> option httpchk http-check expect status 200 default-server inter 3s fall >> 3 rise 2 on-marked-down shutdown-sessions server >> postgresql_10.128.0.10_5432 10.128.0.10:5432 maxconn 100 check port 8008 >> server postgresql_10.128.0.11_5432 10.128.0.11:5432 maxconn 100 check >> port 8008 >> >> Vào Th 4, 21 thg 4, 2021 vào lúc 22:42 Jarno Huuskonen < >> jarno.huusko...@uef.fi> đã viết: >> >>> Hei, >>> >>> On Wed, 2021-04-21 at 16:27 +0100, Andrew Smalley wrote: >>> > From the look of your configuration you are using HTTP Mode, for >>> > PostgreSQL, you will need a TCP VIP >>> > >>> > I noted this because of the HTTP check >>> > >>> > try using "mode tcp" >>> > >>> >>> defaults has mode tcp: >>> >>> defaults >>> log global >>> mode tcp >>> ... >>> >>> -Jarno >>> >>> > >>> > On Wed, 21 Apr 2021 at 16:25, Jarno Huuskonen >>> > wrote: >>> > > >>> > > Hi, >>> > > >>> > > On Wed, 2021-04-21 at 21:55 +0700, thủy bùi wrote: >>> > > > Dear HAproxy dev,I have install all the requirement of HAproxy >>> into the >>> > > > system alongside with patroni and etcd, but finding error while >>> call >>> > > > into >>> > > > port 5000. >>> > > > The information is provided as below. >>> > > > Please help me find out the issue. >>> > > > I have running HAproxy successfully >>> > > > >>> > > > But unable to connect to my database throught port 5000 >>> > > > >>> > > > >>> > > > psql: error: server closed the connection unexpectedly >>> > > > This probably means the server terminated abnormally >>> > > > before or while processing the request. >>> > > > What is your configuration? >>> > > > >>> > > ... >>> > > >>> > > Does haproxy run when you run it from cli >>> > > (haproxy -d -f /path/to/yourconfig.cfg) ? >>> > > >>> > > And do you have errors in your logs ? >>> > > >>> > > > Linux postgre02 3.10.0-1160.21.1.el7.x86_64 #1 SMP Tue Mar 16 >>> 18:28:22 >>> > > > UTC >>> > > > 2021 x86_64 x86_64 x86_64 GNU/Linux >>> > > >>> > > Looks like you're running on CentOS/RHEL 7 ? Do you have selinux >>> enabled >>> > > (getenforce) ? You'll probably need to allow haproxy to connect to >>> all >>> > > ports >>> > > (or allow required ports). (setsebool -P haproxy_connect_any=On might >>> > > help). >>> > > >>> > > (Your logs should show if connections are denied). >>> > > >>> > > -Jarno >>> > > >>> > > -- >>> > > Jarno Huuskonen >>> >>> >> >> -- >> *BUI THANH THUY* >> Tel: 0348672994 >> Email: buithuy.13...@gmail.com >> > -- > *BUI THANH THUY* > Tel: 0348672994 > Email: buithuy.13...@gmail.com >
Re: HAPROXY CAN NOT POINT IN TO PORT 5000 OF PATRONI
Hi, On Thu, 2021-04-22 at 11:39 +0200, Maciej Zdeb wrote: > Hi, > try removing those two lines from config: > option httpchk > http-check expect status 200 > > it is postgres (tcp backend), you should not expect http response on > health check. > I think httpchk/port 8008 is for patroni rest api: https://patroni.readthedocs.io/en/latest/rest_api.html (but should test w/out option httpchk / check port 8008 to rule out this). Also the listen postgres doesn't have explicit mode tcp (it's in defaults though), listen stats has mode tcp (but the stats probably should be mode http). If the system has SELinux enabled I'd check that SELinux has setsebool -P haproxy_connect_any=On because AFAIK the default policy doesn't allow all/random ports. (And logs probably tell what's wrong ...) -Jarno > > czw., 22 kwi 2021 o 02:38 thủy bùi napisał(a): > > I have open all port by setting firewall rules, But still get the same > > error > > > > Vào Th 4, 21 thg 4, 2021 lúc 23:01 thủy bùi đã > > viết: > > > I have change my configuration to TCP mode as below and restart haproxy > > > but still got the same error: > > > global > > > maxconn 100 > > > > > > defaults > > > log global > > > mode tcp > > > retries 2 > > > timeout client 30m > > > timeout connect 4s > > > timeout server 30m > > > timeout check 5s > > > > > > listen stats > > > mode tcp > > > bind *:8009 > > > stats enable > > > stats uri / > > > > > > listen postgres > > > bind *:5000 > > > option httpchk > > > http-check expect status 200 > > > default-server inter 3s fall 3 rise 2 on-marked-down shutdown- > > > sessions > > > server postgresql_10.128.0.10_5432 10.128.0.10:5432 maxconn 100 > > > check port 8008 > > > server postgresql_10.128.0.11_5432 10.128.0.11:5432 maxconn 100 > > > check port 8008 > > > > > > Vào Th 4, 21 thg 4, 2021 vào lúc 22:42 Jarno Huuskonen < > > > jarno.huusko...@uef.fi> đã viết: > > > > Hei, > > > > > > > > On Wed, 2021-04-21 at 16:27 +0100, Andrew Smalley wrote: > > > > > From the look of your configuration you are using HTTP Mode, for > > > > > PostgreSQL, you will need a TCP VIP > > > > > > > > > > I noted this because of the HTTP check > > > > > > > > > > try using "mode tcp" > > > > > > > > > > > > > defaults has mode tcp: > > > > > > > > defaults > > > > log global > > > > mode tcp > > > > ... > > > > > > > > -Jarno > > > > > > > > > > > > > > On Wed, 21 Apr 2021 at 16:25, Jarno Huuskonen > > > > > > > > > wrote: > > > > > > > > > > > > Hi, > > > > > > > > > > > > On Wed, 2021-04-21 at 21:55 +0700, thủy bùi wrote: > > > > > > > Dear HAproxy dev,I have install all the requirement of HAproxy > > > > into the > > > > > > > system alongside with patroni and etcd, but finding error while > > > > call > > > > > > > into > > > > > > > port 5000. > > > > > > > The information is provided as below. > > > > > > > Please help me find out the issue. > > > > > > > I have running HAproxy successfully > > > > > > > > > > > > > > But unable to connect to my database throught port 5000 > > > > > > > > > > > > > > > > > > > > > psql: error: server closed the connection unexpectedly > > > > > > > This probably means the server terminated abnormally > > > > > > > before or while processing the request. > > > > > > > What is your configuration? > > > > > > > > > > > > > ... > > > > > > > > > > > > Does haproxy run when you run it from cli > > > > > > (haproxy -d -f /path/to/yourconfig.cfg) ? > > > > > > > > > > > > And do you have errors in your logs ? > > > > > > > > > > > > > Linux postgre02 3.10.0-1160.21.1.el7.x86_64 #1 SMP Tue Mar 16 > > > > 18:28:22 > > > > > > > UTC > > > > > > > 2021 x86_64 x86_64 x86_64 GNU/Linux > > > > > > > > > > > > Looks like you're running on CentOS/RHEL 7 ? Do you have selinux > > > > enabled > > > > > > (getenforce) ? You'll probably need to allow haproxy to connect to > > > > all > > > > > > ports > > > > > > (or allow required ports). (setsebool -P haproxy_connect_any=On > > > > might > > > > > > help). > > > > > > > > > > > > (Your logs should show if connections are denied). > > > > > > > > > > > > -Jarno > > > > > > > > > > > > -- > > > > > > Jarno Huuskonen > > > > > > > > > > > > > -- > > > BUI THANH THUY > > > Tel: 0348672994 > > > Email: buithuy.13...@gmail.com
Re: Proposal about libslz integration into haproxy
On Wed, Apr 21, 2021 at 08:04:08AM +0200, Willy Tarreau wrote: > Does anyone have any opinion, objection or suggestion on this (especially > those in CC who participated to the first discussion or who could have > packaging concerns) ? Barring any comment I think I'm going to do this > tomorrow so that we have it in -dev17, leaving a bit of time for distro > packagers to test and adjust their build process if needed. Now merged, FWIW. Thanks for all the comments! Willy
Re: Illegal instruction - 2.2 on AMD/Sempron ?
stock 1.8.19 (which runs fine) doesn't also use cmpxchg16b ?
https://en.wikipedia.org/wiki/X86-64#Older_implementations
May be time for a new motherboard (or a Pi4?) ...
Dump of assembler code for function ha_random64:
0x5566bac0 <+0>: push %rbx
0x5566bac1 <+1>: sub$0x10,%rsp
0x5566bac5 <+5>: mov0x17d494(%rip),%rsi# 0x557e8f60
0x5566bacc <+12>: mov0x17d495(%rip),%rdx# 0x557e8f68
0x5566bad3 <+19>: mov%fs:0x28,%rax
0x5566badc <+28>: mov%rax,0x8(%rsp)
0x5566bae1 <+33>: xor%eax,%eax
0x5566bae3 <+35>: mov%rsi,%rcx
0x5566bae6 <+38>: mov%rsi,%rbx
0x5566bae9 <+41>: xor%rdx,%rcx
0x5566baec <+44>: rol$0x18,%rbx
0x5566baf0 <+48>: mov%rcx,%rax
0x5566baf3 <+51>: xor%rcx,%rbx
0x5566baf6 <+54>: ror$0x1b,%rcx
0x5566bafa <+58>: shl$0x10,%rax
0x5566bafe <+62>: xor%rax,%rbx
0x5566bb01 <+65>: mov%rsi,%rax
=> 0x5566bb04 <+68>: lock cmpxchg16b 0x17d453(%rip)#
0x557e8f60
0x5566bb0d <+77>: sete %cl
0x5566bb10 <+80>: test %cl,%cl
0x5566bb12 <+82>: je 0x5566bb40
0x5566bb14 <+84>: lea(%rsi,%rsi,4),%rax
0x5566bb18 <+88>: rol$0x7,%rax
0x5566bb1c <+92>: mov0x8(%rsp),%rdi
0x5566bb21 <+97>: xor%fs:0x28,%rdi
0x5566bb2a <+106>: lea(%rax,%rax,8),%rax
0x5566bb2e <+110>: jne0x5566bb45
0x5566bb30 <+112>: add$0x10,%rsp
On Thu, Apr 22, 2021 at 8:31 AM Willy Tarreau wrote:
>
> Hi Jim,
>
> On Wed, Apr 21, 2021 at 04:46:17AM -0600, Jim Freeman wrote:
> > Stock 1.8.19-1+deb10u3 on Debian10 runs fine, but when I install
> > 2.2.8-1~bpo10+1 from buster-backports, I get "Illegal instruction" ?
> > Is my CPU just too historic ?
>
> Possible but it makes me think that it could also be a matter of lib
> or toolchain that was built for a slightly different arch with certain
> extensions (e.g. sse etc).
>
> Since it seems to happen easily, you should try it again under gdb,
> then disassemble the code around the location:
>
> # gdb --args ./haproxy -f ~/haproxy.cfg
> > run
>
> Once it crashes, issue:
>
> > bt
>
> it will report the backtrace and current function where it died,
> then:
>
> > disassemble $rip
>
> and press Enter until you see a line with "=>" indicating the current
> location. Please post a copy of the surrounding lines here, we may
> possibly figure that we're using an instruction we ought not to use.
>
> > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> > pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm
> > 3dnowext 3dnow rep_good nopl cpuid pni lahf_lm 3dnowprefetch vmmcall
> > bugs : fxsave_leak sysret_ss_attrs null_seg swapgs_fence spectre_v1
> > spectre_v2
>
> I'm not seeing cmpxchg16b here ("cx16"), which could be a serious
> concern, as we've never been aware of any x86_64 CPU without it and
> have been enabling it by default on x86_64 (and it cannot be enabled
> nor disabled at run time as it allows to replace certain structures
> with other ones).
>
> Willy
Re: Illegal instruction - 2.2 on AMD/Sempron ?
Hi Jim,
On Wed, Apr 21, 2021 at 04:46:17AM -0600, Jim Freeman wrote:
> Stock 1.8.19-1+deb10u3 on Debian10 runs fine, but when I install
> 2.2.8-1~bpo10+1 from buster-backports, I get "Illegal instruction" ?
> Is my CPU just too historic ?
Possible but it makes me think that it could also be a matter of lib
or toolchain that was built for a slightly different arch with certain
extensions (e.g. sse etc).
Since it seems to happen easily, you should try it again under gdb,
then disassemble the code around the location:
# gdb --args ./haproxy -f ~/haproxy.cfg
> run
Once it crashes, issue:
> bt
it will report the backtrace and current function where it died,
then:
> disassemble $rip
and press Enter until you see a line with "=>" indicating the current
location. Please post a copy of the surrounding lines here, we may
possibly figure that we're using an instruction we ought not to use.
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm
> 3dnowext 3dnow rep_good nopl cpuid pni lahf_lm 3dnowprefetch vmmcall
> bugs : fxsave_leak sysret_ss_attrs null_seg swapgs_fence spectre_v1 spectre_v2
I'm not seeing cmpxchg16b here ("cx16"), which could be a serious
concern, as we've never been aware of any x86_64 CPU without it and
have been enabling it by default on x86_64 (and it cannot be enabled
nor disabled at run time as it allows to replace certain structures
with other ones).
Willy
Re: Illegal instruction - 2.2 on AMD/Sempron ?
On Thu, Apr 22, 2021 at 09:17:29AM -0600, Jim Freeman wrote: > stock 1.8.19 (which runs fine) doesn't also use cmpxchg16b ? > https://en.wikipedia.org/wiki/X86-64#Older_implementations By default threads were not enabled in 1.8, so there were no atomic ops. And indeed it died on it :-/ > May be time for a new motherboard (or a Pi4?) ... Or you can rebuild to run in single-thread mode. I really didn't know there were any such x86_64 machines without cmpxchg16b, and the way we use implies that it's not even remotely an option to optionally enable it. Given that it's the first report in 3.5 years and that the article above mentions that even Windows 8.1 requires the instruction, I'd rather stay like this. If we figure an easy and reliable method to detect it (probably via CPUID if we want to be cross-os), we could refuse to start and suggest to rebuild with thread support disabled. But yeah, depending on your use case, this could be a good excuse to invest in a different machine (and Pi4 will work fine). Other low-end fanless x86 devices can be pretty good as well (e.g. UP Boards with Atom x5-8350 or Celeron J4105 can do wonders even in fanless mode). Willy
Re: [2.2.11] 100% CPU again
śr., 21 kwi 2021 o 13:53 Christopher Faulet napisał(a): > The fix was merge in upstream : > > * BUG/MAJOR: mux-h2: Properly detect too large frames when decoding headers >(http://git.haproxy.org/?p=haproxy.git;a=commit;h=07f88d75) > Thanks! Building 2.2.13 and 2.3.9 with patches. I'll return with feedback if anything pops out again. Kind regards,
