Re: Illegal instruction - 2.2 on AMD/Sempron ?
On Thu, Apr 22, 2021 at 09:17:29AM -0600, Jim Freeman wrote: > stock 1.8.19 (which runs fine) doesn't also use cmpxchg16b ? > https://en.wikipedia.org/wiki/X86-64#Older_implementations By default threads were not enabled in 1.8, so there were no atomic ops. And indeed it died on it :-/ > May be time for a new motherboard (or a Pi4?) ... Or you can rebuild to run in single-thread mode. I really didn't know there were any such x86_64 machines without cmpxchg16b, and the way we use implies that it's not even remotely an option to optionally enable it. Given that it's the first report in 3.5 years and that the article above mentions that even Windows 8.1 requires the instruction, I'd rather stay like this. If we figure an easy and reliable method to detect it (probably via CPUID if we want to be cross-os), we could refuse to start and suggest to rebuild with thread support disabled. But yeah, depending on your use case, this could be a good excuse to invest in a different machine (and Pi4 will work fine). Other low-end fanless x86 devices can be pretty good as well (e.g. UP Boards with Atom x5-8350 or Celeron J4105 can do wonders even in fanless mode). Willy
Re: Illegal instruction - 2.2 on AMD/Sempron ?
stock 1.8.19 (which runs fine) doesn't also use cmpxchg16b ?
https://en.wikipedia.org/wiki/X86-64#Older_implementations
May be time for a new motherboard (or a Pi4?) ...
Dump of assembler code for function ha_random64:
0x5566bac0 <+0>: push %rbx
0x5566bac1 <+1>: sub$0x10,%rsp
0x5566bac5 <+5>: mov0x17d494(%rip),%rsi# 0x557e8f60
0x5566bacc <+12>: mov0x17d495(%rip),%rdx# 0x557e8f68
0x5566bad3 <+19>: mov%fs:0x28,%rax
0x5566badc <+28>: mov%rax,0x8(%rsp)
0x5566bae1 <+33>: xor%eax,%eax
0x5566bae3 <+35>: mov%rsi,%rcx
0x5566bae6 <+38>: mov%rsi,%rbx
0x5566bae9 <+41>: xor%rdx,%rcx
0x5566baec <+44>: rol$0x18,%rbx
0x5566baf0 <+48>: mov%rcx,%rax
0x5566baf3 <+51>: xor%rcx,%rbx
0x5566baf6 <+54>: ror$0x1b,%rcx
0x5566bafa <+58>: shl$0x10,%rax
0x5566bafe <+62>: xor%rax,%rbx
0x5566bb01 <+65>: mov%rsi,%rax
=> 0x5566bb04 <+68>: lock cmpxchg16b 0x17d453(%rip)#
0x557e8f60
0x5566bb0d <+77>: sete %cl
0x5566bb10 <+80>: test %cl,%cl
0x5566bb12 <+82>: je 0x5566bb40
0x5566bb14 <+84>: lea(%rsi,%rsi,4),%rax
0x5566bb18 <+88>: rol$0x7,%rax
0x5566bb1c <+92>: mov0x8(%rsp),%rdi
0x5566bb21 <+97>: xor%fs:0x28,%rdi
0x5566bb2a <+106>: lea(%rax,%rax,8),%rax
0x5566bb2e <+110>: jne0x5566bb45
0x5566bb30 <+112>: add$0x10,%rsp
On Thu, Apr 22, 2021 at 8:31 AM Willy Tarreau wrote:
>
> Hi Jim,
>
> On Wed, Apr 21, 2021 at 04:46:17AM -0600, Jim Freeman wrote:
> > Stock 1.8.19-1+deb10u3 on Debian10 runs fine, but when I install
> > 2.2.8-1~bpo10+1 from buster-backports, I get "Illegal instruction" ?
> > Is my CPU just too historic ?
>
> Possible but it makes me think that it could also be a matter of lib
> or toolchain that was built for a slightly different arch with certain
> extensions (e.g. sse etc).
>
> Since it seems to happen easily, you should try it again under gdb,
> then disassemble the code around the location:
>
> # gdb --args ./haproxy -f ~/haproxy.cfg
> > run
>
> Once it crashes, issue:
>
> > bt
>
> it will report the backtrace and current function where it died,
> then:
>
> > disassemble $rip
>
> and press Enter until you see a line with "=>" indicating the current
> location. Please post a copy of the surrounding lines here, we may
> possibly figure that we're using an instruction we ought not to use.
>
> > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> > pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm
> > 3dnowext 3dnow rep_good nopl cpuid pni lahf_lm 3dnowprefetch vmmcall
> > bugs : fxsave_leak sysret_ss_attrs null_seg swapgs_fence spectre_v1
> > spectre_v2
>
> I'm not seeing cmpxchg16b here ("cx16"), which could be a serious
> concern, as we've never been aware of any x86_64 CPU without it and
> have been enabling it by default on x86_64 (and it cannot be enabled
> nor disabled at run time as it allows to replace certain structures
> with other ones).
>
> Willy
Re: Illegal instruction - 2.2 on AMD/Sempron ?
Hi Jim,
On Wed, Apr 21, 2021 at 04:46:17AM -0600, Jim Freeman wrote:
> Stock 1.8.19-1+deb10u3 on Debian10 runs fine, but when I install
> 2.2.8-1~bpo10+1 from buster-backports, I get "Illegal instruction" ?
> Is my CPU just too historic ?
Possible but it makes me think that it could also be a matter of lib
or toolchain that was built for a slightly different arch with certain
extensions (e.g. sse etc).
Since it seems to happen easily, you should try it again under gdb,
then disassemble the code around the location:
# gdb --args ./haproxy -f ~/haproxy.cfg
> run
Once it crashes, issue:
> bt
it will report the backtrace and current function where it died,
then:
> disassemble $rip
and press Enter until you see a line with "=>" indicating the current
location. Please post a copy of the surrounding lines here, we may
possibly figure that we're using an instruction we ought not to use.
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm
> 3dnowext 3dnow rep_good nopl cpuid pni lahf_lm 3dnowprefetch vmmcall
> bugs : fxsave_leak sysret_ss_attrs null_seg swapgs_fence spectre_v1 spectre_v2
I'm not seeing cmpxchg16b here ("cx16"), which could be a serious
concern, as we've never been aware of any x86_64 CPU without it and
have been enabling it by default on x86_64 (and it cannot be enabled
nor disabled at run time as it allows to replace certain structures
with other ones).
Willy
