Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Mike Looijmans]

Ancient thread, but might be of interest to others.

A while ago the power supply died. After replacing the PSU, the error 
below hasn't occurred any more.


I'm not 100% certain, but it's very likely the issue was caused by that 
PSU on the brink of failing completely. Not the first thing to come to 
mind...


Apparently gatesgarth is more power demanding than hardknott :)

Kind regards,
Mike


Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topic.nl

Please consider the environment before printing this e-mail
On 27-01-2021 16:10, Mike Looijmans via lists.openembedded.org wrote:
When doing large builds, the GCC compiler tends to crash on random 
spots in the code. There are a few common denominators though.


It only happens when compiling for aarch64 (cortex-A53), not for 
32-bit arm (cortex-A9)


It's random and usually happens on "big" sets like kernel, openssl, 
boost, u-boot etc.


It always reports "during GIMPLE pass: ealias" in the error, for example:

| during GIMPLE pass: ealias
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c: In function 'do_x509_check':
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c:1239:1: internal compiler 
error: Illegal instruction

| 1239 | }

Compiling the same thing again usually goes fine.

I've never experienced this with the zeus and older branches of OE.


I've already tried upgrading to the latest gatesgarth status, and 
cleaning out everything and start from scratch. I've also run "mprime" 
test on my machine (over one hour) just to be confident that the 
system itself is really okay.



Ideas to diagnose, fix or reliably reprodruce are more than welcome.






--
Mike Looijmans


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153408): 
https://lists.openembedded.org/g/openembedded-core/message/153408
Mute This Topic: https://lists.openembedded.org/mt/83863468/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Khem Raj]

if you are using poky then its on by default you might have to disable 
it for this test


conf/distro/poky.conf:require conf/distro/include/yocto-uninative.inc
conf/distro/poky.conf:INHERIT += "uninative"


On 3/14/21 8:00 AM, Mike Looijmans wrote:


Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 12-03-2021 08:37, Mike Looijmans via lists.openembedded.org wrote:

On 12-03-2021 08:33, Khem Raj wrote:
On Thu, Mar 11, 2021 at 11:22 PM Mike Looijmans 
 wrote:


On 05-03-2021 00:32, Randy MacLeod wrote:

On 2021-02-05 2:49 a.m., Mike Looijmans wrote:

I've been doing quite some tests and things last week, my conclusion
so far is that the latest gcc seems to trigger a bug particular to
this CPU. Maybe even this particular batch.

Hours of testing found no issues with RAM.

I've also done huge amounts of builds with zeus, no problems.

Then googling for "illegal instruction gcc ryzen 1700" reveals quite
a few threads on something that sounds suspiciously similar. Though
it makes me wonder why I never encountered it before, since these are
all many years old. I did try a bunch of the suggestions done there,
but so far no luck.



Any update on this bug Mike?
Have you opened a YP bug? If not please do.
I'd like to see if the problem still happens on master.

Interesting.

I just built the 64-kernel using OE master and no crashes, everything
went fine. With the gatesgarth branch, it'd crash at least two times
during kernel compilation.

So something has been fixed since then?

can you try by disabling uninative ?


Probably, if only I knew what you mean by that?

What do I have to configure where?

As far as I can determine I think i'm not using unitative since I didn't 
explicitly enable it anywhere...



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#149432): 
https://lists.openembedded.org/g/openembedded-core/message/149432
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Mike Looijmans]

Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 12-03-2021 08:37, Mike Looijmans via lists.openembedded.org wrote:

On 12-03-2021 08:33, Khem Raj wrote:
On Thu, Mar 11, 2021 at 11:22 PM Mike Looijmans 
 wrote:


On 05-03-2021 00:32, Randy MacLeod wrote:

On 2021-02-05 2:49 a.m., Mike Looijmans wrote:

I've been doing quite some tests and things last week, my conclusion
so far is that the latest gcc seems to trigger a bug particular to
this CPU. Maybe even this particular batch.

Hours of testing found no issues with RAM.

I've also done huge amounts of builds with zeus, no problems.

Then googling for "illegal instruction gcc ryzen 1700" reveals quite
a few threads on something that sounds suspiciously similar. Though
it makes me wonder why I never encountered it before, since these are
all many years old. I did try a bunch of the suggestions done there,
but so far no luck.



Any update on this bug Mike?
Have you opened a YP bug? If not please do.
I'd like to see if the problem still happens on master.

Interesting.

I just built the 64-kernel using OE master and no crashes, everything
went fine. With the gatesgarth branch, it'd crash at least two times
during kernel compilation.

So something has been fixed since then?

can you try by disabling uninative ?


Probably, if only I knew what you mean by that?

What do I have to configure where?

As far as I can determine I think i'm not using unitative since I didn't 
explicitly enable it anywhere...


--
Mike Looijmans


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#149425): 
https://lists.openembedded.org/g/openembedded-core/message/149425
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Mike Looijmans]

Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 12-03-2021 08:33, Khem Raj wrote:

On Thu, Mar 11, 2021 at 11:22 PM Mike Looijmans  wrote:


Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 05-03-2021 00:32, Randy MacLeod wrote:

On 2021-02-05 2:49 a.m., Mike Looijmans wrote:

I've been doing quite some tests and things last week, my conclusion
so far is that the latest gcc seems to trigger a bug particular to
this CPU. Maybe even this particular batch.

Hours of testing found no issues with RAM.

I've also done huge amounts of builds with zeus, no problems.

Then googling for "illegal instruction gcc ryzen 1700" reveals quite
a few threads on something that sounds suspiciously similar. Though
it makes me wonder why I never encountered it before, since these are
all many years old. I did try a bunch of the suggestions done there,
but so far no luck.



Any update on this bug Mike?
Have you opened a YP bug? If not please do.
I'd like to see if the problem still happens on master.

Interesting.

I just built the 64-kernel using OE master and no crashes, everything
went fine. With the gatesgarth branch, it'd crash at least two times
during kernel compilation.

So something has been fixed since then?

can you try by disabling uninative ?


Probably, if only I knew what you mean by that?

What do I have to configure where?



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#149328): 
https://lists.openembedded.org/g/openembedded-core/message/149328
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Khem Raj]

On Thu, Mar 11, 2021 at 11:22 PM Mike Looijmans  wrote:
>
>
> Met vriendelijke groet / kind regards,
>
> Mike Looijmans
> System Expert
>
>
> TOPIC Embedded Products B.V.
> Materiaalweg 4, 5681 RJ Best
> The Netherlands
>
> T: +31 (0) 499 33 69 69
> E: mike.looijm...@topicproducts.com
> W: www.topicproducts.com
>
> Please consider the environment before printing this e-mail
> On 05-03-2021 00:32, Randy MacLeod wrote:
> > On 2021-02-05 2:49 a.m., Mike Looijmans wrote:
> >> I've been doing quite some tests and things last week, my conclusion
> >> so far is that the latest gcc seems to trigger a bug particular to
> >> this CPU. Maybe even this particular batch.
> >>
> >> Hours of testing found no issues with RAM.
> >>
> >> I've also done huge amounts of builds with zeus, no problems.
> >>
> >> Then googling for "illegal instruction gcc ryzen 1700" reveals quite
> >> a few threads on something that sounds suspiciously similar. Though
> >> it makes me wonder why I never encountered it before, since these are
> >> all many years old. I did try a bunch of the suggestions done there,
> >> but so far no luck.
> >>
> >>
> >
> > Any update on this bug Mike?
> > Have you opened a YP bug? If not please do.
> > I'd like to see if the problem still happens on master.
>
> Interesting.
>
> I just built the 64-kernel using OE master and no crashes, everything
> went fine. With the gatesgarth branch, it'd crash at least two times
> during kernel compilation.
>
> So something has been fixed since then?

can you try by disabling uninative ?

>
>
> --
> Mike Looijmans
>
>
> 
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#149327): 
https://lists.openembedded.org/g/openembedded-core/message/149327
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Mike Looijmans]

Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 05-03-2021 00:32, Randy MacLeod wrote:

On 2021-02-05 2:49 a.m., Mike Looijmans wrote:
I've been doing quite some tests and things last week, my conclusion 
so far is that the latest gcc seems to trigger a bug particular to 
this CPU. Maybe even this particular batch.


Hours of testing found no issues with RAM.

I've also done huge amounts of builds with zeus, no problems.

Then googling for "illegal instruction gcc ryzen 1700" reveals quite 
a few threads on something that sounds suspiciously similar. Though 
it makes me wonder why I never encountered it before, since these are 
all many years old. I did try a bunch of the suggestions done there, 
but so far no luck.





Any update on this bug Mike?
Have you opened a YP bug? If not please do.
I'd like to see if the problem still happens on master. 


Interesting.

I just built the 64-kernel using OE master and no crashes, everything 
went fine. With the gatesgarth branch, it'd crash at least two times 
during kernel compilation.


So something has been fixed since then?


--
Mike Looijmans


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#149325): 
https://lists.openembedded.org/g/openembedded-core/message/149325
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Max Krummenacher]

Hi Mike

Am Donnerstag, den 11.03.2021, 10:43 +0100 schrieb Mike Looijmans:
> Met vriendelijke groet / kind regards,
> 
> Mike Looijmans
> System Expert
> 
> 
> TOPIC Embedded Products B.V.
> Materiaalweg 4, 5681 RJ Best
> The Netherlands
> 
> T: +31 (0) 499 33 69 69
> E: mike.looijm...@topicproducts.com
> W: www.topicproducts.com
> 
> Please consider the environment before printing this e-mail
> On 05-03-2021 00:32, Randy MacLeod wrote:
> > On 2021-02-05 2:49 a.m., Mike Looijmans wrote:
> > > I've been doing quite some tests and things last week, my conclusion so 
> > > far 
> > > is that the latest gcc seems to trigger a bug particular to this CPU. 
> > > Maybe 
> > > even this particular batch.
> > > 
> > > Hours of testing found no issues with RAM.
> > > 
> > > I've also done huge amounts of builds with zeus, no problems.
> > > 
> > > Then googling for "illegal instruction gcc ryzen 1700" reveals quite a 
> > > few 
> > > threads on something that sounds suspiciously similar. Though it makes me 
> > > wonder why I never encountered it before, since these are all many years 
> > > old. I did try a bunch of the suggestions done there, but so far no luck.
> > > 
> > > 
> > 
> > Any update on this bug Mike?
> > Have you opened a YP bug? If not please do.
> > I'd like to see if the problem still happens on master.
> 
> So far I've discovered that this only happens when compiling for an aarch64. 
> When targetting 32-bit ARM the compiler crash does not occur.
> 
> I've contacted AMD for an RMA for my CPU, but it's out of warranty now. I 
> plan 
> to borrow a compatible CPU to see if that makes the problem go away. If it 
> does I'll just buy a new CPU...
> 
> As an example, if I take the current gatesgarth branch and meta-raspberrypi, 
> then building for "raspberrypi4" runs without a hitch. If I chose 
> "raspberrypi4-64" as MACHINE, the build will crash multiple times, typically 
> on large components like the linux kernel, qt and boost. It happens on both 
> plain C (kernel, u-boot) and on C++ code (boost, Qt). The same applies to 
> e.g. 
> the Xilinx Zynq, building for the 7-series (cortex A9) is fine but building 
> for the MPSoC (cortex A53) will crash.
> 
> A silly workaround is to run "while ! bitbake ... ; do echo again; done" 
> which 
> will retry until it succeeds.
> 
> I'd be very interested if there's anyone with Ryzen setup who can reproduce 
> this, or fail to, which is also interesting to know.
> 
> I also tried building in tmpfs and on a different SSD to rule out the NVME as 
> a cause.
> 
> I'll try the master branch too, and report back on that (tomorrow probably).
> 
> 
> > ../Randy
> > 
> > 
> > > Met vriendelijke groet / kind regards,
> > > 
> > > Mike Looijmans
> > > System Expert
> > > 
> > > 
> > > TOPIC Embedded Products B.V.
> > > Materiaalweg 4, 5681 RJ Best
> > > The Netherlands
> > > 
> > > T: +31 (0) 499 33 69 69
> > > E: mike.looijm...@topicproducts.com
> > > W: www.topicproducts.com
> > > 
> > > Please consider the environment before printing this e-mail
> > > On 27-01-2021 16:10, Mike Looijmans via lists.openembedded.org wrote:
> > > > When doing large builds, the GCC compiler tends to crash on random 
> > > > spots in 
> > > > the code. There are a few common denominators though.
> > > > 
> > > > It only happens when compiling for aarch64 (cortex-A53), not for 32-bit 
> > > > arm 
> > > > (cortex-A9)
> > > > 
> > > > It's random and usually happens on "big" sets like kernel, openssl, 
> > > > boost, 
> > > > u-boot etc.
> > > > 
> > > > It always reports "during GIMPLE pass: ealias" in the error, for 
> > > > example:
> > > > 
> > > > > during GIMPLE pass: ealias
> > > > > ../openssl-1.1.1i/crypto/x509v3/v3_utl.c: In function 'do_x509_check':
> > > > > ../openssl-1.1.1i/crypto/x509v3/v3_utl.c:1239:1: internal compiler 
> > > > > error: 
> > > > Illegal instruction
> > > > > 1239 | }
> > > > 
> > > > Compiling the same thing again usually goes fine.
> > > > 
> > > > I've never experienced this with the zeus and older branches of OE.
> > > > 
> > > > 
> > > > I've already tried upgrading to the latest gatesgarth status, and 
> > > > cleaning 
> > > > out everything and start from scratch. I've also run "mprime" test on 
> > > > my 
> > > > machine (over one hour) just to be confident that the system itself is 
> > > > really okay.
> > > > 
> > > > 
> > > > Ideas to diagnose, fix or reliably reprodruce are more than welcome. 
> > > 
> > > 

For me a build of core-image-weston on a AMD Ryzen 7 3700X succeeded. The build 
run trough without
any hick-ups. openssl-1.1.1i is built as part of that image.
I used HEAD of today's gatesgarth.
So the issue seems to not affect all Ryzen CPUs.

Max


MACHINE=raspberrypi4-64 bitbake core-image-weston
WARNING: Host distribution "opensuseleap-15.2" has not been validated with this 
version of the build
system; you may possibly experience unexpected failures. It is recommended that 
you use a tested
distribution.
Loading cache: 100%
|  

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Mike Looijmans]

Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 05-03-2021 00:32, Randy MacLeod wrote:

On 2021-02-05 2:49 a.m., Mike Looijmans wrote:
I've been doing quite some tests and things last week, my conclusion so far 
is that the latest gcc seems to trigger a bug particular to this CPU. Maybe 
even this particular batch.


Hours of testing found no issues with RAM.

I've also done huge amounts of builds with zeus, no problems.

Then googling for "illegal instruction gcc ryzen 1700" reveals quite a few 
threads on something that sounds suspiciously similar. Though it makes me 
wonder why I never encountered it before, since these are all many years 
old. I did try a bunch of the suggestions done there, but so far no luck.





Any update on this bug Mike?
Have you opened a YP bug? If not please do.
I'd like to see if the problem still happens on master.


So far I've discovered that this only happens when compiling for an aarch64. 
When targetting 32-bit ARM the compiler crash does not occur.


I've contacted AMD for an RMA for my CPU, but it's out of warranty now. I plan 
to borrow a compatible CPU to see if that makes the problem go away. If it 
does I'll just buy a new CPU...


As an example, if I take the current gatesgarth branch and meta-raspberrypi, 
then building for "raspberrypi4" runs without a hitch. If I chose 
"raspberrypi4-64" as MACHINE, the build will crash multiple times, typically 
on large components like the linux kernel, qt and boost. It happens on both 
plain C (kernel, u-boot) and on C++ code (boost, Qt). The same applies to e.g. 
the Xilinx Zynq, building for the 7-series (cortex A9) is fine but building 
for the MPSoC (cortex A53) will crash.


A silly workaround is to run "while ! bitbake ... ; do echo again; done" which 
will retry until it succeeds.


I'd be very interested if there's anyone with Ryzen setup who can reproduce 
this, or fail to, which is also interesting to know.


I also tried building in tmpfs and on a different SSD to rule out the NVME as 
a cause.


I'll try the master branch too, and report back on that (tomorrow probably).




../Randy




Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 27-01-2021 16:10, Mike Looijmans via lists.openembedded.org wrote:
When doing large builds, the GCC compiler tends to crash on random spots in 
the code. There are a few common denominators though.


It only happens when compiling for aarch64 (cortex-A53), not for 32-bit arm 
(cortex-A9)


It's random and usually happens on "big" sets like kernel, openssl, boost, 
u-boot etc.


It always reports "during GIMPLE pass: ealias" in the error, for example:

| during GIMPLE pass: ealias
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c: In function 'do_x509_check':
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c:1239:1: internal compiler error: 
Illegal instruction

| 1239 | }

Compiling the same thing again usually goes fine.

I've never experienced this with the zeus and older branches of OE.


I've already tried upgrading to the latest gatesgarth status, and cleaning 
out everything and start from scratch. I've also run "mprime" test on my 
machine (over one hour) just to be confident that the system itself is 
really okay.



Ideas to diagnose, fix or reliably reprodruce are more than welcome. 












-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#149277): 
https://lists.openembedded.org/g/openembedded-core/message/149277
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Randy MacLeod]

On 2021-02-05 2:49 a.m., Mike Looijmans wrote:
I've been doing quite some tests and things last week, my conclusion so 
far is that the latest gcc seems to trigger a bug particular to this 
CPU. Maybe even this particular batch.


Hours of testing found no issues with RAM.

I've also done huge amounts of builds with zeus, no problems.

Then googling for "illegal instruction gcc ryzen 1700" reveals quite a 
few threads on something that sounds suspiciously similar. Though it 
makes me wonder why I never encountered it before, since these are all 
many years old. I did try a bunch of the suggestions done there, but so 
far no luck.





Any update on this bug Mike?
Have you opened a YP bug? If not please do.
I'd like to see if the problem still happens on master.

../Randy




Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 27-01-2021 16:10, Mike Looijmans via lists.openembedded.org wrote:
When doing large builds, the GCC compiler tends to crash on random 
spots in the code. There are a few common denominators though.


It only happens when compiling for aarch64 (cortex-A53), not for 
32-bit arm (cortex-A9)


It's random and usually happens on "big" sets like kernel, openssl, 
boost, u-boot etc.


It always reports "during GIMPLE pass: ealias" in the error, for example:

| during GIMPLE pass: ealias
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c: In function 'do_x509_check':
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c:1239:1: internal compiler 
error: Illegal instruction

| 1239 | }

Compiling the same thing again usually goes fine.

I've never experienced this with the zeus and older branches of OE.


I've already tried upgrading to the latest gatesgarth status, and 
cleaning out everything and start from scratch. I've also run "mprime" 
test on my machine (over one hour) just to be confident that the 
system itself is really okay.



Ideas to diagnose, fix or reliably reprodruce are more than welcome. 









--
# Randy MacLeod
# Wind River Linux

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#148948): 
https://lists.openembedded.org/g/openembedded-core/message/148948
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Mike Looijmans]

I've been doing quite some tests and things last week, my conclusion so 
far is that the latest gcc seems to trigger a bug particular to this 
CPU. Maybe even this particular batch.


Hours of testing found no issues with RAM.

I've also done huge amounts of builds with zeus, no problems.

Then googling for "illegal instruction gcc ryzen 1700" reveals quite a 
few threads on something that sounds suspiciously similar. Though it 
makes me wonder why I never encountered it before, since these are all 
many years old. I did try a bunch of the suggestions done there, but so 
far no luck.




Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 27-01-2021 16:10, Mike Looijmans via lists.openembedded.org wrote:
When doing large builds, the GCC compiler tends to crash on random 
spots in the code. There are a few common denominators though.


It only happens when compiling for aarch64 (cortex-A53), not for 
32-bit arm (cortex-A9)


It's random and usually happens on "big" sets like kernel, openssl, 
boost, u-boot etc.


It always reports "during GIMPLE pass: ealias" in the error, for example:

| during GIMPLE pass: ealias
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c: In function 'do_x509_check':
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c:1239:1: internal compiler 
error: Illegal instruction

| 1239 | }

Compiling the same thing again usually goes fine.

I've never experienced this with the zeus and older branches of OE.


I've already tried upgrading to the latest gatesgarth status, and 
cleaning out everything and start from scratch. I've also run "mprime" 
test on my machine (over one hour) just to be confident that the 
system itself is really okay.



Ideas to diagnose, fix or reliably reprodruce are more than welcome. 


--
Mike Looijmans


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147680): 
https://lists.openembedded.org/g/openembedded-core/message/147680
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Leon Woestenberg]

>
> > Let me know if I must replicate a specific set of commits.
>
> Don't know what you mean by that, can you explain?
>
I mean I could try to reproduce your build locally, but I would want
the specific commits of the layers you are testing against, and the
local.conf settings that trigger your faults.
(I.e. be as close to the failing setup as possible).

Leon.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147355): 
https://lists.openembedded.org/g/openembedded-core/message/147355
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Mike Looijmans]

Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 27-01-2021 16:43, Leon Woestenberg wrote:


Hello Mike,

At first sight, this does sound like memory corruption in one specific 
memory area (DIMM?) to me.


On itself, yes, that was my first thought too - hence running memory and 
system tests.
But it only happens with the gatesgarth branch. I've put loads of 
pressure on this machine's memory before, never got errors like that. I 
sometimes have multiple OE branches building at the same time too.



Check dmesg for tripping temperatures etc.
I would reduce both the amount of bitbake tasks and Makefile 
parallelism to 1 on a fresh run to reduce memory pressure.


Reducing somewhat, e.g. 8 each, did not make a difference. Didn't want 
to go to "1", it'll take days to rebuild then...


Not seen anything similar yet (MACHINE=zcu102, build host i7-10700K w/ 
128MB memory.)



Guess that's GB not MB :)

Mine is a Ryzen 7 1700 w/ 32GB.

CPU Temp doesn't rise above 70 degrees while building (during mprime 
testing it got up to 90, still within spec)



I would assume aarch64 is widely run by the community.


I would, too


Let me know if I must replicate a specific set of commits.


Don't know what you mean by that, can you explain?





Regards,

Leon.

On Wed, Jan 27, 2021 at 4:10 PM Mike Looijmans 
mailto:mike.looijm...@topic.nl>> wrote:


When doing large builds, the GCC compiler tends to crash on random
spots
in the code. There are a few common denominators though.

It only happens when compiling for aarch64 (cortex-A53), not for
32-bit
arm (cortex-A9)

It's random and usually happens on "big" sets like kernel, openssl,
boost, u-boot etc.

It always reports "during GIMPLE pass: ealias" in the error, for
example:

| during GIMPLE pass: ealias
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c: In function
'do_x509_check':
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c:1239:1: internal compiler
error: Illegal instruction
| 1239 | }

Compiling the same thing again usually goes fine.

I've never experienced this with the zeus and older branches of OE.


I've already tried upgrading to the latest gatesgarth status, and
cleaning out everything and start from scratch. I've also run
"mprime"
test on my machine (over one hour) just to be confident that the
system
itself is really okay.


Ideas to diagnose, fix or reliably reprodruce are more than welcome.

-- 
Mike Looijmans



Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com

W: www.topicproducts.com 

Please consider the environment before printing this e-mail





--
Mike Looijmans


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147353): 
https://lists.openembedded.org/g/openembedded-core/message/147353
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] GCC crashes on aarch64 since gatesgarth

[Leon Woestenberg]

Hello Mike,

At first sight, this does sound like memory corruption in one specific
memory area (DIMM?) to me.
Check dmesg for tripping temperatures etc.
I would reduce both the amount of bitbake tasks and Makefile parallelism to
1 on a fresh run to reduce memory pressure.

Not seen anything similar yet (MACHINE=zcu102, build host i7-10700K w/
128MB memory.)
I would assume aarch64 is widely run by the community.
Let me know if I must replicate a specific set of commits.

Regards,

Leon.

On Wed, Jan 27, 2021 at 4:10 PM Mike Looijmans 
wrote:

> When doing large builds, the GCC compiler tends to crash on random spots
> in the code. There are a few common denominators though.
>
> It only happens when compiling for aarch64 (cortex-A53), not for 32-bit
> arm (cortex-A9)
>
> It's random and usually happens on "big" sets like kernel, openssl,
> boost, u-boot etc.
>
> It always reports "during GIMPLE pass: ealias" in the error, for example:
>
> | during GIMPLE pass: ealias
> | ../openssl-1.1.1i/crypto/x509v3/v3_utl.c: In function 'do_x509_check':
> | ../openssl-1.1.1i/crypto/x509v3/v3_utl.c:1239:1: internal compiler
> error: Illegal instruction
> | 1239 | }
>
> Compiling the same thing again usually goes fine.
>
> I've never experienced this with the zeus and older branches of OE.
>
>
> I've already tried upgrading to the latest gatesgarth status, and
> cleaning out everything and start from scratch. I've also run "mprime"
> test on my machine (over one hour) just to be confident that the system
> itself is really okay.
>
>
> Ideas to diagnose, fix or reliably reprodruce are more than welcome.
>
> --
> Mike Looijmans
>
>
> Met vriendelijke groet / kind regards,
>
> Mike Looijmans
> System Expert
>
>
> TOPIC Embedded Products B.V.
> Materiaalweg 4, 5681 RJ Best
> The Netherlands
>
> T: +31 (0) 499 33 69 69
> E: mike.looijm...@topicproducts.com
> W: www.topicproducts.com
>
> Please consider the environment before printing this e-mail
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147352): 
https://lists.openembedded.org/g/openembedded-core/message/147352
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] GCC crashes on aarch64 since gatesgarth

[Mike Looijmans]

When doing large builds, the GCC compiler tends to crash on random spots 
in the code. There are a few common denominators though.


It only happens when compiling for aarch64 (cortex-A53), not for 32-bit 
arm (cortex-A9)


It's random and usually happens on "big" sets like kernel, openssl, 
boost, u-boot etc.


It always reports "during GIMPLE pass: ealias" in the error, for example:

| during GIMPLE pass: ealias
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c: In function 'do_x509_check':
| ../openssl-1.1.1i/crypto/x509v3/v3_utl.c:1239:1: internal compiler 
error: Illegal instruction

| 1239 | }

Compiling the same thing again usually goes fine.

I've never experienced this with the zeus and older branches of OE.


I've already tried upgrading to the latest gatesgarth status, and 
cleaning out everything and start from scratch. I've also run "mprime" 
test on my machine (over one hour) just to be confident that the system 
itself is really okay.



Ideas to diagnose, fix or reliably reprodruce are more than welcome.

--
Mike Looijmans


Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijm...@topicproducts.com
W: www.topicproducts.com

Please consider the environment before printing this e-mail

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147351): 
https://lists.openembedded.org/g/openembedded-core/message/147351
Mute This Topic: https://lists.openembedded.org/mt/80159078/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-