Re: Rawhide aarch64: gcc bus error

[Justin Forbes]

On Thu, Dec 1, 2016 at 4:37 PM, Justin Forbes  wrote:
> On Wed, Nov 30, 2016 at 10:45 PM, Orion Poplawski  wrote:
>> On 11/30/2016 04:20 PM, Richard W.M. Jones wrote:
>>>
>>> On Wed, Nov 30, 2016 at 10:08:55PM +, Richard W.M. Jones wrote:

 On Wed, Nov 30, 2016 at 02:07:13PM -0700, Jerry James wrote:
>
> I'm trying to build python-cvxopt, but gcc is failing on aarch64 with
> a bus error:
>
> http://koji.fedoraproject.org/koji/taskinfo?taskID=16684863


 As a data point, the build succeeds on F25/aarch64 with:

   glibc-2.24-3.fc25.aarch64
   gcc-6.2.1-2.fc25.aarch64

 I'll spin up a Rawhide VM later to see if I can reproduce this with
 the suspect glibc etc (don't want to risk upgrading glibc on my main
 machine :-)
>>>
>>>
>>> On Rawhide (updated partially from F25), the compile works OK with:
>>>
>>> glibc-2.24-3.fc25.aarch64
>>> gcc-6.2.1-2.fc26.aarch64
>>>
>>> and it still *works* if I upgrade glibc to:
>>>
>>> glibc-2.24.90-19.fc26.aarch64
>>>
>>> (leaving all other non-glibc-related packages unchanged).
>>>
>>> But it *fails* in exactly the way you describe if I upgrade glibc to:
>>>
>>> glibc-2.24.90-20.fc26.aarch64
>>>
>>> So it looks fairly conclusively like it's something to do with the
>>> latest glibc, and you need to be looking at the differences between
>>> the -19 and -20 packages.
>>
>>
>> Thanks.  I've untaged it.
>>
>
> Seems like it is not actually untagged. My kernel build failed again
> today and it is still using -20.
>

Ignore me, I was looking at the wrong logs it seems. That was
yesterday's build. Today's works.

Justin
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide aarch64: gcc bus error

[Justin Forbes]

On Wed, Nov 30, 2016 at 10:45 PM, Orion Poplawski  wrote:
> On 11/30/2016 04:20 PM, Richard W.M. Jones wrote:
>>
>> On Wed, Nov 30, 2016 at 10:08:55PM +, Richard W.M. Jones wrote:
>>>
>>> On Wed, Nov 30, 2016 at 02:07:13PM -0700, Jerry James wrote:

 I'm trying to build python-cvxopt, but gcc is failing on aarch64 with
 a bus error:

 http://koji.fedoraproject.org/koji/taskinfo?taskID=16684863
>>>
>>>
>>> As a data point, the build succeeds on F25/aarch64 with:
>>>
>>>   glibc-2.24-3.fc25.aarch64
>>>   gcc-6.2.1-2.fc25.aarch64
>>>
>>> I'll spin up a Rawhide VM later to see if I can reproduce this with
>>> the suspect glibc etc (don't want to risk upgrading glibc on my main
>>> machine :-)
>>
>>
>> On Rawhide (updated partially from F25), the compile works OK with:
>>
>> glibc-2.24-3.fc25.aarch64
>> gcc-6.2.1-2.fc26.aarch64
>>
>> and it still *works* if I upgrade glibc to:
>>
>> glibc-2.24.90-19.fc26.aarch64
>>
>> (leaving all other non-glibc-related packages unchanged).
>>
>> But it *fails* in exactly the way you describe if I upgrade glibc to:
>>
>> glibc-2.24.90-20.fc26.aarch64
>>
>> So it looks fairly conclusively like it's something to do with the
>> latest glibc, and you need to be looking at the differences between
>> the -19 and -20 packages.
>
>
> Thanks.  I've untaged it.
>

Seems like it is not actually untagged. My kernel build failed again
today and it is still using -20.

Justin
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide aarch64: gcc bus error

[Florian Weimer]

On 12/01/2016 05:45 AM, Orion Poplawski wrote:


Thanks.  I've untaged it.


Thanks.

I've looked at this briefly and it's not clear to me yet what is going 
on.  TLS variable lookup inside mpfr appears to be broken, but the crash 
does not reproduce with the glibc upstream sources.  I don't know yet if 
this is a problem with a Fedora patch in glibc, an mpfr ABI issue, or 
something that is broken in glibc upstream (either due to a recent 
change or an older, latent bug).


Florian
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide aarch64: gcc bus error

[Orion Poplawski]

On 11/30/2016 04:20 PM, Richard W.M. Jones wrote:

On Wed, Nov 30, 2016 at 10:08:55PM +, Richard W.M. Jones wrote:

On Wed, Nov 30, 2016 at 02:07:13PM -0700, Jerry James wrote:

I'm trying to build python-cvxopt, but gcc is failing on aarch64 with
a bus error:

http://koji.fedoraproject.org/koji/taskinfo?taskID=16684863


As a data point, the build succeeds on F25/aarch64 with:

  glibc-2.24-3.fc25.aarch64
  gcc-6.2.1-2.fc25.aarch64

I'll spin up a Rawhide VM later to see if I can reproduce this with
the suspect glibc etc (don't want to risk upgrading glibc on my main
machine :-)


On Rawhide (updated partially from F25), the compile works OK with:

glibc-2.24-3.fc25.aarch64
gcc-6.2.1-2.fc26.aarch64

and it still *works* if I upgrade glibc to:

glibc-2.24.90-19.fc26.aarch64

(leaving all other non-glibc-related packages unchanged).

But it *fails* in exactly the way you describe if I upgrade glibc to:

glibc-2.24.90-20.fc26.aarch64

So it looks fairly conclusively like it's something to do with the
latest glibc, and you need to be looking at the differences between
the -19 and -20 packages.


Thanks.  I've untaged it.


--
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA/CoRA DivisionFAX: 303-415-9702
3380 Mitchell Lane  or...@cora.nwra.com
Boulder, CO 80301  http://www.cora.nwra.com
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide aarch64: gcc bus error

[Richard W.M. Jones]

On Wed, Nov 30, 2016 at 11:20:13PM +, Richard W.M. Jones wrote:
> On Wed, Nov 30, 2016 at 10:08:55PM +, Richard W.M. Jones wrote:
> > On Wed, Nov 30, 2016 at 02:07:13PM -0700, Jerry James wrote:
> > > I'm trying to build python-cvxopt, but gcc is failing on aarch64 with
> > > a bus error:
> > > 
> > > http://koji.fedoraproject.org/koji/taskinfo?taskID=16684863
> > 
> > As a data point, the build succeeds on F25/aarch64 with:
> > 
> >   glibc-2.24-3.fc25.aarch64
> >   gcc-6.2.1-2.fc25.aarch64
> > 
> > I'll spin up a Rawhide VM later to see if I can reproduce this with
> > the suspect glibc etc (don't want to risk upgrading glibc on my main
> > machine :-)
> 
> On Rawhide (updated partially from F25), the compile works OK with:
> 
> glibc-2.24-3.fc25.aarch64
> gcc-6.2.1-2.fc26.aarch64
> 
> and it still *works* if I upgrade glibc to:
> 
> glibc-2.24.90-19.fc26.aarch64
> 
> (leaving all other non-glibc-related packages unchanged).
> 
> But it *fails* in exactly the way you describe if I upgrade glibc to:
> 
> glibc-2.24.90-20.fc26.aarch64
> 
> So it looks fairly conclusively like it's something to do with the
> latest glibc, and you need to be looking at the differences between
> the -19 and -20 packages.

That glibc is crash-central too.  The act of installing gdb to try to
get a stack trace itself caused a crash :-(

The stack trace may be a bit broken, but here goes:

Core was generated by `/usr/libexec/gcc/aarch64-redhat-linux/6.2.1/cc1 -quiet 
-I /usr/include -I /usr/'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x010b48c0 in aarch64_fallback_frame_state (context=0x3fff9a499c0, 
context=0x3fff9a499c0, fs=0x3fff9a49d80) at ./md-unwind-support.h:71
71if (pc[0] != MOVZ_X8_8B || pc[1] != SVC_0)
Missing separate debuginfos, use: dnf debuginfo-install 
glibc-2.24.90-19.fc26.aarch64 gmp-6.1.1-1.fc25.aarch64 
libmpc-1.0.2-5.fc24.aarch64 mpfr-3.1.5-1.fc25.aarch64 zlib-1.2.8-10.fc24.aarch64
(gdb) t a a bt

Thread 1 (LWP 18161):
#0  0x010b48c0 in aarch64_fallback_frame_state (context=0x3fff9a499c0, 
context=0x3fff9a499c0, fs=0x3fff9a49d80) at ./md-unwind-support.h:71
#1  uw_frame_state_for (context=context@entry=0x3fff9a499c0, 
fs=fs@entry=0x3fff9a49d80) at ../../../libgcc/unwind-dw2.c:1249
#2  0x010b6388 in _Unwind_Backtrace (
trace=trace@entry=0x8657cc , trace_argument=0x3fff9a4a428, 
trace_argument@entry=0x3fff9a4a438) at ../../../libgcc/unwind.inc:290
#3  0x00865920 in backtrace_full (state=0x3ff8b9f, 
skip=skip@entry=2, 
callback=callback@entry=0x84cfbc , 
error_callback=error_callback@entry=0x84cf54 , data=data@entry=0x3fff9a4a494)
at ../../libbacktrace/backtrace.c:127
#4  0x0101bc98 in diagnostic_action_after_output (
context=context@entry=0x15037a0 , 
diag_kind=) at ../../gcc/diagnostic.c:476
#5  0x0101b270 in diagnostic_report_diagnostic (
context=0x15037a0 , diagnostic=0x3fff9a4a568, 
diagnostic@entry=0x3fff9a4a578) at ../../gcc/diagnostic.c:827
#6  0x0084cac8 in internal_error (gmsgid=gmsgid@entry=0x111e3c0 "%s")
at ../../gcc/diagnostic.c:1258
#7  0x006bd188 in crash_signal (signo=11) at ../../gcc/toplev.c:333
#8  
#9  0x010b48c0 in aarch64_fallback_frame_state (context=0x3fff9a4ba60, 
context=0x3fff9a4ba60, fs=0x3fff9a4be20) at ./md-unwind-support.h:71
#10 uw_frame_state_for (context=context@entry=0x3fff9a4ba60, 
fs=fs@entry=0x3fff9a4be20) at ../../../libgcc/unwind-dw2.c:1249
#11 0x010b6388 in _Unwind_Backtrace (
trace=trace@entry=0x8657cc , trace_argument=0x3fff9a4c4c8, 
trace_argument@entry=0x3fff9a4c4d8) at ../../../libgcc/unwind.inc:290
#12 0x00865920 in backtrace_full (state=0x3ff8ba0, 
skip=skip@entry=2, 
callback=callback@entry=0x84cfbc , 
error_callback=error_callback@entry=0x84cf54 , data=data@entry=0x3fff9a4c534)
at ../../libbacktrace/backtrace.c:127
#13 0x0101bc98 in diagnostic_action_after_output (
context=context@entry=0x15037a0 , 
diag_kind=) at ../../gcc/diagnostic.c:476
#14 0x0101b270 in diagnostic_report_diagnostic (
context=0x15037a0 , diagnostic=0x3fff9a4c608, 
diagnostic@entry=0x3fff9a4c618) at ../../gcc/diagnostic.c:827
#15 0x0084cac8 in internal_error (gmsgid=gmsgid@entry=0x111e3c0 "%s")
at ../../gcc/diagnostic.c:1258
#16 0x006bd188 in crash_signal (signo=7) at ../../gcc/toplev.c:333
#17 
#18 0x003f in ?? ()
#19 0x03ff8dee0acc in mpfr_init2 () from /lib64/libmpfr.so.4
#20 0x00b1223c in real_from_string (r=r@entry=0x3fff9a4db48, 
str=0x3fff9a4daa0 "1.0") at ../../gcc/real.c:2106
#21 0x00f50470 in real_from_string3 (r=0x3fff9a4db48, 
s=, fmt=...) at ../../gcc/real.c:2173

Re: Rawhide aarch64: gcc bus error

[Richard W.M. Jones]

On Wed, Nov 30, 2016 at 10:08:55PM +, Richard W.M. Jones wrote:
> On Wed, Nov 30, 2016 at 02:07:13PM -0700, Jerry James wrote:
> > I'm trying to build python-cvxopt, but gcc is failing on aarch64 with
> > a bus error:
> > 
> > http://koji.fedoraproject.org/koji/taskinfo?taskID=16684863
> 
> As a data point, the build succeeds on F25/aarch64 with:
> 
>   glibc-2.24-3.fc25.aarch64
>   gcc-6.2.1-2.fc25.aarch64
> 
> I'll spin up a Rawhide VM later to see if I can reproduce this with
> the suspect glibc etc (don't want to risk upgrading glibc on my main
> machine :-)

On Rawhide (updated partially from F25), the compile works OK with:

glibc-2.24-3.fc25.aarch64
gcc-6.2.1-2.fc26.aarch64

and it still *works* if I upgrade glibc to:

glibc-2.24.90-19.fc26.aarch64

(leaving all other non-glibc-related packages unchanged).

But it *fails* in exactly the way you describe if I upgrade glibc to:

glibc-2.24.90-20.fc26.aarch64

So it looks fairly conclusively like it's something to do with the
latest glibc, and you need to be looking at the differences between
the -19 and -20 packages.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide aarch64: gcc bus error

[Richard W.M. Jones]

On Wed, Nov 30, 2016 at 02:07:13PM -0700, Jerry James wrote:
> I'm trying to build python-cvxopt, but gcc is failing on aarch64 with
> a bus error:
> 
> http://koji.fedoraproject.org/koji/taskinfo?taskID=16684863

As a data point, the build succeeds on F25/aarch64 with:

  glibc-2.24-3.fc25.aarch64
  gcc-6.2.1-2.fc25.aarch64

I'll spin up a Rawhide VM later to see if I can reproduce this with
the suspect glibc etc (don't want to risk upgrading glibc on my main
machine :-)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide aarch64: gcc bus error

[Tom Hughes]

On 30/11/16 21:41, Jerry James wrote:

On Wed, Nov 30, 2016 at 2:19 PM, Tom Hughes  wrote:

On 30/11/16 21:07, Jerry James wrote:
Well have you reported the ICE as it tells you to?

Basically it is, pretty much by definition, a bug in the compiler that needs
to be reported. Biggest pain is going to be getting hold of the preprocessed
source for the bug report.


I don't know that it is a compiler bug, actually.  As Ben Rosser
pointed out, this may be a glibc bug.  There's a brand new glibc in
Rawhide, and now multiple builds are hitting bus errors while building
for aarch64.  Can we back out that glibc build until the problem is
identified, at least on aarch64?


Sure, it could be a library that the compiler uses, though it doesn't 
use very much. So glibc is the other likely culprit sure.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide aarch64: gcc bus error

[Jerry James]

On Wed, Nov 30, 2016 at 2:19 PM, Tom Hughes  wrote:
> On 30/11/16 21:07, Jerry James wrote:
> Well have you reported the ICE as it tells you to?
>
> Basically it is, pretty much by definition, a bug in the compiler that needs
> to be reported. Biggest pain is going to be getting hold of the preprocessed
> source for the bug report.

I don't know that it is a compiler bug, actually.  As Ben Rosser
pointed out, this may be a glibc bug.  There's a brand new glibc in
Rawhide, and now multiple builds are hitting bus errors while building
for aarch64.  Can we back out that glibc build until the problem is
identified, at least on aarch64?
-- 
Jerry James
http://www.jamezone.org/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide aarch64: gcc bus error

[Ben Rosser]

On Wed, Nov 30, 2016 at 4:07 PM, Jerry James  wrote:

> I'm trying to build python-cvxopt, but gcc is failing on aarch64 with
> a bus error:
>
> http://koji.fedoraproject.org/koji/taskinfo?taskID=16684863
>
> I took a quick look at koji and found another recent failing build
> with the same problem:
>
> http://koji.fedoraproject.org/koji/taskinfo?taskID=16684488
>
> Both builds indicate that gcc was compiling a simple arithmetic
> expression when the bus error occurred.  Does anybody know what's
> going on there?  Thanks,
> --
> Jerry James
>

I'm seeing the same thing with elog; koschei alerted me to an aarch64
failure here: https://apps.fedoraproject.org/koschei/build/2381814. I was
able to reproduce the failure by running a scratch-build:
http://koji.fedoraproject.org/koji/taskinfo?taskID=16684965.

Given that the koschei build fired in response to (among a few other
things) a glibc upgrade, I wonder if this is a glibc bug on aarch64?

Ben Rosser
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide aarch64: gcc bus error

[Tom Hughes]

On 30/11/16 21:07, Jerry James wrote:


I'm trying to build python-cvxopt, but gcc is failing on aarch64 with
a bus error:

http://koji.fedoraproject.org/koji/taskinfo?taskID=16684863

I took a quick look at koji and found another recent failing build
with the same problem:

http://koji.fedoraproject.org/koji/taskinfo?taskID=16684488

Both builds indicate that gcc was compiling a simple arithmetic
expression when the bus error occurred.  Does anybody know what's
going on there?  Thanks,


Well have you reported the ICE as it tells you to?

Basically it is, pretty much by definition, a bug in the compiler that 
needs to be reported. Biggest pain is going to be getting hold of the 
preprocessed source for the bug report.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Rawhide aarch64: gcc bus error

[Jerry James]

I'm trying to build python-cvxopt, but gcc is failing on aarch64 with
a bus error:

http://koji.fedoraproject.org/koji/taskinfo?taskID=16684863

I took a quick look at koji and found another recent failing build
with the same problem:

http://koji.fedoraproject.org/koji/taskinfo?taskID=16684488

Both builds indicate that gcc was compiling a simple arithmetic
expression when the bus error occurred.  Does anybody know what's
going on there?  Thanks,
-- 
Jerry James
http://www.jamezone.org/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org