Re: [U-Boot] Fwd: Parallel build is broken
On Mon, Sep 10, 2018 at 11:46 AM, Trent Piepho wrote: > On Wed, 2018-09-05 at 06:57 -0500, Adam Ford wrote: >> On Wed, Sep 5, 2018 at 3:46 AM Alex Kiernan wrote: >> > >> > On Tue, Sep 4, 2018 at 3:54 PM Andy Shevchenko >> > wrote: >> > > >> > > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko >> > > wrote: >> > > > >> > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote: >> > > > > >> > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: >> > > > > >> > > > > > -- Forwarded message - >> > > > > > From: Andy Shevchenko >> > > > > > Date: Tue, Sep 4, 2018 at 3:40 PM >> > > > > > Subject: Parallel build is broken >> > > > > > To: Tom Rini >> > > > > > Cc: Bin Meng , Simon Glass >> > > > > > >> > > > > > >> > > > > > U-Boot can't be build with make -j16. >> > > > > > >> > > > > > Second pass passes fine. >> > > > > > >> > > > > > Didn't investigate myself and probably will not have time (near to >> > > > > > vacation time for me). >> > > > > > So, please, fix it before release. >> > > > > > >> > > > > > sh: echo: I/O error >> > > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] >> > > > > > Error 1 >> >> I am no expert, but I/O error to me indicates a hardware problem on >> the build machine and not necessarily a software issue. > > I doubt that is the issue. While EIO can be returned in response to a > hardware failure, it would be very random and most likely affect > something - the linker, a compiler - that process a lot of files or > data, not an echo command. > > >> > I've seen this problem - building inside a VMware VM. I've always >> > found that logging out and back in appears to fix it, so possibly >> > something related to the session... I've similarly failed to figure >> > out what the problem is as it comes and goes. > > This is likely on track to the real problem. What happened is > something echo did returned EIO as an errno. That failing syscall was > probably a write(3), since echo doesn't do a whole lot else! And why > can write(3) return EIO, besides a hardware error? > >EIOThe process is a member of a background process group > attempting to write to its controlling terminal, TOSTOP > is set, the calling thread is not blocking SIGTTOU, the > process is not ignoring SIGTTOU, and the process group of > the process is orphaned. This error may also be returned > under implementation-defined conditions. > > I think there's a $(shell echo something &) or something like that > somewhere. I observe similar issue when I build QEMU except that its makefiles use 'printf' instead of 'echo' in the 'quiet-command' macro. But I observe the same issue even when building with V=1, i.e. when 'printf' is not used. In that case the error comes from the make itself. I tested a range of make versions from 3.81 to 4.2.1 and see the issue with all of them. When it happens under strace it looks like this (make V=0): 8341 write(1, " CC arch/AArch64/AArch64Di"..., 45) = -1 EAGAIN (Resource temporarily unavailable) or this (make V=1): 31716 write(1, "c++ -I/usr/include/pixman-1 -W"..., 1024) = 1024 31716 write(1, "tcg-op-gvec.o tcg/tcg-common.o t"..., 10240) = 7936 31716 write(1, "ut-keymap.o ../ui/input-legacy.o"..., 2304) = -1 EAGAIN (Resource temporarily unavailable) 31716 write(1, "\n", 1) = 1 I suspect that it's a kernel issue, as I started to notice it after the system upgrade from Debian 8 to Debian 9. -- Thanks. -- Max ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Wed, 2018-09-05 at 06:57 -0500, Adam Ford wrote: > On Wed, Sep 5, 2018 at 3:46 AM Alex Kiernan wrote: > > > > On Tue, Sep 4, 2018 at 3:54 PM Andy Shevchenko > > wrote: > > > > > > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko > > > wrote: > > > > > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote: > > > > > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: > > > > > > > > > > > -- Forwarded message - > > > > > > From: Andy Shevchenko > > > > > > Date: Tue, Sep 4, 2018 at 3:40 PM > > > > > > Subject: Parallel build is broken > > > > > > To: Tom Rini > > > > > > Cc: Bin Meng , Simon Glass > > > > > > > > > > > > > > > > > > U-Boot can't be build with make -j16. > > > > > > > > > > > > Second pass passes fine. > > > > > > > > > > > > Didn't investigate myself and probably will not have time (near to > > > > > > vacation time for me). > > > > > > So, please, fix it before release. > > > > > > > > > > > > sh: echo: I/O error > > > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] > > > > > > Error 1 > > I am no expert, but I/O error to me indicates a hardware problem on > the build machine and not necessarily a software issue. I doubt that is the issue. While EIO can be returned in response to a hardware failure, it would be very random and most likely affect something - the linker, a compiler - that process a lot of files or data, not an echo command. > > I've seen this problem - building inside a VMware VM. I've always > > found that logging out and back in appears to fix it, so possibly > > something related to the session... I've similarly failed to figure > > out what the problem is as it comes and goes. This is likely on track to the real problem. What happened is something echo did returned EIO as an errno. That failing syscall was probably a write(3), since echo doesn't do a whole lot else! And why can write(3) return EIO, besides a hardware error? EIOThe process is a member of a background process group attempting to write to its controlling terminal, TOSTOP is set, the calling thread is not blocking SIGTTOU, the process is not ignoring SIGTTOU, and the process group of the process is orphaned. This error may also be returned under implementation-defined conditions. I think there's a $(shell echo something &) or something like that somewhere. ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Wed, Sep 5, 2018 at 2:58 PM Adam Ford wrote: > > On Wed, Sep 5, 2018 at 3:46 AM Alex Kiernan wrote: > > > > On Tue, Sep 4, 2018 at 3:54 PM Andy Shevchenko > > wrote: > > > > > > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko > > > wrote: > > > > > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote: > > > > > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: > > > > > > > > > > > -- Forwarded message - > > > > > > From: Andy Shevchenko > > > > > > Date: Tue, Sep 4, 2018 at 3:40 PM > > > > > > Subject: Parallel build is broken > > > > > > To: Tom Rini > > > > > > Cc: Bin Meng , Simon Glass > > > > > > > > > > > > > > > > > > U-Boot can't be build with make -j16. > > > > > > > > > > > > Second pass passes fine. > > > > > > > > > > > > Didn't investigate myself and probably will not have time (near to > > > > > > vacation time for me). > > > > > > So, please, fix it before release. > > > > > > > > > > > > sh: echo: I/O error > > > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] > > > > > > Error 1 > > I am no expert, but I/O error to me indicates a hardware problem on > the build machine and not necessarily a software issue. Not exactly. AFAIR it's a message mapped to EIO errno. The scenario can be like that (hypothetically), make runs compiler, but instead of compiler ccache is being called and, in case it's somehow broken, it might try to access to wrong block on the block device which returns EIO on some (read()?) system call. -- With Best Regards, Andy Shevchenko ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Wed, Sep 5, 2018 at 3:46 AM Alex Kiernan wrote: > > On Tue, Sep 4, 2018 at 3:54 PM Andy Shevchenko > wrote: > > > > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko > > wrote: > > > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote: > > > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: > > > > > > > > > -- Forwarded message - > > > > > From: Andy Shevchenko > > > > > Date: Tue, Sep 4, 2018 at 3:40 PM > > > > > Subject: Parallel build is broken > > > > > To: Tom Rini > > > > > Cc: Bin Meng , Simon Glass > > > > > > > > > > > > > > > U-Boot can't be build with make -j16. > > > > > > > > > > Second pass passes fine. > > > > > > > > > > Didn't investigate myself and probably will not have time (near to > > > > > vacation time for me). > > > > > So, please, fix it before release. > > > > > > > > > > sh: echo: I/O error > > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] > > > > > Error 1 I am no expert, but I/O error to me indicates a hardware problem on the build machine and not necessarily a software issue. adam > > > > > make[1]: *** Waiting for unfinished jobs > > > > > CC drivers/serial/ns16550.o > > > > > CC drivers/spi/spi.o > > > > > CC cmd/mmc.o > > > > > CC drivers/watchdog/tangier_wdt.o > > > > > CC drivers/serial/serial_intel_mid.o > > > > > CC cmd/net.o > > > > > CC drivers/usb/common/common.o > > > > > CC arch/x86/lib/acpi.o > > > > > LD drivers/timer/built-in.o > > > > > CC arch/x86/lib/acpi_table.o > > > > > CC drivers/video/console_normal.o > > > > > make: *** [Makefile:1377: common] Error 2 > > > > > make: *** Waiting for unfinished jobs > > > > > > > > What build target (and O= or not?) is this with? Thanks! > > > > > > make clean && make edison_defconfig && make -j16 > > > > > > gcc (Debian 8.2.0-4) 8.2.0 > > > Copyright (C) 2018 Free Software Foundation, Inc. > > > This is free software; see the source for copying conditions. There is NO > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR > > > PURPOSE. > > > > So, first time it's being built, consequent runs usually failed. > > > > I've seen this problem - building inside a VMware VM. I've always > found that logging out and back in appears to fix it, so possibly > something related to the session... I've similarly failed to figure > out what the problem is as it comes and goes. > > -- > Alex Kiernan > ___ > U-Boot mailing list > U-Boot@lists.denx.de > https://lists.denx.de/listinfo/u-boot ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 4, 2018 at 3:54 PM Andy Shevchenko wrote: > > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko > wrote: > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote: > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: > > > > > > > -- Forwarded message - > > > > From: Andy Shevchenko > > > > Date: Tue, Sep 4, 2018 at 3:40 PM > > > > Subject: Parallel build is broken > > > > To: Tom Rini > > > > Cc: Bin Meng , Simon Glass > > > > > > > > > > > > U-Boot can't be build with make -j16. > > > > > > > > Second pass passes fine. > > > > > > > > Didn't investigate myself and probably will not have time (near to > > > > vacation time for me). > > > > So, please, fix it before release. > > > > > > > > sh: echo: I/O error > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error > > > > 1 > > > > make[1]: *** Waiting for unfinished jobs > > > > CC drivers/serial/ns16550.o > > > > CC drivers/spi/spi.o > > > > CC cmd/mmc.o > > > > CC drivers/watchdog/tangier_wdt.o > > > > CC drivers/serial/serial_intel_mid.o > > > > CC cmd/net.o > > > > CC drivers/usb/common/common.o > > > > CC arch/x86/lib/acpi.o > > > > LD drivers/timer/built-in.o > > > > CC arch/x86/lib/acpi_table.o > > > > CC drivers/video/console_normal.o > > > > make: *** [Makefile:1377: common] Error 2 > > > > make: *** Waiting for unfinished jobs > > > > > > What build target (and O= or not?) is this with? Thanks! > > > > make clean && make edison_defconfig && make -j16 > > > > gcc (Debian 8.2.0-4) 8.2.0 > > Copyright (C) 2018 Free Software Foundation, Inc. > > This is free software; see the source for copying conditions. There is NO > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > So, first time it's being built, consequent runs usually failed. > I've seen this problem - building inside a VMware VM. I've always found that logging out and back in appears to fix it, so possibly something related to the session... I've similarly failed to figure out what the problem is as it comes and goes. -- Alex Kiernan ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, 2018-09-04 at 17:43 -0400, Tom Rini wrote:
> CAUTION: This email originated from outside of the organization. Do not click
> links or open attachments unless you recognize the sender and know the
> content is safe.
>
>
> On Tue, Sep 04, 2018 at 09:05:55PM +0300, Andy Shevchenko wrote:
> > On Tue, Sep 4, 2018 at 9:00 PM Tom Rini wrote:
> > >
> > > On Tue, Sep 04, 2018 at 07:33:10PM +0300, Andy Shevchenko wrote:
> > > > On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
> > > > wrote:
> > > > >
> > > > > On Tue, Sep 4, 2018 at 6:14 PM Tom Rini wrote:
> > > > > > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini
> > > > > > > wrote:
> > > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > > > > > > make clean && make edison_defconfig && make -j16
> > > > > > >
> > > > > > > gcc (Debian 8.2.0-4) 8.2.0
> > > > > > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > > > > > This is free software; see the source for copying conditions.
> > > > > > > There is NO
> > > > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A
> > > > > > > PARTICULAR PURPOSE.
> > > > > >
> > > > > > I assume this is on Debian/unstable?
> > > > >
> > > > > testing
> > > > >
> > > > > > I can't directly replicate this on
> > > > > > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines. I'll
> > > > > > setup a chroot soon, but since you've said -j64 is fine there too I
> > > > > > suspect you have more cores than I. This may be something you have
> > > > > > to
> > > > > > bisect for us if I can't replicate it myself. Can you confirm how
> > > > > > many
> > > > > > cores you have? I might be able to spin something up in Google
> > > > > > compute.
> > > > >
> > > > > $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> > > > > cpu cores : 22
> > > >
> > > > Did few runs (~6) on this machine with -j4, no failures so far.
> > > > Reruning same with -j16 brings failure on ~2-3 iteration.
> > > >
> > > > It seems the scope can be narrowed to:
> > > > - many cores build system, and
> > > > - Debian testing/unstable toolchain, and/or
> > > > - U-Boot build system
> > >
> > > I'm pretty sure it's a dependency problem somewhere. Since this was
> > > working reliably for you recently (right?) and I can't reproduce it and
> > > you can, if you can run a git bisect to figure out what commit is
> > > breaking things, that would be very helpful. Thanks!
> >
There was a patch recently to make 4.2.1 to fix some parallel build issue.
No idea if that fix applies to your case though.
Jocke
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
Hi Andy,
On Wed, Sep 5, 2018 at 5:43 AM Tom Rini wrote:
>
> On Tue, Sep 04, 2018 at 09:05:55PM +0300, Andy Shevchenko wrote:
> > On Tue, Sep 4, 2018 at 9:00 PM Tom Rini wrote:
> > >
> > > On Tue, Sep 04, 2018 at 07:33:10PM +0300, Andy Shevchenko wrote:
> > > > On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
> > > > wrote:
> > > > >
> > > > > On Tue, Sep 4, 2018 at 6:14 PM Tom Rini wrote:
> > > > > > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini
> > > > > > > wrote:
> > > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > > > >
> > > > > > > make clean && make edison_defconfig && make -j16
> > > > > > >
> > > > > > > gcc (Debian 8.2.0-4) 8.2.0
> > > > > > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > > > > > This is free software; see the source for copying conditions.
> > > > > > > There is NO
> > > > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A
> > > > > > > PARTICULAR PURPOSE.
> > > > > >
> > > > > > I assume this is on Debian/unstable?
> > > > >
> > > > > testing
> > > > >
> > > > > > I can't directly replicate this on
> > > > > > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines. I'll
> > > > > > setup a chroot soon, but since you've said -j64 is fine there too I
> > > > > > suspect you have more cores than I. This may be something you have
> > > > > > to
> > > > > > bisect for us if I can't replicate it myself. Can you confirm how
> > > > > > many
> > > > > > cores you have? I might be able to spin something up in Google
> > > > > > compute.
> > > > >
> > > > > $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> > > > > cpu cores : 22
> > > >
> > > > Did few runs (~6) on this machine with -j4, no failures so far.
> > > > Reruning same with -j16 brings failure on ~2-3 iteration.
> > > >
> > > > It seems the scope can be narrowed to:
> > > > - many cores build system, and
> > > > - Debian testing/unstable toolchain, and/or
> > > > - U-Boot build system
> > >
> > > I'm pretty sure it's a dependency problem somewhere. Since this was
> > > working reliably for you recently (right?) and I can't reproduce it and
> > > you can, if you can run a git bisect to figure out what commit is
> > > breaking things, that would be very helpful. Thanks!
> >
> > Now after my vacation.
> > So, I guess it's not so critical for release as I thought at the beginning.
>
> I'd love to fix it, but I can't replicate it.
I can't reproduce this on my machine too, on Ubuntu 16.04 with either
native gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10), or
the kernel.org gcc (gcc version 7.3.0 (GCC)), using below commands:
$ make clean && make edison_defconfig && make -j64
Regards,
Bin
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 04, 2018 at 09:05:55PM +0300, Andy Shevchenko wrote:
> On Tue, Sep 4, 2018 at 9:00 PM Tom Rini wrote:
> >
> > On Tue, Sep 04, 2018 at 07:33:10PM +0300, Andy Shevchenko wrote:
> > > On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
> > > wrote:
> > > >
> > > > On Tue, Sep 4, 2018 at 6:14 PM Tom Rini wrote:
> > > > > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote:
> > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > > >
> > > > > > make clean && make edison_defconfig && make -j16
> > > > > >
> > > > > > gcc (Debian 8.2.0-4) 8.2.0
> > > > > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > > > > This is free software; see the source for copying conditions.
> > > > > > There is NO
> > > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> > > > > > PURPOSE.
> > > > >
> > > > > I assume this is on Debian/unstable?
> > > >
> > > > testing
> > > >
> > > > > I can't directly replicate this on
> > > > > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines. I'll
> > > > > setup a chroot soon, but since you've said -j64 is fine there too I
> > > > > suspect you have more cores than I. This may be something you have to
> > > > > bisect for us if I can't replicate it myself. Can you confirm how
> > > > > many
> > > > > cores you have? I might be able to spin something up in Google
> > > > > compute.
> > > >
> > > > $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> > > > cpu cores : 22
> > >
> > > Did few runs (~6) on this machine with -j4, no failures so far.
> > > Reruning same with -j16 brings failure on ~2-3 iteration.
> > >
> > > It seems the scope can be narrowed to:
> > > - many cores build system, and
> > > - Debian testing/unstable toolchain, and/or
> > > - U-Boot build system
> >
> > I'm pretty sure it's a dependency problem somewhere. Since this was
> > working reliably for you recently (right?) and I can't reproduce it and
> > you can, if you can run a git bisect to figure out what commit is
> > breaking things, that would be very helpful. Thanks!
>
> Now after my vacation.
> So, I guess it's not so critical for release as I thought at the beginning.
I'd love to fix it, but I can't replicate it.
--
Tom
signature.asc
Description: PGP signature
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 4, 2018 at 9:00 PM Tom Rini wrote:
>
> On Tue, Sep 04, 2018 at 07:33:10PM +0300, Andy Shevchenko wrote:
> > On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
> > wrote:
> > >
> > > On Tue, Sep 4, 2018 at 6:14 PM Tom Rini wrote:
> > > > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote:
> > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > >
> > > > > make clean && make edison_defconfig && make -j16
> > > > >
> > > > > gcc (Debian 8.2.0-4) 8.2.0
> > > > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > > > This is free software; see the source for copying conditions. There
> > > > > is NO
> > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> > > > > PURPOSE.
> > > >
> > > > I assume this is on Debian/unstable?
> > >
> > > testing
> > >
> > > > I can't directly replicate this on
> > > > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines. I'll
> > > > setup a chroot soon, but since you've said -j64 is fine there too I
> > > > suspect you have more cores than I. This may be something you have to
> > > > bisect for us if I can't replicate it myself. Can you confirm how many
> > > > cores you have? I might be able to spin something up in Google compute.
> > >
> > > $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> > > cpu cores : 22
> >
> > Did few runs (~6) on this machine with -j4, no failures so far.
> > Reruning same with -j16 brings failure on ~2-3 iteration.
> >
> > It seems the scope can be narrowed to:
> > - many cores build system, and
> > - Debian testing/unstable toolchain, and/or
> > - U-Boot build system
>
> I'm pretty sure it's a dependency problem somewhere. Since this was
> working reliably for you recently (right?) and I can't reproduce it and
> you can, if you can run a git bisect to figure out what commit is
> breaking things, that would be very helpful. Thanks!
Now after my vacation.
So, I guess it's not so critical for release as I thought at the beginning.
--
With Best Regards,
Andy Shevchenko
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 04, 2018 at 07:33:10PM +0300, Andy Shevchenko wrote:
> On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
> wrote:
> >
> > On Tue, Sep 4, 2018 at 6:14 PM Tom Rini wrote:
> > > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote:
> > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> >
> > > > make clean && make edison_defconfig && make -j16
> > > >
> > > > gcc (Debian 8.2.0-4) 8.2.0
> > > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > > This is free software; see the source for copying conditions. There is
> > > > NO
> > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> > > > PURPOSE.
> > >
> > > I assume this is on Debian/unstable?
> >
> > testing
> >
> > > I can't directly replicate this on
> > > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines. I'll
> > > setup a chroot soon, but since you've said -j64 is fine there too I
> > > suspect you have more cores than I. This may be something you have to
> > > bisect for us if I can't replicate it myself. Can you confirm how many
> > > cores you have? I might be able to spin something up in Google compute.
> >
> > $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> > cpu cores : 22
>
> Did few runs (~6) on this machine with -j4, no failures so far.
> Reruning same with -j16 brings failure on ~2-3 iteration.
>
> It seems the scope can be narrowed to:
> - many cores build system, and
> - Debian testing/unstable toolchain, and/or
> - U-Boot build system
I'm pretty sure it's a dependency problem somewhere. Since this was
working reliably for you recently (right?) and I can't reproduce it and
you can, if you can run a git bisect to figure out what commit is
breaking things, that would be very helpful. Thanks!
--
Tom
signature.asc
Description: PGP signature
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
wrote:
>
> On Tue, Sep 4, 2018 at 6:14 PM Tom Rini wrote:
> > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote:
> > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
>
> > > make clean && make edison_defconfig && make -j16
> > >
> > > gcc (Debian 8.2.0-4) 8.2.0
> > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > This is free software; see the source for copying conditions. There is NO
> > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> > > PURPOSE.
> >
> > I assume this is on Debian/unstable?
>
> testing
>
> > I can't directly replicate this on
> > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines. I'll
> > setup a chroot soon, but since you've said -j64 is fine there too I
> > suspect you have more cores than I. This may be something you have to
> > bisect for us if I can't replicate it myself. Can you confirm how many
> > cores you have? I might be able to spin something up in Google compute.
>
> $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> cpu cores : 22
Did few runs (~6) on this machine with -j4, no failures so far.
Reruning same with -j16 brings failure on ~2-3 iteration.
It seems the scope can be narrowed to:
- many cores build system, and
- Debian testing/unstable toolchain, and/or
- U-Boot build system
> OTOH, I tried on my desktop (where is unstable) and everything works
> fine with the same (-j16) command line on any gcc I tried.
> $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> cpu cores : 4
--
With Best Regards,
Andy Shevchenko
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 4, 2018 at 6:14 PM Tom Rini wrote:
> On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote:
> > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > make clean && make edison_defconfig && make -j16
> >
> > gcc (Debian 8.2.0-4) 8.2.0
> > Copyright (C) 2018 Free Software Foundation, Inc.
> > This is free software; see the source for copying conditions. There is NO
> > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>
> I assume this is on Debian/unstable?
testing
> I can't directly replicate this on
> my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines. I'll
> setup a chroot soon, but since you've said -j64 is fine there too I
> suspect you have more cores than I. This may be something you have to
> bisect for us if I can't replicate it myself. Can you confirm how many
> cores you have? I might be able to spin something up in Google compute.
$ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
cpu cores : 22
OTOH, I tried on my desktop (where is unstable) and everything works
fine with the same (-j16) command line on any gcc I tried.
$ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
cpu cores : 4
--
With Best Regards,
Andy Shevchenko
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote: > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote: > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: > > > > > -- Forwarded message - > > > From: Andy Shevchenko > > > Date: Tue, Sep 4, 2018 at 3:40 PM > > > Subject: Parallel build is broken > > > To: Tom Rini > > > Cc: Bin Meng , Simon Glass > > > > > > > > > U-Boot can't be build with make -j16. > > > > > > Second pass passes fine. > > > > > > Didn't investigate myself and probably will not have time (near to > > > vacation time for me). > > > So, please, fix it before release. > > > > > > sh: echo: I/O error > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1 > > > make[1]: *** Waiting for unfinished jobs > > > CC drivers/serial/ns16550.o > > > CC drivers/spi/spi.o > > > CC cmd/mmc.o > > > CC drivers/watchdog/tangier_wdt.o > > > CC drivers/serial/serial_intel_mid.o > > > CC cmd/net.o > > > CC drivers/usb/common/common.o > > > CC arch/x86/lib/acpi.o > > > LD drivers/timer/built-in.o > > > CC arch/x86/lib/acpi_table.o > > > CC drivers/video/console_normal.o > > > make: *** [Makefile:1377: common] Error 2 > > > make: *** Waiting for unfinished jobs > > > > What build target (and O= or not?) is this with? Thanks! > > make clean && make edison_defconfig && make -j16 > > gcc (Debian 8.2.0-4) 8.2.0 > Copyright (C) 2018 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. I assume this is on Debian/unstable? I can't directly replicate this on my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines. I'll setup a chroot soon, but since you've said -j64 is fine there too I suspect you have more cores than I. This may be something you have to bisect for us if I can't replicate it myself. Can you confirm how many cores you have? I might be able to spin something up in Google compute. Thanks! -- Tom signature.asc Description: PGP signature ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 4, 2018 at 6:03 PM Andy Shevchenko wrote: > > On Tue, Sep 4, 2018 at 5:54 PM Andy Shevchenko > wrote: > > > > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko > > wrote: > > > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote: > > > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: > > > > > > U-Boot can't be build with make -j16. > > > > > > > > > > Second pass passes fine. > > > > > > > > > > Didn't investigate myself and probably will not have time (near to > > > > > vacation time for me). > > > > > So, please, fix it before release. > > > > > > > > > > sh: echo: I/O error > > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] > > > > > Error 1 > > > > > make[1]: *** Waiting for unfinished jobs > > > > > > make: *** [Makefile:1377: common] Error 2 > > > > > make: *** Waiting for unfinished jobs > > > > > > > > What build target (and O= or not?) is this with? Thanks! > > > > > > make clean && make edison_defconfig && make -j16 > > > > > > gcc (Debian 8.2.0-4) 8.2.0 > > > Copyright (C) 2018 Free Software Foundation, Inc. > > > This is free software; see the source for copying conditions. There is NO > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR > > > PURPOSE. > > Switching to gcc-7 does *not* cure. > > So, first time it's being built, consequent runs usually failed. > > More information: > - when consequential run is being executed no code had been changed > - the object files where it fails is totally arbitrary > - I used ccache, though I disabled for now completely (through > modifying PATH) and problem still exists > - Linux kernel rebuilds on that very same machine fine (like touch > include/linux/perf_event.h; make -j64 in a loop) > > It's total mystery to me. It was rebuilding just fine yesterday, I > updated today from GH and even older (v2018.09-rc2) fails. > > -- > With Best Regards, > Andy Shevchenko -- With Best Regards, Andy Shevchenko ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 4, 2018 at 5:54 PM Andy Shevchenko wrote: > > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko > wrote: > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote: > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: > > > > U-Boot can't be build with make -j16. > > > > > > > > Second pass passes fine. > > > > > > > > Didn't investigate myself and probably will not have time (near to > > > > vacation time for me). > > > > So, please, fix it before release. > > > > > > > > sh: echo: I/O error > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error > > > > 1 > > > > make[1]: *** Waiting for unfinished jobs > > > > make: *** [Makefile:1377: common] Error 2 > > > > make: *** Waiting for unfinished jobs > > > > > > What build target (and O= or not?) is this with? Thanks! > > > > make clean && make edison_defconfig && make -j16 > > > > gcc (Debian 8.2.0-4) 8.2.0 > > Copyright (C) 2018 Free Software Foundation, Inc. > > This is free software; see the source for copying conditions. There is NO > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > So, first time it's being built, consequent runs usually failed. More information: - when consequential run is being executed no code had been changed - the object files where it fails is totally arbitrary - I used ccache, though I disabled for now completely (through modifying PATH) and problem still exists - Linux kernel rebuilds on that very same machine fine (like touch include/linux/perf_event.h; make -j64 in a loop) It's total mystery to me. It was rebuilding just fine yesterday, I updated today from GH and even older (v2018.09-rc2) fails. -- With Best Regards, Andy Shevchenko ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko wrote: > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote: > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: > > > > > -- Forwarded message - > > > From: Andy Shevchenko > > > Date: Tue, Sep 4, 2018 at 3:40 PM > > > Subject: Parallel build is broken > > > To: Tom Rini > > > Cc: Bin Meng , Simon Glass > > > > > > > > > U-Boot can't be build with make -j16. > > > > > > Second pass passes fine. > > > > > > Didn't investigate myself and probably will not have time (near to > > > vacation time for me). > > > So, please, fix it before release. > > > > > > sh: echo: I/O error > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1 > > > make[1]: *** Waiting for unfinished jobs > > > CC drivers/serial/ns16550.o > > > CC drivers/spi/spi.o > > > CC cmd/mmc.o > > > CC drivers/watchdog/tangier_wdt.o > > > CC drivers/serial/serial_intel_mid.o > > > CC cmd/net.o > > > CC drivers/usb/common/common.o > > > CC arch/x86/lib/acpi.o > > > LD drivers/timer/built-in.o > > > CC arch/x86/lib/acpi_table.o > > > CC drivers/video/console_normal.o > > > make: *** [Makefile:1377: common] Error 2 > > > make: *** Waiting for unfinished jobs > > > > What build target (and O= or not?) is this with? Thanks! > > make clean && make edison_defconfig && make -j16 > > gcc (Debian 8.2.0-4) 8.2.0 > Copyright (C) 2018 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. So, first time it's being built, consequent runs usually failed. -- With Best Regards, Andy Shevchenko ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 4, 2018 at 5:00 PM Tom Rini wrote: > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: > > > -- Forwarded message - > > From: Andy Shevchenko > > Date: Tue, Sep 4, 2018 at 3:40 PM > > Subject: Parallel build is broken > > To: Tom Rini > > Cc: Bin Meng , Simon Glass > > > > > > U-Boot can't be build with make -j16. > > > > Second pass passes fine. > > > > Didn't investigate myself and probably will not have time (near to > > vacation time for me). > > So, please, fix it before release. > > > > sh: echo: I/O error > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1 > > make[1]: *** Waiting for unfinished jobs > > CC drivers/serial/ns16550.o > > CC drivers/spi/spi.o > > CC cmd/mmc.o > > CC drivers/watchdog/tangier_wdt.o > > CC drivers/serial/serial_intel_mid.o > > CC cmd/net.o > > CC drivers/usb/common/common.o > > CC arch/x86/lib/acpi.o > > LD drivers/timer/built-in.o > > CC arch/x86/lib/acpi_table.o > > CC drivers/video/console_normal.o > > make: *** [Makefile:1377: common] Error 2 > > make: *** Waiting for unfinished jobs > > What build target (and O= or not?) is this with? Thanks! make clean && make edison_defconfig && make -j16 gcc (Debian 8.2.0-4) 8.2.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > -- > Tom -- With Best Regards, Andy Shevchenko ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] Fwd: Parallel build is broken
On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote: > -- Forwarded message - > From: Andy Shevchenko > Date: Tue, Sep 4, 2018 at 3:40 PM > Subject: Parallel build is broken > To: Tom Rini > Cc: Bin Meng , Simon Glass > > > U-Boot can't be build with make -j16. > > Second pass passes fine. > > Didn't investigate myself and probably will not have time (near to > vacation time for me). > So, please, fix it before release. > > sh: echo: I/O error > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1 > make[1]: *** Waiting for unfinished jobs > CC drivers/serial/ns16550.o > CC drivers/spi/spi.o > CC cmd/mmc.o > CC drivers/watchdog/tangier_wdt.o > CC drivers/serial/serial_intel_mid.o > CC cmd/net.o > CC drivers/usb/common/common.o > CC arch/x86/lib/acpi.o > LD drivers/timer/built-in.o > CC arch/x86/lib/acpi_table.o > CC drivers/video/console_normal.o > make: *** [Makefile:1377: common] Error 2 > make: *** Waiting for unfinished jobs What build target (and O= or not?) is this with? Thanks! -- Tom signature.asc Description: PGP signature ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
[U-Boot] Fwd: Parallel build is broken
-- Forwarded message - From: Andy Shevchenko Date: Tue, Sep 4, 2018 at 3:40 PM Subject: Parallel build is broken To: Tom Rini Cc: Bin Meng , Simon Glass U-Boot can't be build with make -j16. Second pass passes fine. Didn't investigate myself and probably will not have time (near to vacation time for me). So, please, fix it before release. sh: echo: I/O error make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1 make[1]: *** Waiting for unfinished jobs CC drivers/serial/ns16550.o CC drivers/spi/spi.o CC cmd/mmc.o CC drivers/watchdog/tangier_wdt.o CC drivers/serial/serial_intel_mid.o CC cmd/net.o CC drivers/usb/common/common.o CC arch/x86/lib/acpi.o LD drivers/timer/built-in.o CC arch/x86/lib/acpi_table.o CC drivers/video/console_normal.o make: *** [Makefile:1377: common] Error 2 make: *** Waiting for unfinished jobs -- With Best Regards, Andy Shevchenko -- With Best Regards, Andy Shevchenko ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
