Re: Parallel make bug in busybox

2016-08-22 Thread Denys Vlasenko
On Mon, Aug 22, 2016 at 9:09 AM, Richard Purdie wrote: > On Sun, 2016-08-21 at 22:03 +0200, Denys Vlasenko wrote: >> On Fri, Aug 19, 2016 at 3:47 PM, Richard Purdie >> wrote: >> > Index: busybox-1.24.1/applets/applet_tables.c >> >

Re: Parallel make bug in busybox

2016-08-22 Thread Richard Purdie
On Sun, 2016-08-21 at 22:03 +0200, Denys Vlasenko wrote: > On Fri, Aug 19, 2016 at 3:47 PM, Richard Purdie > wrote: > > Index: busybox-1.24.1/applets/applet_tables.c > > === > > ---

Re: Parallel make bug in busybox

2016-08-19 Thread Jacob Kroon
As someone who has had similar problems before I'll give 2 cents.. On Fri, Aug 19, 2016 at 1:56 PM, Richard Purdie wrote: > I think the recent "fix" actually made things worse. Whenever I run "rm > include/applet_tables.h include/NUM_APPLETS.h; make clean; make -j 48", > I see

Re: Parallel make bug in busybox

2016-08-19 Thread Richard Purdie
I think I have got to the bottom of the problem. applet_tables only rewrites NUM_APPLETS.h if the file changes. This leads to make repeatedly trying to rewrite it since the compiled binary itself can be newer that the header, amongst other issues. There is also the issue I mentioned previously,

Re: Parallel make bug in busybox

2016-08-19 Thread Richard Purdie
I think the recent "fix" actually made things worse. Whenever I run "rm include/applet_tables.h include/NUM_APPLETS.h; make clean; make -j 48", I see the applet_tables command running twice in parallel. There are two issues: a) There is nothing stopping make running the appliet_tables command

Re: Parallel make bug in busybox

2016-08-18 Thread Richard Purdie
On Wed, 2016-08-17 at 16:16 +0100, Richard Purdie wrote: > I think this is the real underlying problem which broke our build. I > have backported Denys' fix into our builds but I worry there is > another > bug here which I didn't find a reproducer for as detailed above. Is > there a way we could

Re: Parallel make bug in busybox

2016-08-17 Thread Richard Purdie
On Mon, 2016-08-15 at 08:17 +0200, Peter Korsgaard wrote: > > > > > > "Richard" == Richard Purdie writes: > > Hi, > > Sorry for the slow response, I've been away during the weekend. I see > Denys has committed a fix, so that's great! > > >>

Re: Parallel make bug in busybox

2016-08-17 Thread Richard Purdie
On Sun, 2016-08-14 at 01:53 +0200, Arnout Vandecappelle wrote: > The debug output of make is not easy to parse... The -p output comes > _after_ > the -d output, so the rule for NUM_APPLETS.h you have above is not > the one that > is actually used. Indeed, there is a recursive make with > > make

Re: Parallel make bug in busybox

2016-08-15 Thread Peter Korsgaard
> "Richard" == Richard Purdie writes: Hi, Sorry for the slow response, I've been away during the weekend. I see Denys has committed a fix, so that's great! >> http://autobuild.buildroot.org/?reason=busybox-1.25.0 >> http://autobuild.buildroot.org/?reason=busybox-1.24.1

Re: Parallel make bug in busybox

2016-08-13 Thread Denys Vlasenko
On Sun, Aug 14, 2016 at 2:29 AM, Ralf Friedl wrote: > Arnout Vandecappelle wrote: >> >> As an aside, there may be a second issue: applets/Kbuild.src specifies the >> same >> command for both include/applet_tables.h and include/NUM_APPLETS.h, but >> that >> command will

Re: Parallel make bug in busybox

2016-08-13 Thread Ralf Friedl
Arnout Vandecappelle wrote: As an aside, there may be a second issue: applets/Kbuild.src specifies the same command for both include/applet_tables.h and include/NUM_APPLETS.h, but that command will always generate both files. So if the rules for the two files are launched in parallel, there is

Re: Parallel make bug in busybox

2016-08-13 Thread Arnout Vandecappelle
On 13-08-16 22:29, Denys Vlasenko wrote: > On Fri, Aug 12, 2016 at 8:53 AM, Richard Purdie wrote: >> We (as in the Yocto Project/OpenEmbedded) do a lot of builds of busybox >> One way I did find of breaking the build was: >> >> make; rm libbb/appletlib.o

Re: Parallel make bug in busybox

2016-08-13 Thread Jeremy Chadwick
On Sat, Aug 13, 2016 at 10:29:38PM +0200, Denys Vlasenko wrote: > On Fri, Aug 12, 2016 at 8:53 AM, Richard Purdie wrote: > > We (as in the Yocto Project/OpenEmbedded) do a lot of builds of busybox > > One way I did find of breaking the build was: > > > > make; rm

Re: Parallel make bug in busybox

2016-08-13 Thread Denys Vlasenko
On Fri, Aug 12, 2016 at 8:53 AM, Richard Purdie wrote: > We (as in the Yocto Project/OpenEmbedded) do a lot of builds of busybox > One way I did find of breaking the build was: > > make; rm libbb/appletlib.o include/NUM_APPLETS.h; make > > which results in: > > CC

Re: Parallel make bug in busybox

2016-08-12 Thread Richard Purdie
On Fri, 2016-08-12 at 16:53 +0200, Peter Korsgaard wrote: > > > > > > "Richard" == Richard Purdie writes: > > > We (as in the Yocto Project/OpenEmbedded) do a lot of builds of > busybox > > and one of the failures we've seen recently was: > > >

Re: Parallel make bug in busybox

2016-08-12 Thread Peter Korsgaard
> "Richard" == Richard Purdie writes: > We (as in the Yocto Project/OpenEmbedded) do a lot of builds of busybox > and one of the failures we've seen recently was: > https://autobuilder.yoctoproject.org/main/builders/poky-tiny/builds/908 > /steps/BuildImages/logs/stdio

Parallel make bug in busybox

2016-08-12 Thread Richard Purdie
We (as in the Yocto Project/OpenEmbedded) do a lot of builds of busybox and one of the failures we've seen recently was: https://autobuilder.yoctoproject.org/main/builders/poky-tiny/builds/908 /steps/BuildImages/logs/stdio It basically shows multiple executions of: "make -f