Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-19 Thread Martin Jansa
On Fri, Aug 16, 2019 at 07:22:55PM +0200, Martin Jansa wrote: > On Fri, Aug 16, 2019 at 04:54:48PM +0100, richard.pur...@linuxfoundation.org > wrote: > > On Fri, 2019-08-16 at 17:00 +0200, Martin Jansa wrote: > > > > Will try to bump BB_NUMBER_THREADS from 8 to 72. > > > > > > I've tried to

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-16 Thread Martin Jansa
On Fri, Aug 16, 2019 at 04:54:48PM +0100, richard.pur...@linuxfoundation.org wrote: > On Fri, 2019-08-16 at 17:00 +0200, Martin Jansa wrote: > > > Will try to bump BB_NUMBER_THREADS from 8 to 72. > > > > I've tried to remove icecc.bbclass inherit (because it does things > > while parsing and RP

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-16 Thread richard . purdie
On Fri, 2019-08-16 at 17:00 +0200, Martin Jansa wrote: > > Will try to bump BB_NUMBER_THREADS from 8 to 72. > > I've tried to remove icecc.bbclass inherit (because it does things > while parsing and RP probably doesn't have it inherited), but that > didn't save much time. > > All 3 tests were

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-16 Thread Martin Jansa
> Will try to bump BB_NUMBER_THREADS from 8 to 72. I've tried to remove icecc.bbclass inherit (because it does things while parsing and RP probably doesn't have it inherited), but that didn't save much time. All 3 tests were with bitbake 18d4a31fdcec1f0e5d2199d6142f0ce833fca1a7 94m19.081s 8

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-16 Thread Martin Jansa
On Thu, Aug 15, 2019 at 05:05:48PM +0200, Martin Jansa wrote: > On Tue, Aug 13, 2019 at 10:04:08AM +0100, Richard Purdie wrote: > > On Mon, 2019-08-12 at 20:26 +, Peter Kjellerstedt wrote: > > > Comparing that build to a corresponding do-nothing build with Thud, > > > the time difference

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-15 Thread Richard Purdie
On Thu, 2019-08-15 at 14:56 +0100, richard.pur...@linuxfoundation.org wrote: > On Thu, 2019-08-15 at 14:56 +0200, Alexander Kanavin wrote: > What is really odd is this on both traces: > >524436 405.1020.001 405.2660.001 >

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-15 Thread Martin Jansa
On Tue, Aug 13, 2019 at 10:04:08AM +0100, Richard Purdie wrote: > On Mon, 2019-08-12 at 20:26 +, Peter Kjellerstedt wrote: > > Comparing that build to a corresponding do-nothing build with Thud, > > the time difference matches those three minutes where I have no idea > > what bitbake is doing

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-15 Thread richard . purdie
On Thu, 2019-08-15 at 14:56 +0200, Alexander Kanavin wrote: > On Wed, 14 Aug 2019 at 23:25, > wrote: > > Right, it still definitely needs work. Its a balancing act between > > sorting out the execution bugs in the code and figuring out the > > performance problem. > > > > If anyone wants to

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-15 Thread Alexander Kanavin
On Wed, 14 Aug 2019 at 23:25, wrote: > Right, it still definitely needs work. Its a balancing act between > sorting out the execution bugs in the code and figuring out the > performance problem. > > If anyone wants to experiment, the way I'd debug this is to run the > before and after cases with

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-14 Thread richard . purdie
On Wed, 2019-08-14 at 22:27 +0200, Alexander Kanavin wrote: > On Wed, 14 Aug 2019 at 14:55, > wrote: > > You followed up mentioning this wasn't with master-next. I think > > there > > is a patch in -next which will help with the empty task spin so > > both > > together might get us back to more

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-14 Thread Alexander Kanavin
On Wed, 14 Aug 2019 at 14:55, wrote: > You followed up mentioning this wasn't with master-next. I think there > is a patch in -next which will help with the empty task spin so both > together might get us back to more normal numbers. > As all of these patches are now in master, I re-ran the

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-14 Thread Khem Raj
On Wed, Aug 14, 2019 at 7:57 AM Peter Kjellerstedt wrote: > > > -Original Message- > > From: richard.pur...@linuxfoundation.org > > > > Sent: den 14 augusti 2019 14:56 > > To: Alexander Kanavin > > Cc: Peter Kjellerstedt ; Khem Raj > > ; OE-core > c...@lists.openembedded.org> > >

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-14 Thread Peter Kjellerstedt
> -Original Message- > From: richard.pur...@linuxfoundation.org > > Sent: den 14 augusti 2019 14:56 > To: Alexander Kanavin > Cc: Peter Kjellerstedt ; Khem Raj > ; OE-core c...@lists.openembedded.org> > Subject: Re: [OE-core] Long delays with latest bitbake (was: [PATCH > 1/7]

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-14 Thread richard . purdie
On Wed, 2019-08-14 at 14:08 +0200, Alexander Kanavin wrote: > On Wed, 14 Aug 2019 at 13:36, > wrote: > > On Wed, 2019-08-14 at 13:25 +0200, Alexander Kanavin wrote: > > > On Tue, 13 Aug 2019 at 21:18, Richard Purdie < > > > richard.pur...@linuxfoundation.org> wrote: > > > > I had a glance at the

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-14 Thread Mikko.Rapeli
On Wed, Aug 14, 2019 at 02:08:01PM +0200, Alexander Kanavin wrote: > On Wed, 14 Aug 2019 at 13:36, wrote: > > > On Wed, 2019-08-14 at 13:25 +0200, Alexander Kanavin wrote: > > > On Tue, 13 Aug 2019 at 21:18, Richard Purdie < > > > richard.pur...@linuxfoundation.org> wrote: > > > > I had a glance

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-14 Thread Alexander Kanavin
On Wed, 14 Aug 2019 at 14:08, Alexander Kanavin wrote: > > This might sound slightly crazy but can you try commenting out this >> line in runqueue.py: >> >> logger.debug(2, "Holding off tasks %s" % >> pprint.pformat(self.holdoff_tasks)) >> >> ? >> > > Even crazier is the outcome: it helped! The

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-14 Thread Alexander Kanavin
On Wed, 14 Aug 2019 at 13:36, wrote: > On Wed, 2019-08-14 at 13:25 +0200, Alexander Kanavin wrote: > > On Tue, 13 Aug 2019 at 21:18, Richard Purdie < > > richard.pur...@linuxfoundation.org> wrote: > > > I had a glance at the profile output from master-next and the > > > problem > > > wasn't

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-14 Thread richard . purdie
On Wed, 2019-08-14 at 13:25 +0200, Alexander Kanavin wrote: > On Tue, 13 Aug 2019 at 21:18, Richard Purdie < > richard.pur...@linuxfoundation.org> wrote: > > I had a glance at the profile output from master-next and the > > problem > > wasn't where I thought it would be, it was in the scheduler

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-14 Thread Alexander Kanavin
On Tue, 13 Aug 2019 at 21:18, Richard Purdie < richard.pur...@linuxfoundation.org> wrote: > I had a glance at the profile output from master-next and the problem > wasn't where I thought it would be, it was in the scheduler code. That > is good as those classes are effectively independent of the

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-13 Thread Khem Raj
On Tue, Aug 13, 2019 at 12:18 PM Richard Purdie wrote: > > On Tue, 2019-08-13 at 10:04 +0100, Richard Purdie wrote: > > On Mon, 2019-08-12 at 20:26 +, Peter Kjellerstedt wrote: > > > Comparing that build to a corresponding do-nothing build with Thud, > > > the time difference matches those

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-13 Thread Richard Purdie
On Tue, 2019-08-13 at 10:04 +0100, Richard Purdie wrote: > On Mon, 2019-08-12 at 20:26 +, Peter Kjellerstedt wrote: > > Comparing that build to a corresponding do-nothing build with Thud, > > the time difference matches those three minutes where I have no > > idea > > what bitbake is doing now

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-13 Thread Alexander Kanavin
On Tue, 13 Aug 2019 at 11:04, Richard Purdie < richard.pur...@linuxfoundation.org> wrote: > We talked on irc and you pointed at the commit things started to go > wrong. Just to summarise things for the benefit of the list, this is > some quick testing I did: > > "bitbake -p; time bitbake

Re: [OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

2019-08-13 Thread Richard Purdie
On Mon, 2019-08-12 at 20:26 +, Peter Kjellerstedt wrote: > Comparing that build to a corresponding do-nothing build with Thud, > the time difference matches those three minutes where I have no idea > what bitbake is doing now that it didn’t need to do before… > > Hopefully these time