Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-02-04 Thread Enrico Scholz
Mark Hatle writes: >>> I've noticed that after 4 threads IO becomes a big bottleneck when >>> you have things like webkit, qt, asio etc in the buildqueue. Combine >>> that with issues like every make -j thread taking >2GB ram with asio >>> and webkit this default seems a bit high. I'd use 0.5*num

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-02-04 Thread Mark Hatle
On 2/4/14, 10:13 AM, Enrico Scholz wrote: Koen Kooi writes: +# Default to setting automatically based on cpu count +BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}" I've noticed that after 4 threads IO becomes a big bottleneck when you have things like webkit, qt, asio etc in the buildqueue.

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-02-04 Thread Enrico Scholz
Koen Kooi writes: >> +# Default to setting automatically based on cpu count >> +BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}" > > I've noticed that after 4 threads IO becomes a big bottleneck when you > have things like webkit, qt, asio etc in the buildqueue. Combine that > with issues like eve

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-02-03 Thread Richard Purdie
On Mon, 2014-02-03 at 22:23 +0100, Ulf Samuelsson wrote: > 1 feb 2014 kl. 10:21 skrev Mike Looijmans : > > > On 01/29/2014 01:56 PM, Richard Purdie wrote: > >> On Wed, 2014-01-29 at 13:09 +0100, Ulf Samuelsson wrote: > If bitbakes can track the loading of the CPU, then maybe some parts > in the la

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-02-03 Thread Ulf Samuelsson
1 feb 2014 kl. 10:21 skrev Mike Looijmans : > On 01/29/2014 01:56 PM, Richard Purdie wrote: >> On Wed, 2014-01-29 at 13:09 +0100, Ulf Samuelsson wrote: >>> We discussed this 2.3 months ago. >>> Did some studies on my dual hex-core machine (24 H/W treads) while >>> building a cloud9-gnome-image d

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-02-01 Thread Mike Looijmans
On 01/29/2014 01:56 PM, Richard Purdie wrote: On Wed, 2014-01-29 at 13:09 +0100, Ulf Samuelsson wrote: We discussed this 2.3 months ago. Did some studies on my dual hex-core machine (24 H/W treads) while building a cloud9-gnome-image derivative. This did about 7500 tasks. Enabled the CPU superv

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Laszlo Papp
Thank you for this contribution. As a side note, please consider the documentation bits in the future as well. :-) I have just opened a separate ticket for that here: https://bugzilla.yoctoproject.org/show_bug.cgi?id=5770 On Mon, Jan 27, 2014 at 2:39 PM, Richard Purdie wrote: > Its rather sad t

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Ross Burton
webkit, node are also not running in parallell in later part of the build  WebKit does parallel builds internally so what you’re seeing here is everything that doesn’t depend on webkit has finished, so the remaining tasks are blocking on the webkit build. Ross --  Ross Burton Open Source Techno

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Ulf Samuelsson
2014-01-29 13:56, Richard Purdie skrev: On Wed, 2014-01-29 at 13:09 +0100, Ulf Samuelsson wrote: We discussed this 2.3 months ago. Did some studies on my dual hex-core machine (24 H/W treads) while building a cloud9-gnome-image derivative. This did about 7500 tasks. Enabled the CPU supervisors

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Richard Purdie
On Wed, 2014-01-29 at 13:09 +0100, Ulf Samuelsson wrote: > We discussed this 2.3 months ago. > Did some studies on my dual hex-core machine (24 H/W treads) while > building a cloud9-gnome-image derivative. > This did about 7500 tasks. > > Enabled the CPU supervisors in the panel. > > Everything

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Steffen Sledz
On 29.01.2014 12:52, Paul Eggleton wrote: > On Wednesday 29 January 2014 12:47:54 Steffen Sledz wrote: >> On 29.01.2014 11:59, Richard Purdie wrote: >>> On Wed, 2014-01-29 at 11:14 +0100, Steffen Sledz wrote: On 27.01.2014 15:39, Richard Purdie wrote: > Its rather sad that people don't app

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Ulf Samuelsson
2014-01-28 11:08, Koen Kooi skrev: Op 27 jan. 2014, om 15:39 heeft Richard Purdie het volgende geschreven: Its rather sad that people don't appear to read local.conf and then complain about slow builds when they're just using a single thread. Most systems have more than one core now so we mig

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Paul Eggleton
On Wednesday 29 January 2014 12:47:54 Steffen Sledz wrote: > On 29.01.2014 11:59, Richard Purdie wrote: > > On Wed, 2014-01-29 at 11:14 +0100, Steffen Sledz wrote: > >> On 27.01.2014 15:39, Richard Purdie wrote: > >>> Its rather sad that people don't appear to read local.conf and then > >>> complai

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Steffen Sledz
On 29.01.2014 11:59, Richard Purdie wrote: > On Wed, 2014-01-29 at 11:14 +0100, Steffen Sledz wrote: >> On 27.01.2014 15:39, Richard Purdie wrote: >>> Its rather sad that people don't appear to read local.conf and then complain >>> about slow builds when they're just using a single thread. Most sys

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Steffen Sledz
On 29.01.2014 11:22, Paul Eggleton wrote: > On Wednesday 29 January 2014 11:14:47 Steffen Sledz wrote: >> On 27.01.2014 15:39, Richard Purdie wrote: >>> Its rather sad that people don't appear to read local.conf and then >>> complain about slow builds when they're just using a single thread. Most >

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Richard Purdie
On Wed, 2014-01-29 at 11:14 +0100, Steffen Sledz wrote: > On 27.01.2014 15:39, Richard Purdie wrote: > > Its rather sad that people don't appear to read local.conf and then complain > > about slow builds when they're just using a single thread. Most systems have > > more than one core now so we mig

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Paul Eggleton
On Wednesday 29 January 2014 11:14:47 Steffen Sledz wrote: > On 27.01.2014 15:39, Richard Purdie wrote: > > Its rather sad that people don't appear to read local.conf and then > > complain about slow builds when they're just using a single thread. Most > > systems have more than one core now so we

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-29 Thread Steffen Sledz
On 27.01.2014 15:39, Richard Purdie wrote: > Its rather sad that people don't appear to read local.conf and then complain > about slow builds when they're just using a single thread. Most systems have > more than one core now so we might as well use a more automatic default > for these values. This

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-28 Thread Richard Purdie
On Tue, 2014-01-28 at 11:08 +0100, Koen Kooi wrote: > Op 27 jan. 2014, om 15:39 heeft Richard Purdie > het volgende geschreven: > > > Its rather sad that people don't appear to read local.conf and then complain > > about slow builds when they're just using a single thread. Most systems have > >

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-28 Thread Koen Kooi
Op 27 jan. 2014, om 15:39 heeft Richard Purdie het volgende geschreven: > Its rather sad that people don't appear to read local.conf and then complain > about slow builds when they're just using a single thread. Most systems have > more than one core now so we might as well use a more automatic

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-27 Thread Stewart, David C
On 1/27/14, 8:45 AM, "Otavio Salvador" wrote: >On Mon, Jan 27, 2014 at 12:39 PM, Richard Purdie > wrote: >> Its rather sad that people don't appear to read local.conf and then >>complain >> about slow builds when they're just using a single thread. Most systems >>have >> more than one core now

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-27 Thread Burton, Ross
On 27 January 2014 17:23, Stewart, David C wrote: > Nice! Now if we could just convince people to configure their VM guests > with enough cpus - Now if only we could make bitbake error out when it's running inside a VM. ;) Ross ___ Openembedded-core ma

Re: [OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-27 Thread Otavio Salvador
On Mon, Jan 27, 2014 at 12:39 PM, Richard Purdie wrote: > Its rather sad that people don't appear to read local.conf and then complain > about slow builds when they're just using a single thread. Most systems have > more than one core now so we might as well use a more automatic default > for thes

[OE-core] [PATCH] local.conf.sample: Add automatic defaults for BB_NUMBER_THREADS and PARALLEL_MAKE

2014-01-27 Thread Richard Purdie
Its rather sad that people don't appear to read local.conf and then complain about slow builds when they're just using a single thread. Most systems have more than one core now so we might as well use a more automatic default for these values. This may lead to better experiences for new users. [YO