Re: [OE-core] Improving Build Speed

2013-11-23 Thread Ulf Samuelsson
2013-11-21 14:53, Richard Purdie skrev: On Thu, 2013-11-21 at 09:04 +0100, Ulf Samuelsson wrote: Why restrict PARALLEL_MAKE to anything less than the number of H/W threads in the machine? Came up with a construct PARALLEL_HIGH which is defined alongside PARALLEL_MAKE in

Re: [OE-core] Improving Build Speed

2013-11-23 Thread Nicolas Dechesne
On Thu, Nov 21, 2013 at 8:15 AM, Ulf Samuelsson u...@emagii.com wrote: I currently do not use INHERIT += rm_work since I want to be able to do changes on some packages. Is there a way to defined rm_work on a package basis? Then the majority of the packages can be removed. from

Re: [OE-core] Improving Build Speed

2013-11-21 Thread Ulf Samuelsson
Why restrict PARALLEL_MAKE to anything less than the number of H/W threads in the machine? Came up with a construct PARALLEL_HIGH which is defined alongside PARALLEL_MAKE in conf/local.conf PARALLEL_MAKE = -j8 PARALLEL_HIGH = -j24 In the appropriate recipes,

Re: [OE-core] Improving Build Speed

2013-11-21 Thread Burton, Ross
On 20 November 2013 21:05, Ulf Samuelsson angstrom-...@emagii.com wrote: do_compile() { if [ x$MAKE = x ]; then MAKE=make; fi ... for error_count in 1 2 3; do ... ${MAKE} ${EXTRA_OEMAKE} $@ || exit_code=1 ... done ... } Not sure, but I

Re: [OE-core] Improving Build Speed

2013-11-21 Thread Enrico Scholz
Ulf Samuelsson angstrom-dev-aofpy8dbyrpqt0dzr+a...@public.gmane.org writes: PARALLEL_MAKE = -j6 BB_NUMBER_THREADS = 24 I define | PARALLEL_MAKE = \ | -j ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 2} \ | -l ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) *

Re: [OE-core] Improving Build Speed

2013-11-21 Thread Martin Jansa
On Thu, Nov 21, 2013 at 08:15:08AM +0100, Ulf Samuelsson wrote: 2013-11-21 01:19, Martin Jansa skrev: On Wed, Nov 20, 2013 at 11:43:13PM +0100, Ulf Samuelsson wrote: 2013-11-20 22:29, Richard Purdie skrev: Another idea: I suspect that there is a lot of unpacking and patching of recipes

[OE-core] Improving Build Speed

2013-11-20 Thread Ulf Samuelsson
Finally got my new build machine running. so I thought I'd measure the performance vs the old machine Home Built Core i7-980X 6 core/12 threads @ 3,33GHz 12 GB RAM @ 1333 Mhz. WD Black 1 TB @ 7200 rpm Precision 7500 2 x (X5670 6 core 2,93 MHz) 2 x (24 GB RAM @ 1333 MHz)

Re: [OE-core] Improving Build Speed

2013-11-20 Thread Richard Purdie
Hi Ulf, Nice to see someone else looking at this. I've shared some of my thoughts and observations below based on some of the work I've done trying to speed things up. On Wed, 2013-11-20 at 22:05 +0100, Ulf Samuelsson wrote: Finally got my new build machine running. so I thought I'd measure

Re: [OE-core] Improving Build Speed

2013-11-20 Thread Ulf Samuelsson
2013-11-20 22:29, Richard Purdie skrev: Hi Ulf, Nice to see someone else looking at this. I've shared some of my thoughts and observations below based on some of the work I've done trying to speed things up. On Wed, 2013-11-20 at 22:05 +0100, Ulf Samuelsson wrote: Finally got my new build

Re: [OE-core] Improving Build Speed

2013-11-20 Thread Martin Jansa
On Wed, Nov 20, 2013 at 09:29:16PM +, Richard Purdie wrote: Hi Ulf, (3) Creating the rootfs seems to have zero parallelism. But I have not investigated if anything can be done. This is something I do want to fix in 1.6. We need to convert the core to python to gain access to

Re: [OE-core] Improving Build Speed

2013-11-20 Thread Martin Jansa
On Wed, Nov 20, 2013 at 11:43:13PM +0100, Ulf Samuelsson wrote: 2013-11-20 22:29, Richard Purdie skrev: Another idea: I suspect that there is a lot of unpacking and patching of recipes for the target when the native stuff is built. Does it make sense to have multiple threads reading the

Re: [OE-core] Improving Build Speed

2013-11-20 Thread Ulf Samuelsson
2013-11-21 01:19, Martin Jansa skrev: On Wed, Nov 20, 2013 at 11:43:13PM +0100, Ulf Samuelsson wrote: 2013-11-20 22:29, Richard Purdie skrev: Another idea: I suspect that there is a lot of unpacking and patching of recipes for the target when the native stuff is built. Does it make sense to