On Wed, Nov 20, 2013 at 11:43:13PM +0100, Ulf Samuelsson wrote: > 2013-11-20 22:29, Richard Purdie skrev: > Another idea: > > I suspect that there is a lot of unpacking and patching of recipes > for the target when the native stuff is built. > Does it make sense to have multiple threads reading the disk, for > the target recipes during the native build or will we just lose out > due to seek time? > > Having multiple threads accessing the disk, might force the disk to spend > most of its time seeking. > Found an application which measures seek time performance, > and my WD Black will do 83 seeks per second, and my SAS disk will do > twice that. > The RAID of two SAS disks will provide close to SSD throughput (380 MB/s) > but seek time is no better than a single SAS disk. > > Since there is "empty time" at the end of the native build, does it make > sense > to minimize unpack/patch of target stuff when we reach that point, and > then we let loose?
In my benchmarks increasing PARALLEL_MAKE till number of cores was significantly improving build time, but BB_NUMBER_THREADS had minimal influence somewhere above 6 or 8 (tested on various systems, even only 4 was optimum on my older RAID-0 and 2 on single disk). Of course it was quite different for clean build without sstate prepopulated and build where most of the stuff was reused from sstate. see http://wiki.webos-ports.org/wiki/OE_benchmark > ======================== > > Now with 48 MB of RAM, (which I might grow to 96 GB, if someone proves that > this makes it faster), this might be useful to speed things up. > > Can tmpfs beat the kernel cache system? > > 1. Typically, I work on less than 10 recipes, and if I continuosly > rebuild those, why not create the build directories as links to > a tmpfs file system. > Maybe a configuration file with a list of recipes to build on > tmpfs. > > During a build from scratch, this is not so useful, but once > most stuff is in place, it might, > > 2. If the downloads directory was shadowed in a tmpfs system > then there would be less seek time during the build. > The downloads tmpfs should be poplulated at boot time, > and rsynced with a real disk in the background when new stuff > is downloaded from internet. > > 3. With 96 GB of RAM, maybe the complete build directory will fit. > Would be nice to build everything on tmpfs, and automatically rsync > to a real disk when there is nothing else to do... > > 4. If not tmpfs is used, then It would still be good to have better > control > over the build directory. > It make sense to me to have the metadata on an SSD, but the > build directory should be on my RAID cluster for fast rebuilds. > I can set this up manually, but it would be better to be able to > specify this in a configuration file. > See http://www.mail-archive.com/[email protected]/msg14879.html -- Martin 'JaMa' Jansa jabber: [email protected]
signature.asc
Description: Digital signature
_______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
