Re: [ptxdist] Build speed ups
Hi, cool send my mail as root ... sorry. > Depending on your PTXDIST version this is really easy to setup in > PTXDIST. Do you see the option SETUP_ICECC in ./ptxdist setup? > > Depending on the amount of workstations in your icecream network this is > a real boost! > Setup is easy if SETUP_ICECC is available in your PTXDIST version. 1.) Install packages something like: apt-get install icecc icecc-monitor 2.) Define a scheduler host in /etc/icecc/icecc.conf ICECC_SCHEDULER_HOST="YOUR-SCHEDULER-HOSTNAME-OR-IP" 3.) Restart daemon e.g. service iceccd restart 4.) Build with -ji option e.g. ./p targetinstall kernel -ji32 You can watch your build with icemon. Try something like: export USE_SCHEDULER=YOUR-SCHEDULER-HOSTNAME-OR-IP && icemon Regards Gavin (alias root) Eckelmann AG Vorstand: Dipl.-Ing. Peter Frankenbach (Sprecher) Dipl.-Wi.-Ing. Philipp Eckelmann Dr.-Ing. Marco Münchhof Dr.-Ing. Frank Uhlemann Vorsitzender des Aufsichtsrats: Hubertus G. Krossa Stv. Vorsitzender des Aufsichtsrats: Dr.-Ing. Gerd Eckelmann Sitz der Gesellschaft: Berliner Str. 161, 65205 Wiesbaden, Amtsgericht Wiesbaden HRB 12636 http://www.eckelmann.de ___ ptxdist mailing list ptxdist@pengutronix.de
Re: [ptxdist] Build speed ups
Hi there, On Mon, Sep 18, 2017 at 07:14:52PM +0200, Root wrote: > > What do you use to speed up your builds? > > do you know PTXCONF_PROJECT_CREATE_DEVPKGS and PTXCONF_PROJECT_USE_DEVPKGS? > You find this options in ptxdist menuconfig submenu Project name & version. > > This does not speed up the compile step, it skips the compile step and use > prebuild packages from an earlier build instead for all packages you > want to do this. quite recently we reviewed the corresponding section of the PTXdist documentation, see http://testbed.ptxdist.org/doc/daily_work_section.html#using-pre-built-archives for more information. Best regards Ulrich -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0| Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917- | ___ ptxdist mailing list ptxdist@pengutronix.de
Re: [ptxdist] Build speed ups
Hi, > What do you use to speed up your builds? > do you know PTXCONF_PROJECT_CREATE_DEVPKGS and PTXCONF_PROJECT_USE_DEVPKGS? You find this options in ptxdist menuconfig submenu Project name & version. This does not speed up the compile step, it skips the compile step and use prebuild packages from an earlier build instead for all packages you want to do this. > Distributed CC (distcc/icecc) seem a possibly to scale up things in > smaller steps (ad cheap compile server one by one) Sadly I don't have > any experience with it, but it is something I want to look into. > Depending on your PTXDIST version this is really easy to setup in PTXDIST. Do you see the option SETUP_ICECC in ./ptxdist setup? Depending on the amount of workstations in your icecream network this is a real boost! Regards Gavin ___ ptxdist mailing list ptxdist@pengutronix.de
Re: [ptxdist] Build speed ups
Hei hei, On Mon, Sep 18, 2017 at 02:33:50PM +0200, Erwin Rol wrote: > Distributed CC (distcc/icecc) seem a possibly to scale up things in > smaller steps (ad cheap compile server one by one) Sadly I don't have > any experience with it, but it is something I want to look into. We use icecc here. Before using it, I had some doubts and thought it would be complicated to setup, but it wasn't, also thanks to the transparent integration in ptxdist. You can easily integrate machines already running in your local network. Definitly something I would give a try before buying new hardware. Greets Alex -- »With the first link, the chain is forged. The first speech censured, the first thought forbidden, the first freedom denied, chains us all irrevocably.« (Jean-Luc Picard, quoting Judge Aaron Satie) *** GnuPG-FP: C28E E6B9 0263 95CF 8FAF 08FA 34AD CD00 7221 5CC6 *** signature.asc Description: PGP signature ___ ptxdist mailing list ptxdist@pengutronix.de
[ptxdist] Build speed ups
Hey All, first thanks for the replies on my Yocto mail. Now I have a real PTXdist question; What do you use to speed up your builds? For example I use webkitgtk and that clips my 24 core Xeon at 100% for like 30 minutes. ccache doesn't really help by a first run, it even makes it worse. Also I must build several versions and so the ccache hast to be massive or else the cache is always just updating and before you can use it it is flushed from it already. So get faster hardware? Of course one could get a >100k euro multi CPU system, but I don't think that is the most economic choice. Distributed CC (distcc/icecc) seem a possibly to scale up things in smaller steps (ad cheap compile server one by one) Sadly I don't have any experience with it, but it is something I want to look into. PTXDist's -j functionality I can not use for all my builds, for example I am stuck on kernel 3.1.10 (thanks nVidia), which means I am stuck on a older systemd and can't use gcc 5.4 etc. So I am stuck at a rather old ptxdist, and never got to try the -j stuff. (But with the webkitgtk example it won't help much, the CPU is at 100% already) So what do ppl do to make things faster ? I would really like to hear other ppls experiences with large builds. - Erwin ___ ptxdist mailing list ptxdist@pengutronix.de
