Re: [CFT] build: WITH_FAST_DEPEND and WITH_CCACHE_BUILD
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 12/1/15 5:53 PM, Bryan Drewery wrote: > On 11/12/2015 9:40 AM, Bryan Drewery wrote: >> Hi, >> >> Recently I have introduced two new features into the build. These >> apply to anything using /usr/share/mk including buildworld, >> buildkernel, universe, etc. >> >> - The first is WITH_FAST_DEPEND. Please see the commit for its >> full > > > FYI there is a known issue with FAST_DEPEND with all of the csu > directories and libgcov. Apparently I never thought to do a full > build with having .depend files but no objects. Just today some > thing changes to cause the .depend files for csu to rebuild hitting > the issue. I'll be addressing it soon. The workaround is 'make > cleandepend' before building anything that is failing with > 'multiple targets' errors. > > The issues I know about are all fixed after r291945. You may need to run 'make cleandepend' or do a build without -DNO_CLEAN if you run into "cc: error: cannot specify -o when generating multiple output files ". - -- Regards, Bryan Drewery -BEGIN PGP SIGNATURE- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJWZa+ZAAoJEDXXcbtuRpfPu+MIAKzhoNKoHfzdpAVz2lZGeH4Y tgh4iMt/zbAp8C4g6P4JpP5Yn4sdF/Kibuq6779z9gxYRQIk069YgeejoP9Ekp59 4k88lesY+6yHueuNN5oomRucvOXvHPVyuVTyNboFq7IiZB5wEt/iXyDQw41fDHGI hw8FFebKtvdFUtdz3Vz39wOKzhGtOGzy9sxvcnAgwo+FCR9Jx/LI8bMJbMWvVxjd N08188KJFwITwRfT41CeiAS3Cfx9soNUbwob/QZeavrosMlRQpio0cgJ72mMS07y kse5wHvO9VedpFkZ0nexIums5pmKr+ZSTFZ63TOrQN7nHs2Pb7laqUvg9+/En8c= =PgTt -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [CFT] build: WITH_FAST_DEPEND and WITH_CCACHE_BUILD
On 11/12/2015 9:40 AM, Bryan Drewery wrote: > Hi, > > Recently I have introduced two new features into the build. These apply > to anything using /usr/share/mk including buildworld, buildkernel, > universe, etc. > > - The first is WITH_FAST_DEPEND. Please see the commit for its full FYI there is a known issue with FAST_DEPEND with all of the csu directories and libgcov. Apparently I never thought to do a full build with having .depend files but no objects. Just today some thing changes to cause the .depend files for csu to rebuild hitting the issue. I'll be addressing it soon. The workaround is 'make cleandepend' before building anything that is failing with 'multiple targets' errors. -- Regards, Bryan Drewery signature.asc Description: OpenPGP digital signature
Re: [CFT] build: WITH_FAST_DEPEND and WITH_CCACHE_BUILD
On Thu, 12 Nov 2015 21:04:53 -0800 Craig Rodrigues wrote: > On Thu, Nov 12, 2015 at 9:40 AM, Bryan Drewery wrote: > > > - The first is WITH_FAST_DEPEND. Please see the commit for its full > > > > I have enabled WITH_FAST_DEPEND in make.conf for several builds. I have enabled it in src.conf ( head@r290570, GENERIC-NODEBUG kernel) On hw.model: Intel(R) Xeon(R) CPU E5504 @ 2.00GHz time make -j8 buildworld: 3685.33 real 25176.56 user 1918.05 sys WITH_FAST_DEPEND 3904.77 real 25833.18 user 2327.22 sys without > (1) Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD > Before WITH_FAST_DEPEND: > https://jenkins.freebsd.org/job/FreeBSD_HEAD/3488/1 hour, 23 minutes > After WITH_FAST_DEPEND: > https://jenkins.freebsd.org/job/FreeBSD_HEAD/3497/ , 1 hour, 17 minutes > > Compiler used: clang > > (2) Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/ > Before WITH_FAST_DEPEND: > https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/1309/ , 23 minutes > After WITH_FAST_DEPEND: > https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/1329/ , 22 minutes > > Compiler used: gcc 4.2.1 > > (3) Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/ >Before WITH_FAST_DEPEND: > https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/761/ , 1 hour, 7 > minutes >After WITH_FAST_DEPEND: > https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/762/ , 1 hour 2 > minutes > >Compiler: gcc 4.9.2 > > The timings are not very scientific, because the VM's in which the builds > are occurring are > very loaded, since each build host is running multiple VM's, which are > building different things. > > This gives you a basic idea though, that WITH_FAST_DEPEND works on a few > compilers (clang, gcc 4.2, gcc 4.9), > and there is some build time reduction, and no increase in build time. > > Congratulations for tackling this stuff in FreeBSD. As you mentioned, this > area has not been touched > in 14 years. > -- > Craig -- wbr, tiger ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [CFT] build: WITH_FAST_DEPEND and WITH_CCACHE_BUILD
On Thu, Nov 12, 2015 at 9:40 AM, Bryan Drewery wrote: > - The first is WITH_FAST_DEPEND. Please see the commit for its full > I have enabled WITH_FAST_DEPEND in make.conf for several builds. (1) Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD Before WITH_FAST_DEPEND: https://jenkins.freebsd.org/job/FreeBSD_HEAD/3488/1 hour, 23 minutes After WITH_FAST_DEPEND: https://jenkins.freebsd.org/job/FreeBSD_HEAD/3497/ , 1 hour, 17 minutes Compiler used: clang (2) Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/ Before WITH_FAST_DEPEND: https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/1309/ , 23 minutes After WITH_FAST_DEPEND: https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/1329/ , 22 minutes Compiler used: gcc 4.2.1 (3) Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/ Before WITH_FAST_DEPEND: https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/761/ , 1 hour, 7 minutes After WITH_FAST_DEPEND: https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/762/ , 1 hour 2 minutes Compiler: gcc 4.9.2 The timings are not very scientific, because the VM's in which the builds are occurring are very loaded, since each build host is running multiple VM's, which are building different things. This gives you a basic idea though, that WITH_FAST_DEPEND works on a few compilers (clang, gcc 4.2, gcc 4.9), and there is some build time reduction, and no increase in build time. Congratulations for tackling this stuff in FreeBSD. As you mentioned, this area has not been touched in 14 years. -- Craig ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [CFT] build: WITH_FAST_DEPEND and WITH_CCACHE_BUILD
On 11/12/2015 9:40 AM, Bryan Drewery wrote: > It is only useful for frequent builders who do not use -DNO_CLEAN and > want a reliable incremental build. I forgot to mention that I am also working on making buildworld reliably incremental without ccache. This will not be ready until the Spring probably though. -- Regards, Bryan Drewery signature.asc Description: OpenPGP digital signature
[CFT] build: WITH_FAST_DEPEND and WITH_CCACHE_BUILD
Hi, Recently I have introduced two new features into the build. These apply to anything using /usr/share/mk including buildworld, buildkernel, universe, etc. - The first is WITH_FAST_DEPEND. Please see the commit for its full description, benefits and discussion. It saves 16-30% in build times without gimmicks or risk. It avoids running the preprocessor twice in the build and only runs it during compilation. This is a feature that's 14 years overdue. https://svnweb.freebsd.org/changeset/base/290433 There was a problem with the GCC build, but that has been fixed. I intend to enable this *by default* for the src buildworld/etc... build in a few weeks. I do need to schedule a ports exp-run with this as well to see if anything outside of the src tree is broken by it, which I very much doubt. If you have a downstream fork of FreeBSD, please import r290433 and r290629 and give me feedback on any issues you encounter. There is 1 gotcha that I realized. People running 'make depend' manually may actually want to see a .depend file generated without having to compile first. I may add support for that somehow but am not sure yet. It may be a 'make mkdep' target. - The second is WITH_CCACHE_BUILD. This replaces the previous suggestion of modifying CC and CXX in /etc/make.conf. This is purposely chosen to match the ports name. It can save up to 65% build times when combined with WITH_FAST_DEPEND. This fixes all known issues with buildworld+ccache. There is a rare problem that can occur with header detection that is documented in the ccache manpage in the DIRECT MODE section. **It is only useful for frequent builders who do not use -DNO_CLEAN and want a reliable incremental build. It is not useful for people who build infrequently.** See commit for further details and stats. I do not intend to ever support enabling this by default. I just intend to update the devel/ccache pkg-message to suggest using it once it is known to work for all. https://svnweb.freebsd.org/changeset/base/290526 Thanks! Bryan Drewery signature.asc Description: OpenPGP digital signature
