Re: dmdcache
On Saturday, 25 April 2020 at 16:01:03 UTC, bauss wrote: There is no way to determine the import statements either as those themselves can be generated at compile-time. Question: is then DMD capable to add generated source code in separate files to the compilation process when we can just import code with arbitrary string paths?
Re: dmdcache
On Saturday, 25 April 2020 at 12:30:17 UTC, John Colvin wrote: But how does this differ from just using make? make doesn't support total-dependency-content-digest keying of cached artifacts similar to what, for instance, Bazel does.
Re: dmdcache
On 4/25/20 5:30 AM, John Colvin wrote: > how does this differ from just using make? make is great and I love it (really) but it works at a coarser level. There is no way for it to know that a particular command will produce the same output. As I understand it, dmdcache is supposed to be similar to ccache, which we already use with make to speed up our C++ compilations: https://ccache.dev/ Ali
Re: dmdcache
On 27/04/2020 1:52 AM, Ali Çehreli wrote: However, currently dmdcache doesn't seem to look for string imports; opportunity for improvement. :) That should not be necessary if you base it on -J instead.
Re: dmdcache
On 4/25/20 9:01 AM, bauss wrote: > On Saturday, 25 April 2020 at 10:35:49 UTC, Stefan Koch wrote: >> The main problem with this is that it does not take string imports >> into account [...] > Yeah, doesn't look like it which means it might not be useful in > projects that does a lot of compile-time stuff. > > There is no way to determine the import statements either as those > themselves can be generated at compile-time. Luckily, dmd gives all of that information with both -v and -X command line switches. However, currently dmdcache doesn't seem to look for string imports; opportunity for improvement. :) Ali
Re: dmdcache
On 4/25/20 4:39 AM, Johan wrote: On Saturday, 25 April 2020 at 10:17:50 UTC, Ali Çehreli wrote: A colleague of mine has written dmdcache which may be very useful for some projects: https://github.com/seeraven/dmdcache It drops our build time from 8 minutes to 45 seconds Hey Ali, Did you also try with LDC's built-in object file caching (asm codegen caching) ? Cheers, Johan That sounds very promising. Time to introduce ldc to the project. Ali
Re: dmdcache
On Saturday, 25 April 2020 at 10:35:49 UTC, Stefan Koch wrote: On Saturday, 25 April 2020 at 10:17:50 UTC, Ali Çehreli wrote: A colleague of mine has written dmdcache which may be very useful for some projects: https://github.com/seeraven/dmdcache It drops our build time from 8 minutes to 45 seconds on a particular build environment for about half a dozen D programs, one of which ends up being a 2G executable! WAT! :) And the total cache size is 5.5G. Wow! This build is with dmd 2.084.1 and that one particular application uses tons of template instantiations most of which are in generated source code. If I remember correctly, 2.084.1 does not contain template symbol name improvements and that may be the reason for the large size. Enjoy! Ali The main problem with this is that it does not take string imports into account, (or does it ???, I don't see how it could ) Also the compilers output can depend on the timestamp at which the compilation was done. Yeah, doesn't look like it which means it might not be useful in projects that does a lot of compile-time stuff. There is no way to determine the import statements either as those themselves can be generated at compile-time. I can see this being useful for large projects that does not use CTFE but other than that I think it might just create subtle bugs because you won't immediately know that some part of your code didn't update and isn't working as intended because the code for it was imported from another file etc.
Re: dmdcache
On Saturday, 25 April 2020 at 10:17:50 UTC, Ali Çehreli wrote: A colleague of mine has written dmdcache which may be very useful for some projects: https://github.com/seeraven/dmdcache It drops our build time from 8 minutes to 45 seconds on a particular build environment for about half a dozen D programs, one of which ends up being a 2G executable! WAT! :) And the total cache size is 5.5G. Wow! This build is with dmd 2.084.1 and that one particular application uses tons of template instantiations most of which are in generated source code. If I remember correctly, 2.084.1 does not contain template symbol name improvements and that may be the reason for the large size. Enjoy! Ali Perhaps I'm being very dumb, but how does this differ from just using make?
Re: dmdcache
On Saturday, 25 April 2020 at 10:17:50 UTC, Ali Çehreli wrote: A colleague of mine has written dmdcache which may be very useful for some projects: https://github.com/seeraven/dmdcache It drops our build time from 8 minutes to 45 seconds Hey Ali, Did you also try with LDC's built-in object file caching (asm codegen caching) ? Cheers, Johan
Re: dmdcache
On Saturday, 25 April 2020 at 10:17:50 UTC, Ali Çehreli wrote: A colleague of mine has written dmdcache which may be very useful for some projects: https://github.com/seeraven/dmdcache It drops our build time from 8 minutes to 45 seconds on a particular build environment for about half a dozen D programs, one of which ends up being a 2G executable! WAT! :) And the total cache size is 5.5G. Wow! This build is with dmd 2.084.1 and that one particular application uses tons of template instantiations most of which are in generated source code. If I remember correctly, 2.084.1 does not contain template symbol name improvements and that may be the reason for the large size. Enjoy! Ali The main problem with this is that it does not take string imports into account, (or does it ???, I don't see how it could ) Also the compilers output can depend on the timestamp at which the compilation was done.
dmdcache
A colleague of mine has written dmdcache which may be very useful for some projects: https://github.com/seeraven/dmdcache It drops our build time from 8 minutes to 45 seconds on a particular build environment for about half a dozen D programs, one of which ends up being a 2G executable! WAT! :) And the total cache size is 5.5G. Wow! This build is with dmd 2.084.1 and that one particular application uses tons of template instantiations most of which are in generated source code. If I remember correctly, 2.084.1 does not contain template symbol name improvements and that may be the reason for the large size. Enjoy! Ali
