This is a question as a category, but it's actually a report of what I've done. (If there are any mistakes, I would be very happy if you could specify them.) There are a few stories on github/Nim issues and forums about using ccache, but this will be a report of how much it specifically improves, measured in my environment, so it's something that advanced Nim users(or People who still use C language frequently) know about after all these years. I know this is a new and familiar story to advanced Nim users, **but I would like to introduce this as it has the potential to save a lot of people time and CPU energy.**
**Nim can make very good use of optimized peripheral tools such as gcc and clang.** This reduces build time when using ccache. <https://ccache.dev/> The following is an introduction to ccache. (Some excerpts) * Supports GCC, Clang, MSVC (Microsoft Visual C++) and other similar compilers[1]. (details) * Works on Linux, macOS, other Unix-like operating systems[1] and Windows[1]. (details) My annotations: Works with major environments and compilers supported by Nim. * Optionally uses file cloning (AKA “reflinks”) where possible to avoid copies. * Optionally uses hard links where possible to avoid copies. My annotations: Avoids copying files, thus improving TBW indicators such as SSD. and of course, speed. Another reason to use ccache is that the same cache is used for builds in different directories. If you have several versions or branches of a software stored in different directories, many of the object files in a build directory can probably be taken from the cache even if they were compiled for another version or branch. A third scenario is using ccache to speed up clean builds performed by servers or build farms that regularly check that the code is buildable. My annotations: Due to the deeply thought-out nature of nim, the nim file is translated into one (or more?) c source code, but the same modules used in different projects are not shared (Perhaps simply spec'd for a complete and full build). For example, lib/system.nim is compiled each time. Of course nimcache is cached for a project, but it is not shared between different projects.(by default). That R*** language is very slow to compile in that respect, and its deep dependence on llvm in the early stages (Recently, implementations have been created that also support gcc, but...still unstable & experimental) contrasts greatly with Nim's advantage when compared to Nim's compilation, which is very fast as is. ##### Here's how to use/install the actual ccache. # To measure time, aloso a simple time command would be fine. $ alias t="/usr/bin/time -f 'time: %E, memory: %MkB, exit: %x'" # The example below is the source for the Nim compiler itself, but it works equally well for large projects. # However, Even small sources can be expected to operate efficiently. # ubuntu/debian $ sudo apt-get install ccache # Other *nix #$ wget https://github.com/ccache/ccache/releases/download/v4.7.4/ccache-4.7.4-linux-x86_64.tar.xz #$ tar xvf ccache-4.7.4-linux-x86_64.tar.xz #$ cd ccache-4.7.4-linux-x86_64 #$ make install # /usr/local $ ccache -V ccache version 4.5.1 # Note that the latest is 2022-11-21: Version 4.7.4. (..snip..) ## Example: Compile Nim compiler source. Need to source and cd Nim $ t sh build_all.sh (..snip..) time: 1:37.76, memory: 637376kB, exit: 0 # koch build $ t ./koch boot -d:release time: 0:21.60, memory: 616468kB, exit: 0 ## Under using ccache & nim # clear nimcache's cache $ rm -rf nimcache # clear ccache's cache(Needed for the 2nd time) #$ ccache -c ## using 'ccache gcc' or 'ccache clang' with nim # If you install with apt on debian/ubuntu, a symbolic link to ccache such as gcc will be created in /lib/ccache, so you may prefer that path. $ export PATH=/lib/ccache:$PATH # Other *nix # If you get binaries with wget, make install will be installed in /usr/local/bin by default, so symbolically link to it. #$ mkdir -p ~/.local/bin # Use gcc-11 or clang, but specify the binary version you are using depending on your environment(gcc-10). #$ ln -s /usr/local/bin/ccache ~/.local/bin/gcc #$ ln -s /usr/local/bin/ccache ~/.local/bin/clang #$ export PATH=~/.local/bin:$PATH # 1st $ t sh build_all.sh (..snip..) time: 1:33.28, memory: 638376kB, exit: 0 # 2nd $ t sh build_all.sh (..snip..) time: 0:41.29, memory: 638084kB, exit: 0 # koch differential Build(nim only) $ t ./koch boot -d:release time: 0:22.04, memory: 615804kB, exit: 0 Run **The result of the difference build will not change, but the second full-build will complete more quickly compared to the first full-build. (However, the possibility remains that this may change depending on the environment.)** $ ccache -x # compression_level = 3, default compression_level: 0 (..snip..) Total data: 6.8 MB (7.8 MB disk blocks) Compressed data: 6.8 MB (27.6% of original size) Original size: 24.6 MB Compression ratio: 3.627 x (72.4% space savings) Incompressible data: 0.0 kB # View cache statistics $ ccache --show-stats Cacheable calls: 484 / 524 (92.37%) Hits: 240 / 484 (49.59%) Direct: 240 / 240 (100.0%) Preprocessed: 0 / 240 ( 0.00%) Misses: 244 / 484 (50.41%) Uncacheable calls: 40 / 524 ( 7.63%) Local storage: Cache size (GB): 0.01 / 2.00 ( 0.39%) Run Note: If you set export CC='ccache gcc' in the environment variable, this will cause inconvenience for C projects that are not Nim projects. According to the ccache documentation, some minor compilation options are not supported, so perhaps specifying unsupported options will produce identical object files(Compiler runs without cache). Very small projects and first-time builds are expected to be slightly slower. However, if you are working on multiple projects, the cache will be shared, so the total time across all projects, version upgrades, etc. will be reduced. You can turn off ccache in the configuration file if something goes wrong or if you really want a complete rebuild. $ ccache -o disable=true # Written to ccache.conf Run If used on a shared build server, additional configuration is required, such as using umask=002. See the documentation for details. <https://ccache.dev/manual/4.7.4.html#_sharing_a_local_cache> To use Visual C++ in MS-Windows, see below. <https://github.com/ccache/ccache/wiki/MS-Visual-Studio> Regarding the coordination of such tools, please do not issue this as a Bug to the main developers of Nim, it is very difficult or almost completely impossible to support all of them due to the wide variety of tools in gcc, clang and C language. Instead, it would be much more beneficial to spend time improving core features of the language, such as new gc/mm algorithms, gc algorithms for a wide variety of situations and memory pre-allocation for predictable cases, and implementing memory arenas. Occasional articles appear by Mr.Araq, but thread-shared caches with as few locks as possible, zero-cost exception implementations, etc., are beneficial to all. (I am not sure I understand what you have posted yet...) Disclaimer: I am neither a ccache developer nor an interested party of ccache. I am not responsible for any problems or other damages caused by the changes. **Sorry for the long phraseology and thin content of this post. I just hope it helps someone.**
