Hello, I would like to ask what is the current state of incremental compilation? Was it implemented while I was away ;-)
Looking at github issues, it seems it is partially implemented? I asked 'Claude.ai' about further optimizations inline with nim's design philosophy (that is, not too crazy stuff that complicates the entire codebase stuff), I simply pasted the github issue from: <https://github.com/nim-lang/RFCs/issues/46> i am not such a seasoned programmer to know about nim internals, and not sure if the suggestions are coherent or nonsensical/naive. Perhaps araq would be willing to reply. The idea is to see far and to be aware of more points while writing the design and implementation. There isn't a bit of criticism here, I hope I am received in the correct tone. Here are the suggestions after some filtering. * Finer-grained dependency tracking: Tracking dependencies at the level of individual top-level declarations (functions, types, etc.) instead of just at the module level. This would allow for more targeted recompilation when only a small part of a module changes. * Smarter caching of generic instantiations: Caching generic instantiations separately from the generic definition itself and tracking the dependencies of each instantiation. This would enable more efficient recompilation when only certain instantiations are affected by a change. * Incremental type checking and semantic analysis: Extending incremental computation to other stages of compilation, such as type checking and semantic analysis. By caching and reusing the results of these stages when possible, the compiler could avoid redundant work. * Performing dead code elimination (DCE) in the Nim compiler: Implementing a DCE pass in the Nim compiler itself, before generating the C++ code. This would remove unused code early in the compilation process, reducing the amount of code that needs to be scanned, parsed, and cached during incremental compilation. By eliminating dead code upfront, the compiler can focus its incremental efforts on the code that actually matters. * Adaptive optimization based on previous builds: Using information from previous builds to guide optimization decisions, such as prioritizing the incremental compilation of frequently changing modules or optimizing code paths that are known to be performance-critical. (??) * Multithreading the compiler process: Parallelizing the compilation process to take advantage of multiple CPU cores. This could involve partitioning the work at the module level or even at finer granularities, such as individual functions or declarations. By processing multiple parts of the program concurrently, the overall compilation time could be significantly reduced. * Pipelining and overlapping of compilation stages: Arranging the compilation process as a pipeline of stages (parsing, type checking, code generation, optimization, etc.) and allowing these stages to overlap and run concurrently where possible. This would help maximize CPU utilization and reduce idle time, further speeding up the compilation process.
