Compile-Time autodifferentiation is something that I'm also very interested in and will add to my [deep learning compiler project Laser/Lux](https://github.com/numforge/laser/tree/master/laser/lux_compiler/core)
I suggest you have a look at my experiments on compile-time computation graphs at [https://github.com/mratsim/compute-graph-optim](https://github.com/mratsim/compute-graph-optim) I go through progressive step by step experiments to build a compile-time graph, macros start simple (no macros, pure generics) and grow very complicated. I've looked also in a couple of representations: object algebras and tagless typed encoding before settling on a combined deep embedding using ADTs + shallow embedding using function built on deep embedding, as suggested in [this paper](https://www.cs.ox.ac.uk/people/jeremy.gibbons/publications/embedding-short.pdf). My full research on [computation graphs representations](https://github.com/mratsim/Arraymancer/issues/347#issuecomment-461009747) is available here. Ultimately, I've implemented a proof-of-concept compiler in Nim macros using the [following tree representation](https://github.com/numforge/laser/blob/2f619fdbb2496aa7a5e5538035a8d42d88db8c10/laser/lux_compiler/core/lux_types.nim#L104-L132) which is very similar to Nim macros and that I've rewritten 3 times already. As I want both compile-time and JIT code generation in the future (including autodiff) and I want to reuse the data structure, my design a tiny bit more complex because I can't use generics/concepts at runtime. Instead I have my own AST and compile-time symbolic function evaluation that would mirror runtime function evaluation + codegen from the AST tree. A couple comments: You have taken the object algebra approach, this means that all operations will be encoded in the generic type system, this has the following limitations: * You can only use it for expressions, no for-loop (unless there is a clever way I'm not aware of) * generic symbol resolution is probably one of the slowest part of the compiler * it's much harder to work with types at compile-time than with values within macros Many computation graphs library in other languages uses the Visitor Pattern, you can't do that at compile-time in Nim as inheritance+methods don't work at compile-time. The preferred way (and the way Nim compiler and macros are implemented) is to use ADTs/object variants.
