I've gone through a lot of your material now and I think it could be very valuable, thank you very much :D. But it is a complicated matter that I will need a bit of time to ponder on. There seems to be quite hard to find good examples of shallow vs deep embeddings. Shallow is using semantics and deep is using an AST, that's kinda what I've managed to get thus far. What would you say is the deep and shallow embeddings in Lux? The LuxNodes is deep as they are the AST but then it gets a bit fuzzier. I guess a proc as `transpose` would count as shallow as it is implemented using just the "language features" of the DSL. Operators as "+" and "*" feels especially hard to pinpoint. There's both the addition of two Funcs and also the addition between two LuxNodes. The first one being shallow and the second one deep. But at the same time the shallow version is just a wrapper for the deeper one acting on the LuxNode inside Func?
What I gathered from skimming through some of the linked papers is that as you said a solid core deep embedding (the smallest one to give the desired functionality basically) and then as much as possible is built as shallow embeddings. Two examples that could help me clear my mind is where mathematical functions (sin, exp, etc) and matrices: 1\. Mathematical functions such as exp and sin. We need the deep embedding to do simplifications like `exp(ln(x)) = x` but that limits us to the functions hardcoded in the code. Or perhaps we could use something like [this](https://github.com/mratsim/compute-graph-optim/blob/44ada700cdd95824d1702e9b843293d4aa86ccc4/e07_part1_var_ADTs_tables.nim#L15) to let the user define their own types along with a seq of simplification rules. Let's say someone created a library with a lot of functions this way for the symbolic library. Could the end-user then import the function-library and use the function-kinds defined there along with the default ones? Say someone writes a library with Bessel functions and they want to use them along with the sin and cos that comes by default. 2\. It feels like a symbolic matrix type should be doable just using shallow embeddings. Just a seq of SymbolicExpressions that uses the underlying operations available on the symbolic expressions. The "problem" with this approach is that it feels like it would be hard to generate Nim code at compileTime for this as the AST-builder doesn't know that it's a matrix it is working with. So perhaps it would need to be implemented in the deep embedding after all?
