Re: The cost of doing compile time introspection

[Moritz Maxeiner via Digitalmars-d]

On Thursday, 11 May 2017 at 21:09:05 UTC, Timon Gehr wrote:
[...]
Yes, this works and is a few times faster.
It's slightly faster when inlining the condition:

static foreach(fn;__traits(allMembers, functions)){
    static if (isFunction!(__traits(getMember, functions, fn)) 
&&
               (functionLinkage!(__traits(getMember, functions, 
fn)) == "C" ||
                functionLinkage!(__traits(getMember, functions, 
fn)) == "Windows")){
        mixin("typeof(functions."~fn~")* "~fn~";");
    }
}

With the DMD debug build, I measured the following times on my 
machine:

Baselines:

just imports:
0m0.318s

copy-pasted generated code after printing it with pragma(msg, 
...):
0m0.341s

Compile-time code generation:

old version:
0m2.569s

static foreach, uninlined:
0m0.704s

static foreach inlined:
0m0.610s

Still not great, but a notable improvement.


isFunction and functionLinkage are slow, so I got rid of them 
(as well as the dependency on std.traits):

static foreach(fn;__traits(allMembers, functions)){
    static if(fn != "object" && fn != "llvm" && fn != 
"orEmpty"):
    mixin("typeof(functions."~fn~")* "~fn~";");
}

timing:
0m0.350s

(This is not perfect as you'll need to edit the list in case 
you are adding more non-c-function members to that module, but 
I guess it is a good trade-off.)



You can achieve essentially the same using a string mixin:

mixin({
    string r;
    foreach(fn;__traits(allMembers, functions))
        if(fn != "object" && fn != "llvm" && fn != "orEmpty")
            r~="typeof(functions."~fn~")* "~fn~";";
    return r;
}());

timing:
0m0.370s



In case the original semantics should be preserved, I think 
this is the best option:

mixin({
    string r;
    foreach(fn;CFunctions!functions)
        r~="typeof(functions."~fn~")* "~fn~";";
    return r;
}());

timing:
0m0.740s

Thank you for the detailed comparison. I have applied your 
optimizations (with minor refactoring that did not impact compile 
time for me) and ended up with this (sorry for some name changes, 
wasn't happy with my original ones):

---
import link = llvm.functions.link;

bool isSym(string m) { return m != "object" && m != "llvm" && m 
!= "orEmpty"; }

string declareSymPtr(string m) { return "typeof(link." ~ m ~ ")* 
" ~ m ~ ";"; }
string getSymPtr(string m) { return m ~ " = 
library.getSymbol!(typeof(" ~ m ~ "))(\"" ~ m ~ "\");"; }

mixin ({
        string code;
        foreach (m; __traits(allMembers, link)) if (m.isSym) {
            code ~= m.declareSymPtr;
        }
        return code;
}());

public struct LLVM
{
    static void getSymbols()
    {
        foreach (m; __traits(allMembers, link)) static if 
(m.isSym) {
            mixin (m.getSymPtr);
        }
    }
}
---

I am not particularly happy about (isSym) having to do a name 
based blacklist approach instead of a type based whitelist 
approach, though.
With this I'm at least out of the "OMG why is it still compiling" 
range and thank you to everyone for that. It's still not in the 
ideal range of < 100 milliseconds, but I'll take what I can get.