just an idea regarding your string[TokenType.max + 1] in
immutable(string[TokenType.max + 1]) tokenValues = [..] it seems that you try to reduce the memory usagewouldn't it be a nice idea to generate a combined imutable string at compiletime like this one
"...pragmaexportpackageprivate..." and generated string slice accesses? imuteable string big_one = generated_from(toke_list); imutable string export_token = big_one[10..6];