On Wednesday, 4 June 2014 at 21:12:25 UTC, Brian Schott wrote:
I've been looking at ways to optimize the D lexer's operation using SIMD instructions. I'm not yet sure if I'll need to change the lexer generator's API to do this. I'm going to wait until I have my proof-of-concept code and some benchmark results before asking for a voting thread or creating a pull request.
Implementing some SIMD code just in the lexWhitespace function causes a drop in total lexing time of roughly 3.7%. This looks promising so far, so I'm going to implement similar code in lexStringLiteral, lexSlashStarComment, lexSlashSlashComment, and lexSlashPlusComment.
So far this only uses SSE2 instructions, so all 64-bit x86 processors can benefit from this. (SSE4.2 would allow us to do more interesting things, but it's not as common)
