On Friday, 11 December 2020 at 19:49:12 UTC, vnr wrote:
For a project with good performance, I would need to be able to analyse text. To do so, I would write a parser by hand using the recursive descent algorithm, based on a stream of tokens. I started writing a lexer with the d-lex package (https://code.dlang.org/packages/d-lex), it works really well, unfortunately, it's quite slow for the number of lines I'm aiming to analyse (I did a test, for a million lines, it lasted about 3 minutes). As the parser will only have to manipulate tokens, I think that the performance of the lexer will be more important to consider. Therefore, I wonder what resources there are, in D, for writing an efficient lexer.
Have you looked at Pegged [1]? It will give you the lexer and parser in one go. I'd be very interested to see how it performs on that kind of input.
-- Bastiaan. [1] https://code.dlang.org/packages/pegged
