Re: Ebnf Lexer and Parser generator in nim
As I read, Nim has seamless with any C libraries and code, so for the lexer, you can use Ragel, it produces readable and compact code with -G2 option (I use it on low-end microcontrollers for command parsing). The more interesting question is Nim able to do backtracking to implement DCG parsing for real complex context-sensitive and arbitrary syntaxes.
Re: Ebnf Lexer and Parser generator in nim
Hi @spip, Sorry, only noticed your post just now - for future communication feel free to post into the NPeg issues at github so I get properly notified. > It has its own grammar syntax for rules that does not follow (E)BNF like > Nim's pegs This is a design choice: having a grammar parseable by the Nim compiler has a number of advantages: * Reduced code size because there is no need to create a parser for the grammars itself. Of course this could be made in NPeg itself, but that's a bit of a chicken-and-egg problem :) * Implementation in Nim macros allows for smooth mixing of grammars and nim code, so there is no need for nasty tricks to get nim code back from string grammars etc * Not very important but still nice: syntax highlighting generally keeps working withing macro DSLs This said, it would probably be not too hard to create a (E)BNF compatible parser with everything that is now in place. I do see some problems with this though: (E)BNF and PEG grammers may _look_ similar, but are not trivially compatible (for example, ordered choice in PEGs). You simply can not parse any arbitrary (E)BNF grammar with a PEG, there are always things that need some reordering or rewriting to make them PEG compatible, or at least more efficient to limit backtracking. (On the other hand: the current syntax is not too far from (E)BNF. For example, take a look at src/npeg/lib/uri.nim for a PEG translation of RFC3984.) Also, the (E)BNF syntax would need a number of extensions in order to specify captures or other actions to perform at parse time, which kind of defies the purpose of having a compatible grammar to start with. Last but not least: I see no clean way to mix grammar and Nim code. I'm very much open to any ideas and experiments, so let me hear if you have any practical suggestions! > It does not support Unicode, meaning being able to parse UTF-8 strings using > Runes Like I said in the manual: there is rudimentary UTF-8 support available, and I'm not sure what exactly would be needed to make NPeg really "UTF-8 compatible". Over the last few days I added proper library support for NPeg, and started a bare minimum utf8 lib. The same applies here: I'd be glad to hear any ideas you might have and I'm happy to see if we can make NPeg suit your needs!
Re: Ebnf Lexer and Parser generator in nim
npeg has very nice features: * A complete documentation with clear code. * Peg generation at compile time. * AST captures to build an Abstract Syntax Tree while matching the grammar. * Nim code embedding in the rules. * Debugging and tracing functions, with the nice grammar tree view. Unfortunately, it has two _missing features_ that prevent me using instead of spending my time trying to debug my Nim's pegs grammar: * It has its own grammar syntax for rules that does not follow (E)BNF like Nim's pegs. People are somewhat more used to EBNF syntax, with postfix occurrence patterns, assumed sequences, etc. Existing rules would have to be rewritten and debugged to work with npeg. * It does not support Unicode, meaning being able to parse UTF-8 strings using Runes and Unicode-aware built-in macros, like pegs does, for instance with \white (=any Unicode whitespace character). If you were to make it API compatible with Nim's pegs, it would be a great replacement for the pegs module.
Re: Ebnf Lexer and Parser generator in nim
Add option and repeat in [https://github.com/loloiccl/nimly/pull/22](https://github.com/loloiccl/nimly/pull/22)
Re: Ebnf Lexer and Parser generator in nim
Ah please support "0 or more", it sucks to write simple loops as recursion.
Re: Ebnf Lexer and Parser generator in nim
nimly support only BNF now. For example, these are missing now.
* option ([else] in) : IF cond THEN exp [else]
* 0 or more ({param} in): PROC NAME RPAR {param} LPAR
But you can write BNF which equal to these.
I will support EBNF later
([https://github.com/loloiccl/nimly/issues/21](https://github.com/loloiccl/nimly/issues/21))
Re: Ebnf Lexer and Parser generator in nim
I'm taking the liberty to shamelessly mention my recent project here, as this seems the appropriate thread to do so: NPeg is a PEG-style parser generator which allows free mixing of grammar and Nim code, which should be suitable for the task of lexing and parsing. It can collect simple string captures, complex captures as a JSON tree, or run arbitrary Nim code at match time. NPeg is available in nimble, the manual and project page are at [https://github.com/zevv/npeg](https://github.com/zevv/npeg)
Re: Ebnf Lexer and Parser generator in nim
Wow, nimly looks nice. Don't have the time to try it. When you say "EBNF not supported" what exactly is missing?
Ebnf Lexer and Parser generator in nim
I made a BNF lexer/parser generator library ([https://github.com/loloiccl/nimly)](https://github.com/loloiccl/nimly\)). (unfortunately, EBNF is not supported now)
