Did this activity lead to somewhere? ?For example, do you have a
recommendation for the lexer/parser library to use, if we only want to
handle the expression syntax with it (so starting a new lexing/parsing
should have low overhead)?
Tuesday, August 7, 2018, 12:59:14 AM, Daniel Dekany wrote:
> Sunday, August 5, 2018, 6:58:11 PM, Stephan Müller wrote:
>
>> Am 04.07.2018 um 19:28 schrieb Daniel Dekany:
>>> I wonder what parser libraries could help us, in FM3, to separate the
>>> expression language parsing from the top-level language (like
>>> `<#foo>`, `${...}`, etc.) parsing. Or if a hand written parsers is an
>>> acceptable compromise. It would be good if we can change the top-level
>>> syntax and still reuse the expression syntax. (Or, replace the
>>> expression syntax, and reuse the top-level one.) Like, somebody wants
>>> a syntax like `#foo(exp)` instead of `<#foo exp>`, but still reuse the
>>> expression syntax. (For me it was always part of the FM3 agenda,
>>> though might will be proven to be too much...)
>>> [..]
>>
>> During the last days I had a high-level look at different parser
>> generators, and as one might imagine, there are a lot of parser
>> generators, with different licenses, different maturities, different
>> states of maintenance and so on.
>>
>> Due to https://www.apache.org/legal/resolved.html I ignored all parser
>> generators which may not be included in Apache projects because of their
>> license, especially GNU GPL etc.
>>
>> IMHO this leaves us with:
>>
>> * LL(k) parsers: ANTLR, JavaCC and Grammatica
>> * LALR parsers: CookCC
>> * PEG parsers: Mouse
>> * parser combinators: jparsec, parboiled and PetitParser
>>
>> This list is not exhaustive, so I probably forget some interesting
>> projects. If so, please share, I'd like to have a look into these, too.
>>
>> My idea for the next step: define a really small subset of FTL and try
>> to implement PoCs for this subset with the candidates which I mentioned
>> above.
>>
>> The subset might be something like
>>
>> * interpolations: ${..}
>> * directives: if, assign
>
> Just to be on the safe side, I will note that you shouldn't try to
> hard-code parser logic that's specific to a directive (like "if").
> Instead, you should try to parse an unified/generic directive call
> syntax, and then invoke the Dialect to find out the further rules. And
> that's tricky, as then the parser definition doesn't specify which
> tags have an end-tag pair, and what can be nested between them, only
> the Dialect knows that. Like, if you look at the current parser, it
> basically says that "if" is like
>
> "<#" "if" Expression ">" MixedContent "</#" "if" ">"
>
> which is expressive and all, but sadly it won't be possible in FM3 to
> do it like that.
>
>> * expressions: numbers, variables, +
>> * variants of the parsers with different delimiters
>> * split into two parsers (interpolations/directives vs. expression language)
>>
>> What do you think?
>
> I haven't used any parser library but JavaCC, so I have not tips
> there. Otherwise the plan sounds good.
>
> Anyway, I kind of repeat myself here, but the expectations that may
> filter down the candidates quickly:
>
> - Splitting into two parsers, of course
>
> - Maintainability of custom syntax variations (like new FreeMarker
> versions won't break them, or at least they need no manual work to
> regenerate them)
>
> - How parsing partially driven by the Dialect looks... it won't fit
> JavaCC well for example. (But, probably it won't be very nice with
> any of them.)
>
> In case multiple of the libraries stay alive, some further extras that
> can decide:
>
> - More understandable/helpful error messages is a big plus.
>
> - It would be interesting to see how hard it is to write a parser that
> continues parsing after the first error, to catch more errors. This
> is mostly for IDE-s.
>
>> Stephan.
>>
>> P.S.: my more detailed list of parser generators can be found here:
>> https://gist.github.com/chaquotay/8041096bad36f6f3f0d4166d6f8623b5
>
--
Thanks,
Daniel Dekany
