Hi Daniel,
thanks for your reminder, and sorry that I didn't manage to answer earlier.
I have to confess that I hugely, massively misjudged the amount of time
I'm able to spent on this topic. So apart from my initial list of
possible candidates (which I shared some months ago) and one small
experiment with one library I didn't manage to evaluate anything in
greater detail. So no, right now I cannot provide any recommendation,
and I fear that I won't be able to do so any time soon.
Stephan.
Am 15.11.2018 um 21:15 schrieb Daniel Dekany:
> Did this activity lead to somewhere? ?For example, do you have a
> recommendation for the lexer/parser library to use, if we only want to
> handle the expression syntax with it (so starting a new lexing/parsing
> should have low overhead)?
>
>
> Tuesday, August 7, 2018, 12:59:14 AM, Daniel Dekany wrote:
>
>> Sunday, August 5, 2018, 6:58:11 PM, Stephan Müller wrote:
>>
>>> Am 04.07.2018 um 19:28 schrieb Daniel Dekany:
>>>> I wonder what parser libraries could help us, in FM3, to separate the
>>>> expression language parsing from the top-level language (like
>>>> `<#foo>`, `${...}`, etc.) parsing. Or if a hand written parsers is an
>>>> acceptable compromise. It would be good if we can change the top-level
>>>> syntax and still reuse the expression syntax. (Or, replace the
>>>> expression syntax, and reuse the top-level one.) Like, somebody wants
>>>> a syntax like `#foo(exp)` instead of `<#foo exp>`, but still reuse the
>>>> expression syntax. (For me it was always part of the FM3 agenda,
>>>> though might will be proven to be too much...)
>>>> [..]
>>>
>>> During the last days I had a high-level look at different parser
>>> generators, and as one might imagine, there are a lot of parser
>>> generators, with different licenses, different maturities, different
>>> states of maintenance and so on.
>>>
>>> Due to https://www.apache.org/legal/resolved.html I ignored all parser
>>> generators which may not be included in Apache projects because of their
>>> license, especially GNU GPL etc.
>>>
>>> IMHO this leaves us with:
>>>
>>> * LL(k) parsers: ANTLR, JavaCC and Grammatica
>>> * LALR parsers: CookCC
>>> * PEG parsers: Mouse
>>> * parser combinators: jparsec, parboiled and PetitParser
>>>
>>> This list is not exhaustive, so I probably forget some interesting
>>> projects. If so, please share, I'd like to have a look into these, too.
>>>
>>> My idea for the next step: define a really small subset of FTL and try
>>> to implement PoCs for this subset with the candidates which I mentioned
>>> above.
>>>
>>> The subset might be something like
>>>
>>> * interpolations: ${..}
>>> * directives: if, assign
>>
>> Just to be on the safe side, I will note that you shouldn't try to
>> hard-code parser logic that's specific to a directive (like "if").
>> Instead, you should try to parse an unified/generic directive call
>> syntax, and then invoke the Dialect to find out the further rules. And
>> that's tricky, as then the parser definition doesn't specify which
>> tags have an end-tag pair, and what can be nested between them, only
>> the Dialect knows that. Like, if you look at the current parser, it
>> basically says that "if" is like
>>
>> "<#" "if" Expression ">" MixedContent "</#" "if" ">"
>>
>> which is expressive and all, but sadly it won't be possible in FM3 to
>> do it like that.
>>
>>> * expressions: numbers, variables, +
>>> * variants of the parsers with different delimiters
>>> * split into two parsers (interpolations/directives vs. expression language)
>>>
>>> What do you think?
>>
>> I haven't used any parser library but JavaCC, so I have not tips
>> there. Otherwise the plan sounds good.
>>
>> Anyway, I kind of repeat myself here, but the expectations that may
>> filter down the candidates quickly:
>>
>> - Splitting into two parsers, of course
>>
>> - Maintainability of custom syntax variations (like new FreeMarker
>> versions won't break them, or at least they need no manual work to
>> regenerate them)
>>
>> - How parsing partially driven by the Dialect looks... it won't fit
>> JavaCC well for example. (But, probably it won't be very nice with
>> any of them.)
>>
>> In case multiple of the libraries stay alive, some further extras that
>> can decide:
>>
>> - More understandable/helpful error messages is a big plus.
>>
>> - It would be interesting to see how hard it is to write a parser that
>> continues parsing after the first error, to catch more errors. This
>> is mostly for IDE-s.
>>
>>> Stephan.
>>>
>>> P.S.: my more detailed list of parser generators can be found here:
>>> https://gist.github.com/chaquotay/8041096bad36f6f3f0d4166d6f8623b5
>>
>
