Sunday, September 13, 2015, 1:15:05 PM, Malte Brunnlieb wrote:
> Hi Daniel,
>
> I try to transform the FreeMarker javaCC grammar to an ANTLR4 grammar.
> I did not get the sematics of this lexer state syntax:
>
> <FM_EXPRESSION, IN_PAREN, NAMED_PARAMETER_EXPRESSION> SKIP :
> {
> ...
> }
>
> What does this mean?
The stuff inside <...> means that the patterns inside the {...} are
only used in the lexical states listed there (logical "or").
> Does this match the current stack of lexer states? Does JavaCC
> manage lexer states in a stack anyhow?
At any moment the lexer is in a single state which is identified with
a name (initially DEFAULT), and that's only what it matches. There's
no stack as far as I know. The regular expressions switch between
states directly.
> I googled a lot but could not find this syntax in the web.
> If so, how to push/pop something from the lexer stack in javaCC?
One could maintain a such stack manually, since you can have custom
lexer state fields. It doesn't solve matching against stack content
efficiently, but at least you can push and pop, with your own methods.
> I just could find the swithTo(...) trigger.
>
> Best Regards,
> Malte
>
> -----Ursprüngliche Nachricht-----
> Von: Daniel Dekany [mailto:[email protected]]
> Gesendet: Samstag, 12. September 2015 19:40
> An: Malte Brunnlieb
> Cc: [email protected]
> Betreff: Re: AW: Using FreeMarkers Parser and Lexer
>
> The solution for FTL would be quite similar to Javaparser: there should be
> *another* project that provides an FTL parser, but that's not (or not
> exactly) the same parser than that FTL uses internally, similarly to as
> Javaparser is not the same parser that javac uses internally. Maybe the best
> way of making a such parser is copying and then modifying the current
> parser, and ideally, that project is also helped by people who work on
> FreeMarker itself, so that new language features will get in, and test cases
> for new FTL features are provided. But someone had to take this quite big
> task on himself, with longer term maintenance ideally.
>
> Also, I think you will be better of by copying the current stuff into
> another package (maybe under a liberal license), than accessing things with
> reflection and such that was told to be internal. In the first case you
> might won't support the latest language features after a FreeMarker update,
> but at least your code won't die with ClassNotFound and NoSuchMethod after
> the same FreeMarker update.
>
> (To understand my standpoint more, my opinion is that one day, the current
> parser will have to be heavily reworked, together with some other internals.
> The goal of that is making the language more extensible and making templates
> more introspectable with public API-s.
> Another thing worths knowing is that keeping backward compatibility causes a
> lot of hardship for FM, partly because of API-s negligently exposed in the
> old days. So I have become very careful with undertaking new backward
> compatibility obligations.)
>
> --
> Thanks,
> Daniel Dekany
>
>
> Saturday, September 12, 2015, 3:24:08 PM, Malte Brunnlieb wrote:
>
>> Hi,
>>
>> I am currently trying to work with the Lexer/Parser of FreeMarker as well
> as
>> the Lexer/Parser of the target language to support syntax-safe generation.
>> Thus, it would be nice to just use the FreeMarker library to parse
>> FreeMarker templates as it is also possible with the javaparser I am
> using.
>>
>> I respect the idea of only providing APIs, which will be backward
> compatible
>> in future. Nevertheless, The TokenManager as well as the Parser do follow
> a
>> commonly used API Pattern, which should be quite stable and could be made
>> accessible. I do not claim to support these APIs actively by providing any
>> documentation. It just makes life easier for those, who want to provide
>> further tool support for FreeMarker template editors as well as further
>> language engineers trying to use or adapt FreeMarker as a meta-language
> for
>> any (possibly research) intention. Providing the lexer and parser with
> your
>> deployment for free eventually would establish FreeMarker even more due to
>> more potential tool support.
>>
>> Currently, I somehow managed to work with the non-public TokenManager and
>> Parser API by hacking accessibility via reflection.
>> I know this is not a stable approach, but I am fine with it to just get
> the
>> lexer token stream and the parser token stream to work on further ideas.
>>
>> Best Regards,
>> Malte
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Daniel Dekany [mailto:[email protected]]
>> Gesendet: Freitag, 11. September 2015 17:53
>> An: Malte Brunnlieb
>> Betreff: Re: Using FreeMarkers Parser and Lexer
>>
>> That the related API-s are not public or are public but marked as
>> internal/deprecated is because FM doesn't provide backward compatibility
>> there. So you might as well copy them into your package and change the
>> visibilities, etc.
>>
>> What exactly do you want to achieve?
>>
>>
>> Thursday, September 10, 2015, 11:33:42 AM, Malte Brunnlieb wrote:
>>
>>> Hi all,
>>>
>>> due to my current research, I try to access FreeMarkers Parser and/or
>>> Lexer appropriately, but it seems, that you have already successfully
>>> hidden the most necessary classes to do so.
>>> The only thing, I am currently able to do is to get the parsed element
>>> root. Nevertheless, also this API seems to be deprecated.
>>>
>>> I also tried to work directly with the FTL.jj defintion, but you
>>> adapted the API of the Parser and Lexer Exceptions I cannot simply
>>> reuse from your library.
>>>
>>> I know that my issue is a very special and advanced use case, but do
>>> you see any possibility to work with the latest FreeMarker
>>> Lexer/Parser by simple library reuse?
>>>
>>> Best Regards,
>>> Malte
>>>
>>>
>>
>> --
>> Thanks,
>> Daniel Dekany
>
>
>
--
Thanks,
Daniel Dekany
