Hi Daniel,
I try to transform the FreeMarker javaCC grammar to an ANTLR4 grammar.
I did not get the sematics of this lexer state syntax:
<FM_EXPRESSION, IN_PAREN, NAMED_PARAMETER_EXPRESSION> SKIP :
{
...
}
What does this mean? Does this match the current stack of lexer states?
Does JavaCC manage lexer states in a stack anyhow? I googled a lot but could
not find this syntax in the web.
If so, how to push/pop something from the lexer stack in javaCC? I just
could find the swithTo(...) trigger.
Best Regards,
Malte
-----Ursprüngliche Nachricht-----
Von: Daniel Dekany [mailto:[email protected]]
Gesendet: Samstag, 12. September 2015 19:40
An: Malte Brunnlieb
Cc: [email protected]
Betreff: Re: AW: Using FreeMarkers Parser and Lexer
The solution for FTL would be quite similar to Javaparser: there should be
*another* project that provides an FTL parser, but that's not (or not
exactly) the same parser than that FTL uses internally, similarly to as
Javaparser is not the same parser that javac uses internally. Maybe the best
way of making a such parser is copying and then modifying the current
parser, and ideally, that project is also helped by people who work on
FreeMarker itself, so that new language features will get in, and test cases
for new FTL features are provided. But someone had to take this quite big
task on himself, with longer term maintenance ideally.
Also, I think you will be better of by copying the current stuff into
another package (maybe under a liberal license), than accessing things with
reflection and such that was told to be internal. In the first case you
might won't support the latest language features after a FreeMarker update,
but at least your code won't die with ClassNotFound and NoSuchMethod after
the same FreeMarker update.
(To understand my standpoint more, my opinion is that one day, the current
parser will have to be heavily reworked, together with some other internals.
The goal of that is making the language more extensible and making templates
more introspectable with public API-s.
Another thing worths knowing is that keeping backward compatibility causes a
lot of hardship for FM, partly because of API-s negligently exposed in the
old days. So I have become very careful with undertaking new backward
compatibility obligations.)
--
Thanks,
Daniel Dekany
Saturday, September 12, 2015, 3:24:08 PM, Malte Brunnlieb wrote:
> Hi,
>
> I am currently trying to work with the Lexer/Parser of FreeMarker as well
as
> the Lexer/Parser of the target language to support syntax-safe generation.
> Thus, it would be nice to just use the FreeMarker library to parse
> FreeMarker templates as it is also possible with the javaparser I am
using.
>
> I respect the idea of only providing APIs, which will be backward
compatible
> in future. Nevertheless, The TokenManager as well as the Parser do follow
a
> commonly used API Pattern, which should be quite stable and could be made
> accessible. I do not claim to support these APIs actively by providing any
> documentation. It just makes life easier for those, who want to provide
> further tool support for FreeMarker template editors as well as further
> language engineers trying to use or adapt FreeMarker as a meta-language
for
> any (possibly research) intention. Providing the lexer and parser with
your
> deployment for free eventually would establish FreeMarker even more due to
> more potential tool support.
>
> Currently, I somehow managed to work with the non-public TokenManager and
> Parser API by hacking accessibility via reflection.
> I know this is not a stable approach, but I am fine with it to just get
the
> lexer token stream and the parser token stream to work on further ideas.
>
> Best Regards,
> Malte
>
> -----Ursprüngliche Nachricht-----
> Von: Daniel Dekany [mailto:[email protected]]
> Gesendet: Freitag, 11. September 2015 17:53
> An: Malte Brunnlieb
> Betreff: Re: Using FreeMarkers Parser and Lexer
>
> That the related API-s are not public or are public but marked as
> internal/deprecated is because FM doesn't provide backward compatibility
> there. So you might as well copy them into your package and change the
> visibilities, etc.
>
> What exactly do you want to achieve?
>
>
> Thursday, September 10, 2015, 11:33:42 AM, Malte Brunnlieb wrote:
>
>> Hi all,
>>
>> due to my current research, I try to access FreeMarkers Parser and/or
>> Lexer appropriately, but it seems, that you have already successfully
>> hidden the most necessary classes to do so.
>> The only thing, I am currently able to do is to get the parsed element
>> root. Nevertheless, also this API seems to be deprecated.
>>
>> I also tried to work directly with the FTL.jj defintion, but you
>> adapted the API of the Parser and Lexer Exceptions I cannot simply
>> reuse from your library.
>>
>> I know that my issue is a very special and advanced use case, but do
>> you see any possibility to work with the latest FreeMarker
>> Lexer/Parser by simple library reuse?
>>
>> Best Regards,
>> Malte
>>
>>
>
> --
> Thanks,
> Daniel Dekany
