Re: [fpc-pascal] tply: start conditions

2018-08-10 Thread Martok
Am 10.08.2018 um 12:14 schrieb Martok:
> What is really missing then is %x for exclusive start conditions, which would 
> solve all of that.

Things I never wanted to be able to do: run scanner state machines in my head. 
But: turns out adding %x was less work than
finding an alternative solution, once it was clear how the DFA is actually 
built.

It currently lives on my work-in-progress-branch for fpcres: 
https://github.com/martok/freepascal/commit/79873cd1c8
Proper patches will follow once things settle.

-- 
Regards,
Martok

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] tply: start conditions

2018-08-10 Thread Martok
> This is a bit awkward if one has to write the prefix every time...PS - plex 
> doesn't define a name for the builtin default state, so even that would not 
> actually be possible.

What is really missing then is %x for exclusive start conditions, which would 
solve all of that.


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] tply: start conditions

2018-08-10 Thread Martok
Hi all,

this is more of a generic lex question, but since plex doesn't support the flex 
extension that could easily fix it, I'm asking here.

Take these lex rules:
--
%start INCOMLINE
%%
"//"start(INCOMLINE);
\n   start(0);
.yymore;
[a-zA-Z_]([a-zA-Z0-9_])* return(ID);
--

The idea is that the comment start sequence "//" puts the lexer in the 
INCOMLINE state, which eats everything until the next
newline, then goes back into the start state. This works for lines like that:
  Foo//
But because lex picks the longest matching rule first, it won't work here:
  Foo//Bar
"Bar" still gets parsed as an ID token, because the identifier rule is valid 
for any state and considered longer.

Flex has Condition Scopes (see 
http://dinosaur.compilertools.net/flex/flex_11.html near the end), where one 
could just put all
generic rules in a "INITIAL only" scope. This is a bit awkward if one has to 
write the prefix every time...

I have solved this in the past by including a mini scanner in the action 
because comments are easy, but still... does anyone
have a better idea to do this right in the lexer?

-- 
Regards,
Martok


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal