On 03.05.2017 20:55, Remi Forax wrote:
> It's context-free because a context free grammar defined its input in term of
> terminals and the theory do not say how to map a token to a terminal.
>
> Jay is right that it requires to use either some specific parser generator
> like Tatoo [1] the one i've written 10 years ago (because i wanted the tool to
> help me to extend a grammar easily) or to modify an existing parser generator
so
> the parser can send the production state to the lexer which will
enable/disable
> the automata that recognize the associated keywords .
Just feeding parser state into the Lexer doesn't cut it for Java 9,
because the classification keyword / identifier cannot be made at
the time when the stream passes the Lexer.
Let me remind you of this example:
module foo { exports transitive
How should the poor lexer recognize in this situation that transitive
is an identifier (sic) (if you complete the text accordingly)?
Aside from specific heuristics
(which are not available to any parser generator),
we only know about this classification after the parser has matched
an entire declaration.
I'm not even sure that theory has a name for this kind of grammar.
Maybe we should speak of a constraint solver rather than a parser.
Stephan
Rémi
[1] http://dl.acm.org/citation.cfm?id=1168057
----- Mail original -----
De: "Alex Buckley" <alex.buck...@oracle.com>
À: "Jayaprakash Arthanareeswaran" <jarth...@in.ibm.com>, "Dan Smith"
<daniel.sm...@oracle.com>, "Brian Goetz"
<brian.go...@oracle.com>
Cc: jigsaw-dev@openjdk.java.net
Envoyé: Mercredi 3 Mai 2017 19:46:54
Objet: Re: Java Platform Module System
On 5/2/2017 3:39 PM, Alex Buckley wrote:
On 5/2/2017 7:07 AM, Jayaprakash Arthanareeswaran wrote:
Chapter 2 in [1] describes context-free grammars. The addition to "3.9
Keywords" defines "restricted keywords", which prevent the grammar for
ModuleDeclaration from being context-free. This prevents compilers from
using common parser generators, since those typically only support
context-free grammars. The lexical/syntactic grammar split defined in
chapter 2 is not of much use for actual implementations of
module-info.java parsers.
The spec at least needs to point out that the given grammar for
ModuleDeclaration is not actually context-free.
The syntactic grammar in JLS8 was not context-free either; the opening
line of Chapter 2 has been false for years. For JLS9, I will remove the
claim that the lexical and syntactic grammars are context-free, and
perhaps a future JLS can discuss the difficulties in parsing the
Jan Lahoda pointed out privately that the syntactic grammar in JLS8 and
JLS9 is in fact context-free -- it's just not LL(1). Not being LL(1) is
what I should have said the grammar hasn't been for a long time.
Alex
