Hi,
Ok, I have integrated my changes for tree pattern matching into the
mainline even though they will not be officially supported until 3.2.
We might do a 3.1.3 or more between now and 3.2, which will be the
version associated with the language design patterns book coming out
sometime this summer.
Before 3.2, You folks that have tree grammars working should take a
look at this tree filter mode. It makes tree grammars turn into awk-
like critters. It's like filter mode for lexers except for trees. The
idea is that you specify a bunch of rules and rewrites or actions. The
tree pattern matcher simply tries to match them everywhere in the
tree. The actions and/or rewrites trigger upon match. You don't have
to specify the complete tree grammar structure. For example if you
only want to change one subtree, you only need one tree grammar rule
and then filter=true. You don't need the entire structure. It's
amazingly simple addition.
Here is what you need to do for the upgrade.
1. Updating runtime support -- node streams
The biggest change relates to unbuffered node streams. I copied
CommonTreeNodeStream to BufferedTreeNodeStream and then made
CommonTreeNodeStream unbuffered. This was done with the use of
LookaheadStream<T> extends FastQueue<T>
which contains
protected TreeIterator it;
The generator allows me to walk a doubly linked tree with minimal
state. nodes need to know which child index they are, but that is a
current requirement of our trees anyway. So you need to implement
new implementation of CommonTreeNodeStream
LookaheadStream
FastQueue
TreeIterator
TreeFilter (tree parser when filter=true and no output option)
TreeRewriter (tree parser when filter=true and output=AST)
TreeVisitor and TreeVisitorAction should have been available for
3.1.2. TreeFilter and TreeRewriter use them.
CommonTreeNodeStream is completely unbuffered unless you are
backtracking. No longer making a list of all nodes before tree parsing.
When I get a chance, I will make CommonTokenStream unbuffered as well
using the new LookaheadStream. It will be a nice consolidation of all
of the arbitrary lookahead buffering going on.
2. Altering templates for the filter mode
[this was present in 3.1.2 but you might not be up to date with these]
I had to alter the core template definitions in ANTLRCore.sti so that
I could add attribute filterMode to genericParser and treeParser to
Java.stg. the superClass attribute of treeParser became:
superClass
=
{<
if
(filterMode
)>
<
if(buildAST)>TreeRewriter<else>TreeFilter<endif><else>TreeParser<endif>}
instead of TreeParser.
3. ANTLR changes
Other than that, I only had to change ANTLR itself a little bit. Made
filter mode valid for tree grammars and had it automatically set the
necessary elements: @synpredgate, backtrack=true, rewrite=true (if
output=AST). Added error message for detecting conflicting
options.
Hope that helps when you try to implement this. The tree pattern
matching stuff is super cool.
Ter
_______________________________________________
antlr-dev mailing list
[email protected]
http://www.antlr.org/mailman/listinfo/antlr-dev
