Bob, Thank you very much, that was an excellent explanation in great detail. I'm guessing the next step would be to move away from: r.tree.children[2].toString() to something more useful like: tablename = r.getTableName(); Is this at all possible and where can I find an example of this? I see a similar example here but I'm not having much luck getting it to work: http://codemate.wordpress.com/2009/02/06/writing-a-sql-parser/
Remy. On Fri, Feb 26, 2010 at 10:04 AM, Bob Adolf <[email protected]> wrote: > Hi Remy, > > I think you'll be surprised at how similar the python and java versions > actually are. You made things a little more difficult for yourself than you > needed. I've put some comments in-line with your original python code and > finished off the end. > >> import antlr3 >> import antlr3.tree >> from sqltestLexer import sqltestLexer >> from sqltestParser import sqltestParser >> #import sqltest ? won't import > > The grammar sqltest.g actually doesn't produce an sqltest file or class, so > you just need the first two. > >> import sys >> #from sqltest import sqltest >> #sys.argv[1] >> char_stream = antlr3.ANTLRStringStream("SELECT * FROM BOOKS;") > > Your grammar actually doesn't parse this phrase, but I assume you just > forgot to replace the place-holder string with something else. I used this > instead: > char_stream = antlr3.ANTLRStringStream("CREATE TABLE foo ( bar int ) ;") > which (not knowing SQL) seems to fit your grammar at least. > > It seems like you tried using an input file first. I'm assuming that you > want something like your original java file that selects between a file or a > string. Well, it looks very much like the java version: > if( len(sys.argv)>1 ): > char_stream = antlr3.ANTLRFileStream(sys.argv[1]) > else: > char_stream = antlr3.ANTLRStringStream(raw_input("SQL>")) > >> lexer = sqltestLexer(char_stream) >> tokens = antlr3.CommonTokenStream(lexer) >> parser = sqltestParser(tokens) >> >> r = parser.createtablestmt_return() > > You'll actually want to call the function itself. The types are handled > automatically, so you can focus on the work that's actually being done. The > function parser.createtablestmt (which you can look at directly in > sqltestParser.py) is ANTLR's translation of your root-level grammar rule. > You can walk through the logic in the python source as it tries to match > against input tokens and recursively invokes the other rules. > r = parser.createtablestmt() > > Interestingly enough, this is where your java version stops. You just print > out the tree element. > # System.out.println("tablename ="+((Tree)r.tree).getChild(2).toString()); > # System.out.println("tree="+((Tree)r.tree).toStringTree()); > > You can do the same thing in python! > print "tablename = " + r.tree.children[2].toString() > print "tree = " + r.tree.toStringTree() > > >> >> # this is the root of the AST >> root = r.tree >> >> nodes = antlr3.tree.CommonTreeNodeStream(root) >> nodes.setTokenStream(tokens) >> >> #walker = sqltest(nodes) >> # STUCK! > > As for the rest of this, it looks like you started trying to implement a > tree walker. If you're serious about munging the AST, you'll probably want > to do this later, but for simple things like just dumping the tree, you > don't need to. Likewise, if you're just doing a simple extraction of SQL > snippets or some easy translation, you can do it directly by inserting > actions into your parser at the appropriate place. > > If you do decide that a tree walker is what you need, then you'll need to > create a separate grammar for that (and this is why you got stuck, there was > nothing you could invoke because you hadn't written a tree parser). You > should probably run through the python examples > (http://www.antlr.org/download/examples-v3.tar.gz) or the ANTLR book if > you're going to do that. > > Hope this helps. > > > -Bob > > > > For clarity, here's the whole file I used (driver.py): > import antlr3 > from sqltestLexer import sqltestLexer > from sqltestParser import sqltestParser > import sys > > if( len(sys.argv)>1 ): > char_stream = antlr3.ANTLRFileStream(sys.argv[1]) > else: > char_stream = antlr3.ANTLRStringStream(raw_input("SQL>")) > lexer = sqltestLexer(char_stream) > tokens = antlr3.CommonTokenStream(lexer) > parser = sqltestParser(tokens) > > r = parser.createtablestmt() > > # System.out.println("tablename ="+((Tree)r.tree).getChild(2).toString()); > # System.out.println("tree="+((Tree)r.tree).toStringTree()); > print "tablename = " + r.tree.children[2].toString() > print "tree = " + r.tree.toStringTree() > > And the output: > tablename = foo > tree = CREATE TABLE foo ( bar int ) > > List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
