[il-antlr-interest: 32478] [antlr-interest] IntelliSense, Auto suggest
Hi, We are using ANTLR in our product development. We want to know if there is any API in antlr which can provide the list of valid options (similar to auto suggest) as and when the user is typing. Thanks Regards, Swaroop. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32479] Re: [antlr-interest] Parsing comment-like sequences of arbitrary characters
Hi Rajesh,
Inside a parser rule, the `~` negates tokens, not characters. So if you have
no lexer rule that tokenizes one of: '%', ''^' or '$', then ~SEMICOLON won't
match any of such tokens/characters.
Your grammar (with minor modifications):
grammar Test;
options {
output=AST;
}
tokens {
OPTION;
OPTION_BLOCK;
}
query_options
: OPTIONS^ option_block
;
option_block
: L_BRACE option_def* R_BRACE - ^(OPTION_BLOCK option_def*)
;
option_def
: option_name option_value - ^(OPTION option_name option_value)
;
option_name
: ID (DOT^ ID)*
;
option_value
: COLON^ (~SEMICOLON)* SEMICOLON!
| option_block
;
OPTIONS : 'options';
ID: (LETTER | '_') (LETTER | DIGIT | '_')*;
DOLLAR: '$';
PERCENT: '%';
CARET: '^';
DOT: '.';
L_BRACE: '{';
R_BRACE: '}';
COLON: ':';
SEMICOLON: ';';
DIGIT : '0'..'9';
SL_COMMENT: '#' ~('\r' | '\n')* { skip(); };
WS: (' ' | '\f' | '\r' | '\t')+ { skip(); };
fragment LETTER : 'a'..'z' | 'A'..'Z';
parses the input: options { foo: $ % 1 2 45 ^ $ $$$; } as follows:
(options (OPTION_BLOCK (OPTION foo (: $ % 1 2 4 5 ^ $ $ $ $
as you can see after running the test rig:
import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.*;
public class Main {
public static void main(String[] args) throws Exception {
ANTLRStringStream in = new ANTLRStringStream(options { foo: $ %
1 2 45 ^ $ $$$; });
TestLexer lexer = new TestLexer(in);
CommonTokenStream tokens = new CommonTokenStream(lexer);
TestParser parser = new TestParser(tokens);
TestParser.query_options_return returnValue =
parser.query_options();
CommonTree tree = (CommonTree)returnValue.getTree();
DOTTreeGenerator gen = new DOTTreeGenerator();
StringTemplate st = gen.toDOT(tree);
System.out.println(st);
System.out.println(---\n +
tree.toStringTree());
}
}
Regards,
Bart.
On Wed, May 18, 2011 at 12:55 AM, Rajesh Raman r...@fb.com wrote:
Hello ANTLR-ites,
I'm trying to parse an options structure, like the following:
options {
foo {
bar {
ww: $32.50;
xx: Jekyll Hyde;
}
yy.zz: @15% p/a;
}
}
(Please ignore the non-sensical values for ww, xx and yy.zz -- I'm just
making a point, which will become clearer below). This options structure
will be followed by a query expression whose grammar is more complicated,
and includes ints/floats, identifiers, operators, etc. etc.
The grammar I have for parsing the options structure looks like the below.
(The grammar for the query language is complicated and therefore omitted.)
snip
// ... other stuff here
tokens {
// ... other ad hoc token values
OPTION;
OPTION_BLOCK;
OPTION_VALUE;
}
// ...
query_options
: OPTIONS^ option_block
;
option_block
: L_BRACE option_def* R_BRACE -
^(OPTION_BLOCK option_def*)
;
option_def
: option_name option_value -
^(OPTION option_name option_value)
;
option_name
: ID (DOT^ ID)*
;
option_value
: COLON^ (~SEMICOLON)* SEMICOLON!
| option_block
;
//... other stuff here
//...
OPTIONS: 'options';
ID: (LETTER | '_') (LETTER | DIGIT | '_')*;
DOT: '.';
L_BRACE: '{';
R_BRACE: '}';
COLON: ':';
SEMICOLON: ';';
SL_COMMENT: '#' ~('\r' | '\n')* NEWLINE { skip(); };
WS: (' ' | '\f' | '\r' | '\t')+ { skip(); };
...
/snip
As mentioned, the options clause is part of a larger grammar for a
language that includes operators, identifiers, numbers, etc., However,
within the options clause, I want the characters between the colon and the
semicolon to be treated as a single string, regardless of the fact that it
may contain characters that lex into other tokens used by the language.
This feels like I should be able to use the same techniques as used in
comment-stripping (i.e,. see the line that has COLON^...). But this doesn't
seem to work:
- The stray characters that are not used elsewhere in the grammar are
ignored and don't show up in the parse tree (e.g., $, @, %, , in the
example above)
- Character sequences that form valid tokens for the rest of the language
(like integers or identifiers) are lexed into those respective tokens
instead of being slurped into a single string as intended.
E.g., when I input a string like options { foo: $ % 1 2 45 ^ $ $$$; }
and display the resulting tree.toStringTree(), I get
(options (OPTION_BLOCK (OPTION foo (: 1 2 45
Any guidance you have on the above will be greatly appreciated.
Thanks in advance.
++Rajesh
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send
[il-antlr-interest: 32480] [antlr-interest] Fragment rules inside parser rules
Hello
Do I really need to make LKU in the example below a normal token rule or is
there a way to get this to work for the input 'foo;' not using literals
inside the parser rule or real tokens.
grammar Foo;
program
: (LKU NAME ';')+
;
fragment LKU
: ''
;
NAME
: ('a'..'z')*
;
=
Regards
Ben C.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32481] Re: [antlr-interest] Fragment rules inside parser rules
Hi Ben,
You cannot use fragment rules inside parser rules. So yes, you need to make
LKU a normal token. If you don't want that (for whatever reason), you need
to include the '' in the 'NAME' rule:
NAME
: '' ('a'..'z')+
;
Regards,
Bart.
On Wed, May 18, 2011 at 3:00 PM, Ben Corne ben.co...@gmail.com wrote:
Hello
Do I really need to make LKU in the example below a normal token rule or is
there a way to get this to work for the input 'foo;' not using literals
inside the parser rule or real tokens.
grammar Foo;
program
: (LKU NAME ';')+
;
fragment LKU
: ''
;
NAME
: ('a'..'z')*
;
=
Regards
Ben C.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32487] [antlr-interest] Problem with tree parser using ANTLR 3.x
Hi,
I'm newbie with ANTLR and is currently working on a ANTLR parser. The parser
grammar worked just fine. It created a tree that looked right. But when I
tried to walk the tree with my tree grammar, it failed. I hope that someone
here could help. Any help is appreciated!
The script language I'm working on has the following structure:
assignment or expressionnew line
assignment or expressionnew line
assignment or expressionnew line
where assignment is like
identifier = expression
My parser grammar is:
-
grammar myscript;
options {
language = Java;
output = AST;
ASTLabelType = CommonTree;
}
tokens {
NEGATION;
NEXT;
}
@header {
package a.b.c;
}
@lexer::header {
package a.b.c;
}
codeFragment
: (next!)? statement (next! statement)* EOF
;
next
: ('\r' | '\n' )+ - NEXT
;
statement
: expression
| assignment;
assignment
: IDENT '='^ expression
;
term
: INTEGER
| IDENT
| '('! expression ')'!
;
negate
: '!'^* term
;
unary
: ('+'! | negation^)* negate
;
negation
: '-' - NEGATION
;
mult
: unary (('*'^ | '/'^) unary)*
;
add
: mult (('+'^ | '-'^) mult)*
;
relation
: add (('=='^ | ''^ | '='^ | ''^ | '='^) add)*
;
expression
: relation ((''^ | ''^ | '|'^ | '||'^) relation)*
;
INTEGER: '0' | '1'..'9' DIGIT* 'L'? | '0x' DIGIT+ 'L'?;
IDENT: LETTER (LETTER | DIGIT)*;
WS: (' ' | '\t' | '\f')+ {$channel = HIDDEN;};
COMMENT: '#' .* ('\n'|'r') {$channel=HIDDEN;};
fragment DIGIT: '0'..'9';
fragment LETTER: ('a'..'z' | 'A'..'Z');
-
The tree walker grammar is:
-
tree grammar CodeFragWalker;
options {
language = Java;
tokenVocab = myscript;
ASTLabelType = CommonTree;
output = AST;
}
@header {
package a.b.c;
}
evaluator returns [String result]
: (assignment | expression)* EOF!
;
assignment returns [int result]
: ^('=' IDENT expression)
;
expression returns [int result]
: ^('+' expression expression)
| ^('-' expression expression)
| ^('*' expression expression)
| ^('/' expression expression)
| ^(NEGATION expression)
| IDENT
| INTEGER
;
-
The tree output from the parser is:
(= a (* (* 3 (+ 2 3)) 2)) (= b (+ 3 5)) (+ a b) (+ 1 2) EOF
The error I'm getting when running the following is :
node from line 2:1 mismatched tree node: = expecting EOF
The code:
-
CharStream charStream = new ANTLRStringStream(\na=3*(2+3)*2\r\n\r\n +
b=3+5\r +
a+b\n +
1+2);
myscriptLexer lexer = new myscriptLexer(charStream);
TokenStream tokenStream = new CommonTokenStream(lexer);
myscriptParser parser = new myscriptParser(tokenStream);
codeFragment_return fragment = parser.codeFragment();
CommonTree tree = fragment.tree;
System.out.println(tree.toStringTree());
CommonTreeNodeStream nodeStream = new CommonTreeNodeStream(tree);
CodeFragWalker walker = new CodeFragWalker(nodeStream);
walker.evaluator();
-
--
View this message in context:
http://antlr.1301665.n2.nabble.com/Problem-with-tree-parser-using-ANTLR-3-x-tp6380250p6380250.html
Sent from the ANTLR mailing list archive at Nabble.com.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
