date:20110505

[il-antlr-interest: 32381] Re: [antlr-interest] Inserting missing nodes

2011-05-05 Thread Bart Kiers

How about something like this:

grammar MyGrammar;

options {
  output=AST;
}

tokens {
  DEFAULT_OP;
}

query
  :  andExpression EOF - andExpression
  ;

andExpression
  :  (orExpression - orExpression) ( AND e=orExpression  -
^(AND $e $andExpression)
| (orExpression)= e=orExpression -
^(DEFAULT_OP $e $andExpression)
)*
  ;

orExpression
  :  negation (OR^ negation)*
  ;

negation
  :  NOT operand - ^(NOT operand)
  |  operand
  ;

operand
  :  WORD
  |  '(' andExpression ')' - andExpression
  ;

AND   : 'AND';
OR: 'OR';
NOT   : 'NOT';
WORD  : 'a'..'z'+;
SPACE : (' ' | '\t' | '\r' | '\n') {skip();};


Test class:

import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.*;

public class Main {
public static void main(String[] args) throws Exception {
ANTLRStringStream in = new ANTLRStringStream(software engineer OR
java programmer);
MyGrammarLexer lexer = new MyGrammarLexer(in);
CommonTokenStream tokens = new CommonTokenStream(lexer);
MyGrammarParser parser = new MyGrammarParser(tokens);
MyGrammarParser.query_return returnValue = parser.query();
CommonTree tree = (CommonTree)returnValue.getTree();
DOTTreeGenerator gen = new DOTTreeGenerator();
StringTemplate st = gen.toDOT(tree);
System.out.println(st);
}
}


Regards,

Bart.

On Wed, May 4, 2011 at 4:51 PM, Jean-Sebastien Vachon 
jean-sebastien.vac...@wantedtech.com wrote:

 Thanks for your input. So here is the whole thing with two use cases that
 are not giving me the expected results...
 (Sorry for the long post)

 INPUT = abc def zyx toto
 RESULT = (DEFAULT_OP abc def) (DEFAULT_OP zyx toto)
 EXPECTED = (DEFAULT_OP (DEFAULT_OP abc def) (DEFAULT_OP zyx toto))

 INPUT = software engineer OR java programmer
 RESULT = (DEFAULT_OP software (OR engineer java)) programmer
 EXPECTED =  (DEFAULT_OP (DEFAULT_OP software (OR engineer java))
 programmer)

 I'm also having some trouble using the Interpreter within Eclipse.
 The same expressions are not working in the interpreter. It fails to
 generate the
 tree with a NoViableAltException at input 'abc'  (for the first case).
 I don't think this is related to my other problem since I can't get it to
 generate any tree.

 Thanks again for your time and comments


 --
 Grammar (validation by building a tree and trying to insert missing
 operators)

 --
 grammar MyGrammar;

 options {
  language = Java;
  output = AST;
  ASTLabelType = CommonTree;
 }

 // Rules to build the tree representation of our expression...

 query
  : and_expr+ EOF!
  ;

 // Each AND expression can contain OR expressions...
 and_expr
  : (expr expr+) = default_op
  | (u1=or_expr (AND^ u2=or_expr)*)
  ;

 // A OR expression contains one or more expression
 or_expr
  : u1=expr (OR^ u2=expr)*
  ;

 default_op
  : (e1=or_expr e2=or_expr) - ^(DEFAULT_OP $e1 $e2)
  ;

 expr
  : (NOT^)? (operand)
  ;

 // The leafs of the tree.. Words, sentence and so on...
 // Note that an expression such as '-word' is rewritten in its 'NOT word'
 form
 operand
  : (f=FIELD^)(o=operand)
   | PREFIX
  | WORD
  | SENTENCE
  | WORDLIST
  | NEGATIVE(w=PREFIX|w=WORD|w=SENTENCE|w=WORDLIST) - ^(NOT $w)
  | MUST
   | LPAREN! and_expr RPAREN!
  ;

 // Lexer ...
 NEGATIVE: '-';
 LPAREN  : '(' ;
 RPAREN  : ')' ;
 DOUBLEQUOTE : '';
 STAR  : '*';
 AND : 'AND' | '+';
 OR  : 'OR';
 NOT : 'NOT';
 DEFAULT_OP  : 'DEF_OP';
 FIELD   : ('title'|'TITLE'|'Title')(FIELDSEPARATOR);
 WS  : (WSCHAR)+ { $channel=HIDDEN; };
 PREFIX  : WORDCHAR+(STAR);
 WORD: WORDCHAR+(('-'|'+')WORDCHAR*)*;
 SENTENCE: ((DOUBLEQUOTE)(~(DOUBLEQUOTE))*(DOUBLEQUOTE));
 WORDLIST: ((PREFIX | WORD | SENTENCE)(','(WS)* (PREFIX | WORD |
 SENTENCE))+);
 MUST  : '+'(PREFIX|WORD|SENTENCE|WORDLIST);
 fragment WORDCHAR   : (~( WSCHAR | LPAREN | RPAREN | '-' |':' | '+' |
 ',' | STAR | DOUBLEQUOTE) );
 fragment FIELDSEPARATOR : ':';
 fragment WSCHAR : ( ' ' | '\t' | '\r' | '\n');



 = END OF GRAMMAR ==





 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:
 antlr-interest-boun...@antlr.org] On Behalf Of Bart Kiers
 Sent: May-04-11 10:21 AM
 To: antlr-interest@antlr.org
 Subject: Re: [antlr-interest] Inserting missing nodes

 On Wed, May 4, 2011 at 4:12 PM, Jean-Sebastien Vachon 
 jean-sebastien.vac...@wantedtech.com wrote:

  No one can help me with this? :S
  Let me know if something is not clear. I need to fix this issue as
  soon as I can.
 
  Thanks


 The fact that you didn't provide the lexer rules (although they might be
 straight-forward

[il-antlr-interest: 32382] Re: [antlr-interest] Source code level of ANTLRWorks

2011-05-05 Thread Ben Corne

Here's a screenshot of my antlrworks preferences page + javac info
http://igor.rave.org/antlrjavac.png

I don't see the option where to adjust the commandline call.
And from what I think to understand from the helppage of my javac, it says
1.4 is the default, but I don't find a way to change this default, except by
writing a wrapper shell script that attaches -source 1.5 (which is what I
will do now, until more clues are given)

Ben

2011/5/4 Jim Idle j...@temporal-wave.com

 That just means you are calling a compiler that has a default level less
 than 1.5. So, chance the javacc command in the preferences or remove the
 old compiler and install 1.6, or makes sure that 1.6 is the first on the
 command line. ANTLR works does not influence the Java compiler - it only
 calls the one that you tell it to.

 Jim



  -Original Message-
  From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
  boun...@antlr.org] On Behalf Of Ben Corne
  Sent: Wednesday, May 04, 2011 6:56 AM
  To: antlr-interest@antlr.org
  Subject: [antlr-interest] Source code level of ANTLRWorks
 
  Hi
 
  Is there a way to set the source code level of antlrworks?
  I've checked the preferences pane and launchers with no avail.
 
  I need to set it to  1.5, since I'm using Java Generics:
 
  [15:47:58] 102. ERROR in
  /home/bcorne/Downloads/at2-parser-3/grammar/output/ATGrammar3Lexer.java
  (at line 114) [15:47:58] StackString paraphrase = new
  StackString();
  [15:47:58]  ^^
  [15:47:58] Syntax error, parameterized types are only available if
  source level is 1.5
 
  Regards
  Ben Corne
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
  email-address

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe:
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 32383] Re: [antlr-interest] Source code level of ANTLRWorks

2011-05-05 Thread Jim Idle

Have you tried adding it in with the PATH? Also you can use -source 1.6
-target jsr14 and use 1.6 input and 1.4 output.

Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of Ben Corne
 Sent: Thursday, May 05, 2011 4:58 AM
 To: antlr-interest@antlr.org
 Subject: Re: [antlr-interest] Source code level of ANTLRWorks

 Here's a screenshot of my antlrworks preferences page + javac info
 http://igor.rave.org/antlrjavac.png

 I don't see the option where to adjust the commandline call.
 And from what I think to understand from the helppage of my javac, it
 says
 1.4 is the default, but I don't find a way to change this default,
 except by writing a wrapper shell script that attaches -source 1.5
 (which is what I will do now, until more clues are given)

 Ben

 2011/5/4 Jim Idle j...@temporal-wave.com

  That just means you are calling a compiler that has a default level
  less than 1.5. So, chance the javacc command in the preferences or
  remove the old compiler and install 1.6, or makes sure that 1.6 is
 the
  first on the command line. ANTLR works does not influence the Java
  compiler - it only calls the one that you tell it to.

  Jim

   -Original Message-
   From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
   boun...@antlr.org] On Behalf Of Ben Corne
   Sent: Wednesday, May 04, 2011 6:56 AM
   To: antlr-interest@antlr.org
   Subject: [antlr-interest] Source code level of ANTLRWorks

   Hi

   Is there a way to set the source code level of antlrworks?
   I've checked the preferences pane and launchers with no avail.

   I need to set it to  1.5, since I'm using Java Generics:

   [15:47:58] 102. ERROR in
   /home/bcorne/Downloads/at2-parser-
 3/grammar/output/ATGrammar3Lexer.j
   ava (at line 114) [15:47:58] StackString paraphrase = new
   StackString();
   [15:47:58]  ^^
   [15:47:58] Syntax error, parameterized types are only available if
   source level is 1.5

   Regards
   Ben Corne

   List: http://www.antlr.org/mailman/listinfo/antlr-interest
   Unsubscribe:
   http://www.antlr.org/mailman/options/antlr-interest/your-
   email-address

  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe:
  http://www.antlr.org/mailman/options/antlr-interest/your-email-
 address

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 32384] Re: [antlr-interest] Inserting missing nodes

2011-05-05 Thread Jean-Sebastien Vachon

Hi,

I've integrated your solution to my whole grammar and it works perfectly.

Thanks for your help

From: Bart Kiers [mailto:bki...@gmail.com]
Sent: May-05-11 4:51 AM
To: Jean-Sebastien Vachon
Cc: antlr-interest@antlr.org
Subject: Re: [antlr-interest] Inserting missing nodes

How about something like this:

grammar MyGrammar;

options {
  output=AST;
}

tokens {
  DEFAULT_OP;
}

query
  :  andExpression EOF - andExpression
  ;

andExpression
  :  (orExpression - orExpression) ( AND e=orExpression  - ^(AND 
$e $andExpression)
| (orExpression)= e=orExpression - 
^(DEFAULT_OP $e $andExpression)
)*
  ;

orExpression
  :  negation (OR^ negation)*
  ;

negation
  :  NOT operand - ^(NOT operand)
  |  operand
  ;

operand
  :  WORD
  |  '(' andExpression ')' - andExpression
  ;

AND   : 'AND';
OR: 'OR';
NOT   : 'NOT';
WORD  : 'a'..'z'+;
SPACE : (' ' | '\t' | '\r' | '\n') {skip();};

Test class:

import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.*;

public class Main {
public static void main(String[] args) throws Exception {
ANTLRStringStream in = new ANTLRStringStream(software engineer OR java 
programmer);
MyGrammarLexer lexer = new MyGrammarLexer(in);
CommonTokenStream tokens = new CommonTokenStream(lexer);
MyGrammarParser parser = new MyGrammarParser(tokens);
MyGrammarParser.query_return returnValue = parser.query();
CommonTree tree = (CommonTree)returnValue.getTree();
DOTTreeGenerator gen = new DOTTreeGenerator();
StringTemplate st = gen.toDOT(tree);
System.out.println(st);
}
}

Regards,

Bart.

On Wed, May 4, 2011 at 4:51 PM, Jean-Sebastien Vachon 
jean-sebastien.vac...@wantedtech.commailto:jean-sebastien.vac...@wantedtech.com
 wrote:
Thanks for your input. So here is the whole thing with two use cases that are 
not giving me the expected results...
(Sorry for the long post)

INPUT = abc def zyx toto
RESULT = (DEFAULT_OP abc def) (DEFAULT_OP zyx toto)
EXPECTED = (DEFAULT_OP (DEFAULT_OP abc def) (DEFAULT_OP zyx toto))

INPUT = software engineer OR java programmer
RESULT = (DEFAULT_OP software (OR engineer java)) programmer
EXPECTED =  (DEFAULT_OP (DEFAULT_OP software (OR engineer java)) programmer)

I'm also having some trouble using the Interpreter within Eclipse.
The same expressions are not working in the interpreter. It fails to generate 
the
tree with a NoViableAltException at input 'abc'  (for the first case).
I don't think this is related to my other problem since I can't get it to 
generate any tree.

Thanks again for your time and comments

--
Grammar (validation by building a tree and trying to insert missing operators)
--
grammar MyGrammar;

options {
 language = Java;
 output = AST;
 ASTLabelType = CommonTree;
}
// Rules to build the tree representation of our expression...

query
 : and_expr+ EOF!
 ;

// Each AND expression can contain OR expressions...
and_expr
 : (expr expr+) = default_op
 | (u1=or_expr (AND^ u2=or_expr)*)
 ;
// A OR expression contains one or more expression
or_expr
 : u1=expr (OR^ u2=expr)*
 ;

default_op
 : (e1=or_expr e2=or_expr) - ^(DEFAULT_OP $e1 $e2)
 ;

expr
 : (NOT^)? (operand)
 ;
// The leafs of the tree.. Words, sentence and so on...
// Note that an expression such as '-word' is rewritten in its 'NOT word' form
operand
 : (f=FIELD^)(o=operand)
 | PREFIX
 | WORD
 | SENTENCE
 | WORDLIST
 | NEGATIVE(w=PREFIX|w=WORD|w=SENTENCE|w=WORDLIST) - ^(NOT $w)
 | MUST
 | LPAREN! and_expr RPAREN!
 ;

// Lexer ...
NEGATIVE: '-';
LPAREN  : '(' ;
RPAREN  : ')' ;
DOUBLEQUOTE : '';
STAR  : '*';
AND : 'AND' | '+';
OR  : 'OR';
NOT : 'NOT';
DEFAULT_OP  : 'DEF_OP';
FIELD   : ('title'|'TITLE'|'Title')(FIELDSEPARATOR);
WS  : (WSCHAR)+ { $channel=HIDDEN; };
PREFIX  : WORDCHAR+(STAR);
WORD: WORDCHAR+(('-'|'+')WORDCHAR*)*;
SENTENCE: ((DOUBLEQUOTE)(~(DOUBLEQUOTE))*(DOUBLEQUOTE));
WORDLIST: ((PREFIX | WORD | SENTENCE)(','(WS)* (PREFIX | WORD | 
SENTENCE))+);
MUST  : '+'(PREFIX|WORD|SENTENCE|WORDLIST);
fragment WORDCHAR   : (~( WSCHAR | LPAREN | RPAREN | '-' |':' | '+' | ',' | 
STAR | DOUBLEQUOTE) );
fragment FIELDSEPARATOR : ':';
fragment WSCHAR : ( ' ' | '\t' | '\r' | '\n');



= END OF GRAMMAR ==





-Original Message-
From: antlr-interest-boun...@antlr.orgmailto:antlr-interest-boun...@antlr.org 
[mailto:antlr-interest-boun...@antlr.orgmailto:antlr-interest-boun...@antlr.org]
 On Behalf Of Bart Kiers
Sent: May-04-11 10:21 AM
To: antlr-interest@antlr.orgmailto:antlr-interest@antlr.org
Subject: Re: [antlr-interest] Inserting

[il-antlr-interest: 32385] Re: [antlr-interest] rules that takes rules as arguments

2011-05-05 Thread Bart Kiers

Both parser- and lexer rules can take parameters of any (Java) type.
However, rules in ANTLR are translated to methods, and methods aren't
first class objects in Java. So providing a rule (read: method) to another
rule as parameter is not possible.

Perhaps you can elaborate a bit and explain what problem you're trying to
solve?

Regards,

Bart.


On Tue, May 3, 2011 at 7:29 PM, Vladimir VG gordeev.vladimi...@gmail.comwrote:

 I'm defining some grouping rule ( '(' value ')' ) for operator precedence,
 and it will be helpful if I would use rules that takes other rules as
 arguments.

 Is that possible?

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe:
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 32387] [antlr-interest] rewrite empty input string to *

2011-05-05 Thread Jean-Sebastien Vachon

Hi All,

What is the best way to rewrite an empty input?

I mean if my input is  (empty string) how can I rewrite to * ?

The desired behaviour is to search everything (*) if the input is empty.

I thought about upgrading my top rule to be:

query
  : (andExpression+ - andExpression) EOF
  | (SPACE*) EOF - STAR
  ;

Where STAR and SPACE are defined as:

STAR  : '*';
SPACE : (' ' | '\t' | '\f')+ {$channel=HIDDEN;};

With this set of rules the output of tree in its text form is STAR while I 
was expecting *

What am I doing wrong?

Thanks

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 32381] Re: [antlr-interest] Inserting missing nodes

[il-antlr-interest: 32382] Re: [antlr-interest] Source code level of ANTLRWorks

[il-antlr-interest: 32383] Re: [antlr-interest] Source code level of ANTLRWorks

[il-antlr-interest: 32384] Re: [antlr-interest] Inserting missing nodes

[il-antlr-interest: 32385] Re: [antlr-interest] rules that takes rules as arguments

[il-antlr-interest: 32387] [antlr-interest] rewrite empty input string to *

6 matches

Site Navigation

Mail list logo

Footer information