[il-antlr-interest: 31475] [antlr-interest] [begginer question] could somebody help me to see this simple grammar?

[devdoer bird]

HI,all:

the grammar is list below:
--
logicExp
 :  FIELDNAME ( '' | '' ) (CONST_INT | CONST_STRING)
 | '(' logicExp')'

 ;

fragment DIGIT : '0'..'9';
fragment LOWER  : 'a'..'z';
fragment UPPER  : 'A'..'Z';
FIELDNAME : LOWER (LOWER|DIGIT)*  ;
CONST_INT : DIGIT+;
CONST_STRING : '' (LOWER|UPPER|DIGIT)* '';



When I use the antlrworks to test the grammar,I use the input ( a3 ),but
the interepter failed to recoginze the last ), output as follows:
 logicExp
   / \
 /\
(logicExp
  /\  \
 /  \  \
a 3

As you see, the last ')'  is missed.
So what's wrong with my grammar?

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31476] Re: [antlr-interest] [begginer question] could somebody help me to see this simple grammar?

[Bart Kiers]

On Wed, Feb 16, 2011 at 9:18 AM, devdoer bird devdo...@gmail.com wrote:

 ...
 As you see, the last ')'  is missed.
 So what's wrong with my grammar?


Nothing.

Given your grammar (and adding a `parse` rule to it):

grammar T;

parse
  :  logicExp EOF {System.out.println(parsed ::  + $logicExp.text);}
  ;

logicExp
  :  FIELDNAME ( '' | '' ) (CONST_INT | CONST_STRING)
  | '(' logicExp')'
  ;

fragment DIGIT : '0'..'9';
fragment LOWER  : 'a'..'z';
fragment UPPER  : 'A'..'Z';
FIELDNAME : LOWER (LOWER|DIGIT)*  ;
CONST_INT : DIGIT+;
CONST_STRING : '' (LOWER|UPPER|DIGIT)* '';


and creating the test class:

import org.antlr.runtime.*;

public class Main {
public static void main(String[] args) throws Exception {
ANTLRStringStream in = new ANTLRStringStream((a3));
TLexer lexer = new TLexer(in);
CommonTokenStream tokens = new CommonTokenStream(lexer);
TParser parser = new TParser(tokens);
parser.parse();
}
}


I generated a lexer  parser, compiled all .java files and ran the main
class:

java -cp antlr-3.2.jar org.antlr.Tool T.g
javac -cp antlr-3.2.jar *.java
java -cp .:antlr-3.2.jar Main


which produced the output:

parsed :: (a3)


So, if ANTLRWorks produces something different, there's something wrong with
it.

Regards,

Bart.

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31477] Re: [antlr-interest] what would ANTLR ref guide revised edition have?

[Douglas Godfrey]

How about an example Symbol Table class that can handle C++ or Java style
languages
with nested scopes and drill down access to nested classes via name spaces
and compound names.

On Tue, Feb 15, 2011 at 6:36 PM, Terence Parr pa...@cs.usfca.edu wrote:

 Howdy. I'm thinking about revising the ref guide for ANTLR 3 (ANTLR v4
 might be awhile so I should update book).  Any suggestions to improve?  One
 obvious thing: discuss not just java target :)

 Ter

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe:
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31478] Re: [antlr-interest] what would ANTLR ref guide revised edition have?

[devdoer bird]

Thanks.

But I found it also failed in eclipse plugin.

I tried the following grammar.
---
otherFilterSpec
  : OTHERFILTER '('  CONST_STRING  ')'
  ;

fragment DIGIT : '0'..'9';
fragment LOWER  : 'a'..'z';
fragment UPPER  : 'A'..'Z';

CONST_STRING : '' (LOWER|UPPER|DIGIT)* '';
---
parsed ok.

but the following:
---
 otherFilterSpec
  : OTHERFILTER '('  CONST_STRING  ')'
  ;

constString: '  '  (LOWER|UPPER|DIGIT)* '  ';


fragment DIGIT : '0'..'9';
fragment LOWER  : 'a'..'z';
fragment UPPER  : 'A'..'Z';

CONST_STRING : '' (LOWER|UPPER|DIGIT)* '';
---
failed.
I don't know why changing from rule to TOKEN works.
2011/2/16 Douglas Godfrey douglasgodf...@gmail.com

 How about an example Symbol Table class that can handle C++ or Java style
 languages
 with nested scopes and drill down access to nested classes via name spaces
 and compound names.

 On Tue, Feb 15, 2011 at 6:36 PM, Terence Parr pa...@cs.usfca.edu wrote:

  Howdy. I'm thinking about revising the ref guide for ANTLR 3 (ANTLR v4
  might be awhile so I should update book).  Any suggestions to improve?
  One
  obvious thing: discuss not just java target :)
 
  Ter
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe:
  http://www.antlr.org/mailman/options/antlr-interest/your-email-address
 

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe:
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31479] Re: [antlr-interest] what would ANTLR ref guide revised edition have?

[devdoer bird]

Sorry ,I changed grammar as below with red font.
2011/2/16 devdoer bird devdo...@gmail.com

 Thanks.

 But I found it also failed in eclipse plugin.

 I tried the following grammar.
 ---
 otherFilterSpec
   : OTHERFILTER '('  CONST_STRING  ')'
   ;

 fragment DIGIT : '0'..'9';
 fragment LOWER  : 'a'..'z';
 fragment UPPER  : 'A'..'Z';

 CONST_STRING : '' (LOWER|UPPER|DIGIT)* '';
 ---
 parsed ok.

 but the following:
 ---
  otherFilterSpec
   : OTHERFILTER '('   *constString *  ')' // sorry,
;

 constString: '  '  (LOWER|UPPER|DIGIT)* '  ';


 fragment DIGIT : '0'..'9';
 fragment LOWER  : 'a'..'z';
 fragment UPPER  : 'A'..'Z';

 CONST_STRING : '' (LOWER|UPPER|DIGIT)* '';
 ---
 failed.
 I don't know why changing from rule to TOKEN works.
 2011/2/16 Douglas Godfrey douglasgodf...@gmail.com

 How about an example Symbol Table class that can handle C++ or Java style
 languages
 with nested scopes and drill down access to nested classes via name spaces
 and compound names.

 On Tue, Feb 15, 2011 at 6:36 PM, Terence Parr pa...@cs.usfca.edu wrote:

  Howdy. I'm thinking about revising the ref guide for ANTLR 3 (ANTLR v4
  might be awhile so I should update book).  Any suggestions to improve?
  One
  obvious thing: discuss not just java target :)
 
  Ter
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe:
  http://www.antlr.org/mailman/options/antlr-interest/your-email-address
 

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe:
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address




List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31480] [antlr-interest] NPE when adding @header? (ANTLRWorks 1.4.2, Mac OSX 10.6.6)

[Bill Schongar]

Hi folks,

 I¹m just getting started with Antlr, and have a basic grammar working.
However, as soon as I add an @header declaration I get an NPE when I go to
Debug it, almost as if I hadn¹t properly terminated the @header declaration.

 If I remove @header everything works fine, and I can run debug with my
sample text, get my expected output, etc.

 Any thoughts on what I¹m doing wrong? System info, relevant section of the
grammar and NPE are added at the end for reference.

Thanks,
-Bill



System info:
* AntlrWorks 1.4.2
* Compiler preference setting: javac (in /usr/bin/javac, 1.6.0_22)
* Mac OSX 10.6.6

-- Excerpt of the grammar file ---
grammar TestFile;

@header {package com.mycompany.foo;}

options {
language = Java;
}

tokens {
...



[08:56:45] error(100): TestFile.g:5:1: syntax error: antlr: TestFile.g:5:1:
unexpected token: options {
[08:56:45] error(150):  grammar file TestFile.g has no rules
[08:56:45] error(100): TestFile.g:0:0: syntax error: assign.types:
AST:0:0: unexpected end of subtree
[08:56:45] error(100): TestFile.g:0:0: syntax error: define: AST:0:0:
unexpected end of subtree
[08:56:45] java.lang.NullPointerException
at 
org.antlr.grammar.v2.DefineGrammarItemsWalker.trimGrammar(DefineGrammarItems
Walker.java:94)
at 
org.antlr.grammar.v2.DefineGrammarItemsWalker.finish(DefineGrammarItemsWalke
r.java:77)
at 
org.antlr.grammar.v2.DefineGrammarItemsWalker.grammar(DefineGrammarItemsWalk
er.java:206)
at org.antlr.tool.Grammar.defineGrammarSymbols(Grammar.java:707)
at 
org.antlr.tool.CompositeGrammar.defineGrammarSymbols(CompositeGrammar.java:3
52)
at org.antlr.tool.Grammar.setGrammarContent(Grammar.java:584)
at 
org.antlr.works.grammar.antlr.ANTLRGrammarEngineImpl.createNewGrammar(ANTLRG
rammarEngineImpl.java:192)
at 
org.antlr.works.grammar.antlr.ANTLRGrammarEngineImpl.createParserGrammar(ANT
LRGrammarEngineImpl.java:225)
at 
org.antlr.works.grammar.antlr.ANTLRGrammarEngineImpl.createCombinedGrammar(A
NTLRGrammarEngineImpl.java:203)
at 
org.antlr.works.grammar.antlr.ANTLRGrammarEngineImpl.createGrammars(ANTLRGra
mmarEngineImpl.java:165)
at 
org.antlr.works.grammar.engine.GrammarEngineImpl.getGrammarLanguage(GrammarE
ngineImpl.java:115)
at 
org.antlr.works.components.GrammarWindowMenu.getEditTestRigTitle(GrammarWind
owMenu.java:244)
at 
org.antlr.works.components.GrammarWindowMenu.menuItemState(GrammarWindowMenu
.java:529)
at 
org.antlr.works.components.GrammarWindow.menuItemState(GrammarWindow.java:44
0)
at 
org.antlr.xjlib.appkit.menu.XJMainMenuBar.refreshMenuItemState(XJMainMenuBar
.java:175)
at 
org.antlr.xjlib.appkit.menu.XJMainMenuBar.refreshMenuState(XJMainMenuBar.jav
a:169)
at 
org.antlr.xjlib.appkit.menu.XJMainMenuBar.refreshState(XJMainMenuBar.java:15
3)
at 
org.antlr.xjlib.appkit.menu.XJMainMenuBar.refresh(XJMainMenuBar.java:145)
at 
org.antlr.xjlib.appkit.frame.XJWindow.windowActivated(XJWindow.java:200)
at 
org.antlr.works.components.GrammarWindow.windowActivated(GrammarWindow.java:
413)
at 
org.antlr.xjlib.appkit.frame.XJFrame$3.windowActivated(XJFrame.java:161)
at 
java.awt.AWTEventMulticaster.windowActivated(AWTEventMulticaster.java:372)
at java.awt.Window.processWindowEvent(Window.java:1877)
at javax.swing.JFrame.processWindowEvent(JFrame.java:274)
at java.awt.Window.processEvent(Window.java:1823)
at java.awt.Component.dispatchEventImpl(Component.java:4714)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Window.dispatchEventImpl(Window.java:2478)
at java.awt.Component.dispatchEvent(Component.java:4544)
at 
java.awt.KeyboardFocusManager.redispatchEvent(KeyboardFocusManager.java:1850
)
at 
java.awt.DefaultKeyboardFocusManager.typeAheadAssertions(DefaultKeyboardFocu
sManager.java:910)
at 
java.awt.DefaultKeyboardFocusManager.dispatchEvent(DefaultKeyboardFocusManag
er.java:409)
at java.awt.Component.dispatchEventImpl(Component.java:4586)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Window.dispatchEventImpl(Window.java:2478)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:635)
at java.awt.SentEvent.dispatch(SentEvent.java:55)
at 
java.awt.DefaultKeyboardFocusManager$DefaultKeyboardFocusManagerSentEvent.di
spatch(DefaultKeyboardFocusManager.java:183)
at 
java.awt.DefaultKeyboardFocusManager.sendMessage(DefaultKeyboardFocusManager
.java:210)
at 
java.awt.DefaultKeyboardFocusManager.dispatchEvent(DefaultKeyboardFocusManag
er.java:286)
at java.awt.Component.dispatchEventImpl(Component.java:4586)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Window.dispatchEventImpl(Window.java:2478)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:635)
at 

[il-antlr-interest: 31481] [antlr-interest] ANTLRWorks interpretter bug

[Tristan Aubrey-Jones]

Hi there, 

I've been trying the Expression evaluator tutorial at
http://www.antlr.org/works/help/tutorial/calculator.html [1] and am getting
a MismatchedTokenException(10!=4) when parsing strings containing
subtraction like 1-2n in the interpreter. The grammar works fine when I
generate and debug but in the interpreter it fails. If I swap the '+' and
'-' expressions the same problem occurs with the '+' instead of the '-' so
its positional. Any ideas? 

Thanks, 

Tristan

Links:
--
[1]
http://www.antlr.org/works/help/tutorial/calculator.html

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31482] Re: [antlr-interest] Compile errors with CSharp2 Target

[Sam Harwell]

Hi Simon,

Using the 3.3.1 build I posted last night, the only issue with the C# target
is the use of {$channel=HIDDEN;}. To match .NET naming conventions, the
hidden channel constant is Hidden, so they need to be changed to
{$channel=Hidden;}.

I updated the grammar you sent by:

1. Fixing the CSharp3 build issue (requires the 3.3.1 build I posted last
night).
2. Left factoring to remove the backtrack, memoize, and k=2 options.
3. Using the extended AST operators of the CSharp3 target to remove
unnecessary rewrite rules.

I haven't tested the result aside from making sure it compiles properly with
the CSharp3 target.

Sam

-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Simon
Sent: Wednesday, February 16, 2011 1:42 AM
To: antlr-interest@antlr.org
Subject: Re: [antlr-interest] Compile errors with CSharp2 Target

On Wed, Feb 16, 2011 at 10:47 AM, Johannes Luber jalu...@gmx.de wrote:
 Did you read the other emails stating you may have a grammar which is
meant to work with Java? If possible pls attach the grammar to the email, so
we can check this and any other errors. I forgot which ANTLR versions
require what runtimes exactly, but you need 3.1 or 3.2 for CSharp2. CSharp3
requires ANTLR 3.3 and other assemblies than I've provided, as the targets
aren't compatible with each other yet. Not sure if those assemblies are
provided in distro or where else you can find them. Sam should know more.

Ah, thanks to everyone for pointing out the obvious - I looked more
thoroughly in the grammar and in the end did find some stray bits of Java
code which, once eliminated let it compile just fine using
CSharp2.   I still have the same errors with CSharp3 (incorrect escape
sequences when generated with antlr 3.1.3, different errors with antlr
3.3).   I've attached the purified Php.g grammar that succeeds in
CSharp2 but fails in CSharp3 in case it is useful - however I think I will
proceed with CSharp2 at this point.

Thanks again for the help!

Simon

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address


Php.g
Description: Binary data
-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31483] Re: [antlr-interest] NPE when adding @header? (ANTLRWorks 1.4.2, Mac OSX 10.6.6)

[Sam Harwell]

Hi Bill,

You need to place the @header block after the options{} block.

Sam

-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Bill Schongar
Sent: Wednesday, February 16, 2011 8:04 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] NPE when adding @header? (ANTLRWorks 1.4.2, Mac
OSX 10.6.6)

Hi folks,

 I¹m just getting started with Antlr, and have a basic grammar working.
However, as soon as I add an @header declaration I get an NPE when I go to
Debug it, almost as if I hadn¹t properly terminated the @header declaration.

 If I remove @header everything works fine, and I can run debug with my
sample text, get my expected output, etc.

 Any thoughts on what I¹m doing wrong? System info, relevant section of the
grammar and NPE are added at the end for reference.

Thanks,
-Bill



List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31484] Re: [antlr-interest] NPE when adding @header? (ANTLRWorks 1.4.2, Mac OSX 10.6.6)

[Bill Schongar]

D'oh! I knew I was doing something stupid.. Making sure I put it after
options and tokens fixes it in the combined grammar.

Thanks Sam!
-Bill



On 2/16/11 10:27 AM, Sam Harwell sharw...@pixelminegames.com wrote:

 Hi Bill,
 
 You need to place the @header block after the options{} block.
 
 Sam
 
 -Original Message-
 From: antlr-interest-boun...@antlr.org
 [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Bill Schongar
 Sent: Wednesday, February 16, 2011 8:04 AM
 To: antlr-interest@antlr.org
 Subject: [antlr-interest] NPE when adding @header? (ANTLRWorks 1.4.2, Mac
 OSX 10.6.6)
 
 Hi folks,
 
  I¹m just getting started with Antlr, and have a basic grammar working.
 However, as soon as I add an @header declaration I get an NPE when I go to
 Debug it, almost as if I hadn¹t properly terminated the @header declaration.
 
  If I remove @header everything works fine, and I can run debug with my
 sample text, get my expected output, etc.
 
  Any thoughts on what I¹m doing wrong? System info, relevant section of the
 grammar and NPE are added at the end for reference.
 
 Thanks,
 -Bill
 
 


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31485] [antlr-interest] MissingTokenException

[Wilbur Lang]

Hi,

I'm working on ANTLR to create a parser but failed with 
MissingTokenException. Following is the grammar:


grammar test1;

moduleId
 :TYPEREFERENCE
 ;

WS  :   ( ' '| '\t'| '\r'| '\n') {$channel=HIDDEN;}
 ;

fragment UPPER :('A'..'Z') ;
fragment LOWER :('a'..'z') ;
fragment DIGIT :('0'..'9') ;
HYPHEN :'-' ;

CAPITALREFERENCE
 :UPPER (UPPER|DIGIT)* (HYPHEN (UPPER|DIGIT)+)*
 ;

TYPEREFERENCE
 :UPPER (UPPER|LOWER|DIGIT)* (HYPHEN (UPPER|LOWER|DIGIT)+)*
 ;

When I input sentence 'T3', ANTLR failed with 'MissingTokenException'. 
When I comment 'CAPITALREFERENCE', ANTLR worked. How can I correct the 
grammar?

Please note, I need 'CAPITALREFERENCE' to distinguish such as 'Tabc-01' 
and 'TABC-01'.

Thanks.

Wilbur Lang

--
Ovi Mail: Making email access easy
http://mail.ovi.com


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31486] Re: [antlr-interest] MissingTokenException

[Kevin J. Cummings]

On 02/16/2011 10:38 AM, Wilbur Lang wrote:
 Hi,
 
 I'm working on ANTLR to create a parser but failed with 
 MissingTokenException. Following is the grammar:
 
 
 grammar test1;
 
 moduleId
  :TYPEREFERENCE
  ;
 
 WS  :   ( ' '| '\t'| '\r'| '\n') {$channel=HIDDEN;}
  ;
 
 fragment UPPER :('A'..'Z') ;
 fragment LOWER :('a'..'z') ;
 fragment DIGIT :('0'..'9') ;
 HYPHEN :'-' ;
 
 CAPITALREFERENCE
  :UPPER (UPPER|DIGIT)* (HYPHEN (UPPER|DIGIT)+)*
  ;
 
 TYPEREFERENCE
  :UPPER (UPPER|LOWER|DIGIT)* (HYPHEN (UPPER|LOWER|DIGIT)+)*
  ;
 
 When I input sentence 'T3', ANTLR failed with 'MissingTokenException'. 
 When I comment 'CAPITALREFERENCE', ANTLR worked. How can I correct the 
 grammar?

I'd do the following:

fragment TYPEREFERENCE : ;

CAPITALREFERENCE
: UPPER (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })*
 (HYPHEN (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })+ )*
;

That way you'll only change the token type to TYPEREFERENCE if 1 or more
LOWER characters are found

 Please note, I need 'CAPITALREFERENCE' to distinguish such as 'Tabc-01' 
 and 'TABC-01'.
 
 Thanks.
 
 Wilbur Lang

-- 
Kevin J. Cummings
kjch...@verizon.net
cummi...@kjchome.homeip.net
cummi...@kjc386.framingham.ma.us
Registered Linux User #1232 (http://counter.li.org)

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31487] Re: [antlr-interest] what would ANTLR ref guide revised edition have?

[Terence Parr]

Hi. you mean just like the LIP book does?
Ter
On Feb 16, 2011, at 1:37 AM, Douglas Godfrey wrote:

 How about an example Symbol Table class that can handle C++ or Java style 
 languages 
 with nested scopes and drill down access to nested classes via name spaces 
 and compound names.
 
 On Tue, Feb 15, 2011 at 6:36 PM, Terence Parr pa...@cs.usfca.edu wrote:
 Howdy. I'm thinking about revising the ref guide for ANTLR 3 (ANTLR v4 might 
 be awhile so I should update book).  Any suggestions to improve?  One obvious 
 thing: discuss not just java target :)
 
 Ter
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: 
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address
 


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31488] Re: [antlr-interest] MissingTokenException

[Wilbur Lang]

On 2011/2/16 23:54, Kevin J. Cummings wrote:
 On 02/16/2011 10:38 AM, Wilbur Lang wrote:
 moduleId
   :TYPEREFERENCE
   ;

 I'd do the following:

 fragment TYPEREFERENCE : ;

 CAPITALREFERENCE
  : UPPER (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })*
   (HYPHEN (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })+ )*
  ;

 That way you'll only change the token type to TYPEREFERENCE if 1 or more
 LOWER characters are found
Thanks Kevin.

I rechecked the grammar, it not a context-free one, and it's impossible 
to disambiguate 'T3' is a CAPTIALREFERENCE or TYPEREFERENCE. Maybe I 
should check the 'semantic predicated'.

Wilbur Lang

--
Ovi Mail: Making email access easy
http://mail.ovi.com


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31489] Re: [antlr-interest] MissingTokenException

[Jim Idle]

Kevin's suggestion is the best way to do this as it involves no
predicates.

Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of Wilbur Lang
 Sent: Wednesday, February 16, 2011 9:14 AM
 To: antlr-interest@antlr.org
 Subject: Re: [antlr-interest] MissingTokenException

 On 2011/2/16 23:54, Kevin J. Cummings wrote:
  On 02/16/2011 10:38 AM, Wilbur Lang wrote:
  moduleId
:TYPEREFERENCE
;
 
  I'd do the following:
 
  fragment TYPEREFERENCE : ;
 
  CAPITALREFERENCE
   : UPPER (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })*
(HYPHEN (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })+ )*
   ;
 
  That way you'll only change the token type to TYPEREFERENCE if 1 or
  more LOWER characters are found
 Thanks Kevin.

 I rechecked the grammar, it not a context-free one, and it's impossible
 to disambiguate 'T3' is a CAPTIALREFERENCE or TYPEREFERENCE. Maybe I
 should check the 'semantic predicated'.

 Wilbur Lang

 --
 Ovi Mail: Making email access easy
 http://mail.ovi.com


 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31490] [antlr-interest] Context Sensitive Keyword Support?

[chris king]

Does ANTLR support context sensitive keywords? For example, in C# where is
a keyword only in a LINQ expression. To support this ANTLR would need to
change the lexer behavior given feedback from the parser letting it know
what context it is in. From what I hear ANTLR lexes the entire file before
passing it onto the parser. If that's true than I guess it doesn't support
this language concept. In that case, is there a way to work around?

Thanks,
Chris

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31491] Re: [antlr-interest] Context Sensitive Keyword Support?

[Jim Idle]

No you do this in the parser by allowing the keywords where ID is expected
then by predicating the WHERE keyword in the LINQ rules:


 ((WHERE)=where_clause)? 


id : ID | WHERE | 


I have fully working C# lexer, parser, tree walker if that is what you are
trying to build. It is a commercial product though.

Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of chris king
 Sent: Wednesday, February 16, 2011 1:32 PM
 To: antlr-interest@antlr.org
 Subject: [antlr-interest] Context Sensitive Keyword Support?

 Does ANTLR support context sensitive keywords? For example, in C#
 where is a keyword only in a LINQ expression. To support this ANTLR
 would need to change the lexer behavior given feedback from the parser
 letting it know what context it is in. From what I hear ANTLR lexes the
 entire file before passing it onto the parser. If that's true than I
 guess it doesn't support this language concept. In that case, is there
 a way to work around?

 Thanks,
 Chris

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31492] Re: [antlr-interest] Context Sensitive Keyword Support?

[Scott Stanchfield]

Has anyone looked at the context-sensitive-scanning proposal I sent a
while back?

http://javadude.com/articles/antlr-context-sensitive-scanner.html

-- Scott


Scott Stanchfield
http://javadude.com



On Wed, Feb 16, 2011 at 4:38 PM, Jim Idle j...@temporal-wave.com wrote:
 No you do this in the parser by allowing the keywords where ID is expected
 then by predicating the WHERE keyword in the LINQ rules:


  ((WHERE)=where_clause)? 


 id : ID | WHERE | 


 I have fully working C# lexer, parser, tree walker if that is what you are
 trying to build. It is a commercial product though.

 Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of chris king
 Sent: Wednesday, February 16, 2011 1:32 PM
 To: antlr-interest@antlr.org
 Subject: [antlr-interest] Context Sensitive Keyword Support?

 Does ANTLR support context sensitive keywords? For example, in C#
 where is a keyword only in a LINQ expression. To support this ANTLR
 would need to change the lexer behavior given feedback from the parser
 letting it know what context it is in. From what I hear ANTLR lexes the
 entire file before passing it onto the parser. If that's true than I
 guess it doesn't support this language concept. In that case, is there
 a way to work around?

 Thanks,
 Chris

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: 
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31493] Re: [antlr-interest] Context Sensitive Keyword Support?

[Terence Parr]

hiya. yep, i had a similar proposal way back when for v3.0 but I abandoned; 
can't remember why at this point. i love idea but impl had some issues.
ter
On Feb 16, 2011, at 2:02 PM, Scott Stanchfield wrote:

 Has anyone looked at the context-sensitive-scanning proposal I sent a
 while back?
 
http://javadude.com/articles/antlr-context-sensitive-scanner.html
 
 -- Scott
 
 
 Scott Stanchfield
 http://javadude.com
 
 
 
 On Wed, Feb 16, 2011 at 4:38 PM, Jim Idle j...@temporal-wave.com wrote:
 No you do this in the parser by allowing the keywords where ID is expected
 then by predicating the WHERE keyword in the LINQ rules:
 
 
  ((WHERE)=where_clause)? 
 
 
 id : ID | WHERE | 
 
 
 I have fully working C# lexer, parser, tree walker if that is what you are
 trying to build. It is a commercial product though.
 
 Jim
 
 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of chris king
 Sent: Wednesday, February 16, 2011 1:32 PM
 To: antlr-interest@antlr.org
 Subject: [antlr-interest] Context Sensitive Keyword Support?
 
 Does ANTLR support context sensitive keywords? For example, in C#
 where is a keyword only in a LINQ expression. To support this ANTLR
 would need to change the lexer behavior given feedback from the parser
 letting it know what context it is in. From what I hear ANTLR lexes the
 entire file before passing it onto the parser. If that's true than I
 guess it doesn't support this language concept. In that case, is there
 a way to work around?
 
 Thanks,
 Chris
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: 
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address
 
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: 
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31494] Re: [antlr-interest] Context Sensitive Keyword Support?

[Scott Stanchfield]

If I get a chance I'll play around with it sometime. When I was
thinking of it a while back it sounded like it should fit in pretty
nicely, but there may have been some evil little impl detail I didn't
consider...
-- Scott


Scott Stanchfield
http://javadude.com



On Wed, Feb 16, 2011 at 5:08 PM, Terence Parr pa...@cs.usfca.edu wrote:
 hiya. yep, i had a similar proposal way back when for v3.0 but I abandoned; 
 can't remember why at this point. i love idea but impl had some issues.
 ter
 On Feb 16, 2011, at 2:02 PM, Scott Stanchfield wrote:

 Has anyone looked at the context-sensitive-scanning proposal I sent a
 while back?

    http://javadude.com/articles/antlr-context-sensitive-scanner.html

 -- Scott

 
 Scott Stanchfield
 http://javadude.com



 On Wed, Feb 16, 2011 at 4:38 PM, Jim Idle j...@temporal-wave.com wrote:
 No you do this in the parser by allowing the keywords where ID is expected
 then by predicating the WHERE keyword in the LINQ rules:


  ((WHERE)=where_clause)? 


 id : ID | WHERE | 


 I have fully working C# lexer, parser, tree walker if that is what you are
 trying to build. It is a commercial product though.

 Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of chris king
 Sent: Wednesday, February 16, 2011 1:32 PM
 To: antlr-interest@antlr.org
 Subject: [antlr-interest] Context Sensitive Keyword Support?

 Does ANTLR support context sensitive keywords? For example, in C#
 where is a keyword only in a LINQ expression. To support this ANTLR
 would need to change the lexer behavior given feedback from the parser
 letting it know what context it is in. From what I hear ANTLR lexes the
 entire file before passing it onto the parser. If that's true than I
 guess it doesn't support this language concept. In that case, is there
 a way to work around?

 Thanks,
 Chris

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: 
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: 
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: 
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31495] Re: [antlr-interest] Context Sensitive Keyword Support?

[Jim Idle]

I think that every time we talk about this, we realize that the problems
lie in lookahead where you cannot decide what to do in certain cases. I
think that the quantum token idea is a much better one in that a token can
simultaneously be ID and WHERE or any other token that it is flagged as
being possible to be. This removes context from the lexer and allows the
parser to decide.

Predicates would still work and any other ambiguity should work out? It
means that input.LA(n) would also need

input.LA(n, TYPE)
  and perhaps
input.LT(n).getType(TYPE)

which would return TYPE if LT(n)/LA(n) can be one, otherwise the main type
that it actually is. Could have Boolean versions instead/as well of
course.

WHERE :: ID, KEYWORD : 'WHERE' ;

Or something like that. I think that this is well worth looking in to.



Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of Terence Parr
 Sent: Wednesday, February 16, 2011 2:09 PM
 To: antlr-interest Interest
 Subject: Re: [antlr-interest] Context Sensitive Keyword Support?

 hiya. yep, i had a similar proposal way back when for v3.0 but I
 abandoned; can't remember why at this point. i love idea but impl had
 some issues.
 ter
 On Feb 16, 2011, at 2:02 PM, Scott Stanchfield wrote:

  Has anyone looked at the context-sensitive-scanning proposal I sent a
  while back?
 
 http://javadude.com/articles/antlr-context-sensitive-scanner.html
 
  -- Scott
 
  
  Scott Stanchfield
  http://javadude.com
 
 
 
  On Wed, Feb 16, 2011 at 4:38 PM, Jim Idle j...@temporal-wave.com
 wrote:
  No you do this in the parser by allowing the keywords where ID is
  expected then by predicating the WHERE keyword in the LINQ rules:
 
 
   ((WHERE)=where_clause)? 
 
 
  id : ID | WHERE | 
 
 
  I have fully working C# lexer, parser, tree walker if that is what
  you are trying to build. It is a commercial product though.
 
  Jim
 
  -Original Message-
  From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
  boun...@antlr.org] On Behalf Of chris king
  Sent: Wednesday, February 16, 2011 1:32 PM
  To: antlr-interest@antlr.org
  Subject: [antlr-interest] Context Sensitive Keyword Support?
 
  Does ANTLR support context sensitive keywords? For example, in C#
  where is a keyword only in a LINQ expression. To support this
  ANTLR would need to change the lexer behavior given feedback from
  the parser letting it know what context it is in. From what I hear
  ANTLR lexes the entire file before passing it onto the parser. If
  that's true than I guess it doesn't support this language concept.
  In that case, is there a way to work around?
 
  Thanks,
  Chris
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe:
  http://www.antlr.org/mailman/options/antlr-interest/your-
  email-address
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe:
  http://www.antlr.org/mailman/options/antlr-interest/your-email-
 addres
  s
 
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe:
  http://www.antlr.org/mailman/options/antlr-interest/your-email-
 address


 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31496] [antlr-interest] CSharp tool port

[chris king]

Does anyone know if this is the latest version of *Sam Harwell *C# port or
the Antlr tool?

*http://www.280z28.org/downloads/antlr/antlr-dotnet-binaries-3.2.0.6805.7z*


Thanks,
Chris

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31497] Re: [antlr-interest] CSharp tool port

[Sam Harwell]

Hi Chris,

I just sent an email last night about a new release. :) The links are broken
on that page due to word wrap - just copy everything within the angle braces
(...) and remove the newline to get the correct link.

http://www.antlr.org/pipermail/antlr-interest/2011-February/040767.html

Thanks,
Sam

-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of chris king
Sent: Wednesday, February 16, 2011 4:52 PM
To: antlr-interest@antlr.org
Subject: [antlr-interest] CSharp tool port

Does anyone know if this is the latest version of *Sam Harwell *C# port or
the Antlr tool?

*http://www.280z28.org/downloads/antlr/antlr-dotnet-binaries-3.2.0.6805.7z*


Thanks,
Chris

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31498] Re: [antlr-interest] Context Sensitive Keyword Support?

[Terence Parr]

I also want to try a scannerless version of ANTLR.  Rats guy Robert Grimm has 
groovy combined language / modularity feature.
ter
On Feb 16, 2011, at 2:21 PM, Jim Idle wrote:

 I think that every time we talk about this, we realize that the problems
 lie in lookahead where you cannot decide what to do in certain cases. I
 think that the quantum token idea is a much better one in that a token can
 simultaneously be ID and WHERE or any other token that it is flagged as
 being possible to be. This removes context from the lexer and allows the
 parser to decide.
 
 Predicates would still work and any other ambiguity should work out? It
 means that input.LA(n) would also need
 
 input.LA(n, TYPE)
  and perhaps
 input.LT(n).getType(TYPE)
 
 which would return TYPE if LT(n)/LA(n) can be one, otherwise the main type
 that it actually is. Could have Boolean versions instead/as well of
 course.
 
 WHERE :: ID, KEYWORD : 'WHERE' ;
 
 Or something like that. I think that this is well worth looking in to.
 
 
 
 Jim
 
 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of Terence Parr
 Sent: Wednesday, February 16, 2011 2:09 PM
 To: antlr-interest Interest
 Subject: Re: [antlr-interest] Context Sensitive Keyword Support?
 
 hiya. yep, i had a similar proposal way back when for v3.0 but I
 abandoned; can't remember why at this point. i love idea but impl had
 some issues.
 ter
 On Feb 16, 2011, at 2:02 PM, Scott Stanchfield wrote:
 
 Has anyone looked at the context-sensitive-scanning proposal I sent a
 while back?
 
   http://javadude.com/articles/antlr-context-sensitive-scanner.html
 
 -- Scott
 
 
 Scott Stanchfield
 http://javadude.com
 
 
 
 On Wed, Feb 16, 2011 at 4:38 PM, Jim Idle j...@temporal-wave.com
 wrote:
 No you do this in the parser by allowing the keywords where ID is
 expected then by predicating the WHERE keyword in the LINQ rules:
 
 
  ((WHERE)=where_clause)? 
 
 
 id : ID | WHERE | 
 
 
 I have fully working C# lexer, parser, tree walker if that is what
 you are trying to build. It is a commercial product though.
 
 Jim
 
 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of chris king
 Sent: Wednesday, February 16, 2011 1:32 PM
 To: antlr-interest@antlr.org
 Subject: [antlr-interest] Context Sensitive Keyword Support?
 
 Does ANTLR support context sensitive keywords? For example, in C#
 where is a keyword only in a LINQ expression. To support this
 ANTLR would need to change the lexer behavior given feedback from
 the parser letting it know what context it is in. From what I hear
 ANTLR lexes the entire file before passing it onto the parser. If
 that's true than I guess it doesn't support this language concept.
 In that case, is there a way to work around?
 
 Thanks,
 Chris
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe:
 http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe:
 http://www.antlr.org/mailman/options/antlr-interest/your-email-
 addres
 s
 
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe:
 http://www.antlr.org/mailman/options/antlr-interest/your-email-
 address
 
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31499] Re: [antlr-interest] Context Sensitive Keyword Support?

[Jim Idle]

Yeah - I am skeptical of the performance of scannerless parsing, but if
performance is not the issue, it can help a lot here. I think that the
rats parsing technique also ends up with weak error reporting though I
think it records information to help with this right?

Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of Terence Parr
 Sent: Wednesday, February 16, 2011 3:43 PM
 To: antlr-interest Interest
 Subject: Re: [antlr-interest] Context Sensitive Keyword Support?

 I also want to try a scannerless version of ANTLR.  Rats guy Robert
 Grimm has groovy combined language / modularity feature.
 ter
 On Feb 16, 2011, at 2:21 PM, Jim Idle wrote:

  I think that every time we talk about this, we realize that the
  problems lie in lookahead where you cannot decide what to do in
  certain cases. I think that the quantum token idea is a much better
  one in that a token can simultaneously be ID and WHERE or any other
  token that it is flagged as being possible to be. This removes
 context
  from the lexer and allows the parser to decide.
 
  Predicates would still work and any other ambiguity should work out?
  It means that input.LA(n) would also need
 
  input.LA(n, TYPE)
   and perhaps
  input.LT(n).getType(TYPE)
 
  which would return TYPE if LT(n)/LA(n) can be one, otherwise the main
  type that it actually is. Could have Boolean versions instead/as well
  of course.
 
  WHERE :: ID, KEYWORD : 'WHERE' ;
 
  Or something like that. I think that this is well worth looking in
 to.
 
 
 
  Jim
 
  -Original Message-
  From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
  boun...@antlr.org] On Behalf Of Terence Parr
  Sent: Wednesday, February 16, 2011 2:09 PM
  To: antlr-interest Interest
  Subject: Re: [antlr-interest] Context Sensitive Keyword Support?
 
  hiya. yep, i had a similar proposal way back when for v3.0 but I
  abandoned; can't remember why at this point. i love idea but impl
 had
  some issues.
  ter
  On Feb 16, 2011, at 2:02 PM, Scott Stanchfield wrote:
 
  Has anyone looked at the context-sensitive-scanning proposal I sent
  a while back?
 
http://javadude.com/articles/antlr-context-sensitive-scanner.html
 
  -- Scott
 
  
  Scott Stanchfield
  http://javadude.com
 
 
 
  On Wed, Feb 16, 2011 at 4:38 PM, Jim Idle j...@temporal-wave.com
  wrote:
  No you do this in the parser by allowing the keywords where ID is
  expected then by predicating the WHERE keyword in the LINQ rules:
 
 
   ((WHERE)=where_clause)? 
 
 
  id : ID | WHERE | 
 
 
  I have fully working C# lexer, parser, tree walker if that is what
  you are trying to build. It is a commercial product though.
 
  Jim
 
  -Original Message-
  From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
  boun...@antlr.org] On Behalf Of chris king
  Sent: Wednesday, February 16, 2011 1:32 PM
  To: antlr-interest@antlr.org
  Subject: [antlr-interest] Context Sensitive Keyword Support?
 
  Does ANTLR support context sensitive keywords? For example, in C#
  where is a keyword only in a LINQ expression. To support this
  ANTLR would need to change the lexer behavior given feedback from
  the parser letting it know what context it is in. From what I
 hear
  ANTLR lexes the entire file before passing it onto the parser. If
  that's true than I guess it doesn't support this language
 concept.
  In that case, is there a way to work around?
 
  Thanks,
  Chris
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe:
  http://www.antlr.org/mailman/options/antlr-interest/your-
  email-address
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe:
  http://www.antlr.org/mailman/options/antlr-interest/your-email-
  addres
  s
 
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe:
  http://www.antlr.org/mailman/options/antlr-interest/your-email-
  address
 
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe:
  http://www.antlr.org/mailman/options/antlr-interest/your-
  email-address


 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31500] Re: [antlr-interest] Context Sensitive Keyword Support?

[Sam Harwell]

That's actually on my list of things to discuss when I'm there. :)

Sam

-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Terence Parr
Sent: Wednesday, February 16, 2011 5:43 PM
To: antlr-interest Interest
Subject: Re: [antlr-interest] Context Sensitive Keyword Support?

I also want to try a scannerless version of ANTLR.  Rats guy Robert Grimm
has groovy combined language / modularity feature.
ter



List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31501] Re: [antlr-interest] Context Sensitive Keyword Support?

[Terence Parr]

I'd like to get expression parsing, unbuffered char streams implemented for v3 
soon. would be nice.

Ter
On Feb 16, 2011, at 4:33 PM, Sam Harwell wrote:

 That's actually on my list of things to discuss when I'm there. :)
 
 Sam
 
 -Original Message-
 From: antlr-interest-boun...@antlr.org
 [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Terence Parr
 Sent: Wednesday, February 16, 2011 5:43 PM
 To: antlr-interest Interest
 Subject: Re: [antlr-interest] Context Sensitive Keyword Support?
 
 I also want to try a scannerless version of ANTLR.  Rats guy Robert Grimm
 has groovy combined language / modularity feature.
 ter
 
 
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: 
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31502] Re: [antlr-interest] Context Sensitive Keyword Support?

[Michael Bedward]

On 17 February 2011 09:21, Jim Idle j...@temporal-wave.com wrote:
 I think that the quantum token idea is a much better one in that a token can
 simultaneously be ID and WHERE or any other token that it is flagged as
 being possible to be. This removes context from the lexer and allows the
 parser to decide.


Yes please !!!  This seems like a wonderfully elegant and very useful idea.

Michael

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31503] [antlr-interest] Just For Converting

[辛跃]

i want convert the Stream content like:

attribute1  attibute2   attribute3attribute4   
attribute5

 value1  value2 value3value4value5

 value1  value2 value3value4 value5

 vaue1  value2  value3value4 value5

my goal is store those tokens in the form of

(atribute1 value1) ( attribue2 value2) ( attribue3 value3) ... ...

(atribute1 value1) ( attribue2 value2) ( attribue3 value3) ... ...

(atribute1 value1) ( attribue2 value2) ( attribue3 value3) ... ...

so how could i write my grammer by antlr to reach my goal

thanks

 

 

 




List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31504] Re: [antlr-interest] Just For Converting

[Michael Bedward]

Hello,

For fixed-width or delimited data like these it is a lot easier to
write a program by hand than use ANTLR.

Michael

On 17 February 2011 15:53, 辛跃 xpwit...@163.com wrote:



 i want convert the Stream content like:

 attribute1      attibute2       attribute3        attribute4           
 attribute5

  value1          value2         value3            value4                value5

  value1          value2         value3            value4                 
 value5

  vaue1          value2          value3            value4                 
 value5

 my goal is store those tokens in the form of

 (atribute1 value1) ( attribue2 value2) ( attribue3 value3) ... ...

 (atribute1 value1) ( attribue2 value2) ( attribue3 value3) ... ...

 (atribute1 value1) ( attribue2 value2) ( attribue3 value3) ... ...

 so how could i write my grammer by antlr to reach my goal

 thanks










 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: 
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31505] [antlr-interest] How can I make sure there's one such rule in the input ?

[devdoer bird]

HI:

I have the grammar:

statements:(inputStatement|schemaStatement|filtersStatement|COMMENT)* EOF;


As you see ,it will match as many inputStatements as there are in the input
file.


How can change the grammar to  make sure there is only one inputStatment in
the input file?


Thanks.

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31506] Re: [antlr-interest] How can I make sure there's one such rule in the input ?

[Jim Idle]

You don't. You add a counter and parse everything that comes up, then you
issue a semantic error saying that there are too many inputStatements.
Always push the errors as far down the chain as you can as they gain more
context and are therefore clearer to the end user.

Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of devdoer bird
 Sent: Wednesday, February 16, 2011 9:05 PM
 To: antlr-interest Interest
 Subject: [antlr-interest] How can I make sure there's one such rule in
 the input ?

 HI:

 I have the grammar:

 statements:(inputStatement|schemaStatement|filtersStatement|COMMENT)*
 EOF;


 As you see ,it will match as many inputStatements as there are in the
 input file.


 How can change the grammar to  make sure there is only one
 inputStatment in the input file?


 Thanks.

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.

[il-antlr-interest: 31507] Re: [antlr-interest] How can I make sure there's one such rule in the input ?

[devdoer bird]

Thanks。

2011/2/17 Jim Idle j...@temporal-wave.com

 You don't. You add a counter and parse everything that comes up, then you
 issue a semantic error saying that there are too many inputStatements.
 Always push the errors as far down the chain as you can as they gain more
 context and are therefore clearer to the end user.

 Jim

  -Original Message-
  From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
  boun...@antlr.org] On Behalf Of devdoer bird
  Sent: Wednesday, February 16, 2011 9:05 PM
  To: antlr-interest Interest
  Subject: [antlr-interest] How can I make sure there's one such rule in
  the input ?
 
  HI:
 
  I have the grammar:
 
  statements:(inputStatement|schemaStatement|filtersStatement|COMMENT)*
  EOF;
 
 
  As you see ,it will match as many inputStatements as there are in the
  input file.
 
 
  How can change the grammar to  make sure there is only one
  inputStatment in the input file?
 
 
  Thanks.
 
  List: http://www.antlr.org/mailman/listinfo/antlr-interest
  Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
  email-address

 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe:
 http://www.antlr.org/mailman/options/antlr-interest/your-email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.