[il-antlr-interest: 35042] Re: [antlr-interest] Confused about backtracking

2011-11-28 Thread Kirby Bohling
On Mon, Nov 28, 2011 at 9:02 AM, Christian wrote: > Mh, I am with you. I also do not understand backtracking in this > situation ;) > > Could anyone else explain it, please? Pretty sure you are having problems because you have '.' as an inline/generated token in your rules. I've said it dozens o

[il-antlr-interest: 34804] Re: [antlr-interest] .g vs .g4

2011-11-08 Thread Kirby Bohling
If Sam is saying it is only for the searching on imported grammers, it sounds like the tooling won't care about the main one. Imported grammars already have a bunch of clunky aspects about their placement and naming (at least IMHO, when building them inside Maven). I've used them, and they have g

[il-antlr-interest: 34788] Re: [antlr-interest] .g vs .g4

2011-11-07 Thread Kirby Bohling
On Mon, Nov 7, 2011 at 12:01 PM, Terence Parr wrote: > yeah, maybe forcing tool to do .g4 is good otherwise why use it. .g4 means > v4. .g means v3 (er…and v2). > Ter Please don't do that. If I want to call that file "foobar.g", or "silly" what does it matter? Have a declaration inside of t

[il-antlr-interest: 34345] Re: [antlr-interest] Problem with autotools while using antlr3 C runtime

2011-10-11 Thread Kirby Bohling
Use no-define in the configure.ac file. AM_INIT_AUTOMAKE(PACKAGE, VERSION, [NO-DEFINE]) See: http://gcc.gnu.org/svn/gcc/branches/cilkplus/libstdc++-v3/configure.ac or http://www.delorie.com/gnu/docs/automake/automake_24.html I think this is a sign you or the other package are using the config.h

[il-antlr-interest: 33467] Re: [antlr-interest] Help with a parser

2011-08-02 Thread Kirby Bohling
By examination, I didn't run the grammar through ANTLR, I'm pretty sure this is your problem: FIELDNAME :KEYWORD ':' ; FIELDNAME is a Lexer Rule, and KEYWORD is your lexer rule. I'm pretty sure that is at least one of the problem. For me, every time ANTLR does this the easiest thing t

[il-antlr-interest: 32970] Re: [antlr-interest] No errors on missing tokens?

2011-06-28 Thread Kirby Bohling
omething really obvious. > > BTW, great reply.  Doesn't help me but it will help ANTLR newbies. > >        .bill > > On Jun 28, 2011, at 13:45 , Kirby Bohling wrote: > >> On Tue, Jun 28, 2011 at 12:34 PM, Bill Andersen >> wrote: >>> Partial answer t

[il-antlr-interest: 32588] Re: [antlr-interest] Non-deterministic behaviour in matching lexer tokens

2011-05-27 Thread Kirby Bohling
the type because I'm not sure what > parent > rule context I'm processing under.  The characters of the token alone cannot > tell me that. > > (option 2 - emit different token at parse time) > Same issue as option 1.  I will not be able to differentiate based on the > token

[il-antlr-interest: 32585] Re: [antlr-interest] Non-deterministic behaviour in matching lexer tokens

2011-05-27 Thread Kirby Bohling
'\f')+ >     {$channel = HIDDEN;} >     ; > > Then I parse "action myval" and receive this error: > > line 1:7 mismatched input 'myval' expecting VALUE > > Because the lexer cannot determine whether the token is IDENT or VALUE > my act

[il-antlr-interest: 32582] Re: [antlr-interest] Non-deterministic behaviour in matching lexer tokens

2011-05-27 Thread Kirby Bohling
First grammar: > VALUE: >    (LETTER | DIGIT)+ >    ; Second Grammar: > VALUE: >    (LETTER | DIGIT) '!'+ >    ; > action MYVAL!   (MismatchedTokenException: line 3:7 mismatched input 'MYVAL' You've got the rule in + in the wrong place. I'm pretty sure you meant: VALUE: (LETTER | DIGIT)+ '!'

[il-antlr-interest: 32343] Re: [antlr-interest] Error Handling Advanced

2011-04-29 Thread Kirby Bohling
Maybe it is easier using C#, but it always annoys me that in Java if I implement my own superclass, I have to implement two versions (and duplicate a bunch of goop). Hopefully addressing that is on the ANTLRv4 list. I thought Terrance said he had ideas for that in the future when it was brought

[il-antlr-interest: 32144] Re: [antlr-interest] Problem with MismatchedTokenException

2011-04-10 Thread Kirby Bohling
I'm not much of an expert, and I'm not really sure what you are attempting to accomplish. So it is really hard to advise you on how to resolve the problem in the cleanest way. Hence my comment about parroting common advice. One item about ANTLR to remember is that the lexer runs from start to fi

[il-antlr-interest: 32140] Re: [antlr-interest] Problem with MismatchedTokenException

2011-04-09 Thread Kirby Bohling
The problem is pretty simple there (but a sign you need to think everything through more). "test=toto" will generate the following Tokens: NAME GENERATED_EQUALS NAME Which your parser rules won't allow. "test=toto0" will generate the following Tokens: NAME GENERATED_EQUALS VALUE Which your par

[il-antlr-interest: 31980] Re: [antlr-interest] Problem with Simple Grammar : NoViableAltException

2011-03-27 Thread Kirby Bohling
Rory, I'm pretty sure the problem is that DIGIT is a fragment, but you are using it as part of a parser rule (I would have thought that this wouldn't be successfully parsed by ANTLR). My guess is that is at least part of the problem. You might also want to split up your lexer and grammar rules,

[il-antlr-interest: 31872] Re: [antlr-interest] help for URL grammar

2011-03-22 Thread Kirby Bohling
Pretty sure the generated tokens for the '-' in toplabel and domainlabel are conflicting. Try creating a '-' token (or potentially a fragment). If you are using literal text, I'd only use that as part of a lexer rule, never a parser rule like you are in those two rules. By the by, I don't think

[il-antlr-interest: 31595] Re: [antlr-interest] Token parsing speed

2011-02-24 Thread Kirby Bohling
On Thu, Feb 24, 2011 at 3:25 AM, Richard Druce wrote: > I have a question on the general best practice and speed between of > using  tokens vs rules to construct parts of the grammar. Our language > has many phrases that share words, a simplified sample being 'first' > and 'first second'.  Would I

[il-antlr-interest: 31470] Re: [antlr-interest] what would ANTLR ref guide revised edition have?

2011-02-15 Thread Kirby Bohling
On Tue, Feb 15, 2011 at 5:36 PM, Terence Parr wrote: > Howdy. I'm thinking about revising the ref guide for ANTLR 3 (ANTLR v4 might > be awhile so I should update book).  Any suggestions to improve?  One obvious > thing: discuss not just java target :) > > Ter > The thing I struggled with the m

[il-antlr-interest: 31412] Re: [antlr-interest] Ident with ending In or Out word

2011-02-11 Thread Kirby Bohling
On Fri, Feb 11, 2011 at 9:17 AM, Roohul wrote: > Sorry for that.. > > when i use only letters (a..z | A..Z) in the variables and connectors it give > an error while creating an AST that "no viable alternative at input 'real'" > and when creating variables with digits the grammar do not compile a

[il-antlr-interest: 31343] Re: [antlr-interest] How do I accept input ending with a newline *or* EOF?

2011-02-03 Thread Kirby Bohling
On Thu, Feb 3, 2011 at 6:00 PM, chris king wrote: > Kirby thanks! That helped a ton and thanks for that + vs * tip. A real life > saver. > I have another problem and I'm hoping you can point me in the right > direction. I'm trying to chose between two approaches for building for a > pre-processor.

[il-antlr-interest: 31294] Re: [antlr-interest] How do I accept input ending with a newline *or* EOF?

2011-01-31 Thread Kirby Bohling
On Mon, Jan 31, 2011 at 6:34 PM, Jim Idle wrote: > It is much better to add a NL to the end of the input if it is not present > as the ambiguities are complicated and pointless without that, but with > this simple example you can do: Jim, I've seen you make this suggestion several times. Is

[il-antlr-interest: 31292] Re: [antlr-interest] How do I accept input ending with a newline *or* EOF?

2011-01-31 Thread Kirby Bohling
No idea if it is related to the problem, but you likely really want to have ID use a '+' not a '*' after ('a'..'z'), otherwise ID to match nothing and be cause an infinite loop while lexing at points (generally speaking, any time rules like bar: (foo)*; foo: (baz)*; You are just asking for probl

[il-antlr-interest: 30460] Re: [antlr-interest] Manual Tree Walking Vs. Tree Grammars

2010-11-02 Thread Kirby Bohling
On Tue, Nov 2, 2010 at 5:59 PM, Patrick Niemeyer wrote: > > I had been building up the experience to post something thoughtful on this > topic, but I'll just chime in now :) > > I'd be open to being proved wrong on the tree grammar front... if anyone has > examples of how it simplified their co

[il-antlr-interest: 30216] Re: [antlr-interest] How to distinguish between an assembler comment and a literal integer

2010-09-28 Thread Kirby Bohling
Do the #'s that start a comment have to be the first character? Can there ever be a '#' that is a literal on the first character of the line? If those questions are respectively: Yes, and No, then you use a predicate to determine if the '#' is the first character of the line, and pick the token t

[il-antlr-interest: 29456] Re: [antlr-interest] Getting the Previously Matched Lexer Token in the C Target

2010-07-19 Thread Kirby Bohling
If you aren't going to lex that token to something else, I'm pretty sure the right solution is to just lex it as the invalid token. If you are going to lex it to something else, then likely you have the "keyword" vs. "id" problem (use one token for it in the lexer, and pick which one it really is

[il-antlr-interest: 29104] Re: [antlr-interest] Grammar not detecting extraneous input at end

2010-06-03 Thread Kirby Bohling
Channeling Jim Idle: Remember to use antlr.markmail.org, this is a very common question, and has been answered many times. More then likely you don't require "EOF" at the end of whatever your start rule is. The token stream more then likely has a ')' and if you tried to parse again, you'd get an

[il-antlr-interest: 28917] Re: [antlr-interest] enums in v4 ANTLR Java code generation considered useless

2010-05-19 Thread Kirby Bohling
ll inline > is truly evil. Compiler constant inlining can very easily lead to > incorrect constant values when a library (that provides a constant) > changes (new jar dropped in with a new value for the constant) but the > code using that library isn't recompiled. Safety issue. >

[il-antlr-interest: 28909] Re: [antlr-interest] enums in v4 ANTLR Java code generation considered useless

2010-05-19 Thread Kirby Bohling
On Wed, May 19, 2010 at 2:13 PM, Scott Stanchfield wrote: > Interesting point re common code generation approaches, but as far as > performance goes, it's equivalent - all == tests are done using > pointers, which are the same size as ints. If switch is used the > ordinal values of the enums are u

[il-antlr-interest: 28861] Re: [antlr-interest] Variants of a grammar

2010-05-17 Thread Kirby Bohling
On Mon, May 17, 2010 at 4:14 PM, Marcel Schaible wrote: > Hi, > > I am currently reworking some old big YACC grammar files. Each of them > (about 15 different YACC files with 5kB to 80Kb size) are describing a > command language for a specific hardware device and share a lot of > common rules. Now

[il-antlr-interest: 28517] Re: [antlr-interest] Comments parser and non-alphanum characters

2010-04-19 Thread Kirby Bohling
If you have control of the language, I'd change it to make it easier... If you don't, that's much harder. I'd parse it in two passes. One that handles as a single token, and one is feed the input for and parses it. That's been my plan on handling similar issues in a Wiki-like language. The o

[il-antlr-interest: 28610] Re: [antlr-interest] NoViableAlt

2010-04-13 Thread Kirby Bohling
Without actually running it, I'm pretty sure it's because '3' is lexing to be 'DIGIT' (it's the first rule, if DIGIT had been below NUM I think it might have "worked"). So the lexer returns 'DIGIT', which no rule from your parser accepts. You likely meant to declare the DIGIT as a fragment. At wh

[il-antlr-interest: 28268] Re: [antlr-interest] How to get ANTLR 3.2 to exit upon first error?

2010-03-15 Thread Kirby Bohling
You need to repeat some of that for the lexer. Using the @lexer::members syntax if you're going to do it as a combined lexer/parser grammar (I always separate mine to keep my mental working set smaller). If you don't do the lexer, you can have a lex error and recover from it, but this should catc

[il-antlr-interest: 28202] Re: [antlr-interest] Logic like ~ for parsing

2010-03-08 Thread Kirby Bohling
Kirby > But if you look through the ways you can resync the input in error recovery, > you may get better mileage. > > Jim > >> -Original Message- >> From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- >> boun...@antlr.org] On Behalf Of Kirby Bo

[il-antlr-interest: 28194] [antlr-interest] Logic like ~ for parsing

2010-03-07 Thread Kirby Bohling
All, I have a grammar that is pretty far along. But I really need a rule like this: dangling_match: (non_dangling_a | non_dangling_b) => // Failure to match, get back up to the foo level somehow. | DANGLING_TOKEN ; foo: (non_dangling_a | non_dangling_b | options_including_dangling

[il-antlr-interest: 28130] Re: [antlr-interest] Wiki Markup Like Grammar Example

2010-03-01 Thread Kirby Bohling
Eric, I have a grammar for Wikimedia that recognizes ~90% of the pages on Wikipedia. It probably needs quite a bit of tender loving care. I can't release it, because I did it on the company dime, and I haven't yet won the argument to release it. Sorry. The key question is: Do you expect

[il-antlr-interest: 28059] Re: [antlr-interest] Abstract tree parser

2010-02-23 Thread Kirby Bohling
On Tue, Feb 23, 2010 at 6:44 PM, Gerald Rosenberg wrote: > On 2/23/2010 4:04 PM, Kirby Bohling wrote: >> I wish that superClass option was smart about handling debug vs. >> non-debug, that's the one thing that annoys me any time I choose that >> option is that I can&#x

[il-antlr-interest: 28053] Re: [antlr-interest] Abstract tree parser

2010-02-23 Thread Kirby Bohling
I wish that superClass option was smart about handling debug vs. non-debug, that's the one thing that annoys me any time I choose that option is that I can't integrate with most of the cool debuggers without extra work. In the land of C++ where I can use templates, it's easy to resolve that, in Ja

[il-antlr-interest: 27721] Re: [antlr-interest] "prog : .+ ; " ==> "no viable alternative at character" (antlr-3.1.2)

2010-01-31 Thread Kirby Bohling
On Sun, Jan 31, 2010 at 4:46 PM, Valery Khamenya wrote: > Hi, > > what's wrong with the following trivial lexer grammar? > >  grammar Grammar; >  options { > language=Python; >  output=AST; > ASTLabelType=CommonTree; >  } >  prog : .+  ; > > I am getting "no viable alternative at character ..." at

[il-antlr-interest: 27252] Re: [antlr-interest] simple lexical analysis question

2009-12-14 Thread Kirby Bohling
erating: TOKEN_2, TOKEN_5, TOKEN_4, TOKEN_1 (which technically would match), unlike my previous example. Kirby On Mon, Dec 14, 2009 at 12:46 PM, Kirby Bohling wrote: > On Mon, Dec 14, 2009 at 12:18 PM, Jean-Claude Durand > wrote: >> My lexical grammar (I use antlr v3.2): >>

[il-antlr-interest: 27251] Re: [antlr-interest] simple lexical analysis question

2009-12-14 Thread Kirby Bohling
On Mon, Dec 14, 2009 at 12:18 PM, Jean-Claude Durand wrote: > My lexical grammar (I use antlr v3.2): > lexer grammar Lex; > options > { language=Java; } > WS: ( ' ' | '\t' | '\n' )+ { $channel=HIDDEN; } ; > FIN : '-FIN-' ; > Moins : '-' ; > // Identifiers: > Idf : ('A'..'Z')+ ; > I want to enumera

[il-antlr-interest: 26929] Re: [antlr-interest] Parsing with inverse matches

2009-11-22 Thread Kirby Bohling
On Sun, Nov 22, 2009 at 4:20 PM, Vipul Delwadia wrote: > Hi, > > Suppose I have a very simple grammar: > > line:   x; > > x       :       STRING+; > > fragment BACKSLASH >        :       '\\'; > > NOTA:   BACKSLASH A; > > A       :       'a'; > > STRING >        :       (~(A)|NOTA)+; > > Now I wan