[il-antlr-interest: 31179] Re: [antlr-interest] Strange parse result
Am 16.01.2011 22:23, schrieb Mark Christiaens:
I simplified my test case further. The grammar is now:
grammar test_grammar;
start : t*;
t: ASSIGN | NUM ;
ASSIGN : ':=';
//NUM : '0';
NUM : '0' ':0'?;
I would expect this to be able to match the input
0:=0
but it doesn't. Changing the definition of NUM to the commented version and
the grammar does match the input.
some lookahead may help:
ASSIGN: (':=')= ':=' ;
NUM: '0' ( (':0')=':0' )?;
Michael
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31182] Re: [antlr-interest] WG: Question to the tree grammer
Hi Christian,
this seems like a correct implementation:
...
unary
: MINUS unary - ^(NEGATION unary)
| PLUS unary - unary
| term
;
...
The rest strays as before, only PLUS and MINUS are new tokens.
Von: Christian Lerch [mailto:christian.le...@km-works.eu]
Gesendet: Montag, 17. Januar 2011 13:14
An: antlr-interest@antlr.org; Richard Andrysek
Betreff: Re: [antlr-interest] WG: Question to the tree grammer
Hi Richard,
I also had a fierce fight with unary operators, although in the context of
juxtaposition (which allows to ellide the * and write e.g. 2x instead of 2*x).
Please take a look at my small but tested grammar module that addresses this
problem.
It implements 5 precedence levels for
1. binary +, -(left-assoc)
2. multiplication(left-assoc)
3. unary +, -
4. juxtaposition interpreted as multiplication(left-assoc)
5. exponentiation(right-assoc)
A little bit anoying with this grammar is the fact, that the multiplication
implied by juxtaposition has precedence over normal multiplication.
On the other hand this can also be viewed as a feature, cause it allows you to
write e.g. 1/a b which results in the same AST as 1/(a*b)
Of course you can drop this feature altogether by simply deleting rule arithSec
and substituting all references by arithPrim.
Best wishes,
Chris
/*
Arithmetic expression grammar
with 5-level operator precedence and associativity
with handling of unary minus and unary plus
with juxtaposition mapped to multiplication
without bracketedExpr:
unnecessary parenthesis can be removed later by the code generator
*/
parser grammar ArithExpr2Parser;
options {
language = Java;
output= AST;
tokenVocab = ArithExpr2Lexer;
}
tokens {
NEG;
PAREN;
TIMES;
}
formulae:arithExpr EOF
;
arithExpr:arithTerm ( (PLUS^|MINUS^) arithTerm )*
;
arithTerm:arithFact ( (MULT^|DIV^) arithFact )*
;
arithFact:MINUS arithSec - ^(NEG arithSec)
|PLUS! arithSec
|arithSec
;
arithSec:(arithPrim - arithPrim)
(p=arithPrim - ^(TIMES $arithSec $p))*
;
arithPrim:arithAtom ( POWER^ arithPrim )?
;
arithAtom:IDENT
|NUMBER
|LPAREN arithExpr RPAREN- ^(PAREN arithExpr)
;
Am 17.01.2011 12:13, schrieb Richard Andrysek:
I want to switch from yacc. So I've looked on Videos from Scott
Stanchfield http://vimeo.com/user566590 http://vimeo.com/user566590 ,
especially on Part9 about a
tree grammar.
I've found a problem with an unary operator. The Interpreter in Eclipse
works fine, but the grammar tree not.
It simply can't accept things like this:
-5 - - + - 4
I've attached also my files. Is there a way, how can I ask somebody
else, than me? I assume, it is something with left/right association, is
it?
Thank you for hints
Richard
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
km-works | Ing. Christian Lerch KG
Geschäftsführer: Christian Lerch · FN: 249408 b, LG Wien · UID: ATU57985808
Tel: +43 (720) 616 400 · e-mail: christian.le...@km-works.eu
Die Information in dieser E-Mail-Nachricht samt Anlagen ist vertraulich. Die
Kenntnisnahme und Verwendung der hier enthaltenen Informationen ist nur
denjenigen Personen gestattet, an die diese Kommunikation adressiert ist
und/oder die zur Kenntnisnahme und Verwendung dieser Daten ausdrücklich
ermächtigt wurden. Sollten Sie diesem Personenkreis nicht angehören, werden Sie
hiemit davon in Kenntnis gesetzt, dass jegliche Weiter- und Wiedergabe,
Vervielfältigung, Verbreitung, Verwendung und/oder Handeln aufgrund des Inhalts
dieser Informationen zu unterlassen ist. Sollten Sie diese Nachricht
versehentlich erhalten haben, ersuchen wir Sie, uns über diesen Umstand zu
unterrichten. Bitte löschen Sie dann anschließend diese E-Mail endgültig von
Ihrem System.
All information contained in this e-mail message including attachments is
privileged and confidential. The information contained herein is intended
solely for the use and knowledge of the individual(s) to whom this
communication is addressed and/or others authorised to receive it. If you are
not the intended recipient, you are hereby notified that any disclosure,
copying, distribution, use and/or taking action in reliance on the contents of
this information is strictly prohibited. If you accidentally received this
e-mail, kindly let us know about this. On having done so, please delete this
e-mail permanently from your system.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
[il-antlr-interest: 31183] [antlr-interest] Patches to fix CSharp2 runtime project and add VS2010 project files
Dear All, I submitted a pull request via the Github repo since I was not aware that it was read-only. I have attached the patch files corresponding to these two fixes to this e-mail (as suggested by anatol). These are summarized below: - 0001-Added-missing-file-IAstRuleReturnScope-1.cs-into-sou.patch Adds missing source file IAstRuleReturnScope`1.cs into CSharp2 project (this file is referenced from .csproj files and the interface is consumed elsewhere) - 0002-Generated-project-files-for-VS2010-for-CSharp2-runti.patch I cloned the VS2008 project files and migrated them to VS2010. This might be useful to anybody else who uses this IDE to develop Antlr projects. Thanks, Richard Cook. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address 0001-Added-missing-file-IAstRuleReturnScope-1.cs-into-sou.patch Description: Binary data 0002-Generated-project-files-for-VS2010-for-CSharp2-runti.patch Description: Binary data -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31184] [antlr-interest] collecting tokens without invoking parser rules...
Using Antlr 3.2 with language=C as a target
For parsing Teradata's stored-procedure language (SPL), we have the issue
of context-sensitive token hiding.
I'm trying to use rules for SQL statements embedded in SPL that just
swallow the tokens, so we have rules like:
swallow_to_semi : ~ ( SEMI ) * ;
update_stmt : UPDATE swallow_to_semi;
We take the stream of tokens from this UPDATE rule and pass them off to an
existing SQL parser.
But, because SPL has an assignment statement rule that looks like
assignment_stmt : SET dotted_name '=' expression SEMI;
and teradata SQL uses 'SET' within its own grammar, when I encounter a
source statement like
update mytable set x = y, a = b where a = 'none' ;
I get an error that makes it clear to me that the Antlr parser is 'seeing'
the 'set' and trying to invoke the assignment_stmt rule.
because the complaint is about expecting a SEMI at the source position
where the comma is.
I don't think that redirecting EVERYTHING in the lexer after the UPDATE to
an alternate channel will work in all cases, because there are other
context sensitivities in play - for example:
SELECT has to read everything to a SEMI when it appears in a statement
context, but when there is a select clause in a FOR statement, it must
read upto a USING, FOR, DO or SEMI token.
So, what I tried so far was code that looks like
static ANTLR3_BOOLEAN semicolonMatch ( pplsqlParser ctx, pANTLR3_VECTOR
tokens)
{
pANTLR3_PARSER parser = ctx-pParser;
pANTLR3_TOKEN_STREAM ts = parser-getTokenStream(parser);
ANTLR3_INT32 tok;
if( ! tokens) // If we didn't have a token list, start one now
tokens = ctx-vectors-newVector( ctx-vectors);
if (LA(0) == SEMI) return false; // e.g. COMMIT ;
while( ( tok=LA( 1) ) != EOF)
{
switch( tok)
{
case SEMI: return true;
case EOF:return false;
default:
tokens-add( tokens, LT( 1), NULL);
ts-istream-consume( ts-istream);
continue;
}
}
return false;
}
And a modified swallow_to_semi rule that looks like
swallow_to_semi : tokenlist+=( {semicolonMatch(ctx, $tokenlist) }? )
- $tokenlist+
but that doesn't work correctly because it seems to preemptively swallow
the SEMI and a statement like
COMMIT;
fails.
This feels like something that should be relatively easy to do, but I
don't seem to be able to figure out exactly how to make it happen and I
haven't hit upon the right search terms to find an appropriate example in
the Antlr-interest archives or the Wiki.
NOTICE from Ab Initio: If received in error, please destroy and notify sender,
and make no further use, disclosure, or distribution. This email (including
attachments) may contain information subject to confidentiality obligations,
and sender does not waive confidentiality or privilege.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31185] [antlr-interest] Antlr 3.3 Generate wrong debug code C# target
Hi, i'm writing because i have some problems using Antlr 3.3, target CSharp3 with -debug option. Compile the attached grammar (taken from Antlr 3.3 examples) with -debug, ANTLR generates wrong C# code eg: * generates wrong class definition eg: public partial class Myclass : DebugAntlr.Runtime.Parser instead of Antlr.Runtime.Debug.DebugParser * doesn't genrate static member decisionCanBacktrack * set the DebugListener readonly property in the constructor Best Regards Max List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address TestImport.rar Description: Binary data -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31186] Re: [antlr-interest] Antlr 3.3 Generate wrong debug code C# target
Hi Max, The CSharp3 target does not currently support ANTLRWorks-style debugging. I'm planning to support debugging in a future release of the target. Thanks, Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Massimiliano Donini Sent: Monday, January 17, 2011 4:26 PM To: antlr-interest@antlr.org Subject: [antlr-interest] Antlr 3.3 Generate wrong debug code C# target Hi, i'm writing because i have some problems using Antlr 3.3, target CSharp3 with -debug option. Compile the attached grammar (taken from Antlr 3.3 examples) with -debug, ANTLR generates wrong C# code eg: * generates wrong class definition eg: public partial class Myclass : DebugAntlr.Runtime.Parser instead of Antlr.Runtime.Debug.DebugParser * doesn't genrate static member decisionCanBacktrack * set the DebugListener readonly property in the constructor Best Regards Max List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31187] [antlr-interest] next possible tokens
Folks, I have been struggling with trying to get a list of next possible tokens for a bit now and may use some help. A page here http://www.antlr.org/wiki/pages/viewpage.action?pageId=11567208 talks about how to retrieve a list of possible options but it doesnt seem to cover all bases so to speak. According to my observations ANTLR has two places where next tokens can be obtained (Match function whose parameter indicates next options and PushFollow that preceeds rule invocation and places possibilities in a state follow stack). Also a rule may have optional token (TOKEN1 TOKEN2* for example) in which case TOKEN2 may or may not be present but no mismatch errors are triggered. ANTLR however knows that TOKEN2 may be expected. Another case is when predicates are used in a rule. Only 1 alternative is possible given rule semantics but all possible rule starts are reported as expected tokens in push follow stack. So what is the best strategy to get a list of next expected tokens that is a/ generic (so I dont have to weave code all over parser to make it happen) and b/ reliable (gives me next expected possible tokens only) Rgds. !!! List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31188] [antlr-interest] Strangeness when parsing strings and spaces
Hi,
I know that this is a problem with my lexer and I'm doing something
stupid, but I have a problem with simple k,v pairs of the format:
[String quoted string with spaces and non-alhpa chars]
My grammar
--
LEFT_SQUARE: '[';
RIGHT_SQUARE: ']';
STRING: 'a'..'z'|'A'..'Z';
TEXT: ('a'..z'|'A'..'Z'|' '|',')+
pair: LEFT_SQUARE (info=STRING) ' ' '' (value=TEXT) '' RIGHT_SQUARE
NEWLINE+ - transform(i={$info.text},v={$value.text});
This fails with mismatched input expecting 'Test ' expecting STRING
(note the space that should not be present).
pair: LEFT_SQUARE (info=TEXT) ' ' '' (value=TEXT) '' RIGHT_SQUARE
NEWLINE+ - transform(i={$info.text},v={$value.text});
This grabs the space and passes it to transform (which I really don't want)
Can anyone see what I'm doing wrong? I know the answer is probably simple
Thanks,
Kev
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31189] Re: [antlr-interest] Strangeness when parsing strings and spaces
On Tue, Jan 18, 2011 at 7:41 AM, Kevin Jackson foamd...@gmail.com wrote:
Hi,
I know that this is a problem with my lexer and I'm doing something
stupid, but I have a problem with simple k,v pairs of the format:
[String quoted string with spaces and non-alhpa chars]
My grammar
--
LEFT_SQUARE: '[';
RIGHT_SQUARE: ']';
STRING: 'a'..'z'|'A'..'Z';
TEXT: ('a'..z'|'A'..'Z'|' '|',')+
Your STRING and TEXT have too much in common. Better let TEXT also include
the double quotes. Also, you could just skip the spaces outside you quoted
text and your STRING rule only matches a single character, which is probably
a mistake.
Try something like:
pair
: LEFT_SQUARE IDENTIFIER QUOTED_TEXT RIGHT_SQUARE
;
LEFT_SQUARE : '[';
RIGHT_SQUARE : ']';
IDENTIFIER : ('a'..'z'|'A'..'Z')+;
QUOTED_TEXT : '' ('a'..'z' | 'A'..'Z' | ' ' | ',' | '-')+ '';
SPACES : (' ' | '\t')+ {skip();};
Regards,
Bart.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31190] Re: [antlr-interest] Strangeness when parsing strings and spaces
Hi,
pair
: LEFT_SQUARE IDENTIFIER QUOTED_TEXT RIGHT_SQUARE
;
LEFT_SQUARE : '[';
RIGHT_SQUARE : ']';
IDENTIFIER : ('a'..'z'|'A'..'Z')+;
QUOTED_TEXT : '' ('a'..'z' | 'A'..'Z' | ' ' | ',' | '-')+ '';
SPACES : (' ' | '\t')+ {skip();};
Thanks for the reply. This nearly works! However I don't want to
retain the '' (quotes) in the rewrite rule, so although the
IDENTIFIER has fixed the first problem, I now have a problem with the
QUOTED_TEXT.
I should have mentioned that I suppose.
Thanks,
Kev
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31192] Re: [antlr-interest] Strangeness when parsing strings and spaces
Hi,
Try something like this:
QUOTED_TEXT
: '' ('a'..'z' | 'A'..'Z' | ' ' | ',' | '-')+ ''
{setText(getText().substring(0, getText().length()-1));}
;
Thanks, that works perfectly. I thought that there was a way to
achieve this through the lexer/tokenization process without resorting
to code fragments - I recall seeing something similar but cannot find
where I saw it in my history :(
A further question which I cannot find the answer too:
What is the difference between 'a'..'z'+ and ('a'..'z')+ ??
Thanks,
Kev
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
