[il-antlr-interest: 28458] [antlr-interest] nope. doesnt fix it. :-(
that last hack doesn't fix it. now it runs but creates a dodgy parser. jamie List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28459] [antlr-interest] narrowing it down
A NonLLStarDecisionException was being throw in the DFA constructor, but
then the DecisionProbe was not printing any information because in
public void issueWarnings() {
// NONREGULAR DUE TO RECURSION 1 ALTS
// Issue this before aborted analysis, which might also occur
// if we take too long to terminate
if ( nonLLStarDecision !dfa.getAutoBacktrackMode() ) {
ErrorManager.nonLLStarDecision(this);
}
the if clause was ( true false). so dfa.getAutoBacktrackMode is returning
true. When I nuked that part of the test I got this error message:
error(211): JavaDef.g:378:5: [fatal] rule statement has non-LL(*) decision
due to recursive rule invocations reachable from alts 2,3. Resolve by
left-factoring or using syntactic predicates or using backtrack=true option.
The DFA that had this problem in the constructor is the one failing later
because the resetStateNumbersToBeContiguous() method was never called.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28460] [antlr-interest] posting
Hi im trying to post to the antlr interest group but it seems I get bouced everytime Cheers Amitesh Kumar Amitesh Kumar |CIB Integration | Business Infrastructure Technology | Standard Bank CIB International | Ground Floor, 20 Gresham Street, London, EC2V 7JE T: +44 [0]203 145 5575 | E: amitesh.ku...@standardbank.com * More information on Standard Bank is available at www.standardbank.com Everything in this email and any attachments relating to the official business of Standard Bank Group Limited and any or all subsidiaries, (the Company), is proprietary to the Company. It is confidential, legally privileged and protected by relevant laws. The Company does not own and endorse any other content. Views and opinions are those of the sender unless clearly stated as being that of the Company. The person or persons addressed in this email are the sole authorised recipient. Please notify the sender immediately if it has unintentionally, or inadvertently reached you and do not read, disclose or use the content in any way and delete this e-mail from your system. The Company cannot ensure that the integrity of this email has been maintained nor that it is free of errors, virus, interception or interference. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. * This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28461] [antlr-interest] Parsing Large Files
Hi Guys what we are looking for is just parsing the file and recording the errors we don't need to keep a track of any tokens or a AST. Im getting Exception in thread main java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2760) at java.util.Arrays.copyOf(Arrays.java:2734) at java.util.ArrayList.ensureCapacity(ArrayList.java:167) at java.util.ArrayList.add(ArrayList.java:351) at org.antlr.runtime.CommonTokenStream.fillBuffer(CommonTokenStream.java:11 6) at org.antlr.runtime.CommonTokenStream.LT(CommonTokenStream.java:238) at org.antlr.runtime.Parser.getCurrentInputSymbol(Parser.java:54) at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:104) at DealsAll2Parser.header(DealsAll2Parser.java:123) at DealsAll2Parser.file(DealsAll2Parser.java:67) at AntlrMain.main(AntlrMain.java:53) I see where the error is coming from the CommonTokenStream is keeping track of all past tokens, how can I make it so it doesn't. Do I have to create my own Token Stream? Or is there a easy way. Cheers Kumaap0 * More information on Standard Bank is available at www.standardbank.com Everything in this email and any attachments relating to the official business of Standard Bank Group Limited and any or all subsidiaries, (the Company), is proprietary to the Company. It is confidential, legally privileged and protected by relevant laws. The Company does not own and endorse any other content. Views and opinions are those of the sender unless clearly stated as being that of the Company. The person or persons addressed in this email are the sole authorised recipient. Please notify the sender immediately if it has unintentionally, or inadvertently reached you and do not read, disclose or use the content in any way and delete this e-mail from your system. The Company cannot ensure that the integrity of this email has been maintained nor that it is free of errors, virus, interception or interference. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. * This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28462] Re: [antlr-interest] posting
On Thu, Apr 1, 2010 at 11:05 AM, Kumar, Amitesh amitesh.ku...@standardbank.com wrote: Hi im trying to post to the antlr interest group but it seems I get bouced everytime Not any more: this one went through just fine. Regards, Bart. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28463] Re: [antlr-interest] Factorization of Logic Expressions
Hi, 2010/3/17 Nazim Oztahtaci nazim_oztaht...@hotmail.com [...] a DNF representation: ab+ac. [...] Now Im working on factorizing the DNF formed expressions back to originial [...]. So I need to factorize the DNF expression. [...] I know that I cant reach the original expression everytime for the complicated expressions. Also for the reverse Demorgan, I cant use this matrix probably. I wanted to ask users of this mail group if they have any advice to me regarding an algorithm for this problem or a new way as solution. Maybe http://en.wikipedia.org/wiki/And-inverter_graph can help you. regards -- Mark Ruvald. A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28464] [antlr-interest] ClassNotFound exception when run gUnit
I'm on my way to set up gUnit for my grammar. I'm learning the tutorial found there : http://www.antlr.org/wiki/display/ANTLR3/gUnit+-+Grammar+Unit+Testing I follow all the instruction but when run gUnit, it give an error like this : C:\__GUnit\examples\java\LL-starjava org.antlr.Tool SimpleC.g C:\__GUnit\examples\java\LL-starjavac *.java C:\__GUnit\examples\java\LL-starjava org.antlr.gunit.Interp SimpleC.gunit java.lang.ClassNotFoundException: SimpleCLexer at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:252) at org.antlr.gunit.gUnitExecutor.classForName(gUnitExecutor.java:98) at org.antlr.gunit.gUnitExecutor.runParser(gUnitExecutor.java:308) at org.antlr.gunit.gUnitExecutor.runCorrectParser(gUnitExecutor.java:159) at org.antlr.gunit.gUnitExecutor.executeTests(gUnitExecutor.java:175) at org.antlr.gunit.gUnitExecutor.execTest(gUnitExecutor.java:123) at org.antlr.gunit.Interp.main(Interp.java:71) Java code for lexer and parser are generated and compile successfully and all files are located in the same folder: c:\__GUnit\examples\java\LL-star\input c:\__GUnit\examples\java\LL-star\SimpleC.g c:\__GUnit\examples\java\LL-star\SimpleC.gunit c:\__GUnit\examples\java\LL-star\SimpleC.tokens c:\__GUnit\examples\java\LL-star\SimpleCLexer$DFA4.class c:\__GUnit\examples\java\LL-star\SimpleCLexer.class c:\__GUnit\examples\java\LL-star\SimpleCLexer.java c:\__GUnit\examples\java\LL-star\SimpleCParser$DFA2.class c:\__GUnit\examples\java\LL-star\SimpleCParser.class c:\__GUnit\examples\java\LL-star\SimpleCParser.java I can't figure out what is wrong to cause ClassNotFound exception so would someone please help me fix it. I'm on a WinXP 64, using antlr 3.1.3, string template 3.2 and jdk1.6.0_11. Thank in advance List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28465] [antlr-interest] Syntax of rewrite rule
Hi, I am wondering how to transform the rewrite rule Expr: mul_expr ( ( PLUS | MINUS )^ mul_expr )*; Into the explicit form with the arrow Expr: a=mul_expr ( op=( PLUS | MINUS ) b=mul_expr )* - ? ; For later use in an AST tree parser. What is the proper way to do this? I checked ² The definitive ANTL reference²/online documentation but could not find anything about the syntax. Thanks and best regards, Chris List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28466] Re: [antlr-interest] Syntax of rewrite rule
On Thu, Apr 1, 2010 at 1:21 PM, Christian Mayer christian.ma...@bsse.ethz.ch wrote: I am wondering how to transform the rewrite rule Expr: mul_expr ( ( PLUS | MINUS )^ mul_expr )*; Into the explicit form with the arrow Expr: a=mul_expr ( op=( PLUS | MINUS ) b=mul_expr )* - ? ; For later use in an AST tree parser. What is the proper way to do this? I checked ² The definitive ANTL reference²/online documentation but could not find anything about the syntax. I did not test it, but I believe this would work: Expr : (a=mul_expr - $a ) (op=(PLUS | MINUS) b=mul_expr - ^($op $a $b) )* ; Also see Chapter 7, section Constructing AST's with rewrite rules paragraph Rewrite Rules in Subrules from The definitive ANTLR reference. Kind regards, Bart. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28467] [antlr-interest] backtrack + rule arguments + C target
Hi.
I have some problems with backtrack option and rule arguments.
Example grammar:
grammar testbt;
options
{
language = C;
backtrack = true;
}
expr
@init {
int i = 1;
}
:
num[i] | id[i]
;
num[int i] : '0'..'9'+;
id[int i] : 'a'..'z'+;
Generated files does not compile with error error C2065: 'i' :
undeclared identifier.
(The problem is that C target does not insert rule arguments into
argument list of autogenerated 'synpred' functions.)
How can I fix this?
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28468] Re: [antlr-interest] crash with tree filter
This is usually because one of your tokens has not formal definition and ends up out of range. Unfortunately there is a bug (I think I trigger it usually when developing import grammars) that does not record that a token has no definition under certain circumstances. So, check your token defs - the answer is buried in there somewhere. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of James Briant Sent: Wednesday, March 31, 2010 8:06 PM To: ANTLR Interest Subject: [antlr-interest] crash with tree filter error(10): internal error: com/binaryfinery/comb/javaparser/generated/JavaDef.g : java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 50 java.util.Vector.set(Vector.java:712) org.antlr.analysis.DFA.createMinMaxTables(DFA.java:531) org.antlr.analysis.DFA.createStateTables(DFA.java:435) org.antlr.codegen.CodeGenerator.genLookaheadDecision(CodeGenerator.java :653) org.antlr.grammar.v2.CodeGenTreeWalker.block(CodeGenTreeWalker.java:101 8) org.antlr.grammar.v2.CodeGenTreeWalker.rule(CodeGenTreeWalker.java:797) org.antlr.grammar.v2.CodeGenTreeWalker.rules(CodeGenTreeWalker.java:588 ) org.antlr.grammar.v2.CodeGenTreeWalker.grammarSpec(CodeGenTreeWalker.ja va:530) org.antlr.grammar.v2.CodeGenTreeWalker.grammar(CodeGenTreeWalker.java:3 36) org.antlr.codegen.CodeGenerator.genRecognizer(CodeGenerator.java:432) org.antlr.Tool.generateRecognizer(Tool.java:641) org.antlr.Tool.process(Tool.java:454) org.antlr.mojo.antlr3.Antlr3Mojo.execute(Antlr3Mojo.java:391) org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginM anager.java:451) org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(Defaul tLifecycleExecutor.java:558) org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandaloneGo al(DefaultLifecycleExecutor.java:512) org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(Default LifecycleExecutor.java:482) org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandl eFailures(DefaultLifecycleExecutor.java:330) org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments (DefaultLifecycleExecutor.java:291) org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLife cycleExecutor.java:142) org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:336) org.apache.maven.DefaultMaven.execute(DefaultMaven.java:129) org.apache.maven.cli.MavenCli.main(MavenCli.java:287) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja va:39) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso rImpl.java:25) java.lang.reflect.Method.invoke(Method.java:597) org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315) org.codehaus.classworlds.Launcher.launch(Launcher.java:255) org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430) org.codehaus.classworlds.Launcher.main(Launcher.java:375) This is where it tanks: switchBlockLabels : ^(SWITCH_BLOCK_LABEL_LIST switchCaseLabels* ) ; switchCaseLabels : ^(CASE expression blockStatement*) | ^(DEFAULT blockStatement*) ; Any ideas? Thanks, Jamie List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28469] Re: [antlr-interest] hack for crash
Yeah - we need to fix the problem with token defs. I will try to make some time
today to raise a few JIRAs for various bugs that have been reported. Then at
least Ter can check them in V4.
Thanks for your efforts.
Jim
-Original Message-
From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
boun...@antlr.org] On Behalf Of James Briant
Sent: Wednesday, March 31, 2010 9:26 PM
To: ANTLR Interest
Subject: [antlr-interest] hack for crash
in the org.antlr.analysis.DFA class, I hacked in the for loop. Total
hackery
on my part, but it works. Fixes symptoms not disease.
public int getNumberOfStates() {
if ( getUserMaxLookahead()0 ) {
// if using fixed lookahead then uniqueSets not set
return states.size();
}
for(int i = 0; i states.size(); ++i) {
DFAState state = states.get(i);
int s = state.stateNumber;
if ( s=numberOfStates) {
numberOfStates=s+1;
}
}
return numberOfStates;
}
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28470] Re: [antlr-interest] posting
-Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Kumar, Amitesh Sent: Thursday, April 01, 2010 2:06 AM To: antlr-interest@antlr.org Subject: [antlr-interest] posting Hi im trying to post to the antlr interest group but it seems I get bouced everytime Not this time. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28471] Re: [antlr-interest] Parsing Large Files
Kumaap0,
It looks like just some inexperience here. My first guess is that you have a
lexer rule that matches and empty string and so you are getting an infinite
number of tokens.
Look for:
LEXERRULE: ('x')* ;
Where a * should be a + as that is the most common error.
Jim
-Original Message-
From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
boun...@antlr.org] On Behalf Of Kumar, Amitesh
Sent: Thursday, April 01, 2010 2:13 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] Parsing Large Files
Hi Guys what we are looking for is just parsing the file and recording
the errors we don't need to keep a track of any tokens or a AST.
Im getting
Exception in thread main java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2760)
at java.util.Arrays.copyOf(Arrays.java:2734)
at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
at java.util.ArrayList.add(ArrayList.java:351)
at
org.antlr.runtime.CommonTokenStream.fillBuffer(CommonTokenStream.java:1
1
6)
at
org.antlr.runtime.CommonTokenStream.LT(CommonTokenStream.java:238)
at
org.antlr.runtime.Parser.getCurrentInputSymbol(Parser.java:54)
at
org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:104)
at DealsAll2Parser.header(DealsAll2Parser.java:123)
at DealsAll2Parser.file(DealsAll2Parser.java:67)
at AntlrMain.main(AntlrMain.java:53)
I see where the error is coming from the CommonTokenStream is keeping
track of all past tokens, how can I make it so it doesn't. Do I have to
create my own Token Stream? Or is there a easy way.
Cheers
Kumaap0
***
**
More information on Standard Bank is available at www.standardbank.com
Everything in this email and any attachments relating to the official
business
of Standard Bank Group Limited and any or all subsidiaries, (the
Company), is
proprietary to the Company. It is confidential, legally privileged and
protected
by relevant laws. The Company does not own and endorse any other
content.
Views and opinions are those of the sender unless clearly stated as
being
that of the Company.
The person or persons addressed in this email are the sole authorised
recipient. Please notify the sender immediately if it has
unintentionally,
or inadvertently reached you and do not read, disclose or use the
content
in any way and delete this e-mail from your system.
The Company cannot ensure that the integrity of this email has been
maintained nor that it is free of errors, virus, interception or
interference.
The sender therefore does not accept liability for any errors or
omissions
in the contents of this message which arise as a result of e-mail
transmission.
If verification is required please request a hard-copy version. This
message
is provided for informational purposes and should not be construed as a
solicitation or offer to buy or sell any securities or related
financial instruments.
***
**
This message has been scanned for viruses by BlackSpider MailControl -
www.blackspider.com
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28472] Re: [antlr-interest] Parsing Large Files
The other possibility is of course that you are trying to parse a massive file in one lump. You probably just want to reinvoke the parser for each deal record (break it up in the string tream. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Kumar, Amitesh Sent: Thursday, April 01, 2010 2:13 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Parsing Large Files Hi Guys what we are looking for is just parsing the file and recording the errors we don't need to keep a track of any tokens or a AST. Im getting Exception in thread main java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2760) at java.util.Arrays.copyOf(Arrays.java:2734) at java.util.ArrayList.ensureCapacity(ArrayList.java:167) at java.util.ArrayList.add(ArrayList.java:351) at org.antlr.runtime.CommonTokenStream.fillBuffer(CommonTokenStream.java:1 1 6) at org.antlr.runtime.CommonTokenStream.LT(CommonTokenStream.java:238) at org.antlr.runtime.Parser.getCurrentInputSymbol(Parser.java:54) at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:104) at DealsAll2Parser.header(DealsAll2Parser.java:123) at DealsAll2Parser.file(DealsAll2Parser.java:67) at AntlrMain.main(AntlrMain.java:53) I see where the error is coming from the CommonTokenStream is keeping track of all past tokens, how can I make it so it doesn't. Do I have to create my own Token Stream? Or is there a easy way. Cheers Kumaap0 *** ** More information on Standard Bank is available at www.standardbank.com Everything in this email and any attachments relating to the official business of Standard Bank Group Limited and any or all subsidiaries, (the Company), is proprietary to the Company. It is confidential, legally privileged and protected by relevant laws. The Company does not own and endorse any other content. Views and opinions are those of the sender unless clearly stated as being that of the Company. The person or persons addressed in this email are the sole authorised recipient. Please notify the sender immediately if it has unintentionally, or inadvertently reached you and do not read, disclose or use the content in any way and delete this e-mail from your system. The Company cannot ensure that the integrity of this email has been maintained nor that it is free of errors, virus, interception or interference. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. *** ** This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28473] Re: [antlr-interest] backtrack + rule arguments + C target
Please read the documentation:
@declarations {}
@init {}
Then search the list using antlr.markmail.org for hoisted predicates and local
variables. It isn't the C target it is just that the local variable or
parameter is out of scope for the predicate, so you must use scopes if you have
to use semantic predicates (with parameters).
Jim
-Original Message-
From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
boun...@antlr.org] On Behalf Of Anton Bychkov
Sent: Thursday, April 01, 2010 4:20 AM
To: antlr-interest@antlr.org interest
Subject: [antlr-interest] backtrack + rule arguments + C target
Hi.
I have some problems with backtrack option and rule arguments.
Example grammar:
grammar testbt;
options
{
language = C;
backtrack = true;
}
expr
@init {
int i = 1;
}
:
num[i] | id[i]
;
num[int i]: '0'..'9'+;
id[int i] : 'a'..'z'+;
Generated files does not compile with error error C2065: 'i' :
undeclared identifier.
(The problem is that C target does not insert rule arguments into
argument list of autogenerated 'synpred' functions.)
How can I fix this?
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28475] Re: [antlr-interest] Parsing Large Files
On Thu, Apr 1, 2010 at 4:26 PM, Kumar, Amitesh amitesh.ku...@standardbank.com wrote: But my general issue is that not all my data is a simple CSV file some will be multi line records. Hence I didn't want to keep a record of the tokens. Any ideas . By the way thanks for your reply. Hi You can easily implement your own TokenStream that is optimized for your use case eg. does not try to keep everything in one big array. If you explore this possibility, you will quickly discover that it is very easy thing to do and test. Hope it helps. Cheers Kumaap0 -- Greetings Marcin Rzeźnicki List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28476] Re: [antlr-interest] Parsing Large Files
Actually, I think that if you use UnbufferedTokenStream(), that this will pretty much do what you want already, but it is easy to derive from one of the token streams, and add methods tah can discard buffered tokens once you know you have dealt with them. Also, if you have comma separated files, then it is usually easier to use awk. Finally, your grammar has myriad lexical ambiguities and I am afraid it is not going to work as you have written it. You cannot have more than one lexer rule that matches the same text as the lexer is not syntax directed, it just tokenizes what it sees. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Marcin Rzeznicki Sent: Thursday, April 01, 2010 8:02 AM To: Kumar, Amitesh Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files On Thu, Apr 1, 2010 at 4:26 PM, Kumar, Amitesh amitesh.ku...@standardbank.com wrote: But my general issue is that not all my data is a simple CSV file some will be multi line records. Hence I didn't want to keep a record of the tokens. Any ideas . By the way thanks for your reply. Hi You can easily implement your own TokenStream that is optimized for your use case eg. does not try to keep everything in one big array. If you explore this possibility, you will quickly discover that it is very easy thing to do and test. Hope it helps. Cheers Kumaap0 -- Greetings Marcin Rzeźnicki List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28477] [antlr-interest] Objective-C and Antlr
Is there a string template to generate Objective-C from Antlr3.2? If not what version of Antlr does one need to go back to in order to get a version that will generate Objective-C code? Thanks, Alan --- Alan Condit 1085 Tierra Ct. Woodburn, OR 97071 Email -- acon...@ipns.com Home-Office (503) 982-0906 List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28479] Re: [antlr-interest] Parsing Large Files
Great where is this UnbufferedTokenStream. Cheers Jim Amitesh Kumar |CIB Integration | Business Infrastructure Technology | Standard Bank CIB International | Ground Floor, 20 Gresham Street, London, EC2V 7JE T: +44 [0]203 145 5575 | E: amitesh.ku...@standardbank.com -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Jim Idle Sent: 01 April 2010 16:27 Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Actually, I think that if you use UnbufferedTokenStream(), that this will pretty much do what you want already, but it is easy to derive from one of the token streams, and add methods tah can discard buffered tokens once you know you have dealt with them. Also, if you have comma separated files, then it is usually easier to use awk. Finally, your grammar has myriad lexical ambiguities and I am afraid it is not going to work as you have written it. You cannot have more than one lexer rule that matches the same text as the lexer is not syntax directed, it just tokenizes what it sees. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Marcin Rzeznicki Sent: Thursday, April 01, 2010 8:02 AM To: Kumar, Amitesh Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files On Thu, Apr 1, 2010 at 4:26 PM, Kumar, Amitesh amitesh.ku...@standardbank.com wrote: But my general issue is that not all my data is a simple CSV file some will be multi line records. Hence I didn't want to keep a record of the tokens. Any ideas . By the way thanks for your reply. Hi You can easily implement your own TokenStream that is optimized for your use case eg. does not try to keep everything in one big array. If you explore this possibility, you will quickly discover that it is very easy thing to do and test. Hope it helps. Cheers Kumaap0 -- Greetings Marcin Rzeźnicki List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address * More information on Standard Bank is available at www.standardbank.com Everything in this email and any attachments relating to the official business of Standard Bank Group Limited and any or all subsidiaries, (the Company), is proprietary to the Company. It is confidential, legally privileged and protected by relevant laws. The Company does not own and endorse any other content. Views and opinions are those of the sender unless clearly stated as being that of the Company. The person or persons addressed in this email are the sole authorised recipient. Please notify the sender immediately if it has unintentionally, or inadvertently reached you and do not read, disclose or use the content in any way and delete this e-mail from your system. The Company cannot ensure that the integrity of this email has been maintained nor that it is free of errors, virus, interception or interference. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. * This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28480] Re: [antlr-interest] Parsing Large Files
Just replace new CommonTokenStream(...). But you will need to fix your grammar before it will all work of course. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Kumar, Amitesh Sent: Thursday, April 01, 2010 8:52 AM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Great where is this UnbufferedTokenStream. Cheers Jim Amitesh Kumar |CIB Integration | Business Infrastructure Technology | Standard Bank CIB International | Ground Floor, 20 Gresham Street, London, EC2V 7JE T: +44 [0]203 145 5575 | E: amitesh.ku...@standardbank.com -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Jim Idle Sent: 01 April 2010 16:27 Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Actually, I think that if you use UnbufferedTokenStream(), that this will pretty much do what you want already, but it is easy to derive from one of the token streams, and add methods tah can discard buffered tokens once you know you have dealt with them. Also, if you have comma separated files, then it is usually easier to use awk. Finally, your grammar has myriad lexical ambiguities and I am afraid it is not going to work as you have written it. You cannot have more than one lexer rule that matches the same text as the lexer is not syntax directed, it just tokenizes what it sees. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Marcin Rzeznicki Sent: Thursday, April 01, 2010 8:02 AM To: Kumar, Amitesh Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files On Thu, Apr 1, 2010 at 4:26 PM, Kumar, Amitesh amitesh.ku...@standardbank.com wrote: But my general issue is that not all my data is a simple CSV file some will be multi line records. Hence I didn't want to keep a record of the tokens. Any ideas . By the way thanks for your reply. Hi You can easily implement your own TokenStream that is optimized for your use case eg. does not try to keep everything in one big array. If you explore this possibility, you will quickly discover that it is very easy thing to do and test. Hope it helps. Cheers Kumaap0 -- Greetings Marcin Rzeźnicki List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr- interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address *** ** More information on Standard Bank is available at www.standardbank.com Everything in this email and any attachments relating to the official business of Standard Bank Group Limited and any or all subsidiaries, (the Company), is proprietary to the Company. It is confidential, legally privileged and protected by relevant laws. The Company does not own and endorse any other content. Views and opinions are those of the sender unless clearly stated as being that of the Company. The person or persons addressed in this email are the sole authorised recipient. Please notify the sender immediately if it has unintentionally, or inadvertently reached you and do not read, disclose or use the content in any way and delete this e-mail from your system. The Company cannot ensure that the integrity of this email has been maintained nor that it is free of errors, virus, interception or interference. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. *** ** This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at
[il-antlr-interest: 28481] Re: [antlr-interest] Parsing Large Files
Ive got ANTLR version 3.2 it doesn't seem to have UnbufferedTokenStream, there doesn't seem to be a newer version on the site Amitesh Kumar |CIB Integration | Business Infrastructure Technology | Standard Bank CIB International | Ground Floor, 20 Gresham Street, London, EC2V 7JE T: +44 [0]203 145 5575 | E: amitesh.ku...@standardbank.com -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Jim Idle Sent: 01 April 2010 16:55 To: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Just replace new CommonTokenStream(...). But you will need to fix your grammar before it will all work of course. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Kumar, Amitesh Sent: Thursday, April 01, 2010 8:52 AM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Great where is this UnbufferedTokenStream. Cheers Jim Amitesh Kumar |CIB Integration | Business Infrastructure Technology | Standard Bank CIB International | Ground Floor, 20 Gresham Street, London, EC2V 7JE T: +44 [0]203 145 5575 | E: amitesh.ku...@standardbank.com -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Jim Idle Sent: 01 April 2010 16:27 Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Actually, I think that if you use UnbufferedTokenStream(), that this will pretty much do what you want already, but it is easy to derive from one of the token streams, and add methods tah can discard buffered tokens once you know you have dealt with them. Also, if you have comma separated files, then it is usually easier to use awk. Finally, your grammar has myriad lexical ambiguities and I am afraid it is not going to work as you have written it. You cannot have more than one lexer rule that matches the same text as the lexer is not syntax directed, it just tokenizes what it sees. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Marcin Rzeznicki Sent: Thursday, April 01, 2010 8:02 AM To: Kumar, Amitesh Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files On Thu, Apr 1, 2010 at 4:26 PM, Kumar, Amitesh amitesh.ku...@standardbank.com wrote: But my general issue is that not all my data is a simple CSV file some will be multi line records. Hence I didn't want to keep a record of the tokens. Any ideas . By the way thanks for your reply. Hi You can easily implement your own TokenStream that is optimized for your use case eg. does not try to keep everything in one big array. If you explore this possibility, you will quickly discover that it is very easy thing to do and test. Hope it helps. Cheers Kumaap0 -- Greetings Marcin Rzeźnicki List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr- interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address ** * ** More information on Standard Bank is available at www.standardbank.com Everything in this email and any attachments relating to the official business of Standard Bank Group Limited and any or all subsidiaries, (the Company), is proprietary to the Company. It is confidential, legally privileged and protected by relevant laws. The Company does not own and endorse any other content. Views and opinions are those of the sender unless clearly stated as being that of the Company. The person or persons addressed in this email are the sole authorised recipient. Please notify the sender immediately if it has unintentionally, or inadvertently reached you and do not read, disclose or use the content in any way and delete this e-mail from your system. The Company cannot ensure that the integrity of this email has been maintained nor that it is free of errors, virus, interception or interference. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. ** * ** This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com List:
[il-antlr-interest: 28482] Re: [antlr-interest] Parsing Large Files
Ah - this is still in development. In which case you would need to extend CommonTokenStream and create your own system. Or you could switch the development snapshot. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Kumar, Amitesh Sent: Thursday, April 01, 2010 9:07 AM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Ive got ANTLR version 3.2 it doesn't seem to have UnbufferedTokenStream, there doesn't seem to be a newer version on the site Amitesh Kumar |CIB Integration | Business Infrastructure Technology | Standard Bank CIB International | Ground Floor, 20 Gresham Street, London, EC2V 7JE T: +44 [0]203 145 5575 | E: amitesh.ku...@standardbank.com -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Jim Idle Sent: 01 April 2010 16:55 To: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Just replace new CommonTokenStream(...). But you will need to fix your grammar before it will all work of course. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Kumar, Amitesh Sent: Thursday, April 01, 2010 8:52 AM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Great where is this UnbufferedTokenStream. Cheers Jim Amitesh Kumar |CIB Integration | Business Infrastructure Technology | Standard Bank CIB International | Ground Floor, 20 Gresham Street, London, EC2V 7JE T: +44 [0]203 145 5575 | E: amitesh.ku...@standardbank.com -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Jim Idle Sent: 01 April 2010 16:27 Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Actually, I think that if you use UnbufferedTokenStream(), that this will pretty much do what you want already, but it is easy to derive from one of the token streams, and add methods tah can discard buffered tokens once you know you have dealt with them. Also, if you have comma separated files, then it is usually easier to use awk. Finally, your grammar has myriad lexical ambiguities and I am afraid it is not going to work as you have written it. You cannot have more than one lexer rule that matches the same text as the lexer is not syntax directed, it just tokenizes what it sees. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Marcin Rzeznicki Sent: Thursday, April 01, 2010 8:02 AM To: Kumar, Amitesh Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files On Thu, Apr 1, 2010 at 4:26 PM, Kumar, Amitesh amitesh.ku...@standardbank.com wrote: But my general issue is that not all my data is a simple CSV file some will be multi line records. Hence I didn't want to keep a record of the tokens. Any ideas . By the way thanks for your reply. Hi You can easily implement your own TokenStream that is optimized for your use case eg. does not try to keep everything in one big array. If you explore this possibility, you will quickly discover that it is very easy thing to do and test. Hope it helps. Cheers Kumaap0 -- Greetings Marcin Rzeźnicki List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr- interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr- interest/your- email-address ** * ** More information on Standard Bank is available at www.standardbank.com Everything in this email and any attachments relating to the official business of Standard Bank Group Limited and any or all subsidiaries, (the Company), is proprietary to the Company. It is confidential, legally privileged and protected by relevant laws. The Company does not own and endorse any other content. Views and opinions are those of the sender unless clearly stated as being that of the Company. The person or persons addressed in this email are the sole authorised recipient. Please notify the sender immediately if it has unintentionally, or inadvertently reached you and do not read, disclose or use the content in any way and delete this e-mail from your system. The Company cannot ensure that the integrity of this email has been maintained nor that it is free of errors, virus, interception or interference. The sender therefore does not accept liability for any errors or omissions in the
[il-antlr-interest: 28483] [antlr-interest] online version of this forum?
I don't like to get email from this list, but I would very much like to be able to reply to discussions here. How do I do that without actually receiving emails to my inbox first? There doesn't appear to be a way. Is it possible we could move to a forum that would allow for online discussion/reply as well as supporting the email users? Jamie List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28484] Re: [antlr-interest] Parsing Large Files
I had for different reasons the same problem with CommonTokenStream and ended up implementing my own stream. It is available under the name XQTokenStream in my open source project xqgrammar at : http://code.google.com/p/xqgrammar/. Regards Nikolay On 04/01/2010 07:07 PM, Kumar, Amitesh wrote: Ive got ANTLR version 3.2 it doesn't seem to have UnbufferedTokenStream, there doesn't seem to be a newer version on the site Amitesh Kumar |CIB Integration | Business Infrastructure Technology | Standard Bank CIB International | Ground Floor, 20 Gresham Street, London, EC2V 7JE T: +44 [0]203 145 5575 | E: amitesh.ku...@standardbank.com -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Jim Idle Sent: 01 April 2010 16:55 To: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Just replace new CommonTokenStream(...). But you will need to fix your grammar before it will all work of course. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Kumar, Amitesh Sent: Thursday, April 01, 2010 8:52 AM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Great where is this UnbufferedTokenStream. Cheers Jim Amitesh Kumar |CIB Integration | Business Infrastructure Technology | Standard Bank CIB International | Ground Floor, 20 Gresham Street, London, EC2V 7JE T: +44 [0]203 145 5575 | E: amitesh.ku...@standardbank.com -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Jim Idle Sent: 01 April 2010 16:27 Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files Actually, I think that if you use UnbufferedTokenStream(), that this will pretty much do what you want already, but it is easy to derive from one of the token streams, and add methods tah can discard buffered tokens once you know you have dealt with them. Also, if you have comma separated files, then it is usually easier to use awk. Finally, your grammar has myriad lexical ambiguities and I am afraid it is not going to work as you have written it. You cannot have more than one lexer rule that matches the same text as the lexer is not syntax directed, it just tokenizes what it sees. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Marcin Rzeznicki Sent: Thursday, April 01, 2010 8:02 AM To: Kumar, Amitesh Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Parsing Large Files On Thu, Apr 1, 2010 at 4:26 PM, Kumar, Amitesh amitesh.ku...@standardbank.com wrote: But my general issue is that not all my data is a simple CSV file some will be multi line records. Hence I didn't want to keep a record of the tokens. Any ideas . By the way thanks for your reply. Hi You can easily implement your own TokenStream that is optimized for your use case eg. does not try to keep everything in one big array. If you explore this possibility, you will quickly discover that it is very easy thing to do and test. Hope it helps. Cheers Kumaap0 -- Greetings Marcin Rzeźnicki List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr- interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address ** * ** More information on Standard Bank is available at www.standardbank.com Everything in this email and any attachments relating to the official business of Standard Bank Group Limited and any or all subsidiaries, (the Company), is proprietary to the Company. It is confidential, legally privileged and protected by relevant laws. The Company does not own and endorse any other content. Views and opinions are those of the sender unless clearly stated as being that of the Company. The person or persons addressed in this email are the sole authorised recipient. Please notify the sender immediately if it has unintentionally, or inadvertently reached you and do not read, disclose or use the content in any way and delete this e-mail from your system. The Company cannot ensure that the integrity of this email has been maintained nor that it is free of errors, virus, interception or interference. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided
[il-antlr-interest: 28485] [antlr-interest] fixed
Once I managed to get ANTLR to give me the error message, the problem was easy to solve. I had these two lines: | ^(IF parenthesizedExpression statement statement) | ^(IF parenthesizedExpression statement) And I just changed it to: | ^(IF parenthesizedExpression statement statement?) And now it works fine. So the problem was definitely an exception in the DFA constructor that was then caught, but ignored, with no warning or error. Once I forced it to report the error, I could fix it. Jamie List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28486] [antlr-interest] building antlr: feedback
I spent several hours building antlr, which on the one hand is a tiny amount
of time to go from nothing to having a fully working parser generator :-)
Thank you all for creating this awesome software. I have some requests to
make it even easier:
1. Try building it from a fresh checkout (like I had to). I got errors below
(at the end of this email).
2. Can we distinguish Unit Tests from Functional Tests please?
Unit tests need to be *short* time-wise, so that developers are motivated to
run them every time they build. I gave up running the unit tests after about
10 minutes, and eventually had to delete the test folders entirely.
Functional Tests on the other hand can take as long as you like.
May I suggest using http://maven.apache.org/pom.html#Profiles to
differentiate between builds that are local and builds that are intended to
be pushed out to the world?
Thanks for listening,
Jamie
neutron:antlr-3.2 jamie$ mvn package
[INFO] Scanning for projects...
[INFO] Reactor build order:
[INFO] ANTLR Master build control POM
[INFO] Antlr 3 Runtime
[INFO] ANTLR Grammar Tool
[INFO] Maven plugin for ANTLR V3
[INFO]
[INFO] Building ANTLR Master build control POM
[INFO]task-segment: [package]
[INFO]
[INFO] [site:attach-descriptor]
[INFO]
[INFO] Building Antlr 3 Runtime
[INFO]task-segment: [package]
[INFO]
[INFO] [buildnumber:create {execution: default}]
[INFO] Storing buildNumber: Mar 31, 2010 22:30:11 at timestamp:
1270099811460
[INFO]
[ERROR] FATAL ERROR
[INFO]
[INFO] The scm url cannot be null.
[INFO]
[INFO] Trace
java.lang.NullPointerException: The scm url cannot be null.
at
org.apache.maven.scm.manager.AbstractScmManager.makeScmRepository(AbstractScmManager.java:181)
at org.codehaus.mojo.build.CreateMojo.getScmRepository(CreateMojo.java:722)
at org.codehaus.mojo.build.CreateMojo.getScmBranch(CreateMojo.java:593)
at org.codehaus.mojo.build.CreateMojo.execute(CreateMojo.java:452)
at
org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:451)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:558)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:499)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:478)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:330)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:291)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:142)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:336)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:129)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
[INFO]
[INFO] Total time: 5 seconds
[INFO] Finished at: Wed Mar 31 22:30:11 PDT 2010
[INFO] Final Memory: 23M/79M
[INFO]
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28487] Re: [antlr-interest] Partial parsing
Would a filtering lexer work better for you? That's used to pick out the content of interest and ignore the rest. I was able to write a SQL parser that picks out table definitions in one dialect and translates them to another dialect in something under 100 lines. ...R On Mar 30, 2010, at 9:51 AM, Anton Bychkov wrote: Hi. I'm trying to implement partial SQL parsing in my application with the help of ANTLR. It is hard to write proper grammar for some SQL expressions, so I want to leave them as a plain text. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28490] [antlr-interest] Visual Studio plugin for ANTLR grammar files
Hello all, I've been working on and off with ANTLR over the past couple years and one thing that always got me, not being a Java developer, was trying to remember how to call ANTLR to compile my grammars to C#. Moreover, I really wanted it to be part of Visual Studio so that it generated code from the grammar on change, like a Makefile. I searched around and couldn't find any such tool that already existed. Visual Studio has an API that allows you to create your own code generators, so I took a shot at it. I have a working proof of concept that works as you would expect. The generated code files are seen as child nodes hanging off the grammar file. I was curious if anyone were willing to give it a shot and let me know how it works for them. I'd like to package it up into an installer and put the source up on something like github or ohloh. Any input would be greatly appreciated. --- Shawn Poulson spoul...@explodingcoder.com List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28491] Re: [antlr-interest] antlr-interest Digest, Vol 65, Issue 1
Jim, I guess the question then is how do you go about adding import header statements into the generated c files, so that you can use Objective C objects and messages in the tree handling code? I'd rather not have to edit them by hand everytime that I make a change to the .g file. Also, is it possible to configure the lexer, so that you pass it a buffer, rather than a file stream? I currently have a lexer written in Objective-C that works that way, but I like the higher level lexer definition of antlr for ease of maintenance. Alan --- On Apr 1, 2010, at 12:00 PM, antlr-interest-requ...@antlr.org wrote: 28. Objective-C and Antlr (Alan Condit) 29. Re: Objective-C and Antlr (Jim Idle) -- Message: 29 Date: Thu, 01 Apr 2010 08:29:36 -0700 From: Jim Idle j...@temporal-wave.com Subject: Re: [antlr-interest] Objective-C and Antlr To: antlr-interest@antlr.org antlr-interest@antlr.org Message-ID: ae5820b41a49164fabb46295b2fee...@temporal-wave.com Content-Type: text/plain; charset=us-ascii Well, the C output is compatible with Objective C of course, but the ObjC target was pretty much abandoned too many years ago to be useful. I doubt that there is much advantage to using an objective C oriented target vs a straight C target to be honest. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Alan Condit Sent: Thursday, April 01, 2010 8:28 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Objective-C and Antlr Is there a string template to generate Objective-C from Antlr3.2? If not what version of Antlr does one need to go back to in order to get a version that will generate Objective-C code? Thanks, Alan List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28492] Re: [antlr-interest] Objective-C and Antlr
Jim, I changed the subject line to try and keep it in the correct thread, sorry about that. I guess the question then is how do you go about adding import header statements into the generated c files, so that you can use Objective C objects and messages in the tree handling code? I'd rather not have to edit them by hand everytime that I make a change to the .g file. Also, is it possible to configure the lexer, so that you pass it a buffer, rather than a file stream? I currently have a lexer written in Objective-C that works that way, but I like the higher level lexer definition of antlr for ease of maintenance. Alan --- On Apr 1, 2010, at 12:00 PM, antlr-interest-requ...@antlr.org wrote: 28. Objective-C and Antlr (Alan Condit) 29. Re: Objective-C and Antlr (Jim Idle) -- Message: 29 Date: Thu, 01 Apr 2010 08:29:36 -0700 From: Jim Idle j...@temporal-wave.com Subject: Re: [antlr-interest] Objective-C and Antlr To: antlr-interest@antlr.org antlr-interest@antlr.org Message-ID: ae5820b41a49164fabb46295b2fee...@temporal-wave.com Content-Type: text/plain; charset=us-ascii Well, the C output is compatible with Objective C of course, but the ObjC target was pretty much abandoned too many years ago to be useful. I doubt that there is much advantage to using an objective C oriented target vs a straight C target to be honest. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Alan Condit Sent: Thursday, April 01, 2010 8:28 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Objective-C and Antlr Is there a string template to generate Objective-C from Antlr3.2? If not what version of Antlr does one need to go back to in order to get a version that will generate Objective-C code? Thanks, Alan List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28493] Re: [antlr-interest] Visual Studio plugin for ANTLR grammar files
Hi Shawn, I went the route of tying it into MSBuild. The generated file is placed in the intermediate output directory (often obj\Debug or obj\Release). This system properly handles dependent grammars (eg. tree grammars are automatically recompiled when the lexer defining the tokens is recompiled). The latest source is in source control as the AntlrBuildTask project that's part of the C# 3 port. I still need to post an updated set of binaries for it since I fixed several bugs. Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Shawn Poulson Sent: Thursday, April 01, 2010 6:36 PM To: ANTLR Interest Subject: [antlr-interest] Visual Studio plugin for ANTLR grammar files Hello all, I've been working on and off with ANTLR over the past couple years and one thing that always got me, not being a Java developer, was trying to remember how to call ANTLR to compile my grammars to C#. Moreover, I really wanted it to be part of Visual Studio so that it generated code from the grammar on change, like a Makefile. I searched around and couldn't find any such tool that already existed. Visual Studio has an API that allows you to create your own code generators, so I took a shot at it. I have a working proof of concept that works as you would expect. The generated code files are seen as child nodes hanging off the grammar file. I was curious if anyone were willing to give it a shot and let me know how it works for them. I'd like to package it up into an installer and put the source up on something like github or ohloh. Any input would be greatly appreciated. --- Shawn Poulson spoul...@explodingcoder.com List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
