[il-antlr-interest: 28950] [antlr-interest] Java Target - How to Generate the Parser and Lexer at Runtime?

2010-05-22 Thread Sameh W. Zaky
Greetings,

I am still an ANTLR beginner..

I run a software where I generate the .g grammar file automatically. So I
would love to know how to generate the lexer and parser at runtime given the
.g file?

Thanks!
-- 
Sameh W. Zaky

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28951] Re: [antlr-interest] Java Target - How to Generate the Parser and Lexer at Runtime?

2010-05-22 Thread Bart Kiers
On Sat, May 22, 2010 at 1:33 PM, Sameh W. Zaky sameh...@gmail.com wrote:

 Greetings,

 I am still an ANTLR beginner..

 I run a software where I generate the .g grammar file automatically. So I
 would love to know how to generate the lexer and parser at runtime given
 the
 .g file?


When generating lexers/parser on the command line, you'd do something like
this:

java -cp .:antlr-3.2.jar org.antlr.Tool /path/to/your/grammar.g

(the Tool class has a static main method taking a command line parameter!)
So generating them in your own code would look like:

String yourGrammarFile = /path/to/your/grammar.g;
org.antlr.Tool.main(new String[]{yourGrammarFile});

Kind regards,

Bart.
**

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28953] Re: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35 Speed/Memory

2010-05-22 Thread Terence Parr
Very interesting. The token stream in v3.2 I don't think but first tokens by 
default, but it still buffers all of the input characters...

Ter
On May 21, 2010, at 10:00 PM, Jim Idle wrote:

 Well, you're comparing apples to cheese here. Bison/Flex do not create 
 complicated tokens with method calls and so on, so ANTLR is winning here in 
 reality. That said, I am going to implementing some of this stuff with a more 
 fly weight pattern in v4. However, it sounds like you need to implement your 
 own token stream that discards the tokens at certain points. 
 
 Jim


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28954] Re: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35 Speed/Memory

2010-05-22 Thread Jim Idle
There needs to be a 3.2.1 to catch up the other targets to the Java Target. We 
released 3.2 for your book but I had not implemented the extensions you made 
for 3.2 yet.

There is also the fact that the input stream is implemented by method calls in 
ANTLR and not direct pointers (but there is a macro you can turn on to do it 
like bison if you know the encoding in advance), and we are also keeping track 
of the follow stack for error recovery, which LALR isn't very good at (and 
there is a macro in the development branch that allows turning off the stack). 

So, the ANTLR C runtime is performing a whole bunch of tasks that bison isn't. 
The only thing to add is the token stream that can discard after certain 
checkpoints. However, if you are producing a tree, then you need the tokens in 
the tree anyway, so such things are really only good for record traversal and 
so on, which is probably better done with awk anyway.

Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of Terence Parr
 Sent: Saturday, May 22, 2010 9:19 AM
 To: ANTLR interest
 Subject: Re: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35
 Speed/Memory
 
 Very interesting. The token stream in v3.2 I don't think but first
 tokens by default, but it still buffers all of the input characters...
 
 Ter
 On May 21, 2010, at 10:00 PM, Jim Idle wrote:
 
  Well, you're comparing apples to cheese here. Bison/Flex do not
 create complicated tokens with method calls and so on, so ANTLR is
 winning here in reality. That said, I am going to implementing some of
 this stuff with a more fly weight pattern in v4. However, it sounds
 like you need to implement your own token stream that discards the
 tokens at certain points.
 
  Jim
 
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address




List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28955] Re: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35 Speed/Memory

2010-05-22 Thread Jim Idle
The other thing to add here is that you are using the $xxx.text references, and 
these do not free up the string memory until you free the parser. With this 
many inputs, the memory usage you are seeing is probably this first and not the 
tokens.

Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of Bob
 Sent: Friday, May 21, 2010 7:47 PM
 To: antlr-interest@antlr.org
 Subject: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35
 Speed/Memory
 
 A tiny grammar was implemented in both Antlr and Bison+Flex (shown
 below).
 
 Test files repeating two lines (shown below) were made in 6 different
 
 sizes.
 
 One executable compiled with command line switch choosing either
 
 Antlr or Bison+Flex.
 
 One run with empty actions, one run with actions populated, to compare
 
 pure parsing with some actual work.
 
 
 
 Results:
 
 
 CPU timePeak Memory
 
 File Name File Size # modules #tokens  Bison Antlr  Bison Antlr
 
 Action bodies empty:
 
 source.v10m 460mb  10m  150m 28s  572k  *
 
 source.v5m   230mb   5m75m   15s  572k
 *
 
 source.v2.5m115mb   2.5m 37m   7s   572k  *
 
 source.v1m   46mb 1m15m2s
 572k  *
 
 source.v500k23mb500k  7.5m  1s572k
 *
 
 source.v250k11mb250k  3.7m1s   4s 572k
 1.7g
 ---
 
 Action bodies populated:
 
 source.v250k11mb250k  3.7m 9s   13s477m
 1.7g
 ---
 
 
 
 * Antlr ran out of memory at 2gb
 
 
 
 Comments:
 
 
 
 1. I expected the requirement that the entire file be resident in
 memory
 
to be the memory glut. Surprise! Quick inspection suggests an
 initial
 
tokenizing of the entire in-memory file consumes gobbs of memory,
 pushing
 
a small footprint up to 1.7gb before releasing it. Only the smallest
 
test file was under the runable 32 bit 2gb limit. Please fix!!
 
 
 
 2. Speed is clearly slower than bison+flex, however empty actions don't
 make
 
interesting programs. The test with actions enabled shows a 9s vs.
 13s
 
difference, considerable less than the empty action case.
 
 
 
 3. If you've never setup bison+flex I have only one comment: !...@%$#.
 Two
 
thumbs up for Antlr.
 
 
 
 Details:
 
 
 
   Vista 64, AMD opteron 2.4Ghz, 16gb ram
 
   Visual Studio 2008 Sp1
 
   One exe file with both Antlr and Bison+Flex, targeting 32 bit
 
   Full Optimization (/Ox), Inline Any suitable (/Ob2), Favor Small Code
 (/Os)
 
   Versions:
 
 Antlr 3.2
 
 Bison 2.4.2 LR(1)
 
 Flex  2.5.35
 
 
 
 
 
 --- Input file -
 
 module tiptop #(int p1=3, p2=4 );
 
 endmodule
 
 ... repeat to the indicated number of modules ...
 
 --- Antlr Grammar --
 
 source_text : description ( description )*
 
 ;
 
 description : module_declaration
 
 ;
 
 module_declaration : module_ansi_header ENDMODULE ( ':'
 module_identifier )?
 
 { act_module(); }
 
 ;
 
 module_ansi_header : MODULE_KEYWORD module_identifier (
 parameter_port_list
 )? ';'
 
 ;
 
 module_identifier : identifier
 
 ;
 
 parameter_port_list
 
 : '#' '(' parameter_port_declaration ( ','
 parameter_port_declaration )*
 ')'
 
 | '#' '(' ')'
 
 ;
 
 parameter_port_declaration returns [void* node]
 
 scope {
 
 void* type;
 
 void* head;
 
 void* tail;
 
 }
 
 : data_type
 
 { $parameter_port_declaration::type = $data_type.node;
 
 $parameter_port_declaration::head=NULL; }
 list_of_param_assignments
 
 { $node = $parameter_port_declaration::head; }
 
 ;
 
 list_of_param_assignments
 
 : param_assignment ( ',' param_assignment )*
 
 ;
 
 param_assignment
 
 : parameter_identifier '=' constant_param_expression
 
 { act_param_assignment
 
 (
 
  $parameter_port_declaration::head,
 
  $parameter_port_declaration::tail,
 
 $parameter_identifier.node,
 
 $parameter_port_declaration::type,
 
 $constant_param_expression.node
 
 );
 
 }
 
 ;
 
 constant_param_expression returns [void* node]
 
 : constant_mintypmax_expression
 
 { $node = $constant_mintypmax_expression.node; }
 
 //| '$'
 
 ;
 
 constant_mintypmax_expression returns [void* node]
 
 : constant_expression
 
 { $node = $constant_expression.node; }
 
 ;
 
 // Deviate from LRM
 
 constant_expression returns [void* node]
 
 : expr { $node = $expr.node; }
 
 ;
 
 parameter_identifier returns [void* node]
 
 : identifier { $node = $identifier.node; }
 
 ;
 
 data_type returns [void* node]
 
 : integer_atom_type signing
 

[il-antlr-interest: 28956] Re: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35 Speed/Memory

2010-05-22 Thread Bob
Perhaps you are correct regarding the strings. Looking at the antlr
generated identifier rule code, I see 

SIMPLE_IDENTIFIER12 is not freed.  Is there a mechanism whereby I can free
the string when the rule is complete?

 

After reading the file into memory, the process size is ~11meg (size of
input file), then it rises to 853Meg before the 1st module .. endmodule
from the input is recognized. As subsequent module..endmodule's are
recognized memory rises to 1.2+Gb.

 

If you have a solution to free the strings when a rule is complete, I'll try
that and see where it stands. The expansion from 11Meg to over 1.2Gb is
quite a bit for strings!

 

 

Rule:

 

identifier returns [void* node] : SIMPLE_IDENTIFIER

{ $node = act_identifier( $SIMPLE_IDENTIFIER.text-chars ); }

;

 

Code:

 

static void*

identifier(pNuVParser ctx)

{   

void* node = NULL;

 

pANTLR3_COMMON_TOKENSIMPLE_IDENTIFIER12;

 

/* Initialize rule variables

 */

 

 

SIMPLE_IDENTIFIER12   = NULL;

 

{

// NuV.g:102:33: ( SIMPLE_IDENTIFIER )

// NuV.g:102:35: SIMPLE_IDENTIFIER

{

SIMPLE_IDENTIFIER12 = (pANTLR3_COMMON_TOKEN)
MATCHT(SIMPLE_IDENTIFIER, FOLLOW_SIMPLE_IDENTIFIER_in_identifier608); 

if  (HASEXCEPTION())

{

goto ruleidentifierEx;

}

 

{

 node= act_identifier(
(SIMPLE_IDENTIFIER12-getText(SIMPLE_IDENTIFIER12))-chars ); 

}

 

}

 

}



 

// This is where rules clean up and exit

//

goto ruleidentifierEx; /* Prevent compiler warnings */

ruleidentifierEx: ;

 

if (HASEXCEPTION())

{

PREPORTERROR();

PRECOVER();

}

 

 

return node;

}

 

-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Jim Idle
Sent: Saturday, May 22, 2010 10:37 AM
To: antlr-interest@antlr.org
Subject: Re: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35
Speed/Memory

 

The other thing to add here is that you are using the $xxx.text references,
and these do not free up the string memory until you free the parser. With
this many inputs, the memory usage you are seeing is probably this first and
not the tokens.

 

Jim

 

 -Original Message-

 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-

 boun...@antlr.org] On Behalf Of Bob

 Sent: Friday, May 21, 2010 7:47 PM

 To: antlr-interest@antlr.org

 Subject: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35

 Speed/Memory

 

 A tiny grammar was implemented in both Antlr and Bison+Flex (shown

 below).

 

 Test files repeating two lines (shown below) were made in 6 different

 

 sizes.

 

 One executable compiled with command line switch choosing either

 

 Antlr or Bison+Flex.

 

 One run with empty actions, one run with actions populated, to compare

 

 pure parsing with some actual work.

 

 

 

 Results:

 

 

 CPU timePeak Memory

 

 File Name File Size # modules #tokens  Bison Antlr  Bison Antlr

 

 Action bodies empty:

 

 source.v10m 460mb  10m  150m 28s  572k  *

 

 source.v5m   230mb   5m75m   15s  572k

 *

 

 source.v2.5m115mb   2.5m 37m   7s   572k  *

 

 source.v1m   46mb 1m15m2s

 572k  *

 

 source.v500k23mb500k  7.5m  1s572k

 *

 

 source.v250k11mb250k  3.7m1s   4s 572k

 1.7g

 ---

 

 Action bodies populated:

 

 source.v250k11mb250k  3.7m 9s   13s477m

 1.7g

 ---

 

 

 

 * Antlr ran out of memory at 2gb

 

 

 

 Comments:

 

 

 

 1. I expected the requirement that the entire file be resident in

 memory

 

to be the memory glut. Surprise! Quick inspection suggests an

 initial

 

tokenizing of the entire in-memory file consumes gobbs of memory,

 pushing

 

a small footprint up to 1.7gb before releasing it. Only the smallest

 

test file was under the runable 32 bit 2gb limit. Please fix!!

 

 

 

 2. Speed is clearly slower than bison+flex, however empty actions don't

 make

 

interesting programs. The test with actions enabled shows a 9s vs.

 13s

 

difference, considerable less than the empty action case.

 

 

 

 3. If you've never setup bison+flex I have only one comment: !...@%$#.

 Two

 

thumbs up for Antlr.

 

 

 

 Details:

 

 

 

   Vista 64, AMD opteron 2.4Ghz, 16gb ram

 

   Visual Studio 2008 Sp1

 

   One exe file with both Antlr and Bison+Flex, targeting 32 bit

 

   Full Optimization (/Ox), Inline Any suitable (/Ob2), Favor Small Code

 (/Os)

 

   Versions:

 

 Antlr 3.2

 

 Bison 2.4.2 LR(1)

 

 Flex  2.5.35

 

 

 

 

 

 --- Input file 

[il-antlr-interest: 28957] Re: [antlr-interest] Java Target - How to Generate the Parser and Lexer at Runtime?

2010-05-22 Thread Sameh W. Zaky
Dear Brat,

You're right..

But now I need to change my question because I found out that Parser.java
generated from this action is different than Parser.java generated when I
click 'Run..' in ANTLRWorks and give it an input text..

So my new question is: How, from the command line, do I run the grammar
(giving it an input text file, let's say)?

On Sat, May 22, 2010 at 1:54 PM, Bart Kiers bki...@gmail.com wrote:

 On Sat, May 22, 2010 at 1:33 PM, Sameh W. Zaky sameh...@gmail.com wrote:

 Greetings,

 I am still an ANTLR beginner..

 I run a software where I generate the .g grammar file automatically. So I
 would love to know how to generate the lexer and parser at runtime given
 the
 .g file?


 When generating lexers/parser on the command line, you'd do something like
 this:

 java -cp .:antlr-3.2.jar org.antlr.Tool /path/to/your/grammar.g

 (the Tool class has a static main method taking a command line parameter!)
 So generating them in your own code would look like:

 String yourGrammarFile = /path/to/your/grammar.g;
 org.antlr.Tool.main(new String[]{yourGrammarFile});

 Kind regards,

 Bart.
 **




-- 
Sameh W. Zaky

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28959] Re: [antlr-interest] Java Target - How to Generate the Parser and Lexer at Runtime?

2010-05-22 Thread Sameh W. Zaky
OK, I will clarify my question (sorry that I was not clear in my previous
mail).

The 2 files are different mainly in the constructors

*When I use the method you told me about in the first message, the output
Parser.java has the constructors:*
*public **RulesGrammarParser(TokenStream input) {*
*this(input, new RecognizerSharedState());*
*}*
*public RulesGrammarParser(TokenStream input, RecognizerSharedState state) {
*
*super(input, state);*
* *
*}*


*But when I use click 'Run..' in ANTLRWorks I get the output Parser.java has
the constructors:*

*public RulesGrammarParser(TokenStream input) {*
*this(input, DebugEventSocketProxy.DEFAULT_DEBUGGER_PORT, new
RecognizerSharedState());*
*}*
*public RulesGrammarParser(TokenStream input, int port,
RecognizerSharedState state) {*
*super(input, state);*
*DebugEventSocketProxy proxy =*
*new DebugEventSocketProxy(this, port, null);*
*setDebugListener(proxy);*
*try {*
*proxy.handshake();*
*}*
*catch (IOException ioe) {*
*reportError(ioe);*
*}*
*}*
*public RulesGrammarParser(TokenStream input, DebugEventListener dbg) {*
*super(input, dbg, new RecognizerSharedState());*

*}*


*In addition to this method:*
*protected boolean evalPredicate(boolean result, String predicate) {*
*dbg.semanticPredicate(result, predicate);*
*return result;*
*}*

*and these declarations:*
*public static final String[] ruleNames = new String[] {*
*invalidRule, in, evaluation, gt, consequence, lte, *
*times, lte_math, divided_by, enumeration_value,
gte_math, *
*range, check, variable, last_connection, numeric_value, *
*term, gte, modulus, plus, source_name, connection,
lt, *
*source, function, equal, name_of_quality, condition,
primary, *
*minus, value_of_quality, arithmeticExpression*
*};*
* *
*public int ruleLevel = 0;*
*public int getRuleLevel() { return ruleLevel; }*
*public void incRuleLevel() { ruleLevel++; }*
*public void decRuleLevel() { ruleLevel--; }*


And by input text, I mean the input.. The text that I write in ANTLRWorks
when I click 'Run..'

Thanks for your time :-)

On Sat, May 22, 2010 at 8:41 PM, Bart Kiers bki...@gmail.com wrote:

 But now I need to change my question because I found out that Parser.java
 generated from this action is different than Parser.java generated when I
 click 'Run..' in ANTLRWorks and give it an input text..


 Please explain what different means. And what do you mean by input
 text?

 Kind regards,

 Bart.




-- 
Sameh W. Zaky

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28960] Re: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35 Speed/Memory

2010-05-22 Thread Jim Idle
The string stuff is just a convenience method for simple stuff. For real 
programs you should not do that but use the pointer to start and end that is 
contained in the token. Then you can just memcpy straight from the input source 
into whatever structure you are using.

Also, if your input is 8 bit then follow the examples and use direct pointers. 
If you don't need error recovery then in the next release you can turn off the 
stack overhead, or you can do it yourself by undefining the macrothat does this 
in the generated code.

Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of Bob
 Sent: Saturday, May 22, 2010 11:33 AM
 To: antlr-interest@antlr.org
 Subject: Re: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35
 Speed/Memory
 
 Perhaps you are correct regarding the strings. Looking at the antlr
 generated identifier rule code, I see
 
 SIMPLE_IDENTIFIER12 is not freed.  Is there a mechanism whereby I can
 free
 the string when the rule is complete?
 
 
 
 After reading the file into memory, the process size is ~11meg (size of
 input file), then it rises to 853Meg before the 1st module ..
 endmodule
 from the input is recognized. As subsequent module..endmodule's are
 recognized memory rises to 1.2+Gb.
 
 
 
 If you have a solution to free the strings when a rule is complete,
 I'll try
 that and see where it stands. The expansion from 11Meg to over 1.2Gb is
 quite a bit for strings!
 
 
 
 
 
 Rule:
 
 
 
 identifier returns [void* node] : SIMPLE_IDENTIFIER
 
 { $node = act_identifier( $SIMPLE_IDENTIFIER.text-chars ); }
 
 ;
 
 
 
 Code:
 
 
 
 static void*
 
 identifier(pNuVParser ctx)
 
 {
 
 void* node = NULL;
 
 
 
 pANTLR3_COMMON_TOKENSIMPLE_IDENTIFIER12;
 
 
 
 /* Initialize rule variables
 
  */
 
 
 
 
 
 SIMPLE_IDENTIFIER12   = NULL;
 
 
 
 {
 
 // NuV.g:102:33: ( SIMPLE_IDENTIFIER )
 
 // NuV.g:102:35: SIMPLE_IDENTIFIER
 
 {
 
 SIMPLE_IDENTIFIER12 = (pANTLR3_COMMON_TOKEN)
 MATCHT(SIMPLE_IDENTIFIER, FOLLOW_SIMPLE_IDENTIFIER_in_identifier608);
 
 if  (HASEXCEPTION())
 
 {
 
 goto ruleidentifierEx;
 
 }
 
 
 
 {
 
  node= act_identifier(
 (SIMPLE_IDENTIFIER12-getText(SIMPLE_IDENTIFIER12))-chars );
 
 }
 
 
 
 }
 
 
 
 }
 
 
 
 
 
 // This is where rules clean up and exit
 
 //
 
 goto ruleidentifierEx; /* Prevent compiler warnings */
 
 ruleidentifierEx: ;
 
 
 
 if (HASEXCEPTION())
 
 {
 
 PREPORTERROR();
 
 PRECOVER();
 
 }
 
 
 
 
 
 return node;
 
 }
 
 
 
 -Original Message-
 From: antlr-interest-boun...@antlr.org
 [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Jim Idle
 Sent: Saturday, May 22, 2010 10:37 AM
 To: antlr-interest@antlr.org
 Subject: Re: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35
 Speed/Memory
 
 
 
 The other thing to add here is that you are using the $xxx.text
 references,
 and these do not free up the string memory until you free the parser.
 With
 this many inputs, the memory usage you are seeing is probably this
 first and
 not the tokens.
 
 
 
 Jim
 
 
 
  -Original Message-
 
  From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 
  boun...@antlr.org] On Behalf Of Bob
 
  Sent: Friday, May 21, 2010 7:47 PM
 
  To: antlr-interest@antlr.org
 
  Subject: [antlr-interest] Antlr 3.2 vs. Bison 2.4.2+Flex 2.5.35
 
  Speed/Memory
 
 
 
  A tiny grammar was implemented in both Antlr and Bison+Flex (shown
 
  below).
 
 
 
  Test files repeating two lines (shown below) were made in 6 different
 
 
 
  sizes.
 
 
 
  One executable compiled with command line switch choosing either
 
 
 
  Antlr or Bison+Flex.
 
 
 
  One run with empty actions, one run with actions populated, to
 compare
 
 
 
  pure parsing with some actual work.
 
 
 
 
 
 
 
  Results:
 
 
 
 
 
  CPU timePeak Memory
 
 
 
  File Name File Size # modules #tokens  Bison Antlr  Bison Antlr
 
 
 
  Action bodies empty:
 
 
 
  source.v10m 460mb  10m  150m 28s  572k  *
 
 
 
  source.v5m   230mb   5m75m   15s
 572k
 
  *
 
 
 
  source.v2.5m115mb   2.5m 37m   7s   572k
 *
 
 
 
  source.v1m   46mb 1m15m2s
 
  572k  *
 
 
 
  source.v500k23mb500k  7.5m  1s
 572k
 
  *
 
 
 
  source.v250k11mb250k  3.7m1s   4s 572k
 
  1.7g
 
  ---
 
 
 
  Action bodies populated:
 
 
 
  source.v250k11mb250k  3.7m 9s   13s477m
 
  1.7g
 
  ---
 
 
 
 
 
 
 
  * Antlr ran out of memory at 2gb
 
 
 
 
 
 
 
  Comments:
 
 
 
 
 
 
 
  1. I expected the requirement that the entire file be resident in
 
  memory
 
 
 
 to 

[il-antlr-interest: 28961] Re: [antlr-interest] Java Target - How to Generate the Parser and Lexer at Runtime?

2010-05-22 Thread Jim Idle
You need to read the wiki articles and follow the examples.

Jim

 -Original Message-
 From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
 boun...@antlr.org] On Behalf Of Sameh W. Zaky
 Sent: Saturday, May 22, 2010 11:56 AM
 To: Bart Kiers
 Cc: antlr-interest@antlr.org
 Subject: Re: [antlr-interest] Java Target - How to Generate the Parser
 and Lexer at Runtime?
 
 OK, I will clarify my question (sorry that I was not clear in my
 previous
 mail).
 
 The 2 files are different mainly in the constructors
 
 *When I use the method you told me about in the first message, the
 output
 Parser.java has the constructors:*
 *public **RulesGrammarParser(TokenStream input) {*
 *this(input, new RecognizerSharedState());*
 *}*
 *public RulesGrammarParser(TokenStream input, RecognizerSharedState
 state) {
 *
 *super(input, state);*
 * *
 *}*
 
 
 *But when I use click 'Run..' in ANTLRWorks I get the output
 Parser.java has
 the constructors:*
 
 *public RulesGrammarParser(TokenStream input) {*
 *this(input, DebugEventSocketProxy.DEFAULT_DEBUGGER_PORT,
 new
 RecognizerSharedState());*
 *}*
 *public RulesGrammarParser(TokenStream input, int port,
 RecognizerSharedState state) {*
 *super(input, state);*
 *DebugEventSocketProxy proxy =*
 *new DebugEventSocketProxy(this, port, null);*
 *setDebugListener(proxy);*
 *try {*
 *proxy.handshake();*
 *}*
 *catch (IOException ioe) {*
 *reportError(ioe);*
 *}*
 *}*
 *public RulesGrammarParser(TokenStream input, DebugEventListener dbg)
 {*
 *super(input, dbg, new RecognizerSharedState());*
 
 *}*
 
 
 *In addition to this method:*
 *protected boolean evalPredicate(boolean result, String predicate) {*
 *dbg.semanticPredicate(result, predicate);*
 *return result;*
 *}*
 
 *and these declarations:*
 *public static final String[] ruleNames = new String[] {*
 *invalidRule, in, evaluation, gt, consequence, lte,
 *
 *times, lte_math, divided_by, enumeration_value,
 gte_math, *
 *range, check, variable, last_connection,
 numeric_value, *
 *term, gte, modulus, plus, source_name, connection,
 lt, *
 *source, function, equal, name_of_quality, condition,
 primary, *
 *minus, value_of_quality, arithmeticExpression*
 *};*
 * *
 *public int ruleLevel = 0;*
 *public int getRuleLevel() { return ruleLevel; }*
 *public void incRuleLevel() { ruleLevel++; }*
 *public void decRuleLevel() { ruleLevel--; }*
 
 
 And by input text, I mean the input.. The text that I write in
 ANTLRWorks
 when I click 'Run..'
 
 Thanks for your time :-)
 
 On Sat, May 22, 2010 at 8:41 PM, Bart Kiers bki...@gmail.com wrote:
 
  But now I need to change my question because I found out that
 Parser.java
  generated from this action is different than Parser.java generated
 when I
  click 'Run..' in ANTLRWorks and give it an input text..
 
 
  Please explain what different means. And what do you mean by input
  text?
 
  Kind regards,
 
  Bart.
 
 
 
 
 --
 Sameh W. Zaky
 
 List: http://www.antlr.org/mailman/listinfo/antlr-interest
 Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
 email-address




List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28962] Re: [antlr-interest] Coding rule checking for Ada 95

2010-05-22 Thread Pro
I give a try to TXL.
Here is my TXL grammar for Ada 2005, under BSD-style licence :
  http://github.com/bu2/ada-2005-txl-grammar

And my roadmap :
http://wiki.github.com/bu2/fadacop/roadmap

Bruno.

On 22 avr. 2010, at 22:02, bruno le hyaric bruno.lehya...@gmail.com  
wrote:

 Hum, okay...

 My problem is : the better I get it running, the better it is for my  
 ass ^^

 Now, I'm evaluating the feasibility of a solution :
 - based on ANTLR and Java target (thanks to Aonix  Hibachi eclipse  
 plugin)
 - with a Scala overlay to get a functionnal approach for all tree
 walking/matching algorithm

 I'm on the way to make a little prototype...

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28965] [antlr-interest] Invitation to connect on LinkedIn

2010-05-22 Thread Kunal Sawlani
LinkedIn
Kunal Sawlani requested to add you as a connection on LinkedIn:
--

Ian,

I'd like to add you to my professional network on LinkedIn.

- Kunal

Accept invitation from Kunal Sawlani
http://www.linkedin.com/e/XJ_Jh42eHeYQ_eJ2S5_J_0Ueq9jQFT3yBwA6v-/blk/I2064900695_2/1BpC5vrmRLoRZcjkkZt5YCpnlOt3RApnhMpmdzgmhxrSNBszYOnPkVdz0MejgSc399bTFniAllrDESbPAOc3cMcPkOcP4LrCBxbOYWrSlI/EML_comm_afe/

View invitation from Kunal Sawlani
http://www.linkedin.com/e/XJ_Jh42eHeYQ_eJ2S5_J_0Ueq9jQFT3yBwA6v-/blk/I2064900695_2/39vdjASc30Vd3oMcAALqnpPbOYWrSlI/svi/
 
--

DID YOU KNOW LinkedIn can help you find the right service providers using 
recommendations from your trusted network? Using LinkedIn Services, you can 
take the risky guesswork out of selecting service providers by reading the 
recommendations of credible, trustworthy members of your network. 
http://www.linkedin.com/e/svp/inv-25/

 
--
(c) 2010, LinkedIn Corporation

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28966] [antlr-interest] Dynamic scope for lexer rule

2010-05-22 Thread Junkman
Greetings,

I've added an attribute with dynamic scoping to a lexer rule, and when
generating code, I'm encountering an internal error.  Listed below is
partial call stack reported:

error(10): internal error: Junkscript.g : java.lang.NullPointerException

org.antlr.grammar.v2.DefineGrammarItemsWalker.ruleScopeSpec(DefineGrammarItemsWalker.java:1050)
at
org.antlr.grammar.v2.DefineGrammarItemsWalker.rule(DefineGrammarItemsWalker.java:891)
at
org.antlr.grammar.v2.DefineGrammarItemsWalker.rules(DefineGrammarItemsWalker.java:576)
at
org.antlr.grammar.v2.DefineGrammarItemsWalker.grammarSpec(DefineGrammarItemsWalker.java:361)
at
org.antlr.grammar.v2.DefineGrammarItemsWalker.grammar(DefineGrammarItemsWalker.java:193)
at org.antlr.tool.Grammar.defineGrammarSymbols(Grammar.java:702)
at
org.antlr.tool.CompositeGrammar.defineGrammarSymbols(CompositeGrammar.java:351)
...


Is dynamic scoping allowed for lexer rule attributes?

Thanks for any info.

J





List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28967] Re: [antlr-interest] Building Antlr-3.2 from source on a Mac with Maven

2010-05-22 Thread Naveen
On May 22, 3:13 pm, Alan Condit acon...@ipns.com wrote:
 Has anybody with a Mac successfully rebuilt Antlr-3.2 from source?

I built it successfully like this:
download latest source from http://github.com/antlr/antlr
edit the pom.xml in the root folder, remove the following two lines:
modulegunit/module
modulegunit-maven-plugin/module
there were errors building the gunit plugin otherwise on mac.
you may need an existing antlr-3.2 and antlr-2.7.7.jar in your
classpath
in the root folder again, run:
mvn -N install
mvn -Dmaven.test.skip=true
mvn -Dmaven.test.skip=true package assembly:assembly
this will put a antlr-master-3.2.1-SNAPSHOT-completejar.jar weighing
1.6mb in the target folder.



List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.



[il-antlr-interest: 28968] Re: [antlr-interest] Building Antlr-3.2 from source on a Mac with Maven

2010-05-22 Thread Alan Condit
Naveen,

Thanks, I got it working. I just wondered if it was possible for others and I 
was just messing up.
I unzipped a new copy of the files, fixed the pom.xml so that it didn't get the 
scm error, and everything worked.
Actually it stopped with errors a couple of times but by restarting it, it 
finally went on to completion.

Alan
---

Alan Condit
1085 Tierra Ct.
Woodburn, OR 97071

Email -- acon...@ipns.com
Home-Office (503) 982-0906


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-- 
You received this message because you are subscribed to the Google Groups 
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to 
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en.