Re: [sqlite] Lemon Parser vs bubble-generator.tcl

[Richard Hipp]

On 8/31/18, Warren Young  wrote:
> They’re separate.  Here’s the Tcl source for the bubble diagrams:
>
> https://www.sqlite.org/docsrc/file/?name=art/syntax/bubble-generator-data.tcl
>
> …and here’s the Lemon grammar for SQLite’s SQL parser:
>
> https://www.sqlite.org/src/file?name=src/parse.y
>

The parse.y file is designed to generate a fast parser automaton.  The
bubble-generator-data.tcl file is designed to generate output that is
easily understandable by humans.  These two files are therefore at
cross-purposes.

One little-noticed aspect of SQLite that the source code to the
LALR(1) parser generator (Lemon) is included in the source tree.  The
fact that SQLite uses its own parser generator, rather than relying
yacc/bison or similar, has proven very helpful over the years.  We've
been able to add features, fix bugs, and make optimizations to the
parser that would not have been possible had we been dependent on an
outside tool.
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon Parser vs bubble-generator.tcl

[Scott Robison]

On Fri, Aug 31, 2018 at 2:59 PM Warren Young  wrote:
>
> On Aug 31, 2018, at 1:55 PM, Scott Robison  wrote:
> >
> > Is one generated from the other, or are they maintained separately?
>
> They’re separate.  Here’s the Tcl source for the bubble diagrams:

As I suspected having looked at them in the source tree. Thanks for
the confirmation.

-- 
Scott Robison
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon Parser vs bubble-generator.tcl

[Warren Young]

On Aug 31, 2018, at 1:55 PM, Scott Robison  wrote:
> 
> Is one generated from the other, or are they maintained separately?

They’re separate.  Here’s the Tcl source for the bubble diagrams:


https://www.sqlite.org/docsrc/file/?name=art/syntax/bubble-generator-data.tcl

…and here’s the Lemon grammar for SQLite’s SQL parser:

https://www.sqlite.org/src/file?name=src/parse.y

Here’s a brief article on the script that may explain a few things:

https://wiki.tcl.tk/21708

> I...want to do something related to
> identifying SQL syntax at C++ compile time.

In that case, I’d study how ./src/parse.y gets turned into ./parse.[ch] in the 
SQLite source tree, then try to reuse as much of parse.y and its associated 
bits as you can.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon Parser vs bubble-generator.tcl

[Scott Robison]

SQLite language as implemented uses a Lemon based parser. The syntax
diagrams are created from the bubble-generator.tcl script. Is one
generated from the other, or are they maintained separately? Is one
(or another file that I don't know exists yet) the canonical
description of the "complete" SQLite syntax?

I ask because I'm working on a project idea for a senior project class
I'll be taking this fall and want to do something related to
identifying SQL syntax at C++ compile time. In doing a little research
/ planning, I started by creating my own syntax description from some
of the diagrams, then realized "I should just use portions of the
syntax diagram script directly", then started wondering about the
parser vs the diagram script.

-- 
Scott Robison
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] lemon parser bug, %destructors directive not appended into source code

[Richard Hipp]

On 8/16/16, xiaobing  wrote:
> In lemon.c:ReportTable() , when " Combine duplicate destructors into a
> single case ", sp2->destructor is set to 0, but later it is used in
> tranlate_code to generate destructor.
> so if you have grammar like this:
> %destructor expr_a { expr_free($$); }
> %destructor expr_b { expr_free($$); }  // this one will be lost, unless
> you insert some comment so strcmp return nonzero
> expr ::= expr_a expr_b.
> the generated code will include destructor of expr_a, however, destructor
> of expr_b will be lost.

Excellent analysis, Xiaobing.  Thanks.

I have checked in a slightly different fix.  Please confirm that what
I have checked in actually fixes your problem.
https://www.sqlite.org/src/fdiff?dc=20=e3aa9ba3469804d7=e4fb7d73ac88


>
> Here is my patch to lemon.c:
>
> --- old/lemon.c 2016-05-18 18:07:00.0 +0800
> +++ new/lemon.c   2016-08-16 20:57:36.799178091 +0800
> @@ -263,6 +263,7 @@
>int useCnt;  /* Number of times used */
>char *destructor;/* Code which executes whenever this symbol is
> ** popped from the stack during error
> processing */
> +  int destructor_emitted;  /* Is the destructor code emitted */
>int destLineno;  /* Line number for start of destructor */
>char *datatype;  /* The data type of information held by this
> ** object. Only used if type==NONTERMINAL */
> @@ -4351,8 +4352,9 @@
>}
>for(i=0; insymbol; i++){
>  struct symbol *sp = lemp->symbols[i];
> -if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ) continue;
> +if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ||
> sp->destructor_emitted) continue;
>  fprintf(out,"case %d: /* %s */\n", sp->index, sp->name); lineno++;
> +sp->destructor_emitted = 1;
>
>  /* Combine duplicate destructors into a single case */
>  for(j=i+1; jnsymbol; j++){
> @@ -4362,7 +4364,7 @@
>&& strcmp(sp->destructor,sp2->destructor)==0 ){
>   fprintf(out,"case %d: /* %s */\n",
>   sp2->index, sp2->name); lineno++;
> - sp2->destructor = 0;
> + sp2->destructor_emitted = 1;
>}
>  }
>
> @@ -4876,6 +4878,7 @@
>  sp->firstset = 0;
>  sp->lambda = LEMON_FALSE;
>  sp->destructor = 0;
> +sp->destructor_emitted = 0;
>  sp->destLineno = 0;
>  sp->datatype = 0;
>  sp->useCnt = 0;
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] lemon parser bug, %destructors directive not appended into source code

[xiaobing]

In lemon.c:ReportTable() , when " Combine duplicate destructors into a
single case ", sp2->destructor is set to 0, but later it is used in
tranlate_code to generate destructor.
so if you have grammar like this:
%destructor expr_a { expr_free($$); }
%destructor expr_b { expr_free($$); }  // this one will be lost, unless
you insert some comment so strcmp return nonzero
expr ::= expr_a expr_b.
the generated code will include destructor of expr_a, however, destructor
of expr_b will be lost.

Here is my patch to lemon.c:

--- old/lemon.c 2016-05-18 18:07:00.0 +0800
+++ new/lemon.c   2016-08-16 20:57:36.799178091 +0800
@@ -263,6 +263,7 @@
   int useCnt;  /* Number of times used */
   char *destructor;/* Code which executes whenever this symbol is
** popped from the stack during error
processing */
+  int destructor_emitted;  /* Is the destructor code emitted */
   int destLineno;  /* Line number for start of destructor */
   char *datatype;  /* The data type of information held by this
** object. Only used if type==NONTERMINAL */
@@ -4351,8 +4352,9 @@
   }
   for(i=0; insymbol; i++){
 struct symbol *sp = lemp->symbols[i];
-if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ) continue;
+if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ||
sp->destructor_emitted) continue;
 fprintf(out,"case %d: /* %s */\n", sp->index, sp->name); lineno++;
+sp->destructor_emitted = 1;

 /* Combine duplicate destructors into a single case */
 for(j=i+1; jnsymbol; j++){
@@ -4362,7 +4364,7 @@
   && strcmp(sp->destructor,sp2->destructor)==0 ){
  fprintf(out,"case %d: /* %s */\n",
  sp2->index, sp2->name); lineno++;
- sp2->destructor = 0;
+ sp2->destructor_emitted = 1;
   }
 }

@@ -4876,6 +4878,7 @@
 sp->firstset = 0;
 sp->lambda = LEMON_FALSE;
 sp->destructor = 0;
+sp->destructor_emitted = 0;
 sp->destLineno = 0;
 sp->datatype = 0;
 sp->useCnt = 0;
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] lemon parser bug, %destructors directive not appended into source code

[xiaobing]

In lemon.c:ReportTable() , when " Combine duplicate destructors into a
single case ", sp2->destructor is set to 0, but later it is used in
tranlate_code to generate destructor.
so if you have grammar like this:
%destructor expr_a { expr_free($$); }
%destructor expr_b { expr_free($$); }  // this one will be lost, unless
you insert some comment so strcmp return nonzero
expr ::= expr_a expr_b.
the generated code will include destructor of expr_a, however, destructor
of expr_b will be lost.

Here is my patch to lemon.c:

--- old/lemon.c 2016-05-18 18:07:00.0 +0800
+++ new/lemon.c   2016-08-16 20:57:36.799178091 +0800
@@ -263,6 +263,7 @@
   int useCnt;  /* Number of times used */
   char *destructor;/* Code which executes whenever this symbol is
** popped from the stack during error
processing */
+  int destructor_emitted;  /* Is the destructor code emitted */
   int destLineno;  /* Line number for start of destructor */
   char *datatype;  /* The data type of information held by this
** object. Only used if type==NONTERMINAL */
@@ -4351,8 +4352,9 @@
   }
   for(i=0; insymbol; i++){
 struct symbol *sp = lemp->symbols[i];
-if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ) continue;
+if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ||
sp->destructor_emitted) continue;
 fprintf(out,"case %d: /* %s */\n", sp->index, sp->name); lineno++;
+sp->destructor_emitted = 1;

 /* Combine duplicate destructors into a single case */
 for(j=i+1; jnsymbol; j++){
@@ -4362,7 +4364,7 @@
   && strcmp(sp->destructor,sp2->destructor)==0 ){
  fprintf(out,"case %d: /* %s */\n",
  sp2->index, sp2->name); lineno++;
- sp2->destructor = 0;
+ sp2->destructor_emitted = 1;
   }
 }

@@ -4876,6 +4878,7 @@
 sp->firstset = 0;
 sp->lambda = LEMON_FALSE;
 sp->destructor = 0;
+sp->destructor_emitted = 0;
 sp->destLineno = 0;
 sp->datatype = 0;
 sp->useCnt = 0;
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser

[Chris verBurg]

On Tue, Apr 6, 2010 at 10:40 PM, Dan Kennedy  wrote:

>
> On Apr 7, 2010, at 3:23 AM, Chris verBurg wrote:
>
> > Hehe, okay, here I go.  :)
> >
> >
> > I'm trying to replace an existing flex/bison parser with an re2c/lemon
> > parser, but I'm running into a methodological problem.  I have a
> > hypothetical grammar like this:
> >
> >  file ::= FOO str .
> >  file ::= BAR str .
>
> See the use of the "%fallback" directive in the parse.y
> file that is part of SQLite sources. I think you want
> something like this:
>
>   %fallback str BAR.
>
>
That does almost exactly what I want.  Thank you thank you thank you!  :)
Do you know why that isn't documented?  (Or, am I looking at the wrong
documentation?)

-Chris
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser

[Dan Kennedy]

On Apr 7, 2010, at 3:23 AM, Chris verBurg wrote:

> Hehe, okay, here I go.  :)
>
>
> I'm trying to replace an existing flex/bison parser with an re2c/lemon
> parser, but I'm running into a methodological problem.  I have a
> hypothetical grammar like this:
>
>  file ::= FOO str .
>  file ::= BAR str .

See the use of the "%fallback" directive in the parse.y
file that is part of SQLite sources. I think you want
something like this:

   %fallback str BAR.


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser

[Chris verBurg]

Hehe, okay, here I go.  :)


I'm trying to replace an existing flex/bison parser with an re2c/lemon
parser, but I'm running into a methodological problem.  I have a
hypothetical grammar like this:

  file ::= FOO str .
  file ::= BAR str .

Where my keywords are FOO and BAR, and str is any ol' generic string.

I'm giving it the input "FOO BAR" in the hopes that it will match the first
rule and give str the value "BAR".  To keep Lemon from choking on a syntax
error when it sees BAR, I want to tell the lexer (re2c) to force the next
token after "FOO" to be a string.  Thus I broke the rule up like so:

  file ::= fooDefinition .

  fooDefinition ::= fooStart fooEnd .
  fooStart ::= FOO . { lexer->forceNextThingToBeAString(); }
  fooEnd ::= str .

However, the call to forceNextThingToBeAString() happens after re2c reads
"BAR", and by that time it's already identified it as the keyword BAR
instead of as a string that happens to be set to "BAR".  It seems like Lemon
always reads a lookahead token, even though it doesn't need to.

I considered using the lexer-only hack of having the lexer force a string
after "FOO", but that only works if every FOO in the grammar is followed by
str.  In my real-world problem, that's not the case.  The CFG of the lexer
is not adequate to know when to force strings; it requires control from the
parser.


Would you have any comments/ideas?

Thanks!
-Chris


On Tue, Apr 6, 2010 at 1:12 PM, Wilson, Ronald  wrote:

> > I'm using the Lemon parser and running into a methodological problem
> that
> > I
> > wanted to ask the user base about.  Is there a mailing list or forum
> > specifically for Lemon, or is this it?  :)
> >
> > Thanks,
> > -Chris
>
> There is no mailing list specifically for lemon.  Some of us are
> familiar to various degrees with lemon, so feel free to ask here.
> Though I must say, lemon questions tend languish for a while before
> someone replies.  Don't be afraid to bump your own message if it isn't
> answered.  I think it's safe to say that most sqlite users aren't
> initimately familiar with lemon.
>
> RW
>
> Ron Wilson, Engineering Project Lead
> (o) 434.455.6453, (m) 434.851.1612, www.harris.com
>
> HARRIS CORPORATION   |   RF Communications Division
> assuredcommunications(tm)
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser

[Wilson, Ronald]

> I'm using the Lemon parser and running into a methodological problem
that
> I
> wanted to ask the user base about.  Is there a mailing list or forum
> specifically for Lemon, or is this it?  :)
> 
> Thanks,
> -Chris

There is no mailing list specifically for lemon.  Some of us are
familiar to various degrees with lemon, so feel free to ask here.
Though I must say, lemon questions tend languish for a while before
someone replies.  Don't be afraid to bump your own message if it isn't
answered.  I think it's safe to say that most sqlite users aren't
initimately familiar with lemon.

RW

Ron Wilson, Engineering Project Lead
(o) 434.455.6453, (m) 434.851.1612, www.harris.com

HARRIS CORPORATION   |   RF Communications Division
assuredcommunications(tm)
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon parser

[Chris verBurg]

Hey guys,

I'm using the Lemon parser and running into a methodological problem that I
wanted to ask the user base about.  Is there a mailing list or forum
specifically for Lemon, or is this it?  :)

Thanks,
-Chris
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser : compile error when using "%token_destructor" directive

[Wilson, Ron P]

Very interesting.  I had no idea that lemon.c and lempar.c were being revised.  
I assumed they were static.

Would it be too much to increment the version that lemon -x prints?  Currently 
it prints out "Lemon version 1.0" which led me to believe it was not being 
actively developed.

RW

P.S.  I like what I see in the revision history.

Ron Wilson, Engineering Project Lead, 434.455.6453

HARRIS CORPORATION   |   RF Communications Division 
assuredcommunications(tm)


-Original Message-
From: Vincent Zweije [mailto:vzwe...@wcc-group.com] 
Sent: Monday, June 15, 2009 12:54 PM
To: Wilson, Ron P
Cc: General Discussion of SQLite Database
Subject: Re: [sqlite] Lemon parser : compile error when using 
"%token_destructor" directive

On Mon, Jun 15, 2009 at 11:42:26AM -0400, Wilson, Ron P wrote:

||  It has been a while since I used lemon (big fan though).  Did you resolve
||  this issue or do you still need help?

[It appears my previous response did not get through.]

Looks suspiciously like this problem, which was fixed in version 3.6.2:

http://www.sqlite.org/cvstrac/tktview?tn=3299

Ciao.

||  -Original Message-
||  From: sqlite-users-boun...@sqlite.org 
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of ferrety ferrety
||  Sent: Monday, June 08, 2009 7:15 PM
||  To: sqlite-users@sqlite.org
||  Subject: [sqlite] Lemon parser : compile error when using 
"%token_destructor" directive
||
||  Hi List,
||  This is the only place I found to ask for "lemon parser" error.
||
||  When trying to use the "%token_destructor" directive:
||
||  %include {
||  #include 
||  void token_dtor (struct Token * t)
||{
||  fprintf(stderr, "In token_destructor: t -> value=%s\n", t -> value);
||}
||  }
||  %token_destructor { token_dtor($$); }
||
||
||  I got the following error:
||  error: too many arguments to function 'yy_destructor'
||
||  I tried to understand how "lemon" generated the calls for that function and
||  found that:
||
||  1) The definition of "yy_destructor" is with only 2 arguments:
||  static void yy_destructor(YYCODETYPE yymajor, YYMINORTYPE *yypminor){
||switch( yymajor ){
||  /* Here is inserted the actions which take place when a
||  ...
||
||  2) "Lemon" generates different calls to "yy_destructor", sometimes with 2 or
||  3 parameters :
||  yy_destructor( yymajor, >minor); <- HERE with 2 arguments,
||  which is fine
||  yy_destructor(yypParser,4,[0].minor); <- HERE with 3 arguments, which
||  is an error
||  yy_destructor(yypParser,5,[-2].minor); <- same
||  yy_destructor(yypParser,6,[0].minor);  <- same
||  ...
||
||  Sometime, the "yy_destructor" si called with 2 arguments as defined and some
||  other time,
||  it's called with 3 arguments.
||
||  Is it a bug or am I missing something?
-- 
WCC - Smart Search & Match
NL  +31 30 7503222
vzwe...@wcc-group.com
www.wcc-group.com
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser : compile error when using "%token_destructor" directive

[Vincent Zweije]

On Mon, Jun 15, 2009 at 11:42:26AM -0400, Wilson, Ron P wrote:

||  It has been a while since I used lemon (big fan though).  Did you resolve
||  this issue or do you still need help?

[It appears my previous response did not get through.]

Looks suspiciously like this problem, which was fixed in version 3.6.2:

http://www.sqlite.org/cvstrac/tktview?tn=3299

Ciao.

||  -Original Message-
||  From: sqlite-users-boun...@sqlite.org 
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of ferrety ferrety
||  Sent: Monday, June 08, 2009 7:15 PM
||  To: sqlite-users@sqlite.org
||  Subject: [sqlite] Lemon parser : compile error when using 
"%token_destructor" directive
||
||  Hi List,
||  This is the only place I found to ask for "lemon parser" error.
||
||  When trying to use the "%token_destructor" directive:
||
||  %include {
||  #include 
||  void token_dtor (struct Token * t)
||{
||  fprintf(stderr, "In token_destructor: t -> value=%s\n", t -> value);
||}
||  }
||  %token_destructor { token_dtor($$); }
||
||
||  I got the following error:
||  error: too many arguments to function 'yy_destructor'
||
||  I tried to understand how "lemon" generated the calls for that function and
||  found that:
||
||  1) The definition of "yy_destructor" is with only 2 arguments:
||  static void yy_destructor(YYCODETYPE yymajor, YYMINORTYPE *yypminor){
||switch( yymajor ){
||  /* Here is inserted the actions which take place when a
||  ...
||
||  2) "Lemon" generates different calls to "yy_destructor", sometimes with 2 or
||  3 parameters :
||  yy_destructor( yymajor, >minor); <- HERE with 2 arguments,
||  which is fine
||  yy_destructor(yypParser,4,[0].minor); <- HERE with 3 arguments, which
||  is an error
||  yy_destructor(yypParser,5,[-2].minor); <- same
||  yy_destructor(yypParser,6,[0].minor);  <- same
||  ...
||
||  Sometime, the "yy_destructor" si called with 2 arguments as defined and some
||  other time,
||  it's called with 3 arguments.
||
||  Is it a bug or am I missing something?
-- 
WCC - Smart Search & Match
NL  +31 30 7503222
vzwe...@wcc-group.com
www.wcc-group.com
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser : compile error when using "%token_destructor" directive

[Wilson, Ron P]

It has been a while since I used lemon (big fan though).  Did you resolve this 
issue or do you still need help?

RW

Ron Wilson, Engineering Project Lead, 434.455.6453

HARRIS CORPORATION   |   RF Communications Division 
assuredcommunications(tm)


-Original Message-
From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] 
On Behalf Of ferrety ferrety
Sent: Monday, June 08, 2009 7:15 PM
To: sqlite-users@sqlite.org
Subject: [sqlite] Lemon parser : compile error when using "%token_destructor" 
directive

Hi List,
This is the only place I found to ask for "lemon parser" error.

When trying to use the "%token_destructor" directive:

%include {
#include 
void token_dtor (struct Token * t)
  {
fprintf(stderr, "In token_destructor: t -> value=%s\n", t -> value);
  }
}
%token_destructor { token_dtor($$); }


I got the following error:
error: too many arguments to function 'yy_destructor'

I tried to understand how "lemon" generated the calls for that function and
found that:

1) The definition of "yy_destructor" is with only 2 arguments:
static void yy_destructor(YYCODETYPE yymajor, YYMINORTYPE *yypminor){
  switch( yymajor ){
/* Here is inserted the actions which take place when a
...

2) "Lemon" generates different calls to "yy_destructor", sometimes with 2 or
3 parameters :
yy_destructor( yymajor, >minor); <- HERE with 2 arguments,
which is fine
yy_destructor(yypParser,4,[0].minor); <- HERE with 3 arguments, which
is an error
yy_destructor(yypParser,5,[-2].minor); <- same
yy_destructor(yypParser,6,[0].minor);  <- same
...

Sometime, the "yy_destructor" si called with 2 arguments as defined and some
other time,
it's called with 3 arguments.

Is it a bug or am I missing something?

Thanks in advance
Frederic
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon parser : compile error when using "%token_destructor" directive

[ferrety ferrety]

Hi List,
This is the only place I found to ask for "lemon parser" error.

When trying to use the "%token_destructor" directive:

%include {
#include 
void token_dtor (struct Token * t)
  {
fprintf(stderr, "In token_destructor: t -> value=%s\n", t -> value);
  }
}
%token_destructor { token_dtor($$); }


I got the following error:
error: too many arguments to function 'yy_destructor'

I tried to understand how "lemon" generated the calls for that function and
found that:

1) The definition of "yy_destructor" is with only 2 arguments:
static void yy_destructor(YYCODETYPE yymajor, YYMINORTYPE *yypminor){
  switch( yymajor ){
/* Here is inserted the actions which take place when a
...

2) "Lemon" generates different calls to "yy_destructor", sometimes with 2 or
3 parameters :
yy_destructor( yymajor, >minor); <- HERE with 2 arguments,
which is fine
yy_destructor(yypParser,4,[0].minor); <- HERE with 3 arguments, which
is an error
yy_destructor(yypParser,5,[-2].minor); <- same
yy_destructor(yypParser,6,[0].minor);  <- same
...

Sometime, the "yy_destructor" si called with 2 arguments as defined and some
other time,
it's called with 3 arguments.

Is it a bug or am I missing something?

Thanks in advance
Frederic
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon Parser: Modular

[Uma Krishnan]

I had posted earlier to find if Lemon parser can be made modular. As per 
previous post responses, who had advised that I look in lemon parser forum. But 
I could not find a separate forum.  

  Please refer to the following link to get a better idea of what I'm trying to 
do:
   
  http://www.patentstorm.us/patents/7089541-claims.html

Thanks
   
  Uma

Re: [sqlite] Lemon Parser - Modular & Extensible ?

[Christian Smith]

Uma Krishnan uttered:

Hey, There's no need to be offensive. I did not mean to be critical. Far 
from it, it does a great a job (far more than I'm capable of producing). 
What I was trying to find out was, if it is possible for a .y files to 
be broken such that it can be built on top on other .y files.



Sorry if I came across as offensive. That was not the intention. I was 
just a little confused about the question.


I think lemon can only handle single input files. But you can can include 
C source into your output C file using the %include directive. Check out 
the documentation at, if you haven't already done so:

http://www.hwaci.com/sw/lemon/lemon.html

Now, what may draw some critical analysis is top posting and hijacking an 
existing thread for a new topic... [snip]





 Not sure if this is the right group. But could not find a lemon parser 
user group.



This is the best group to ask. While not tied to SQLite, it appears to be 
maintained as part of SQLite (but I may be wrong.)


Christian

--
/"\
\ /ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
 X   - AGAINST MS ATTACHMENTS
/ \

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon Parser - Modular & Extensible ?

[Uma Krishnan]

Hey, There's no need to be offensive. I did not mean to be critical. Far from 
it, it does a great a job (far more than I'm capable of producing). What I was 
trying to find out was, if it is possible for a .y files to be broken such that 
it can be built on top on other .y files. 
   
  Not sure if this is the right group. But could not find a lemon parser user 
group.
   
  

Christian Smith <[EMAIL PROTECTED]> wrote:
  Uma Krishnan uttered:

> Hello:
>
> Is lemon parser modular and extensible?


Extensible to do what? It generates parsers, and is self contained. It 
does a single job, and does it well. What more could you ask for?


>
> Thanks
>
> Uma
>
> Asif Lodhi wrote:
> Hi Everybody,
>
> I have just joined this mailing list as Sqlite looks like a good
> software solution to my needs. What I need right now is RE-assurance
> of "crash-recovery" that is mentioned on your front page. So, I would
> be thankful if you experts would give me an "accurate" and fair
> picture of the crash-recovery aspects of SQLite - without any hype.
>
> --
> Best regards,
>
> Asif
>
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
>
>
>

--
/"\
\ / ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
X - AGAINST MS ATTACHMENTS
/ \

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon Parser - Modular & Extensible ?

[Christian Smith]

Uma Krishnan uttered:


Hello:

 Is lemon parser modular and extensible?



Extensible to do what? It generates parsers, and is self contained. It 
does a single job, and does it well. What more could you ask for?





 Thanks

 Uma

Asif Lodhi <[EMAIL PROTECTED]> wrote:
 Hi Everybody,

I have just joined this mailing list as Sqlite looks like a good
software solution to my needs. What I need right now is RE-assurance
of "crash-recovery" that is mentioned on your front page. So, I would
be thankful if you experts would give me an "accurate" and fair
picture of the crash-recovery aspects of SQLite - without any hype.

--
Best regards,

Asif

-
To unsubscribe, send email to [EMAIL PROTECTED]
-





--
/"\
\ /ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
 X   - AGAINST MS ATTACHMENTS
/ \

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

[sqlite] Lemon Parser - Modular & Extensible ?

[Uma Krishnan]

Hello:
   
  Is lemon parser modular and extensible? 
   
  Thanks
   
  Uma

Asif Lodhi <[EMAIL PROTECTED]> wrote:
  Hi Everybody,

I have just joined this mailing list as Sqlite looks like a good
software solution to my needs. What I need right now is RE-assurance
of "crash-recovery" that is mentioned on your front page. So, I would
be thankful if you experts would give me an "accurate" and fair
picture of the crash-recovery aspects of SQLite - without any hype.

--
Best regards,

Asif

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

RE: [sqlite] Lemon parser generator question

[Medi Montaseri]

Thanks again Ulrik...the problem was with the tokenizer and not the
parser...

Medi 

-Original Message-
From: Medi Montaseri [mailto:[EMAIL PROTECTED] 
Sent: Friday, May 04, 2007 5:14 PM
To: sqlite-users@sqlite.org
Subject: RE: [sqlite] Lemon parser generator question

Thanks Ulrik...I am now looking for the lexer...

Medi

-Original Message-
From: Ulrik Petersen [mailto:[EMAIL PROTECTED]
Sent: Friday, May 04, 2007 4:30 PM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] Lemon parser generator question

Medi,

Lemon is a parser, not a lexer.  The terminals are defined outside of
Lemon.  Perhaps you inherited the .y file but did not receive the
lexer/tokenizer?

SQLite, for example, has a hand-coded tokenizer.  Other projects (such
as my own) may use a lexer-generator such as flex.


In Lemon, the tokenizer drives the parser, not the other way around.  
For more information, you can see the sources of SQLite, which includes
a document on Lemon, or you can read this:

http://www.hwaci.com/sw/lemon/lemon.html


HTH


Ulrik Petersen




Medi Montaseri wrote:
> Hi,
>  
> Firstly, if this is not the proper forum for Lemon questions, please
let
> me know where I need to go...
>  
> Second, I am looking at a SQL grammer written for Lemon parser
generator
> and am failing to see where some terminals are defined. For example 
> COMMA, FROM, SELECT are terminals and releatively easy to deduce.
> However in the grammer I am studying, I see references to NAME and 
> STRING (all in uppercase, indicating a terminal production).
>  
> Here is my higher level problem...
>  
> I have inherited a SqlGrammer.y that does not parse quoted-table-name 
> and is failing in cases where the FROM clause have things like "my, 
> table". For example select * from "my, table". Currently it works with

> select * from "table one"
> Chasing the grammer rules, it see a rule that reads
> from_source(P) ::= NAME(C).
> {
> someClass:someAction()
> }
> Based on Lemons' doc, NAME must be a terminal rule (all upper case)
and
> since I don't define it, then lemon must.
>  
> Can someone shed some light on this
>  
> Thanks
> Medi
>
>   



-
To unsubscribe, send email to [EMAIL PROTECTED]

-



-
To unsubscribe, send email to [EMAIL PROTECTED]

-


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

RE: [sqlite] Lemon parser generator question

[Medi Montaseri]

Thanks Ulrik...I am now looking for the lexer...

Medi

-Original Message-
From: Ulrik Petersen [mailto:[EMAIL PROTECTED] 
Sent: Friday, May 04, 2007 4:30 PM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] Lemon parser generator question

Medi,

Lemon is a parser, not a lexer.  The terminals are defined outside of
Lemon.  Perhaps you inherited the .y file but did not receive the
lexer/tokenizer?

SQLite, for example, has a hand-coded tokenizer.  Other projects (such
as my own) may use a lexer-generator such as flex.


In Lemon, the tokenizer drives the parser, not the other way around.  
For more information, you can see the sources of SQLite, which includes 
a document on Lemon, or you can read this:

http://www.hwaci.com/sw/lemon/lemon.html


HTH


Ulrik Petersen




Medi Montaseri wrote:
> Hi,
>  
> Firstly, if this is not the proper forum for Lemon questions, please
let
> me know where I need to go...
>  
> Second, I am looking at a SQL grammer written for Lemon parser
generator
> and am failing to see where some terminals are defined. For example
> COMMA, FROM, SELECT are terminals and releatively easy to deduce.
> However in the grammer I am studying, I see references to NAME and
> STRING (all in uppercase, indicating a terminal production). 
>  
> Here is my higher level problem...
>  
> I have inherited a SqlGrammer.y that does not parse quoted-table-name
> and is failing in cases where the FROM clause have things like "my,
> table". For example select * from "my, table". Currently it works with
> select * from "table one" 
> Chasing the grammer rules, it see a rule that reads
> from_source(P) ::= NAME(C).
> {
> someClass:someAction()
> }
> Based on Lemons' doc, NAME must be a terminal rule (all upper case)
and
> since I don't define it, then lemon must.
>  
> Can someone shed some light on this
>  
> Thanks
> Medi
>
>   



-
To unsubscribe, send email to [EMAIL PROTECTED]

-


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon parser generator question

[Ulrik Petersen]

Medi,

Lemon is a parser, not a lexer.  The terminals are defined outside of 
Lemon.  Perhaps you inherited the .y file but did not receive the 
lexer/tokenizer?


SQLite, for example, has a hand-coded tokenizer.  Other projects (such 
as my own) may use a lexer-generator such as flex.



In Lemon, the tokenizer drives the parser, not the other way around.  
For more information, you can see the sources of SQLite, which includes 
a document on Lemon, or you can read this:


http://www.hwaci.com/sw/lemon/lemon.html


HTH


Ulrik Petersen




Medi Montaseri wrote:

Hi,
 
Firstly, if this is not the proper forum for Lemon questions, please let

me know where I need to go...
 
Second, I am looking at a SQL grammer written for Lemon parser generator

and am failing to see where some terminals are defined. For example
COMMA, FROM, SELECT are terminals and releatively easy to deduce.
However in the grammer I am studying, I see references to NAME and
STRING (all in uppercase, indicating a terminal production). 
 
Here is my higher level problem...
 
I have inherited a SqlGrammer.y that does not parse quoted-table-name

and is failing in cases where the FROM clause have things like "my,
table". For example select * from "my, table". Currently it works with
select * from "table one" 
Chasing the grammer rules, it see a rule that reads

from_source(P) ::= NAME(C).
{
someClass:someAction()
}
Based on Lemons' doc, NAME must be a terminal rule (all upper case) and
since I don't define it, then lemon must.
 
Can someone shed some light on this
 
Thanks

Medi

  



-
To unsubscribe, send email to [EMAIL PROTECTED]
-

[sqlite] Lemon parser generator question

[Medi Montaseri]

Hi,
 
Firstly, if this is not the proper forum for Lemon questions, please let
me know where I need to go...
 
Second, I am looking at a SQL grammer written for Lemon parser generator
and am failing to see where some terminals are defined. For example
COMMA, FROM, SELECT are terminals and releatively easy to deduce.
However in the grammer I am studying, I see references to NAME and
STRING (all in uppercase, indicating a terminal production). 
 
Here is my higher level problem...
 
I have inherited a SqlGrammer.y that does not parse quoted-table-name
and is failing in cases where the FROM clause have things like "my,
table". For example select * from "my, table". Currently it works with
select * from "table one" 
Chasing the grammer rules, it see a rule that reads
from_source(P) ::= NAME(C).
{
someClass:someAction()
}
Based on Lemons' doc, NAME must be a terminal rule (all upper case) and
since I don't define it, then lemon must.
 
Can someone shed some light on this
 
Thanks
Medi

Re: [sqlite] Lemon Parser Examples - I'll show you my examples

[sporkey]

On Thu, Aug 05, 2004 at 09:07:36AM -0400, Joseph Stewart wrote:
> Many thanks for your contribution!
> -joe
> 
> On Wed, 4 Aug 2004 22:02:48 -0400, [EMAIL PROTECTED]
> <[EMAIL PROTECTED]> wrote:
> > A user earlier posted a question about terminating
> > the grammer for a simple calculator done with lemon.
> > 
> > I ran into the same problem and had to use
> > the following, which worked fine:
> > 
> >   main ::= in.
> >   in ::= .
> >   in ::= in state NEWLINE.
> > 
> > I cannot reduce it any further. For instance,
> > the following does not work:
> > 
> >   in ::= .
> >   in ::= in stmt ENDLINE.
> > 
> > The version of lemon I'm using is from the
> > cvs 3.0.3.
> > 
> > The full calculator, there are 4 examples, can
> > be downloaded from the following link:
> > 
> > http://prdownloads.sourceforge.net/souptonuts/lemon_examples.tar.gz?download
> > 
> > Look at example4.y, if you want to go directly to
> > the terminating issue above.  Everything should be in
> > this download including lemon.c and lempar.c, so you
> > won't have to download anything else.
> > 
> > Take a look at the README. I tried to add documentation
> > on creating custom tokens, destructors, and a little bit
> > on what the grammer does.
> > 
> > To run everything, just run make
> > 
> >$ make
> > 
> > Then, each example can be run separately
> > 
> >$ ./ex1
> > 
> >$ ./ex2
> > 
> > 
> >$ ./ex4
> > 
> > Anyway, I hope this examples will help. Or
> > inspire you to take a took at lemon.
> > 
> > Regards,
> > 
> > Mike Chirico
> > 
> >

I've added an example with flex, and I'm trying write some engaging
documentation. If enough people find it useful, and if it can help
bring people to SQLite, I'll change the license from GPL to the public
domain. But, it will probably have to go through a few revisions first.

Regards,

Mike Chirico

Re: [sqlite] Lemon Parser Examples - I'll show you my examples

[Joseph Stewart]

Many thanks for your contribution!
-joe

On Wed, 4 Aug 2004 22:02:48 -0400, [EMAIL PROTECTED]
<[EMAIL PROTECTED]> wrote:
> A user earlier posted a question about terminating
> the grammer for a simple calculator done with lemon.
> 
> I ran into the same problem and had to use
> the following, which worked fine:
> 
>   main ::= in.
>   in ::= .
>   in ::= in state NEWLINE.
> 
> I cannot reduce it any further. For instance,
> the following does not work:
> 
>   in ::= .
>   in ::= in stmt ENDLINE.
> 
> The version of lemon I'm using is from the
> cvs 3.0.3.
> 
> The full calculator, there are 4 examples, can
> be downloaded from the following link:
> 
> http://prdownloads.sourceforge.net/souptonuts/lemon_examples.tar.gz?download
> 
> Look at example4.y, if you want to go directly to
> the terminating issue above.  Everything should be in
> this download including lemon.c and lempar.c, so you
> won't have to download anything else.
> 
> Take a look at the README. I tried to add documentation
> on creating custom tokens, destructors, and a little bit
> on what the grammer does.
> 
> To run everything, just run make
> 
>$ make
> 
> Then, each example can be run separately
> 
>$ ./ex1
> 
>$ ./ex2
> 
> 
>$ ./ex4
> 
> Anyway, I hope this examples will help. Or
> inspire you to take a took at lemon.
> 
> Regards,
> 
> Mike Chirico
> 
>

[sqlite] Lemon Parser Examples - I'll show you my examples

[sporkey]

A user earlier posted a question about terminating
the grammer for a simple calculator done with lemon. 

I ran into the same problem and had to use
the following, which worked fine:

  main ::= in.
  in ::= .
  in ::= in state NEWLINE.

I cannot reduce it any further. For instance,
the following does not work:

  in ::= .
  in ::= in stmt ENDLINE.

The version of lemon I'm using is from the
cvs 3.0.3.

The full calculator, there are 4 examples, can
be downloaded from the following link:

http://prdownloads.sourceforge.net/souptonuts/lemon_examples.tar.gz?download

Look at example4.y, if you want to go directly to
the terminating issue above.  Everything should be in 
this download including lemon.c and lempar.c, so you 
won't have to download anything else.

Take a look at the README. I tried to add documentation
on creating custom tokens, destructors, and a little bit
on what the grammer does.

To run everything, just run make

   $ make

Then, each example can be run separately

   $ ./ex1
   
   $ ./ex2

...
   $ ./ex4


Anyway, I hope this examples will help. Or
inspire you to take a took at lemon.

Regards,

Mike Chirico