[sqlite] lemon - namespace support

[Mark Olesen]

I've fairly recently been using lemon for building several parsers in 
C++ and found what I believe to be a *minimalist* means of avoiding 
symbol clashes without adding bloat, or affecting C code generation.

- New '-e' command line option to define the code extension. By default 
this is 'c', but with this option can define somethinglike -ecxx etc for 
using a C++ compiler.

- New (optional) '%namespace' directive. This can be used to embed the 
'Parse*' routines into a C++ namespace. If the '%namespace' is not 
defined, there is no change in behaviour.
The namespace can be anonymous or contain multiple nested namespaces. 
For example,
 %namespace {}
 %namespace {ns1::ns2::ns3}

This makes it easy to generate lemon parsers for C++ without any 
potential symbol clashes, and it imposes no C++ interface on the user.
It does not fundamentally change how lemon works.

I hope that the changes are useful enough for broader interest and/or 
sufficiently encapsulated that they could be incorporated into the sources.

Here are references to the patches:

https://develop.openfoam.com/Development/openfoam/blob/develop/wmake/src/lemon.c.patch

https://develop.openfoam.com/Development/openfoam/blob/develop/wmake/etc/lempar.c.patch


Cheers,
/mark
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] [lemon] lempar function visibility

[Luke Drummond]

Hi all

I have been using lemon successfully to generate a parser used in a
library, and I'd like some source-level way of limiting the visibility
of some lempar symbols.

I threw together a proof of concept for a macro that can be user defined
to allow setting `static` or e.g.
`__attribute___((visibility("hidden")))` for the `Parse.*` functions. By
default it is empty which maintains the existing behaviour.

My proof of concept patch is included below using the macro name
`YYAPIFUNC`.

I feel this is of general utility, and if you find it useful, please
feel free to adopt as you see fit under the usual sqlite terms.

All the Best

Luke

---
Index: tool/lempar.c
==
--- tool/lempar.c
+++ tool/lempar.c
@@ -231,10 +231,18 @@
 #include 
 static FILE *yyTraceFILE = 0;
 static char *yyTracePrompt = 0;
 #endif /* NDEBUG */
 
+/*
+** Any required function qualifiers for public parts of the parser should be
+** defined ** here, e.g. static, declspec, etc.
+*/
+#ifndef YYAPIFUNC
+#define YYAPIFUNC /* default is empty */
+#endif
+
 #ifndef NDEBUG
 /* 
 ** Turn parser tracing on by giving a stream to which to write the trace
 ** and a prompt to preface each trace message.  Tracing is turned off
 ** by making either argument NULL 
@@ -249,11 +257,11 @@
 ** 
 **
 ** Outputs:
 ** None.
 */
-void ParseTrace(FILE *TraceFILE, char *zTracePrompt){
+YYAPIFUNC void ParseTrace(FILE *TraceFILE, char *zTracePrompt){
   yyTraceFILE = TraceFILE;
   yyTracePrompt = zTracePrompt;
   if( yyTraceFILE==0 ) yyTracePrompt = 0;
   else if( yyTracePrompt==0 ) yyTraceFILE = 0;
 }
@@ -318,11 +326,11 @@
 # define YYMALLOCARGTYPE size_t
 #endif
 
 /* Initialize a new parser that has already been allocated.
 */
-void ParseInit(void *yypRawParser ParseCTX_PDECL){
+YYAPIFUNC void ParseInit(void *yypRawParser ParseCTX_PDECL){
   yyParser *yypParser = (yyParser*)yypRawParser;
   ParseCTX_STORE
 #ifdef YYTRACKMAXSTACKDEPTH
   yypParser->yyhwm = 0;
 #endif
@@ -357,11 +365,11 @@
 **
 ** Outputs:
 ** A pointer to a parser.  This pointer is used in subsequent calls
 ** to Parse and ParseFree.
 */
-void *ParseAlloc(void *(*mallocProc)(YYMALLOCARGTYPE) ParseCTX_PDECL){
+YYAPIFUNC void *ParseAlloc(void *(*mallocProc)(YYMALLOCARGTYPE) 
ParseCTX_PDECL){
   yyParser *yypParser;
   yypParser = (yyParser*)(*mallocProc)( (YYMALLOCARGTYPE)sizeof(yyParser) );
   if( yypParser ){
 ParseCTX_STORE
 ParseInit(yypParser ParseCTX_PARAM);
@@ -425,11 +433,11 @@
 }
 
 /*
 ** Clear all secondary memory allocations from the parser
 */
-void ParseFinalize(void *p){
+YYAPIFUNC void ParseFinalize(void *p){
   yyParser *pParser = (yyParser*)p;
   while( pParser->yytos>pParser->yystack ) yy_pop_parser_stack(pParser);
 #if YYSTACKDEPTH<=0
   if( pParser->yystack!=>yystk0 ) free(pParser->yystack);
 #endif
@@ -442,11 +450,11 @@
 **
 ** If the YYPARSEFREENEVERNULL macro exists (for example because it
 ** is defined in a %include section of the input grammar) then it is
 ** assumed that the input pointer is never NULL.
 */
-void ParseFree(
+YYAPIFUNC void ParseFree(
   void *p,/* The parser to be deleted */
   void (*freeProc)(void*) /* Function used to reclaim memory */
 ){
 #ifndef YYPARSEFREENEVERNULL
   if( p==0 ) return;
@@ -458,11 +466,11 @@
 
 /*
 ** Return the peak depth of the stack for a parser.
 */
 #ifdef YYTRACKMAXSTACKDEPTH
-int ParseStackPeak(void *p){
+YYAPIFUNC int ParseStackPeak(void *p){
   yyParser *pParser = (yyParser*)p;
   return pParser->yyhwm;
 }
 #endif
 
@@ -482,11 +490,11 @@
 **   (2)  is not a syntax error.
 **
 ** Return the number of missed state/lookahead combinations.
 */
 #if defined(YYCOVERAGE)
-int ParseCoverage(FILE *out){
+YYAPIFUNC int ParseCoverage(FILE *out){
   int stateno, iLookAhead, i;
   int nMissed = 0;
   for(stateno=0; stateno
 **
 ** Outputs:
 ** None.
 */
-void Parse(
+YYAPIFUNC void Parse(
   void *yyp,   /* The parser */
   int yymajor, /* The major token code number */
   ParseTOKENTYPE yyminor   /* The value for the token */
   ParseARG_PDECL   /* Optional %extra_argument parameter */
 ){
@@ -1063,14 +1071,14 @@
 
 /*
 ** Return the fallback token corresponding to canonical token iToken, or
 ** 0 if iToken has no fallback.
 */
-int ParseFallback(int iToken){
+YYAPIFUNC int ParseFallback(int iToken){
 #ifdef YYFALLBACK
   assert( iToken<(int)(sizeof(yyFallback)/sizeof(yyFallback[0])) );
   return yyFallback[iToken];
 #else
   (void)iToken;
   return 0;
 #endif
 }

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon source code?

[Richard Hipp]

On 12/20/19, David Given  wrote:
> Does anyone know if the original source code for the lemon parser still
> exists, and if so, where to find it?
>
> The lemon.c file shipped with sqlite, which the hwaci website points me at,
> does not appear to be it --- instead it's an amalgamation of a bunch of
> other files, including some generated by aagen from data files; but I have
> been able to track down neither the data files nor the aagen tool itself.
>
> Or does maintainence these days happen directly on the amalgamation?

I have long since lost the original source files (which I wrote on a
Sun4 in K C).  All maintenance these days happens on the amalgamated
"lemon.c" source file, which isn't that big really - less than 6K
lines total and less than 5K SLOC.

The complete edit history of the "lemon.c" source file, going back
almost 20 years, can be seen here:
https://www.sqlite.org/src/finfo/tool/lemon.c

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon source code?

[David Given]

Does anyone know if the original source code for the lemon parser still
exists, and if so, where to find it?

The lemon.c file shipped with sqlite, which the hwaci website points me at,
does not appear to be it --- instead it's an amalgamation of a bunch of
other files, including some generated by aagen from data files; but I have
been able to track down neither the data files nor the aagen tool itself.

Or does maintainence these days happen directly on the amalgamation?

Thanks!

-- 
┌─── http://www.cowlark.com ───
│ "I have always wished for my computer to be as easy to use as my
│ telephone; my wish has come true because I can no longer figure out
│ how to use my telephone." --- Bjarne Stroustrup
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon Parser vs bubble-generator.tcl

[Richard Hipp]

On 8/31/18, Warren Young  wrote:
> They’re separate.  Here’s the Tcl source for the bubble diagrams:
>
> https://www.sqlite.org/docsrc/file/?name=art/syntax/bubble-generator-data.tcl
>
> …and here’s the Lemon grammar for SQLite’s SQL parser:
>
> https://www.sqlite.org/src/file?name=src/parse.y
>

The parse.y file is designed to generate a fast parser automaton.  The
bubble-generator-data.tcl file is designed to generate output that is
easily understandable by humans.  These two files are therefore at
cross-purposes.

One little-noticed aspect of SQLite that the source code to the
LALR(1) parser generator (Lemon) is included in the source tree.  The
fact that SQLite uses its own parser generator, rather than relying
yacc/bison or similar, has proven very helpful over the years.  We've
been able to add features, fix bugs, and make optimizations to the
parser that would not have been possible had we been dependent on an
outside tool.
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon Parser vs bubble-generator.tcl

[Scott Robison]

On Fri, Aug 31, 2018 at 2:59 PM Warren Young  wrote:
>
> On Aug 31, 2018, at 1:55 PM, Scott Robison  wrote:
> >
> > Is one generated from the other, or are they maintained separately?
>
> They’re separate.  Here’s the Tcl source for the bubble diagrams:

As I suspected having looked at them in the source tree. Thanks for
the confirmation.

-- 
Scott Robison
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon Parser vs bubble-generator.tcl

[Warren Young]

On Aug 31, 2018, at 1:55 PM, Scott Robison  wrote:
> 
> Is one generated from the other, or are they maintained separately?

They’re separate.  Here’s the Tcl source for the bubble diagrams:


https://www.sqlite.org/docsrc/file/?name=art/syntax/bubble-generator-data.tcl

…and here’s the Lemon grammar for SQLite’s SQL parser:

https://www.sqlite.org/src/file?name=src/parse.y

Here’s a brief article on the script that may explain a few things:

https://wiki.tcl.tk/21708

> I...want to do something related to
> identifying SQL syntax at C++ compile time.

In that case, I’d study how ./src/parse.y gets turned into ./parse.[ch] in the 
SQLite source tree, then try to reuse as much of parse.y and its associated 
bits as you can.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon Parser vs bubble-generator.tcl

[Scott Robison]

SQLite language as implemented uses a Lemon based parser. The syntax
diagrams are created from the bubble-generator.tcl script. Is one
generated from the other, or are they maintained separately? Is one
(or another file that I don't know exists yet) the canonical
description of the "complete" SQLite syntax?

I ask because I'm working on a project idea for a senior project class
I'll be taking this fall and want to do something related to
identifying SQL syntax at C++ compile time. In doing a little research
/ planning, I started by creating my own syntax description from some
of the diagrams, then realized "I should just use portions of the
syntax diagram script directly", then started wondering about the
parser vs the diagram script.

-- 
Scott Robison
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] [Lemon] report file description

[Oleg]

  Hi, all.

Can anybody help me with interpretation of
lemon report file?

For example, i have the next report part:

State 10:
  (2) head ::= init_stmts *
  init_stmts ::= init_stmts * init_stmt
  init_stmt ::= * WORD init_stmt_prms SEMICOLON

  WORD shift5  
  WORD reduce   2   ** Parsing conflict (SR)**
 init_stmt shift-reduce 3  init_stmts ::= init_stmts 
init_stmt

Do i understand correctly that:

1. * marks a current position of a parser
2. 5 in "WORD shift5" is a number of a state to go
3. 2 in "WORD reduce   2" is a number of a rule to reduce
4. 3 in "init_stmt shift-reduce 3" is a number of a rule to reduce

Thanks!

-- 
Олег Неманов (Oleg Nemanov)
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon-generated parser gives an assertion failure

[Richard Hipp]

On 4/25/17, Kelvin Sherlock  wrote:
>
> I believe the issue is line 4164 which should be … = LEMON_TRUE.  Currently,
> when i=0, all rules will erroneously be optimized out.
>
>   4157/* Mark rules that are actually used for reduce actions after all
>   4158** optimizations have been applied
>   4159*/
>   4160for(rp=lemp->rule; rp; rp=rp->next) rp->doesReduce = LEMON_FALSE;
>   4161for(i=0; inxstate; i++){
>   4162  for(ap=lemp->sorted[i]->ap; ap; ap=ap->next){
>   4163if( ap->type==REDUCE || ap->type==SHIFTREDUCE ){
>   4164  ap->x.rp->doesReduce = i;
>   4165}
>   4166  }
>   4167}
>

Well done.  I fear that you have had to learn more about Lemon and
LALR(1) parsing than perhaps you really ever wanted to know.  Your
efforts are greatly appreciated.
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon-generated parser gives an assertion failure

[Kelvin Sherlock]

The test case can be simplified to:

bug.lemon:
---

%include {
#include 
#include 
#include 
#include "bug.h"
}
%code {
int main(void) {

void *pParser;

pParser = ParseAlloc(malloc);
if (!pParser) {
printf("out of memory\n");
exit(1);
}

ParseTrace(stderr, "Debug: ");

Parse(pParser, X, 0);
Parse(pParser, 0, 0);
ParseFree(pParser, free);
return 0;
}
}


main ::= decls.
decls ::= . 
decls ::= decls decl.

decl ::= X

—

which generates this code:

  default:
  /* (0) main ::= decls */ yytestcase(yyruleno==0);
  /* (1) decls ::= (OPTIMIZED OUT) */ assert(yyruleno!=1);
break;

I believe the issue is line 4164 which should be … = LEMON_TRUE.  Currently, 
when i=0, all rules will erroneously be optimized out.

  4157/* Mark rules that are actually used for reduce actions after all
  4158** optimizations have been applied
  4159*/
  4160for(rp=lemp->rule; rp; rp=rp->next) rp->doesReduce = LEMON_FALSE;
  4161for(i=0; inxstate; i++){
  4162  for(ap=lemp->sorted[i]->ap; ap; ap=ap->next){
  4163if( ap->type==REDUCE || ap->type==SHIFTREDUCE ){
  4164  ap->x.rp->doesReduce = i;
  4165}
  4166  }
  4167}

Kelvin Sherlock

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon: Simple recursive rule causes assertion failed: stateno <= YY_SHIFT_COUNT

[Kelvin Sherlock]

This lemon bug was reported about 6 months ago:


8<

%include {
#include 
#include 
#include 
#include "lemon-bug.h"
}

%code {
int main()
{
void *pParser;

pParser = ParseAlloc(malloc);
if (!pParser)
{
printf("out of memory\n");
exit(1);
}

ParseTrace(stderr, "Debug: ");

Parse(pParser, CMDNAME, 0);
Parse(pParser, INTEGER, 0);
Parse(pParser, TEXT, 0);
Parse(pParser, EOL, 0);

Parse(pParser, CMDNAME, 0);
Parse(pParser, INTEGER, 0);
Parse(pParser, TEXT, 0);
Parse(pParser, EOL, 0);

Parse(pParser, 0, 0);
ParseFree(pParser, free);
return 0;
}
}


database ::= entrylist.

entrylist ::= command.
entrylist ::= entrylist command.

command ::= CMDNAME cmdargs EOL. 
cmdargs ::= .
cmdargs ::= cmdargs cmdarg.

cmdarg ::= INTEGER.
cmdarg ::= TEXT.


8<


./lemon-bug 
Debug: Input 'CMDNAME'
Debug: Shift 'CMDNAME', go to state 3
Debug: Return. Stack=[CMDNAME]
Debug: Input 'INTEGER'
Assertion failed: (stateno <= YY_SHIFT_COUNT), function yy_find_shift_action, 
file lemon-bug.c, line 512.
Abort trap: 6

which generates this code:

#define YY_MAX_SHIFT 3
#define YY_SHIFT_COUNT(2)
#define YY_SHIFT_USE_DFLT (13)
static const unsigned char yy_shift_ofst[] = {
 /* 0 */ 7,1,6,
};
…

  assert( stateno <= YY_SHIFT_COUNT );

without the shift table compression -- lemon.c line 4235:
while( n>0 && lemp->sorted[n-1]->iTknOfst==NO_OFFSET ) n—;

it will generate this:

#define YY_SHIFT_COUNT(3)
static const unsigned char yy_shift_ofst[] = {
 /* 0 */ 7,1,6,   13,
};

and the assert doesn’t fail.

Most of the time (and in the case of SQLite parse.y) the shift table won’t 
compress and YY_MAX_SHIFT == YY_SHIFT_COUNT.
Given that, it probably shouldn’t try to compress the shift table.

Kelvin Sherlock



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon-generated parser gives an assertion failure

[Guy Harris]

If you use the version of lemon.c and lempar.c in the Fossil repository for 
SQLite as of 2017-04-16 20:54:23 UTC, take the following Lemon parser, compile 
it, and run it, it fails with

Assertion failed: (yyruleno!=116), function yy_reduce, file 
mate_grammar.c, line 2165.

(It's a grammar for a domain-specific language in Wireshark, with the actions 
trimmed down to avoid having to drag in the entire MATE mechanism, and other 
changes to make it a stand-along program.  As such, you get a bunch of warnings 
from Lemon about unused labels; ignore them.)

%include {

/* mate_grammar.lemon
* MATE's configuration language grammar
*
* Copyright 2005, Luis E. Garcia Ontanon 
*
* Wireshark - Network traffic analyzer
* By Gerald Combs 
* Copyright 1998 Gerald Combs
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/

#include 
#include 
#include 
#include 

#include "mate_grammar.h"

typedef enum _gop_tree_mode_t {
GOP_NULL_TREE,
GOP_BASIC_TREE,
GOP_FULL_TREE
} gop_tree_mode_t;

typedef enum _gop_pdu_tree {
GOP_NO_TREE,
GOP_PDU_TREE,
GOP_FRAME_TREE,
GOP_BASIC_PDU_TREE
} gop_pdu_tree_t;

typedef enum _accept_mode_t {
ACCEPT_MODE,
REJECT_MODE
} accept_mode_t;
typedef enum _avpl_match_mode {
AVPL_NO_MATCH,
AVPL_STRICT,
AVPL_LOOSE,
AVPL_EVERY
} avpl_match_mode;

typedef enum _avpl_replace_mode {
AVPL_NO_REPLACE,
AVPL_INSERT,
AVPL_REPLACE
} avpl_replace_mode;

#define DUMMY void*

#define MATE_PARSE(token_type, text) \
MateParser(pParser, (token_type), strdup(text));

int
main(void)
{
void MateParser(void*,int, char*);
void *MateParserAlloc(void *(*)(size_t));
void MateParserFree( void*, void(*)(void*) );
void MateParseTrace(FILE*,char*);
void* pParser;

pParser = MateParserAlloc(malloc);

/* Here come the tokens */
MATE_PARSE(TOKEN_PDU_KW, "Pdu");
MATE_PARSE(TOKEN_NAME, "sip_pdu");
MATE_PARSE(TOKEN_PROTO_KW, "Proto");
MATE_PARSE(TOKEN_NAME, "sip");
MATE_PARSE(TOKEN_TRANSPORT_KW, "Transport");
MATE_PARSE(TOKEN_NAME, "ip");
MATE_PARSE(TOKEN_OPEN_BRACE, "{");
MATE_PARSE(TOKEN_EXTRACT_KW, "Extract");
MATE_PARSE(TOKEN_NAME, "call_id");
MATE_PARSE(TOKEN_FROM_KW, "From");
MATE_PARSE(TOKEN_NAME, "sip.Call-Id");
MATE_PARSE(TOKEN_SEMICOLON, ";");
MATE_PARSE(TOKEN_EXTRACT_KW, "Extract");
MATE_PARSE(TOKEN_NAME, "method");
MATE_PARSE(TOKEN_FROM_KW, "From");
MATE_PARSE(TOKEN_NAME, "sip.Method");
MATE_PARSE(TOKEN_SEMICOLON, ";");
MATE_PARSE(TOKEN_CLOSE_BRACE, "}");
MATE_PARSE(TOKEN_GOP_KW, "Gop");
MATE_PARSE(TOKEN_NAME, "sip");
MATE_PARSE(TOKEN_ON_KW, "On");
MATE_PARSE(TOKEN_NAME, "sip_pdu");
MATE_PARSE(TOKEN_MATCH_KW, "Match");
MATE_PARSE(TOKEN_OPEN_PARENS, "(");
MATE_PARSE(TOKEN_NAME, "call_id");
MATE_PARSE(TOKEN_CLOSE_PARENS, ")");
MATE_PARSE(TOKEN_OPEN_BRACE, "{");
MATE_PARSE(TOKEN_START_KW, "Start");
MATE_PARSE(TOKEN_OPEN_PARENS, "(");
MATE_PARSE(TOKEN_NAME, "method");
MATE_PARSE(TOKEN_AVP_OPERATOR, "=");
MATE_PARSE(TOKEN_QUOTED, "SUBSCRIBE");
MATE_PARSE(TOKEN_CLOSE_PARENS, ")");
MATE_PARSE(TOKEN_SEMICOLON, ";");
MATE_PARSE(TOKEN_STOP_KW, "Stop");
MATE_PARSE(TOKEN_OPEN_PARENS, "(");
MATE_PARSE(TOKEN_NAME, "method");
MATE_PARSE(TOKEN_AVP_OPERATOR, "=");
MATE_PARSE(TOKEN_QUOTED, "NOTIFY");
MATE_PARSE(TOKEN_CLOSE_PARENS, ")");
MATE_PARSE(TOKEN_SEMICOLON, ";");
MATE_PARSE(TOKEN_CLOSE_BRACE, "}");
MATE_PARSE(TOKEN_DONE_KW, "Done");
MATE_PARSE(TOKEN_SEMICOLON, ";");

/* Inform parser that end of input has reached. */
MateParser(pParser, 0, NULL);

MateParserFree(pParser, free);
}

}

%name MateParser

%token_prefix TOKEN_

%token_type { char* }
%token_destructor {
if ($$) free($$);
}

%syntax_error {
fprintf(stderr, "Syntax Error before %s",yyminor);
}

%parse_failure {
fprintf(stderr, "Parse Error");
}

%type   transform_decl  { char* }
%type   

Re: [sqlite] Lemon documention (small error)

[Richard Hipp]

On 10/8/16, Conor O'Rourke  wrote:
> I believe the Lemon documentation at:
> http://www.hwaci.com/sw/lemon/lemon.html is out of date with respect to the
> default token type. If I create a parser with no token_type override, the
> default type is:
>
> #define ParseTOKENTYPE void*
>
> not int as stated.

Fix checked in.  Thanks.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] lemon parser bug, %destructors directive not appended into source code

[Richard Hipp]

On 8/16/16, xiaobing  wrote:
> In lemon.c:ReportTable() , when " Combine duplicate destructors into a
> single case ", sp2->destructor is set to 0, but later it is used in
> tranlate_code to generate destructor.
> so if you have grammar like this:
> %destructor expr_a { expr_free($$); }
> %destructor expr_b { expr_free($$); }  // this one will be lost, unless
> you insert some comment so strcmp return nonzero
> expr ::= expr_a expr_b.
> the generated code will include destructor of expr_a, however, destructor
> of expr_b will be lost.

Excellent analysis, Xiaobing.  Thanks.

I have checked in a slightly different fix.  Please confirm that what
I have checked in actually fixes your problem.
https://www.sqlite.org/src/fdiff?dc=20=e3aa9ba3469804d7=e4fb7d73ac88


>
> Here is my patch to lemon.c:
>
> --- old/lemon.c 2016-05-18 18:07:00.0 +0800
> +++ new/lemon.c   2016-08-16 20:57:36.799178091 +0800
> @@ -263,6 +263,7 @@
>int useCnt;  /* Number of times used */
>char *destructor;/* Code which executes whenever this symbol is
> ** popped from the stack during error
> processing */
> +  int destructor_emitted;  /* Is the destructor code emitted */
>int destLineno;  /* Line number for start of destructor */
>char *datatype;  /* The data type of information held by this
> ** object. Only used if type==NONTERMINAL */
> @@ -4351,8 +4352,9 @@
>}
>for(i=0; insymbol; i++){
>  struct symbol *sp = lemp->symbols[i];
> -if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ) continue;
> +if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ||
> sp->destructor_emitted) continue;
>  fprintf(out,"case %d: /* %s */\n", sp->index, sp->name); lineno++;
> +sp->destructor_emitted = 1;
>
>  /* Combine duplicate destructors into a single case */
>  for(j=i+1; jnsymbol; j++){
> @@ -4362,7 +4364,7 @@
>&& strcmp(sp->destructor,sp2->destructor)==0 ){
>   fprintf(out,"case %d: /* %s */\n",
>   sp2->index, sp2->name); lineno++;
> - sp2->destructor = 0;
> + sp2->destructor_emitted = 1;
>}
>  }
>
> @@ -4876,6 +4878,7 @@
>  sp->firstset = 0;
>  sp->lambda = LEMON_FALSE;
>  sp->destructor = 0;
> +sp->destructor_emitted = 0;
>  sp->destLineno = 0;
>  sp->datatype = 0;
>  sp->useCnt = 0;
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] lemon parser bug, %destructors directive not appended into source code

[xiaobing]

In lemon.c:ReportTable() , when " Combine duplicate destructors into a
single case ", sp2->destructor is set to 0, but later it is used in
tranlate_code to generate destructor.
so if you have grammar like this:
%destructor expr_a { expr_free($$); }
%destructor expr_b { expr_free($$); }  // this one will be lost, unless
you insert some comment so strcmp return nonzero
expr ::= expr_a expr_b.
the generated code will include destructor of expr_a, however, destructor
of expr_b will be lost.

Here is my patch to lemon.c:

--- old/lemon.c 2016-05-18 18:07:00.0 +0800
+++ new/lemon.c   2016-08-16 20:57:36.799178091 +0800
@@ -263,6 +263,7 @@
   int useCnt;  /* Number of times used */
   char *destructor;/* Code which executes whenever this symbol is
** popped from the stack during error
processing */
+  int destructor_emitted;  /* Is the destructor code emitted */
   int destLineno;  /* Line number for start of destructor */
   char *datatype;  /* The data type of information held by this
** object. Only used if type==NONTERMINAL */
@@ -4351,8 +4352,9 @@
   }
   for(i=0; insymbol; i++){
 struct symbol *sp = lemp->symbols[i];
-if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ) continue;
+if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ||
sp->destructor_emitted) continue;
 fprintf(out,"case %d: /* %s */\n", sp->index, sp->name); lineno++;
+sp->destructor_emitted = 1;

 /* Combine duplicate destructors into a single case */
 for(j=i+1; jnsymbol; j++){
@@ -4362,7 +4364,7 @@
   && strcmp(sp->destructor,sp2->destructor)==0 ){
  fprintf(out,"case %d: /* %s */\n",
  sp2->index, sp2->name); lineno++;
- sp2->destructor = 0;
+ sp2->destructor_emitted = 1;
   }
 }

@@ -4876,6 +4878,7 @@
 sp->firstset = 0;
 sp->lambda = LEMON_FALSE;
 sp->destructor = 0;
+sp->destructor_emitted = 0;
 sp->destLineno = 0;
 sp->datatype = 0;
 sp->useCnt = 0;
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] lemon parser bug, %destructors directive not appended into source code

[xiaobing]

In lemon.c:ReportTable() , when " Combine duplicate destructors into a
single case ", sp2->destructor is set to 0, but later it is used in
tranlate_code to generate destructor.
so if you have grammar like this:
%destructor expr_a { expr_free($$); }
%destructor expr_b { expr_free($$); }  // this one will be lost, unless
you insert some comment so strcmp return nonzero
expr ::= expr_a expr_b.
the generated code will include destructor of expr_a, however, destructor
of expr_b will be lost.

Here is my patch to lemon.c:

--- old/lemon.c 2016-05-18 18:07:00.0 +0800
+++ new/lemon.c   2016-08-16 20:57:36.799178091 +0800
@@ -263,6 +263,7 @@
   int useCnt;  /* Number of times used */
   char *destructor;/* Code which executes whenever this symbol is
** popped from the stack during error
processing */
+  int destructor_emitted;  /* Is the destructor code emitted */
   int destLineno;  /* Line number for start of destructor */
   char *datatype;  /* The data type of information held by this
** object. Only used if type==NONTERMINAL */
@@ -4351,8 +4352,9 @@
   }
   for(i=0; insymbol; i++){
 struct symbol *sp = lemp->symbols[i];
-if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ) continue;
+if( sp==0 || sp->type==TERMINAL || sp->destructor==0 ||
sp->destructor_emitted) continue;
 fprintf(out,"case %d: /* %s */\n", sp->index, sp->name); lineno++;
+sp->destructor_emitted = 1;

 /* Combine duplicate destructors into a single case */
 for(j=i+1; jnsymbol; j++){
@@ -4362,7 +4364,7 @@
   && strcmp(sp->destructor,sp2->destructor)==0 ){
  fprintf(out,"case %d: /* %s */\n",
  sp2->index, sp2->name); lineno++;
- sp2->destructor = 0;
+ sp2->destructor_emitted = 1;
   }
 }

@@ -4876,6 +4878,7 @@
 sp->firstset = 0;
 sp->lambda = LEMON_FALSE;
 sp->destructor = 0;
+sp->destructor_emitted = 0;
 sp->destLineno = 0;
 sp->datatype = 0;
 sp->useCnt = 0;
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Nick Wellnhofer]

On 12/07/2016 22:01, Richard Hipp wrote:

OK.  Another fix.  Please try the latest trunk version.


This version works for me. Thanks.

Nick

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Richard Hipp]

On 7/9/16, Nick Wellnhofer  wrote:
>
> This still doesn't work for me.

OK.  Another fix.  Please try the latest trunk version.

Note to passive readers of this thread: none of this has any impact on
SQLite.  SQLite does not use the feature of the Lemon LALR(1) parser
generator that Nick is fixing.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Nick Wellnhofer]

On 08/07/2016 21:54, Richard Hipp wrote:

Please try again with the latest version of Lemon.  Thanks.


This still doesn't work for me. I created a GitHub repo to demonstrate the 
problem:


https://github.com/nwellnhof/lemon-bug

Nick

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Nick Wellnhofer]

On 08/07/2016 21:54, Richard Hipp wrote:

Please try again with the latest version of Lemon.  Thanks.

On 7/6/16, Nick Wellnhofer  wrote:

On 05/07/2016 18:12, Richard Hipp wrote:

Please try https://www.sqlite.org/src/info/2683b375ad129117 and verify
that the changes on trunk are working.  Thanks.


Still doesn't work for me. The structure of the #ifdefs in `Parse` is:

 #ifdef YYERRORSYMBOL
   ...
 #elif defined(YYNOERRORRECOVERY)
   ...
 #else  /* YYERRORSYMBOL is not defined */
   ...
 #endif

Your first check-in modifies the first branch, your second check-in the
second
branch, resulting in:

 #ifdef YYERRORSYMBOL
   ...
   #ifndef YYNOERRORRECOVERY
 yypParser->yyerrcnt = -1;
   #endif
   ...
 #elif defined(YYNOERRORRECOVERY)
   ...
   #ifndef YYNOERRORRECOVERY
 yypParser->yyerrcnt = -1;
   #endif
   ...
 #else  /* YYERRORSYMBOL is not defined */
   ...
 #endif

The change to the second branch has no effect because YYNOERRORRECOVERY is
always defined. My patch modifies the third branch ("YYERRORSYMBOL is not
defined"). This fixes code that defines neither YYERRORSYMBOL nor
YYNOERRORRECOVERY. I think the code should look like this:

 #ifdef YYERRORSYMBOL
   ...
   #ifndef YYNOERRORRECOVERY
 yypParser->yyerrcnt = -1;
   #endif
   ...
 #elif defined(YYNOERRORRECOVERY)
   ...
 #else  /* YYERRORSYMBOL is not defined */
   ...
   yypParser->yyerrcnt = -1;
   ...
 #endif

(Another check for YYNOERRORRECOVERY isn't really needed in the third
branch.
It will always be undef.)

Nick








--
aevum GmbH
Nadistr. 12
80809 München
Germany

Tel: +49 89 35747589
http://aevum.de/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Richard Hipp]

Please try again with the latest version of Lemon.  Thanks.

On 7/6/16, Nick Wellnhofer  wrote:
> On 05/07/2016 18:12, Richard Hipp wrote:
>> Please try https://www.sqlite.org/src/info/2683b375ad129117 and verify
>> that the changes on trunk are working.  Thanks.
>
> Still doesn't work for me. The structure of the #ifdefs in `Parse` is:
>
>  #ifdef YYERRORSYMBOL
>...
>  #elif defined(YYNOERRORRECOVERY)
>...
>  #else  /* YYERRORSYMBOL is not defined */
>...
>  #endif
>
> Your first check-in modifies the first branch, your second check-in the
> second
> branch, resulting in:
>
>  #ifdef YYERRORSYMBOL
>...
>#ifndef YYNOERRORRECOVERY
>  yypParser->yyerrcnt = -1;
>#endif
>...
>  #elif defined(YYNOERRORRECOVERY)
>...
>#ifndef YYNOERRORRECOVERY
>  yypParser->yyerrcnt = -1;
>#endif
>...
>  #else  /* YYERRORSYMBOL is not defined */
>...
>  #endif
>
> The change to the second branch has no effect because YYNOERRORRECOVERY is
> always defined. My patch modifies the third branch ("YYERRORSYMBOL is not
> defined"). This fixes code that defines neither YYERRORSYMBOL nor
> YYNOERRORRECOVERY. I think the code should look like this:
>
>  #ifdef YYERRORSYMBOL
>...
>#ifndef YYNOERRORRECOVERY
>  yypParser->yyerrcnt = -1;
>#endif
>...
>  #elif defined(YYNOERRORRECOVERY)
>...
>  #else  /* YYERRORSYMBOL is not defined */
>...
>yypParser->yyerrcnt = -1;
>...
>  #endif
>
> (Another check for YYNOERRORRECOVERY isn't really needed in the third
> branch.
> It will always be undef.)
>
> Nick
>
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Nick Wellnhofer]

On 05/07/2016 18:12, Richard Hipp wrote:

Please try https://www.sqlite.org/src/info/2683b375ad129117 and verify
that the changes on trunk are working.  Thanks.


Still doesn't work for me. The structure of the #ifdefs in `Parse` is:

#ifdef YYERRORSYMBOL
  ...
#elif defined(YYNOERRORRECOVERY)
  ...
#else  /* YYERRORSYMBOL is not defined */
  ...
#endif

Your first check-in modifies the first branch, your second check-in the second 
branch, resulting in:


#ifdef YYERRORSYMBOL
  ...
  #ifndef YYNOERRORRECOVERY
yypParser->yyerrcnt = -1;
  #endif
  ...
#elif defined(YYNOERRORRECOVERY)
  ...
  #ifndef YYNOERRORRECOVERY
yypParser->yyerrcnt = -1;
  #endif
  ...
#else  /* YYERRORSYMBOL is not defined */
  ...
#endif

The change to the second branch has no effect because YYNOERRORRECOVERY is 
always defined. My patch modifies the third branch ("YYERRORSYMBOL is not 
defined"). This fixes code that defines neither YYERRORSYMBOL nor 
YYNOERRORRECOVERY. I think the code should look like this:


#ifdef YYERRORSYMBOL
  ...
  #ifndef YYNOERRORRECOVERY
yypParser->yyerrcnt = -1;
  #endif
  ...
#elif defined(YYNOERRORRECOVERY)
  ...
#else  /* YYERRORSYMBOL is not defined */
  ...
  yypParser->yyerrcnt = -1;
  ...
#endif

(Another check for YYNOERRORRECOVERY isn't really needed in the third branch. 
It will always be undef.)


Nick

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Richard Hipp]

Please try https://www.sqlite.org/src/info/2683b375ad129117 and verify
that the changes on trunk are working.  Thanks.

On 7/5/16, Nick Wellnhofer  wrote:
> On 05/07/2016 17:15, Richard Hipp wrote:
>> On 7/5/16, Nick Wellnhofer  wrote:
>>> No, this doesn't fix my problem. The check-in only changes the "#ifdef
>>> YYERRORSYMBOL" branch which I don't define. But if I add the change to
>>> the
>>> "YYERRORSYMBOL is not defined" branch as well, everything works as
>>> expected.
>>
>> Can you show me your patch, please?
>
> Here it is.
>
> Nick
>
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Nick Wellnhofer]

On 05/07/2016 17:15, Richard Hipp wrote:

On 7/5/16, Nick Wellnhofer  wrote:

No, this doesn't fix my problem. The check-in only changes the "#ifdef
YYERRORSYMBOL" branch which I don't define. But if I add the change to the
"YYERRORSYMBOL is not defined" branch as well, everything works as
expected.


Can you show me your patch, please?


Here it is.

Nick

diff --git a/lemon/lempar.c b/lemon/lempar.c
index 112d0bd..25a9525 100644
--- a/lemon/lempar.c
+++ b/lemon/lempar.c
@@ -919,6 +919,7 @@ void Parse(
   if( yyendofinput ){
 yy_parse_failed(yypParser);
   }
+  yypParser->yyerrcnt = -1;
   yymajor = YYNOCODE;
 #endif
 }
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Richard Hipp]

On 7/5/16, Nick Wellnhofer  wrote:
> No, this doesn't fix my problem. The check-in only changes the "#ifdef
> YYERRORSYMBOL" branch which I don't define. But if I add the change to the
> "YYERRORSYMBOL is not defined" branch as well, everything works as
> expected.

Can you show me your patch, please?

>
> Thanks for the quick response!
>
> Nick
>
> On 05/07/2016 14:48, Richard Hipp wrote:
>> Please try the latest check-in
>> (https://www.sqlite.org/src/info/91889fa30e84760e) and let me know
>> whether or not it clears your problem.
>>
>> On 7/5/16, Nick Wellnhofer  wrote:
>>> Hello,
>>>
>>> I hope this is right place to report Lemon issues. I ran into a problem
>>> after
>>> upgrading to the latest version of the Lemon source code from `trunk`.
>>>
>>> The following commit removed the initialization of `yyerrcnt` from the
>>> `Parse`
>>> function:
>>>
>>>
>>> http://www.sqlite.org/src/fdiff?sbs=1=872383ebf36c13fd=8569dd3e4c22831e
>>>
>>> Then this commit added the initialization to `ParseAlloc`:
>>>
>>>
>>> http://www.sqlite.org/src/fdiff?sbs=1=f06b7e98a6b7efb4=66a16b5e00fefff2
>>>
>>> I'd like to use a parser instance for multiple inputs without
>>> reallocating.
>>> If
>>> I parse input that causes a syntax error, the error is reported for the
>>> first
>>> time. But subsequent errors aren't reported anymore. I presume this is
>>> because
>>> `yyerrcnt` should be reset to -1 somewhere in the code.
>>>
>>> Nick
>>> ___
>>> sqlite-users mailing list
>>> sqlite-users@mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>>
>>
>>
>
>
> --
> aevum GmbH
> Nadistr. 12
> 80809 München
> Germany
>
> Tel: +49 89 35747589
> http://aevum.de/
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Nick Wellnhofer]

No, this doesn't fix my problem. The check-in only changes the "#ifdef 
YYERRORSYMBOL" branch which I don't define. But if I add the change to the 
"YYERRORSYMBOL is not defined" branch as well, everything works as expected.


Thanks for the quick response!

Nick

On 05/07/2016 14:48, Richard Hipp wrote:

Please try the latest check-in
(https://www.sqlite.org/src/info/91889fa30e84760e) and let me know
whether or not it clears your problem.

On 7/5/16, Nick Wellnhofer  wrote:

Hello,

I hope this is right place to report Lemon issues. I ran into a problem
after
upgrading to the latest version of the Lemon source code from `trunk`.

The following commit removed the initialization of `yyerrcnt` from the
`Parse`
function:


http://www.sqlite.org/src/fdiff?sbs=1=872383ebf36c13fd=8569dd3e4c22831e

Then this commit added the initialization to `ParseAlloc`:


http://www.sqlite.org/src/fdiff?sbs=1=f06b7e98a6b7efb4=66a16b5e00fefff2

I'd like to use a parser instance for multiple inputs without reallocating.
If
I parse input that causes a syntax error, the error is reported for the
first
time. But subsequent errors aren't reported anymore. I presume this is
because
`yyerrcnt` should be reset to -1 somewhere in the code.

Nick
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users







--
aevum GmbH
Nadistr. 12
80809 München
Germany

Tel: +49 89 35747589
http://aevum.de/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon doesn't reset `yyerrcnt` after error

[Richard Hipp]

Please try the latest check-in
(https://www.sqlite.org/src/info/91889fa30e84760e) and let me know
whether or not it clears your problem.

On 7/5/16, Nick Wellnhofer  wrote:
> Hello,
>
> I hope this is right place to report Lemon issues. I ran into a problem
> after
> upgrading to the latest version of the Lemon source code from `trunk`.
>
> The following commit removed the initialization of `yyerrcnt` from the
> `Parse`
> function:
>
>
> http://www.sqlite.org/src/fdiff?sbs=1=872383ebf36c13fd=8569dd3e4c22831e
>
> Then this commit added the initialization to `ParseAlloc`:
>
>
> http://www.sqlite.org/src/fdiff?sbs=1=f06b7e98a6b7efb4=66a16b5e00fefff2
>
> I'd like to use a parser instance for multiple inputs without reallocating.
> If
> I parse input that causes a syntax error, the error is reported for the
> first
> time. But subsequent errors aren't reported anymore. I presume this is
> because
> `yyerrcnt` should be reset to -1 somewhere in the code.
>
> Nick
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon doesn't reset `yyerrcnt` after error

[Nick Wellnhofer]

Hello,

I hope this is right place to report Lemon issues. I ran into a problem after 
upgrading to the latest version of the Lemon source code from `trunk`.


The following commit removed the initialization of `yyerrcnt` from the `Parse` 
function:



http://www.sqlite.org/src/fdiff?sbs=1=872383ebf36c13fd=8569dd3e4c22831e

Then this commit added the initialization to `ParseAlloc`:


http://www.sqlite.org/src/fdiff?sbs=1=f06b7e98a6b7efb4=66a16b5e00fefff2

I'd like to use a parser instance for multiple inputs without reallocating. If 
I parse input that causes a syntax error, the error is reported for the first 
time. But subsequent errors aren't reported anymore. I presume this is because 
`yyerrcnt` should be reset to -1 somewhere in the code.


Nick
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon bug in shiftreduce action for error symbol

[Vincent Zweije]

On Fri, Jun 17, 2016 at 11:50:48AM -0400, Richard Hipp wrote:

> On 6/6/16, Vincent Zweije  wrote:
>
> >
> > When the shiftreduce action is used in an error context,
> > such as:
> >
> >X -> alpha error.
> >
> > the adjustment is not made. This causes error handling to fail.
> >
>
> Is this problem fixed by
> https://www.sqlite.org/src/fdiff?sbs=1=66a16b5e00fefff2=8c4e9d8517e50da3
> from earlier this month?  Or have you identified a new problem?

It is essentially the same change, and made on the day I sent
the bug report for the first time. I conclude it was made
because of my bug report, even though I haven't seen the bug
report back on the mailing list, or seen any response.

Some sort of feedback would have been nice.

I'll let you know what happens when sqlite-3.14 is released.

Vincent.
-- 
WCC - Smart Search & Match
NL  +31 30 7503222
vzwe...@wcc-group.com
www.wcc-group.com
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon bug in shiftreduce action for error symbol

[Richard Hipp]

On 6/6/16, Vincent Zweije  wrote:

>
> When the shiftreduce action is used in an error context,
> such as:
>
>X -> alpha error.
>
> the adjustment is not made. This causes error handling to fail.
>

Is this problem fixed by
https://www.sqlite.org/src/fdiff?sbs=1=66a16b5e00fefff2=8c4e9d8517e50da3
from earlier this month?  Or have you identified a new problem?
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon bug in shiftreduce action for error symbol

[Vincent Zweije]

[resent without pgp signature since i've not seen it
come back on the list or had any reaction so far]

The shiftreduce action is a shift action followed by a
reduce action. It is executed in two parts: a shift part,
shifting to an anonymous state. In this state, the state
number is actually an indication of the reduction to perform,
independently of the lookahead symbol.

When performing the shift part, the argument of the action
is adjusted before being stored in the state of the anonymous
state.

Problem:

When the shiftreduce action is used in an error context,
such as:

   X -> alpha error.

the adjustment is not made. This causes error handling to fail.

Solution:

Moving the adjustment into the yy_shift function fixes the
problem.

I think check-in 2c17a1358353a0845b039283be79353f033e2491
is missing this adjustment, and so it would affect releases
3.9.0 and up (3.13.*).

The following patch fixes the problem for me, but it
is not against the latest trunk, and I think it will
not apply to it because the line to be removed has been
reformatted. Nevertheless I suppose it will be useful.

Index: tool/lempar.c
===
--- tool/lempar.c
+++ tool/lempar.c
@@ -563,6 +563,7 @@
   int yyMajor,  /* The major token to shift in */
   ParseTOKENTYPE yyMinor/* The minor token to shift in */
 ){
+  if( yyNewState > YY_MAX_SHIFT ) yyNewState += YY_MIN_REDUCE - 
YY_MIN_SHIFTREDUCE;
   yyStackEntry *yytos;
   yypParser->yyidx++;
 #ifdef YYTRACKMAXSTACKDEPTH
@@ -813,7 +814,6 @@
   do{
 yyact = yy_find_shift_action(yypParser,(YYCODETYPE)yymajor);
 if( yyact <= YY_MAX_SHIFTREDUCE ){
-  if( yyact > YY_MAX_SHIFT ) yyact += YY_MIN_REDUCE - 
YY_MIN_SHIFTREDUCE;
   yy_shift(yypParser,yyact,yymajor,yyminor);
 #ifndef YYNOERRORRECOVERY
   yypParser->yyerrcnt--;

Thanks.
Vincent.
-- 
WCC - Smart Search & Match
NL  +31 30 7503222
vzwe...@wcc-group.com
www.wcc-group.com
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon reduce action merge bug

[Richard Hipp]

On 4/30/16, Kelvin Sherlock  wrote:
>
> Here are a couple more edge cases I noticed:
>

Thanks for the deep-diving on Lemon!  All changes are now checked in.

For passive listeners - none of the bugs that Kelvin has found in
Lemon have any impact on SQLite.  They only come up when using Lemon
on other non-SQLite grammars.  Furthermore, all of the bugs arise (I
think) from recent enhancements I made to Lemon that help the SQLite
parser run faster.
-- 
D. Richard Hipp
drh at sqlite.org

[sqlite] Lemon reduce action merge bug

[Kelvin Sherlock]

> On Apr 29, 2016, at 7:29 AM, Richard Hipp  wrote:
> 
> On 4/28/16, Kelvin Sherlock  wrote:
>> I believe the lemon reduce action optimizer needs to compare the codePrefix
>> and codeSuffix.
> 
> Thanks for the bug report.  A fix has now been checked in.
> 
> -- 
> D. Richard Hipp
> drh at sqlite.org

Here are a couple more edge cases I noticed:

---

bug_report ::= rule.

%destructor malloc{free($$)}

rule ::= .
rule ::= malloc . /* 1 */
rule ::= malloc malloc . /* 2 */

malloc(X) ::= ID . { X = malloc(100); }

---

1. the rhs[0] destructor will not be called since the lhsalias null check 
happens first and the codePrefix is never generated.

The empty lhsalias check can be moved down to compensate:

@@ -3552,30 +3552,30 @@ PRIVATE int translate_code(struct lemon *lemp, struct 
rule *rp){
   if( rp->code==0 ){
 static char newlinestr[2] = { '\n', '\0' };
 rp->code = newlinestr;
 rp->line = rp->ruleline;
   }


-  if( rp->lhsalias==0 ){
-/* There is no LHS value symbol. */
-lhsdirect = 1;
-  }else if( rp->nrhs==0 ){
+  if( rp->nrhs==0 ){
 /* If there are no RHS symbols, then writing directly to the LHS is ok */
 lhsdirect = 1;
   }else if( rp->rhsalias[0]==0 ){
 /* The left-most RHS symbol has not value.  LHS direct is ok.  But
 ** we have to call the distructor on the RHS symbol first. */
 lhsdirect = 1;
 if( has_destructor(rp->rhs[0],lemp) ){
   append_str(0,0,0,0);
   append_str("  yy_destructor(yypParser,%d,[%d].minor);\n", 0,
  rp->rhs[0]->index,1-rp->nrhs);
   rp->codePrefix = Strsafe(append_str(0,0,0,0));
 }
+  }else if( rp->lhsalias==0 ){
+/* There is no LHS value symbol. */
+lhsdirect = 1;
   }else if( strcmp(rp->lhsalias,rp->rhsalias[0])==0 ){
 /* The LHS symbol and the left-most RHS symbol are the same, so 
 ** direct writing is allowed */
 lhsdirect = 1;
 lhsused = 1;
 used[0] = 1;


2. if there is a codePrefix or codeSuffix but no code, it will be merged into 
the default rule and the prefix/suffix will be lost.

@@ -3715,7 +3715,7 @@ PRIVATE int translate_code(struct lemon *lemp, struct 
rule *rp){

   /* Suffix code generation complete */
   cp = append_str(0,0,0,0);
-  if( cp ) rp->codeSuffix = Strsafe(cp);
+  if( cp && cp[0] ) rp->codeSuffix = Strsafe(cp);

   return rc;
 }

@@ -4397,7 +4397,8 @@ void ReportTable(
   for(rp=lemp->rule; rp; rp=rp->next){
 struct rule *rp2;   /* Other rules with the same action */
 if( rp->code==0 ) continue;
-if( rp->code[0]=='\n' && rp->code[1]==0 ) continue; /* Will be default: */
+if( rp->code[0]=='\n' && rp->code[1]==0 && rp->codePrefix==0
+  && rp->codeSuffix==0 ) continue; /* Will be default: */
 fprintf(out,"  case %d: /* ", rp->iRule);
 writeRuleText(out, rp);
 fprintf(out, " */\n"); lineno++;
@@ -4419,7 +4420,8 @@ void ReportTable(
   fprintf(out,"  default:\n"); lineno++;
   for(rp=lemp->rule; rp; rp=rp->next){
 if( rp->code==0 ) continue;
-assert( rp->code[0]=='\n' && rp->code[1]==0 );
+assert( rp->code[0]=='\n' && rp->code[1]==0 && rp->codePrefix==0
+  && rp->codeSuffix==0 );
 fprintf(out,"  /* (%d) ", rp->iRule);
 writeRuleText(out, rp);
 fprintf(out, " */ yytestcase(yyruleno==%d);\n", rp->iRule); lineno++;


Thanks,
Kelvin

[sqlite] Lemon reduce action merge bug

[Richard Hipp]

On 4/28/16, Kelvin Sherlock  wrote:
> I believe the lemon reduce action optimizer needs to compare the codePrefix
> and codeSuffix.

Thanks for the bug report.  A fix has now been checked in.

-- 
D. Richard Hipp
drh at sqlite.org

[sqlite] Lemon reduce action merge bug

[Kelvin Sherlock]

I believe the lemon reduce action optimizer needs to compare the codePrefix and 
codeSuffix. Consider this example:

8<

%type malloc {void *}
%destructor malloc {free($$)}
%type contrived_example {int}

bug_report ::= contrived_example.

contrived_example(LHS) ::= INT INT INT. { LHS = 0; }
contrived_example(LHS) ::= malloc malloc malloc. { LHS = 0; }

malloc(LHS) ::= . { LHS = malloc(100); } 

8< 

Since the code (but not the codeSuffix) matches, they will be merged together 
into this:

  case 0: /* contrived_example ::= INT INT INT */
  case 1: /* contrived_example ::= malloc malloc malloc */ 
yytestcase(yyruleno==1);
#line 8 "bug.lemon"
{ yymsp[-2].minor.yy4 = 0; }
#line 725 "bug.c"
break;

or this:

/** Begin reduce actions **/
  case 0: /* contrived_example ::= malloc malloc malloc */
  case 1: /* contrived_example ::= INT INT INT */ yytestcase(yyruleno==1);
{  yy_destructor(yypParser,3,[-2].minor);
#line 8 "bug.lemon"
{ yymsp[-2].minor.yy4 = 0; }
#line 726 "bug.c"
  yy_destructor(yypParser,3,[-1].minor);
  yy_destructor(yypParser,3,[0].minor);
}
break;

depending on which rule is first.

patch:

diff --git a/lemon.c b/lemon.c
index 903fe97..dca5a08 100644
--- a/lemon.c
+++ b/lemon.c
@@ -4402,7 +4402,11 @@ void ReportTable(
 writeRuleText(out, rp);
 fprintf(out, " */\n"); lineno++;
 for(rp2=rp->next; rp2; rp2=rp2->next){
-  if( rp2->code==rp->code ){
+  if(
+rp2->code==rp->code && 
+rp2->codePrefix==rp->codePrefix &&
+rp2->codeSuffix==rp->codeSuffix
+  ){
 fprintf(out,"  case %d: /* ", rp2->iRule);
 writeRuleText(out, rp2);
 fprintf(out," */ yytestcase(yyruleno==%d);\n", rp2->iRule); lineno++;


Thanks,
Kelvin Sherlock

[sqlite] lemon problem with two versions of lempar.c

[Neo Anderson]

I am utilizing lemon in one of my Apple Store Apps. My parser works fine with 
the 'old' lempar.c found in source tree TRUNK/tool/.I also found a 
newer version in TRUNK/src, but when I compile myparser.c generated using this 
lempar.c, I got compilation errors:test.c:376:7: error: use of 
undeclared identifier 'NEVER' if( NEVER(pParser-yyidx0) ) 
return 0;How can I solve this error? BTW, does this newer lempar.c 
benefit my parser more than the old one?

[sqlite] lemon problem with two versions of lempar.c

[Richard Hipp]

On 4/15/15, Neo Anderson  wrote:
> I am utilizing lemon in one of my Apple Store Apps. My parser works fine
> with the 'old' lempar.c found in source tree TRUNK/tool/.I also
> found a newer version in TRUNK/src, but when I compile myparser.c generated
> using this lempar.c, I got compilation errors:test.c:376:7: error:
> use of undeclared identifier 'NEVER' if(
> NEVER(pParser-yyidx0) ) return 0;How can I solve this error?
> BTW, does this newer lempar.c benefit my parser more than the old one?
>   

Use the one in tool/lempar.c.  The template in src/lempar.c contains
SQLite-specific modifications.
https://www.sqlite.org/src/fdiff?sbs=1=01ca97f8=7274c97
-- 
D. Richard Hipp
drh at sqlite.org

Re: [sqlite] lemon - %wildcar

[Sylvain Pointeau]

Hello,

Le samedi 26 juillet 2014, Richard Hipp  a écrit :
>
> Historical note:  I wrote Lemon back in the 80s (on a Sun3, IIRC)


You were a visionary, Lemon is probably the only viable option for a parser
generator in C/C++ at this moment.

because
> the $1,$2,$3 notation of Yacc wasn't working for me and also because Yacc
> wanted the parser to call the lexer, not the other way around, and because
> it was difficult to stop memory leaks in Yacc when there was a syntax
> error.  There was a companion LL(1) parser generator called "Lime" which
> has been lost.   Lemon was originally a bunch of separate source files,
> many of which were automatically generated.  But at some point I
> concatenated them all together into the single "lemon.c" file.  All that
> happened prior to the beginning of SQLite.  Anyhow, the current lemon.c
> code is a mess, for which I apologize.

But it does mostly work, so I've
> never gotten around to rewriting it from scratch, which is what it
> desperately needs.


Lemon is a master piece, it deserves it.
I propose my help, I am interested in the domain, but I might not be
capable enough. However I would be highly motivated if I am guided by
someone like you.


>
> The %wildcard token matches any token that cannot be matched by another
> rule in the current context.
>
>
It is exactly what I needed, I tried it and it is wonderful. I try to parse
some plsql code, but I only need to be superficial, then everything between
begin / end will be eaten by the wildcar


> I had to add the %wildcard feature to the parser generator when we added
> virtual tables to SQLite back in 2006 - in order to support the full text
> search extension.  The virtual table syntax allows one put any arbitrary
> sequence of tokens as the argument, as long as parentheses match up.
> Keywords, literals, operators - any tokens are allowed.  And the most
> memory-efficient way to do that was to enhance the parser generate with
> some special features to support it.  Hence:  "%wildcard".
>
>
I am lucky ;-)



> The lack of documentation is probably just because we never got around to
> it


> > if yes, could it be added in the documentation?
> >
>
> Do you have a patch :-)
>
>
Your documentation is among the best, I can help also there I think, how do
you want the patch to be done, I just need a little bit of explanation to
start. (I am on MacOS X)


>
> >
> > ps: I am pretty impressed by lemon, I am really having fun.
> >
> > Best regards,
> > Sylvain
> > ___
> > sqlite-users mailing list
> > sqlite-users@sqlite.org 
> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> >
>
>
>
> --
> D. Richard Hipp
> d...@sqlite.org 
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org 
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>

Best regards,
Sylvain
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] lemon - %wildcar

[Richard Hipp]

On Sat, Jul 26, 2014 at 11:10 AM, Sylvain Pointeau <
sylvain.point...@gmail.com> wrote:

> Hello,
>
> I would like to know if %wildcard is an undocumented feature on purpose, is
> this safe to use it?
>

Historical note:  I wrote Lemon back in the 80s (on a Sun3, IIRC) because
the $1,$2,$3 notation of Yacc wasn't working for me and also because Yacc
wanted the parser to call the lexer, not the other way around, and because
it was difficult to stop memory leaks in Yacc when there was a syntax
error.  There was a companion LL(1) parser generator called "Lime" which
has been lost.   Lemon was originally a bunch of separate source files,
many of which were automatically generated.  But at some point I
concatenated them all together into the single "lemon.c" file.  All that
happened prior to the beginning of SQLite.  Anyhow, the current lemon.c
code is a mess, for which I apologize.  But it does mostly work, so I've
never gotten around to rewriting it from scratch, which is what it
desperately needs.

The %wildcard token matches any token that cannot be matched by another
rule in the current context.

I had to add the %wildcard feature to the parser generator when we added
virtual tables to SQLite back in 2006 - in order to support the full text
search extension.  The virtual table syntax allows one put any arbitrary
sequence of tokens as the argument, as long as parentheses match up.
Keywords, literals, operators - any tokens are allowed.  And the most
memory-efficient way to do that was to enhance the parser generate with
some special features to support it.  Hence:  "%wildcard".

The lack of documentation is probably just because we never got around to
it


> if yes, could it be added in the documentation?
>

Do you have a patch :-)


>
> ps: I am pretty impressed by lemon, I am really having fun.
>
> Best regards,
> Sylvain
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] lemon - %wildcar

[Sylvain Pointeau]

Hello,

I would like to know if %wildcard is an undocumented feature on purpose, is
this safe to use it?
if yes, could it be added in the documentation?

ps: I am pretty impressed by lemon, I am really having fun.

Best regards,
Sylvain
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] (Lemon) (Patch) adding a -f option to Lemon to emit function prototypes

[Tiago Rodrigues]

I see; well, that's unfortunate.  At any rate, parsing lempar.c will
probably be much more complex than the code I have now, which solves a
simple (but common) case.  Maybe I'll look into parsing lempar.c, but since
this solves my use case, I probably won't be in a hurry to do it.  I
apologise to everyone this doesn't help, then.

Cheers,

  -Tiago


On Mon, Jan 7, 2013 at 9:13 PM, Richard Hipp  wrote:

> On Mon, Jan 7, 2013 at 4:18 PM, Tiago Rodrigues  wrote:
>
> > Oops, Dominique alerted me to the fact that the patch I included was
> > stripped by the listserver...  I'm including it inline, then.
> >
>
>
> This patch does not work in the general case.  I don't doubt that it works
> for your use case, but in general it does not.  The reason:  The function
> prototypes depend on the "lempar.c" template file.  Some applications (ex:
> SQLite) change the lempar.c parser template in ways that could change the
> function signatures.  (The function signatures for SQLite are unchanged,
> but that doesn't exclude the possibility that they might in the future.)
>
> For this to really work in general, I suppose you would have to somehow
> extract the function signatures from the lempar.c template file.  Or maybe
> have a separate lempar.h template that contains the header.  Or something.
>
>
>
> >
> > (Begin patch)
> >
> > --- lemon.c2013-01-04 20:39:20 +
> > +++ lemon-new.c2013-01-04 23:09:59 +
> > @@ -109,7 +109,7 @@
> >  void Reprint(struct lemon *);
> >  void ReportOutput(struct lemon *);
> >  void ReportTable(struct lemon *, int);
> > -void ReportHeader(struct lemon *);
> > +void ReportHeader(struct lemon *, int);
> >  void CompressTables(struct lemon *);
> >  void ResortStates(struct lemon *);
> >
> > @@ -1393,11 +1393,13 @@
> >static int mhflag = 0;
> >static int nolinenosflag = 0;
> >static int noResort = 0;
> > +  static int fpflag = 0;
> >static struct s_options options[] = {
> >  {OPT_FLAG, "b", (char*), "Print only the basis in
> report."},
> >  {OPT_FLAG, "c", (char*), "Don't compress the action
> table."},
> >  {OPT_FSTR, "D", (char*)handle_D_option, "Define an %ifdef macro."},
> >  {OPT_FSTR, "T", (char*)handle_T_option, "Specify a template file."},
> > +{OPT_FLAG, "f", (char*), "Generate function prototypes in
> > header."},
> >  {OPT_FLAG, "g", (char*), "Print grammar without actions."},
> >  {OPT_FLAG, "m", (char*), "Output a makeheaders compatible
> > file."},
> >  {OPT_FLAG, "l", (char*), "Do not print #line
> > statements."},
> > @@ -1502,7 +1504,7 @@
> >  /* Produce a header file for use by the scanner.  (This step is
> >  ** omitted if the "-m" option is used because makeheaders will
> >  ** generate the file for us.) */
> > -if( !mhflag ) ReportHeader();
> > +if( !mhflag ) ReportHeader(, fpflag);
> >}
> >if( statistics ){
> >  printf("Parser statistics: %d terminals, %d nonterminals, %d
> rules\n",
> > @@ -4009,16 +4011,20 @@
> >  }
> >
> >  /* Generate a header file for the parser */
> > -void ReportHeader(struct lemon *lemp)
> > +void ReportHeader(struct lemon *lemp, int fpflag)
> >  {
> >FILE *out, *in;
> > +  const char *name;
> >const char *prefix;
> >char line[LINESIZE];
> >char pattern[LINESIZE];
> >int i;
> > +  int protok = 1;
> >
> >if( lemp->tokenprefix ) prefix = lemp->tokenprefix;
> >elseprefix = "";
> > +  if( lemp->name ) name = lemp->name;
> > +  else name = "Parse";
> >in = file_open(lemp,".h","rb");
> >if( in ){
> >  int nextChar;
> > @@ -4026,9 +4032,24 @@
> >sprintf(pattern,"#define %s%-30s
> > %2d\n",prefix,lemp->symbols[i]->name,i);
> >if( strcmp(line,pattern) ) break;
> >  }
> > +if( fpflag ){
> > +sprintf(pattern,"void *%sAlloc(void *(*)(size_t));\n",name);
> > +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> > 0; goto after; }
> > +
> > +if( lemp->arg ){
> > +sprintf(pattern,"void %s(void
> > *,int,%s,%s);\n",name,lemp->tokentype,lemp->arg);
> > +}else{
> > +sprintf(pattern,"void %s(void
> > *,int,%s);\n",name,lemp->tokentype);
> > +}
> > +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> > 0; goto after; }
> > +
> > +sprintf(pattern,"void *%sFree(void *,void (*)(void
> *));\n",name);
> > +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> > 0; goto after; }
> > +}
> > +after:
> >  nextChar = fgetc(in);
> >  fclose(in);
> > -if( i==lemp->nterminal && nextChar==EOF ){
> > +if( i==lemp->nterminal && protok && nextChar==EOF ){
> >/* No change in the file.  Don't rewrite it. */
> >return;
> >  }
> > @@ -4038,6 +4059,16 @@
> >  for(i=1; interminal; i++){
> >fprintf(out,"#define %s%-30s
> > %2d\n",prefix,lemp->symbols[i]->name,i);
> >  }
> > +

Re: [sqlite] (Lemon) (Patch) adding a -f option to Lemon to emit function prototypes

[Richard Hipp]

On Mon, Jan 7, 2013 at 4:18 PM, Tiago Rodrigues  wrote:

> Oops, Dominique alerted me to the fact that the patch I included was
> stripped by the listserver...  I'm including it inline, then.
>


This patch does not work in the general case.  I don't doubt that it works
for your use case, but in general it does not.  The reason:  The function
prototypes depend on the "lempar.c" template file.  Some applications (ex:
SQLite) change the lempar.c parser template in ways that could change the
function signatures.  (The function signatures for SQLite are unchanged,
but that doesn't exclude the possibility that they might in the future.)

For this to really work in general, I suppose you would have to somehow
extract the function signatures from the lempar.c template file.  Or maybe
have a separate lempar.h template that contains the header.  Or something.



>
> (Begin patch)
>
> --- lemon.c2013-01-04 20:39:20 +
> +++ lemon-new.c2013-01-04 23:09:59 +
> @@ -109,7 +109,7 @@
>  void Reprint(struct lemon *);
>  void ReportOutput(struct lemon *);
>  void ReportTable(struct lemon *, int);
> -void ReportHeader(struct lemon *);
> +void ReportHeader(struct lemon *, int);
>  void CompressTables(struct lemon *);
>  void ResortStates(struct lemon *);
>
> @@ -1393,11 +1393,13 @@
>static int mhflag = 0;
>static int nolinenosflag = 0;
>static int noResort = 0;
> +  static int fpflag = 0;
>static struct s_options options[] = {
>  {OPT_FLAG, "b", (char*), "Print only the basis in report."},
>  {OPT_FLAG, "c", (char*), "Don't compress the action table."},
>  {OPT_FSTR, "D", (char*)handle_D_option, "Define an %ifdef macro."},
>  {OPT_FSTR, "T", (char*)handle_T_option, "Specify a template file."},
> +{OPT_FLAG, "f", (char*), "Generate function prototypes in
> header."},
>  {OPT_FLAG, "g", (char*), "Print grammar without actions."},
>  {OPT_FLAG, "m", (char*), "Output a makeheaders compatible
> file."},
>  {OPT_FLAG, "l", (char*), "Do not print #line
> statements."},
> @@ -1502,7 +1504,7 @@
>  /* Produce a header file for use by the scanner.  (This step is
>  ** omitted if the "-m" option is used because makeheaders will
>  ** generate the file for us.) */
> -if( !mhflag ) ReportHeader();
> +if( !mhflag ) ReportHeader(, fpflag);
>}
>if( statistics ){
>  printf("Parser statistics: %d terminals, %d nonterminals, %d rules\n",
> @@ -4009,16 +4011,20 @@
>  }
>
>  /* Generate a header file for the parser */
> -void ReportHeader(struct lemon *lemp)
> +void ReportHeader(struct lemon *lemp, int fpflag)
>  {
>FILE *out, *in;
> +  const char *name;
>const char *prefix;
>char line[LINESIZE];
>char pattern[LINESIZE];
>int i;
> +  int protok = 1;
>
>if( lemp->tokenprefix ) prefix = lemp->tokenprefix;
>elseprefix = "";
> +  if( lemp->name ) name = lemp->name;
> +  else name = "Parse";
>in = file_open(lemp,".h","rb");
>if( in ){
>  int nextChar;
> @@ -4026,9 +4032,24 @@
>sprintf(pattern,"#define %s%-30s
> %2d\n",prefix,lemp->symbols[i]->name,i);
>if( strcmp(line,pattern) ) break;
>  }
> +if( fpflag ){
> +sprintf(pattern,"void *%sAlloc(void *(*)(size_t));\n",name);
> +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> 0; goto after; }
> +
> +if( lemp->arg ){
> +sprintf(pattern,"void %s(void
> *,int,%s,%s);\n",name,lemp->tokentype,lemp->arg);
> +}else{
> +sprintf(pattern,"void %s(void
> *,int,%s);\n",name,lemp->tokentype);
> +}
> +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> 0; goto after; }
> +
> +sprintf(pattern,"void *%sFree(void *,void (*)(void *));\n",name);
> +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> 0; goto after; }
> +}
> +after:
>  nextChar = fgetc(in);
>  fclose(in);
> -if( i==lemp->nterminal && nextChar==EOF ){
> +if( i==lemp->nterminal && protok && nextChar==EOF ){
>/* No change in the file.  Don't rewrite it. */
>return;
>  }
> @@ -4038,6 +4059,16 @@
>  for(i=1; interminal; i++){
>fprintf(out,"#define %s%-30s
> %2d\n",prefix,lemp->symbols[i]->name,i);
>  }
> +if( fpflag ){
> +/* emit function prototypes */
> +fprintf(out,"void *%sAlloc(void *(*)(size_t));\n",name);
> +if( lemp->arg ){
> +fprintf(out,"void %s(void
> *,int,%s,%s);\n",name,lemp->tokentype,lemp->arg);
> +}else{
> +fprintf(out,"void %s(void *,int,%s);\n",name,lemp->tokentype);
> +}
> +fprintf(out,"void *%sFree(void *,void (*)(void *));\n",name);
> +}
>  fclose(out);
>}
>return;
>
> (End patch)
>
> Thanks,
>
>   -Tiago
>
>
> On Mon, Jan 7, 2013 at 6:15 AM, Dominique Devienne  >wrote:
>
> > On Sat, Jan 5, 2013 at 6:22 PM, Tiago Rodrigues 

Re: [sqlite] (Lemon) (Patch) adding a -f option to Lemon to emit function prototypes

[Tiago Rodrigues]

Oops, Dominique alerted me to the fact that the patch I included was
stripped by the listserver...  I'm including it inline, then.

(Begin patch)

--- lemon.c2013-01-04 20:39:20 +
+++ lemon-new.c2013-01-04 23:09:59 +
@@ -109,7 +109,7 @@
 void Reprint(struct lemon *);
 void ReportOutput(struct lemon *);
 void ReportTable(struct lemon *, int);
-void ReportHeader(struct lemon *);
+void ReportHeader(struct lemon *, int);
 void CompressTables(struct lemon *);
 void ResortStates(struct lemon *);

@@ -1393,11 +1393,13 @@
   static int mhflag = 0;
   static int nolinenosflag = 0;
   static int noResort = 0;
+  static int fpflag = 0;
   static struct s_options options[] = {
 {OPT_FLAG, "b", (char*), "Print only the basis in report."},
 {OPT_FLAG, "c", (char*), "Don't compress the action table."},
 {OPT_FSTR, "D", (char*)handle_D_option, "Define an %ifdef macro."},
 {OPT_FSTR, "T", (char*)handle_T_option, "Specify a template file."},
+{OPT_FLAG, "f", (char*), "Generate function prototypes in
header."},
 {OPT_FLAG, "g", (char*), "Print grammar without actions."},
 {OPT_FLAG, "m", (char*), "Output a makeheaders compatible
file."},
 {OPT_FLAG, "l", (char*), "Do not print #line
statements."},
@@ -1502,7 +1504,7 @@
 /* Produce a header file for use by the scanner.  (This step is
 ** omitted if the "-m" option is used because makeheaders will
 ** generate the file for us.) */
-if( !mhflag ) ReportHeader();
+if( !mhflag ) ReportHeader(, fpflag);
   }
   if( statistics ){
 printf("Parser statistics: %d terminals, %d nonterminals, %d rules\n",
@@ -4009,16 +4011,20 @@
 }

 /* Generate a header file for the parser */
-void ReportHeader(struct lemon *lemp)
+void ReportHeader(struct lemon *lemp, int fpflag)
 {
   FILE *out, *in;
+  const char *name;
   const char *prefix;
   char line[LINESIZE];
   char pattern[LINESIZE];
   int i;
+  int protok = 1;

   if( lemp->tokenprefix ) prefix = lemp->tokenprefix;
   elseprefix = "";
+  if( lemp->name ) name = lemp->name;
+  else name = "Parse";
   in = file_open(lemp,".h","rb");
   if( in ){
 int nextChar;
@@ -4026,9 +4032,24 @@
   sprintf(pattern,"#define %s%-30s
%2d\n",prefix,lemp->symbols[i]->name,i);
   if( strcmp(line,pattern) ) break;
 }
+if( fpflag ){
+sprintf(pattern,"void *%sAlloc(void *(*)(size_t));\n",name);
+if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
0; goto after; }
+
+if( lemp->arg ){
+sprintf(pattern,"void %s(void
*,int,%s,%s);\n",name,lemp->tokentype,lemp->arg);
+}else{
+sprintf(pattern,"void %s(void
*,int,%s);\n",name,lemp->tokentype);
+}
+if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
0; goto after; }
+
+sprintf(pattern,"void *%sFree(void *,void (*)(void *));\n",name);
+if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
0; goto after; }
+}
+after:
 nextChar = fgetc(in);
 fclose(in);
-if( i==lemp->nterminal && nextChar==EOF ){
+if( i==lemp->nterminal && protok && nextChar==EOF ){
   /* No change in the file.  Don't rewrite it. */
   return;
 }
@@ -4038,6 +4059,16 @@
 for(i=1; interminal; i++){
   fprintf(out,"#define %s%-30s %2d\n",prefix,lemp->symbols[i]->name,i);
 }
+if( fpflag ){
+/* emit function prototypes */
+fprintf(out,"void *%sAlloc(void *(*)(size_t));\n",name);
+if( lemp->arg ){
+fprintf(out,"void %s(void
*,int,%s,%s);\n",name,lemp->tokentype,lemp->arg);
+}else{
+fprintf(out,"void %s(void *,int,%s);\n",name,lemp->tokentype);
+}
+fprintf(out,"void *%sFree(void *,void (*)(void *));\n",name);
+}
 fclose(out);
   }
   return;

(End patch)

Thanks,

  -Tiago


On Mon, Jan 7, 2013 at 6:15 AM, Dominique Devienne wrote:

> On Sat, Jan 5, 2013 at 6:22 PM, Tiago Rodrigues  wrote:
> > [...]
> > This patch works against the current version of lemon.c, as linked to by
> > the Lemon page.  This is probably not correct, as the file is amalgamated
> > from other sources, but still might be useful to somebody.
>
> Hi Tiago. Looks like the patch was stripped by the mailing list.
> Perhaps you should include it inline to your message. Thanks, --DD
>



-- 
In those days, in those distant days, in those nights, in those remote
nights, in those years, in those distant years...
  - Gilgamesh, Enkidu and the Underworld
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] (Lemon) (Patch) adding a -f option to Lemon to emit function prototypes

[Tiago Rodrigues]

Hello, all,

First, forgive me if this is the wrong medium for submitting this, but I'm
rather new at this patch submission business.  Anyway, I was using Lemon to
generate a few parsers for a program I'm writing (which uses SQLite, too,
for storage), and I noticed that the files I used to call the parser were
giving me a strange "Pointer made from integer without a cast" on
ParseAlloc(), which is strange, since ParseAlloc() returns a void *.  It
took me a while to figure out that ParseAlloc(), ParseFree() and Parse()
weren't actually declared anywhere, so the compiler was assuming they
received no parameters and returned int.  Since I have more than one parser
to work with, I decided to have lemon emit the function prototypes on the
header file upon a new (-f) flag.  Then I decided that I might as well
submit this back to the community and let it decide whether the inclusion
is useful or not.

This patch works against the current version of lemon.c, as linked to by
the Lemon page.  This is probably not correct, as the file is amalgamated
from other sources, but still might be useful to somebody.

Thanks for reading,

  -Tiago

-- 
In those days, in those distant days, in those nights, in those remote
nights, in those years, in those distant years...
  - Gilgamesh, Enkidu and the Underworld
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon: Non-terminal destructors and cleanup

[Ben]

On 18 Oct 2012, at 20:07, Richard Hipp  wrote:

> On Thu, Oct 18, 2012 at 3:03 PM, Ben  wrote:
> 
>> Hi list,
>> 
>> I'm having a little trouble getting my head around memory management
>> within a Lemon-generated parser. Specifically the part of the docs stating
>> when a destructor will or will not be called.
>> 
>> For example, this is a portion of a grammar based on the SQLite parse.y
>> file:
>> 
>> 
>> columnName ::= nm(N). {
>>BSSQLiteColumn *col = [[BSSQLiteColumn alloc] init];
>>col.name = N.textValue;
>>[[parsedTable columns] addObject:col];
>> }
>> 
>> nm(A) ::= id(X). { A = X; }
>> nm(A) ::= STRING(X). { A = X; }
>> id(A) ::= ID(X). { A = X; }
>> 
>> Notes:
>> - The token type here is a struct containing an Objective-C string which
>> needs freeing when done with.
>> - Only a %token_destructor is defined, not any others
>> 
>> 
>> I know that the last three assignments are leaking memory, but I don't
>> know when I should be explicitly freeing my allocated memory within a token
>> and when I should be relying on the destructor defined by
>> %token_destructor{}. Or for that matter whether I should be declaring a
>> more specific symbol destructor.
>> 
>> Can anyone shed some light on how this should be done?
>> 
> 
> If the nonterminal payload is passed into an action (as in your example
> where N is processed because of nm(N)) then Lemon assume that your code
> will free the content, if needed.
> 
> If the rule had been:  columnName ::= nm {...}   (without the (N) argument
> to nm) then the destructor would have been called.

Got it. I've added two release calls for X at the end of the C code blocks for 
these two:
nm(A) ::= STRING(X). { A = X; }
id(A) ::= ID(X). { A = X; }

and now it's working leak-free.

Thank you.

Ben


> The destructor is also called if nm is popped from the stack for any reason
> other than the columnName ::= nm rule, such as when the stack is popped
> during error recovery.
> 
> 
>> 
>> Thanks,
>> 
>> Ben
>> 
>> 
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>> 
> 
> 
> 
> -- 
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon: Non-terminal destructors and cleanup

[Richard Hipp]

On Thu, Oct 18, 2012 at 3:03 PM, Ben  wrote:

> Hi list,
>
> I'm having a little trouble getting my head around memory management
> within a Lemon-generated parser. Specifically the part of the docs stating
> when a destructor will or will not be called.
>
> For example, this is a portion of a grammar based on the SQLite parse.y
> file:
>
>
> columnName ::= nm(N). {
> BSSQLiteColumn *col = [[BSSQLiteColumn alloc] init];
> col.name = N.textValue;
> [[parsedTable columns] addObject:col];
> }
>
> nm(A) ::= id(X). { A = X; }
> nm(A) ::= STRING(X). { A = X; }
> id(A) ::= ID(X). { A = X; }
>
> Notes:
> - The token type here is a struct containing an Objective-C string which
> needs freeing when done with.
> - Only a %token_destructor is defined, not any others
>
>
> I know that the last three assignments are leaking memory, but I don't
> know when I should be explicitly freeing my allocated memory within a token
> and when I should be relying on the destructor defined by
> %token_destructor{}. Or for that matter whether I should be declaring a
> more specific symbol destructor.
>
> Can anyone shed some light on how this should be done?
>

If the nonterminal payload is passed into an action (as in your example
where N is processed because of nm(N)) then Lemon assume that your code
will free the content, if needed.

If the rule had been:  columnName ::= nm {...}   (without the (N) argument
to nm) then the destructor would have been called.

The destructor is also called if nm is popped from the stack for any reason
other than the columnName ::= nm rule, such as when the stack is popped
during error recovery.


>
> Thanks,
>
> Ben
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon: Non-terminal destructors and cleanup

[Ben]

Hi list,

I'm having a little trouble getting my head around memory management within a 
Lemon-generated parser. Specifically the part of the docs stating when a 
destructor will or will not be called.

For example, this is a portion of a grammar based on the SQLite parse.y file:


columnName ::= nm(N). {
BSSQLiteColumn *col = [[BSSQLiteColumn alloc] init];
col.name = N.textValue;
[[parsedTable columns] addObject:col];
}

nm(A) ::= id(X). { A = X; }
nm(A) ::= STRING(X). { A = X; }
id(A) ::= ID(X). { A = X; }

Notes:
- The token type here is a struct containing an Objective-C string which needs 
freeing when done with.
- Only a %token_destructor is defined, not any others


I know that the last three assignments are leaking memory, but I don't know 
when I should be explicitly freeing my allocated memory within a token and when 
I should be relying on the destructor defined by %token_destructor{}. Or for 
that matter whether I should be declaring a more specific symbol destructor.

Can anyone shed some light on how this should be done?

Thanks,

Ben


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon: broken lookahead propagation?

[Vincent Zweije]

On Sat, Jan 07, 2012 at 10:18:12AM -0500, Richard Hipp wrote:

||  On Wed, Jan 4, 2012 at 11:00 AM, Vincent Zweije  wrote:
||
||  > I may have hit a bug in the lemon parser generator.
||  >
||
||  Please see if the following fix clears your problem:
||
||  http://www.sqlite.org/src/info/ce32775b23

Looks good: it produces a few more 's and other terminals in
the starters list, but I'm sure you checked that too. Haven't checked
the generated parser yet on actual input yet.

Thanks.  Vincent.
-- 
Vincent Zweije    | "If you're flamed in a group you
  | don't read, does anybody get burnt?"
[Xhost should be taken out and shot] |-- Paul Tomblin on a.s.r.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon: broken lookahead propagation?

[Richard Hipp]

On Wed, Jan 4, 2012 at 11:00 AM, Vincent Zweije  wrote:

> I may have hit a bug in the lemon parser generator.
>

Please see if the following fix clears your problem:

http://www.sqlite.org/src/info/ce32775b23



>
> It looks like lookahead symbols aren't propagated properly, in some cases.
>
> The consequence is that in some states, valid symbols are rejected with
> a syntax error.
>
> The complete grammar can be found at
>
>http://www.zweije.nl/~vzweije/parser.y
>
> I'm processing the grammar with lemon version 3.7.7-2ubuntu2. I've
> compared that version to the current head in sqlite, and it only differs
> in non-essential areas (calls to fclose recently added in trunk, calls
> to snprintf added in ubuntu). I'm pretty sure the current head version
> of lemon will exhibit the problem as well.
>
> For completeness' sake, this was found on an armv7l system.
>
> I have just reproduced the problem with the head lemon.c on an i386
> system. Valgrind reports no stray memory accesses (though quite a few
> memory leaks, but those are probably harmless).
>
> The command to process the parser is:
>
> lemon -c -s parser.y
>
> The generated parser has, among others, the following state:
>
> State 244:
>  constructor_def ::= existental_quant_variables_opt upper_name *
> fix_opt type_clos
>(200) fix_opt ::= *
>  fix_opt ::= * Fix
>
>  Open reduce 200
> CurlyOpen reduce 200
>  Hash reduce 200
>  RectOpen reduce 200
> A reduce 200
>   Dot reduce 200
>   Exclaim reduce 200
>  Star reduce 200
>   Int reduce 200
>  Real reduce 200
>  Char reduce 200
>  Bool reduce 200
>  RectOpenExclaimRectClose reduce 200
>   Dynamic reduce 200
>  File reduce 200
>String reduce 200
> World reduce 200
>   Fix shift  486
>   LowerCaseId reduce 200
>   UpperCaseId reduce 200
>   FunnyId reduce 200
>   fix_opt shift  130
>
> As can be seen, fix_opt is nullable - it may derive the empty string.
> Therefore, any symbol that can start what follows it (type_clos),
> or any lookahead symbol (because type_clos is nullable too), must
> be acceptable in this state. However, some tokens (Bar, Semicolon,
> LowerCaseId, UpperCaseId, FunnyId) are not.
>
> The next state lists the acceptable symbols there:
>
> State 130:
>  type_clos ::= * open_type_clos
>  type_clos ::= * closed_type_clos
>  open_type_clos ::= * closed_type_clos type_prefix_seq
>  closed_type_seq ::= * closed_type_clos type_prefix_clos
> type_expression
>  closed_type_clos ::= * closed_type_seq
>(144) closed_type_clos ::= *
>  constructor_def ::= existental_quant_variables_opt upper_name
> fix_opt * type_clos
>
>  Open reduce 144
> CurlyOpen reduce 144
>   Bar reduce 144
>  Hash reduce 144
>  RectOpen reduce 144
> A reduce 144
>   Dot reduce 144
>   Exclaim reduce 144
>  Star reduce 144
>   Int reduce 144
>  Real reduce 144
>  Char reduce 144
>  Bool reduce 144
>  RectOpenExclaimRectClose reduce 144
>   Dynamic reduce 144
>  File reduce 144
>String reduce 144
> World reduce 144
>   LowerCaseId reduce 144
>   UpperCaseId reduce 144
>   FunnyId reduce 144
> Semicolon reduce 144
> type_clos shift  563
>   closed_type_seq shift  375
>  closed_type_clos shift  179
>open_type_clos shift  491
>
> Some of the missing symbols (LowerCaseId, UpperCaseId, FunnyId) are
> starters of type_clos:
>
>  164: type_clos: Open CurlyOpen Hash RectOpen A Dot Exclaim Star Int Real
> Char Bool RectOpenExclaimRectClose Dynamic File String World LowerCaseId
> UpperCaseId FunnyId
>
> The other two (Bar, Semicolon) are not; these are followers of the
> constructor_def symbol.
>
> Looking at it now,  is not in the starters of type_clos,
> though according to the grammar rules, it is obviously nullable
> (type_clos => closed_type_close => ). Curious.
>
> The problem disappears when fix_opt is deleted from the RHS of the
> rule. (Of course, this means that a present Fix token is no longer
> accepted.)
>
> Ciao.   

[sqlite] Lemon: broken lookahead propagation?

[Vincent Zweije]

I may have hit a bug in the lemon parser generator.

It looks like lookahead symbols aren't propagated properly, in some cases.

The consequence is that in some states, valid symbols are rejected with
a syntax error.

The complete grammar can be found at

http://www.zweije.nl/~vzweije/parser.y

I'm processing the grammar with lemon version 3.7.7-2ubuntu2. I've
compared that version to the current head in sqlite, and it only differs
in non-essential areas (calls to fclose recently added in trunk, calls
to snprintf added in ubuntu). I'm pretty sure the current head version
of lemon will exhibit the problem as well.

For completeness' sake, this was found on an armv7l system.

I have just reproduced the problem with the head lemon.c on an i386
system. Valgrind reports no stray memory accesses (though quite a few
memory leaks, but those are probably harmless).

The command to process the parser is:

lemon -c -s parser.y

The generated parser has, among others, the following state:

State 244:
  constructor_def ::= existental_quant_variables_opt upper_name * 
fix_opt type_clos
(200) fix_opt ::= *
  fix_opt ::= * Fix

  Open reduce 200
 CurlyOpen reduce 200
  Hash reduce 200
  RectOpen reduce 200
 A reduce 200
   Dot reduce 200
   Exclaim reduce 200
  Star reduce 200
   Int reduce 200
  Real reduce 200
  Char reduce 200
  Bool reduce 200
  RectOpenExclaimRectClose reduce 200
   Dynamic reduce 200
  File reduce 200
String reduce 200
 World reduce 200
   Fix shift  486
   LowerCaseId reduce 200
   UpperCaseId reduce 200
   FunnyId reduce 200
   fix_opt shift  130

As can be seen, fix_opt is nullable - it may derive the empty string.
Therefore, any symbol that can start what follows it (type_clos),
or any lookahead symbol (because type_clos is nullable too), must
be acceptable in this state. However, some tokens (Bar, Semicolon,
LowerCaseId, UpperCaseId, FunnyId) are not.

The next state lists the acceptable symbols there:

State 130:
  type_clos ::= * open_type_clos
  type_clos ::= * closed_type_clos
  open_type_clos ::= * closed_type_clos type_prefix_seq
  closed_type_seq ::= * closed_type_clos type_prefix_clos 
type_expression
  closed_type_clos ::= * closed_type_seq
(144) closed_type_clos ::= *
  constructor_def ::= existental_quant_variables_opt upper_name fix_opt 
* type_clos

  Open reduce 144
 CurlyOpen reduce 144
   Bar reduce 144
  Hash reduce 144
  RectOpen reduce 144
 A reduce 144
   Dot reduce 144
   Exclaim reduce 144
  Star reduce 144
   Int reduce 144
  Real reduce 144
  Char reduce 144
  Bool reduce 144
  RectOpenExclaimRectClose reduce 144
   Dynamic reduce 144
  File reduce 144
String reduce 144
 World reduce 144
   LowerCaseId reduce 144
   UpperCaseId reduce 144
   FunnyId reduce 144
 Semicolon reduce 144
 type_clos shift  563
   closed_type_seq shift  375
  closed_type_clos shift  179
open_type_clos shift  491

Some of the missing symbols (LowerCaseId, UpperCaseId, FunnyId) are
starters of type_clos:

  164: type_clos: Open CurlyOpen Hash RectOpen A Dot Exclaim Star Int Real Char 
Bool RectOpenExclaimRectClose Dynamic File String World LowerCaseId UpperCaseId 
FunnyId

The other two (Bar, Semicolon) are not; these are followers of the
constructor_def symbol.

Looking at it now,  is not in the starters of type_clos,
though according to the grammar rules, it is obviously nullable
(type_clos => closed_type_close => ). Curious.

The problem disappears when fix_opt is deleted from the RHS of the
rule. (Of course, this means that a present Fix token is no longer
accepted.)

Ciao.  Vincent.
-- 
Vincent Zweije    | "If you're flamed in a group you
  | don't read, does anybody get burnt?"
[Xhost should be taken out and shot] |-- Paul Tomblin on a.s.r.
___
sqlite-users mailing 

[sqlite] Lemon and errors

[Igmar Palsenberg]

Hi,

I'm attempting to rewrite a buggy hand-written parser with a re2c / lemon one. 
I've ran into a strange thing (which is/was actually a bug in the lexer), but I 
can't determine why lemon finishes parsing, instead of throwing up a parser 
error.

The stripped-down parser :

[igmar@devel native]$ lemon -g parser.le 
// Reprint of input file "parser.le".
// Symbols:
//   0 $  4 CURLY_OPEN 8 start 12 identparam  
//   1 DOUBLE_QUOTE   5 CURLY_CLOSE9 groups  
//   2 STRING 6 SEMICOLON 10 group   
//   3 IDENTIFIER 7 error 11 identparams 
start ::= groups.
groups ::= groups group.
groups ::= group.
identparams ::= identparams identparam.
identparams ::= identparam.
identparam ::= DOUBLE_QUOTE STRING|IDENTIFIER DOUBLE_QUOTE.
identparam ::= STRING|IDENTIFIER.
group ::= IDENTIFIER identparams CURLY_OPEN CURLY_CLOSE SEMICOLON.
group ::= IDENTIFIER CURLY_OPEN CURLY_CLOSE SEMICOLON.

Should be fairly simple.

Input :

service transparent nntp {
};

acl telnet-service-acl {
};

I've put the tracelog on pastebin : http://pastebin.com/9FiUmd6a , can anyone 
en lighten me why the parser doesn't throw parser errors as I expect ?


Regards,


Igmar
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon behavior

[Christian Smith]

On Tue, Dec 07, 2010 at 08:09:53PM +0100, Begelman, Jamie wrote:
> I'm using Lemon for a non-sqlite related project and it is exiting with an 
> assertion failure that I would like to understand. I have extracted the 
> following small set of productions from a larger grammar. The "list" 
> production happens to be the start symbol in the larger grammar.
> 
> list::= list DELIMITER command.
> list::= command.
> command ::= TERMINAL1.
> command ::= TERMINAL2.
> 
> When I place these in a .y file by themselves and build the file, Lemon fails 
> with:
> 
> Assertion failed: apx->type==SH_RESOLVED || apx->type==RD_RESOLVED || 
> apx->type==SSCONFLICT || apx->type==SRCONFLICT || apx->type==RRCONFLICT || 
> apy->type==SH_RESOLVED || apy->type==RD_RESOLVED || apy->type==SSCONFLICT || 
> apy->type==SRCONFLICT || apy->type==RRCONFLICT, file lemon.c, line 1065
> 
> The odd thing is I use this pattern to parse lists of things elsewhere in the 
> grammar without issue. Any insight appreciated.
> 


I observed that adding a single rule to the beginning of the grammar:

main::= list.

Fixed things for me. Try that. I've no idea why it makes a difference, mind. 
This was tried
with lemon in SQLite 3.5.9, from Debian lenny.

Christian
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon behavior

[Begelman, Jamie]

I'm using Lemon for a non-sqlite related project and it is exiting with an 
assertion failure that I would like to understand. I have extracted the 
following small set of productions from a larger grammar. The "list" production 
happens to be the start symbol in the larger grammar.

list::= list DELIMITER command.
list::= command.
command ::= TERMINAL1.
command ::= TERMINAL2.

When I place these in a .y file by themselves and build the file, Lemon fails 
with:

Assertion failed: apx->type==SH_RESOLVED || apx->type==RD_RESOLVED || 
apx->type==SSCONFLICT || apx->type==SRCONFLICT || apx->type==RRCONFLICT || 
apy->type==SH_RESOLVED || apy->type==RD_RESOLVED || apy->type==SSCONFLICT || 
apy->type==SRCONFLICT || apy->type==RRCONFLICT, file lemon.c, line 1065

The odd thing is I use this pattern to parse lists of things elsewhere in the 
grammar without issue. Any insight appreciated.

TIA,
Jamie


The information contained in this message may be confidential and legally 
protected under applicable law. The message is intended solely for the 
addressee(s). If you are not the intended recipient, you are hereby notified 
that any use, forwarding, dissemination, or reproduction of this message is 
strictly prohibited and may be unlawful. If you are not the intended recipient, 
please contact the sender by return e-mail and destroy all copies of the 
original message.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon maintainer

[Richard Hipp]

On Tue, Oct 12, 2010 at 6:54 PM, Vincent Adam Burns wrote:

> Is there an active maintainer for the Lemon Parser? I'm getting some
> parsing conflicts, ex:
>
> statement ::= IF LEFT_PAREN expression RIGHT_PAREN statement ELSE
> statement.
> statement ::= IF LEFT_PAREN expression RIGHT_PAREN statement.
>
> expr ::= expr EQ|NE|GT|GE|LT|LE expr.
> expr ::= expr MOD|DIV|MOD expr.
> expr ::= expr ADD|SUB expr.
> expr ::= primaryExpr.
>

Problems with Lemon should be directed to the SQLite mailing list.

However, parsing conflicts are not problems with Lemon, but rather problems
with your grammar.  Unfortunately there is (to my knowledge) no tutorial on
how to write conflict-free LALR(1) grammers for Lemon.  You have to grok
LALR(1) first, then Lemon just makes sense.

Perhaps you should start with a copy of the Dragon book.  New copies are
wildly over-priced.  But it looks like you can get a used copy from Amazon
for just a few bucks.
http://www.amazon.com/Compilers-Principles-Techniques-Alfred-Aho/dp/0201100886



>
> --
> --- You know you've achieved perfection in design, not when you have
> nothing more to add, but when you have nothing more to take away.
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] lemon-generated parser cooperation with a scanner not under my control

[Aspra Flavius Adrian]

Hi

How could I force lemon assign a specific major number to a terminal? I need
that in order to make a lemon-generated parser work together with an already
existing scanner which is not under my control, and I'd like not to fall
back to "manual" translation.

Actually "manual" translation would complicate the stuff I'm doing by an
order of magnitude which should really not be
necessary.

Thanks
--Flavius
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser

[Chris verBurg]

On Tue, Apr 6, 2010 at 10:40 PM, Dan Kennedy  wrote:

>
> On Apr 7, 2010, at 3:23 AM, Chris verBurg wrote:
>
> > Hehe, okay, here I go.  :)
> >
> >
> > I'm trying to replace an existing flex/bison parser with an re2c/lemon
> > parser, but I'm running into a methodological problem.  I have a
> > hypothetical grammar like this:
> >
> >  file ::= FOO str .
> >  file ::= BAR str .
>
> See the use of the "%fallback" directive in the parse.y
> file that is part of SQLite sources. I think you want
> something like this:
>
>   %fallback str BAR.
>
>
That does almost exactly what I want.  Thank you thank you thank you!  :)
Do you know why that isn't documented?  (Or, am I looking at the wrong
documentation?)

-Chris
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser

[Dan Kennedy]

On Apr 7, 2010, at 3:23 AM, Chris verBurg wrote:

> Hehe, okay, here I go.  :)
>
>
> I'm trying to replace an existing flex/bison parser with an re2c/lemon
> parser, but I'm running into a methodological problem.  I have a
> hypothetical grammar like this:
>
>  file ::= FOO str .
>  file ::= BAR str .

See the use of the "%fallback" directive in the parse.y
file that is part of SQLite sources. I think you want
something like this:

   %fallback str BAR.


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser

[Chris verBurg]

Hehe, okay, here I go.  :)


I'm trying to replace an existing flex/bison parser with an re2c/lemon
parser, but I'm running into a methodological problem.  I have a
hypothetical grammar like this:

  file ::= FOO str .
  file ::= BAR str .

Where my keywords are FOO and BAR, and str is any ol' generic string.

I'm giving it the input "FOO BAR" in the hopes that it will match the first
rule and give str the value "BAR".  To keep Lemon from choking on a syntax
error when it sees BAR, I want to tell the lexer (re2c) to force the next
token after "FOO" to be a string.  Thus I broke the rule up like so:

  file ::= fooDefinition .

  fooDefinition ::= fooStart fooEnd .
  fooStart ::= FOO . { lexer->forceNextThingToBeAString(); }
  fooEnd ::= str .

However, the call to forceNextThingToBeAString() happens after re2c reads
"BAR", and by that time it's already identified it as the keyword BAR
instead of as a string that happens to be set to "BAR".  It seems like Lemon
always reads a lookahead token, even though it doesn't need to.

I considered using the lexer-only hack of having the lexer force a string
after "FOO", but that only works if every FOO in the grammar is followed by
str.  In my real-world problem, that's not the case.  The CFG of the lexer
is not adequate to know when to force strings; it requires control from the
parser.


Would you have any comments/ideas?

Thanks!
-Chris


On Tue, Apr 6, 2010 at 1:12 PM, Wilson, Ronald  wrote:

> > I'm using the Lemon parser and running into a methodological problem
> that
> > I
> > wanted to ask the user base about.  Is there a mailing list or forum
> > specifically for Lemon, or is this it?  :)
> >
> > Thanks,
> > -Chris
>
> There is no mailing list specifically for lemon.  Some of us are
> familiar to various degrees with lemon, so feel free to ask here.
> Though I must say, lemon questions tend languish for a while before
> someone replies.  Don't be afraid to bump your own message if it isn't
> answered.  I think it's safe to say that most sqlite users aren't
> initimately familiar with lemon.
>
> RW
>
> Ron Wilson, Engineering Project Lead
> (o) 434.455.6453, (m) 434.851.1612, www.harris.com
>
> HARRIS CORPORATION   |   RF Communications Division
> assuredcommunications(tm)
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser

[Wilson, Ronald]

> I'm using the Lemon parser and running into a methodological problem
that
> I
> wanted to ask the user base about.  Is there a mailing list or forum
> specifically for Lemon, or is this it?  :)
> 
> Thanks,
> -Chris

There is no mailing list specifically for lemon.  Some of us are
familiar to various degrees with lemon, so feel free to ask here.
Though I must say, lemon questions tend languish for a while before
someone replies.  Don't be afraid to bump your own message if it isn't
answered.  I think it's safe to say that most sqlite users aren't
initimately familiar with lemon.

RW

Ron Wilson, Engineering Project Lead
(o) 434.455.6453, (m) 434.851.1612, www.harris.com

HARRIS CORPORATION   |   RF Communications Division
assuredcommunications(tm)
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon parser

[Chris verBurg]

Hey guys,

I'm using the Lemon parser and running into a methodological problem that I
wanted to ask the user base about.  Is there a mailing list or forum
specifically for Lemon, or is this it?  :)

Thanks,
-Chris
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] lemon mods

[Wilson, Ronald]

With all the good changes to lemon recently, I thought I'd post changes that I 
have made to my personal copy.  One of my customers requires that I demonstrate 
the version of all build tools I use at build time, so I added a few command 
line parameters to help with that.

It would be nice if the version identifier for lemon would increment as it 
matures, but I understand why it doesn't currently increment.  It doesn't make 
sense to tie the version of lemon.exe to sqlite3.exe.  I've toyed with using 
the build script to query fossil finfo lemon.c --limit 1 and use the artifact 
id or version id to uniquely identify a particular lemon build.

RW

Ron Wilson, Engineering Project Lead
(o) 434.455.6453, (m) 434.851.1612, www.harris.com

project-name: SQLite
repository:   C:/rev/fossil/sqlite3.f
local-root:   C:/rev/src/sqlite3/
user-home:    C:/Documents and Settings/ma088024/Application Data
project-code: 2ab58778c2967968b94284e989e43dc11791f548
server-code:  dcd2b579c69c56d9973f76b372db9806df4a3252
checkout: 1e8b842039cc06b57a321226633c55b94eb8dcd7 2010-02-22 19:37:44 UTC
parent:   a8076aede33c07e9a2aaa05be8a888f37b45e41c 2010-02-22 19:32:32 UTC
parent:   721f33e7221c5fc907e9e293ac3242843f4fcfb7 2010-02-17 20:31:32 UTC
tags: trunk

--- lemon.c
+++ lemon.c
@@ -4,10 +4,13 @@
 ** single file to make it easy to include LEMON in the source tree
 ** and Makefile of another program.
 **
 ** The author of this program disclaims copyright.
 */
+
+#define SZVERSION "1.0"
+
 #include 
 #include 
 #include 
 #include 
 #include 
@@ -99,10 +102,11 @@
   enum option_type type;
   const char *label;
   char *arg;
   const char *message;
 };
+void   OptVersion(void);
 int    OptInit(char**,struct s_options*,FILE*);
 int    OptNArgs(void);
 char  *OptArg(int);
 void   OptErr(int);
 void   OptPrint(void);
@@ -1395,10 +1399,12 @@
 }
 
 /* The main program.  Parse the command line and do it... */
 int main(int argc, char **argv)
 {
+  static int help = 0;
+  static int versiononly = 0;
   static int version = 0;
   static int rpflag = 0;
   static int basisflag = 0;
   static int compress = 0;
   static int quiet = 0;
@@ -1409,31 +1415,46 @@
 {OPT_FLAG, "b", (char*), "Print only the basis in report."},
 {OPT_FLAG, "c", (char*), "Don't compress the action table."},
 {OPT_FSTR, "D", (char*)handle_D_option, "Define an %ifdef macro."},
 {OPT_FSTR, "T", (char*)handle_T_option, "Specify a template file."},
 {OPT_FLAG, "g", (char*), "Print grammar without actions."},
+    {OPT_FLAG, "h", (char*), "Print this help message."},
 {OPT_FLAG, "m", (char*), "Output a makeheaders compatible file."},
 {OPT_FLAG, "l", (char*), "Do not print #line statements."},
 {OPT_FLAG, "q", (char*), "(Quiet) Don't print the report file."},
 {OPT_FLAG, "s", (char*),
    "Print parser stats to standard output."},
-    {OPT_FLAG, "x", (char*), "Print the version number."},
+    {OPT_FLAG, "v", (char*), "Print the version number and continue."},
+    {OPT_FLAG, "x", (char*), "Print the version number and exit."},
 {OPT_FLAG,0,0,0}
   };
   int i;
   int exitcode;
   struct lemon lem;
 
   atexit(LemonAtExit);
 
   OptInit(argv,options,stderr);
+
+  if ( help ) {
+    OptVersion();
+    printf("Valid command line options are:\n");
+    OptPrint();
+    exit(0);
+  }
+  if( versiononly ){
+    OptVersion();
+    exit(0);
+  }
   if( version ){
- printf("Lemon version 1.0\n");
- exit(0);
+    OptVersion();
   }
   if( OptNArgs()!=1 ){
 fprintf(stderr,"Exactly one filename argument is required.\n");
+    OptVersion();
+    printf("Valid command line options are:\n");
+    OptPrint();
 exit(1);
   }
   memset(, 0, sizeof(lem));
   lem.errorcnt = 0;
 
@@ -1821,10 +1842,15 @@
     (*(void(*)(char *))(op[j].arg))(sv);
 break;
 }
   }
   return errcnt;
+}
+
+void OptVersion()
+{
+ printf("Lemon version %s\n", SZVERSION);
 }
 
 int OptInit(char **a, struct s_options *o, FILE *err)
 {
   int errcnt = 0;

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] [Lemon] Jump to %syntax_error

[Igmar Palsenberg]

Hi,

Can I somehow simulate a syntax error from within a rule action ? This 
parser looks up the token it receives, and I want to abort futher 
parsing in case that the lookup fails. as if there was a syntax error in 
the grammar.


Regards,


 Igmr
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] lemon: error handling

[D. Richard Hipp]

On Dec 22, 2009, at 6:06 PM, Benjamin Peterson wrote:

> I'm using lemon to write a parser for a little language I'm writing.
> I'm wondering how I indicate to lemon that an error has occurred in
> processing and an exit is needed. For example, if I have:
>
> stmt(A) ::= NAME(B). { A = malloc(sizeof(stmt)); A->name = B; }
>
> If malloc returns NULL, what should I do to escape the parser?

Configure the %extra_argument to be a pointer to a structure that  
holds the state of your parser.

  %extra_argument {Parser *pParser}

The pParser pointer is directly accessible from within your actions.   
So set a flag in that structure that tell the tokenizer to stop  
sending tokens and abort with an error.

 stmt(A) ::= NAME(B).  {
A = malloc(sizeof(stmt));
if( A==0 ){
   pParser->errFlag = 1;
}else{
  A->name = B;
   }
}

>
> -- 
> Regards,
> Benjamin
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

D. Richard Hipp
d...@hwaci.com



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] lemon: error handling

[Benjamin Peterson]

I'm using lemon to write a parser for a little language I'm writing.
I'm wondering how I indicate to lemon that an error has occurred in
processing and an exit is needed. For example, if I have:

stmt(A) ::= NAME(B). { A = malloc(sizeof(stmt)); A->name = B; }

If malloc returns NULL, what should I do to escape the parser?

-- 
Regards,
Benjamin
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser : compile error when using "%token_destructor" directive

[Wilson, Ron P]

Very interesting.  I had no idea that lemon.c and lempar.c were being revised.  
I assumed they were static.

Would it be too much to increment the version that lemon -x prints?  Currently 
it prints out "Lemon version 1.0" which led me to believe it was not being 
actively developed.

RW

P.S.  I like what I see in the revision history.

Ron Wilson, Engineering Project Lead, 434.455.6453

HARRIS CORPORATION   |   RF Communications Division 
assuredcommunications(tm)


-Original Message-
From: Vincent Zweije [mailto:vzwe...@wcc-group.com] 
Sent: Monday, June 15, 2009 12:54 PM
To: Wilson, Ron P
Cc: General Discussion of SQLite Database
Subject: Re: [sqlite] Lemon parser : compile error when using 
"%token_destructor" directive

On Mon, Jun 15, 2009 at 11:42:26AM -0400, Wilson, Ron P wrote:

||  It has been a while since I used lemon (big fan though).  Did you resolve
||  this issue or do you still need help?

[It appears my previous response did not get through.]

Looks suspiciously like this problem, which was fixed in version 3.6.2:

http://www.sqlite.org/cvstrac/tktview?tn=3299

Ciao.

||  -Original Message-
||  From: sqlite-users-boun...@sqlite.org 
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of ferrety ferrety
||  Sent: Monday, June 08, 2009 7:15 PM
||  To: sqlite-users@sqlite.org
||  Subject: [sqlite] Lemon parser : compile error when using 
"%token_destructor" directive
||
||  Hi List,
||  This is the only place I found to ask for "lemon parser" error.
||
||  When trying to use the "%token_destructor" directive:
||
||  %include {
||  #include 
||  void token_dtor (struct Token * t)
||{
||  fprintf(stderr, "In token_destructor: t -> value=%s\n", t -> value);
||}
||  }
||  %token_destructor { token_dtor($$); }
||
||
||  I got the following error:
||  error: too many arguments to function 'yy_destructor'
||
||  I tried to understand how "lemon" generated the calls for that function and
||  found that:
||
||  1) The definition of "yy_destructor" is with only 2 arguments:
||  static void yy_destructor(YYCODETYPE yymajor, YYMINORTYPE *yypminor){
||switch( yymajor ){
||  /* Here is inserted the actions which take place when a
||  ...
||
||  2) "Lemon" generates different calls to "yy_destructor", sometimes with 2 or
||  3 parameters :
||  yy_destructor( yymajor, >minor); <- HERE with 2 arguments,
||  which is fine
||  yy_destructor(yypParser,4,[0].minor); <- HERE with 3 arguments, which
||  is an error
||  yy_destructor(yypParser,5,[-2].minor); <- same
||  yy_destructor(yypParser,6,[0].minor);  <- same
||  ...
||
||  Sometime, the "yy_destructor" si called with 2 arguments as defined and some
||  other time,
||  it's called with 3 arguments.
||
||  Is it a bug or am I missing something?
-- 
WCC - Smart Search & Match
NL  +31 30 7503222
vzwe...@wcc-group.com
www.wcc-group.com
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser : compile error when using "%token_destructor" directive

[Vincent Zweije]

On Mon, Jun 15, 2009 at 11:42:26AM -0400, Wilson, Ron P wrote:

||  It has been a while since I used lemon (big fan though).  Did you resolve
||  this issue or do you still need help?

[It appears my previous response did not get through.]

Looks suspiciously like this problem, which was fixed in version 3.6.2:

http://www.sqlite.org/cvstrac/tktview?tn=3299

Ciao.

||  -Original Message-
||  From: sqlite-users-boun...@sqlite.org 
[mailto:sqlite-users-boun...@sqlite.org] On Behalf Of ferrety ferrety
||  Sent: Monday, June 08, 2009 7:15 PM
||  To: sqlite-users@sqlite.org
||  Subject: [sqlite] Lemon parser : compile error when using 
"%token_destructor" directive
||
||  Hi List,
||  This is the only place I found to ask for "lemon parser" error.
||
||  When trying to use the "%token_destructor" directive:
||
||  %include {
||  #include 
||  void token_dtor (struct Token * t)
||{
||  fprintf(stderr, "In token_destructor: t -> value=%s\n", t -> value);
||}
||  }
||  %token_destructor { token_dtor($$); }
||
||
||  I got the following error:
||  error: too many arguments to function 'yy_destructor'
||
||  I tried to understand how "lemon" generated the calls for that function and
||  found that:
||
||  1) The definition of "yy_destructor" is with only 2 arguments:
||  static void yy_destructor(YYCODETYPE yymajor, YYMINORTYPE *yypminor){
||switch( yymajor ){
||  /* Here is inserted the actions which take place when a
||  ...
||
||  2) "Lemon" generates different calls to "yy_destructor", sometimes with 2 or
||  3 parameters :
||  yy_destructor( yymajor, >minor); <- HERE with 2 arguments,
||  which is fine
||  yy_destructor(yypParser,4,[0].minor); <- HERE with 3 arguments, which
||  is an error
||  yy_destructor(yypParser,5,[-2].minor); <- same
||  yy_destructor(yypParser,6,[0].minor);  <- same
||  ...
||
||  Sometime, the "yy_destructor" si called with 2 arguments as defined and some
||  other time,
||  it's called with 3 arguments.
||
||  Is it a bug or am I missing something?
-- 
WCC - Smart Search & Match
NL  +31 30 7503222
vzwe...@wcc-group.com
www.wcc-group.com
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon parser : compile error when using "%token_destructor" directive

[Wilson, Ron P]

It has been a while since I used lemon (big fan though).  Did you resolve this 
issue or do you still need help?

RW

Ron Wilson, Engineering Project Lead, 434.455.6453

HARRIS CORPORATION   |   RF Communications Division 
assuredcommunications(tm)


-Original Message-
From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] 
On Behalf Of ferrety ferrety
Sent: Monday, June 08, 2009 7:15 PM
To: sqlite-users@sqlite.org
Subject: [sqlite] Lemon parser : compile error when using "%token_destructor" 
directive

Hi List,
This is the only place I found to ask for "lemon parser" error.

When trying to use the "%token_destructor" directive:

%include {
#include 
void token_dtor (struct Token * t)
  {
fprintf(stderr, "In token_destructor: t -> value=%s\n", t -> value);
  }
}
%token_destructor { token_dtor($$); }


I got the following error:
error: too many arguments to function 'yy_destructor'

I tried to understand how "lemon" generated the calls for that function and
found that:

1) The definition of "yy_destructor" is with only 2 arguments:
static void yy_destructor(YYCODETYPE yymajor, YYMINORTYPE *yypminor){
  switch( yymajor ){
/* Here is inserted the actions which take place when a
...

2) "Lemon" generates different calls to "yy_destructor", sometimes with 2 or
3 parameters :
yy_destructor( yymajor, >minor); <- HERE with 2 arguments,
which is fine
yy_destructor(yypParser,4,[0].minor); <- HERE with 3 arguments, which
is an error
yy_destructor(yypParser,5,[-2].minor); <- same
yy_destructor(yypParser,6,[0].minor);  <- same
...

Sometime, the "yy_destructor" si called with 2 arguments as defined and some
other time,
it's called with 3 arguments.

Is it a bug or am I missing something?

Thanks in advance
Frederic
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon parser : compile error when using "%token_destructor" directive

[ferrety ferrety]

Hi List,
This is the only place I found to ask for "lemon parser" error.

When trying to use the "%token_destructor" directive:

%include {
#include 
void token_dtor (struct Token * t)
  {
fprintf(stderr, "In token_destructor: t -> value=%s\n", t -> value);
  }
}
%token_destructor { token_dtor($$); }


I got the following error:
error: too many arguments to function 'yy_destructor'

I tried to understand how "lemon" generated the calls for that function and
found that:

1) The definition of "yy_destructor" is with only 2 arguments:
static void yy_destructor(YYCODETYPE yymajor, YYMINORTYPE *yypminor){
  switch( yymajor ){
/* Here is inserted the actions which take place when a
...

2) "Lemon" generates different calls to "yy_destructor", sometimes with 2 or
3 parameters :
yy_destructor( yymajor, >minor); <- HERE with 2 arguments,
which is fine
yy_destructor(yypParser,4,[0].minor); <- HERE with 3 arguments, which
is an error
yy_destructor(yypParser,5,[-2].minor); <- same
yy_destructor(yypParser,6,[0].minor);  <- same
...

Sometime, the "yy_destructor" si called with 2 arguments as defined and some
other time,
it's called with 3 arguments.

Is it a bug or am I missing something?

Thanks in advance
Frederic
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon: Functionality like Bison's %error-verbose / Access to last follow set from %syntax_error

[Markus Thiele]

Greetings,

> Lemon does not have any feature that will provide the application with  
> access to the follow-set.  You could perhaps tease that informatino  
> out of the "*.out" output file using a script, though.

Capital idea! That does indeed do the trick. It's straightforward to 
extract the terminals that can be shifted in each state from the .out 
file and thus generate a mapping from states to follow sets.

Afterwards, the current state can be queried from %syntax_error as 
yypParser->yystack[yypParser->yyidx].stateno (not part of the public 
interface of course, but it works), and thus a nice error message can be 
generated from the previously created collection of follow sets.

Thank you very much,
-- Markus Thiele
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon: Functionality like Bison's %error-verbose / Access to last follow set from %syntax_error

[D. Richard Hipp]

On Aug 17, 2008, at 1:48 PM, Markus Thiele wrote:

> Greetings,
>
> I've been using Lemon for a small custom compiler project. I've used
> Bison before, and I very much prefer the way Lemon does things,  
> there's
> just one feature I'm missing and haven't been able to find.
>
> Bison generates a human readable error message on syntax errors, and
> when setting %error-verbose this message contains some nice extra
> information, e.g. "Unexpected token A, expected C or D".
>
> As far as I can see, Lemon does not generate human readable messages  
> at
> all (except when ParseTrace is set, but then only to a stream and
> non-selectively).
>
> Now obviously I can generate the "Unexpected token A." part from the
> last token read, but for the rest I'd need access to the last follow  
> set
> before the error. So I've been wondering if there's any halfway nice  
> and
> clean solution for this, or if I'm maybe just missing an existing  
> feature.


Lemon does not have any feature that will provide the application with  
access to the follow-set.  You could perhaps tease that informatino  
out of the "*.out" output file using a script, though.

D. Richard Hipp
[EMAIL PROTECTED]



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon: Functionality like Bison's %error-verbose / Access to last follow set from %syntax_error

[Markus Thiele]

Greetings,

I've been using Lemon for a small custom compiler project. I've used 
Bison before, and I very much prefer the way Lemon does things, there's 
just one feature I'm missing and haven't been able to find.

Bison generates a human readable error message on syntax errors, and 
when setting %error-verbose this message contains some nice extra 
information, e.g. "Unexpected token A, expected C or D".

As far as I can see, Lemon does not generate human readable messages at 
all (except when ParseTrace is set, but then only to a stream and 
non-selectively).

Now obviously I can generate the "Unexpected token A." part from the 
last token read, but for the rest I'd need access to the last follow set 
before the error. So I've been wondering if there's any halfway nice and 
clean solution for this, or if I'm maybe just missing an existing feature.

Thank you,
-- Markus Thiele
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon

[arjunkumar keerti]

Hi Arjen,

Thanks for the mail...I found it very helpful and i started to execute
it..I succeeded in doing it...

However i started writing a sample program for converting "Infix to
Prefix"...I am struck at this point ...so can u help me in
writing this program...

Thanks in advance

Regards,
Arjun

On Sat, Jun 28, 2008 at 12:54 AM, Arjen Markus <[EMAIL PROTECTED]>
wrote:

> > Hi,
> > Can u send me any documentation on LEMON that u have worked out.
> > I have some queries also.
> > 1.Does LEMON work on Windows environment?
> > 2.I tried but it is not.I heard that it works on Linux environment.I am
> > trying to know why not on windows...can u give me some info about it
> >
> > I am reading the material u have suggested me but it did not have any
> > information regarding to my queries.
> >
> > Can u mail me if u have made any documentation on LEMON.
> >
>
> Well, basically lemon works as follows:
> 1. It reads the definition of the parser from the input file
>   you give it as the first argument. From that it produces
>   a set of tables that implement the parser. This information
>   gets filled in in a template.
> 2. The result is a C file that you can use to parse (interpret)
>   your input.
> 3. What it does not do is split up the input into tokens. You
>   have to do that yourself.
>
> I attach a small - not very efficient - example that shows a
> few features of lemon.
>
> As for your questions:
> There is no reason it would not work on Windows: it is a
> perfectly straghtforward C program, you get the source code
> and compile it. One thing though: the template file must be
> in the directory where you run the program, where the
> executable lives or in the PATH.
>
> (I am using it myself on Windows :). I am adapting it so that
> it will produce Fortran code instead of C code. But that is
> off topic.)
>
> Here is my sample parser definition:
>
> // expr.y
> // Simple parser:
> // expr = vaue + value + value + ...
>
> %extra_argument {State *state}
> %token_type {Token*}
> %type term {int}
>
> expr ::= firstterm plusterms . {
>printf( "Result: %d\n", state->sum );
> }
>
> firstterm ::= term(T) . {
>state->sum = T;
>printf("First term: %d\n", T );
> }
>
> term(V) ::= NUMBER(N) . {
>sscanf( N->token, "%d",  );
>printf("Term: %d -- %s\n", V, N->token );
> }
>
> plusterms ::= .
> plusterms ::= plusterms plusterm .
>
> plusterm ::= PLUS term(T) . {
>state->sum = state->sum + T;
>printf( "Result so far: %d\n", state->sum );
> }
>
> --
> And here is the main program that includes the
> resulting C code:
>
> #include 
> #include 
> #include 
>
> typedef struct {
>char *token;
> } Token;
> typedef struct {
>int sum;
> } State;
>
> #include "expr.h"
> #include "expr.c"
>
> int main( int argc, char *argv[] ) {
>Token token[10];
>State state;
>
>void *pParser = (void *) ParseAlloc( malloc );
>
>ParseTrace( stdout, ">>" );
>
>token[0].token = "1"; Parse( pParser, NUMBER, [0],  );
>token[1].token = "+"; Parse( pParser, PLUS,   [1],  );
>token[2].token = "2"; Parse( pParser, NUMBER, [2],  );
>token[3].token = "+"; Parse( pParser, PLUS,   [3],  );
>token[4].token = "3"; Parse( pParser, NUMBER, [4],  );
>
>Parse( pParser, 0, [0],  );
>
>ParseFree( pParser, free );
>
>return 0;
> }
>
> -
> One thing to note:
> As the tokens are not always treated the moment they are passed
> to the parser - this depends on the grammar and the moment in the
> parsing process, you need to make sure that the tokens remain
> available. I have done that in this silly program by using an
> array of them. But a smarter program would use lemon's
> abilities to destroy the tokens when ready.
>
> Regards,
>
> Arjen
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon

[Arjen Markus]

> Hi,
> Can u send me any documentation on LEMON that u have worked out.
> I have some queries also.
> 1.Does LEMON work on Windows environment?
> 2.I tried but it is not.I heard that it works on Linux environment.I am
> trying to know why not on windows...can u give me some info about it
>
> I am reading the material u have suggested me but it did not have any
> information regarding to my queries.
>
> Can u mail me if u have made any documentation on LEMON.
>

Well, basically lemon works as follows:
1. It reads the definition of the parser from the input file
   you give it as the first argument. From that it produces
   a set of tables that implement the parser. This information
   gets filled in in a template.
2. The result is a C file that you can use to parse (interpret)
   your input.
3. What it does not do is split up the input into tokens. You
   have to do that yourself.

I attach a small - not very efficient - example that shows a
few features of lemon.

As for your questions:
There is no reason it would not work on Windows: it is a
perfectly straghtforward C program, you get the source code
and compile it. One thing though: the template file must be
in the directory where you run the program, where the
executable lives or in the PATH.

(I am using it myself on Windows :). I am adapting it so that
it will produce Fortran code instead of C code. But that is
off topic.)

Here is my sample parser definition:

// expr.y
// Simple parser:
// expr = vaue + value + value + ...

%extra_argument {State *state}
%token_type {Token*}
%type term {int}

expr ::= firstterm plusterms . {
printf( "Result: %d\n", state->sum );
}

firstterm ::= term(T) . {
state->sum = T;
printf("First term: %d\n", T );
}

term(V) ::= NUMBER(N) . {
sscanf( N->token, "%d",  );
printf("Term: %d -- %s\n", V, N->token );
}

plusterms ::= .
plusterms ::= plusterms plusterm .

plusterm ::= PLUS term(T) . {
state->sum = state->sum + T;
printf( "Result so far: %d\n", state->sum );
}

--
And here is the main program that includes the
resulting C code:

#include 
#include 
#include 

typedef struct {
char *token;
} Token;
typedef struct {
int sum;
} State;

#include "expr.h"
#include "expr.c"

int main( int argc, char *argv[] ) {
Token token[10];
State state;

void *pParser = (void *) ParseAlloc( malloc );

ParseTrace( stdout, ">>" );

token[0].token = "1"; Parse( pParser, NUMBER, [0],  );
token[1].token = "+"; Parse( pParser, PLUS,   [1],  );
token[2].token = "2"; Parse( pParser, NUMBER, [2],  );
token[3].token = "+"; Parse( pParser, PLUS,   [3],  );
token[4].token = "3"; Parse( pParser, NUMBER, [4],  );

Parse( pParser, 0, [0],  );

ParseFree( pParser, free );

return 0;
}

-
One thing to note:
As the tokens are not always treated the moment they are passed
to the parser - this depends on the grammar and the moment in the
parsing process, you need to make sure that the tokens remain
available. I have done that in this silly program by using an
array of them. But a smarter program would use lemon's
abilities to destroy the tokens when ready.

Regards,

Arjen

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon

[Arjen Markus]

arjunkumar keerti wrote:

>Hi,
>I found in wikipedia that Lemon parser is a part of SQLite project but i
>couldn't found any sort of information regarding to LEMON.
>Can u give me any documentation regarding how to install it and how to work
>for some programs on Lemon parser generator or any URL's that might be
>helpful and can download the information regarding LEMON.
>
>  
>
You can find information on lemon at: http://www.hwaci.com/sw/lemon/

(And having gained some limited experience with it myself, this last 
week, I can
say it works quite nicely, with one or two caveats, perhaps :))

Regards,

Arjen
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon

[arjunkumar keerti]

Hi,
I found in wikipedia that Lemon parser is a part of SQLite project but i
couldn't found any sort of information regarding to LEMON.
Can u give me any documentation regarding how to install it and how to work
for some programs on Lemon parser generator or any URL's that might be
helpful and can download the information regarding LEMON.

Thanks
Arjun
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon: Conflicts with repeated TERMINALS

[Ralf Junker]

Joe Wilson <[EMAIL PROTECTED]> wrote:

>The following grammar may be clearer to you:

Yes, it is many thanks! I believe I am making progress! At least I can see the 
picture much clearer now and was able to come up with the following grammar 
with just one conflict unsolved:

  %left NEWLINE.   /* Do these matter here at all? */
  %nonassoc TEXT LINK.
  %left HEADING_START.
  %left HEADING_END.

  article ::= blocks.

  blocks ::= block. /* EOF */
  blocks ::= blocks NEWLINE./* EOF */
  blocks ::= blocks NEWLINE NEWLINE block.

  block ::= .   /* EOF */
  block ::= paragraph.
  block ::= heading.

  heading ::= HEADING_START text HEADING_END.

  paragraph ::= line.
  paragraph ::= paragraph NEWLINE line.

  line ::= text.

  text ::= textpiece.
  text ::= text textpiece.

  textpiece ::= TEXT.
  textpiece ::= LINK.

I of course appreciate any comments ;-) My idea is that

* A block can be either a paragraph or a heading. Multiple blocks are separated 
by two NEWLINEs.

* A paragraph is made up of n >= 1 lines. Each line within a paragraph ends 
with a single NEWLINE. Two NEWLINEs start a new block (see above).

* A line consists of text, which can be TEXT or LINK.

Not all works well with the grammer, and unfortunately I do not understand why. 
Given this input, for example:

  TEXT, NEWLINE

the parser gets stuck at

  paragraph ::= paragraph NEWLINE line.

instead of falling back to the line above
  
  paragraph ::= line.

to find the conditions of a paragraph fulfilled. Why does it not try the other 
alternatives? Or are there none in the grammar?

>Try reading some papers on parsing or search for the book
>"Compilers: Principles, Techniques, and Tools" (a.k.a. 
>the dragon book).

I certainly will.

>Also try writing on paper random sequences of tokens and 
>manually parse your grammar to see the conflicts firsthand.

As I throw different token sequences to my experimental parser I am slowly 
starting to make sense of the debugger output.

Ralf 


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon: Conflicts with repeated TERMINALS

[Joe Wilson]

--- Ralf Junker <[EMAIL PROTECTED]> wrote:
> >  paragraph ::= PARA text.
> 
> I observed the new PARA terminal token (the clear separator!?). Unfortunately 
> the lexer does not
> generate such a token. Paragraph repeats are also removed.

It was just an HTML-like example. I just wanted to demonstrate one
possible way to remove the conflicts by adding a special tag. 
I'm not suggesting that you alter your grammar in this way.

> >Here's another:
> >
> >  article ::= blocks.
> >
> >  blocks ::= block.
> >  blocks ::= blocks block.
> >
> >  block ::= heading NEWLINE.
> >  block ::= paragraph NEWLINE.
> >
> >  heading ::= HEADING_START text HEADING_END.
> >  heading ::= HEADING_START text.
> >  heading ::= HEADING_START.
> >
> >  paragraph ::= text.
> >
> >  text ::= textpiece.
> >  text ::= text textpiece.
> >
> >  textpiece ::= TEXT.
> >  textpiece ::= LINK.
> 
> This one also removes paragraph repeats, doesn't it? Unfortunately paragraphs 
> need to repeat for
> my grammar. Is there a way to achieve this without conflicts?

In your original grammar, you could have random sequences of TEXT 
and LINK and NEWLINE tokens without any way of differentiating whether 
they were part of a "text" or "paragraph" or "heading", hence the conflict.
So I figured that a paragraph may as well be any combination of
TEXT and LINK tokens ending with NEWLINE. The headings in my 2nd grammar
also have to end with NEWLINE. "paragraph" will not repeat, per se, but you 
can repeat "block" (see the "blocks" rules), where you can have several 
consecutive "block"s that happen to be of type paragraph so you can achieve 
the same effect. 

The following grammar may be clearer to you:

  article ::= blocks.

  blocks ::= block.
  blocks ::= blocks block.

  block ::= heading.
  block ::= paragraph.

  heading ::= HEADING_START text HEADING_END.
  heading ::= HEADING_START text NEWLINE.
  heading ::= HEADING_START NEWLINE.

  paragraph ::= NEWLINE.
  paragraph ::= text NEWLINE.

  text ::= textpiece.
  text ::= text textpiece.

  textpiece ::= TEXT.
  textpiece ::= LINK.

It's much the same as the previous grammar, but puts the NEWLINEs as part 
of the paragraph and heading rules instead of in the block rule.
The difference being that heading no longer needs to end with NEWLINE 
in one case if HEADING_END is encountered, as it is not ambiguous:

  heading ::= HEADING_START text HEADING_END.

and a paragraph in this grammar may also be empty so you can parse 
consecutive NEWLINE tokens with this rule:

  paragraph ::= NEWLINE.

Again, this was just an example of how to disambiguate the grammar. 
You can find other ways.

> >Lemon generates an .out file for the .y file processed.
> >You can examine it for errors.
> 
> I have tried to make sense of the .out file before. It tells me where to look 
> for the problem,
> but not how to fix it ...

Try reading some papers on parsing or search for the book
"Compilers: Principles, Techniques, and Tools" (a.k.a. 
the dragon book).

Also try writing on paper random sequences of tokens and 
manually parse your grammar to see the conflicts firsthand.



  

Never miss a thing.  Make Yahoo your home page. 
http://www.yahoo.com/r/hs

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon: Conflicts with repeated TERMINALS

[Ralf Junker]

Many thanks, Joe,

>Your grammar is ambiguous. The text tokens run together for 
>various rules because the grammar lacks clear separators between 
>them. 

OK, I begin to understand. The "clear separators" need to be TERMINALs, right? 
I believed that these were imlicit because there are TEXT and LINK after all 
text tokens are fully expanded. Therefore I thought that the grammar would not 
be ambiguous.

>You can fix it a million ways by altering your grammar.

Thanks for the suggestions - I can see that they do not generate conflicts, but 
they certainly alter the grammar.

>Here is one way:
>
>  article ::= blocks.
>
>  blocks ::= block.
>  blocks ::= blocks block.
>
>  block ::= heading.
>  block ::= paragraph.
>
>  heading ::= HEADING_START text HEADING_END.
>  heading ::= HEADING_START text.
>  heading ::= HEADING_START.
>
>  paragraph ::= PARA text.
>
>  text ::= textpiece.
>  text ::= text textpiece.
>
>  textpiece ::= TEXT.
>  textpiece ::= LINK.

I observed the new PARA terminal token (the clear separator!?). Unfortunately 
the lexer does not generate such a token. Paragraph repeats are also removed.

>Here's another:
>
>  article ::= blocks.
>
>  blocks ::= block.
>  blocks ::= blocks block.
>
>  block ::= heading NEWLINE.
>  block ::= paragraph NEWLINE.
>
>  heading ::= HEADING_START text HEADING_END.
>  heading ::= HEADING_START text.
>  heading ::= HEADING_START.
>
>  paragraph ::= text.
>
>  text ::= textpiece.
>  text ::= text textpiece.
>
>  textpiece ::= TEXT.
>  textpiece ::= LINK.

This one also removes paragraph repeats, doesn't it? Unfortunately paragraphs 
need to repeat for my grammar. Is there a way to achieve this without conflicts?

>Lemon generates an .out file for the .y file processed.
>You can examine it for errors.

I have tried to make sense of the .out file before. It tells me where to look 
for the problem, but not how to fix it ...

I am sorry to appear stupid, but I still can not make sense of it all. Can 
someone still help, please?

Ralf 


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon: Conflicts with repeated TERMINALS

[Joe Wilson]

--- Ralf Junker <[EMAIL PROTECTED]> wrote:
> article ::= blocks.
> 
> blocks ::= block.
> blocks ::= blocks block.
> 
> block ::= heading.
> block ::= paragraph.
> 
> heading ::= HEADING_START text HEADING_END.
> heading ::= HEADING_START text.
> heading ::= HEADING_START.
> 
> paragraph ::= text NEWLINE.
> paragraph ::= paragraph text NEWLINE.
> paragraph ::= text.
> paragraph ::= paragraph text.
> 
> text ::= textpiece.
> text ::= text textpiece.
> 
> textpiece ::= TEXT.
> textpiece ::= LINK.

Your grammar is ambiguous. The text tokens run together for 
various rules because the grammar lacks clear separators between 
them. You can fix it a million ways by altering your grammar.

Here is one way:

  article ::= blocks.

  blocks ::= block.
  blocks ::= blocks block.

  block ::= heading.
  block ::= paragraph.

  heading ::= HEADING_START text HEADING_END.
  heading ::= HEADING_START text.
  heading ::= HEADING_START.

  paragraph ::= PARA text.

  text ::= textpiece.
  text ::= text textpiece.

  textpiece ::= TEXT.
  textpiece ::= LINK.

Here's another:

  article ::= blocks.

  blocks ::= block.
  blocks ::= blocks block.

  block ::= heading NEWLINE.
  block ::= paragraph NEWLINE.

  heading ::= HEADING_START text HEADING_END.
  heading ::= HEADING_START text.
  heading ::= HEADING_START.

  paragraph ::= text.

  text ::= textpiece.
  text ::= text textpiece.

  textpiece ::= TEXT.
  textpiece ::= LINK.

Lemon generates an .out file for the .y file processed.
You can examine it for errors.



  

Be a better sports nut!  Let your teams follow you 
with Yahoo Mobile. Try it now.  
http://mobile.yahoo.com/sports;_ylt=At9_qDKvtAbMuh1G1SQtBI7ntAcJ

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

[sqlite] Lemon: Conflicts with repeated TERMINALS

[Ralf Junker]

I am trying to write a Wiki parser with Lemon. The Lemon features suite my 
needs perfectly, but I am unfortunately stuck with the problem of parsing 
conflicts.

All conflicts seem caused by repeat constructs like this:

  text ::= textpiece.
  text ::= text textpiece.

The complete grammar follows below and results in 10 conflicts.

I have read the manual, looked at tutorials, and searched the mailing list, but 
nothing helped me to reduce the number of conflicts. Changing token order even 
tends cause more of them.

Reading similar grammars for Bison makes me wonder why Bison apparently has no 
problems with them but Lemon does. Am I doing something wrong or is this simply 
not possible with Lemon?

Ralf

---

article ::= blocks.

blocks ::= block.
blocks ::= blocks block.

block ::= heading.
block ::= paragraph.

heading ::= HEADING_START text HEADING_END.
heading ::= HEADING_START text.
heading ::= HEADING_START.

paragraph ::= text NEWLINE.
paragraph ::= paragraph text NEWLINE.
paragraph ::= text.
paragraph ::= paragraph text.

text ::= textpiece.
text ::= text textpiece.

textpiece ::= TEXT.
textpiece ::= LINK.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon problem

[Téragone]

Hello,

I finally find the bug. In fact that was many bugs.

- Grammar rule order.
- Input string not reseted properly.

Thank you.

David
On Nov 18, 2007 10:30 AM, Joe Wilson <[EMAIL PROTECTED]> wrote:

> It's most likely your bug. Just add some debug prints in your grammar
> and tokenizer to see what's going on.
> See also: ParseTrace() in http://www.hwaci.com/sw/lemon/lemon.html
>
> --- Téragone <[EMAIL PROTECTED]> wrote:
> > I have a problem with a rule of a small calculator which accept
> variables :
> >
> > assignment(A) ::= VARIABLE(C) EQUAL expr(B).
> >
> > I can set variable with simple value or expression like :
> > a=2
> > b=2+3
> >
> > but when I try :
> > c=a+b
> >
> > The result is put in variable b instead of c.
> >
> > Is it my bug or a Lemon bug ?
>
>
>
>
>  
> 
> Be a better pen pal.
> Text or chat with friends inside Yahoo! Mail. See how.
> http://overview.mail.yahoo.com/
>
>
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
>
> -
>
>

Re: [sqlite] Lemon problem

[Joe Wilson]

It's most likely your bug. Just add some debug prints in your grammar 
and tokenizer to see what's going on.
See also: ParseTrace() in http://www.hwaci.com/sw/lemon/lemon.html

--- Téragone <[EMAIL PROTECTED]> wrote:
> I have a problem with a rule of a small calculator which accept variables :
> 
> assignment(A) ::= VARIABLE(C) EQUAL expr(B).
> 
> I can set variable with simple value or expression like :
> a=2
> b=2+3
> 
> but when I try :
> c=a+b
> 
> The result is put in variable b instead of c.
> 
> Is it my bug or a Lemon bug ?



  

Be a better pen pal. 
Text or chat with friends inside Yahoo! Mail. See how.  
http://overview.mail.yahoo.com/

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

[sqlite] Lemon problem

[Téragone]

Hello,

I have a problem with a rule of a small calculator which accept variables :

assignment(A) ::= VARIABLE(C) EQUAL expr(B).

I can set variable with simple value or expression like :
a=2
b=2+3

but when I try :
c=a+b

The result is put in variable b instead of c.

Is it my bug or a Lemon bug ?

Thank you,
David

Re: [sqlite] Lemon not reducing

[Gaspard Bucher]

For those reading this thread, I could solve my problme by using ragel
(http://www.cs.queensu.ca/~thurston/ragel/). You can define leaving
actions but also 'any change' actions. It was also easier to include
in a C++ project then lemon/flex. If my grammar becomes more
complicated, I heard it is possible to make a good collaboration
between ragel and lemon (ragel is then used as a tokenizer).

2007/10/25, Gaspard Bucher <[EMAIL PROTECTED]>:
> >  I do not understand why lemon waits for one more token when it has 
> >  enough information to reduce
> ...
> > 
> > >>> I don't think you can.  Why do you want to?  Why not just go
> > >>> ahead and send it the next token?
> > >>>
> > >> Most people find a way around this problem using white-space. This
> > >> could be a solution but then my grammar will be filled with
> > >> "white-space | nothing" rules and I thought Lemon could reduce when
> > >> there is no other way out of the current stack as it is more elegant.
> > >>
> > >>
> > LA(LR) is the answer - just drop Your's tokens as their arrive and give
> > a chance for the parser to be LALR, not LR or SLR :)
> >
> > mak
> >
> Lookahead improves what we can do with a grammar, but when there is no
> ambiguity, it should resolve without waiting for the next token. If
> your grammar is so simple it's an SLR, why not treat it so and ease
> the creation of grammars without needing to add "separator" tokens
> just so there is a lookahead (most languages use ";" or white space
> for that purpose).
>
> My grammar works with all "whitespace or nothing" rules, but I have to
> be careful to avoid conflicts. When I write it without the whitespace
> stuff (eaten in the tokenizer), I have no conflict. Simple example:
> main ::= commands.
> commands ::= .
> commands ::= commands ws command.
> command  ::= variable EQUAL value.
> variable ::= ws IDENTIFIER ws.
> value::= ws NUMBER ws.
> ws   ::= .
> ws   ::= WHITE.
> > 2 parsing conflicts.
>
> Whithout whitespace stuff:
> main ::= commands.
> commands ::= .
> commands ::= commands command.
> command  ::= variable EQUAL value.
> variable ::= IDENTIFIER.
> value::= NUMBER.
> > no conflicts, easier and cleaner.
>
> I know how to fix the first grammar, but you have to think of the
> actual succession to avoid two "ws" from coming next to the other,
> this thinking is not really related to the grammar from my point of
> view.
>
> Gaspard
>

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon not reducing

[Gaspard Bucher]

>  I do not understand why lemon waits for one more token when it has 
>  enough information to reduce
...
> 
> >>> I don't think you can.  Why do you want to?  Why not just go
> >>> ahead and send it the next token?
> >>>
> >> Most people find a way around this problem using white-space. This
> >> could be a solution but then my grammar will be filled with
> >> "white-space | nothing" rules and I thought Lemon could reduce when
> >> there is no other way out of the current stack as it is more elegant.
> >>
> >>
> LA(LR) is the answer - just drop Your's tokens as their arrive and give
> a chance for the parser to be LALR, not LR or SLR :)
>
> mak
>
Lookahead improves what we can do with a grammar, but when there is no
ambiguity, it should resolve without waiting for the next token. If
your grammar is so simple it's an SLR, why not treat it so and ease
the creation of grammars without needing to add "separator" tokens
just so there is a lookahead (most languages use ";" or white space
for that purpose).

My grammar works with all "whitespace or nothing" rules, but I have to
be careful to avoid conflicts. When I write it without the whitespace
stuff (eaten in the tokenizer), I have no conflict. Simple example:
main ::= commands.
commands ::= .
commands ::= commands ws command.
command  ::= variable EQUAL value.
variable ::= ws IDENTIFIER ws.
value::= ws NUMBER ws.
ws   ::= .
ws   ::= WHITE.
> 2 parsing conflicts.

Whithout whitespace stuff:
main ::= commands.
commands ::= .
commands ::= commands command.
command  ::= variable EQUAL value.
variable ::= IDENTIFIER.
value::= NUMBER.
> no conflicts, easier and cleaner.

I know how to fix the first grammar, but you have to think of the
actual succession to avoid two "ws" from coming next to the other,
this thinking is not really related to the grammar from my point of
view.

Gaspard

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon not reducing

[Grzegorz Makarewicz]

Gaspard Bucher wrote:
> PS: There is another reason, aside from aesthetics and simpler grammar
> to filter white spaces inside the tokenizer: you avoid all the parser
> conflicts you could get with "empty | or space" rules.
>
> 2007/10/24, Gaspard Bucher <[EMAIL PROTECTED]>:
>   
>>> Gaspard Bucher <[EMAIL PROTECTED]> wrote:
>>>   
 I do not understand why lemon waits for one more token when it has
 enough information to reduce.

 I want to recognize :
 foo = Bar()
 when the token CLOSE_PAR is received, not when an extra token is parsed..

 How can I avoid lemon waiting for the extra token before reducing ?

 
>>> I don't think you can.  Why do you want to?  Why not just go
>>> ahead and send it the next token?
>>>   
>> Most people find a way around this problem using white-space. This
>> could be a solution but then my grammar will be filled with
>> "white-space | nothing" rules and I thought Lemon could reduce when
>> there is no other way out of the current stack as it is more elegant.
>> I went into the sources and saw this comment:
>>
>> 
LA(LR) is the answer - just drop Your's tokens as their arrive and give
a chance for the parser to be LALR, not LR or SLR :)

mak


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon not reducing

[Gaspard Bucher]

PS: There is another reason, aside from aesthetics and simpler grammar
to filter white spaces inside the tokenizer: you avoid all the parser
conflicts you could get with "empty | or space" rules.

2007/10/24, Gaspard Bucher <[EMAIL PROTECTED]>:
> > Gaspard Bucher <[EMAIL PROTECTED]> wrote:
> > > I do not understand why lemon waits for one more token when it has
> > > enough information to reduce.
> > >
> > > I want to recognize :
> > > foo = Bar()
> > > when the token CLOSE_PAR is received, not when an extra token is parsed..
> > >
> > > How can I avoid lemon waiting for the extra token before reducing ?
> > >
> >
> > I don't think you can.  Why do you want to?  Why not just go
> > ahead and send it the next token?
> Most people find a way around this problem using white-space. This
> could be a solution but then my grammar will be filled with
> "white-space | nothing" rules and I thought Lemon could reduce when
> there is no other way out of the current stack as it is more elegant.
> I went into the sources and saw this comment:
>
> I think I might find something to do here:
> in yy_find_shift_action :
> when we find the action:
>
> // return yy_action[i];
> result = yy_action[i];
> printf("==> next : %i\n", result);
> printf("==> next_next: %i\n", yy_shift_ofst[result % (YYNSTATE+YYNRULE)]);
> // if the shift results in a default action do this action now.
> if (result < (YYNSTATE+YYNRULE) && yy_shift_ofst[result] ==
> YY_SHIFT_USE_DFLT) {
>   // do the change ourself
>
>   YYMINORTYPE yyminorunion;
>   yy_shift(pParser,result,iLookAhead,);
>   pParser->yyerrcnt--;
>   return yy_default[result];
> } else
>   return result;
>
> This works except when the reduction reaches state '1'. Somehow, it
> should jump directly to state '0', clearing the current lookahead.
>
> > --
> > D. Richard Hipp <[EMAIL PROTECTED]>
> >
> >
> > -
> > To unsubscribe, send email to [EMAIL PROTECTED]
> > -
> >
> >
> > !DSPAM:471f82d0320381804284693!
> >
> >
>

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon not reducing

[Gaspard Bucher]

> Gaspard Bucher <[EMAIL PROTECTED]> wrote:
> > I do not understand why lemon waits for one more token when it has
> > enough information to reduce.
> >
> > I want to recognize :
> > foo = Bar()
> > when the token CLOSE_PAR is received, not when an extra token is parsed..
> >
> > How can I avoid lemon waiting for the extra token before reducing ?
> >
>
> I don't think you can.  Why do you want to?  Why not just go
> ahead and send it the next token?
Most people find a way around this problem using white-space. This
could be a solution but then my grammar will be filled with
"white-space | nothing" rules and I thought Lemon could reduce when
there is no other way out of the current stack as it is more elegant.
I went into the sources and saw this comment:

I think I might find something to do here:
in yy_find_shift_action :
when we find the action:

// return yy_action[i];
result = yy_action[i];
printf("==> next : %i\n", result);
printf("==> next_next: %i\n", yy_shift_ofst[result % (YYNSTATE+YYNRULE)]);
// if the shift results in a default action do this action now.
if (result < (YYNSTATE+YYNRULE) && yy_shift_ofst[result] ==
YY_SHIFT_USE_DFLT) {
  // do the change ourself

  YYMINORTYPE yyminorunion;
  yy_shift(pParser,result,iLookAhead,);
  pParser->yyerrcnt--;
  return yy_default[result];
} else
  return result;

This works except when the reduction reaches state '1'. Somehow, it
should jump directly to state '0', clearing the current lookahead.

> --
> D. Richard Hipp <[EMAIL PROTECTED]>
>
>
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
>
>
> !DSPAM:471f82d0320381804284693!
>
>

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon not reducing

[drh]

Gaspard Bucher <[EMAIL PROTECTED]> wrote:
> I do not understand why lemon waits for one more token when it has
> enough information to reduce.
> 
> I want to recognize :
> foo = Bar()
> when the token CLOSE_PAR is received, not when an extra token is parsed.
> 
> How can I avoid lemon waiting for the extra token before reducing ?
> 

I don't think you can.  Why do you want to?  Why not just go
ahead and send it the next token?
--
D. Richard Hipp <[EMAIL PROTECTED]>


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

[sqlite] Lemon not reducing

[Gaspard Bucher]

I do not understand why lemon waits for one more token when it has
enough information to reduce.

I want to recognize :
foo = Bar()
when the token CLOSE_PAR is received, not when an extra token is parsed.

How can I avoid lemon waiting for the extra token before reducing ?

Thanks for your help !

Gaspard

=== GRAMMAR   ==
main::= commands.
commands::= . /* can be empty  */
commands::= WHITE_SPACE.
commands::= commands command. /* many commands */

command ::= variable EQUAL class parameters. { printf("[new  
object]\n"); }

variable::= IDENTIFIER.
class   ::= CONST_IDENTIFIER.
parameters  ::= OPEN_PAR CLOSE_PAR.

===  DEBUG OUTPUT ==

Parser started.
> foo = Bar()
>>Input IDENTIFIER
>>Reduce [commands ::=].
>>Shift 1
>>Stack: commands
>>Shift 21
>>Stack: commands IDENTIFIER
>>Input EQUAL
>>Reduce [variable ::= IDENTIFIER].
>>Shift 3
>>Stack: commands variable
>>Shift 4
>>Stack: commands variable EQUAL
>>Input CONST_IDENTIFIER
>>Shift 20
>>Stack: commands variable EQUAL CONST_IDENTIFIER
>>Input OPEN_PAR
>>Reduce [class ::= CONST_IDENTIFIER].
>>Shift 5
>>Stack: commands variable EQUAL class
>>Shift 7
>>Stack: commands variable EQUAL class OPEN_PAR
>>Input CLOSE_PAR
>>Shift 8
>>Stack: commands variable EQUAL class OPEN_PAR CLOSE_PAR

At this point, lemon has all the information needed, but it waits
before reducing...

---
http://rubyk.org, scriptable multimedia controller



-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon: Help on conflic resolution?

[Ralf Junker]

Richard,

this helped me greatly! I also derived from your example that I can use 
multiple characters without conflicts like this:

---

doc ::= inline_list.

// List of allowed characters. Add more as you like.

c ::= CHAR.
c ::= SPACE.

// The c character repeat.

chars ::= c.
chars ::= chars CHAR.

// Any sequence of just c and 'c' (c surrounded by apostrophes).

inline ::= c.
inline ::= APOS chars APOS.
 
// The inline repeat.
 
inline_list ::= inline.
inline_list ::= inline_list inline. 

-

Many thanks!

Ralf


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon: Help on conflic resolution?

[Ralf Junker]

Richard,

this helped me greatly! I also derived from your example that I can use 
multiple characters without conflicts like this:

---

doc ::= inline_list.

// List of allowed characters. Add more as you like.

c ::= CHAR.
c ::= SPACE.

// The c character repeat.

chars ::= c.
chars ::= chars CHAR.

// Any sequence of just c and 'c' (c surrounded by apostrophes).

inline ::= c.
inline ::= APOS chars APOS.
 
// The inline repeat.
 
inline_list ::= inline.
inline_list ::= inline_list inline. 

-

Many thanks!

Ralf


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon: Help on conflic resolution?

[drh]

Ralf Junker <[EMAIL PROTECTED]> wrote:
> I am writing to ask for help about how to solve The Lemon parser conflicts.
> 
> As part of a larger grammar, I am need to implement this regular expression 
> in Lemon:
> 
>   (.+|'.+')+
> 
> I tried lots of grammars, but all of them generated Lemon warnings. 
> 
> Maybe someone could have a look at the grammar below and let me know
> how the conflicts can be solved, and why they are generated in the 
> first place?
> 

doc ::= inline_list. 
 
// One ore more CHARs. 
 
chars ::= CHAR. 
chars ::= chars CHAR. 
 
// Any sequence of just CHARs and 'CHARs' (surrounded by apostrophes). 
 
inline ::= CHAR. 
inline ::= APOS chars APOS. 
 
// The repeat. This causes conflicts. Isn't it allowed? Workarounds? 
 
inline_list ::= inline.
inline_list ::= inline_list inline. 

--
D. Richard Hipp <[EMAIL PROTECTED]>


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

[sqlite] Lemon: Help on conflic resolution?

[Ralf Junker]

I am writing to ask for help about how to solve The Lemon parser conflicts.

As part of a larger grammar, I am need to implement this regular expression in 
Lemon:

  (.+|'.+')+

I tried lots of grammars, but all of them generated Lemon warnings. 

Maybe someone could have a look at the grammar below and let me know how the 
conflicts can be solved, and why they are generated in the first place?

Many thanks,

Ralf

--

%left CHAR. // Any character, except for apostrophe.
%left APOS. // Apostrophe only.

doc ::= inline.

// One ore more CHARs.

chars ::= CHAR.
chars ::= chars CHAR.

// Any sequence of just CHARs and 'CHARs' (surrounded by apostrophes).

inline ::= chars.
inline ::= APOS chars APOS.

// The repeat. This causes conflicts. Isn't it allowed? Workarounds?

inline ::= inline inline.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

[sqlite] Lemon Parser: Modular

[Uma Krishnan]

I had posted earlier to find if Lemon parser can be made modular. As per 
previous post responses, who had advised that I look in lemon parser forum. But 
I could not find a separate forum.  

  Please refer to the following link to get a better idea of what I'm trying to 
do:
   
  http://www.patentstorm.us/patents/7089541-claims.html

Thanks
   
  Uma

Re: [sqlite] Lemon Parser - Modular & Extensible ?

[Christian Smith]

Uma Krishnan uttered:

Hey, There's no need to be offensive. I did not mean to be critical. Far 
from it, it does a great a job (far more than I'm capable of producing). 
What I was trying to find out was, if it is possible for a .y files to 
be broken such that it can be built on top on other .y files.



Sorry if I came across as offensive. That was not the intention. I was 
just a little confused about the question.


I think lemon can only handle single input files. But you can can include 
C source into your output C file using the %include directive. Check out 
the documentation at, if you haven't already done so:

http://www.hwaci.com/sw/lemon/lemon.html

Now, what may draw some critical analysis is top posting and hijacking an 
existing thread for a new topic... [snip]





 Not sure if this is the right group. But could not find a lemon parser 
user group.



This is the best group to ask. While not tied to SQLite, it appears to be 
maintained as part of SQLite (but I may be wrong.)


Christian

--
/"\
\ /ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
 X   - AGAINST MS ATTACHMENTS
/ \

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon Parser - Modular & Extensible ?

[Uma Krishnan]

Hey, There's no need to be offensive. I did not mean to be critical. Far from 
it, it does a great a job (far more than I'm capable of producing). What I was 
trying to find out was, if it is possible for a .y files to be broken such that 
it can be built on top on other .y files. 
   
  Not sure if this is the right group. But could not find a lemon parser user 
group.
   
  

Christian Smith <[EMAIL PROTECTED]> wrote:
  Uma Krishnan uttered:

> Hello:
>
> Is lemon parser modular and extensible?


Extensible to do what? It generates parsers, and is self contained. It 
does a single job, and does it well. What more could you ask for?


>
> Thanks
>
> Uma
>
> Asif Lodhi wrote:
> Hi Everybody,
>
> I have just joined this mailing list as Sqlite looks like a good
> software solution to my needs. What I need right now is RE-assurance
> of "crash-recovery" that is mentioned on your front page. So, I would
> be thankful if you experts would give me an "accurate" and fair
> picture of the crash-recovery aspects of SQLite - without any hype.
>
> --
> Best regards,
>
> Asif
>
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
>
>
>

--
/"\
\ / ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
X - AGAINST MS ATTACHMENTS
/ \

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Lemon Parser - Modular & Extensible ?

[Christian Smith]

Uma Krishnan uttered:


Hello:

 Is lemon parser modular and extensible?



Extensible to do what? It generates parsers, and is self contained. It 
does a single job, and does it well. What more could you ask for?





 Thanks

 Uma

Asif Lodhi <[EMAIL PROTECTED]> wrote:
 Hi Everybody,

I have just joined this mailing list as Sqlite looks like a good
software solution to my needs. What I need right now is RE-assurance
of "crash-recovery" that is mentioned on your front page. So, I would
be thankful if you experts would give me an "accurate" and fair
picture of the crash-recovery aspects of SQLite - without any hype.

--
Best regards,

Asif

-
To unsubscribe, send email to [EMAIL PROTECTED]
-





--
/"\
\ /ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
 X   - AGAINST MS ATTACHMENTS
/ \

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

[sqlite] Lemon Parser - Modular & Extensible ?

[Uma Krishnan]

Hello:
   
  Is lemon parser modular and extensible? 
   
  Thanks
   
  Uma

Asif Lodhi <[EMAIL PROTECTED]> wrote:
  Hi Everybody,

I have just joined this mailing list as Sqlite looks like a good
software solution to my needs. What I need right now is RE-assurance
of "crash-recovery" that is mentioned on your front page. So, I would
be thankful if you experts would give me an "accurate" and fair
picture of the crash-recovery aspects of SQLite - without any hype.

--
Best regards,

Asif

-
To unsubscribe, send email to [EMAIL PROTECTED]
-