I have already referred to this change several times. Our current
scanner is very complex and has subtle interactions with our parser.
This is hard to maintain, and also prevents easy extensions of
our grammar (which defeats the whole point of having moved to
using bison for Bison).
I was about to install this patch when I read Paul's message about
2.3 being out soon?
Index: ChangeLog
from Akim Demaille <[EMAIL PROTECTED]>
Extract the parsing of user actions from the grammar scanner.
As a consequence, the relation between the grammar scanner and
parser is much simpler. We can also split "composite tokens" back
into simple tokens.
* src/gram.h (ITEM_NUMBER_MAX, RULE_NUMBER_MAX): New.
* src/scan-gram.l (add_column_width, adjust_location): Move to and
rename as...
* src/location.h, src/location.c (add_column_width)
(location_compute): these.
(location_print): Be robust to ending column being 0.
* src/location.h (boundary_set): New.
* src/main.c: Adjust to scanner_free being renamed as
gram_scanner_free.
* src/output.c: Include scan-code.h.
* src/parse-gram.y: Include scan-gram.h and scan-code.h.
Use boundary_set.
(PERCENT_DESTRUCTOR, PERCENT_PRINTER, PERCENT_INITIAL_ACTION)
(PERCENT_LEX_PARAM, PERCENT_PARSE_PARAM): Remove the {...} part,
which is now, again, a separate token.
Adjust all dependencies.
Whereever actions with $ and @ are used, use translate_code.
(action): Remove this nonterminal which is now useless.
* src/reader.c: Include assert.h, scan-gram.h and scan-code.h.
(grammar_current_rule_action_append): Use translate_code.
(packgram): Bound check ruleno, itemno, and rule_length.
* src/reader.h (gram_in, gram__flex_debug, scanner_cursor)
(last_string, last_braced_code_loc, max_left_semantic_context)
(scanner_initialize, scanner_free, scanner_last_string_free)
(gram_out, gram_lineno, YY_DECL_): Move to...
* src/scan-gram.h: this new file.
(YY_DECL): Rename as...
(GRAM_DECL): this.
* src/scan-code.h, src/scan-code.l, src/scan-code-c.c: New.
* src/scan-gram.l (gram_get_lineno, gram_get_in, gram_get_out):
(gram_get_leng, gram_get_text, gram_set_lineno, gram_set_in):
(gram_set_out, gram_get_debug, gram_set_debug, gram_lex_destroy):
Move these declarations, and...
(obstack_for_string, STRING_GROW, STRING_FINISH, STRING_FREE):
these to...
* src/flex-scanner.h: this new file.
* src/scan-gram.l (rule_length, rule_length_overflow)
(increment_rule_length): Remove.
(last_braced_code_loc): Rename as...
(gram_last_braced_code_loc): this.
Adjust to the changes of the parser.
Move all the handling of $ and @ into...
* src/scan-code.l: here.
* src/scan-gram.l (handle_dollar, handle_at): Remove.
(handle_action_dollar, handle_action_at): Move to...
* src/scan-code.l: here.
* src/Makefile.am (bison_SOURCES): Add flex-scanner.h,
scan-code.h, scan-code-c.c, scan-gram.h.
(EXTRA_bison_SOURCES): Add scan-code.l.
(BUILT_SOURCES): Add scan-code.c.
(yacc): Be robust to white spaces.
* tests/regression.at: Adjust the error message.
Index: src/Makefile.am
===================================================================
RCS file: /cvsroot/bison/bison/src/Makefile.am,v
retrieving revision 1.67
diff -u -r1.67 Makefile.am
--- src/Makefile.am 9 Dec 2005 23:51:25 -0000 1.67
+++ src/Makefile.am 29 May 2006 06:53:54 -0000
@@ -1,4 +1,4 @@
-## Copyright (C) 2001, 2002, 2003, 2004, 2005 Free Software Foundation, Inc.
+## Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation,
Inc.
## This program is free software; you can redistribute it and/or modify
## it under the terms of the GNU General Public License as published by
@@ -39,6 +39,7 @@
conflicts.c conflicts.h \
derives.c derives.h \
files.c files.h \
+ flex-scanner.h \
getargs.c getargs.h \
gram.c gram.h \
lalr.h lalr.c \
@@ -53,8 +54,9 @@
reader.c reader.h \
reduce.c reduce.h \
relation.c relation.h \
- scan-gram-c.c \
- scan-skel-c.c scan-skel.h \
+ scan-code.h scan-code-c.c \
+ scan-gram.h scan-gram-c.c \
+ scan-skel.h scan-skel-c.c \
state.c state.h \
symlist.c symlist.h \
symtab.c symtab.h \
@@ -64,15 +66,15 @@
vcg.c vcg.h \
vcg_defaults.h
-EXTRA_bison_SOURCES = scan-skel.l scan-gram.l
+EXTRA_bison_SOURCES = scan-code.l scan-skel.l scan-gram.l
-BUILT_SOURCES = scan-skel.c scan-gram.c parse-gram.c parse-gram.h
+BUILT_SOURCES = scan-code.c scan-skel.c scan-gram.c parse-gram.c parse-gram.h
MOSTLYCLEANFILES = yacc
yacc:
echo '#! /bin/sh' >$@
- echo 'exec $(bindir)/bison -y "$$@"' >>$@
+ echo "exec '$(bindir)/bison' -y \"[EMAIL PROTECTED]"" >>$@
chmod a+x $@
echo:
Index: src/flex-scanner.h
===================================================================
RCS file: src/flex-scanner.h
diff -N src/flex-scanner.h
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ src/flex-scanner.h 29 May 2006 06:53:54 -0000
@@ -0,0 +1,65 @@
+/* Common parts between scan-code.l and scan-gram.l.
+
+ Copyright (C) 2006 Free Software Foundation, Inc.
+
+ This file is part of Bison, the GNU Compiler Compiler.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ 02110-1301 USA
+*/
+
+#ifndef FLEX_PREFIX
+# error "FLEX_PREFIX not defined"
+#endif
+
+#include "system.h"
+
+/* Pacify "gcc -Wmissing-prototypes" when flex 2.5.31 is used. */
+int FLEX_PREFIX (get_lineno) (void);
+FILE *FLEX_PREFIX (get_in) (void);
+FILE *FLEX_PREFIX (get_out) (void);
+int FLEX_PREFIX (get_leng) (void);
+char *FLEX_PREFIX (get_text) (void);
+void FLEX_PREFIX (set_lineno) (int);
+void FLEX_PREFIX (set_in) (FILE *);
+void FLEX_PREFIX (set_out) (FILE *);
+int FLEX_PREFIX (get_debug) (void);
+void FLEX_PREFIX (set_debug) (int);
+int FLEX_PREFIX (lex_destroy) (void);
+
+#define last_string FLEX_PREFIX (last_string)
+
+/* OBSTACK_FOR_STRING -- Used to store all the characters that we need to
+ keep (to construct ID, STRINGS etc.). Use the following macros to
+ use it.
+
+ Use STRING_GROW to append what has just been matched, and
+ STRING_FINISH to end the string (it puts the ending 0).
+ STRING_FINISH also stores this string in LAST_STRING, which can be
+ used, and which is used by STRING_FREE to free the last string. */
+
+static struct obstack obstack_for_string;
+
+#define STRING_GROW \
+ obstack_grow (&obstack_for_string, yytext, yyleng)
+
+#define STRING_FINISH \
+ do { \
+ obstack_1grow (&obstack_for_string, '\0'); \
+ last_string = obstack_finish (&obstack_for_string); \
+ } while (0)
+
+#define STRING_FREE \
+ obstack_free (&obstack_for_string, FLEX_PREFIX (last_string))
Index: src/gram.h
===================================================================
RCS file: /cvsroot/bison/bison/src/gram.h,v
retrieving revision 1.58
diff -u -r1.58 gram.h
--- src/gram.h 9 Nov 2005 15:48:05 -0000 1.58
+++ src/gram.h 29 May 2006 06:53:54 -0000
@@ -1,6 +1,6 @@
/* Data definitions for internal representation of Bison's input.
- Copyright (C) 1984, 1986, 1989, 1992, 2001, 2002, 2003, 2004, 2005
+ Copyright (C) 1984, 1986, 1989, 1992, 2001, 2002, 2003, 2004, 2005, 2006
Free Software Foundation, Inc.
This file is part of Bison, the GNU Compiler Compiler.
@@ -115,6 +115,7 @@
extern int nvars;
typedef int item_number;
+#define ITEM_NUMBER_MAX INT_MAX
extern item_number *ritem;
extern unsigned int nritems;
@@ -146,6 +147,7 @@
/* Rule numbers. */
typedef int rule_number;
+#define RULE_NUMBER_MAX INT_MAX
extern rule_number nrules;
static inline item_number
Index: src/location.c
===================================================================
RCS file: /cvsroot/bison/bison/src/location.c,v
retrieving revision 1.6
diff -u -r1.6 location.c
--- src/location.c 9 Dec 2005 23:51:26 -0000 1.6
+++ src/location.c 29 May 2006 06:53:54 -0000
@@ -1,6 +1,5 @@
/* Locations for Bison
-
- Copyright (C) 2002, 2005 Free Software Foundation, Inc.
+ Copyright (C) 2002, 2005, 2006 Free Software Foundation, Inc.
This file is part of Bison, the GNU Compiler Compiler.
@@ -28,11 +27,80 @@
location const empty_location;
+/* If BUF is null, add BUFSIZE (which in this case must be less than
+ INT_MAX) to COLUMN; otherwise, add mbsnwidth (BUF, BUFSIZE, 0) to
+ COLUMN. If an overflow occurs, or might occur but is undetectable,
+ return INT_MAX. Assume COLUMN is nonnegative. */
+
+static inline int
+add_column_width (int column, char const *buf, size_t bufsize)
+{
+ size_t width;
+ unsigned int remaining_columns = INT_MAX - column;
+
+ if (buf)
+ {
+ if (INT_MAX / 2 <= bufsize)
+ return INT_MAX;
+ width = mbsnwidth (buf, bufsize, 0);
+ }
+ else
+ width = bufsize;
+
+ return width <= remaining_columns ? column + width : INT_MAX;
+}
+
+/* Set *LOC and adjust scanner cursor to account for token TOKEN of
+ size SIZE. */
+
+void
+location_compute (location *loc, boundary *cur, char const *token, size_t size)
+{
+ int line = cur->line;
+ int column = cur->column;
+ char const *p0 = token;
+ char const *p = token;
+ char const *lim = token + size;
+
+ loc->start = *cur;
+
+ for (p = token; p < lim; p++)
+ switch (*p)
+ {
+ case '\n':
+ line += line < INT_MAX;
+ column = 1;
+ p0 = p + 1;
+ break;
+
+ case '\t':
+ column = add_column_width (column, p0, p - p0);
+ column = add_column_width (column, NULL, 8 - ((column - 1) & 7));
+ p0 = p + 1;
+ break;
+
+ default:
+ break;
+ }
+
+ cur->line = line;
+ cur->column = column = add_column_width (column, p0, p - p0);
+
+ loc->end = *cur;
+
+ if (line == INT_MAX && loc->start.line != INT_MAX)
+ warn_at (*loc, _("line number overflow"));
+ if (column == INT_MAX && loc->start.column != INT_MAX)
+ warn_at (*loc, _("column number overflow"));
+}
+
+
/* Output to OUT the location LOC.
Warning: it uses quotearg's slot 3. */
void
location_print (FILE *out, location loc)
{
+ int end_col = 0 < loc.end.column ? loc.end.column - 1 : 0;
fprintf (out, "%s:%d.%d",
quotearg_n_style (3, escape_quoting_style, loc.start.file),
loc.start.line, loc.start.column);
@@ -40,9 +108,9 @@
if (loc.start.file != loc.end.file)
fprintf (out, "-%s:%d.%d",
quotearg_n_style (3, escape_quoting_style, loc.end.file),
- loc.end.line, loc.end.column - 1);
+ loc.end.line, end_col);
else if (loc.start.line < loc.end.line)
- fprintf (out, "-%d.%d", loc.end.line, loc.end.column - 1);
- else if (loc.start.column < loc.end.column - 1)
- fprintf (out, "-%d", loc.end.column - 1);
+ fprintf (out, "-%d.%d", loc.end.line, end_col);
+ else if (loc.start.column < end_col)
+ fprintf (out, "-%d", end_col);
}
Index: src/location.h
===================================================================
RCS file: /cvsroot/bison/bison/src/location.h,v
retrieving revision 1.13
diff -u -r1.13 location.h
--- src/location.h 9 Mar 2006 23:23:11 -0000 1.13
+++ src/location.h 29 May 2006 06:53:54 -0000
@@ -40,6 +40,15 @@
} boundary;
+/* Set the position of \a a. */
+static inline void
+boundary_set (boundary *b, const char *f, int l, int c)
+{
+ b->file = f;
+ b->line = l;
+ b->column = c;
+}
+
/* Return nonzero if A and B are equal boundaries. */
static inline bool
equal_boundaries (boundary a, boundary b)
@@ -64,6 +73,11 @@
extern location const empty_location;
+/* Set *LOC and adjust scanner cursor to account for token TOKEN of
+ size SIZE. */
+void location_compute (location *loc,
+ boundary *cur, char const *token, size_t size);
+
void location_print (FILE *out, location loc);
#endif /* ! defined LOCATION_H_ */
Index: src/main.c
===================================================================
RCS file: /cvsroot/bison/bison/src/main.c,v
retrieving revision 1.85
diff -u -r1.85 main.c
--- src/main.c 9 Dec 2005 23:51:26 -0000 1.85
+++ src/main.c 29 May 2006 06:53:54 -0000
@@ -1,6 +1,7 @@
/* Top level entry point of Bison.
- Copyright (C) 1984, 1986, 1989, 1992, 1995, 2000, 2001, 2002, 2004, 2005
+ Copyright (C) 1984, 1986, 1989, 1992, 1995, 2000, 2001, 2002, 2004, 2005,
+ 2006
Free Software Foundation, Inc.
This file is part of Bison, the GNU Compiler Compiler.
@@ -169,7 +170,7 @@
/* The scanner memory cannot be released right after parsing, as it
contains things such as user actions, prologue, epilogue etc. */
- scanner_free ();
+ gram_scanner_free ();
muscle_free ();
uniqstrs_free ();
timevar_pop (TV_FREE);
Index: src/output.c
===================================================================
RCS file: /cvsroot/bison/bison/src/output.c,v
retrieving revision 1.247
diff -u -r1.247 output.c
--- src/output.c 14 May 2006 20:40:35 -0000 1.247
+++ src/output.c 29 May 2006 06:53:54 -0000
@@ -36,6 +36,7 @@
#include "muscle_tab.h"
#include "output.h"
#include "reader.h"
+#include "scan-code.h" /* max_left_semantic_context */
#include "scan-skel.h"
#include "symtab.h"
#include "tables.h"
Index: src/parse-gram.y
===================================================================
RCS file: /cvsroot/bison/bison/src/parse-gram.y,v
retrieving revision 1.74
diff -u -r1.74 parse-gram.y
--- src/parse-gram.y 14 May 2006 19:14:10 -0000 1.74
+++ src/parse-gram.y 29 May 2006 06:53:55 -0000
@@ -32,6 +32,8 @@
#include "quotearg.h"
#include "reader.h"
#include "symlist.h"
+#include "scan-gram.h"
+#include "scan-code.h"
#include "strverscmp.h"
#define YYLLOC_DEFAULT(Current, Rhs, N) (Current) = lloc_default (Rhs, N)
@@ -84,9 +86,8 @@
{
/* Bison's grammar can initial empty locations, hence a default
location is needed. */
- @$.start.file = @$.end.file = current_file;
- @$.start.line = @$.end.line = 1;
- @$.start.column = @$.end.column = 0;
+ boundary_set (&@$.start, current_file, 1, 0);
+ boundary_set (&@$.end, current_file, 1, 0);
}
/* Only NUMBERS have a value. */
@@ -109,8 +110,8 @@
%token PERCENT_NTERM "%nterm"
%token PERCENT_TYPE "%type"
-%token PERCENT_DESTRUCTOR "%destructor {...}"
-%token PERCENT_PRINTER "%printer {...}"
+%token PERCENT_DESTRUCTOR "%destructor"
+%token PERCENT_PRINTER "%printer"
%token PERCENT_UNION "%union {...}"
@@ -137,8 +138,8 @@
PERCENT_EXPECT_RR "%expect-rr"
PERCENT_FILE_PREFIX "%file-prefix"
PERCENT_GLR_PARSER "%glr-parser"
- PERCENT_INITIAL_ACTION "%initial-action {...}"
- PERCENT_LEX_PARAM "%lex-param {...}"
+ PERCENT_INITIAL_ACTION "%initial-action"
+ PERCENT_LEX_PARAM "%lex-param"
PERCENT_LOCATIONS "%locations"
PERCENT_NAME_PREFIX "%name-prefix"
PERCENT_NO_DEFAULT_PREC "%no-default-prec"
@@ -146,7 +147,7 @@
PERCENT_NONDETERMINISTIC_PARSER
"%nondeterministic-parser"
PERCENT_OUTPUT "%output"
- PERCENT_PARSE_PARAM "%parse-param {...}"
+ PERCENT_PARSE_PARAM "%parse-param"
PERCENT_PURE_PARSER "%pure-parser"
PERCENT_REQUIRE "%require"
PERCENT_SKELETON "%skeleton"
@@ -167,23 +168,14 @@
%token EPILOGUE "epilogue"
%token BRACED_CODE "{...}"
-
%type <chars> STRING string_content
- "%destructor {...}"
- "%initial-action {...}"
- "%lex-param {...}"
- "%parse-param {...}"
- "%printer {...}"
+ "{...}"
"%union {...}"
PROLOGUE EPILOGUE
%printer { fprintf (stderr, "\"%s\"", $$); }
STRING string_content
%printer { fprintf (stderr, "{\n%s\n}", $$); }
- "%destructor {...}"
- "%initial-action {...}"
- "%lex-param {...}"
- "%parse-param {...}"
- "%printer {...}"
+ "{...}"
"%union {...}"
PROLOGUE EPILOGUE
%type <uniqstr> TYPE
@@ -214,7 +206,8 @@
declaration:
grammar_declaration
-| PROLOGUE { prologue_augment ($1, @1); }
+| PROLOGUE { prologue_augment (translate_code ($1, @1),
+ @1); }
| "%debug" { debug_flag = true; }
| "%define" string_content
{
@@ -232,17 +225,17 @@
nondeterministic_parser = true;
glr_parser = true;
}
-| "%initial-action {...}"
+| "%initial-action" "{...}"
{
- muscle_code_grow ("initial_action", $1, @1);
+ muscle_code_grow ("initial_action", translate_symbol_action ($2, @2),
@2);
}
-| "%lex-param {...}" { add_param ("lex_param", $1, @1); }
+| "%lex-param" "{...}" { add_param ("lex_param", $2, @2); }
| "%locations" { locations_flag = true; }
| "%name-prefix" "=" string_content { spec_name_prefix = $3; }
| "%no-lines" { no_lines_flag = true; }
| "%nondeterministic-parser" { nondeterministic_parser = true; }
| "%output" "=" string_content { spec_outfile = $3; }
-| "%parse-param {...}" { add_param ("parse_param", $1, @1);
}
+| "%parse-param" "{...}" { add_param ("parse_param", $2, @2);
}
| "%pure-parser" { pure_parser = true; }
| "%require" string_content { version_check (&@2, $2); }
| "%skeleton" string_content { skeleton = $2; }
@@ -275,19 +268,21 @@
typed = true;
muscle_code_grow ("stype", body, @1);
}
-| "%destructor {...}" symbols.1
+| "%destructor" "{...}" symbols.1
{
symbol_list *list;
- for (list = $2; list; list = list->next)
- symbol_destructor_set (list->sym, $1, @1);
- symbol_list_free ($2);
+ const char *action = translate_symbol_action ($2, @2);
+ for (list = $3; list; list = list->next)
+ symbol_destructor_set (list->sym, action, @2);
+ symbol_list_free ($3);
}
-| "%printer {...}" symbols.1
+| "%printer" "{...}" symbols.1
{
symbol_list *list;
- for (list = $2; list; list = list->next)
- symbol_printer_set (list->sym, $1, @1);
- symbol_list_free ($2);
+ const char *action = translate_symbol_action ($2, @2);
+ for (list = $3; list; list = list->next)
+ symbol_printer_set (list->sym, action, @2);
+ symbol_list_free ($3);
}
| "%default-prec"
{
@@ -346,7 +341,6 @@
;
/* One or more nonterminals to be %typed. */
-
symbols.1:
symbol { $$ = symbol_list_new ($1, @1); }
| symbols.1 symbol { $$ = symbol_list_prepend ($1, $2, @2); }
@@ -426,7 +420,9 @@
{ grammar_current_rule_begin (current_lhs, current_lhs_location); }
| rhs symbol
{ grammar_current_rule_symbol_append ($2, @2); }
-| rhs action
+| rhs "{...}"
+ { grammar_current_rule_action_append (gram_last_string,
+ gram_last_braced_code_loc); }
| rhs "%prec" symbol
{ grammar_current_rule_prec_set ($3, @3); }
| rhs "%dprec" INT
@@ -440,23 +436,6 @@
| string_as_id { $$ = $1; }
;
-/* Handle the semantics of an action specially, with a mid-rule
- action, so that grammar_current_rule_action_append is invoked
- immediately after the braced code is read by the scanner.
-
- This implementation relies on the LALR(1) parsing algorithm.
- If grammar_current_rule_action_append were executed in a normal
- action for this rule, then when the input grammar contains two
- successive actions, the scanner would have to read both actions
- before reducing this rule. That wouldn't work, since the scanner
- relies on all preceding input actions being processed by
- grammar_current_rule_action_append before it scans the next
- action. */
-action:
- { grammar_current_rule_action_append (last_string, last_braced_code_loc); }
- BRACED_CODE
-;
-
/* A string used as an ID: quote it. */
string_as_id:
STRING
@@ -477,8 +456,8 @@
/* Nothing. */
| "%%" EPILOGUE
{
- muscle_code_grow ("epilogue", $2, @2);
- scanner_last_string_free ();
+ muscle_code_grow ("epilogue", translate_code ($2, @2), @2);
+ gram_scanner_last_string_free ();
}
;
@@ -563,7 +542,7 @@
free (name);
}
- scanner_last_string_free ();
+ gram_scanner_last_string_free ();
}
static void
Index: src/reader.c
===================================================================
RCS file: /cvsroot/bison/bison/src/reader.c,v
retrieving revision 1.254
diff -u -r1.254 reader.c
--- src/reader.c 14 May 2006 19:14:10 -0000 1.254
+++ src/reader.c 29 May 2006 06:53:55 -0000
@@ -22,6 +22,7 @@
#include <config.h>
#include "system.h"
+#include <assert.h>
#include <quotearg.h>
@@ -34,6 +35,8 @@
#include "reader.h"
#include "symlist.h"
#include "symtab.h"
+#include "scan-gram.h"
+#include "scan-code.h"
static void check_and_convert_grammar (void);
@@ -77,6 +80,8 @@
!typed ? &pre_prologue_obstack : &post_prologue_obstack;
obstack_fgrow1 (oout, "]b4_syncline(%d, [[", loc.start.line);
+ /* FIXME: Protection of M4 characters missing here. See
+ output.c:escaped_output. */
MUSCLE_OBSTACK_SGROW (oout,
quotearg_style (c_quoting_style, loc.start.file));
obstack_sgrow (oout, "]])[\n");
@@ -398,9 +403,7 @@
void
grammar_current_rule_action_append (const char *action, location loc)
{
- /* There's no need to invoke grammar_midrule_action here, since the
- scanner already did it if necessary. */
- current_rule->action = action;
+ current_rule->action = translate_rule_action (current_rule, action, loc);
current_rule->action_location = loc;
}
@@ -426,6 +429,7 @@
while (p)
{
+ int rule_length = 0;
symbol *ruleprec = p->ruleprec;
rules[ruleno].user_number = ruleno;
rules[ruleno].number = ruleno;
@@ -440,18 +444,22 @@
rules[ruleno].action = p->action;
rules[ruleno].action_location = p->action_location;
- p = p->next;
- while (p && p->sym)
+ for (p = p->next; p && p->sym; p = p->next)
{
+ ++rule_length;
+
+ /* Don't allow rule_length == INT_MAX, since that might
+ cause confusion with strtol if INT_MAX == LONG_MAX. */
+ if (rule_length == INT_MAX)
+ fatal_at (rules[ruleno].location, _("rule is too long"));
+
/* item_number = symbol_number.
But the former needs to contain more: negative rule numbers. */
ritem[itemno++] = symbol_number_as_item_number (p->sym->number);
/* A rule gets by default the precedence and associativity
- of the last token in it. */
+ of its last token. */
if (p->sym->class == token_sym && default_prec)
rules[ruleno].prec = p->sym;
- if (p)
- p = p->next;
}
/* If this rule has a %prec,
@@ -461,8 +469,11 @@
rules[ruleno].precsym = ruleprec;
rules[ruleno].prec = ruleprec;
}
+ /* An item ends by the rule number (negated). */
ritem[itemno++] = rule_number_as_item_number (ruleno);
+ assert (itemno < ITEM_NUMBER_MAX);
++ruleno;
+ assert (ruleno < RULE_NUMBER_MAX);
if (p)
p = p->next;
@@ -511,7 +522,7 @@
gram__flex_debug = trace_flag & trace_scan;
gram_debug = trace_flag & trace_parse;
- scanner_initialize ();
+ gram_scanner_initialize ();
gram_parse ();
if (! complaint_issued)
Index: src/reader.h
===================================================================
RCS file: /cvsroot/bison/bison/src/reader.h,v
retrieving revision 1.49
diff -u -r1.49 reader.h
--- src/reader.h 9 Mar 2006 23:23:11 -0000 1.49
+++ src/reader.h 29 May 2006 06:53:55 -0000
@@ -35,26 +35,6 @@
uniqstr type;
} merger_list;
-/* From the scanner. */
-extern FILE *gram_in;
-extern int gram__flex_debug;
-extern boundary scanner_cursor;
-extern char *last_string;
-extern location last_braced_code_loc;
-extern int max_left_semantic_context;
-void scanner_initialize (void);
-void scanner_free (void);
-void scanner_last_string_free (void);
-
-/* These are declared by the scanner, but not used. We put them here
- to pacify "make syntax-check". */
-extern FILE *gram_out;
-extern int gram_lineno;
-
-# define YY_DECL int gram_lex (YYSTYPE *val, location *loc)
-YY_DECL;
-
-
/* From the parser. */
extern int gram_debug;
int gram_parse (void);
Index: src/scan-code-c.c
===================================================================
RCS file: src/scan-code-c.c
diff -N src/scan-code-c.c
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ src/scan-code-c.c 29 May 2006 06:53:55 -0000
@@ -0,0 +1,2 @@
+#include <config.h>
+#include "scan-code.c"
Index: src/scan-code.h
===================================================================
RCS file: src/scan-code.h
diff -N src/scan-code.h
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ src/scan-code.h 29 May 2006 06:53:55 -0000
@@ -0,0 +1,47 @@
+/* Bison Action Scanner
+
+ Copyright (C) 2006 Free Software Foundation, Inc.
+
+ This file is part of Bison, the GNU Compiler Compiler.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ 02110-1301 USA
+*/
+
+#ifndef SCAN_CODE_H_
+# define SCAN_CODE_H_
+
+# include "location.h"
+# include "symlist.h"
+
+/* Keeps track of the maximum number of semantic values to the left of
+ a handle (those referenced by $0, $-1, etc.) are required by the
+ semantic actions of this grammar. */
+extern int max_left_semantic_context;
+
+void code_scanner_free (void);
+
+/* The action A contains $$, $1 etc. referring to the values
+ of the rule R. */
+const char *translate_rule_action (symbol_list *r, const char *a, location l);
+
+/* The action A refers to $$ and @$ only, referring to a symbol. */
+const char *translate_symbol_action (const char *a, location l);
+
+/* The action contains no special escapes, just protect M4 special
+ symbols. */
+const char *translate_code (const char *a, location l);
+
+#endif /* !SCAN_CODE_H_ */
Index: src/scan-code.l
===================================================================
RCS file: src/scan-code.l
diff -N src/scan-code.l
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ src/scan-code.l 29 May 2006 06:53:55 -0000
@@ -0,0 +1,358 @@
+/* Bison Action Scanner -*- C -*-
+
+ Copyright (C) 2006 Free Software Foundation, Inc.
+
+ This file is part of Bison, the GNU Compiler Compiler.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ 02110-1301 USA
+*/
+
+%option debug nodefault nounput noyywrap never-interactive
+%option prefix="code_" outfile="lex.yy.c"
+
+%{
+/* Work around a bug in flex 2.5.31. See Debian bug 333231
+ <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */
+#undef code_wrap
+#define code_wrap() 1
+
+#define FLEX_PREFIX(Id) code_ ## Id
+#include "flex-scanner.h"
+#include "reader.h"
+#include "getargs.h"
+#include <assert.h>
+#include <get-errno.h>
+#include <quote.h>
+
+#include "scan-code.h"
+
+/* The current calling start condition: SC_RULE_ACTION or
+ SC_SYMBOL_ACTION. */
+# define YY_DECL const char *code_lex (int sc_context)
+YY_DECL;
+
+#define YY_USER_ACTION location_compute (loc, &loc->end, yytext, yyleng);
+
+static void handle_action_dollar (char *cp, location loc);
+static void handle_action_at (char *cp, location loc);
+static location the_location;
+static location *loc = &the_location;
+
+/* The rule being processed. */
+symbol_list *current_rule;
+%}
+ /* C and C++ comments in code. */
+%x SC_COMMENT SC_LINE_COMMENT
+ /* Strings and characters in code. */
+%x SC_STRING SC_CHARACTER
+ /* Whether in a rule or symbol action. Specifies the translation
+ of $ and @. */
+%x SC_RULE_ACTION SC_SYMBOL_ACTION
+
+
+/* POSIX says that a tag must be both an id and a C union member, but
+ historically almost any character is allowed in a tag. We disallow
+ NUL and newline, as this simplifies our implementation. */
+tag [^\0\n>]+
+
+/* Zero or more instances of backslash-newline. Following GCC, allow
+ white space between the backslash and the newline. */
+splice (\\[ \f\t\v]*\n)*
+
+%%
+
+%{
+ /* This scanner is special: it is invoked only once, henceforth
+ is expected to return only once. This initialization is
+ therefore done once per action to translate. */
+ assert (sc_context == SC_SYMBOL_ACTION
+ || sc_context == SC_RULE_ACTION
+ || sc_context == INITIAL);
+ BEGIN sc_context;
+%}
+
+ /*------------------------------------------------------------.
+ | Scanning a C comment. The initial `/ *' is already eaten. |
+ `------------------------------------------------------------*/
+
+<SC_COMMENT>
+{
+ "*"{splice}"/" STRING_GROW; BEGIN sc_context;
+}
+
+
+ /*--------------------------------------------------------------.
+ | Scanning a line comment. The initial `//' is already eaten. |
+ `--------------------------------------------------------------*/
+
+<SC_LINE_COMMENT>
+{
+ "\n" STRING_GROW; BEGIN sc_context;
+ {splice} STRING_GROW;
+}
+
+
+ /*--------------------------------------------.
+ | Scanning user-code characters and strings. |
+ `--------------------------------------------*/
+
+<SC_CHARACTER,SC_STRING>
+{
+ {splice}|\\{splice}. STRING_GROW;
+}
+
+<SC_CHARACTER>
+{
+ "'" STRING_GROW; BEGIN sc_context;
+}
+
+<SC_STRING>
+{
+ "\"" STRING_GROW; BEGIN sc_context;
+}
+
+
+<SC_RULE_ACTION,SC_SYMBOL_ACTION>{
+ "'" {
+ STRING_GROW;
+ BEGIN SC_CHARACTER;
+ }
+ "\"" {
+ STRING_GROW;
+ BEGIN SC_STRING;
+ }
+ "/"{splice}"*" {
+ STRING_GROW;
+ BEGIN SC_COMMENT;
+ }
+ "/"{splice}"/" {
+ STRING_GROW;
+ BEGIN SC_LINE_COMMENT;
+ }
+}
+
+<SC_RULE_ACTION>
+{
+ "$"("<"{tag}">")?(-?[0-9]+|"$") handle_action_dollar (yytext, *loc);
+ "@"(-?[0-9]+|"$") handle_action_at (yytext, *loc);
+
+ "$" {
+ warn_at (*loc, _("stray `$'"));
+ obstack_sgrow (&obstack_for_string, "$][");
+ }
+ "@" {
+ warn_at (*loc, _("stray `@'"));
+ obstack_sgrow (&obstack_for_string, "@@");
+ }
+}
+
+<SC_SYMBOL_ACTION>
+{
+ "$$" obstack_sgrow (&obstack_for_string, "]b4_dollar_dollar[");
+ "@$" obstack_sgrow (&obstack_for_string, "]b4_at_dollar[");
+}
+
+
+ /*-----------------------------------------.
+ | Escape M4 quoting characters in C code. |
+ `-----------------------------------------*/
+
+<*>
+{
+ \$ obstack_sgrow (&obstack_for_string, "$][");
+ \@ obstack_sgrow (&obstack_for_string, "@@");
+ \[ obstack_sgrow (&obstack_for_string, "@{");
+ \] obstack_sgrow (&obstack_for_string, "@}");
+}
+
+ /*-----------------------------------------------------.
+ | By default, grow the string obstack with the input. |
+ `-----------------------------------------------------*/
+
+<*>.|\n STRING_GROW;
+
+ /* End of processing. */
+<*><<EOF>> {
+ obstack_1grow (&obstack_for_string, '\0');
+ return obstack_finish (&obstack_for_string);
+ }
+
+%%
+
+/* Keeps track of the maximum number of semantic values to the left of
+ a handle (those referenced by $0, $-1, etc.) are required by the
+ semantic actions of this grammar. */
+int max_left_semantic_context = 0;
+
+
+/*------------------------------------------------------------------.
+| TEXT is pointing to a wannabee semantic value (i.e., a `$'). |
+| |
+| Possible inputs: $[<TYPENAME>]($|integer) |
+| |
+| Output to OBSTACK_FOR_STRING a reference to this semantic value. |
+`------------------------------------------------------------------*/
+
+static void
+handle_action_dollar (char *text, location loc)
+{
+ const char *type_name = NULL;
+ char *cp = text + 1;
+ int rule_length = symbol_list_length (current_rule->next);
+
+ /* Get the type name if explicit. */
+ if (*cp == '<')
+ {
+ type_name = ++cp;
+ while (*cp != '>')
+ ++cp;
+ *cp = '\0';
+ ++cp;
+ }
+
+ if (*cp == '$')
+ {
+ if (!type_name)
+ type_name = symbol_list_n_type_name_get (current_rule, loc, 0);
+ if (!type_name && typed)
+ complain_at (loc, _("$$ of `%s' has no declared type"),
+ current_rule->sym->tag);
+ if (!type_name)
+ type_name = "";
+ obstack_fgrow1 (&obstack_for_string,
+ "]b4_lhs_value([%s])[", type_name);
+ current_rule->used = true;
+ }
+ else
+ {
+ long int num;
+ set_errno (0);
+ num = strtol (cp, 0, 10);
+ if (INT_MIN <= num && num <= rule_length && ! get_errno ())
+ {
+ int n = num;
+ if (1-n > max_left_semantic_context)
+ max_left_semantic_context = 1-n;
+ if (!type_name && n > 0)
+ type_name = symbol_list_n_type_name_get (current_rule, loc, n);
+ if (!type_name && typed)
+ complain_at (loc, _("$%d of `%s' has no declared type"),
+ n, current_rule->sym->tag);
+ if (!type_name)
+ type_name = "";
+ obstack_fgrow3 (&obstack_for_string,
+ "]b4_rhs_value(%d, %d, [%s])[",
+ rule_length, n, type_name);
+ symbol_list_n_used_set (current_rule, n, true);
+ }
+ else
+ complain_at (loc, _("integer out of range: %s"), quote (text));
+ }
+}
+
+
+/*------------------------------------------------------.
+| TEXT is a location token (i.e., a [EMAIL PROTECTED]'). Output to |
+| OBSTACK_FOR_STRING a reference to this location. |
+`------------------------------------------------------*/
+
+static void
+handle_action_at (char *text, location loc)
+{
+ char *cp = text + 1;
+ int rule_length = symbol_list_length (current_rule->next);
+ locations_flag = true;
+
+ if (*cp == '$')
+ obstack_sgrow (&obstack_for_string, "]b4_lhs_location[");
+ else
+ {
+ long int num;
+ set_errno (0);
+ num = strtol (cp, 0, 10);
+
+ if (INT_MIN <= num && num <= rule_length && ! get_errno ())
+ {
+ int n = num;
+ obstack_fgrow2 (&obstack_for_string, "]b4_rhs_location(%d, %d)[",
+ rule_length, n);
+ }
+ else
+ complain_at (loc, _("integer out of range: %s"), quote (text));
+ }
+}
+
+
+/*-------------------------.
+| Initialize the scanner. |
+`-------------------------*/
+
+/* Translate the dollars and ats in \a a, whose location is l.
+ Depending on the \a sc_context (SC_RULE_ACTION, SC_SYMBOL_ACTION,
+ INITIAL), the processing is different. */
+
+static const char *
+translate_action (int sc_context, const char *a, location l)
+{
+ const char *res;
+ static bool initialized = false;
+ if (!initialized)
+ {
+ obstack_init (&obstack_for_string);
+ /* The initial buffer, never used. */
+ yy_delete_buffer (YY_CURRENT_BUFFER);
+ yy_flex_debug = 0;
+ initialized = true;
+ }
+
+ loc->start = loc->end = l.start;
+ yy_switch_to_buffer (yy_scan_string (a));
+ res = code_lex (sc_context);
+ yy_delete_buffer (YY_CURRENT_BUFFER);
+
+ return res;
+}
+
+const char *
+translate_rule_action (symbol_list *r, const char *a, location l)
+{
+ current_rule = r;
+ return translate_action (SC_RULE_ACTION, a, l);
+}
+
+const char *
+translate_symbol_action (const char *a, location l)
+{
+ return translate_action (SC_SYMBOL_ACTION, a, l);
+}
+
+const char *
+translate_code (const char *a, location l)
+{
+ return translate_action (INITIAL, a, l);
+}
+
+/*-----------------------------------------------.
+| Free all the memory allocated to the scanner. |
+`-----------------------------------------------*/
+
+void
+code_scanner_free (void)
+{
+ obstack_free (&obstack_for_string, 0);
+ /* Reclaim Flex's buffers. */
+ yy_delete_buffer (YY_CURRENT_BUFFER);
+}
Index: src/scan-gram.l
===================================================================
RCS file: /cvsroot/bison/bison/src/scan-gram.l,v
retrieving revision 1.86
diff -u -r1.86 scan-gram.l
--- src/scan-gram.l 3 Apr 2006 13:50:10 -0000 1.86
+++ src/scan-gram.l 29 May 2006 06:53:55 -0000
@@ -29,112 +29,48 @@
#undef gram_wrap
#define gram_wrap() 1
-#include "system.h"
-
-#include <mbswidth.h>
-#include <quote.h>
+#define FLEX_PREFIX(Id) gram_ ## Id
+#include "flex-scanner.h"
#include "complain.h"
#include "files.h"
-#include "getargs.h"
+#include "getargs.h" /* yacc_flag */
#include "gram.h"
#include "quotearg.h"
#include "reader.h"
#include "uniqstr.h"
+#include <mbswidth.h>
+#include <quote.h>
+
+#include "scan-gram.h"
+
+#define YY_DECL GRAM_LEX_DECL
+
#define YY_USER_INIT \
- do \
- { \
- scanner_cursor.file = current_file; \
- scanner_cursor.line = 1; \
- scanner_cursor.column = 1; \
- code_start = scanner_cursor; \
- } \
- while (0)
-
-/* Pacify "gcc -Wmissing-prototypes" when flex 2.5.31 is used. */
-int gram_get_lineno (void);
-FILE *gram_get_in (void);
-FILE *gram_get_out (void);
-int gram_get_leng (void);
-char *gram_get_text (void);
-void gram_set_lineno (int);
-void gram_set_in (FILE *);
-void gram_set_out (FILE *);
-int gram_get_debug (void);
-void gram_set_debug (int);
-int gram_lex_destroy (void);
+ code_start = scanner_cursor = loc->start; \
/* Location of scanner cursor. */
boundary scanner_cursor;
-static void adjust_location (location *, char const *, size_t);
-#define YY_USER_ACTION adjust_location (loc, yytext, yyleng);
+#define YY_USER_ACTION location_compute (loc, &scanner_cursor, yytext,
yyleng);
static size_t no_cr_read (FILE *, char *, size_t);
#define YY_INPUT(buf, result, size) ((result) = no_cr_read (yyin, buf, size))
-
-/* OBSTACK_FOR_STRING -- Used to store all the characters that we need to
- keep (to construct ID, STRINGS etc.). Use the following macros to
- use it.
-
- Use STRING_GROW to append what has just been matched, and
- STRING_FINISH to end the string (it puts the ending 0).
- STRING_FINISH also stores this string in LAST_STRING, which can be
- used, and which is used by STRING_FREE to free the last string. */
-
-static struct obstack obstack_for_string;
-
/* A string representing the most recently saved token. */
char *last_string;
-/* The location of the most recently saved token, if it was a
- BRACED_CODE token; otherwise, this has an unspecified value. */
-location last_braced_code_loc;
-
-#define STRING_GROW \
- obstack_grow (&obstack_for_string, yytext, yyleng)
-
-#define STRING_FINISH \
- do { \
- obstack_1grow (&obstack_for_string, '\0'); \
- last_string = obstack_finish (&obstack_for_string); \
- } while (0)
-
-#define STRING_FREE \
- obstack_free (&obstack_for_string, last_string)
-
void
-scanner_last_string_free (void)
+gram_scanner_last_string_free (void)
{
STRING_FREE;
}
-/* Within well-formed rules, RULE_LENGTH is the number of values in
- the current rule so far, which says where to find `$0' with respect
- to the top of the stack. It is not the same as the rule->length in
- the case of mid rule actions.
-
- Outside of well-formed rules, RULE_LENGTH has an undefined value. */
-static int rule_length;
-
-static void rule_length_overflow (location) __attribute__ ((__noreturn__));
-
-/* Increment the rule length by one, checking for overflow. */
-static inline void
-increment_rule_length (location loc)
-{
- rule_length++;
-
- /* Don't allow rule_length == INT_MAX, since that might cause
- confusion with strtol if INT_MAX == LONG_MAX. */
- if (rule_length == INT_MAX)
- rule_length_overflow (loc);
-}
+/* The location of the most recently saved token, if it was a
+ BRACED_CODE token; otherwise, this has an unspecified value. */
+location gram_last_braced_code_loc;
-static void handle_dollar (int token_type, char *cp, location loc);
-static void handle_at (int token_type, char *cp, location loc);
static void handle_syncline (char *, location);
static unsigned long int scan_integer (char const *p, int base, location loc);
static int convert_ucn_to_byte (char const *hex_text);
@@ -142,11 +78,26 @@
static void unexpected_newline (boundary, char const *);
%}
-%x SC_COMMENT SC_LINE_COMMENT SC_YACC_COMMENT
-%x SC_STRING SC_CHARACTER
-%x SC_AFTER_IDENTIFIER
+ /* A C-like comment in directives/rules. */
+%x SC_YACC_COMMENT
+ /* Strings and characters in directives/rules. */
%x SC_ESCAPED_STRING SC_ESCAPED_CHARACTER
-%x SC_PRE_CODE SC_BRACED_CODE SC_PROLOGUE SC_EPILOGUE
+ /* A identifier was just read in directives/rules. Special state
+ to capture the sequence `identifier :'. */
+%x SC_AFTER_IDENTIFIER
+ /* A keyword that should be followed by some code was read (e.g.
+ %printer). */
+%x SC_PRE_CODE
+
+ /* Three types of user code:
+ - prologue (code between `%{' `%}' in the first section, before %%);
+ - actions, printers, union, etc, (between braced in the middle section);
+ - epilogue (everything after the second %%). */
+%x SC_PROLOGUE SC_BRACED_CODE SC_EPILOGUE
+ /* C and C++ comments in code. */
+%x SC_COMMENT SC_LINE_COMMENT
+ /* Strings and characters in code. */
+%x SC_STRING SC_CHARACTER
letter [.abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_]
id {letter}({letter}|[0-9])*
@@ -221,17 +172,17 @@
"%default"[-_]"prec" return PERCENT_DEFAULT_PREC;
"%define" return PERCENT_DEFINE;
"%defines" return PERCENT_DEFINES;
- "%destructor" token_type = PERCENT_DESTRUCTOR; BEGIN
SC_PRE_CODE;
+ "%destructor" /* FIXME: Remove once %union handled
differently. */ token_type = BRACED_CODE; return PERCENT_DESTRUCTOR;
"%dprec" return PERCENT_DPREC;
"%error"[-_]"verbose" return PERCENT_ERROR_VERBOSE;
"%expect" return PERCENT_EXPECT;
"%expect"[-_]"rr" return PERCENT_EXPECT_RR;
"%file-prefix" return PERCENT_FILE_PREFIX;
"%fixed"[-_]"output"[-_]"files" return PERCENT_YACC;
- "%initial-action" token_type = PERCENT_INITIAL_ACTION; BEGIN
SC_PRE_CODE;
+ "%initial-action" /* FIXME: Remove once %union handled differently.
*/ token_type = BRACED_CODE; return PERCENT_INITIAL_ACTION;
"%glr-parser" return PERCENT_GLR_PARSER;
"%left" return PERCENT_LEFT;
- "%lex-param" token_type = PERCENT_LEX_PARAM; BEGIN SC_PRE_CODE;
+ "%lex-param" /* FIXME: Remove once %union handled differently. */
token_type = BRACED_CODE; return PERCENT_LEX_PARAM;
"%locations" return PERCENT_LOCATIONS;
"%merge" return PERCENT_MERGE;
"%name"[-_]"prefix" return PERCENT_NAME_PREFIX;
@@ -241,9 +192,9 @@
"%nondeterministic-parser" return PERCENT_NONDETERMINISTIC_PARSER;
"%nterm" return PERCENT_NTERM;
"%output" return PERCENT_OUTPUT;
- "%parse-param" token_type = PERCENT_PARSE_PARAM; BEGIN SC_PRE_CODE;
- "%prec" rule_length--; return PERCENT_PREC;
- "%printer" token_type = PERCENT_PRINTER; BEGIN SC_PRE_CODE;
+ "%parse-param" /* FIXME: Remove once %union handled differently. */
token_type = BRACED_CODE; return PERCENT_PARSE_PARAM;
+ "%prec" return PERCENT_PREC;
+ "%printer" /* FIXME: Remove once %union handled differently.
*/ token_type = BRACED_CODE; return PERCENT_PRINTER;
"%pure"[-_]"parser" return PERCENT_PURE_PARSER;
"%require" return PERCENT_REQUIRE;
"%right" return PERCENT_RIGHT;
@@ -262,13 +213,12 @@
}
"=" return EQUAL;
- "|" rule_length = 0; return PIPE;
+ "|" return PIPE;
";" return SEMICOLON;
{id} {
val->symbol = symbol_get (yytext, *loc);
id_loc = *loc;
- increment_rule_length (*loc);
BEGIN SC_AFTER_IDENTIFIER;
}
@@ -335,7 +285,6 @@
<SC_AFTER_IDENTIFIER>
{
":" {
- rule_length = 0;
*loc = id_loc;
BEGIN INITIAL;
return ID_COLON;
@@ -401,7 +350,6 @@
STRING_FINISH;
loc->start = token_start;
val->chars = last_string;
- increment_rule_length (*loc);
BEGIN INITIAL;
return STRING;
}
@@ -428,7 +376,6 @@
last_string_1 = last_string[1];
symbol_user_token_number_set (val->symbol, last_string_1, *loc);
STRING_FREE;
- increment_rule_length (*loc);
BEGIN INITIAL;
return ID;
}
@@ -501,7 +448,7 @@
<SC_CHARACTER,SC_STRING>
{
- {splice}|[EMAIL PROTECTED] STRING_GROW;
+ {splice}|\\{splice}[^\n\[\]] STRING_GROW;
}
<SC_CHARACTER>
@@ -622,8 +569,7 @@
STRING_FINISH;
loc->start = code_start;
val->chars = last_string;
- increment_rule_length (*loc);
- last_braced_code_loc = *loc;
+ gram_last_braced_code_loc = *loc;
BEGIN INITIAL;
return token_type;
}
@@ -633,18 +579,6 @@
(as `<' `<%'). */
"<"{splice}"<" STRING_GROW;
- "$"("<"{tag}">")?(-?[0-9]+|"$") handle_dollar (token_type, yytext, *loc);
- "@"(-?[0-9]+|"$") handle_at (token_type, yytext, *loc);
-
- "$" {
- warn_at (*loc, _("stray `$'"));
- obstack_sgrow (&obstack_for_string, "$][");
- }
- "@" {
- warn_at (*loc, _("stray `@'"));
- obstack_sgrow (&obstack_for_string, "@@");
- }
-
<<EOF>> unexpected_eof (code_start, "}"); BEGIN INITIAL;
}
@@ -684,19 +618,6 @@
}
- /*-----------------------------------------.
- | Escape M4 quoting characters in C code. |
- `-----------------------------------------*/
-
-<SC_COMMENT,SC_LINE_COMMENT,SC_STRING,SC_CHARACTER,SC_BRACED_CODE,SC_PROLOGUE,SC_EPILOGUE>
-{
- \$ obstack_sgrow (&obstack_for_string, "$][");
- \@ obstack_sgrow (&obstack_for_string, "@@");
- \[ obstack_sgrow (&obstack_for_string, "@{");
- \] obstack_sgrow (&obstack_for_string, "@}");
-}
-
-
/*-----------------------------------------------------.
| By default, grow the string obstack with the input. |
`-----------------------------------------------------*/
@@ -706,79 +627,6 @@
%%
-/* Keeps track of the maximum number of semantic values to the left of
- a handle (those referenced by $0, $-1, etc.) are required by the
- semantic actions of this grammar. */
-int max_left_semantic_context = 0;
-
-/* If BUF is null, add BUFSIZE (which in this case must be less than
- INT_MAX) to COLUMN; otherwise, add mbsnwidth (BUF, BUFSIZE, 0) to
- COLUMN. If an overflow occurs, or might occur but is undetectable,
- return INT_MAX. Assume COLUMN is nonnegative. */
-
-static inline int
-add_column_width (int column, char const *buf, size_t bufsize)
-{
- size_t width;
- unsigned int remaining_columns = INT_MAX - column;
-
- if (buf)
- {
- if (INT_MAX / 2 <= bufsize)
- return INT_MAX;
- width = mbsnwidth (buf, bufsize, 0);
- }
- else
- width = bufsize;
-
- return width <= remaining_columns ? column + width : INT_MAX;
-}
-
-/* Set *LOC and adjust scanner cursor to account for token TOKEN of
- size SIZE. */
-
-static void
-adjust_location (location *loc, char const *token, size_t size)
-{
- int line = scanner_cursor.line;
- int column = scanner_cursor.column;
- char const *p0 = token;
- char const *p = token;
- char const *lim = token + size;
-
- loc->start = scanner_cursor;
-
- for (p = token; p < lim; p++)
- switch (*p)
- {
- case '\n':
- line += line < INT_MAX;
- column = 1;
- p0 = p + 1;
- break;
-
- case '\t':
- column = add_column_width (column, p0, p - p0);
- column = add_column_width (column, NULL, 8 - ((column - 1) & 7));
- p0 = p + 1;
- break;
-
- default:
- break;
- }
-
- scanner_cursor.line = line;
- scanner_cursor.column = column = add_column_width (column, p0, p - p0);
-
- loc->end = scanner_cursor;
-
- if (line == INT_MAX && loc->start.line != INT_MAX)
- warn_at (*loc, _("line number overflow"));
- if (column == INT_MAX && loc->start.column != INT_MAX)
- warn_at (*loc, _("column number overflow"));
-}
-
-
/* Read bytes from FP into buffer BUF of size SIZE. Return the
number of bytes read. Remove '\r' from input, treating \r\n
and isolated \r as \n. */
@@ -826,173 +674,6 @@
}
-/*------------------------------------------------------------------.
-| TEXT is pointing to a wannabee semantic value (i.e., a `$'). |
-| |
-| Possible inputs: $[<TYPENAME>]($|integer) |
-| |
-| Output to OBSTACK_FOR_STRING a reference to this semantic value. |
-`------------------------------------------------------------------*/
-
-static inline bool
-handle_action_dollar (char *text, location loc)
-{
- const char *type_name = NULL;
- char *cp = text + 1;
-
- if (! current_rule)
- return false;
-
- /* Get the type name if explicit. */
- if (*cp == '<')
- {
- type_name = ++cp;
- while (*cp != '>')
- ++cp;
- *cp = '\0';
- ++cp;
- }
-
- if (*cp == '$')
- {
- if (!type_name)
- type_name = symbol_list_n_type_name_get (current_rule, loc, 0);
- if (!type_name && typed)
- complain_at (loc, _("$$ of `%s' has no declared type"),
- current_rule->sym->tag);
- if (!type_name)
- type_name = "";
- obstack_fgrow1 (&obstack_for_string,
- "]b4_lhs_value([%s])[", type_name);
- current_rule->used = true;
- }
- else
- {
- long int num = strtol (cp, NULL, 10);
-
- if (1 - INT_MAX + rule_length <= num && num <= rule_length)
- {
- int n = num;
- if (max_left_semantic_context < 1 - n)
- max_left_semantic_context = 1 - n;
- if (!type_name && 0 < n)
- type_name = symbol_list_n_type_name_get (current_rule, loc, n);
- if (!type_name && typed)
- complain_at (loc, _("$%d of `%s' has no declared type"),
- n, current_rule->sym->tag);
- if (!type_name)
- type_name = "";
- obstack_fgrow3 (&obstack_for_string,
- "]b4_rhs_value(%d, %d, [%s])[",
- rule_length, n, type_name);
- symbol_list_n_used_set (current_rule, n, true);
- }
- else
- complain_at (loc, _("integer out of range: %s"), quote (text));
- }
-
- return true;
-}
-
-
-/*----------------------------------------------------------------.
-| Map `$?' onto the proper M4 symbol, depending on its TOKEN_TYPE |
-| (are we in an action?). |
-`----------------------------------------------------------------*/
-
-static void
-handle_dollar (int token_type, char *text, location loc)
-{
- switch (token_type)
- {
- case BRACED_CODE:
- if (handle_action_dollar (text, loc))
- return;
- break;
-
- case PERCENT_DESTRUCTOR:
- case PERCENT_INITIAL_ACTION:
- case PERCENT_PRINTER:
- if (text[1] == '$')
- {
- obstack_sgrow (&obstack_for_string, "]b4_dollar_dollar[");
- return;
- }
- break;
-
- default:
- break;
- }
-
- complain_at (loc, _("invalid value: %s"), quote (text));
-}
-
-
-/*------------------------------------------------------.
-| TEXT is a location token (i.e., a [EMAIL PROTECTED]'). Output to |
-| OBSTACK_FOR_STRING a reference to this location. |
-`------------------------------------------------------*/
-
-static inline bool
-handle_action_at (char *text, location loc)
-{
- char *cp = text + 1;
- locations_flag = true;
-
- if (! current_rule)
- return false;
-
- if (*cp == '$')
- obstack_sgrow (&obstack_for_string, "]b4_lhs_location[");
- else
- {
- long int num = strtol (cp, NULL, 10);
-
- if (1 - INT_MAX + rule_length <= num && num <= rule_length)
- {
- int n = num;
- obstack_fgrow2 (&obstack_for_string, "]b4_rhs_location(%d, %d)[",
- rule_length, n);
- }
- else
- complain_at (loc, _("integer out of range: %s"), quote (text));
- }
-
- return true;
-}
-
-
-/*----------------------------------------------------------------.
-| Map [EMAIL PROTECTED]' onto the proper M4 symbol, depending on its
TOKEN_TYPE |
-| (are we in an action?). |
-`----------------------------------------------------------------*/
-
-static void
-handle_at (int token_type, char *text, location loc)
-{
- switch (token_type)
- {
- case BRACED_CODE:
- handle_action_at (text, loc);
- return;
-
- case PERCENT_INITIAL_ACTION:
- case PERCENT_DESTRUCTOR:
- case PERCENT_PRINTER:
- if (text[1] == '$')
- {
- obstack_sgrow (&obstack_for_string, "]b4_at_dollar[");
- return;
- }
- break;
-
- default:
- break;
- }
-
- complain_at (loc, _("invalid value: %s"), quote (text));
-}
-
/*------------------------------------------------------.
| Scan NUMBER for a base-BASE integer at location LOC. |
@@ -1087,20 +768,8 @@
warn_at (loc, _("line number overflow"));
lineno = INT_MAX;
}
- scanner_cursor.file = current_file = uniqstr_new (file);
- scanner_cursor.line = lineno;
- scanner_cursor.column = 1;
-}
-
-
-/*---------------------------------.
-| Report a rule that is too long. |
-`---------------------------------*/
-
-static void
-rule_length_overflow (location loc)
-{
- fatal_at (loc, _("rule is too long"));
+ current_file = uniqstr_new (file);
+ boundary_set, (&scanner_cursor, current_file, lineno, 1);
}
@@ -1148,7 +817,7 @@
`-------------------------*/
void
-scanner_initialize (void)
+gram_scanner_initialize (void)
{
obstack_init (&obstack_for_string);
}
@@ -1159,7 +828,7 @@
`-----------------------------------------------*/
void
-scanner_free (void)
+gram_scanner_free (void)
{
obstack_free (&obstack_for_string, 0);
/* Reclaim Flex's buffers. */
Index: src/system.h
===================================================================
RCS file: /cvsroot/bison/bison/src/system.h,v
retrieving revision 1.76
diff -u -r1.76 system.h
--- src/system.h 22 Jan 2006 07:38:49 -0000 1.76
+++ src/system.h 29 May 2006 06:53:55 -0000
@@ -113,6 +113,8 @@
# define ATTRIBUTE_UNUSED __attribute__ ((__unused__))
#endif
+#define FUNCTION_PRINT() fprintf (stderr, "%s: ", __func__)
+
/*------.
| NLS. |
`------*/
Index: tests/input.at
===================================================================
RCS file: /cvsroot/bison/bison/tests/input.at,v
retrieving revision 1.43
diff -u -r1.43 input.at
--- tests/input.at 3 Apr 2006 13:50:10 -0000 1.43
+++ tests/input.at 29 May 2006 06:53:56 -0000
@@ -25,33 +25,17 @@
## Invalid $n. ##
## ------------ ##
-AT_SETUP([Invalid dollar-n])
+AT_SETUP([Invalid \$n and @n])
AT_DATA([input.y],
[[%%
exp: { $$ = $1 ; };
-]])
-
-AT_CHECK([bison input.y], [1], [],
-[[input.y:2.13-14: integer out of range: `$1'
-]])
-
-AT_CLEANUP
-
-
-## ------------ ##
-## Invalid @n. ##
-## ------------ ##
-
-AT_SETUP([Invalid @n])
-
-AT_DATA([input.y],
-[[%%
exp: { @$ = @1 ; };
]])
AT_CHECK([bison input.y], [1], [],
-[[input.y:2.13-14: integer out of range: [EMAIL PROTECTED]'
+[[input.y:2.13-14: integer out of range: `$1'
+input.y:3.13-14: integer out of range: [EMAIL PROTECTED]'
]])
AT_CLEANUP
Index: tests/regression.at
===================================================================
RCS file: /cvsroot/bison/bison/tests/regression.at,v
retrieving revision 1.98
diff -u -r1.98 regression.at
--- tests/regression.at 21 May 2006 04:48:47 -0000 1.98
+++ tests/regression.at 29 May 2006 06:53:56 -0000
@@ -346,9 +346,7 @@
]])
AT_CHECK([bison input.y], [1], [],
-[[input.y:3.1: missing `{' in "%destructor {...}"
-input.y:4.1: missing `{' in "%initial-action {...}"
-input.y:4.1: syntax error, unexpected %initial-action {...}, expecting string
or identifier
+[[input.y:3.1-15: syntax error, unexpected %initial-action, expecting {...}
]])
AT_CLEANUP
