On Mon, 18 Dec 2006, Joel E. Denny wrote: > > This will break a lot of code, I'm afraid, since many apps assume > > yyparse is a true function and is not a macro. > > No apps assume that for %push-parser since no apps use %push-parser yet.
> > While we're on the subject, the "&yylex" isn't portable either. yylex > > might be a macro, or it might not have exactly the signature you > > expect (it might return 'wchar_t', say). > > Thanks. Bummer. Again, we'd need something like %pull-parser. I decided to try out something along those lines. Here are some advantages: 1. Using a lexer function pointer can have a measurable negative impact on performance (especially when I pass -O3 to gcc), so in this regard it's better to hard-code the yylex call. (If the user actually wants the flexibility to call multiple lexer functions, he can write his own yypush_parse wrappers. This wasn't actually one of my objectives in the existing implementation anyway.) 2. If the user wishes to define yylex as a macro, that can have an additional measurable positive impact on performance, so we should probably allow that for yyparse in push mode. 3. I believe that making yyparse a function rather than a macro will make the push mode interface completely compatible with the pull mode interface. This should simplify the documentation and the test suite. I still want yypull_parse since it can be mixed with yypush_parse, and I see no harm to performance by adding this function to the call chain. This makes sense since yyparse and/or yypull_parse are called only once per parse. At the moment, I favor: %push-pull-parser rather than: %push-parser %pull-parser The latter begs the question: what does %pull-parser do when there's no %push-parser? Moreover, the former is shorter. What do people think of the following uncommitted patch? Index: ChangeLog =================================================================== RCS file: /sources/bison/bison/ChangeLog,v retrieving revision 1.1639 diff -p -u -r1.1639 ChangeLog --- ChangeLog 26 Dec 2006 18:09:53 -0000 1.1639 +++ ChangeLog 26 Dec 2006 23:47:57 -0000 @@ -1,5 +1,45 @@ 2006-12-26 Joel E. Denny <[EMAIL PROTECTED]> + For push mode, convert yyparse from a macro to a function, invoke yylex + instead of passing a yylexp argument to yypull_parse, and don't + generate yypull_parse or yyparse unless %push-pull-parser is declared. + Discussed starting at + <http://lists.gnu.org/archive/html/bison-patches/2006-12/msg00163.html>. + * data/bison.m4 (b4_pull_if): New. + * data/c.m4 (YYPULL): New similar to YYPUSH. + * data/push.c: Improve M4 quoting a little. + (b4_generate_macro_args, b4_parenthesize): Remove. + (yyparse): If there's a b4_prefix, #define this to b4_prefix[parse] + any time a pull parser is requested. + Don't #define this as a wrapper around yypull_parse. Instead, when + both push and pull are requested, make it a function that does that + same thing. + (yypull_parse): If there's a b4_prefix, #define this to + b4_prefix[pull_parse] when both push and pull are requested. + Don't define this as a function unless both push and pull are + requested. + Remove the yylexp argument and hard-code yylex invocation instead. + * etc/bench.pl.in (bench_grammar): Use %push-pull-parser instead of + %push-parser. + * src/getargs.c (pull_parser): New global initialized to true. + * getargs.h (pull_parser): extern it. + * src/output.c (prepare): Insert pull_flag muscle. + * src/parse-gram.y (PERCENT_PUSH_PULL_PARSER): New token. + (prologue_declaration): Set both push_parser and pull_parser = true for + %push-pull-parser. Set push_parser = true and pull_parser = false for + %push-parser. + * src/scan-gram.l: Don't accept %push_parser as an alternative to + %push-parser since there's no backward-compatibility concern here. + Scan %push-pull-parser. + * tests/calc.at (Simple LALR(1) Calculator): Use %push-pull-parser + instead of %push-parser. + * tests/headers.at (export YYLTYPE): Make yylex static, and don't + prototype it in the module that calls yyparse. + * tests/input.at (Torturing the Scanner): Likewise. + * tests/local.at (AT_PUSH_IF): Check for %push-pull-parser as well. + +2006-12-26 Joel E. Denny <[EMAIL PROTECTED]> + Update etc/bench.pl. Optimize push mode a little (the yyn change deserves most of the credit). * Makefile.am (SUBDIRS): Add etc subdirectory. Index: data/bison.m4 =================================================================== RCS file: /sources/bison/bison/data/bison.m4,v retrieving revision 1.2 diff -p -u -r1.2 bison.m4 --- data/bison.m4 15 Oct 2006 12:37:06 -0000 1.2 +++ data/bison.m4 26 Dec 2006 23:47:57 -0000 @@ -127,8 +127,9 @@ b4_define_flag_if([error_verbose]) # Whe b4_define_flag_if([glr]) # Whether a GLR parser is requested. b4_define_flag_if([locations]) # Whether locations are tracked. b4_define_flag_if([nondeterministic]) # Whether conflicts should be handled. +b4_define_flag_if([pull]) # Whether pull parsing is requested. b4_define_flag_if([pure]) # Whether the interface is pure. -b4_define_flag_if([push]) # Whether push parsing is supported. +b4_define_flag_if([push]) # Whether push parsing is requested. b4_define_flag_if([yacc]) # Whether POSIX Yacc is emulated. Index: data/c.m4 =================================================================== RCS file: /sources/bison/bison/data/c.m4,v retrieving revision 1.67 diff -p -u -r1.67 c.m4 --- data/c.m4 8 Oct 2006 11:07:02 -0000 1.67 +++ data/c.m4 26 Dec 2006 23:47:57 -0000 @@ -46,6 +46,9 @@ m4_define([b4_identification], /* Push parsers. */ [#]define YYPUSH b4_push_flag +/* Pull parsers. */ +[#]define YYPULL b4_pull_flag + /* Using locations. */ [#]define YYLSP_NEEDED b4_locations_flag ]) Index: data/push.c =================================================================== RCS file: /sources/bison/bison/data/push.c,v retrieving revision 1.26 diff -p -u -r1.26 push.c --- data/push.c 26 Dec 2006 18:09:53 -0000 1.26 +++ data/push.c 26 Dec 2006 23:47:57 -0000 @@ -63,32 +63,6 @@ b4_locations_if([, [[YYLTYPE *], [&yyllo m4_ifdef([b4_lex_param], b4_lex_param))) -# b4_generate_macro_args([A], [B], [C], ...) -# --------------------------------------------------- -# Generate a comma-delimited list whose size is equal to the number of input -# arguments and whose form is: -# -# YYARG1, YYARG2, YYARG3, ... -# -# No argument should be the empty string except A in the special invocation -# b4_generate_macro_args(), which generates an empty string. -m4_define([b4_generate_macro_args], -[m4_if([$1], [], [], [$#], [1], [[YYARG1]], - [b4_generate_macro_args(m4_shift($@)), [YYARG$#]])]) - - -# b4_parenthesize([A], [B], [C], ...) -# --------------------------------------------------- -# Convert arguments to the form: -# -# (A), (B), (C), ... -# -# No argument should be the empty string except A in the special invocation -# b4_parenthesize(), which generates an empty string. -m4_define([b4_parenthesize], -[m4_if([$1], [], [], [$#], [1], [[($1)]], - [($1), b4_parenthesize(m4_shift($@))])]) - ## ------------ ## ## Data Types. ## ## ------------ ## @@ -175,20 +149,20 @@ b4_copyright([Skeleton implementation fo ]b4_identification m4_if(b4_prefix, [yy], [], -[/* Substitute the variable and function names. */ -]b4_push_if([#define yypush_parse b4_prefix[]push_parse -#define yypull_parse b4_prefix[]pull_parse -#define yypstate_new b4_prefix[]pstate_new -#define yypstate_delete b4_prefix[]pstate_delete -#define yypstate b4_prefix[]pstate], -[#define yyparse b4_prefix[]parse])[ -#define yylex b4_prefix[]lex -#define yyerror b4_prefix[]error -#define yylval b4_prefix[]lval -#define yychar b4_prefix[]char -#define yydebug b4_prefix[]debug -#define yynerrs b4_prefix[]nerrs -b4_locations_if([#define yylloc b4_prefix[]lloc])])[ +[[/* Substitute the variable and function names. */ +]b4_pull_if([[#define yyparse ]b4_prefix[parse +]])b4_push_if([[#define yypush_parse ]b4_prefix[push_parse +]b4_pull_if([[#define yypull_parse ]b4_prefix[pull_parse +]])[#define yypstate_new ]b4_prefix[pstate_new +#define yypstate_delete ]b4_prefix[pstate_delete +#define yypstate ]b4_prefix[pstate +]])[#define yylex ]b4_prefix[lex +#define yyerror ]b4_prefix[error +#define yylval ]b4_prefix[lval +#define yychar ]b4_prefix[char +#define yydebug ]b4_prefix[debug +#define yynerrs ]b4_prefix[nerrs +]b4_locations_if([[#define yylloc ]b4_prefix[lloc]])])[ /* Copy the first part of user declarations. */ ]b4_user_pre_prologue[ @@ -248,20 +222,20 @@ b4_push_if([[#ifndef YYPUSH_DECLS struct yypstate; typedef struct yypstate yypstate; enum { YYPUSH_MORE = 4 }; -# define yyparse(]b4_generate_macro_args(b4_parse_param))[ yypull_parse (0, &yylex]m4_ifset([b4_parse_param], [, b4_parenthesize(b4_generate_macro_args(b4_parse_param))])[) -]b4_c_function_decl([[yypstate_new]], [[yypstate *]], [[[void]], []]) -b4_c_function_decl([[yypstate_delete]], [[void]], - [[[yypstate *yyps]], [[yyps]]]) -b4_c_function_decl([[yypush_parse]], [[int]], + +]b4_pull_if([b4_c_function_decl([[yyparse]], [[int]], b4_parse_param) +])b4_c_function_decl([[yypush_parse]], [[int]], [[[yypstate *yyps]], [[yyps]]]b4_pure_if([, [[[int yypushed_char]], [[yypushed_char]]], [[[YYSTYPE const *yypushed_val]], [[yypushed_val]]]b4_locations_if([, [[[YYLTYPE const *yypushed_loc]], [[yypushed_loc]]]])])m4_ifset([b4_parse_param], [, b4_parse_param])) -b4_c_function_decl([[yypull_parse]], [[int]], - [[[yypstate *yyps]], [[yyps]]], - [[[int (*yylexp)(]b4_c_ansi_formals(b4_lex_param)[)]], [[yylexp]]]m4_ifset([b4_parse_param], [, - b4_parse_param]))[ +b4_pull_if([b4_c_function_decl([[yypull_parse]], [[int]], + [[[yypstate *yyps]], [[yyps]]]m4_ifset([b4_parse_param], [, + b4_parse_param]))]) +b4_c_function_decl([[yypstate_new]], [[yypstate *]], [[[void]], []]) +b4_c_function_decl([[yypstate_delete]], [[void]], + [[[yypstate *yyps]], [[yyps]]])[ #endif ]]) m4_ifdef([b4_provides], @@ -1087,9 +1061,14 @@ b4_push_if( int yynew; }; +]b4_pull_if([b4_c_function_def([[yyparse]], [[int]], b4_parse_param)[ +{ + return yypull_parse (0]m4_ifset([b4_parse_param], + [[, ]b4_c_args(b4_parse_param)])[); +} + ]b4_c_function_def([[yypull_parse]], [[int]], - [[[yypstate *yyps]], [[yyps]]], - [[[int (*yylexp)(]b4_c_ansi_formals(b4_lex_param)[)]], [[yylexp]]]m4_ifset([b4_parse_param], [, + [[[yypstate *yyps]], [[yyps]]]m4_ifset([b4_parse_param], [, b4_parse_param]))[ { int yystatus; @@ -1103,7 +1082,7 @@ b4_push_if( else yyps_local = yyps; do { - yychar = ]b4_c_function_call([yylexp], [int], b4_lex_param)[; + yychar = YYLEX; yystatus = yypush_parse (yyps_local]b4_pure_if([[, yychar, &yylval]b4_locations_if([[, &yylloc]])])m4_ifset([b4_parse_param], [, b4_c_args(b4_parse_param)])[); } while (yystatus == YYPUSH_MORE); @@ -1111,7 +1090,7 @@ b4_push_if( yypstate_delete (yyps_local); return yystatus; } - +]])[ /* Initialize the parser data structure. */ ]b4_c_function_def([[yypstate_new]], [[yypstate *]])[ { @@ -1703,21 +1682,20 @@ b4_push_if([[#ifndef YYPUSH_DECLS struct ]b4_prefix[pstate; typedef struct ]b4_prefix[pstate ]b4_prefix[pstate; enum { YYPUSH_MORE = 4 }; -# define ]b4_prefix[parse(]b4_generate_macro_args(b4_parse_param)) b4_prefix[pull_parse (0, &]b4_prefix[lex]m4_ifset([b4_parse_param], [, b4_parenthesize(b4_generate_macro_args(b4_parse_param))])[) -]b4_c_function_decl([b4_prefix[pstate_new]], [b4_prefix[pstate *]], - [[[void]], []]) -b4_c_function_decl([b4_prefix[pstate_delete]], [[void]], - [[b4_prefix[pstate *yyps]], [[yyps]]]) -b4_c_function_decl([b4_prefix[push_parse]], [[int]], +]b4_pull_if([b4_c_function_decl([b4_prefix[parse]], [[int]], b4_parse_param) +])b4_c_function_decl([b4_prefix[push_parse]], [[int]], [[b4_prefix[pstate *yyps]], [[yyps]]]b4_pure_if([, [[[int yypushed_char]], [[yypushed_char]]], [[[YYSTYPE const *yypushed_val]], [[yypushed_val]]]b4_locations_if([, [[[YYLTYPE const *yypushed_loc]], [[yypushed_loc]]]])])m4_ifset([b4_parse_param], [, b4_parse_param])) -b4_c_function_decl([b4_prefix[pull_parse]], [[int]], - [[b4_prefix[pstate *yyps]], [[yyps]]], - [[[int (*yylexp)(]b4_c_ansi_formals(b4_lex_param)[)]], [[yylexp]]]m4_ifset([b4_parse_param], [, - b4_parse_param]))[ +b4_pull_if([b4_c_function_decl([b4_prefix[pull_parse]], [[int]], + [[b4_prefix[pstate *yyps]], [[yyps]]]m4_ifset([b4_parse_param], [, + b4_parse_param]))]) +b4_c_function_decl([b4_prefix[pstate_new]], [b4_prefix[pstate *]], + [[[void]], []]) +b4_c_function_decl([b4_prefix[pstate_delete]], [[void]], + [[b4_prefix[pstate *yyps]], [[yyps]]])[ #endif ]]) m4_ifdef([b4_provides], Index: etc/bench.pl.in =================================================================== RCS file: /sources/bison/bison/etc/bench.pl.in,v retrieving revision 1.1 diff -p -u -r1.1 bench.pl.in --- etc/bench.pl.in 26 Dec 2006 18:09:53 -0000 1.1 +++ etc/bench.pl.in 26 Dec 2006 23:47:58 -0000 @@ -349,8 +349,8 @@ sub bench_grammar ($) "yacc.c-pull-pure" => '%pure-parser', "push.c-pull-impure" => '%skeleton "push.c"', "push.c-pull-pure" => '%skeleton "push.c" %pure-parser', - "push.c-push-impure" => '%skeleton "push.c" %push-parser', - "push.c-push-pure" => '%skeleton "push.c" %push-parser %pure-parser', + "push.c-push-impure" => '%skeleton "push.c" %push-pull-parser', + "push.c-push-pure" => '%skeleton "push.c" %push-pull-parser %pure-parser', ); my %bench; Index: src/getargs.c =================================================================== RCS file: /sources/bison/bison/src/getargs.c,v retrieving revision 1.87 diff -p -u -r1.87 getargs.c --- src/getargs.c 19 Dec 2006 00:34:36 -0000 1.87 +++ src/getargs.c 26 Dec 2006 23:47:58 -0000 @@ -59,6 +59,7 @@ bool error_verbose = false; bool nondeterministic_parser = false; bool glr_parser = false; +bool pull_parser = true; bool pure_parser = false; bool push_parser = false; Index: src/getargs.h =================================================================== RCS file: /sources/bison/bison/src/getargs.h,v retrieving revision 1.37 diff -p -u -r1.37 getargs.h --- src/getargs.h 19 Dec 2006 00:34:36 -0000 1.37 +++ src/getargs.h 26 Dec 2006 23:47:58 -0000 @@ -52,13 +52,16 @@ extern bool error_verbose; extern bool glr_parser; +/* PULL_PARSER is true if should generate a pull parser. */ + +extern bool pull_parser; + /* PURE_PARSER is true if should generate a parser that is all pure and reentrant. */ extern bool pure_parser; -/* PUSH_PARSER is true if should generate a parser that is capable of being - called asynchronously. Is must be pure and reentrant. */ +/* PUSH_PARSER is true if should generate a push parser. */ extern bool push_parser; Index: src/output.c =================================================================== RCS file: /sources/bison/bison/src/output.c,v retrieving revision 1.261 diff -p -u -r1.261 output.c --- src/output.c 19 Dec 2006 00:34:36 -0000 1.261 +++ src/output.c 26 Dec 2006 23:47:58 -0000 @@ -593,6 +593,7 @@ prepare (void) MUSCLE_INSERT_BOOL ("glr_flag", glr_parser); MUSCLE_INSERT_BOOL ("locations_flag", locations_flag); MUSCLE_INSERT_BOOL ("nondeterministic_flag", nondeterministic_parser); + MUSCLE_INSERT_BOOL ("pull_flag", pull_parser); MUSCLE_INSERT_BOOL ("pure_flag", pure_parser); MUSCLE_INSERT_BOOL ("push_flag", push_parser); MUSCLE_INSERT_BOOL ("synclines_flag", !no_lines_flag); Index: src/parse-gram.y =================================================================== RCS file: /sources/bison/bison/src/parse-gram.y,v retrieving revision 1.103 diff -p -u -r1.103 parse-gram.y --- src/parse-gram.y 25 Dec 2006 17:25:39 -0000 1.103 +++ src/parse-gram.y 26 Dec 2006 23:47:59 -0000 @@ -156,6 +156,8 @@ static int current_prec = 0; PERCENT_PROVIDES "%provides" PERCENT_PURE_PARSER "%pure-parser" PERCENT_PUSH_PARSER "%push-parser" + PERCENT_PUSH_PULL_PARSER + "%push-pull-parser" PERCENT_REQUIRE "%require" PERCENT_REQUIRES "%requires" PERCENT_SKELETON "%skeleton" @@ -257,7 +259,8 @@ prologue_declaration: | "%output" "=" STRING { spec_outfile = $3; } /* deprecated */ | "%parse-param" "{...}" { add_param ("parse_param", $2, @2); } | "%pure-parser" { pure_parser = true; } -| "%push-parser" { push_parser = true; } +| "%push-parser" { push_parser = true; pull_parser = false; } +| "%push-pull-parser" { push_parser = true; pull_parser = true; } | "%require" STRING { version_check (&@2, $2); } | "%skeleton" STRING { skeleton_arg ($2, 1, &@1); } | "%token-table" { token_table_flag = true; } Index: src/scan-gram.l =================================================================== RCS file: /sources/bison/bison/src/scan-gram.l,v retrieving revision 1.112 diff -p -u -r1.112 scan-gram.l --- src/scan-gram.l 19 Dec 2006 00:34:36 -0000 1.112 +++ src/scan-gram.l 26 Dec 2006 23:47:59 -0000 @@ -189,7 +189,8 @@ splice (\\[ \f\t\v]*\n)* "%printer" return PERCENT_PRINTER; "%provides" return PERCENT_PROVIDES; "%pure"[-_]"parser" return PERCENT_PURE_PARSER; - "%push"[-_]"parser" return PERCENT_PUSH_PARSER; + "%push-parser" return PERCENT_PUSH_PARSER; + "%push-pull-parser" return PERCENT_PUSH_PULL_PARSER; "%require" return PERCENT_REQUIRE; "%requires" return PERCENT_REQUIRES; "%right" return PERCENT_RIGHT; Index: tests/calc.at =================================================================== RCS file: /sources/bison/bison/tests/calc.at,v retrieving revision 1.101 diff -p -u -r1.101 calc.at --- tests/calc.at 25 Dec 2006 18:21:52 -0000 1.101 +++ tests/calc.at 26 Dec 2006 23:47:59 -0000 @@ -562,7 +562,7 @@ AT_CHECK_CALC_LALR([%yacc]) AT_CHECK_CALC_LALR([%error-verbose]) AT_CHECK_CALC_LALR([%pure-parser %locations]) -AT_CHECK_CALC_LALR([%push-parser %pure-parser %locations %skeleton "push.c"]) +AT_CHECK_CALC_LALR([%push-pull-parser %pure-parser %locations %skeleton "push.c"]) AT_CHECK_CALC_LALR([%error-verbose %locations]) AT_CHECK_CALC_LALR([%error-verbose %locations %defines %name-prefix "calc" %verbose %yacc]) @@ -571,7 +571,7 @@ AT_CHECK_CALC_LALR([%debug]) AT_CHECK_CALC_LALR([%error-verbose %debug %locations %defines %name-prefix "calc" %verbose %yacc]) AT_CHECK_CALC_LALR([%pure-parser %error-verbose %debug %locations %defines %name-prefix "calc" %verbose %yacc]) -AT_CHECK_CALC_LALR([%push-parser %pure-parser %error-verbose %debug %locations %defines %name-prefix "calc" %verbose %yacc %skeleton "push.c"]) +AT_CHECK_CALC_LALR([%push-pull-parser %pure-parser %error-verbose %debug %locations %defines %name-prefix "calc" %verbose %yacc %skeleton "push.c"]) AT_CHECK_CALC_LALR([%pure-parser %error-verbose %debug %locations %defines %name-prefix "calc" %verbose %yacc %parse-param {semantic_value *result} %parse-param {int *count}]) Index: tests/headers.at =================================================================== RCS file: /sources/bison/bison/tests/headers.at,v retrieving revision 1.16 diff -p -u -r1.16 headers.at --- tests/headers.at 25 Dec 2006 18:21:52 -0000 1.16 +++ tests/headers.at 26 Dec 2006 23:47:59 -0000 @@ -94,9 +94,7 @@ AT_DATA_GRAMMAR([input.y], #include <stdio.h> #include <stdlib.h> -int my_lex (void); - -int +static int my_lex (void) { return EOF; @@ -122,8 +120,6 @@ YYLTYPE *my_llocp = &my_lloc; #ifndef YYPUSH_DECLS int my_parse (void); -#else -int my_lex (void); #endif int Index: tests/input.at =================================================================== RCS file: /sources/bison/bison/tests/input.at,v retrieving revision 1.65 diff -p -u -r1.65 input.at --- tests/input.at 25 Dec 2006 18:21:52 -0000 1.65 +++ tests/input.at 26 Dec 2006 23:47:59 -0000 @@ -470,7 +470,7 @@ char quote[] = "@:>@@:>@,"; %{ static void yyerror (const char *s); -int yylex (void); +static int yylex (void); %} %type <ival> '@<:@' @@ -503,7 +503,7 @@ value_as_yystype (value val) return res; } -int +static int yylex (void) { static char const input[] = "@<:@[EMAIL PROTECTED]@[EMAIL PROTECTED]@&[EMAIL PROTECTED] @@ -531,8 +531,6 @@ AT_DATA([main.c], #ifndef YYPUSH_DECLS int yyparse (void); -#else -int yylex (void); #endif int Index: tests/local.at =================================================================== RCS file: /sources/bison/bison/tests/local.at,v retrieving revision 1.21 diff -p -u -r1.21 local.at --- tests/local.at 25 Dec 2006 17:25:39 -0000 1.21 +++ tests/local.at 26 Dec 2006 23:47:59 -0000 @@ -59,7 +59,7 @@ m4_pushdef([AT_LOCATION_IF], m4_pushdef([AT_PURE_IF], [m4_bmatch([$3], [%pure-parser], [$1], [$2])]) m4_pushdef([AT_PUSH_IF], -[m4_bmatch([$3], [%push-parser], [$1], [$2])]) +[m4_bmatch([$3], [%push-parser\|%push-pull-parser], [$1], [$2])]) m4_pushdef([AT_YACC_OR_PUSH_IF], [AT_YACC_IF([$1], [AT_PUSH_IF([$1], [$2])])]) m4_pushdef([AT_PURE_AND_LOC_IF],
