Hi,
I'd like to document the parser in two stages if that's OK. The first
part will be everything but the example. The second part will contain an
example if that's considered necessary.
Joel, thanks for all your quick responses! They allowed me to get the
most out of my time, when I had some free time!
Here's a first attempt at a documentation patch. Don't laugh, I am
terrible at english! Please provide any necessary feedback.
BTW, I have trouble writing ChangeLog's for texinfo files. It's more
difficult to understand what the section is than with c code. Sorry if
this is all wrong.
+2007-07-30 Bob Rossi <[EMAIL PROTECTED]>
+
+ * doc/bison.texinfo (Push Decl): Document the push parser.
+ (Table of Symbols): Ditto.
+ (Pure Decl): Ditto.
+ (Declaration Summary): Ditto.
+ (Push Parser Function, Pull Parser Function, Parser Create Function,
+ Parser Delete Function): Add new push parser symbols.
+ (Bison symbols): Document push-parser, push-pull-parser, yypush_parse,
+ yypull_parse, yypstate_new and yypstate_delete.
Bob Rossi
Index: doc/bison.texinfo
===================================================================
RCS file: /sources/bison/bison/doc/bison.texinfo,v
retrieving revision 1.232
diff -u -r1.232 bison.texinfo
--- doc/bison.texinfo 26 May 2007 20:08:18 -0000 1.232
+++ doc/bison.texinfo 31 Jul 2007 02:26:07 -0000
@@ -224,6 +224,7 @@
* Expect Decl:: Suppressing warnings about parsing conflicts.
* Start Decl:: Specifying the start symbol.
* Pure Decl:: Requesting a reentrant parser.
+* Push Decl:: Requesting a push parser.
* Decl Summary:: Table of all Bison declarations.
Parser C-Language Interface
@@ -3978,6 +3979,7 @@
* Expect Decl:: Suppressing warnings about parsing conflicts.
* Start Decl:: Specifying the start symbol.
* Pure Decl:: Requesting a reentrant parser.
+* Push Decl:: Requesting a push parser.
* Decl Summary:: Table of all Bison declarations.
@end menu
@@ -4510,8 +4512,9 @@
@code{yylloc} become local variables in @code{yyparse}, and a different
calling convention is used for the lexical analyzer function
@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
-Parsers}, for the details of this. The variable @code{yynerrs} also
-becomes local in @code{yyparse} (@pxref{Error Reporting, ,The Error
+Parsers}, for the details of this. The variable @code{yynerrs}
+becomes local in @code{yyparse} in pull mode but it becomes a member
+of yypstate in push mode. (@pxref{Error Reporting, ,The Error
Reporting Function @code{yyerror}}). The convention for calling
@code{yyparse} itself is unchanged.
@@ -4519,6 +4522,99 @@
You can generate either a pure parser or a nonreentrant parser from any
valid grammar.
[EMAIL PROTECTED] Push Decl
[EMAIL PROTECTED] A Push Parser
[EMAIL PROTECTED] push parser
[EMAIL PROTECTED] push parser
[EMAIL PROTECTED] %push-parser
+
+A pull parser is called once with a given amount of input and it takes
+control (blocks) until all it's input is completely parsed. A push parser,
+on the other hand, is called each time some new input is made available.
+
+A pull parser can parse it's input faster, but it must have all the
+input up front and it blocks while it is parsing. The push parser
+generally takes longer to parse the same amount of data, but it can
+be told about input while the application is receiving it.
+The push parser is typically useful when the parser is part of a
+main event loop and it is important for the event loop to be triggered
+within a certain time period. This is often the case with a GUI
+application.
+
+Normally, Bison generates a pull parser. The Bison declaration
[EMAIL PROTECTED] says that you want the parser to be a push parser.
+It looks like this:
+
[EMAIL PROTECTED]
+%push-parser
[EMAIL PROTECTED] example
+
+When a push parser is selected, Bison will generate some new symbols in
+the generated parser. @code{yypstate} is a structure that the generated
+parser uses to store the parsers state. @code{yypstate_new} is the
+function that will create a new parser instance. @code{yypstate_delete}
+will free the resources associated with the corresponding parser instance.
+Finally, @code{yypush_parse} is the function that should be called whenever a
+token is available to provide the parser. A trivial example
+of using a push parser would look like this:
+
[EMAIL PROTECTED]
+int yystatus;
+struct yypstate *yyps = yypstate_new ();
+do @{
+ yychar = yylex ();
+ yystatus = yypush_parse (yyps);
[EMAIL PROTECTED] while (yystatus == YYPUSH_MORE);
+yypstate_delete (yyps);
[EMAIL PROTECTED] example
+
+It is acceptable to have many parser instances, of the same type of parser,
+in memory at the same time. However, in order to ensure that the parsers
+are reentrant, you must provide the @code{pure-parser} declaration in the
+grammar. @xref{Pure Decl, ,A Pure (Reentrant) Parser}. When this
+is done, the @code{yychar} variable becomes a local variable in the
[EMAIL PROTECTED] function. In order to allow the the next token to be
+passed to the @code{yypush_parse} function, its signature is changed to
+accept the next token as a parameter. A reentrant push parser example
+would thus look like this:
+
[EMAIL PROTECTED]
+int yystatus;
+struct yypstate *yyps = yypstate_new ();
+do @{
+ yystatus = yypush_parse (yyps, yylex ());
[EMAIL PROTECTED] while (yystatus == YYPUSH_MORE);
+yypstate_delete (yyps);
[EMAIL PROTECTED] example
+
+That's it. Simply pass the next token into the @code{yypush_parse} function
+as a parameter.
+
+Bison also supports both the push parser interface along with the pull parser
+interface in the same generated parser. In order to get this functionality,
+you should provide the grammar with the @code{%push-pull-parser} declaration.
+Doing this will create all of the symbols mentioned earlier along with the
+two extra symbols, @code{yyparse} and @code{yypull_parse}. @code{yyparse}
+can be used exactly as it normally would be used. However, the user should
+not that it is implemented in the generated parser by calling
[EMAIL PROTECTED] This makes the @code{yyparse} function that is generated
+with the @code{%push-pull-parser} declaration slower than the normal
[EMAIL PROTECTED] function. If the user calls the @code{yypull_parse} function
+it will parse the rest of the input stream. It is possible to
+yypush_parse tokens to select a subgrammar and then yypull_parse the rest
+of the input stream. If you would like to switch back and forth between
+between parsing styles, you would have to write your own yypull_parse function
+that knows when to quit looking for input.
+
+Adding the @code{pure-parser} declaration does exactly the same thing to the
+generated parser with @code{%push-pull-parser} as it did for
[EMAIL PROTECTED]
+
+When the @code{%push-parser} or @code{%push-pull-parser} declaration is used
+then it is important to understand that all references to the @code{yyparse}
+function in this manual corresponds to the @code{yypush_parser} function
+unless otherwise stated.
+
@node Decl Summary
@subsection Bison Declaration Summary
@cindex Bison declaration summary
@@ -4797,10 +4893,12 @@
in C parsers
is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
@code{yylval}, @code{yychar}, @code{yydebug}, and
-(if locations are used) @code{yylloc}. For example, if you use
[EMAIL PROTECTED] "c_"}, the names become @code{c_parse}, @code{c_lex},
-and so on. In C++ parsers, it is only the surrounding namespace which is
-named @var{prefix} instead of @samp{yy}.
+(if locations are used) @code{yylloc}. If you use a push parser,
+yypush_parse, yypull_parse, yypstate, yypstate_new and yypstate_delete will
+also be renamed. For example, if you use @samp{%name-prefix "c_"}, the
+names become @code{c_parse}, @code{c_lex}, and so on. In C++ parsers,
+it is only the surrounding namespace which is named @var{prefix} instead
+of @samp{yy}.
@xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
@end deffn
@@ -4923,8 +5021,11 @@
The precise list of symbols renamed is @code{yyparse}, @code{yylex},
@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
[EMAIL PROTECTED] and @code{yydebug}. For example, if you use @samp{-p c},
-the names become @code{cparse}, @code{clex}, and so on.
[EMAIL PROTECTED] and @code{yydebug}. If you use a push parser,
[EMAIL PROTECTED], @code{yypull_parse}, @code{yypstate},
[EMAIL PROTECTED] and @code{yypstate_delete} will also be renamed.
+For example, if you use @samp{-p c}, the names become @code{cparse},
[EMAIL PROTECTED], and so on.
@strong{All the other variables and macros associated with Bison are not
renamed.} These others are not global; there is no conflict if the same
@@ -4953,6 +5054,10 @@
@menu
* Parser Function:: How to call @code{yyparse} and what it returns.
+* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
+* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
+* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
+* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
* Lexical:: You must supply a function @code{yylex}
which reads tokens.
* Error Reporting:: You must supply a function @code{yyerror}.
@@ -5035,6 +5140,61 @@
exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
@end example
[EMAIL PROTECTED] Push Parser Function
[EMAIL PROTECTED] The Push Parser Function @code{yypush_parse}
[EMAIL PROTECTED] yypush_parse
+
+You call the function @code{yypush_parse} to parse a single token. This
+function is available if either the @code{%push-parser} or
[EMAIL PROTECTED] declaration is used.
[EMAIL PROTECTED] Decl, ,A Push Parser}.
+
[EMAIL PROTECTED] int yypush_parse (yypstate *yyps)
+The value returned by @code{yypush_parse} is the same as for yyparse with the
+following exception. @code{yypush_parse} will return YYPUSH_MORE if more input
+is required to finish parsing the grammar.
[EMAIL PROTECTED] deftypefun
+
[EMAIL PROTECTED] Pull Parser Function
[EMAIL PROTECTED] The Pull Parser Function @code{yypull_parse}
[EMAIL PROTECTED] yypull_parse
+
+You call the function @code{yypull_parse} to parse the rest of the input
+stream. This function is available if either the @code{%push-parser} or
[EMAIL PROTECTED] declaration is used.
[EMAIL PROTECTED] Decl, ,A Push Parser}.
+
[EMAIL PROTECTED] int yypull_parse (yypstate *yyps)
+The value returned by @code{yypull_parse} is the same as for yyparse.
[EMAIL PROTECTED] deftypefun
+
[EMAIL PROTECTED] Parser Create Function
[EMAIL PROTECTED] The Parser Create Function @code{yystate_new}
[EMAIL PROTECTED] yypstate_new
+
+You call the function @code{yypstate_new} to create a new parser instance.
+This function is available if either the @code{%push-parser} or
[EMAIL PROTECTED] declaration is used.
[EMAIL PROTECTED] Decl, ,A Push Parser}.
+
[EMAIL PROTECTED] yypstate *yypstate_new (void)
+The fuction will return a valid parser instance if there was memory available
+or NULL if no memory was avialable.
[EMAIL PROTECTED] deftypefun
+
[EMAIL PROTECTED] Parser Delete Function
[EMAIL PROTECTED] The Parser Delete Function @code{yystate_delete}
[EMAIL PROTECTED] yypstate_delete
+
+You call the function @code{yypstate_delete} to delete a parser instance. This
+function is available if either the @code{%push-parser} or
[EMAIL PROTECTED] declaration is used.
[EMAIL PROTECTED] Decl, ,A Push Parser}.
+
[EMAIL PROTECTED] void yypstate_delete (yypstate *yyps)
+This function will reclaim the memory associate with a parser instance. After
+this call, you shoul no longer attempt to use the parser instance.
[EMAIL PROTECTED] deftypefun
@node Lexical
@section The Lexical Analyzer Function @code{yylex}
@@ -9279,6 +9439,16 @@
@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
@end deffn
[EMAIL PROTECTED] {Directive} %push-parser
+Bison declaration to request a push parser.
[EMAIL PROTECTED] Decl, ,A Push Parser}.
[EMAIL PROTECTED] deffn
+
[EMAIL PROTECTED] {Directive} %push-pull-parser
+Bison declaration to request a push and a pull parser.
[EMAIL PROTECTED] Decl, ,A Push Parser}.
[EMAIL PROTECTED] deffn
+
@deffn {Directive} %require "@var{version}"
Require version @var{version} or higher of Bison. @xref{Require Decl, ,
Require a Version of Bison}.
@@ -9453,7 +9623,8 @@
@deffn {Variable} yynerrs
Global variable which Bison increments each time it reports a syntax error.
-(In a pure parser, it is a local variable within @code{yyparse}.)
+(In a pure parser, it is a local variable within @code{yyparse}. In a
+push parser, it is a member of yypstate.)
@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
@end deffn
@@ -9462,6 +9633,31 @@
parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
@end deffn
[EMAIL PROTECTED] {Function} yypush_parse
+The parser function produced by Bison in push mode; call this function to
+parse a single token. @xref{Push Parser Function, ,The Push Parser Function
[EMAIL PROTECTED]
[EMAIL PROTECTED] deffn
+
[EMAIL PROTECTED] {Function} yypull_parse
+The parser function produced by Bison in push mode; call this function to
+parse the rest of the input stream.
[EMAIL PROTECTED] Parser Function, ,The Pull Parser Function
[EMAIL PROTECTED]
[EMAIL PROTECTED] deffn
+
[EMAIL PROTECTED] {Function} yypstate_new
+The function to create a parser instance, produced by Bison in push mode;
+call this function to create a new parser.
[EMAIL PROTECTED] Create Function, ,The Parser Create Function @code{yypstate_new}}.
[EMAIL PROTECTED] deffn
+
[EMAIL PROTECTED] {Function} yypstate_delete
+The function to delete a parser instance, produced by Bison in push mode;
+call this function to delete the memory associate with a parser.
[EMAIL PROTECTED] Delete Function, ,The Parser Delete Function @code{yypstate_delete}}.
[EMAIL PROTECTED] deffn
+
@deffn {Macro} YYPARSE_PARAM
An obsolete macro for specifying the name of a parameter that
@code{yyparse} should accept. The use of this macro is deprecated, and
