I think I originally resisted to the idea (I believe Christian Schoenebeck was the first to suggest it), but there are many advantages in exposing the symbol (internal) numbers:
- custom error messages can use them to decide how to represent a given symbol, or a set of symbols. See my message about custom error messages for examples. (https://lists.gnu.org/archive/html/bison-patches/2020-01/msg00000.html) - we need something similar in uses of yyexpected_tokens. For instance, currently, bistromathic's completion() reads: int ntokens = expected_tokens (line, tokens, YYNTOKENS); [...] for (int i = 0; i < ntokens; ++i) if (tokens[i] == YYTRANSLATE (TOK_VAR)) [...] else if (tokens[i] == YYTRANSLATE (TOK_FUN)) [...] else [...] - now that's it's a compile-time expression, we can easily build static tables, use switch, etc. - some users depended on the ability to get the token number from a symbol to write test cases for their scanners. But Bison 3.5 removed the table this feature depended upon (a reverse yytranslate). Now they can check against the actual symbol number, without having pay (space and time) a conversion. See https://lists.gnu.org/r/bug-bison/2020-01/msg00001.html, and https://lists.gnu.org/archive/html/bug-bison/2020-03/msg00015.html. - it helps us clearly separate the internal symbol numbers from the external token numbers, whose difference is sometimes blurred in the code when values coincide (e.g. "yychar = yytoken = YYEOF"). - it allows us to get rid of ugly macros with inconsistent names such as YYUNDEFTOK and YYTERROR, and to group related definitions together. - similarly it provides a clean access to the $accept symbol (which proves convenient in a current experimentation of mine with several %start symbols). I have left the 'regen' commits, because it's useful to see the impact of these changes. These changes are only for yacc.c, but of course they will be installed in the other skeletons too. And I would also like to move the lone #defines for YYEOF and YYEMPTY into yytokentype, so that we can use strong typing for both yytoken and yychar. And get rid of more macros. I'd be happy to receive comments, as usual. Cheers! Akim Demaille (9): style: comment changes about token numbers yacc.c: introduce an enum that defines the symbol's number regen yacc.c: use yysymbol_code_t instead of int for yytoken yacc.c: also define a symbol number for the empty token regen yacc.c: prefer YYSYMBOL_YYERROR to YYSYMBOL_error regen bistromathic: use symbol numbers instead of YYTRANSLATE TODO | 5 ++ data/skeletons/bison.m4 | 31 ++++++-- data/skeletons/c.m4 | 43 +++++++++-- data/skeletons/yacc.c | 53 +++++++------ examples/c/bistromathic/parse.y | 26 ++++--- src/parse-gram.c | 132 +++++++++++++++++++++++++++----- tests/regression.at | 2 +- 7 files changed, 222 insertions(+), 70 deletions(-) -- 2.25.1
