Hi Frank, Hi all, > Le 16 déc. 2018 à 10:02, Frank Heckenbach <f.heckenb...@fh-soft.de> a écrit : > > So to make it safe, we might need something like this: > > static inline > symbol_type > make_symbol (token_type type, b4_locations_if([const location_type& l, ])T&& > v); > > auto-generated for each semantic type T of any token (plus one > without the "v" parameter for untyped tokens) that checks (at > runtime) the "type" parameter against the (statically known) valid > token types for T.
I like this idea. I have a draft for it in my repo, as "make-symbol". Please, try it and report about it. There are a few issues: - make_symbol will collide if the user has a token named symbol Any idea of a better name? - In the signature of make_symbol, I've had to use int for the token type, instead of the enum token_type, and then to convert the int into token_type. I don't like that, but I probably don't have much of a choice. (A template would be overkill IMHO). - I'm not sure I should emit the assert only when parse.assert was set, maybe we should always do it. After all, if the user does not want the check, he can pass -DNDEBUG. parse.assert was made to check what Bison generates, more than to check what the user does. It's used like this: yy::parser::symbol_type yylex () { static char const input[] = "12"; static size_t toknum = 0; int res = input[toknum++]; if (res == '1') return yy::parser::make_symbol (res, std::make_unique<int> (10)); else if (res == '2') return yy::parser::make_symbol (res, std::make_pair<int, int> (21, 22)); else return yy::parser::make_symbol (res); } The generated code looks like this (I had a token named m4_define to make sure everything was properly quoted): Input: %token <pair<int, int>> FOO '+' BAR "BAR" m4_define %nterm <pair<int, int>> exp exp1 Output: inline parser::symbol_type parser::make_symbol (int tok) { assert (tok == 0); return symbol_type (token_type (tok)); } inline parser::symbol_type parser::make_symbol (int tok, pair<int, int> v) { assert (tok == token::FOO || tok == 43 || tok == token::BAR || tok == token::m4_define); return symbol_type (token_type (tok), std::move (v)); } commit 1f86c28ea2ff69f68a569568ad6890dee3ac9684 Author: Akim Demaille <akim.demai...@gmail.com> Date: Tue Dec 18 07:16:55 2018 +0100 c++: provide symbol constructors per type On %token <int> FOO BAR we currently generate make_FOO(int) and make_bar(int). However, in order to factor their scanners, some users would also like to have make_symbol(tok, int), where tok is FOO or BAR. To ensure type safety, add assertions that do check that value type and token type match. Bind this assertion to the parse.assert %define variable. Suggested by Frank Heckenbach. http://lists.gnu.org/archive/html/bug-bison/2018-12/msg00034.html Should also match expectations from Аскар Сафин. http://lists.gnu.org/archive/html/bug-bison/2018-12/msg00023.html * data/variant.hh: Use b4_token_visible_if where applicable. (_b4_type_constructor_declare, _b4_type_constructor_define): New. Use them. diff --git a/data/variant.hh b/data/variant.hh index 9696cf7b..75811b0d 100644 --- a/data/variant.hh +++ b/data/variant.hh @@ -335,6 +335,16 @@ m4_define([b4_symbol_value_template], ## ------------- ## +# _b4_includes_tokens(SYMBOL-NUM...) +# ---------------------------------- +# Expands to non-empty iff one of the SYMBOL-NUM denotes +# a token. +m4_define([_b4_is_token], + [b4_symbol_if([$1], [is_token], [1])]) +m4_define([_b4_includes_tokens], + [m4_map([_b4_is_token], [$@])]) + + # _b4_symbol_constructor_declare(SYMBOL-NUM) # ------------------------------------------ # Declare make_SYMBOL for SYMBOL-NUM. Use at class-level. @@ -358,12 +368,38 @@ b4_join(b4_symbol_if([$1], [has_type], ])]) +# _b4_type_constructor_declare(SYMBOL-NUM...) +# ------------------------------------------- +# Declare a unique make_symbol for all the SYMBOL-NUM (they +# have the same type). Use at class-level. +m4_define([_b4_type_constructor_declare], +[m4_ifval(_b4_includes_tokens($@), +[#if 201103L <= YY_CPLUSPLUS + static + symbol_type + make_symbol (dnl +b4_join([int tok], + b4_symbol_if([$1], [has_type], + [b4_symbol([$1], [type]) v]), + b4_locations_if([location_type l]))); +#else + static + symbol_type + make_symbol (dnl +b4_join([int tok], + b4_symbol_if([$1], [has_type], + [const b4_symbol([$1], [type])& v]), + b4_locations_if([const location_type& l]))); +#endif +])]) + + # b4_symbol_constructor_declare # ----------------------------- -# Declare symbol constructors for all the value types. -# Use at class-level. +# Declare symbol constructors. Use at class-level. m4_define([b4_symbol_constructor_declare], [ // Symbol constructors declarations. +b4_type_foreach([_b4_type_constructor_declare]) b4_symbol_foreach([_b4_symbol_constructor_declare])]) @@ -401,6 +437,50 @@ b4_join(b4_symbol_if([$1], [has_type], ])]) +# _b4_type_constructor_define(SYMBOL-NUM...) +# ------------------------------------------ +# Declare a unique make_symbol for all the SYMBOL-NUM (they +# have the same type). Use at class-level. +m4_define([_b4_type_clause], +[b4_symbol_if([$1], [is_token], + [b4_symbol_if([$1], [has_id], + [tok == token::b4_symbol([$1], [id])], + [tok == b4_symbol([$1], [user_number])])])]) + +m4_define([_b4_type_constructor_define], +[m4_ifval(_b4_includes_tokens($@), +[#if 201103L <= YY_CPLUSPLUS + inline + b4_parser_class_name::symbol_type + b4_parser_class_name::make_symbol (dnl +b4_join([int tok], + b4_symbol_if([$1], [has_type], + [b4_symbol([$1], [type]) v]), + b4_locations_if([location_type l]))) + {b4_parse_assert_if([ + assert (m4_join([ || ], m4_map_sep([_b4_type_clause], [, ], [$@])));])[ + return symbol_type (]b4_join([token_type (tok)], + b4_symbol_if([$1], [has_type], [std::move (v)]), + b4_locations_if([std::move (l)]))); + } +#else + inline + b4_parser_class_name::symbol_type + b4_parser_class_name::make_symbol (dnl +b4_join([int tok], + b4_symbol_if([$1], [has_type], + [const b4_symbol([$1], [type])& v]), + b4_locations_if([const location_type& l]))) + {b4_parse_assert_if([ + assert (m4_join([ || ], m4_map_sep([_b4_type_clause], [, ], [$@])));])[ + return symbol_type (]b4_join([token_type (tok)], + b4_symbol_if([$1], [has_type], [v]), + b4_locations_if([l]))); + } +#endif +])]) + + # b4_basic_symbol_constructor_declare(SYMBOL-NUM) # ----------------------------------------------- # Generate a constructor declaration for basic_symbol from given type. @@ -451,4 +531,5 @@ m4_define([b4_basic_symbol_constructor_define], # Define the overloaded versions of make_symbol for all the value types. m4_define([b4_symbol_constructor_define], [ // Implementation of make_symbol for each symbol type. +b4_type_foreach([_b4_type_constructor_define]) b4_symbol_foreach([_b4_symbol_constructor_define])]) diff --git a/tests/types.at b/tests/types.at index 2289f751..adc8ec6a 100644 --- a/tests/types.at +++ b/tests/types.at @@ -327,6 +327,26 @@ m4_foreach([b4_skel], [[yacc.c], [glr.c], [lalr1.cc], [glr.cc]], [10, 21, 22], [AT_REQUIRE_CXX_STD(14, [echo "$at_std not supported"; continue])]) + # Type-based token constructors on move-only types, and types with commas. + AT_TEST([%skeleton "]b4_skel[" + %code requires { #include <memory> } + %define api.value.type variant + %define api.token.constructor], + [[%token <std::unique_ptr<int>> '1'; + %token <std::pair<int, int>> '2';]], + ['1' '2' { std::cout << *$1 << ", " + << $2.first << ", " + << $2.second << '\n'; }], + ["12"], + [[if (res == '1') + return yy::parser::make_symbol ('1', std::make_unique<int> (10)); + else if (res == '2') + return yy::parser::make_symbol ('2', std::make_pair<int, int> (21, 22)); + else + return yy::parser::make_symbol (0)]], + [10, 21, 22], + [AT_REQUIRE_CXX_STD(14, [echo "$at_std not supported"; continue])]) + ]) ])