> On 17 Dec 2018, at 10:48, Frank Heckenbach <f.heckenb...@fh-soft.de> wrote: > > Hans Åberg wrote: > >>> On 16 Dec 2018, at 15:48, Frank Heckenbach <f.heckenb...@fh-soft.de> wrote: >>> >>> Hans Åberg wrote: >>> >>>> The idea would be that rather than returning the token value, >>>> something that does not need translation. I don't know if that >>>> helps up the static approach, though. >>> >>> Not sure what you mean here. Please elaborate. >> >> I couldn't see the details when I looked at it. I don't use the typed >> parser, but might if it is safe. > > The parser is type safe. This is only about an alternative way of > creating tokens by the lexer, which is also type safe when used > properly (as mine does). It's only about adding an additional safety > net.
Right. >>> I think Akim made it clear enough that he doesn't like the overhead. >>> (I don't mind so much, as I used std::variant in my implementation, >>> but I only have a few parsers to care about.) >> >> In that case, my impression was that he thought he could do without it. > > Well, he can. :) In that case. >>> One might validly say that preventing it is the job of the lexer >>> (and my lexer does so), not Bison's, just like other kinds of >>> undefined or wrong behaviour outside of the generated code, also >>> dynamic token types are a somewhat special usage anyway, so Bison >>> can just do nothing about it, that's fine. >> >> I use the same thing, returning the token value found on a lookup >> table, but I would not want to use the typed parser if I would >> have to add translations for every possibility. The information >> about it is in Bison, therefore it should not be put on the >> writing of the lexer. > > I think we agree here, and that was actually my concern when I > started this thread. I don't want to have to write a separate case > for each token kind in my lexer. Of course, we need a separate case > for each semantic type because that involves a different type in the > constructor/builder call already, but these are relatively few, > compared to token kinds, in my lexers. Might Bison generate a function with a switch statement, generate the right return for the lexer to use? >>> I also suggested an approach in my previous mail with a few more >>> generated functions that help runtime checking. I'd prefer Bison to >>> add them, and then we'd have runtime checking as good as we'd have >>> with std::variant in this respect. >> >> Maybe an option. Akim perhaps haven't used this dynamic token >> lookup. > > I guess he hasn't. But I don't think we need an option. These would > just be additional functions that one can use or not. The with an option would be that those that do not need this feature could use a more optimal variant. >> Those that do might prefer not risking the program to bomb. > > It's not that bad actually. Again, my lexers work fine as is. > I just brought this up because Akim proposed to call the function > "unsafe_..." which I thought was too harsh and proposed > "unchecked_..." -- but by adding the checks, it would be neither > unsafe nor unchecked. :) This worries me. But also having having to use something more complex to be returned by the lexer than a value on the lookup table .
