original string is ascii or utf-8 or other. The cost during lexing is essentially zero.
I am not so sure when it comes to SIMD lexing. I think the specified behaviour should be done in a way which encourage later optimizations.
original string is ascii or utf-8 or other. The cost during lexing is essentially zero.
I am not so sure when it comes to SIMD lexing. I think the specified behaviour should be done in a way which encourage later optimizations.