Re: [pypy-dev] Making the most of internal UTF8

2020-02-27 Thread Matt Billenstein via pypy-dev
On Thu, Feb 27, 2020 at 11:54:45AM -0600, Jerry Spicklemire wrote: > Thanks for all the replies. > Dan, re: > > "I think you mostly don't want u'foo' in 3.x or b'foo' in 2.x" > > Actually, I don't want either, anywhere. > > If UTF8 is used internally, and ASCII is > already UTF8, then it is all

Re: [pypy-dev] Making the most of internal UTF8

2020-02-26 Thread Dan Stromberg
On Wed, Feb 26, 2020 at 10:41 AM Matt Billenstein via pypy-dev < pypy-dev@python.org> wrote: > You can think of 'u' as being the default in python3 where 'b' was the > default in python2 (not ascii) - but most stdlib functions would accept > bytes as strings. > > So in python3, you don't need 'u'

Re: [pypy-dev] Making the most of internal UTF8

2020-02-26 Thread Matt Billenstein via pypy-dev
On Wed, Feb 26, 2020 at 09:08:49AM -0600, Jerry Spicklemire wrote: > In other words, within the range of ASCII characters, the UTF8 representation  > is identical to the ASCII representation. So, does that mean we can put the > 'u'  > and 'b' prefix nightmares behind us? It would help some diehards

Re: [pypy-dev] Making the most of internal UTF8

2020-02-26 Thread Antonio Cuni
To expand Armin's answer, the two most "visible" effects for end users are: - some_unicode.encode('utf-8') is essentially for free (because it is already UTF-8 internally) - some_bytes.decode('utf-8') is very chep (it just needs to check that some_bytes is valid utf-8) ciao, Anto On Wed, Feb 26

Re: [pypy-dev] Making the most of internal UTF8

2020-02-26 Thread Armin Rigo
Hi Jerry, On Wed, 26 Feb 2020 at 16:09, Jerry Spicklemire wrote: > Is there a tutorial about how to best take advantage of PyPy's internal UTF8? For best or for worse, this is only an internal feature. It has no effect for the end user. In particular, Python programs written for PyPy3.6 and fo