On Thu, 17 Mar 2016 at 07:56 Guido van Rossum <gu...@python.org> wrote:
> On Thu, Mar 17, 2016 at 5:04 AM, Serhiy Storchaka <storch...@gmail.com> > wrote: > >> Should we recommend that everyone use tokenize.detect_encoding()? > > > > Likely. However the interface of tokenize.detect_encoding() is not very > > simple. > > I just found that out yesterday. You have to give it a readline() > function, which is cumbersome if all you have is a (byte) string and > you don't want to split it on lines just yet. And the readline() > function raises SyntaxError when the encoding isn't right. I wish > there were a lower-level helper that just took a line and told you > what the encoding in it was, if any. Then the rest of the logic can be > handled by the caller (including the logic of trying up to two lines). > Since this is for mypy my guess is you only want to know the encoding, but if you're simply trying to decode bytes of syntax then importilb.util.decode_source() will handle that for you.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
