STINNER Victor <[EMAIL PROTECTED]> added the comment: It looks like the problem of fix_latin.diff is the decoding_state: it's set to STATE_NORMAL whereas current behaviour is to stay in state STATE_RAW.
I wrote another patch which is a mix of case 1 (utf-8: just set tok->encoding) and case 2 (another charset: set tok->enc, tok->encoding and tok>decoding_state): a new case 3 which set enc, encoding but stay a the state STATE_RAW. I don't understand my patch, so review it (twice or more :-D). Using my patch: - compile(...) works - test_shlex.py works - test_pep3120.py - iso.py works Added file: http://bugs.python.org/file11698/tokenizer_iso-8859-1.patch _______________________________________ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3574> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
