[issue1065986] Fix pydoc crashing on unicode strings
Roundup Robot added the comment: New changeset bf077fc97fdd by R David Murray in branch '2.7': #1065986: Make pydoc handle unicode strings. http://hg.python.org/cpython/rev/bf077fc97fdd -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
R. David Murray added the comment: Committed, thanks Akira. The support for --disable-unicode is not fully tested. I tried running the tests but the _io module wasn't built, so regrtest doesn't work. A command line invocation of pydoc worked fine, though. -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Roundup Robot added the comment: New changeset e57660acc6d4 by R David Murray in branch '2.7': #1065986: add missing error handler in pydoc unicode fix. http://hg.python.org/cpython/rev/e57660acc6d4 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
R. David Murray added the comment: Made some review comments. Looks good in general and it seems like the tests are fairly comprehensive. I haven't tried to run any additional experiments, but I don't see how it could make things worse, since the new code paths will only do something different if unicode objects are actually involved. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Akira Kitada added the comment: Made a few more adjustments to fix things r.david.murray pointed out. -- Added file: http://bugs.python.org/file33316/issue1065986-6.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
R. David Murray added the comment: Benjamin: the patch looks pretty good to me, for fixing the problem of docstrings that are explicitly unicode. But before I go to the trouble of a full review and test, is this a level of change you think is acceptable in 2.7 at this point it its lifecycle? -- nosy: +benjamin.peterson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Akira Kitada added the comment: Added meta charset=utf-8 to html pydoc generates. -- Added file: http://bugs.python.org/file32721/issue1065986-4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Benjamin Peterson added the comment: Okay with me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Éric Araujo added the comment:
LGTM.
One thing: did you mean assertEqual in Lib/test/test_pydoc.py:466:
self.assertTrue(open('pipe').read(), pydoc._encode(doc))
?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1065986
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Akira Kitada added the comment: Good catch. Fixed. -- Added file: http://bugs.python.org/file32738/issue1065986-5.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Akira Kitada added the comment:
Updated the previous patch to test unicode strings in
__{version,date,author,credits}__ don't crash.
--
Added file: http://bugs.python.org/file31832/issue1065986-3.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1065986
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Akira Kitada added the comment: Now we have a working fix for 2.7. Could someone please review the attached patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Changes by Antoine Pitrou pit...@free.fr: -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Akira Kitada added the comment: With this patch applied, the example from issue15791 works fine. $ echo __author__ = u'Michele Orr\xf9' foo.py ./python -c import foo; print foo.__author__; help(foo) Michele Orrù Help on module foo: NAME foo FILE /tmp/cpython/foo.py DATA __author__ = u'Michele Orr\xf9' AUTHOR Michele Orrù -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Akira Kitada added the comment: Attaching a modified version of issue1065986.patch. The differences are: - Added _binstr(), which is str() that works with unicode objects. - Changed getdoc() to return encoded docstrings/comments - Used _binstr() to convert __version__, __date__, __author__ and __credits__ to str -- Added file: http://bugs.python.org/file31793/issue1065986-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Changes by Akira Kitada akit...@gmail.com: -- nosy: +akitada ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
David Barnett added the comment:
I guess it must be more complicated than it looks, because I thought checking
for unicode strings and doing .encode('utf-8') would help at least some cases
without making anything worse.
Anyways, if it's too hard or not worth fixing correctly, couldn't we at least
do something to prevent a crash? Maybe strip out / replace special characters
and try again?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1065986
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Changes by David Barnett davidbarne...@gmail.com: -- nosy: +mu_mind ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
David Barnett added the comment: I just ran into this, and I'd like to communicate how unfortunate it is that it's not a priority to fix this fairly trivial (?) bug. It means there's no way to define a unicode string literal with non-ascii characters that won't crash the builtin help() command. I ran into this with the desktop package (http://pypi.python.org/pypi/desktop) where the only useful documentation right now is the source code and the docstrings. Apparently the author, who has non-ascii characters in his name, did me a favor by using broken encoding on the doc string so that at least I could read everything except for his name in the help. I tried to correct the encoding and found I get a nice traceback instead of help. And to top it all off, googling for things like help unicode docstring and python help ascii codec turns up nothing. I only found this issue once I thought to include pipepager in the search... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
David Barnett added the comment: Also, the resolution is still marked as fixed, which is not correct... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
R. David Murray added the comment: It is not so much that it isn't a priority, as that no one has suggested a working fix that is suitable for 2.7. Do you have a suggestion? -- resolution: fixed - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Martin v. Löwis added the comment:
I fail to see how this patch solves this issue. Taking the example from
issue15791, I still get the traceback of that issue, namely in the line
result = result + self.section('AUTHOR', str(object.__author__))
If __author__ is a unicode object, it's the str call that fails. This is long
before any attempt is made to render the resulting string to an output device.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1065986
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Stefano Taschini tasch...@ieee.org added the comment: Here's my patch, along the lines of the work-around I posted earlier. A few remarks: 1. The modifications in pydoc only touch the four console pagers and the html pager (html.page). 2. A module-wide default encoding is initialized from locale.getpreferredencoding. Pagers that write to a file use the encoding of that file if defined, else they use the module-wide default. The html pager uses ascii. All of them use xml character entity replacement as fall-back. 3. An additional set of tests has been added to test.test_pydoc to verify the behaviour of the modifications. 4. No functionality is broken if Python is built without unicode support. -- Added file: http://bugs.python.org/file25380/issue1065986.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Changes by Éric Araujo mer...@netwok.org: -- nosy: +eric.araujo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Stefano Taschini tasch...@ieee.org added the comment: Shouldn't this be reopened for Python 2.7 ? -- type: - behavior versions: +Python 2.7 -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
R. David Murray rdmur...@bitdance.com added the comment: I don't think so. We aren't promising unicode support in pydoc in 2.x, and it is too late to add it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Stefano Taschini tasch...@ieee.org added the comment: Oh well, in that case I guess we'll have to work around it. Here's the monkey patch I use to overcome this limitation in pydoc, in case others wish to add it to their PYTHONSTARTUP or sitecustomize: def pipepager(text, cmd): Page through text by feeding it to another program. try: import locale except ImportError: encoding = ascii else: encoding = locale.getpreferredencoding() pipe = os.popen(cmd, 'w') try: pipe.write(text.encode(encoding, 'xmlcharrefreplace') if isinstance(text, unicode) else text) pipe.close() except IOError: pass # Ignore broken pipes caused by quitting the pager program. import pydoc pydoc.pipepager = pipepager del pydoc, pipepager -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
R. David Murray rdmur...@bitdance.com added the comment: Hmm. Making it not raise an error while still producing useful output would be acceptable as a bug fix if that's all it takes, I think. -- status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Changes by Stefano Taschini tasch...@ieee.org: -- nosy: +taschini ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Tom Bachmann e_mc...@web.de added the comment: Hello, [this is my first bug report, so I'm sorry if I'm not adhering to some conventions] in what versions of python is this supposed to be fixed? Consider: % python Python 2.7.2+ (default, Nov 30 2011, 19:22:03) [GCC 4.6.2] on linux2 Type help, copyright, credits or license for more information. from pydoc import pager from locale import getpreferredencoding expr = u'\u211a' pager(expr) # error pager(expr.encode(getdefaultencoding())) # works The error is: Traceback (most recent call last): File stdin, line 1, in module File /usr/lib/python2.7/pydoc.py, line 1318, in pager pager(text) File /usr/lib/python2.7/pydoc.py, line 1332, in lambda return lambda text: pipepager(text, os.environ['PAGER']) File /usr/lib/python2.7/pydoc.py, line 1359, in pipepager pipe.write(text) UnicodeEncodeError: 'ascii' codec can't encode character u'\u211a' in position 0: ordinal not in range(128) Best, Tom -- nosy: +ness ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
R. David Murray rdmur...@bitdance.com added the comment: It is fixed in Python3. Apparently Raymond was wrong about it having been fixed earlier (or perhaps he was referring to the unicode being removed from the pydoc __credits__ string). -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1065986] Fix pydoc crashing on unicode strings
Tom Bachmann e_mc...@web.de added the comment: I see. Thank you. On 12.04.2012 16:08, R. David Murray wrote: R. David Murrayrdmur...@bitdance.com added the comment: It is fixed in Python3. Apparently Raymond was wrong about it having been fixed earlier (or perhaps he was referring to the unicode being removed from the pydoc __credits__ string). -- nosy: +r.david.murray ___ Python trackerrep...@bugs.python.org http://bugs.python.org/issue1065986 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1065986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
