[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Zachary Ware added the comment: New changeset b36349a647b2bf8174f0e736a4fc347e92ae204e by Peter Donis in branch 'master': bpo-43049: Use io.IncrementalNewlineDecoder for doctest newline conversion (GH-24359) https://github.com/python/cpython/commit/b36349a647b2bf8174f0e736a4fc347e92ae204e

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Change by Peter Donis : -- pull_requests: +23184 pull_request: https://github.com/python/cpython/pull/24359 ___ Python tracker ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Zachary Ware added the comment: 12 years later, we finally landed your fix :). Thanks for the patch, and for bearing with us (and me in particular for the past couple months). As mentioned in the PR, there is probably opportunity to clean up the cleanup code in the new test a bit, and as

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

miss-islington added the comment: New changeset b05fbe9f374bc660af3c589a1146b2026606d442 by Miss Islington (bot) in branch '3.8': bpo-1812: Fix newline conversion when doctest.testfile loads from a package whose loader has a get_data method (GH-17385)

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

miss-islington added the comment: New changeset 9387678f8a580726aca5f836b2c50456f236ecbb by Miss Islington (bot) in branch '3.7': bpo-1812: Fix newline conversion when doctest.testfile loads from a package whose loader has a get_data method (GH-17385)

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Change by miss-islington : -- nosy: +miss-islington nosy_count: 4.0 -> 5.0 pull_requests: +18535 pull_request: https://github.com/python/cpython/pull/19175 ___ Python tracker

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Zachary Ware added the comment: New changeset e0b8101492f6c61dee831425b4d3dae39a953599 by Peter Donis in branch 'master': bpo-1812: Fix newline conversion when doctest.testfile loads from a package whose loader has a get_data method (GH-17385)

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Change by miss-islington : -- pull_requests: +18536 pull_request: https://github.com/python/cpython/pull/19176 ___ Python tracker ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Éric Araujo added the comment: Ah, I got it wrong, get_data itself uses `rb`, not default `r`. It makes sense given that package data could be any kind of file, not necessarily text. Too bad there isn’t a `get_text(encoding)` method in the loader API! With Python 3’s rich IO system, isn’t

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Éric Araujo added the comment: Allow me to ask a maybe very naive question: Wouldn't it be possible to fix the problem at the source, that is, use `open(filename, "rU")` in the two spots you found rather than doing ad-hoc line ending translation? (I get that the second one uses `rb` at the

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Change by Zachary Ware : -- nosy: +zach.ware ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis added the comment: Pinging as a reminder that there is a pull request for this issue awaiting review. -- ___ Python tracker ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Change by Peter Donis : -- versions: +Python 3.9 -Python 2.7, Python 3.4, Python 3.5 ___ Python tracker ___ ___ Python-bugs-list

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis added the comment: I have submitted pull request #17385 regarding this issue: https://github.com/python/cpython/pull/17385 -- ___ Python tracker ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Change by Peter Donis : -- pull_requests: +16867 pull_request: https://github.com/python/cpython/pull/17385 ___ Python tracker ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Changes by Terry J. Reedy tjre...@udel.edu: -- versions: +Python 3.4, Python 3.5 -Python 3.1, Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Changes by Alexander Belopolsky alexander.belopol...@gmail.com: -- assignee: belopolsky - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Changes by Mark Lawrence breamore...@yahoo.co.uk: -- nosy: -BreamoreBoy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: I recently noticed that there has been a minor code change in the _load_testfile function in doctest, so I generated a new patch against the latest pull from Mercurial (cpython). No actual changes to the issue fix, but this patch should

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: Updated patch to ensure that tests pass when the -v flag is set running the test suite. This is done by having the helper script, doctest_testfile.py, call doctest.testfile with verbose=False to ensure there is no output if the test

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Mark Lawrence breamore...@yahoo.co.uk added the comment: can committers take a look please. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Mark Lawrence breamore...@yahoo.co.uk added the comment: @Peter again the py3k patch is fine and the 2.7 fails. Is this worth all your effort? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: @Mark, I'm probably stubborn, yes. :-) Could you post verbose output from your testing on Windows? I'd at least like to be able to duplicate your findings; it's possible there's something simple I'm missing. --

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Mark Lawrence breamore...@yahoo.co.uk added the comment: @Peter apologies hereby offered, the 2.7 patch is fine, I'd screwed up doctest_testfile.txt. As the patch is ok and has been very thoroughly tested please can this be committed. -- stage: - commit review

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Alexander Belopolsky belopol...@users.sourceforge.net added the comment: I'll take a look. Mark, commit review is a stage after a patch was reviewed by a committer and is deemed appropriate. Usually this comes with an accepted resolution. Sometimes commit review is skipped for simple

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: @Mark, no problem, thanks for keeping up with all my patches. :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Mark Lawrence breamore...@yahoo.co.uk added the comment: @Alexander: If I understand this correctly it means that there is effectively no distinction betwen patch review and commit review. Hence it is perfectly possible that the work that myself and Peter have put in goes down the drain?

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Mark Lawrence breamore...@yahoo.co.uk added the comment: The py3k stuff is fine on Windows but the 2.7 maintainance branch now fails. 1 items had failures: 1 of 6 in doctest_testfile.txt ***Test Failed*** 1 failures. -- ___ Python

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: I don't normally run Windows, so it will take a little time for me to set up a Windows build environment. However, I've made a number of other improvements as a result of further testing on Linux, and I've uploaded the improved patch as

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: Uploaded revised diff against py3k branch, doctest-fixes6-py3k.diff, with same improvements as doctest-fixes6.diff. Tests still pass on my Linux box. -- Added file: http://bugs.python.org/file18134/doctest-fixes6-py3k.diff

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: Re my msg110822, I think I have a better solution: have the test create a temporary txt file with intentionally mismatched newlines, and use that as the doctest. That means we can control the exact byte by byte content of the txt file,

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: Uploaded doctest-fixes5.diff with one minor correction: removed some comments that were reminders for the py3k version (which I'll upload shortly). -- Added file: http://bugs.python.org/file18118/doctest-fixes5.diff

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: Uploaded doctest-fixes5-py3k.diff, diff against py3k branch implementing same improved test method as doctest-fixes5.diff. -- Added file: http://bugs.python.org/file18119/doctest-fixes5-py3k.diff

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: I saw that this issue was bumped and re-tested against the current trunk (r82970). A further change in doctest_testfile.py was needed to make the test pass when called from regrtest.py: the test importer for the loader.get_data test case

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Mark Lawrence breamore...@yahoo.co.uk added the comment: Patched files work fine on Windows against the 2.7 maintainance branch. doctest (doctest) ... 66 tests with zero failures doctest (test.test_doctest) ... 440 tests with zero failures They fail on py3k, it's a unicode problem that's

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: You'll probably want someone else to confirm, but for the record, my testing was on a Linux box (SuSE 11.2) using Python 2.7 built from the SVN trunk: pe...@powerspec:~/.local/lib/python2.7/test uname -a Linux Powerspec

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: I realized on comparing doctest-fixes2.diff with doctest-fixes1.diff that doctest-fixes2.diff doesn't capture the different newlines correctly, so patch on my machine wouldn't apply the diff (I had done testing from the modified svn

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Mark Lawrence breamore...@yahoo.co.uk added the comment: I've again tried running the test against the 2.7 maintainance release and got:- ValueError: line 12 of the docstring for doctest_testfile.txt has inconsistent leading whitespace: 'This doctest verifies universal newline support; each

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: Uploaded a diff implementing the fix for the head of the py3k branch. Test passes on my Linux box: pe...@powerspec:~/.local/lib/python3.2/test python3.2 Python 3.2a0 (py3k:82984, Jul 19 2010, 16:03:06) [GCC 4.4.1 [gcc-4_4-branch revision

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: Re msg110808, on thinking it over I realized it may not be so simple to get diff and patch to behave properly with a file like doctest_testfile.txt, where we want to intentionally mismatch newlines. We basically need to treat the file as

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: Uploading py3k version of doctest_testfile.txt as well, in case it's needed for testing. -- Added file: http://bugs.python.org/file18072/doctest_testfile.txt.py3k ___ Python tracker

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis peterdo...@alum.mit.edu added the comment: Also, can someone please clear the spam flag on my msg110813? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Alexander Belopolsky belopol...@users.sourceforge.net added the comment: Mark L, This could use some shaking. Please take a look. -- nosy: +BreamoreBoy, belopolsky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Changes by Christian Heimes: -- priority: - normal __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1812 __ ___ Python-bugs-list mailing list Unsubscribe:

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Changes by Christian Heimes: -- keywords: +easy __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1812 __ ___ Python-bugs-list mailing list Unsubscribe:

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis added the comment: I've uploaded a revised diff with two small improvements: (1) Removed a redundant os.isfile check in PackageLoaderTestImporter.get_data() in doctest_testfile.py. (The open() builtin already raises IOError if the file can't be opened.) (2) Added doctest.master =

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

New submission from Peter Donis: When running doctest.testfile on a Linux machine, testing a txt file saved on a Windows machine, doctest raised a SyntaxError exception for each Windows newline in the txt file. On examining the code in the _load_testfile function, it looks to me like there

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

Peter Donis added the comment: Edit: I should have said that the attached diff also includes changes to test_doctest.py to test for the correct newline behavior. Because the test setup is a little complex, I added an auxiliary script, doctest_testfile.py, and an accompanying text file,