[issue43049] Use io.IncrementalNewlineDecoder for doctest newline conversion
Peter Donis added the comment: Thanks for merging! -- ___ Python tracker <https://bugs.python.org/issue43049> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43049] Use io.IncrementalNewlineDecoder for doctest newline conversion
Change by Peter Donis : -- keywords: +patch pull_requests: +23183 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24359 ___ Python tracker <https://bugs.python.org/issue43049> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Change by Peter Donis : -- pull_requests: +23184 pull_request: https://github.com/python/cpython/pull/24359 ___ Python tracker <https://bugs.python.org/issue1812> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43049] Use io.IncrementalNewlineDecoder for doctest newline conversion
New submission from Peter Donis : This is a follow-up to issue 1812: https://bugs.python.org/issue1812 It was suggested in the discussion on that issue that the newline conversion in doctest that was corrected in that issue could be done using already defined resources in the io module instead of by a custom function. This issue is to provide an issue number for my pull request on github to do that: the io module provides the IncrementalNewlineDecoder object that does exactly what is needed. -- components: Tests messages: 385830 nosy: pdonis priority: normal severity: normal status: open title: Use io.IncrementalNewlineDecoder for doctest newline conversion type: enhancement versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue43049> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis added the comment: Pinging as a reminder that there is a pull request for this issue awaiting review. -- ___ Python tracker <https://bugs.python.org/issue1812> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Change by Peter Donis : -- versions: +Python 3.9 -Python 2.7, Python 3.4, Python 3.5 ___ Python tracker <https://bugs.python.org/issue1812> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis added the comment: I have submitted pull request #17385 regarding this issue: https://github.com/python/cpython/pull/17385 -- ___ Python tracker <https://bugs.python.org/issue1812> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Change by Peter Donis : -- pull_requests: +16867 pull_request: https://github.com/python/cpython/pull/17385 ___ Python tracker <https://bugs.python.org/issue1812> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25045] smtplib throws exception TypeError: readline()
Peter Donis added the comment: Investigating further, the problem appears to be with the SSLFakeFile object used for SSL/TLS connections. Here is a console session showing the issue (I have redacted the host name and port number used for privacy reasons): >>> import smtplib >>> smtp = smtplib.SMTP(hostname, portnum) >>> smtp.file <_io.BufferedReader name=3> >>> smtp.starttls() (220, b'2.0.0 Ready to start TLS') >>> smtp.file >>> smtp.file.readline(1) Traceback (most recent call last): File "", line 1, in TypeError: readline() takes exactly 1 positional argument (2 given) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25045> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25045] smtplib throws exception TypeError: readline()
Peter Donis added the comment: I am having the same problem; the error occurs in the call to the login method of the smtplib.SMTP object. That method takes two arguments, username and password, and that is what I am calling it with. -- nosy: +pdonis ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25045> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25045] smtplib throws exception TypeError: readline()
Peter Donis added the comment: It looks like this was fixed some time in between my Python 3.2 version (3.2.3) and the current one (3.2.6); the code in the current 3.2 head in the repository has a size parameter in the readline function for SSLFakeFile: https://hg.python.org/cpython/file/3.2/Lib/smtplib.py -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25045> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: I recently noticed that there has been a minor code change in the _load_testfile function in doctest, so I generated a new patch against the latest pull from Mercurial (cpython). No actual changes to the issue fix, but this patch should apply cleanly against a checkout from Mercurial. This patch also adds my name to the Misc/ACKS file. :-) I have not re-done patches in Mercurial's format against any other branches except the default. If this fix is still under consideration, I can generate patches against other branches and test them. -- versions: +Python 3.3 Added file: http://bugs.python.org/file26505/doctest-fixes7-py3k.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: Updated patch to ensure that tests pass when the -v flag is set running the test suite. This is done by having the helper script, doctest_testfile.py, call doctest.testfile with verbose=False to ensure there is no output if the test passes (which is what the master test_doctest script expects) even if the -v flag is set on the command line. -- Added file: http://bugs.python.org/file26508/doctest-fixes8-py3k.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: @Mark, I'm probably stubborn, yes. :-) Could you post verbose output from your testing on Windows? I'd at least like to be able to duplicate your findings; it's possible there's something simple I'm missing. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: @Mark, no problem, thanks for keeping up with all my patches. :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: I don't normally run Windows, so it will take a little time for me to set up a Windows build environment. However, I've made a number of other improvements as a result of further testing on Linux, and I've uploaded the improved patch as doctest-fixes6.diff. When I apply this patch to a regular Python 2.7 installation on Windows (Windows 2000 running under VirtualBox on Linux), the tests pass (as well as on my Linux box when applied against the release27-maint SVN branch). Output from testing on Windows: C:\Python27\Lib\testpython test_doctest.py doctest (doctest) ... 66 tests with zero failures doctest (test.test_doctest) ... 443 tests with zero failures C:\Python27\Lib\testpython Python 2.7 (r27:82525, Jul 4 2010, 09:01:59) [MSC v.1500 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. Hopefully the improved patch will test OK on your box as well. If not, I'll work on testing it in a Windows build environment against an SVN checkout. Improvements in doctest-fixes6.diff: - Uses with statements to guard all file reads (in earlier patches writes were guarded but reads were not); - Saves and restores sys.path the same way as test_importhooks; - Checks for byte-compiled files in __pycache__ when deleting temporary files (this was in the py3k patch already, but reading PEP 3147 it looks like this feature may be backported to 2.7 as well); - Test setup/cleanup is now done in a TestFixture class, for clarity and because it's easier that way to store the original sys.path in the setup and restore it in the cleanup. -- Added file: http://bugs.python.org/file18133/doctest-fixes6.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: Uploaded revised diff against py3k branch, doctest-fixes6-py3k.diff, with same improvements as doctest-fixes6.diff. Tests still pass on my Linux box. -- Added file: http://bugs.python.org/file18134/doctest-fixes6-py3k.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: Re my msg110822, I think I have a better solution: have the test create a temporary txt file with intentionally mismatched newlines, and use that as the doctest. That means we can control the exact byte by byte content of the txt file, without worrying about how it is stored/transferred, etc. I've attached a Python 2.7 patch, doctest-fixes4.diff, which implements this test method (same actual tests as before, just the setup/cleanup is changed); the test passes on my machine. The test code in this patch also includes comments explaining why this approach is taken. -- Added file: http://bugs.python.org/file18117/doctest-fixes4.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: Uploaded doctest-fixes5.diff with one minor correction: removed some comments that were reminders for the py3k version (which I'll upload shortly). -- Added file: http://bugs.python.org/file18118/doctest-fixes5.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: Uploaded doctest-fixes5-py3k.diff, diff against py3k branch implementing same improved test method as doctest-fixes5.diff. -- Added file: http://bugs.python.org/file18119/doctest-fixes5-py3k.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: I saw that this issue was bumped and re-tested against the current trunk (r82970). A further change in doctest_testfile.py was needed to make the test pass when called from regrtest.py: the test importer for the loader.get_data test case now stores the absolute path where doctest_testfile.txt is located, so that it can always find it. I've uploaded a 2nd revised diff with this change. -- keywords: +patch Added file: http://bugs.python.org/file18063/doctest-fixes2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: You'll probably want someone else to confirm, but for the record, my testing was on a Linux box (SuSE 11.2) using Python 2.7 built from the SVN trunk: pe...@powerspec:~/.local/lib/python2.7/test uname -a Linux Powerspec 2.6.31.12-0.2-desktop #1 SMP PREEMPT 2010-03-16 21:25:39 +0100 i686 i686 i386 GNU/Linux pe...@powerspec:~/.local/lib/python2.7/test python2.7 Python 2.7 (trunk:82970M, Jul 19 2010, 12:44:35) [GCC 4.4.1 [gcc-4_4-branch revision 150839]] on linux2 Type help, copyright, credits or license for more information. pe...@powerspec:~/.local/lib/python2.7/test python2.7 test_doctest.py doctest (doctest) ... 66 tests with zero failures doctest (test.test_doctest) ... 443 tests with zero failures -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: I realized on comparing doctest-fixes2.diff with doctest-fixes1.diff that doctest-fixes2.diff doesn't capture the different newlines correctly, so patch on my machine wouldn't apply the diff (I had done testing from the modified svn checkout without reverting and then re-applying the patch). Uploaded doctest-fixes3.diff which is generated using the external diff command (svn diff --diff-cmd diff) so that patch will apply it cleanly. Not sure if this is something specific to my machine. -- keywords: +patch Added file: http://bugs.python.org/file18066/doctest-fixes3.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: Uploaded a diff implementing the fix for the head of the py3k branch. Test passes on my Linux box: pe...@powerspec:~/.local/lib/python3.2/test python3.2 Python 3.2a0 (py3k:82984, Jul 19 2010, 16:03:06) [GCC 4.4.1 [gcc-4_4-branch revision 150839]] on linux2 Type help, copyright, credits or license for more information. pe...@powerspec:~/.local/lib/python3.2/test python3.2 test_doctest.py doctest (doctest) ... 66 tests with zero failures doctest (test.test_doctest) ... 415 tests with zero failures -- Added file: http://bugs.python.org/file18070/doctest-fixes-py3k.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: Re msg110808, on thinking it over I realized it may not be so simple to get diff and patch to behave properly with a file like doctest_testfile.txt, where we want to intentionally mismatch newlines. We basically need to treat the file as binary rather than text (which also means that, since the file as I've uploaded it contains all Unix newlines except for the Windows and Mac specific lines, on a Windows or Mac system most of the file, including all the comment lines, ends up being part of the test). Maybe there's a way to set the svn:mime-type property to do that. In the meantime, I've uploaded the raw doctest_testfile.txt file, so if necessary it can just be copied into the appropriate directory for testing. -- Added file: http://bugs.python.org/file18071/doctest_testfile.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: Uploading py3k version of doctest_testfile.txt as well, in case it's needed for testing. -- Added file: http://bugs.python.org/file18072/doctest_testfile.txt.py3k ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis peterdo...@alum.mit.edu added the comment: Also, can someone please clear the spam flag on my msg110813? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1812 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis added the comment: I've uploaded a revised diff with two small improvements: (1) Removed a redundant os.isfile check in PackageLoaderTestImporter.get_data() in doctest_testfile.py. (The open() builtin already raises IOError if the file can't be opened.) (2) Added doctest.master = None in three places in doctest_testfile.py so the expected output is now nothing instead of three lines of *** DocTestRunner.merge: 'doctest_testfile.txt' in both testers; summing outcomes. Changed the expected output in test_doctest.py to correspond. Added file: http://bugs.python.org/file9142/doctest-fixes1.diff __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1812 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
New submission from Peter Donis: When running doctest.testfile on a Linux machine, testing a txt file saved on a Windows machine, doctest raised a SyntaxError exception for each Windows newline in the txt file. On examining the code in the _load_testfile function, it looks to me like there are actually two issues: (1) When there is no package keyword argument given, or the package argument points to a package without a __loader__.get_data method, the txt file data is retrieved by calling open(filename).read(); this is the code path that my Windows-saved file triggered. However, since the default file mode for open is 'r', not 'rU', there is no universal newline conversion done on the file data. This was the issue I initially observed. (2) When there is a package.__loader__.get_data method found, that method reads the data (using file mode 'rb'), and then newline conversion is performed by this line: return file_contents.replace(os.linesep, '\n') This does not seem to match what universal newline conversion does; that is supposed to convert both '\r\n' and '\r' to '\n', but running on Linux, os.linesep is '\n', so the replace operation in the current code is a no-op, even if the txt file was saved with Windows or Mac newlines. It seems to me that the correct operation would be: for linesep in ('\r\n', '\r'): file_contents = file_contents.replace(linesep, '\n') I have attached a diff against the current svn trunk showing my suggested fix for both of the above issues. Peter Donis -- components: Tests files: doctest-fixes.diff messages: 59801 nosy: pdonis severity: normal status: open title: doctest _load_testfile function -- newline handling seems incorrect type: behavior versions: Python 2.6 Added file: http://bugs.python.org/file9138/doctest-fixes.diff __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1812 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1812] doctest _load_testfile function -- newline handling seems incorrect
Peter Donis added the comment: Edit: I should have said that the attached diff also includes changes to test_doctest.py to test for the correct newline behavior. Because the test setup is a little complex, I added an auxiliary script, doctest_testfile.py, and an accompanying text file, doctest_testfile.txt, which intentionally contains mismatched newlines for use in the test. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1812 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com