[issue7198] Extraneous newlines with csv.writer on Windows
Roundup Robot devnull@devnull added the comment: New changeset 9201455f950b by R David Murray in branch '3.1': #7198: really add newline='' to csv.writer docs. http://hg.python.org/cpython/rev/9201455f950b New changeset fa0563f3b7f7 by R David Murray in branch '3.2': Really merge #7198 http://hg.python.org/cpython/rev/fa0563f3b7f7 New changeset ed0d1e07ce79 by R David Murray in branch 'default': Dummy merge #7198 http://hg.python.org/cpython/rev/ed0d1e07ce79 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
R. David Murray rdmur...@bitdance.com added the comment: OK, now it's really done (I hope!). -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Skip Montanaro s...@pobox.com added the comment: John Skip, The changes that I suggested have NOT been made. Please John re-read the doc page you pointed to. The writer paragraph does John NOT mention that newline='' is required when writing. The writer John examples do NOT include newline=''. The examples have NOT been John enhanced by using a with statement and not using space as an John example delimiter. I copied the statement about using newline= from the reader() doc to the writer() doc. All the examples I see (I'm looking at the cpython repo - that is, what will be 3.3) use the with statement and open files using newline=''. I don't think more changes are necessary. I will consult with other Python developers about merging these changes to other active branches. I simply don't understand the new Mercurial workflow well enough to do it properly. Skip -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Roundup Robot devnull@devnull added the comment: New changeset 88876a264ebe by R David Murray in branch '3.1': Markup fixes for #7198 patch. http://hg.python.org/cpython/rev/88876a264ebe New changeset d0d1235cb66e by R David Murray in branch '3.2': Merge markup fixes for #7198 patch. http://hg.python.org/cpython/rev/d0d1235cb66e New changeset 2a8580f4897c by R David Murray in branch 'default': Markup fixes for #7198 patch. http://hg.python.org/cpython/rev/2a8580f4897c -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
John Machin sjmac...@lexicon.net added the comment: Can somebody please review my doc patch submitted 2 months ago? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Skip Montanaro s...@pobox.com added the comment: John John Machin sjmac...@lexicon.net added the comment: John Can somebody please review my doc patch submitted 2 months ago? My apologies. I have it in my sandbox, but a combination of the switch to Mercurial and lack of round tuits has conspired to keep me from checking it in. Skip -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Skip Montanaro s...@pobox.com added the comment: Actually, I was thinking of another doc patch for the csv module. Your changes (or something very like them) made it into the 3.2 release, as you can see here: http://docs.python.org/py3k/library/csv.html S -- resolution: - accepted status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
John Machin sjmac...@lexicon.net added the comment: Skip, The changes that I suggested have NOT been made. Please re-read the doc page you pointed to. The writer paragraph does NOT mention that newline='' is required when writing. The writer examples do NOT include newline=''. The examples have NOT been enhanced by using a with statement and not using space as an example delimiter. PLEASE RE-OPEN THIS ISSUE. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Roundup Robot devnull@devnull added the comment: New changeset ab27f16f707a by R David Murray in branch 'default': #7198: add newlines='' to csv.writer docs. http://hg.python.org/cpython/rev/ab27f16f707a New changeset 959f666470cc by R David Murray in branch 'default': Merge #7198 doc fix. http://hg.python.org/cpython/rev/959f666470cc New changeset 9d1b1a95bc8f by R David Murray in branch 'default': Merge #7198 doc fix. http://hg.python.org/cpython/rev/9d1b1a95bc8f -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
R. David Murray rdmur...@bitdance.com added the comment: Fixed now. Thanks, and sorry for the delay, and the confusion. -- resolution: accepted - fixed stage: needs patch - committed/rejected ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
R. David Murray rdmur...@bitdance.com added the comment: Gah, I messed up the push. Now I have to backport the doc fix :( -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Changes by R. David Murray rdmur...@bitdance.com: -- assignee: skip.montanaro - r.david.murray status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
John Machin sjmac...@lexicon.net added the comment:
docpatch for 3.x csv docs:
In the csv.writer docs, insert the sentence If csvfile is a file object, it
should be opened with newline=''. immediately after the sentence csvfile can
be any object with a write() method.
In the closely-following example, change the open call from open('eggs.csv',
'w') to open('eggs.csv', 'w', newline='').
In section 13.1.5 Examples, there are 2 reader cases and 1 writer case that
likewise need inserting , newline='' in the open call.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7198
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
John Machin sjmac...@users.sourceforge.net added the comment: Skip, I'm WRITING, not reading.. Please read the 3.1 documentation for csv.writer. It does NOT mention newline='', and neither does the example. Please fix. Other problems with the examples: (1) They encourage a bad habit (open inside the call to reader/writer); good practice is to retain the reference to the file handle (preferably with a with statement) so that it can be closed properly. (2) delimiter=' ' is very unrealistic. The documentation for both 2.x and 3.x should be much more explicit about what is needed in open() for csv to work properly and portably: 2.x read: use mode='rb' -- otherwise fail on Windows 2.x write: use mode='wb' -- otherwise fail on Windows 3.x read: use newline='' -- otherwise fail unconditionally(?) 3.x write: use newline='' -- otherwise fail on Windows The 2.7 documentation says If csvfile is a file object, it must be opened with the 'b' flag on platforms where that makes a difference ... in my experience, people are left asking what platforms? what difference?; Windows should be mentioned explicitly. -- versions: +Python 2.7, Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
R. David Murray rdmur...@bitdance.com added the comment: OK, I'm reopening this as a doc issue, since currently the Python3 writer docs do not mention newline='', and it is indeed required on Windows. John, would you care to suggest a doc patch? I agree with Skip that where it makes a difference is more precise than specifically mentioning Windows, even if less useful in this context. That is how the 'b' mode is documented in the open documentation. To fix the problem with the CSV docs, the recommendation to use 'b' can simply be made unconditional, as it is for newline='' in python3. -- components: +Documentation nosy: +r.david.murray resolution: invalid - stage: - needs patch status: closed - open versions: -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Skip Montanaro s...@pobox.com added the comment: John, The API for the open() builtin function has changed. You should open the output file with newline= instead of using the default. Take a look at the documentation for open() and csv.reader: http://docs.python.org/py3k/library/functions.html?highlight=open#open http://docs.python.org/py3k/library/csv.html?highlight=csv.reader#csv.reader Note the form of the open() call in the csv.reader example. This one snuck by me as well. Python 3 underwent a lot of change in the I/O subsystem. This was one of them. If changing the form of the open() call doesn't fix the problem, let me know. Skip -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
John Machin sjmac...@users.sourceforge.net added the comment:
Please re-open this. The binary/text mode problem still exists with Python 3.X
on Windows. Quite simply, there is no option available to the caller to open
the output file in binary mode, because the module is throwing str objects at
the file. The module's idea of taking control in the default case appears to
be to write \r\n which is then processed by the Windows runtime and becomes
\r\r\n.
Python 3.1.3 (r313:86834, Nov 27 2010, 18:30:53) [MSC v.1500 32 bit (Intel)] on
win32
Type help, copyright, credits or license for more information.
import csv
f = open('terminator31.csv', 'w')
row = ['foo', None, 3.14159]
writer = csv.writer(f)
writer.writerow(row)
14
writer.writerow(row)
14
f.close()
open('terminator31.csv', 'rb').read()
b'foo,,3.14159\r\r\nfoo,,3.14159\r\r\n'
And it's not just a row terminator problem; newlines embedded in fields are
likewise expanded to \r\n by the Windows runtime.
--
nosy: +sjmachin
versions: +Python 3.1 -Python 2.6
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7198
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Skip Montanaro s...@pobox.com added the comment: If the documentation is not clear enough about requiring binary, it is a doc bug. The documentation for both csv.reader and csv.writer state (this is from the Python 2.7 version): If *csvfile* is a file object, it must be opened with the 'b' flag on platforms where that makes a difference. I suppose we could be explicit and mention Windows here, but the wording is quite clear. There is really no harm in always opening the file in binary mode, and I do that myself even though I only program on Unix or Mac platforms where it's safe to open the file in text mode. This all changed in Python 3. There, the choice of line ending is up to the programmer, so file objects for use by the csv module are opened with newline='' and when writing CSV data the writer object takes complete control of proper line termination according to the programmer's stated choice of lineterminator. Skip -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Éric Araujo mer...@netwok.org added the comment: Bob, can you give us some code to reproduce the problem, in the form or a unit test or even just a regular function? It will help confirm the bug and fix it. -- nosy: +merwok stage: - unit test needed title: csv.writer - Extraneous newlines with csv.writer on Windows ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Bob Cannon b...@neqn.net added the comment: Eric, This issue was resolved for me by Skip Montanaro's response less than an hour after I posted it. I didn't understand why a text file had to be binary, but I no longer had a problem with extraneous. In looking back at my message 94441, I think that it was ambiguous and that I should have made it clear that I no longer had a problem. Perhaps in my ignorance as a newbie I didn't close the issue properly. I don't know that I can reproduce the problem any more. I think that I was writing snippets of code to try to isolate the problem and when I used Skip's solution I changed the program and deleted the test code. Please let me know what I can do to help you now. Bob Éric Araujo wrote: Éric Araujo mer...@netwok.org added the comment: Bob, can you give us some code to reproduce the problem, in the form or a unit test or even just a regular function? It will help confirm the bug and fix it. -- nosy: +merwok stage: - unit test needed title: csv.writer - Extraneous newlines with csv.writer on Windows ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Éric Araujo mer...@netwok.org added the comment: If the documentation is not clear enough about requiring binary, it is a doc bug. (P.S. Please strip unneeded quotes. Thanks) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7198] Extraneous newlines with csv.writer on Windows
Skip Montanaro s...@pobox.com added the comment: I got access to Python 2.6.5 on Windows and ran this simple example: Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit (Intel)] on win32 Type copyright, credits or license() for more information. Personal firewall software may warn about the connection IDLE makes to its subprocess using this computer's internal loopback interface. This connection is not visible on any external interface and no data is sent to or received from the Internet. IDLE 2.6.5 f = open(H:sample.csv, wb) import csv writer = csv.writer(f) writer.writerow([1,2,3]) writer.writerow(['a', 'b', 'c']) del writer f.close() I then looked at the CSV file which it generated. Looked find to me. Each of the two rows was terminated by a single CRLF pair. Then I repeated the test, opening the file in text mode: f = open(H:sample2.csv, w) writer = csv.writer(f) writer.writerow([1,2,3]) writer.writerow(['a', 'b', 'c']) del writer f.close() That output does indeed terminate each line with CRCRLF and when viewed in a spreadsheet program such as OpenOffice Calc (probably Excel as well), displays a blank line between the 123 row and the abc row. I've removed the unit test needed attribute from the ticket as there is a test_writerows test case in the Python test suite. Also closing again and marking invalid. If you still believe there is actually a problem, feel free to reopen this issue, but also please send me (s...@pobox.com) a short example and the erroneous output it produces for you (attach your two files - don't just embed them in your mail msg). -- resolution: - invalid stage: unit test needed - status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
