Bugs item #1606092, was opened at 2006-11-30 08:46 Message generated for change (Comment added) made by montanaro You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1606092&group_id=5470
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Unicode Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: JettLogic (jettlogic) Assigned to: Nobody/Anonymous (nobody) Summary: csv module broken for unicode Initial Comment: The csv module does not accept data to write/read as anything other than ascii-or-utf-8 str, and the do-it-yourself example in the Python 2.5 Manual to write in another encoding is extremely clunky: 1) convert unicode to utf-8 2) use csv on utf-8 with cStringIO output 3) convert utf-8 to unicode 4) convert unicode to target encoding (may be utf-8...) So clunky as to be a bug - csv clearly can't handle unicode at all. The module functions are in dire need of either accepting unicode objects (letting the output stream worry about the encoding, like codecs.StreamWriter), or at the very least accepting data directly in a target encoding instead of roundabout utf-8. To read another encoding is a bit less onerous than writing: 1) wrap file to return utf-8 2) use csv, getting utf-8 output 3) convert utf-8 to unicode object Anyone willing to fix the csv module? ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2006-12-03 12:22 Message: Logged In: YES user_id=44345 Originator: NO I must admit I don't understand the criticism of the UnicodeReader and UnicodeWriter example classes in the module documentation. Sure, their implementations jump through some hoops, but that's so you don't have to. If you use them as written I believe their API's should be about the same as the csv.reader and csv.writer classes with the added improvement that the reader returns Unicode and the writer accepts Unicode. If your desire is to read and write Unicode why do you care that those objects are encoded using utf-8 in the file? Like Martin asked, are you willing to come up with better examples? Better yet, are you willing to provide a patch for the underlying extension module so it handles Unicode? Hint: I'm fairly certain that if it was trivial it would have been done by now. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2006-12-03 06:54 Message: Logged In: YES user_id=38388 Originator: NO It should be easy to provide a wrapper class which implements the above in plain Python. However, if noone volunteers to write such code, it's not going to happen. I've found that the builtin csv module is not flexible enough to deal with the often broken CSV data you typically find in practice, so perhaps adding a pure Python implementation which works with Unicode might prove to be a better approach. Unassigning the report, since I don't have time for this. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2006-12-03 04:34 Message: Logged In: YES user_id=21627 Originator: NO Are you willing to fix it? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1606092&group_id=5470 _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
