On 16 August 2016 at 18:47, Mike Dewhirst <[email protected]> wrote:
> Postscript ... > > Just went back into the csv file and it had switched *itself* back to > utf-16. This does not compute. I need to lift my understanding somewhat. > LibreOffice asks what encoding you want to save in, but other programs, including text editors, may not. On 16 August 2016 at 18:35, Mike Dewhirst <[email protected]> wrote: > On 16/08/2016 5:36 PM, William ML Leslie wrote: > >> ​What does something like this do for you?​ >> > > That is an official, gold plated win! > > Thanks for your persistence William. Everything is utf-8 now. I found a > Windows registry hack to convert the codepage to utf-8 or actually cp65001 > as Microsoft prefer to call it. No more cp850 or cp1252. Then I adjusted > your ... > > with open(csvfile, "r", encoding='utf-16') as csv: > > ... to utf-8 and we have readable output. I'm cooking again. > Cool, I learned a few things too. I think the main points to take away for me are: * You can get errors="replace" behaviour for select stdout lines by encoding to and then decoding from that encoding, with the replace handler. I wonder how difficult it would be for print() and TextIOBase.write() to support errors= as a keyword argument. * Check if things are really UTF-8 - they often aren't. * Also, repr(str) no longer returns an ASCII-clean string in python3. > > Apart from beer next time we meet and a glowing credit in the project > contribution list you have my sincere gratitude. > Great, looking forward to it! I hope csvkit helps, too. Offhand, I'm sorry if I was overly assertive at any point on this list. I tend to get twitchy when dealing with Unicode, but I don't think that's uncommon. -- William Leslie Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement.
_______________________________________________ melbourne-pug mailing list [email protected] https://mail.python.org/mailman/listinfo/melbourne-pug
