Re: Python 3.7+ cannot print unicode characters when output is redirected to file - is this a bug?

2022-11-13 Thread Eryk Sun
On 11/13/22, Jessica Smith <12jessicasmit...@gmail.com> wrote: > Consider the following code ran in Powershell or cmd.exe: > > $ python -c "print('└')" > └ > > $ python -c "print('└')" > test_file.txt > Traceback (most recent call last): > File "", line 1, in > File "C:\Program Files\Python38\

Re: Python 3.7+ cannot print unicode characters when output is redirected to file - is this a bug?

2022-11-13 Thread Thomas Passin
On 11/13/2022 9:49 AM, Jessica Smith wrote: Consider the following code ran in Powershell or cmd.exe: $ python -c "print('└')" └ $ python -c "print('└')" > test_file.txt Traceback (most recent call last): File "", line 1, in File "C:\Program Files\Python38\lib\encodings\cp1252.py", line

Re: Python 3.7+ cannot print unicode characters when output is redirected to file - is this a bug?

2022-11-13 Thread Barry
> On 13 Nov 2022, at 14:52, Jessica Smith <12jessicasmit...@gmail.com> wrote: > > Consider the following code ran in Powershell or cmd.exe: > > $ python -c "print('└')" > └ > > $ python -c "print('└')" > test_file.txt > Traceback (most recent call last): > File "", line 1, in > File "C:\Pr

Python 3.7+ cannot print unicode characters when output is redirected to file - is this a bug?

2022-11-13 Thread Jessica Smith
Consider the following code ran in Powershell or cmd.exe: $ python -c "print('└')" └ $ python -c "print('└')" > test_file.txt Traceback (most recent call last): File "", line 1, in File "C:\Program Files\Python38\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(i

Re: Putting Unicode characters in JSON

2018-03-24 Thread Peter J. Holzer
On 2018-03-25 06:30:54 +1100, Chris Angelico wrote: > On Sun, Mar 25, 2018 at 3:35 AM, Peter J. Holzer wrote: > > On 2018-03-24 11:21:09 +1100, Chris Angelico wrote: > >> If the database has been configured to use UTF-8 (as mentioned, that's > >> "utf8mb4" in MySQL), you won't get that byte sequen

Re: Putting Unicode characters in JSON

2018-03-24 Thread Chris Angelico
On Sun, Mar 25, 2018 at 3:35 AM, Peter J. Holzer wrote: > On 2018-03-24 11:21:09 +1100, Chris Angelico wrote: >> If the database has been configured to use UTF-8 (as mentioned, that's >> "utf8mb4" in MySQL), you won't get that byte sequence back. You'll get >> back valid UTF-8. > > Actually (with

Re: Putting Unicode characters in JSON

2018-03-24 Thread Peter J. Holzer
On 2018-03-24 11:21:09 +1100, Chris Angelico wrote: > On Sat, Mar 24, 2018 at 11:11 AM, Steven D'Aprano > wrote: > > On Fri, 23 Mar 2018 07:46:16 -0700, Tobiah wrote: > >> If I changed my database tables to all be UTF-8 would this work cleanly > >> without any decoding? > > > > Not reliably or saf

Re: Putting Unicode characters in JSON

2018-03-23 Thread Steven D'Aprano
On Sat, 24 Mar 2018 11:21:09 +1100, Chris Angelico wrote: >>> If I changed my database tables to all be UTF-8 would this work >>> cleanly without any decoding? >> >> Not reliably or safely. It will appear to work so long as you have only >> pure ASCII strings from the database, and then crash when

Re: Putting Unicode characters in JSON

2018-03-23 Thread Chris Angelico
On Sat, Mar 24, 2018 at 11:11 AM, Steven D'Aprano wrote: > On Fri, 23 Mar 2018 07:46:16 -0700, Tobiah wrote: > >> If I changed my database tables to all be UTF-8 would this work cleanly >> without any decoding? > > Not reliably or safely. It will appear to work so long as you have only > pure ASCI

Re: Putting Unicode characters in JSON

2018-03-23 Thread Steven D'Aprano
On Fri, 23 Mar 2018 07:46:16 -0700, Tobiah wrote: > If I changed my database tables to all be UTF-8 would this work cleanly > without any decoding? Not reliably or safely. It will appear to work so long as you have only pure ASCII strings from the database, and then crash when you don't: py> te

Re: Putting Unicode characters in JSON

2018-03-23 Thread Chris Angelico
On Sat, Mar 24, 2018 at 1:46 AM, Tobiah wrote: > On 03/22/2018 12:46 PM, Tobiah wrote: >> >> I have some mailing information in a Mysql database that has >> characters from various other countries. The table says that >> it's using latin-1 encoding. I want to send this data out >> as JSON. >> >>

Re: Putting Unicode characters in JSON

2018-03-23 Thread Grant Edwards
On 2018-03-23, Richard Damon wrote: > One comment on this whole argument, the original poster asked how to get > data from a database that WAS using Latin-1 encoding into JSON (which > wants UTF-8 encoding) and was asking if something needed to be done > beyond using .decode('Latin-1'), and in

Re: Putting Unicode characters in JSON

2018-03-23 Thread Grant Edwards
On 2018-03-23, Chris Angelico wrote: > On Fri, Mar 23, 2018 at 10:47 AM, Steven D'Aprano > wrote: >> On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote: >> I was reading though, that JSON files must be encoded with UTF-8. So should I be doing string.decode('latin-1').encode('utf-8

Re: Putting Unicode characters in JSON

2018-03-23 Thread Tobiah
On 03/22/2018 12:46 PM, Tobiah wrote: I have some mailing information in a Mysql database that has characters from various other countries.  The table says that it's using latin-1 encoding.  I want to send this data out as JSON. So I'm just taking each datum and doing 'name'.decode('latin-1') an

Re: Putting Unicode characters in JSON

2018-03-23 Thread Richard Damon
On 3/23/18 6:35 AM, Chris Angelico wrote: On Fri, Mar 23, 2018 at 9:29 PM, Steven D'Aprano wrote: On Fri, 23 Mar 2018 18:35:20 +1100, Chris Angelico wrote: That doesn't seem to be a strictly-correct Latin-1 decoder, then. There are a number of unassigned byte values in ISO-8859-1. That's inc

Re: Putting Unicode characters in JSON

2018-03-23 Thread Chris Angelico
On Fri, Mar 23, 2018 at 9:29 PM, Steven D'Aprano wrote: > On Fri, 23 Mar 2018 18:35:20 +1100, Chris Angelico wrote: > >> That doesn't seem to be a strictly-correct Latin-1 decoder, then. There >> are a number of unassigned byte values in ISO-8859-1. > > That's incorrect, but I don't blame you for

Re: Putting Unicode characters in JSON

2018-03-23 Thread Steven D'Aprano
On Fri, 23 Mar 2018 18:35:20 +1100, Chris Angelico wrote: > That doesn't seem to be a strictly-correct Latin-1 decoder, then. There > are a number of unassigned byte values in ISO-8859-1. That's incorrect, but I don't blame you for getting it wrong. Who thought that it was a good idea to disting

Re: Putting Unicode characters in JSON

2018-03-23 Thread Paul Moore
On 23 March 2018 at 00:27, Thomas Jollans wrote: > On 22/03/18 20:46, Tobiah wrote: >> I was reading though, that JSON files must be encoded with UTF-8. So >> should I be doing string.decode('latin-1').encode('utf-8')? Or does >> the json module do that for me when I give it a unicode object? >

Re: Putting Unicode characters in JSON

2018-03-23 Thread Chris Angelico
On Fri, Mar 23, 2018 at 4:35 PM, Steven D'Aprano wrote: > On Fri, 23 Mar 2018 12:05:34 +1100, Chris Angelico wrote: > >> Latin-1 is not "arbitrary bytes". It is a very specific encoding that >> cannot decode every possible byte value. > > Yes it can. > > py> blob = bytes(range(256)) > py> len(blob

Re: Putting Unicode characters in JSON

2018-03-22 Thread Steven D'Aprano
On Fri, 23 Mar 2018 12:05:34 +1100, Chris Angelico wrote: > Latin-1 is not "arbitrary bytes". It is a very specific encoding that > cannot decode every possible byte value. Yes it can. py> blob = bytes(range(256)) py> len(blob) 256 py> blob[45:55] b'-./0123456' py> s = blob.decode('latin1') py>

Re: Putting Unicode characters in JSON

2018-03-22 Thread Chris Angelico
On Fri, Mar 23, 2018 at 11:39 AM, Steven D'Aprano wrote: > On Fri, 23 Mar 2018 11:08:56 +1100, Chris Angelico wrote: >> Okay. Give me a good reason for the database itself to be locked to >> Latin-1. Make sure you explain how potentially saving the occasional >> byte of storage (compared to UTF-8)

Re: Putting Unicode characters in JSON

2018-03-22 Thread Steven D'Aprano
On Fri, 23 Mar 2018 11:08:56 +1100, Chris Angelico wrote: > On Fri, Mar 23, 2018 at 10:47 AM, Steven D'Aprano > wrote: >> On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote: >> I was reading though, that JSON files must be encoded with UTF-8. So should I be doing string.decode('l

Re: Putting Unicode characters in JSON

2018-03-22 Thread Chris Angelico
On Fri, Mar 23, 2018 at 11:39 AM, Ben Finney wrote: > Chris Angelico writes: > >> There is NOT always a good reason for a suboptimal configuration. > > True. Did anyone claim otherwise? > > What I saw Steven responding to was your claim that there is *never* a > good reason to do it. > > To refut

Re: Putting Unicode characters in JSON

2018-03-22 Thread Thomas Jollans
On 22/03/18 20:46, Tobiah wrote: > I was reading though, that JSON files must be encoded with UTF-8.  So > should I be doing string.decode('latin-1').encode('utf-8')?  Or does > the json module do that for me when I give it a unicode object? Definitely not. In fact, that won't even work. >>> impo

Re: Putting Unicode characters in JSON

2018-03-22 Thread Ben Finney
Chris Angelico writes: > There is NOT always a good reason for a suboptimal configuration. True. Did anyone claim otherwise? What I saw Steven responding to was your claim that there is *never* a good reason to do it. To refute that, it's sufficient to show that good reason can exist in some c

Re: Putting Unicode characters in JSON

2018-03-22 Thread Chris Angelico
On Fri, Mar 23, 2018 at 11:25 AM, Ben Finney wrote: > Chris Angelico writes: > >> On Fri, Mar 23, 2018 at 10:47 AM, Steven D'Aprano >> wrote: >> > On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote: >> >> Reconfigure your MySQL database to use UTF-8. There is no reason to >> >> use Latin-1

Re: Putting Unicode characters in JSON

2018-03-22 Thread Ben Finney
Chris Angelico writes: > On Fri, Mar 23, 2018 at 10:47 AM, Steven D'Aprano > wrote: > > On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote: > >> Reconfigure your MySQL database to use UTF-8. There is no reason to > >> use Latin-1 in the database. > > > > You don't know that. You don't know

Re: Putting Unicode characters in JSON

2018-03-22 Thread Chris Angelico
On Fri, Mar 23, 2018 at 10:47 AM, Steven D'Aprano wrote: > On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote: > >>> I was reading though, that JSON files must be encoded with UTF-8. So >>> should I be doing string.decode('latin-1').encode('utf-8')? Or does >>> the json module do that for

Re: Putting Unicode characters in JSON

2018-03-22 Thread Steven D'Aprano
On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote: >> I was reading though, that JSON files must be encoded with UTF-8. So >> should I be doing string.decode('latin-1').encode('utf-8')? Or does >> the json module do that for me when I give it a unicode object? > > Reconfigure your MySQL

Re: Putting Unicode characters in JSON

2018-03-22 Thread Tobiah
On 03/22/2018 01:09 PM, Chris Angelico wrote: On Fri, Mar 23, 2018 at 6:46 AM, Tobiah wrote: I have some mailing information in a Mysql database that has characters from various other countries. The table says that it's using latin-1 encoding. I want to send this data out as JSON. So I'm jus

Re: Putting Unicode characters in JSON

2018-03-22 Thread Chris Angelico
On Fri, Mar 23, 2018 at 6:46 AM, Tobiah wrote: > I have some mailing information in a Mysql database that has > characters from various other countries. The table says that > it's using latin-1 encoding. I want to send this data out > as JSON. > > So I'm just taking each datum and doing 'name'.d

Putting Unicode characters in JSON

2018-03-22 Thread Tobiah
I have some mailing information in a Mysql database that has characters from various other countries. The table says that it's using latin-1 encoding. I want to send this data out as JSON. So I'm just taking each datum and doing 'name'.decode('latin-1') and adding the resulting Unicode value ri

Re: read Unicode characters one by one in python2

2018-02-25 Thread Chris Angelico
On Mon, Feb 26, 2018 at 3:57 AM, Steven D'Aprano wrote: > On Mon, 26 Feb 2018 01:50:16 +1100, Chris Angelico wrote: > >> If you actually need character-by-character, you'd need "for character >> in fh.read()" rather than iterating over the file itself. Iterating over >> a file yields lines. > > In

Re: read Unicode characters one by one in python2

2018-02-25 Thread Steven D'Aprano
On Mon, 26 Feb 2018 01:50:16 +1100, Chris Angelico wrote: > If you actually need character-by-character, you'd need "for character > in fh.read()" rather than iterating over the file itself. Iterating over > a file yields lines. Indeed. But I wonder if there's a performance cost/gain to iterating

Re: read Unicode characters one by one in python2

2018-02-25 Thread Chris Angelico
On Mon, Feb 26, 2018 at 12:33 AM, Chris Warrick wrote: > On 24 February 2018 at 17:17, Peng Yu wrote: >> Here shows some code for reading Unicode characters one by one in >> python2. Is it the best code for reading Unicode characters one by one >> in python2? >> >

Re: read Unicode characters one by one in python2

2018-02-25 Thread Chris Warrick
On 24 February 2018 at 17:17, Peng Yu wrote: > Here shows some code for reading Unicode characters one by one in > python2. Is it the best code for reading Unicode characters one by one > in python2? > > https://rosettacode.org/wiki/Read_a_file_character_by_character/UTF8#P

Re: read Unicode characters one by one in python2

2018-02-24 Thread Karsten Hilbert
On Sat, Feb 24, 2018 at 10:17:35AM -0600, Peng Yu wrote: > Here shows some code for reading Unicode characters one by one in > python2. Is it the best code for reading Unicode characters one by one > in python2? > > https://rosettacode.org/wiki/Read_a_file_character_by_charac

read Unicode characters one by one in python2

2018-02-24 Thread Peng Yu
Here shows some code for reading Unicode characters one by one in python2. Is it the best code for reading Unicode characters one by one in python2? https://rosettacode.org/wiki/Read_a_file_character_by_character/UTF8#Python -- Regards, Peng -- https://mail.python.org/mailman/listinfo/python

Re: How to print unicode characters with yaml.safe_dump()?

2016-05-31 Thread Peter Otten
Peng Yu wrote: > Hi, The following code shows that "Michał" is printed differently for > print(yaml.safe_dump(...)) and the direct print. Does anybody know how > to use yaml.safe_dump() so that "Michał" will be printed as is. > > ~$ cat main.py > #!/usr/bin/env python > # vim: set noexpandtab tab

How to print unicode characters with yaml.safe_dump()?

2016-05-31 Thread Peng Yu
Hi, The following code shows that "Michał" is printed differently for print(yaml.safe_dump(...)) and the direct print. Does anybody know how to use yaml.safe_dump() so that "Michał" will be printed as is. ~$ cat main.py #!/usr/bin/env python # vim: set noexpandtab tabstop=2 shiftwidth=2 softtabsto

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-05 Thread Wiktor
On 05 Aug 2014 20:26:08 GMT, Tony the Tiger wrote: > On Mon, 04 Aug 2014 00:52:29 +0200, Wiktor wrote: > >> okumenty\python\kolony\menu.py", line 14, in > > This works for me on Linux: I believe you, but I use Windows and its cmd.exe (as mentioned in subject). -- Best regards, Wiktor M

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-05 Thread Grant Edwards
On 2014-08-05, Tony the Tiger wrote: > On Mon, 04 Aug 2014 00:52:29 +0200, Wiktor wrote: > >> okumenty\python\kolony\menu.py", line 14, in > > This works for me on Linux: > > ---8<- > # coding:utf-8 > > test = """ > ┌──╖ > │ Construction ║ > │ Production ║ > │ Re

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-05 Thread Wiktor
On Tue, 05 Aug 2014 04:51:15 +0400, Akira Li wrote: > Unicode has line drawing characters [1]. win_unicode_console [2] allows > to print Unicode in cmd.exe. win_unicode_console and colorama will > probably conflict. You could look at the source to see how hard to > combine both functionalities. >

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Akira Li
Wiktor writes: > On Mon, 04 Aug 2014 15:17:04 -0400, Terry Reedy wrote: > >>>I'm taking next step, so I tried to draw nice frame around menu (that's >>> why I posted yesterday). >> >> Is there no working codepage with ascii text and the line chars? I >> suppose I am not surprised if not. > >

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread giacomo boffi
Wiktor writes: > I'm not starting from scratch. I'm using packages 'termcolor', 'colorama' > and 'colorconsole' the 'urwid' package could be useful for similar projects but requires Linux, OSX, Cygwin or other unix-like OS so I guess it's of no use for you... ciao

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Christian Gollwitzer
Am 04.08.14 01:08, schrieb Chris Angelico: On Mon, Aug 4, 2014 at 8:52 AM, Wiktor wrote: I have to ask - is there a way to make that original concept work? I know, that CP437 has symbols "╖", "╢" and "╘", but does not have polish letters - and I need to display them too. Yeah, that's exactly

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Wiktor
On Mon, 04 Aug 2014 15:17:04 -0400, Terry Reedy wrote: >>I'm taking next step, so I tried to draw nice frame around menu (that's >> why I posted yesterday). > > Is there no working codepage with ascii text and the line chars? I > suppose I am not surprised if not. With single line (└┘┌┐─│

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Chris Angelico
On Tue, Aug 5, 2014 at 5:17 AM, Terry Reedy wrote: > Is there no working codepage with ascii text and the line chars? I suppose I > am not surprised if not. That would be codepage 437. I grew up with that on DOS, as the one and only 256-character set, and then when we moved to OS/2 and actual cod

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Terry Reedy
On 8/4/2014 1:22 PM, Wiktor wrote: On Tue, 5 Aug 2014 03:06:41 +1000, Chris Angelico wrote: On Tue, Aug 5, 2014 at 2:48 AM, Wiktor wrote: From colorama I just use one function - init(). Without this initialization all those ansii escape characters (used by colorama itself, but also by term

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Grant Edwards
On 2014-08-04, Glenn Linderman wrote: > I believe that most Unix terminal emulators, which are used for running > shells and command lines, support cursor controls, and I believe most of > them have a mode that emulates the DEC VT-52 terminal, I'm not aware of any that are in common use, but t

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Terry Reedy
On 8/4/2014 6:24 AM, Wolfgang Maier wrote: On 08/04/2014 11:53 AM, Glenn Linderman wrote: I've never used the API from Python but random console access is documented at http://msdn.microsoft.com/en-us/library/windows/desktop/ms687404%28v=vs.85%29.aspx Would using the API from Python involve

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Terry Reedy
On 8/4/2014 11:00 AM, Wiktor wrote: Yes, I'd like to make text game, that looks like window-based, with popup boxes, inactive windows grayed out and all this stuff. And all this running on standard console window (cmd.exe). Your problem doing this is that cmd.exe is not a standard since 30

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Glenn Linderman
On 8/4/2014 3:24 AM, Wolfgang Maier wrote: On 08/04/2014 11:53 AM, Glenn Linderman wrote: I've never used the API from Python but random console access is documented at http://msdn.microsoft.com/en-us/library/windows/desktop/ms687404%28v=vs.85%29.aspx Would using the API from Python invol

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Glenn Linderman
On 8/4/2014 3:33 AM, Andrew Berg wrote: If you want to save your users the hassle, I would definitely recommend a graphical environment. If I had realized that you intended your application to be widely deployed, I would have simply recommended that from the start. Graphical environments are go

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Wiktor
On Tue, 5 Aug 2014 03:06:41 +1000, Chris Angelico wrote: > On Tue, Aug 5, 2014 at 2:48 AM, Wiktor wrote: >> From colorama I just use one function - init(). Without this >> initialization all those ansii escape characters (used by colorama itself, >> but also by termcolor.colored()) don't work i

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Chris Angelico
On Tue, Aug 5, 2014 at 2:48 AM, Wiktor wrote: > From colorama I just use one function - init(). Without this > initialization all those ansii escape characters (used by colorama itself, > but also by termcolor.colored()) don't work in cmd.exe. At least I couldn't > make it work. I dug into colo

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Wiktor
On Mon, 04 Aug 2014 17:43:41 +0200, Wolfgang Maier wrote: >>I'm not starting from scratch. I'm using packages 'termcolor', 'colorama' >> and 'colorconsole' - they provide functions to print text at desired >> position on screen, and change color of foreground/background of this text. > > Than

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Wolfgang Maier
On 08/04/2014 05:00 PM, Wiktor wrote: Hi, first, thank you all for responses. I decided to just use single line frame around menu. Yes, those double+single line corners work fine in ConEmu, but I don't want this Python script to be dependent on external program. Maybe one day it will be worth

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Wiktor
Hi, first, thank you all for responses. I decided to just use single line frame around menu. Yes, those double+single line corners work fine in ConEmu, but I don't want this Python script to be dependent on external program. Maybe one day it will be worth of showing to others, and it's silly to t

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Andrew Berg
On 2014.08.04 04:46, Glenn Linderman wrote: > How does one "directly run" another application using ConEmu? That wasn't > clear > from what I found to read. It sounded like you run ConEmu, run one or more > shells within it, and launch programs from those shells? And so it was also > unclear if a

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Wolfgang Maier
On 08/04/2014 11:53 AM, Glenn Linderman wrote: I've never used the API from Python but random console access is documented at http://msdn.microsoft.com/en-us/library/windows/desktop/ms687404%28v=vs.85%29.aspx Would using the API from Python involve doing the wrapping yourself or do you know

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Glenn Linderman
k much keep track of stacking order and how to restore what was hidden when a box goes away or is moved down in the stacking order. I would not be surprised if the Atari had at least a rudimentary widget framework. > I tried to use 'standard' Unicode characters (I can see that most of m

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Glenn Linderman
On 8/3/2014 10:06 PM, Andrew Berg wrote: On 2014.08.03 23:14, Glenn Linderman wrote: Having read a bit about ConEmu, it seems that it is a "pretty face" built on top of Windows Console, by screen scraping the real (but hidden) Windows Console, and providing a number of interesting display featur

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Chris “Kwpolska” Warrick
On Mon, Aug 4, 2014 at 2:17 AM, Glenn Linderman wrote: > For this OP problem, it is mostly a matter of finding a fixed-width font > that supports the box drawing characters and the Polish characters that are > desired. Lucida Console has a fair repertoire, and Consolas has a fair > repertoire, in

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-04 Thread Terry Reedy
tore what was hidden when a box goes away or is moved down in the stacking order. I would not be surprised if the Atari had at least a rudimentary widget framework. > I tried to use 'standard' Unicode characters (I can see that most of my Windows monospaced fonts have them) to draw

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-03 Thread Andrew Berg
On 2014.08.03 23:14, Glenn Linderman wrote: > Having read a bit about ConEmu, it seems that it is a "pretty face" built on > top of Windows Console, by screen scraping the real (but hidden) Windows > Console, and providing a number of interesting display features and modes. So > while it adds funct

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-03 Thread Glenn Linderman
e has been that input can be difficult, but output works well). I personally have used an IRC bot written in Python with logging output containing Unicode characters that display just fine (both locally and over SSH). [1] I recommend ConEmu:https://code.google.com/p/conemu-maximus5/ I will be re

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-03 Thread Glenn Linderman
tput works well). I personally have used an IRC bot written in Python with logging output containing Unicode characters that display just fine (both locally and over SSH). [1] I recommend ConEmu: https://code.google.com/p/conemu-maximus5/ I will be reading more about conemu, thanks for the refere

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-03 Thread Chris Angelico
On Mon, Aug 4, 2014 at 9:39 AM, Chris Angelico wrote: > I just played around with a CP-437 decode of everything 128-255, > rendered in various different fonts, all using my MUD client on > Windows. (For what it's worth, it renders using GTK2 and Pango. But I > suspect this is more a font issue tha

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-03 Thread Chris Angelico
fect (my experience has been that input can be > difficult, but output works well). I personally have used an IRC bot written > in > Python with logging output containing Unicode characters that display just > fine > (both locally and over SSH). > > [1] I recommend ConEmu: http

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-03 Thread Mark Lawrence
tput works well). I personally have used an IRC bot written in Python with logging output containing Unicode characters that display just fine (both locally and over SSH). [1] I recommend ConEmu: https://code.google.com/p/conemu-maximus5/ *facepalm* forgot all about ConEmu, but then I only use it

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-03 Thread Andrew Berg
nally have used an IRC bot written in Python with logging output containing Unicode characters that display just fine (both locally and over SSH). [1] I recommend ConEmu: https://code.google.com/p/conemu-maximus5/ -- https://mail.python.org/mailman/listinfo/python-list

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-03 Thread Mark Lawrence
odern rewritten (for Atari emulators) version: Kolony 2106 https://www.youtube.com/watch?v=eX20Qqqm5eg - you get the idea? ;-)). OO Design is one thing, but I want to make it look as near as possible to the original (those windows-like menus in console window). I tried to use 'standard' Unic

Re: cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-03 Thread Chris Angelico
On Mon, Aug 4, 2014 at 8:52 AM, Wiktor wrote: > I have to ask - is there a way to make that original concept work? I know, > that CP437 has symbols "╖", "╢" and "╘", but does not have polish letters - > and I need to display them too. Yeah, that's exactly the problem with codepages :) The best w

cmd.exe on WIndows - problem with displaying some Unicode characters

2014-08-03 Thread Wiktor
version: Kolony 2106 https://www.youtube.com/watch?v=eX20Qqqm5eg - you get the idea? ;-)). OO Design is one thing, but I want to make it look as near as possible to the original (those windows-like menus in console window). I tried to use 'standard' Unicode characters (I can see that most

Re: pyw program not displaying unicode characters properly

2012-10-15 Thread Roy Smith
In article , Dennis Lee Bieber wrote: > Classically, NNTP did not have "attachments" as seen in MIME email. NNTP (Network News Transport Protocol) and SMTP (Simple Mail Transfer Protocol) are both just ways of shipping around messages. Neither one really knows about attachments. In bo

Re: pyw program not displaying unicode characters properly

2012-10-14 Thread Alain Ketterlin
Steven D'Aprano writes: > On Sun, 14 Oct 2012 19:19:33 +0200, Alain Ketterlin wrote: > >> Usenet has no attachments. > > *snarfle* > > You almost owed me a new monitor. I nearly sprayed my breakfast all over > it. [...] I owe you nothing, and you can do whatever you want with your breakfast.

Re: pyw program not displaying unicode characters properly

2012-10-14 Thread jjmeric
In article , ian.g.ke...@gmail.com says... > > On Sun, Oct 14, 2012 at 1:36 PM, jjmeric wrote: > > Is there some sort of defaut font, or is there in Python or Python for > > Windows any ini file where the font used can be seen, eventually changed > > to a more appropriate one with all the requir

Re: pyw program not displaying unicode characters properly

2012-10-14 Thread Ian Kelly
On Sun, Oct 14, 2012 at 1:36 PM, jjmeric wrote: > Is there some sort of defaut font, or is there in Python or Python for > Windows any ini file where the font used can be seen, eventually changed > to a more appropriate one with all the required glyphs (like Lucida Sans > Unicode has). No, this i

Re: pyw program not displaying unicode characters properly

2012-10-14 Thread Steven D'Aprano
On Sun, 14 Oct 2012 19:19:33 +0200, Alain Ketterlin wrote: > Usenet has no attachments. *snarfle* You almost owed me a new monitor. I nearly sprayed my breakfast all over it. "Usenet has no attachments" -- that's like saying that the Web has no advertisements. Maybe the websites you visit ha

Re: pyw program not displaying unicode characters properly

2012-10-14 Thread Roy Smith
In article , MRAB wrote: > Which codepoint is it? What is the codepoint's name? > > Here's how to find out: > > >>> hex(ord("?")) > '0x190' > >>> import unicodedata > >>> unicodedata.name("?") > 'LATIN CAPITAL LETTER OPEN E' Wow, I never knew you could do that. I usually just google for

Re: pyw program not displaying unicode characters properly

2012-10-14 Thread jjmeric
Alain, MRAB Thank you for prompt responses. What they suggest to me is I should look into what font is being used by this Python for Windows program. I am not the programmer, so not idea where to look for. The program settings do not include a choice for display font. The font that used for disp

Re: pyw program not displaying unicode characters properly

2012-10-14 Thread MRAB
On 2012-10-14 17:55, jjmeric wrote: Hi everybody ! Our language lab at INALCO is using a nice language parsing and analysis program written in Python. As you well know a lot of languages use characters that can only be handled by unicode. Here is an example of the problem we have on some Windo

Re: pyw program not displaying unicode characters properly

2012-10-14 Thread Alain Ketterlin
isplay the > characters on others suggests that it is a user configuration issue: > Recent observations: it's OK on Windows 7 but not on Vista computers, > it's OK on some Windows XP computers, it's not on others Windows XP... You need a font that has glyphs for all unico

pyw program not displaying unicode characters properly

2012-10-14 Thread jjmeric
Hi everybody ! Our language lab at INALCO is using a nice language parsing and analysis program written in Python. As you well know a lot of languages use characters that can only be handled by unicode. Here is an example of the problem we have on some Windows computers. In the attached screen

Benchmarking stripping of Unicode characters which are invalid XML

2012-03-18 Thread Alex Willmer
Last week I was surprised to discover that there are Unicode characters that aren't valid in an XML document. That is regardless of escaping (e.g. �) and unicode encoding (e.g. UTF-8) - not every Unicode string can be stored in XML. The valid characters are (as of XML 1.0) #x9 | #xA

Re: Py3: Read file with Unicode characters

2010-04-08 Thread Gnarlodious
On Apr 8, 11:04 am, "Martin v. Loewis" wrote: > That's because you need to re-learn some things. Apparently so, every little item is a lesson. Thank you. -- Gnarlie -- http://mail.python.org/mailman/listinfo/python-list

Re: Py3: Read file with Unicode characters

2010-04-08 Thread Martin v. Loewis
Gnarlodious wrote: > On Apr 8, 9:14 am, "Martin v. Loewis" wrote: > >> When opening the file, you need to specify the file encoding. > > OK, I had tried this: > > open(path, 'r').read().encode('utf-8') No, when *opening* the file, you need to specify the encoding: open(path, 'r', encoding='utf

Re: Py3: Read file with Unicode characters

2010-04-08 Thread Gnarlodious
On Apr 8, 9:14 am, "Martin v. Loewis" wrote: > When opening the file, you need to specify the file encoding. OK, I had tried this: open(path, 'r').read().encode('utf-8') however I get error TypeError: Can't convert 'bytes' object to str implicitly I had assumed a Unicode string was a Unicode

Re: Py3: Read file with Unicode characters

2010-04-08 Thread Martin v. Loewis
Gnarlodious wrote: > Attempting to read a file containing Unicode characters such as ±: > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position > 5007: ordinal not in range(128) > > I did succeed by converting all the characters to HTML entities

Py3: Read file with Unicode characters

2010-04-08 Thread Gnarlodious
Attempting to read a file containing Unicode characters such as ±: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 5007: ordinal not in range(128) I did succeed by converting all the characters to HTML entities such as "±", but I want the characters

Re: Unicode characters in btye-strings

2010-03-12 Thread Martin v. Loewis
Michael Rudolf wrote: > Am 12.03.2010 21:56, schrieb Martin v. Loewis: >> (*) If a source encoding was given, the source is actually recoded to >> UTF-8, parsed, and then re-encoded back into the original encoding. > > Why is that? Why is what? That string literals get reencoded into the source e

Re: Unicode characters in btye-strings

2010-03-12 Thread John Bokma
Michael Rudolf writes: > Am 12.03.2010 21:56, schrieb Martin v. Loewis: >> (*) If a source encoding was given, the source is actually recoded to >> UTF-8, parsed, and then re-encoded back into the original encoding. > > Why is that? So "unicode"-strings (as in u"string") are not really > unicode-

Re: Unicode characters in btye-strings

2010-03-12 Thread Michael Rudolf
Am 12.03.2010 21:56, schrieb Martin v. Loewis: (*) If a source encoding was given, the source is actually recoded to UTF-8, parsed, and then re-encoded back into the original encoding. Why is that? So "unicode"-strings (as in u"string") are not really unicode-, but utf8-strings? Need citatio

Re: Unicode characters in btye-strings

2010-03-12 Thread Martin v. Loewis
>> Can somebody explain what happens when I put non-ASCII characters into a >> non-unicode string? My guess is that the result will depend on the >> current encoding of my terminal. > > Exactly right. To elaborate on the "what happens" part: the string that gets entered is typically passed as a b

Re: Unicode characters in btye-strings

2010-03-12 Thread Robert Kern
On 2010-03-12 06:35 AM, Steven D'Aprano wrote: I know this is wrong, but I'm not sure just how wrong it is, or why. Using Python 2.x: s = "éâÄ" print s éâÄ len(s) 6 list(s) ['\xc3', '\xa9', '\xc3', '\xa2', '\xc3', '\x84'] Can somebody explain what happens when I put non-ASCII characters i

Unicode characters in btye-strings

2010-03-12 Thread Steven D'Aprano
I know this is wrong, but I'm not sure just how wrong it is, or why. Using Python 2.x: >>> s = "éâÄ" >>> print s éâÄ >>> len(s) 6 >>> list(s) ['\xc3', '\xa9', '\xc3', '\xa2', '\xc3', '\x84'] Can somebody explain what happens when I put non-ASCII characters into a non-unicode string? My guess is

Re: How do I decode unicode characters in the subject using email.message_from_string()?

2009-02-25 Thread Gabriel Genellina
ou would think the header contains Unicode characters >>> when it says "us-ascii" ("=?us-ascii?Q?"). I think there is a tendency >>> to label everything "Unicode" someone does not understand. >> And I wonder why you would think the h

Re: How do I decode unicode characters in the subject using email.message_from_string()?

2009-02-25 Thread Gabriel Genellina
En Wed, 25 Feb 2009 15:44:18 -0200, escribió: Tab is not mentioned in RFC 2822 except to say that it is a valid whitespace character. Header folding (insertion of ) can occur most places whitespace appears, and is defined in section 2.2.3 thusly: [...] So, the whitespace characters are suppose

Re: How do I decode unicode characters in the subject using email.message_from_string()?

2009-02-25 Thread Thorsten Kampe
* Tim Golden (Wed, 25 Feb 2009 17:27:07 +) > Thorsten Kampe wrote: > > * Gabriel Genellina (Wed, 25 Feb 2009 14:00:16 -0200) > >> En Wed, 25 Feb 2009 13:40:31 -0200, Thorsten Kampe [...] > >>> And I wonder why you would think the header contains Unicode cha

  1   2   >