subject:"Re\: Unicode Problem"

Re: Unicode Problem

2015-01-29 Thread Kubilay Kocak

On 29/01/2015 6:13 PM, Robert Simmons wrote:
 On further inspection I've found the following:
 
 FreeBSD
 import sys
 print(sys.getdefaultencoding())
 utf-8
 print(sys.stdout.encoding)
 US-ASCII
 
 MacOS X:
 import sys
 print(sys.getdefaultencoding())
 utf-8
 print(sys.stdout.encoding)
 UTF-8
 
 How do I modify stdout encoding to set it to UTF-8 in FreeBSD?

Another data point from my 9-STABLE:

Python 3.4.2 (default, Nov  3 2014, 13:38:18)
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
208032)] on freebsd9
Type help, copyright, credits or license for more information.
 b'\xc3\xa2'.decode('utf-8')
'â'
 import sys
 print(sys.getdefaultencoding())
utf-8
 print(sys.stdout.encoding)
UTF-8


Python 2.7.9 (default, Jan 24 2015, 20:39:40)
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
208032)] on freebsd9
Type help, copyright, credits or license for more information.
 b'\xc3\xa2'.decode('utf-8')
u'\xe2'
 import sys
 print(sys.getdefaultencoding())
ascii
 print(sys.stdout.encoding)
UTF-8

___
freebsd-python@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-python
To unsubscribe, send any mail to freebsd-python-unsubscr...@freebsd.org

Re: Unicode Problem

2015-01-29 Thread Konstantin Belousov

On Thu, Jan 29, 2015 at 08:32:35PM +1100, Kubilay Kocak wrote:
 On 29/01/2015 6:13 PM, Robert Simmons wrote:
  On further inspection I've found the following:
  
  FreeBSD
  import sys
  print(sys.getdefaultencoding())
  utf-8
  print(sys.stdout.encoding)
  US-ASCII
  
  MacOS X:
  import sys
  print(sys.getdefaultencoding())
  utf-8
  print(sys.stdout.encoding)
  UTF-8
  
  How do I modify stdout encoding to set it to UTF-8 in FreeBSD?
 
 Another data point from my 9-STABLE:
 
 Python 3.4.2 (default, Nov  3 2014, 13:38:18)
 [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
 208032)] on freebsd9
 Type help, copyright, credits or license for more information.
  b'\xc3\xa2'.decode('utf-8')
 '??'
  import sys
  print(sys.getdefaultencoding())
 utf-8
  print(sys.stdout.encoding)
 UTF-8
 
 
 Python 2.7.9 (default, Jan 24 2015, 20:39:40)
 [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
 208032)] on freebsd9
 Type help, copyright, credits or license for more information.
  b'\xc3\xa2'.decode('utf-8')
 u'\xe2'
  import sys
  print(sys.getdefaultencoding())
 ascii
  print(sys.stdout.encoding)
 UTF-8
 

Encoding of the sys.stdout is determined by the language environment.
Set LANG environment variable to something like en_US.UTF-8 to get
UTF-8 output from python.

The 'data points' about python version, FreeBSD version, compiler
version, colour of the enclosure for hard disk used, or even phase of
the Moon have lesser impact on the sys.stdout.encoding.
___
freebsd-python@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-python
To unsubscribe, send any mail to freebsd-python-unsubscr...@freebsd.org

Re: Unicode Problem

2015-01-29 Thread Robert Simmons

I appreciate the assistance. Setting the LANG variable does the trick.
I made the change system-wide by adding the following to
/etc/login.conf
LC_COLLATE=C
charset=UTF-8
lang=en_US.UTF-8

Now the output is correct:
 b'\xc3\xa2'.decode('utf-8')
'â'
 import sys
 print(sys.stdout.encoding)
UTF-8

On Thu, Jan 29, 2015 at 4:53 AM, Konstantin Belousov
kostik...@gmail.com wrote:
 Encoding of the sys.stdout is determined by the language environment.
 Set LANG environment variable to something like en_US.UTF-8 to get
 UTF-8 output from python.
___
freebsd-python@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-python
To unsubscribe, send any mail to freebsd-python-unsubscr...@freebsd.org

Re: Unicode Problem

2015-01-29 Thread Rainer Hurling

Am 29.01.2015 um 10:53 schrieb Konstantin Belousov:
 On Thu, Jan 29, 2015 at 08:32:35PM +1100, Kubilay Kocak wrote:
 On 29/01/2015 6:13 PM, Robert Simmons wrote:
 On further inspection I've found the following:

 FreeBSD
 import sys
 print(sys.getdefaultencoding())
 utf-8
 print(sys.stdout.encoding)
 US-ASCII

 MacOS X:
 import sys
 print(sys.getdefaultencoding())
 utf-8
 print(sys.stdout.encoding)
 UTF-8

 How do I modify stdout encoding to set it to UTF-8 in FreeBSD?

 Another data point from my 9-STABLE:

 Python 3.4.2 (default, Nov  3 2014, 13:38:18)
 [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
 208032)] on freebsd9
 Type help, copyright, credits or license for more information.
 b'\xc3\xa2'.decode('utf-8')
 '??'
 import sys
 print(sys.getdefaultencoding())
 utf-8
 print(sys.stdout.encoding)
 UTF-8


 Python 2.7.9 (default, Jan 24 2015, 20:39:40)
 [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
 208032)] on freebsd9
 Type help, copyright, credits or license for more information.
 b'\xc3\xa2'.decode('utf-8')
 u'\xe2'
 import sys
 print(sys.getdefaultencoding())
 ascii
 print(sys.stdout.encoding)
 UTF-8


On my box, with recent HEAD amd64, it is like for Koobs:

#locale
LANG=de_DE.UTF-8
LC_CTYPE=de_DE.UTF-8
LC_COLLATE=C
LC_TIME=de_DE.UTF-8
LC_NUMERIC=de_DE.UTF-8
LC_MONETARY=de_DE.UTF-8
LC_MESSAGES=de_DE.UTF-8
LC_ALL=


#python3
Python 3.4.2 (default, Jan 11 2015, 07:51:41)
[GCC 4.2.1 Compatible FreeBSD Clang 3.5.0 (tags/RELEASE_350/final
216957)] on freebsd11
Type help, copyright, credits or license for more information.
### b'\xc3\xa2'.decode('utf-8')
'â'


[For python 27, option UCS4 enabled]
#python
Python 2.7.9 (default, Jan 24 2015, 10:35:50)
[GCC 4.2.1 Compatible FreeBSD Clang 3.5.1 (tags/RELEASE_351/final
225668)] on freebsd11
Type help, copyright, credits or license for more information.
### b'\xc3\xa2'.decode('utf-8')
u'\xe2'

So, obviously there is a difference between the python versions,
independently from the locale settings?

___
freebsd-python@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-python
To unsubscribe, send any mail to freebsd-python-unsubscr...@freebsd.org

Re: Unicode Problem

2015-01-29 Thread Roland Smith

On Thu, Jan 29, 2015 at 02:42:31AM -0500, Robert Simmons wrote:
 On Thu, Jan 29, 2015 at 2:29 AM, Roland Smith rsm...@xs4all.nl wrote:
  On Thu, Jan 29, 2015 at 01:38:21AM -0500, Robert Simmons wrote:
  I'm having a unicode problem on FreeBSD lang/python34 that does not
  appear on MacOS X. I've condensed the problem to one single line to
  enter in the interpreter:
 
  FreeBSD:
  Python 3.4.2 (default, Jan 28 2015, 22:23:57)
  [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
  208032)] on freebsd10
  Type help, copyright, credits or license for more information.
   b'\xc3\xa2'.decode('utf-8')
  '\xe2'
 
  MacOS X:
  Python 3.4.2 (default, Oct 19 2014, 17:55:38)
  [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin
  Type help, copyright, credits or license for more information.
   b'\xc3\xa2'.decode('utf-8')
  'â'
 
  Why is Python on FreeBSD incorrectly decoding this?
 
  Works fine here (FreeBSD 10.1-STABLE #0 r276653 amd64):
 
  Python 3.4.2 (default, Nov  4 2014, 19:34:48)
  [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 
  208032)] on freebsd10
  Type help, copyright, credits or license for more information.
   b'\xc3\xa2'.decode('utf-8')
  'â'

(please don't top-post)

 What is the output from print(sys.stdout.encoding) on your system?

Python 3.4.2 (default, Nov  4 2014, 19:34:48) 
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 
208032)] on freebsd10
Type help, copyright, credits or license for more information.
 import sys
 print(sys.stdout.encoding)
UTF-8

 And, can you explain how to change that on mine so that it is UTF-8?
 Mine is a default fresh install, btw.

In /etc/login.conf, I set LC_ALL=en_US.UTF-8;

default:\
:passwd_format=sha512:\
:copyright=/etc/COPYRIGHT:\
:welcome=/etc/motd:\
:setenv=MAIL=/var/mail/$,BLOCKSIZE=K,LC_ALL=en_US.UTF-8:\
:path=/sbin /bin /usr/sbin /usr/bin /usr/games /usr/local/sbin 
/usr/local/bin

And I use a unicode aware X terminal (rxvt-unicode).

In case you're not using X11, the new vt(4) device uses UTF-8, but the old
sc(4) doesn't support it at all, AFAIK.

Roland
-- 
R.F.Smith   http://rsmith.home.xs4all.nl/
[plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated]
pgp: 5753 3324 1661 B0FE 8D93  FCED 40F6 D5DC A38A 33E0 (keyID: A38A33E0)


pgpp4MY4jhhiJ.pgp
Description: PGP signature

Re: Unicode Problem

2015-01-28 Thread Roland Smith

On Thu, Jan 29, 2015 at 01:38:21AM -0500, Robert Simmons wrote:
 I'm having a unicode problem on FreeBSD lang/python34 that does not
 appear on MacOS X. I've condensed the problem to one single line to
 enter in the interpreter:
 
 FreeBSD:
 Python 3.4.2 (default, Jan 28 2015, 22:23:57)
 [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
 208032)] on freebsd10
 Type help, copyright, credits or license for more information.
  b'\xc3\xa2'.decode('utf-8')
 '\xe2'
 
 MacOS X:
 Python 3.4.2 (default, Oct 19 2014, 17:55:38)
 [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin
 Type help, copyright, credits or license for more information.
  b'\xc3\xa2'.decode('utf-8')
 'â'
 
 Why is Python on FreeBSD incorrectly decoding this?

Works fine here (FreeBSD 10.1-STABLE #0 r276653 amd64):

Python 3.4.2 (default, Nov  4 2014, 19:34:48)
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 
208032)] on freebsd10
Type help, copyright, credits or license for more information.
 b'\xc3\xa2'.decode('utf-8')
'â'


Roland
-- 
R.F.Smith   http://rsmith.home.xs4all.nl/
[plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated]
pgp: 5753 3324 1661 B0FE 8D93  FCED 40F6 D5DC A38A 33E0 (keyID: A38A33E0)


pgp2zogLjVeoo.pgp
Description: PGP signature

Re: Unicode Problem

2015-01-28 Thread Robert Simmons

What is the output from print(sys.stdout.encoding) on your system?
And, can you explain how to change that on mine so that it is UTF-8?
Mine is a default fresh install, btw.

On Thu, Jan 29, 2015 at 2:29 AM, Roland Smith rsm...@xs4all.nl wrote:
 On Thu, Jan 29, 2015 at 01:38:21AM -0500, Robert Simmons wrote:
 I'm having a unicode problem on FreeBSD lang/python34 that does not
 appear on MacOS X. I've condensed the problem to one single line to
 enter in the interpreter:

 FreeBSD:
 Python 3.4.2 (default, Jan 28 2015, 22:23:57)
 [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
 208032)] on freebsd10
 Type help, copyright, credits or license for more information.
  b'\xc3\xa2'.decode('utf-8')
 '\xe2'

 MacOS X:
 Python 3.4.2 (default, Oct 19 2014, 17:55:38)
 [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin
 Type help, copyright, credits or license for more information.
  b'\xc3\xa2'.decode('utf-8')
 'â'

 Why is Python on FreeBSD incorrectly decoding this?

 Works fine here (FreeBSD 10.1-STABLE #0 r276653 amd64):

 Python 3.4.2 (default, Nov  4 2014, 19:34:48)
 [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 
 208032)] on freebsd10
 Type help, copyright, credits or license for more information.
  b'\xc3\xa2'.decode('utf-8')
 'â'


 Roland
 --
 R.F.Smith   http://rsmith.home.xs4all.nl/
 [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated]
 pgp: 5753 3324 1661 B0FE 8D93  FCED 40F6 D5DC A38A 33E0 (keyID: A38A33E0)
___
freebsd-python@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-python
To unsubscribe, send any mail to freebsd-python-unsubscr...@freebsd.org

Re: Unicode Problem

2015-01-28 Thread Robert Simmons

On further inspection I've found the following:

FreeBSD
 import sys
 print(sys.getdefaultencoding())
utf-8
 print(sys.stdout.encoding)
US-ASCII

MacOS X:
 import sys
 print(sys.getdefaultencoding())
utf-8
 print(sys.stdout.encoding)
UTF-8

How do I modify stdout encoding to set it to UTF-8 in FreeBSD?

On Thu, Jan 29, 2015 at 1:38 AM, Robert Simmons rsimmo...@gmail.com wrote:
 I'm having a unicode problem on FreeBSD lang/python34 that does not
 appear on MacOS X. I've condensed the problem to one single line to
 enter in the interpreter:

 FreeBSD:
 Python 3.4.2 (default, Jan 28 2015, 22:23:57)
 [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
 208032)] on freebsd10
 Type help, copyright, credits or license for more information.
 b'\xc3\xa2'.decode('utf-8')
 '\xe2'

 MacOS X:
 Python 3.4.2 (default, Oct 19 2014, 17:55:38)
 [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin
 Type help, copyright, credits or license for more information.
 b'\xc3\xa2'.decode('utf-8')
 'â'

 Why is Python on FreeBSD incorrectly decoding this?
___
freebsd-python@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-python
To unsubscribe, send any mail to freebsd-python-unsubscr...@freebsd.org

Re: Unicode Problem

Re: Unicode Problem

Re: Unicode Problem

Re: Unicode Problem

Re: Unicode Problem

Re: Unicode Problem

Re: Unicode Problem

Re: Unicode Problem

8 matches

Site Navigation

Mail list logo

Footer information