Re: Another å, ä, ö question

[Martin Schöön]

Den 2016-09-22 skrev Peter Otten <__pete...@web.de>:
> Martin Schöön wrote:
>> 
>> I am not sure I answer your questions since I am not quite sure I
>> understand it but here goes: The complete file is UTF-8 encoded as
>> that is how Geany is set up to create new files (I just checked).
>
> When the encoding used for the file and the encoding used by the terminal 
> differ the output of non-ascii characters gets messed up. Example script:
>
> # -*- coding: iso-8859-15 -*-
>
> print "first unicode:"
> print u"Schöön"
>
> print "then bytes:"
> print "Schöön"
>
> When I dump that in my UTF-8 terminal all "ö"s are lost because it gets the 
> invalid byte sequence b"\xf6" rather than the required b"\xc3\xb6":
>
> $ cat demo.py
> # -*- coding: iso-8859-15 -*-
>
> print "first unicode:"
> print u"Sch��n"
>
> print "then bytes:"
> print "Sch��n"
>
> But when I run the code:
>
> $ python demo.py
> first unicode:
> Schöön
> then bytes:
> Sch��n
>
> There are other advantages, too:
>
 print "Schöön".upper()
> SCHööN
 print u"Schöön".upper()
> SCHÖÖN
 print "Schöön"[:4]
> Sch�
 print u"Schöön"[:4]
> Schö
>
Cool :-)
Thanks for the education.

/Martin
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Chris Angelico]

On Thu, Sep 22, 2016 at 10:27 PM, Peter Otten <__pete...@web.de> wrote:
> When the encoding used for the file and the encoding used by the terminal
> differ the output of non-ascii characters gets messed up. Example script:
>
> # -*- coding: iso-8859-15 -*-
>
> print "first unicode:"
> print u"Schöön"
>
> print "then bytes:"
> print "Schöön"
>
> When I dump that in my UTF-8 terminal all "ö"s are lost because it gets the
> invalid byte sequence b"\xf6" rather than the required b"\xc3\xb6":
>
> $ cat demo.py
> # -*- coding: iso-8859-15 -*-
>
> print "first unicode:"
> print u"Sch��n"
>
> print "then bytes:"
> print "Sch��n"
>
> But when I run the code:
>
> $ python demo.py
> first unicode:
> Schöön
> then bytes:
> Sch��n

What this really means is that you (almost certainly) shouldn't be
storing non-ASCII text in byte strings. Most stuff will "just work" if
you're using a Unicode string (obviously cat doesn't acknowledge the
coding cookie, but Python itself does, as do a number of editors), and
of course, you can avoid all the u"..." prefixes by going to Py3.
Trying to use text in byte strings is extremely encoding-dependent,
and thus dangerous. Sure, it'll generally work for ASCII... but only
because you're highly likely to have your terminal set to an
ASCII-compatible encoding. If you pick something else you're in
for a whole new world of fun. Acres of entertainment.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Peter Otten]

Martin Schöön wrote:

> Den 2016-09-20 skrev Peter Otten <__pete...@web.de>:
>> Martin Schöön wrote:
>>
>>> Den 2016-09-19 skrev Christian Gollwitzer :
 Am 19.09.16 um 22:21 schrieb Martin Schöön:
> I am studying some of these tutorials:
> https://pythonprogramming.net/matplotlib-intro-tutorial/
> 

 Assuming that you use UTF-8 (you should check with an emacs expert, I
 am not an emacs user), try putting the header

 #!/usr/bin/python
 # -*- coding: utf-8 -*-

 on top of your files.

>>> I already have this and since it doesn't work from command line
>>> either it can't be an emacs unique problem.
>>> 
>>
>> Are all non-ascii strings unicode? I. e.
>>
>> u"Schöön" rather than just "Schöön"
>>
>> ?
> 
> I am not sure I answer your questions since I am not quite sure I
> understand it but here goes: The complete file is UTF-8 encoded as
> that is how Geany is set up to create new files (I just checked).

When the encoding used for the file and the encoding used by the terminal 
differ the output of non-ascii characters gets messed up. Example script:

# -*- coding: iso-8859-15 -*-

print "first unicode:"
print u"Schöön"

print "then bytes:"
print "Schöön"

When I dump that in my UTF-8 terminal all "ö"s are lost because it gets the 
invalid byte sequence b"\xf6" rather than the required b"\xc3\xb6":

$ cat demo.py
# -*- coding: iso-8859-15 -*-

print "first unicode:"
print u"Sch��n"

print "then bytes:"
print "Sch��n"

But when I run the code:

$ python demo.py
first unicode:
Schöön
then bytes:
Sch��n

There are other advantages, too:

>>> print "Schöön".upper()
SCHööN
>>> print u"Schöön".upper()
SCHÖÖN
>>> print "Schöön"[:4]
Sch�
>>> print u"Schöön"[:4]
Schö


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Martin Schöön]

Den 2016-09-20 skrev Peter Otten <__pete...@web.de>:
> Martin Schöön wrote:
>
>> Den 2016-09-19 skrev Christian Gollwitzer :
>>> Am 19.09.16 um 22:21 schrieb Martin Schöön:
 I am studying some of these tutorials:
 https://pythonprogramming.net/matplotlib-intro-tutorial/

>>>
>>> Assuming that you use UTF-8 (you should check with an emacs expert, I am
>>> not an emacs user), try putting the header
>>>
>>> #!/usr/bin/python
>>> # -*- coding: utf-8 -*-
>>>
>>> on top of your files.
>>>
>> I already have this and since it doesn't work from command line
>> either it can't be an emacs unique problem.
>> 
>
> Are all non-ascii strings unicode? I. e.
>
> u"Schöön" rather than just "Schöön"
>
> ?

I am not sure I answer your questions since I am not quite sure I
understand it but here goes: The complete file is UTF-8 encoded as
that is how Geany is set up to create new files (I just checked).

/Martin
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Martin Schöön]

Den 2016-09-20 skrev Chris Angelico :
> On Wed, Sep 21, 2016 at 6:21 AM, Martin Schöön  
> wrote:
>> Den 2016-09-19 skrev Lawrence D’Oliveiro :
>>> On Tuesday, September 20, 2016 at 8:21:25 AM UTC+12, Martin Schöön wrote:
 But -- now I tested using emacs instead using C-c C-c to execute.
 Noting happens so I try to run the program from command line and
 find that now Python can't stand my å, ä and ö.
>>>
>>> What version of Python? Python 3 accepts Unicode UTF-8 as a matter of 
>>> course.
>>
>> Python 2.7.
>> I just tried running my code in Python 3 and that worked like charm.
>
> Then you've found the solution. Py2's Unicode support is weaker than
> Py3's, and it often depends on encodings.
>
Yes, so it seems. I have told emacs to invoke python3 instead of python
and then it worked like just fine.

I assume there is a setting somewhere in Geany that tells it to run
python3 rather than python but I have failed to find it.

Case closed as far as I am concerned.

Thanks for the help and patience.

/Martin
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Chris Angelico]

On Wed, Sep 21, 2016 at 6:21 AM, Martin Schöön  wrote:
> Den 2016-09-19 skrev Lawrence D’Oliveiro :
>> On Tuesday, September 20, 2016 at 8:21:25 AM UTC+12, Martin Schöön wrote:
>>> But -- now I tested using emacs instead using C-c C-c to execute.
>>> Noting happens so I try to run the program from command line and
>>> find that now Python can't stand my å, ä and ö.
>>
>> What version of Python? Python 3 accepts Unicode UTF-8 as a matter of course.
>
> Python 2.7.
> I just tried running my code in Python 3 and that worked like charm.

Then you've found the solution. Py2's Unicode support is weaker than
Py3's, and it often depends on encodings.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Peter Otten]

Martin Schöön wrote:

> Den 2016-09-19 skrev Christian Gollwitzer :
>> Am 19.09.16 um 22:21 schrieb Martin Schöön:
>>> I am studying some of these tutorials:
>>> https://pythonprogramming.net/matplotlib-intro-tutorial/
>>>
>>> I am recreating the code and I use my native Swedish for comments,
>>> labels and titles. Until now I have been doing so using Geany
>>> and executing using F5. This works fine.
>>>
>>> But -- now I tested using emacs instead using C-c C-c to execute.
>>> Noting happens so I try to run the program from command line and
>>> find that now Python can't stand my å, ä and ö.
>>>
>>> I am puzzled: With Geany there is no problem but outside Geany
>>> I am punished by Python for using Swedish.
>>
>> you are not "punished for Swedish", you need to tell Python the encoding
>> of the file it runs. See here:
>> https://www.python.org/dev/peps/pep-0263/
>>
>> Assuming that you use UTF-8 (you should check with an emacs expert, I am
>> not an emacs user), try putting the header
>>
>> #!/usr/bin/python
>> # -*- coding: utf-8 -*-
>>
>> on top of your files.
>>
> I already have this and since it doesn't work from command line
> either it can't be an emacs unique problem.
> 
> Still confused...
> 
> Too late to find a emacs forum tonight.
> 
> /Martin

Are all non-ascii strings unicode? I. e.

u"Schöön" rather than just "Schöön"

?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Martin Schöön]

Den 2016-09-19 skrev Lawrence D’Oliveiro :
> On Tuesday, September 20, 2016 at 8:21:25 AM UTC+12, Martin Schöön wrote:
>> But -- now I tested using emacs instead using C-c C-c to execute.
>> Noting happens so I try to run the program from command line and
>> find that now Python can't stand my å, ä and ö.
>
> What version of Python? Python 3 accepts Unicode UTF-8 as a matter of course.

Python 2.7.
I just tried running my code in Python 3 and that worked like charm.

/Martin
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Martin Schöön]

Den 2016-09-19 skrev Christian Gollwitzer :
> Am 19.09.16 um 22:21 schrieb Martin Schöön:
>> I am studying some of these tutorials:
>> https://pythonprogramming.net/matplotlib-intro-tutorial/
>>
>> I am recreating the code and I use my native Swedish for comments,
>> labels and titles. Until now I have been doing so using Geany
>> and executing using F5. This works fine.
>>
>> But -- now I tested using emacs instead using C-c C-c to execute.
>> Noting happens so I try to run the program from command line and
>> find that now Python can't stand my å, ä and ö.
>>
>> I am puzzled: With Geany there is no problem but outside Geany
>> I am punished by Python for using Swedish.
>
> you are not "punished for Swedish", you need to tell Python the encoding 
> of the file it runs. See here:
> https://www.python.org/dev/peps/pep-0263/
>
> Assuming that you use UTF-8 (you should check with an emacs expert, I am 
> not an emacs user), try putting the header
>
> #!/usr/bin/python
> # -*- coding: utf-8 -*-
>
> on top of your files.
>
I already have this and since it doesn't work from command line
either it can't be an emacs unique problem.

Still confused...

Too late to find a emacs forum tonight.

/Martin
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Steven D'Aprano]

On Tuesday 20 September 2016 06:21, Martin Schöön wrote:

> I am puzzled: With Geany there is no problem but outside Geany
> I am punished by Python for using Swedish.
> 
> Any ideas?

Yes, you are being punished for what Sweden did to Julian Assange.

No, only kidding.

We can't really help you if you don't tell us exactly what error you get. Is 
this an error in emacs? A Python error? What exception do you get?

But if I were to take a guess, I think you should put an encoding cookie in the 
first or second line of your text file. Make sure your editor is configured to 
use UTF-8 as the encoding, and put this in line 1 or line 2 of the file:


# -*- coding: utf-8 -*-


That will tell Python to read your file as UTF-8 instead of ASCII.



-- 
Steven
git gets easier once you get the basic idea that branches are homeomorphic 
endofunctors mapping submanifolds of a Hilbert space.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Lawrence D’Oliveiro]

On Tuesday, September 20, 2016 at 8:21:25 AM UTC+12, Martin Schöön wrote:
> But -- now I tested using emacs instead using C-c C-c to execute.
> Noting happens so I try to run the program from command line and
> find that now Python can't stand my å, ä and ö.

What version of Python? Python 3 accepts Unicode UTF-8 as a matter of course.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Christian Gollwitzer]

Am 19.09.16 um 22:21 schrieb Martin Schöön:

I am studying some of these tutorials:
https://pythonprogramming.net/matplotlib-intro-tutorial/

I am recreating the code and I use my native Swedish for comments,
labels and titles. Until now I have been doing so using Geany
and executing using F5. This works fine.

But -- now I tested using emacs instead using C-c C-c to execute.
Noting happens so I try to run the program from command line and
find that now Python can't stand my å, ä and ö.

I am puzzled: With Geany there is no problem but outside Geany
I am punished by Python for using Swedish.


you are not "punished for Swedish", you need to tell Python the encoding 
of the file it runs. See here:

https://www.python.org/dev/peps/pep-0263/

Assuming that you use UTF-8 (you should check with an emacs expert, I am 
not an emacs user), try putting the header


#!/usr/bin/python
# -*- coding: utf-8 -*-

on top of your files.

Christian

--
https://mail.python.org/mailman/listinfo/python-list