Re: [Zope] utf-8 problem in Zope when using Localizer

2007-08-25 Thread Sascha Welter
(Fri, Aug 24, 2007 at 10:26:42AM +0200) Patrick Ulmer wrote/schrieb/egrapse:
 Now I only must find the right place for setdefaultencoding() so
 it only affect my zope-instance.

Install a separate python that is used only by your zope instance. Use
setdefaultencoding only there.

Regards,

Sascha

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] utf-8 problem in Zope when using Localizer

2007-08-25 Thread Andreas Jung



--On 25. August 2007 15:48:04 +0200 Sascha Welter [EMAIL PROTECTED] 
wrote:



(Fri, Aug 24, 2007 at 10:26:42AM +0200) Patrick Ulmer
wrote/schrieb/egrapse:

Now I only must find the right place for setdefaultencoding() so
it only affect my zope-instance.


Install a separate python that is used only by your zope instance. Use
setdefaultencoding only there.



This is nonense. A custom sitecustomize.py script does the job without 
installing a complete new Python.


-aj

pgpen3iqRl5zw.pgp
Description: PGP signature
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


[Zope] utf-8 problem in Zope when using Localizer

2007-08-24 Thread Patrick Ulmer
Hi,

because nobody in der Localizer-Mailings list react and I thinks it could
be a common problem with utf-8 in zope, I ask my question here also.
I use Localizer 1.2.1 and I try out some things and all looks
great. After testing Localizer my next plan was to switch from encoding
ISO-8859-15 to utf-8 in Zope 2.10.3 but run into Problems when
inserting text through a MessageCatalog (object of Localizer).

Here is what I do in Zope:

1. I add default-zpublisher-encoding utf-8 to my zope.conf and restart
zope.
2. In ZMI I add the property management_page_charset and set it to utf-8
3. I add a new DTML Document and start editing. I check the encoding
in my Browser and it switched to unicode (utf-8). Looks good.
4. I insert html-code and some russian chars for testing

html
head
  meta http-equiv=content-type content=text/html; charset=utf-8
/head
body
   ...russian chars...
/body
/html

After Saving all looks great in ZMI and also in my webbrowser.

5. Now I add one line to use a MessageCatalog:

dtml-var MessageCatalog('Hallo Welt!')

After that, all of my utf-8-chars are broken in my webbrowser. I check
browser-encoding and it is still utf-8. In ZMI my DTML Document looks
good. This line must switch something in the output-encoding in the
internal page-render (I don't know, how to better explain it). Is this
an error in Localizer or in Zope or have I overlook a fault?

Thanks
Patrick


___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] utf-8 problem in Zope when using Localizer

2007-08-24 Thread Andreas Jung



--On 24. August 2007 08:32:17 +0200 Patrick Ulmer [EMAIL PROTECTED] 
wrote:



Hi,

because nobody in der Localizer-Mailings list react and I thinks it could
be a common problem with utf-8 in zope, I ask my question here also.
I use Localizer 1.2.1 and I try out some things and all looks
great. After testing Localizer my next plan was to switch from encoding
ISO-8859-15 to utf-8 in Zope 2.10.3 but run into Problems when
inserting text through a MessageCatalog (object of Localizer).

Here is what I do in Zope:

1. I add default-zpublisher-encoding utf-8 to my zope.conf and restart
zope.
2. In ZMI I add the property management_page_charset and set it to
utf-8 3. I add a new DTML Document and start editing. I check the
encoding in my Browser and it switched to unicode (utf-8). Looks good.
4. I insert html-code and some russian chars for testing

html
head
  meta http-equiv=content-type content=text/html; charset=utf-8
/head
body
   ...russian chars...
/body
/html

After Saving all looks great in ZMI and also in my webbrowser.

5. Now I add one line to use a MessageCatalog:

dtml-var MessageCatalog('Hallo Welt!')

After that, all of my utf-8-chars are broken in my webbrowser. I check
browser-encoding and it is still utf-8. In ZMI my DTML Document looks
good. This line must switch something in the output-encoding in the
internal page-render (I don't know, how to better explain it). Is this
an error in Localizer or in Zope or have I overlook a fault?



The http-equiv tag means not much to browser. Try setting the content-type
including the charset within the HTTP response 
(REQUEST.RESPONSE.setHeader(...)).


-aj

pgpyJCX7p3dYG.pgp
Description: PGP signature
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] utf-8 problem in Zope when using Localizer

2007-08-24 Thread Patrick Ulmer
Hi,

 4. I insert html-code and some russian chars for testing

 html
 head
   meta http-equiv=content-type content=text/html; charset=utf-8
 /head
 body
...russian chars...
 /body
 /html

 After Saving all looks great in ZMI and also in my webbrowser.

 5. Now I add one line to use a MessageCatalog:

 dtml-var MessageCatalog('Hallo Welt!')

 After that, all of my utf-8-chars are broken in my webbrowser. I check
 browser-encoding and it is still utf-8. In ZMI my DTML Document looks
 good. This line must switch something in the output-encoding in the
 internal page-render (I don't know, how to better explain it). Is this
 an error in Localizer or in Zope or have I overlook a fault?


 The http-equiv tag means not much to browser. Try setting the
 content-type
 including the charset within the HTTP response
 (REQUEST.RESPONSE.setHeader(...)).

I have checked it but it is the same problem. Browser is encoding in
utf-8 correct, but the result (html sourcecode from zope) is not utf-8
anymore after inserting dtml-var MessageCatalog('Hallo Welt!') and I
don't know why it changed. How can a dtml-var-tag changed the encoding
for the document?

Thanks,
Patrick
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] utf-8 problem in Zope when using Localizer

2007-08-24 Thread Jaroslav Lukesh


- Original Message - 
From: Patrick Ulmer [EMAIL PROTECTED]

The http-equiv tag means not much to browser. Try setting the
content-type
including the charset within the HTTP response
(REQUEST.RESPONSE.setHeader(...)).


I have checked it but it is the same problem. Browser is encoding in
utf-8 correct, but the result (html sourcecode from zope) is not utf-8
anymore after inserting dtml-var MessageCatalog('Hallo Welt!') and I
don't know why it changed. How can a dtml-var-tag changed the encoding
for the document?



What about

dtml-var _.str(MessageCatalog('Hallo Welt!'))

Use _.str() solves many potential problems

Regards JL.
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] utf-8 problem in Zope when using Localizer

2007-08-24 Thread Andreas Jung



--On 24. August 2007 09:36:30 +0200 Jaroslav Lukesh [EMAIL PROTECTED] 
wrote:




- Original Message - From: Patrick Ulmer [EMAIL PROTECTED]

The http-equiv tag means not much to browser. Try setting the
content-type
including the charset within the HTTP response
(REQUEST.RESPONSE.setHeader(...)).


I have checked it but it is the same problem. Browser is encoding in
utf-8 correct, but the result (html sourcecode from zope) is not utf-8
anymore after inserting dtml-var MessageCatalog('Hallo Welt!') and I
don't know why it changed. How can a dtml-var-tag changed the encoding
for the document?



What about

dtml-var _.str(MessageCatalog('Hallo Welt!'))

Use _.str() solves many potential problems



That's the mallet method.

-aj

pgpNZuRDJFJSt.pgp
Description: PGP signature
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] utf-8 problem in Zope when using Localizer

2007-08-24 Thread Patrick Ulmer

 I have checked it but it is the same problem. Browser is encoding in
 utf-8 correct, but the result (html sourcecode from zope) is not utf-8
 anymore after inserting dtml-var MessageCatalog('Hallo Welt!') and I
 don't know why it changed. How can a dtml-var-tag changed the encoding
 for the document?


 What about

 dtml-var _.str(MessageCatalog('Hallo Welt!'))

 Use _.str() solves many potential problems

I think it's the right way for a solution. Now my page is utf-8, but if
the string returned from MessageCatalog have utf-8 chars and not online
chars in ascii range I get this error-message:

*Error Type: UnicodeEncodeError*
*Error Value: 'ascii' codec can't encode character u'\xf4' in position
3: ordinal not in range(128)*

Regards
Patrick
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] utf-8 problem in Zope when using Localizer

2007-08-24 Thread Patrick Ulmer
Hi,

I think I solved the problem.

dtml-var MessageCatalog('Hallo Welt!').encode('utf-8')

works. I read some further and found out that I should use something like

import sys
sys.setdefaultencoding('utf-8')

I set it global in sitecustomize.py under /usr/lib/python2.4/site-packages and 
then 


dtml-var MessageCatalog('Hallo Welt!')

works. Now I only must find the right place for setdefaultencoding() so
it only affect my zope-instance.

Regars
Patrick


Patrick Ulmer schrieb:
 I think it's the right way for a solution. Now my page is utf-8, but if
 the string returned from MessageCatalog have utf-8 chars and not online
 chars in ascii range I get this error-message:

 Error Type: UnicodeEncodeError
 Error Value: 'ascii' codec can't encode character u'\xf4' in position
 3: ordinal not in range(128)
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


RE: [Zope] utf-8 problem in Zope when using Localizer

2007-08-24 Thread Doyon, Jean-Francois
If you can, I would strongly recommend using the Zope3/Five way of doing
translations and i18n in general!

See here:

http://worldcookery.com/files/fivei18n/

http://codespeak.net/z3/five/i18n.html

I've made my site fully internationalized, localized and utf-8 using
this method without resorting to any trickery.

setdefaultencoding() is apparently frowned upon ... Not sure why though.

Good luck!
J.F.

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of
Patrick Ulmer
Sent: August 24, 2007 04:27
To: zope@zope.org
Subject: Re: [Zope] utf-8 problem in Zope when using Localizer

Hi,

I think I solved the problem.

dtml-var MessageCatalog('Hallo Welt!').encode('utf-8')

works. I read some further and found out that I should use something
like

import sys
sys.setdefaultencoding('utf-8')

I set it global in sitecustomize.py under
/usr/lib/python2.4/site-packages and then 


dtml-var MessageCatalog('Hallo Welt!')

works. Now I only must find the right place for setdefaultencoding() so
it only affect my zope-instance.

Regars
Patrick


Patrick Ulmer schrieb:
 I think it's the right way for a solution. Now my page is utf-8, but 
 if the string returned from MessageCatalog have utf-8 chars and not 
 online chars in ascii range I get this error-message:

 Error Type: UnicodeEncodeError
 Error Value: 'ascii' codec can't encode character u'\xf4' in position
 3: ordinal not in range(128)
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] utf-8 problem in Zope when using Localizer

2007-08-24 Thread Dieter Maurer
Patrick Ulmer wrote at 2007-8-24 10:26 +0200:
I think I solved the problem.

dtml-var MessageCatalog('Hallo Welt!').encode('utf-8')

works. I read some further and found out that I should use something like

import sys
sys.setdefaultencoding('utf-8')

I set it global in sitecustomize.py under /usr/lib/python2.4/site-packages and 
then 


dtml-var MessageCatalog('Hallo Welt!')

works. Now I only must find the right place for setdefaultencoding() so
it only affect my zope-instance.

$INSTANCE_HOME/lib/python would be a good place for the
site-customize.py.



-- 
Dieter
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


RE: [Zope] utf-8 problem in Zope when using Localizer

2007-08-24 Thread Dieter Maurer
Doyon, Jean-Francois wrote at 2007-8-24 09:36 -0400:
 ...
setdefaultencoding() is apparently frowned upon ... Not sure why though.

Occasionally, a package writer expects that str converts to ASCII,
such that

 try: ascii = str(unicodeStr)
 except UnicodeError:
   # non ascii

can be used to check for ASCIIness.

Of course, this is true only when the defaultencoding is ASCII.

Fortunately, unicode(unicodeStr, 'ascii') can safely be
used instead.


The code above actually occured in Python's xmlrpclib.
When I reported the bug, people said we should eliminate
setdefaultencoding
But, then, they nevertheless fixed the bug...



-- 
Dieter
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )