Re: [Zope-dev] UnicodeDecodeError problems with Zope 2.12.0b4
Chris Withers wrote: > Annoyingly, this required even more evil, since sitecustomize.py is no > longer found anywhere sensible in Python 2.5, correction: it never was, but prior to Zope 2.12, the INSTANCE_HOME ended up on PYTHONPATH and so sitecustomize.py could be put there. Using the buildout approach for Zope 2.12, it not longer is on PYTHONPATH, so it doesn't get found... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] UnicodeDecodeError problems with Zope 2.12.0b4
Am 30.08.2009, 19:16 Uhr, schrieb Sidnei da Silva : > You mean the latest binary doesn't have the fix? Or you haven't seen > that a binary was released recently? > http://www.zope.org/Products/Zope/2.11.4/ ah, I hadn't seen that one - my last Windows install was in July. Will give a try this week and am very much looking forward to removing my hackish override. Charlie -- Charlie Clark Managing Director Clark Consulting & Research Ltd. German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 GSM: +49-178-782-6226 Company No. 06770088 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] UnicodeDecodeError problems with Zope 2.12.0b4
Hi Charlie, > I only know about it because I keep on it hitting it in Windows installs > as there isn't a binary of 2.11 with the fix. You mean the latest binary doesn't have the fix? Or you haven't seen that a binary was released recently? http://www.zope.org/Products/Zope/2.11.4/ -- Sidnei ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] UnicodeDecodeError problems with Zope 2.12.0b4
On 30.08.09 13:58, Charlie Clark wrote: > Am 30.08.2009, 13:26 Uhr, schrieb Andreas Jung : > > >> Possibly fixed in new versions of zope.publisher. I think there was some >> open issue >> with backporting some fix - but my brain can't remember :-> >> > See > https://bugs.launchpad.net/zope2/+bug/160968 > and > https://bugs.launchpad.net/bugs/143873 No, I am referring to a controversial discussion with Dieter - however I really can not find it anymore. Anyway...my conclusion from this discussion was that a custom unicode conflict resolver should be used for resolving such custom problems (unless with a real bug in the processing of the http headers). A clean solution would be to introduce the notion of a default charset for a site manager (as the 'default_charset' property of CMF sites or Plone sites). A unicode conflict resolver could get hold of the (local) site manager and check for a (registered) local utiltiy return the charset through some API...something like that (instead of guessing/sniffing in the HTTP client headers). Andreas <>___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] UnicodeDecodeError problems with Zope 2.12.0b4
Am 30.08.2009, 13:26 Uhr, schrieb Andreas Jung : > Possibly fixed in new versions of zope.publisher. I think there was some > open issue > with backporting some fix - but my brain can't remember :-> See https://bugs.launchpad.net/zope2/+bug/160968 and https://bugs.launchpad.net/bugs/143873 So, it should be in the current code-base and shouldn't be happening at all in Zope 2.12. So it could be something else. I only know about it because I keep on it hitting it in Windows installs as there isn't a binary of 2.11 with the fix. Charlie -- Charlie Clark Managing Director Clark Consulting & Research Ltd. German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 GSM: +49-178-782-6226 Company No. 06770088 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] UnicodeDecodeError problems with Zope 2.12.0b4
On 30.08.09 13:01, Charlie Clark wrote: > Am 30.08.2009, 12:51 Uhr, schrieb Andreas Jung : > > >>> - Why am I getting this decoding error only with these browsers? Firefox >>> and Chrome work just fine... Is the publisher doing something special >>> with a header these browsers send that IE and Safari are not? If so, how >>> do I make it do the "right thing" for IE and Safari? >>> > I thought this had been fixed? I'm pretty sure it's related to > 'HTTP_ACCEPT_CHARSET' missing in the headers from IE and Safari and the > charset detector incorrectly extracting it from the REQUEST which is > always True for Zope 2. > > zope.publisher.http.getPreferredCharsets() is where this magic happens. > Possibly fixed in new versions of zope.publisher. I think there was some open issue with backporting some fix - but my brain can't remember :-> Andreas <>___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] UnicodeDecodeError problems with Zope 2.12.0b4
Am 30.08.2009, 12:51 Uhr, schrieb Andreas Jung : >> - Why am I getting this decoding error only with these browsers? Firefox >> and Chrome work just fine... Is the publisher doing something special >> with a header these browsers send that IE and Safari are not? If so, how >> do I make it do the "right thing" for IE and Safari? I thought this had been fixed? I'm pretty sure it's related to 'HTTP_ACCEPT_CHARSET' missing in the headers from IE and Safari and the charset detector incorrectly extracting it from the REQUEST which is always True for Zope 2. zope.publisher.http.getPreferredCharsets() is where this magic happens. Charlie -- Charlie Clark Managing Director Clark Consulting & Research Ltd. German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 GSM: +49-178-782-6226 Company No. 06770088 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] UnicodeDecodeError problems with Zope 2.12.0b4
There is a related bug report on LP (with a controversial discussion) about the issue (can't find the URL right now). Andreas On 21.08.09 21:40, Chris Withers wrote: > Hi All, > > I'm suffering with two different types of UnicodeDecodeError with a Zope > 2.12 project... > > The first type of problem occurs when a form is submitted and the form > fields contain encoded strings. Somewhere down the line I get: > >Module zope.i18n, line 166, in interpolate >Module zope.i18n, line 161, in replace > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5: > ordinal not in range(128) > > So, my question is how do I get strings submitted to the publisher > (ideally) end up as unicode objects not encoded strings? Failing that, > how do I find out what encoding the browser has provided? > > The second type of problem only occurs in IE and Safari, where viewing a > page that contains some non-ascii-able data: > >Module ZPublisher.Publish, line 127, in publish >Module ZPublisher.mapply, line 77, in mapply >Module ZPublisher.Publish, line 47, in call_object >Module Products.Five.browser.metaconfigure, line 427, in __call__ >Module Products.Five.browser.pagetemplatefile, line 126, in __call__ >Module Products.Five.browser.pagetemplatefile, line 60, in __call__ >Module zope.pagetemplate.pagetemplate, line 116, in pt_render > - Warning: Macro expansion failed > - Warning: : macro nesting > limit (100) exceeded by u'root/standard_template.pt/macros/nav' >Module StringIO, line 270, in getvalue > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 10: > ordinal not in range(128) > > Two things are weird here: > > - Why am I getting a macro nesting limit? This seems to be a recurring > problem (excuse the pun) with Zope 2.12 page templates where if there's > an error in a standard template, I get this error when a template that > uses it is rendered. > > - Why am I getting this decoding error only with these browsers? Firefox > and Chrome work just fine... Is the publisher doing something special > with a header these browsers send that IE and Safari are not? If so, how > do I make it do the "right thing" for IE and Safari? > > cheers, > > Chris > > -- ZOPYX Ltd. & Co KG \ ZOPYX & Friends Charlottenstr. 37/1 \ The experts for your Python, Zope and D-72070 Tübingen \ Plone projects www.zopyx.com, i...@zopyx.com \ www.zopyx.de/friends, frie...@zopyx.de E-Publishing, Python, Zope & Plone development, Consulting <>___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] UnicodeDecodeError problems with Zope 2.12.0b4
Takayuki Shimizukawa wrote: > 2009/8/22 Chris Withers : >> The first type of problem occurs when a form is submitted and the form >> fields contain encoded strings. Somewhere down the line I get: > > The first type, I don't know. > >> The second type of problem only occurs in IE and Safari, where viewing a >> page that contains some non-ascii-able data: > > The second type, (maybe) similer probrem reported to launchpad. > https://bugs.launchpad.net/zope2/+bug/160968 > >> - Why am I getting this decoding error only with these browsers? Firefox >> and Chrome work just fine... Is the publisher doing something special >> with a header these browsers send that IE and Safari are not? If so, how >> do I make it do the "right thing" for IE and Safari? > > Are concerned that HTTP_ACCEPT_CHARSET http header will probably > not be sent from client browser (IE6,7, Safari). > When HTTP_ACCEPT_CHARSET not in HTTP request headers, > zope.publisher use UTF-8 encoding for publishing. Well, I just solved the problem with a large setdefaultencoding hammer :-) Annoyingly, this required even more evil, since sitecustomize.py is no longer found anywhere sensible in Python 2.5, so I ended up with a buildout that looks like the following: [buildout] parts = instance extends = http://svn.zope.org/*checkout*/Zope/tags/2.12.0b4/versions.cfg?rev=102515 versions = versions [versions] zope2 = 2.12.0b4 [instance] recipe = zc.recipe.egg eggs = ${buildout:eggs} interpreter = py entry-points= runzope=Zope2.Startup.run:run zopectl=Zope2.Startup.zopectl:main scripts = runzope zopectl initialization = import sys reload(sys) sys.setdefaultencoding('utf-8') sys.argv[1:1] = ['-C','${buildout:directory}/etc/instance.conf'] Evil huh? ;-) Hope this helps someone... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] UnicodeDecodeError problems with Zope 2.12.0b4
Hi Chris. 2009/8/22 Chris Withers : > The first type of problem occurs when a form is submitted and the form > fields contain encoded strings. Somewhere down the line I get: The first type, I don't know. > The second type of problem only occurs in IE and Safari, where viewing a > page that contains some non-ascii-able data: The second type, (maybe) similer probrem reported to launchpad. https://bugs.launchpad.net/zope2/+bug/160968 > - Why am I getting this decoding error only with these browsers? Firefox > and Chrome work just fine... Is the publisher doing something special > with a header these browsers send that IE and Safari are not? If so, how > do I make it do the "right thing" for IE and Safari? Are concerned that HTTP_ACCEPT_CHARSET http header will probably not be sent from client browser (IE6,7, Safari). When HTTP_ACCEPT_CHARSET not in HTTP request headers, zope.publisher use UTF-8 encoding for publishing. regards. -- Takayuki Shimizukawa ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )