RE: [Zope-dev] [ZOPE 2.6 B1] Unicode/locale problems withOFS/dtml/properties.dtml
> I can second this. With CVS-Zope (did the last cvs up this moment) > I'm getting a very curios thing: > Displaying .../index_html is ok. > But > return context.index_html(context,request) > creates broken characters instead is isolatin1 Umlaute. > In my case (Konqueror on Linux) it seems that the > text/html;charset=UTF-8 breaks the page because the byte values are > correct for the "Umlaute". This is further confirmed by the fact that > forcing Konq to display iso8859-1 fixes the display. Hmm, you may check out http://collector.zope.org/Zope/517 but it could be the same difficulties as we experienced earlier. The problem here was that Zope thought it was returning UTF-8, while it was really returning ISO-8859-1. This was due to the statement not having the desired effect. (notice the space) seemed to work brilliantly. > So how are these Unicode changes supposed to work? Are non-ascii > characters forbidden now? And how do I get UTF-8 text into Zope? There are converters inside ZOPE. UTF-8 is simply a transport format, although it may be used for storage to save space. There is lots of software that supports UTF-8 today. This is the future. > While I'm quite sure that this will help Zope in the Asiatic region, it > seems quite inconvienent for isolatin1 world :( This will be a win in Europe as well, especially for multilingual sites. IIRC there are 15 variants of ISO-8859-1. I18N is *very* important, and Unicode is an essential ingredient. Arnar Lundesgaard ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] [ZOPE 2.6 B1] Unicode/locale problems with OFS/dtml/properties.dtml
> Thanks for taking the time to help debug this. Thanks for fixing it so quickly! > This line, intended to force the dtml to be rendered as a unicode > object, is not doing its job. If the dtml is not a unicode object > then ZPublisher's encoding mechanism is not engaged. Has there been > some recent dtml optimisation to ignore empty strings perhaps? > This patch below works for me. Please let me know if this works for > you and I will apply it to 2.6. (and various sources of documentation > that all recommend including this line.) Jepp, that seems to work. I'm at home though, so I haven't been able to try with the original browser configuration. 'wget' gives me UTF-8 though, so it should work nicely. :-) Arnar Lundesgaard ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] [ZOPE 2.6 B1] Unicode/locale problems with OFS/dtml/properties.dtml
> what do you mean by "inserted into the form"? Put there by the dtml-method in a dtml-var statement. > That the response contains a single byte where you properties > contain character whose unicode value is greater than 127 ? Yes > how have you checked this? if so, thats a bug. In a Hex editor... The character 'æ' for instance is inserted as '0xE6' in the returned HTML. > > Secondly, the type attribute of all inputfields contain an extra > > ':utf8:' that we assumed is a server directive to interpret the > > contents as UTF-8. This apperently what crashes when storing the > > second time. > > This is a directive to tell zope when you submit the form > that your browser will have encoded the form response using > utf-8. Browsers stupidly dont put this information anywhere > more suitable. Yeah, we figured this would be similar to ':method'. > What browser are you using? is it correctly using utf8 for > this page? (for example, Mozilla has a View/Encoding menu > that can override server-supplied encoding information) We have so far tried Opera 6, IE 6 and Mozilla 1.1. > > lib/python/OFS/dtml/properties.dtml contains the following > > that seems to us to be debug code: > > > > > > and several ':utf8:' directives. > > > > When we removed those, it worked fine. Was this dtml-method > > merged in mistakenly > > Those are supposed to be there. The first inserts the > text/html;charset=utf-8 header into the management page. > The second ensures that the dtml which computes this page > content returns a unicode object. > > If the dtml returns a unicode object, then ZPublisher looks > at the charset header to determine how to encode it. This is DTML/ZOPE internal, right? I though the first two letters to signify endianness 'FFFE' or 'FEFF', tells you that this is UTF-8. The u'' string is not on the firs line. > please put ib/python/OFS/dtml/properties.dtml back the way is > was originally, then send me > > 1. which browser you are using I am using Opera 6.02, but we are seeing it on all browsers. > 2. a export file containing one object that demonstrates the problem. It is on every property page in the whole site, but see the attached .zexp of /standard_template.pt > 3. a copy of the page obtained using wget or similar. > (please dont use your browsers 'save' feature because > that sometimes performs transcoding) I used wget... -- Arnar Lundesgaard standard_template.pt.zexp Description: standard_template.pt.zexp manage_propertiesForm.1 Description: manage_propertiesForm.1
[Zope-dev] [ZOPE 2.6 B1] Unicode/locale problems with OFS/dtml/properties.dtml
We are trying out 2.6 b1. So far it seems very snappy (!!), and the
new error-log object is exceptionally usefull! Kudos to Zope Corp and
the community. :-)
While trying the beta we have found a few problems in the handling of
propertysheets. We can successfully store Norwegian characters
('æ','ø', and 'å') in property fields, but when we reload the page, we
get results that display wrong.
When studying the generated sources, we see that the data from ZOPE
inserted into the form is ISO-8859-1.
OTOH this field is inserted in the html-header:
Therefore the browser tries to display utf-8, which of course becomes
garbage when you get an ISO-8859-1 character with a larger value than
127.
All other pages (than the "properies" tab) in the have
Secondly, the type attribute of all inputfields contain an extra
':utf8:' that we assumed is a server directive to interpret the
contents as UTF-8. This apperently what crashes when storing the
second time.
lib/python/OFS/dtml/properties.dtml contains the following that seems
to us to be debug code:
and several ':utf8:' directives.
When we removed those, it worked fine. Was this dtml-method merged in
mistakenly, or are there problems in our setup.
Arnar Lundesgaard
---
phone: (+47) 982 38 036
mailto:arnar.lundesgaard(at)creuna.no
Creuna as
Bryggegata 3
NO-0250 Oslo
phone office: (+47) 23 23 88 00
fax: (+47) 23 23 88 50
http://www.creuna.no/
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [Zope-CMF] Performance problem in ZOPE 2.5.1, 'getOwner' in/lib/python/AccessControl/Owned.py
> What kind of user folder are you using? We are using the extensible User Folder. ZODB Auth BTree Source Basic Membership Source Authentication Type:cookie-based Credential Cache Timeout:600 (We have also tried 0 without much success) Negative Credential Cache:0 memberdata is the standard CMF with a exUserfolder wrapper for it. I have just tested the performance with a normal userfolder with only two users, and the response was instantaneous as it should be. This seems to be the reason for our troubles. :-/ We have about 100k users (many inactive) which we cannot remove, so we would very much appreciate experiences with this number of users and similar setups. Arnar Lundesgaard ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Performance problem in ZOPE 2.5.1, 'getOwner' in /lib/python/AccessControl/Owned.py
Hi, as I have written on this list before, we have had serious performance problems on one of our CMF based sites. For a long time we believed that this was mostly due to load, and differences in our application. Unfortunately the more we searched for answers the less we understood. Recently we have converted to a ZEO setup, and that has made it much easier to locate hotspots in the application. One of things we have seen, is in our skinned 'folder_contents', which includes the name of the objects owner. This is fetched by using the 'Creator' method from the DublinCore mixin. This again calls getOwner of 'AccessControl/Owned.py'. On the first site we have set up (using the same application, whose performance currently is quite good) and folder_contents is virtualy instantaneous. Therefore we tried using the default skin. The difference was astonishing, and we are seeing a difference of 20-30 seconds. We started removing functionality, and we quickly found the culprit. In the CVS log for Owned.py it says (around 1.15.10.2): Remove cache from getOwner, since it would have cached both unwrapped and wrapped objects (caching wrapped objects being a bad thing) This change seems to be the difference between our two sites, and it will also affect other parts of our application. For instance when we reindex objects on change, since Creator is both an index and a metadata field. Can we simply change this file? What are the consequences of having or not having the cache? Arnar Lundesgaard - phone: (+47) 982 38 036 mailto:arnar.lundesgaard(at)creuna.no Creuna as Bryggegata 3 NO-0250 Oslo phone office: (+47) 23 23 88 00 fax: (+47) 23 23 88 50 http://www.creuna.no/ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [Zope-CMF] Fieldindex searches very slow when anonymous.
THANK YOU This worked extremely well, and the site is almost faster than the squid cache now. :-) http://www.forskning.no is a major ZOPE site (for us), that will be released tomorrow noon local time. This is the first site that uses our soon to be opensourced TopicMap engine that builds on CMF. The engine lets CMF handle all Topic Map data items, giving us access to CMF functionality like versioning and workflow. It should be opensourced when we have time to clean it up a bit and document it. - Arnar Lundesgaard > -Original Message- > From: Tres Seaver [mailto:[EMAIL PROTECTED]] > Sent: 9. april 2002 05:43 > To: Arnar Lundesgaard > Cc: [EMAIL PROTECTED] > Subject: Re: [Zope-CMF] Fieldindex searches very slow when anonymous. > > What is killing your query is the test for "effective range" > of content. If > your site does not use the concept (i.e., all your content > has 'None' for > both its effective and expires dates), you can apply the > following patch to > remove the check:: > > -- patch starts here - > --- CMFCore/CatalogTool.py4 Jan 2002 19:50:04 - 1.22 > +++ CMFCore/CatalogTool.py9 Apr 2002 03:42:14 - > @@ -186,7 +186,7 @@ > user = _getAuthenticatedUser(self) > kw[ 'allowedRolesAndUsers' ] = > self._listAllowedRolesAndUsers( user ) > > -if not _checkPermission( > +if 0 and not _checkPermission( > CMFCorePermissions.AccessInactivePortalContent, self ): > base = aq_base( self ) > now = DateTime() > -- patch ends here --- > > Tres. > -- > === > Tres Seaver[EMAIL PROTECTED] > Zope Corporation "Zope Dealers" http://www.zope.com > > ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
