[Zope-dev] Re: Do we need a Packages directory in the new repository
Jim Fulton wrote: Historically, we've had Packages, Products, Packages3 and Products3 directories in the CBS repository. I wonder of we need these going forward. Perhaps we should just have top-level projct directories in the new subversion repository. +1 Philipp ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Unicode in Zope 2 (ZMI, Archetypes, Plone, Formulator)
While we're all waiting for Zope 3 and Plone 3, I'd like to know what the standard practice way of using Unicode with Zope 2. In particular, we'd like to store all text as Unicode in the ZODB, and have Zope do the encoding/decoding as automatically and transparently as possible. We've been using Zope 2's ZPublisher to do this encoding/decoding for over 2 years, and it's working fine. We just have to ensure that we set the appropriate encoding in a HTTP Content-type header, and that we add :utext/ustring:ENCODING to HTML form field names. Regardless of what you may have heard, THIS WORKS FINE! We also store Unicode, not UTF-8 (or other encodings), strings in the ZODB. The problem we're running into are with other components, basically making our Unicode-with-Zope experience, shall we say, less than ecstatic (To put it this way, I seem to lose hair much faster when dealing with Unicode problems :) I'm wondering why components/products aren't all relying on the ZPublisher for Unicode encoding/decoding? Is there another standard way? Here is a summary of what we've found: ZMI * gets charset from manage_page_charset encoding * relies on ZPublisher for encoding (but doesn't do decoding, see below) * in PropertyManager you can add ustrings, but since it doesn't add :ENCODING to the field names, you get a Unicode error when trying to save since it tries to decode the text assuming ASCII (big problem) * DTML Methods/Documents: doesn't support Unicode (annoying) * can't use Unicode id's (not a big problem) Archetypes: * gets charset from portal_url.getCharset() or portal_properties.site_properties.default_charset * doesn't rely on ZPublisher, does its own encoding/decoding * returns encoded strings, not Unicode strings, to Zope apps, leading to problems such as: - SearcableText() encodes, and as such can't be used with Unicode-aware ZCatalogs - transform() encodes (and because of that SearchableText() sometimes decodes/encodes 2 times instead of 0 times) - get()ing field values will encode them, so if you want Unicode, you have to decode yourself (adding both unnecessary overhead for data access, and unnecessary dependency on the global variable for the charset) Plone: * no special Unicode support for HTML forms; relies on Archetypes Formulator: * gets charset from manage_page_charset (same as ZMI), but can be overridden * stores field values as encoded text (not Unicode), but lets you specify which encoding to use (confusingly calls this unicode mode) * messages are stored as UTF-8 (hardcoded) I suggest this way of dealing with Unicode right now in Zope 2: (1) Let ZPublisher do the encoding/decoding of form input and HTML output: a. Always set a character encoding in a HTTP Content-type request b. Always append :ustring/utext/ulines/utokens:ENCODING to field names of fields that support Unicode (we may need some library code to make this easier) (2) Store Unicode strings directly in the ZODB. The ZODB is perfectly capable of storing strings in Python's internal Unicode format; no need to encode the text to UTF-8 or some other encoding. (3) Encode/decode yourself when reading from/ writing to other external data sources such as files and other databases. Do it just before you write, or just after you read, so that as much code as possible can be encoding-agnostic. Keep the encoding/decoding as close to the source data as possible. The best way to do it is (in most cases) to specify the encoding on the IO stream, and let Python do the encoding/decoding for you transparently. If possible, get the encoding from the external data source (e.g., the file) instead of relying on a magical global variable. If you have to rely on a global variable, let it be manage_page_charset. (4) [This is really just advice...] Resist patching your code to work with components that doesn't deal with Unicode. Others are likely having the same problem, so to avoid ending up with lots of ugly patches (that are the source of mysterious Unicode problems), fix the problem at its source: the other component. It's really not that difficult to fix (if we agree on how it should be fixed ;) None of the above components handles Unicode in this way, but it seems to be how the Unicode support in Zope 2 was meant to be used. Let me know if there is another better way, but please do let me know... I think we need to resolve this once and for all or I know some people that'll just go mad (or bald, or both) :) I'll be willing to contribute patches, but since this applies to so many products, it would be good to get some consensus first. At the very least, can we create a Standard Unicode Practices page? Bye, -- Bjorn Stabell mailto:[EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists -
[Zope-dev] Re: [Zope3-dev] Decouple Interface and zope.interface (Martijn was right)
Jim Fulton wrote: [decouple interface implementation] *falls into a dead faint* *wakes up and starts bouncing around* *Loud cheering!* Awesome, thanks, Jim! A good start of my working week, too. *cough* *regains composure* *ahum* +1 Regards, Martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Unicode in Zope 2 (ZMI, Archetypes, Plone, Formulator)
On Monday 26 April 2004 09:27, Bjorn Stabell wrote: While we're all waiting for Zope 3 and Plone 3, I'd like to know what the standard practice way of using Unicode with Zope 2. In particular, we'd like to store all text as Unicode in the ZODB, and have Zope do the encoding/decoding as automatically and transparently as possible. I added most of the Unicode support in core Zope in 2.6. Transparency was my goal too. For the applications I was working on at the time this goal was achieved. Naturally I stopped without converting all the components that I dont use, and sadly noone has picked up this work in a consistent manner. * in PropertyManager you can add ustrings, but since it doesn't add :ENCODING to the field names, you get a Unicode error when trying to save since it tries to decode the text assuming ASCII (big problem) It works for me in 2.6. Note that it only adds the :ENCODING after adding your first unicode property. This is ugly, but necessary for legacy applications using 8-bit string properties to hold pre-encoded text. I suggest this way of dealing with Unicode right now in Zope 2: good advice. None of the above components handles Unicode in this way, but it seems to be how the Unicode support in Zope 2 was meant to be used. That is correct. -- Toby Dickenson ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Archetypes-devel] Unicode in Zope 2 (ZMI, Archetypes, Plone, Formulator)
Hi Bjorn, I always believed that unicode and utf-8 were same encoding, but reading you let me think i was wrong. Can you tell me what the difference is between unicode and utf-8 ? Bjorn Stabell wrote: While we're all waiting for Zope 3 and Plone 3, I'd like to know what the standard practice way of using Unicode with Zope 2. In particular, we'd like to store all text as Unicode in the ZODB, and have Zope do the encoding/decoding as automatically and transparently as possible. We've been using Zope 2's ZPublisher to do this encoding/decoding for over 2 years, and it's working fine. We just have to ensure that we set the appropriate encoding in a HTTP Content-type header, and that we add :utext/ustring:ENCODING to HTML form field names. Regardless of what you may have heard, THIS WORKS FINE! We also store Unicode, not UTF-8 (or other encodings), strings in the ZODB. The problem we're running into are with other components, basically making our Unicode-with-Zope experience, shall we say, less than ecstatic (To put it this way, I seem to lose hair much faster when dealing with Unicode problems :) I'm wondering why components/products aren't all relying on the ZPublisher for Unicode encoding/decoding? Is there another standard way? Here is a summary of what we've found: ZMI * gets charset from manage_page_charset encoding * relies on ZPublisher for encoding (but doesn't do decoding, see below) * in PropertyManager you can add ustrings, but since it doesn't add :ENCODING to the field names, you get a Unicode error when trying to save since it tries to decode the text assuming ASCII (big problem) * DTML Methods/Documents: doesn't support Unicode (annoying) * can't use Unicode id's (not a big problem) Archetypes: * gets charset from portal_url.getCharset() or portal_properties.site_properties.default_charset * doesn't rely on ZPublisher, does its own encoding/decoding * returns encoded strings, not Unicode strings, to Zope apps, leading to problems such as: - SearcableText() encodes, and as such can't be used with Unicode-aware ZCatalogs - transform() encodes (and because of that SearchableText() sometimes decodes/encodes 2 times instead of 0 times) - get()ing field values will encode them, so if you want Unicode, you have to decode yourself (adding both unnecessary overhead for data access, and unnecessary dependency on the global variable for the charset) Plone: * no special Unicode support for HTML forms; relies on Archetypes Formulator: * gets charset from manage_page_charset (same as ZMI), but can be overridden * stores field values as encoded text (not Unicode), but lets you specify which encoding to use (confusingly calls this unicode mode) * messages are stored as UTF-8 (hardcoded) I suggest this way of dealing with Unicode right now in Zope 2: (1) Let ZPublisher do the encoding/decoding of form input and HTML output: a. Always set a character encoding in a HTTP Content-type request b. Always append :ustring/utext/ulines/utokens:ENCODING to field names of fields that support Unicode (we may need some library code to make this easier) (2) Store Unicode strings directly in the ZODB. The ZODB is perfectly capable of storing strings in Python's internal Unicode format; no need to encode the text to UTF-8 or some other encoding. (3) Encode/decode yourself when reading from/ writing to other external data sources such as files and other databases. Do it just before you write, or just after you read, so that as much code as possible can be encoding-agnostic. Keep the encoding/decoding as close to the source data as possible. The best way to do it is (in most cases) to specify the encoding on the IO stream, and let Python do the encoding/decoding for you transparently. If possible, get the encoding from the external data source (e.g., the file) instead of relying on a magical global variable. If you have to rely on a global variable, let it be manage_page_charset. (4) [This is really just advice...] Resist patching your code to work with components that doesn't deal with Unicode. Others are likely having the same problem, so to avoid ending up with lots of ugly patches (that are the source of mysterious Unicode problems), fix the problem at its source: the other component. It's really not that difficult to fix (if we agree on how it should be fixed ;) None of the above components handles Unicode in this way, but it seems to be how the Unicode support in Zope 2 was meant to be used. Let me know if there is another better way, but please do let me know... I think we need to resolve this once and for all or I know some people that'll just go mad (or bald, or both) :) I'll be willing to contribute patches, but since this applies to so many products, it would be good to get some consensus first. At the very least, can we create a Standard Unicode Practices page? Bye, -- David Convent ___ Zope-Dev maillist - [EMAIL PROTECTED]
[Zope-dev] RE: [Archetypes-devel] Unicode in Zope 2 (ZMI, Archetypes, Plone, Formulator)
--On Montag, 26. April 2004 10:53 Uhr +0200 David Convent [EMAIL PROTECTED] wrote: I always believed that unicode and utf-8 were same encoding, but reading you let me think i was wrong. Can you tell me what the difference is between unicode and utf-8 ? Andreas Jung wrote: Unicode is common database for almost all characters. UTF-8 is an *encoding* that allows you to represent any element of this character database as set for 1,2,3 or 4 bytes. There are also other encoding e.g. like UTF16 that encode an element in a different wayso we are talking about completely different things. Yes, the difference is that Python has a whole different understanding of Unicode strings (type(u)) than it has of text of some character encoding (e.g., UTF-8, GB18030, ISO8859-1, ASCII, stored as type()). Python will of course represent these unicode strings internally some way (maybe as a 16-bit integer?), but we don't need to know what that is like. All we need to know is that this is a string that can contain any character on the planet, and that we can reasonably expect normal text operations to work on it. UTF-8 is, similar to ISO-8869-1 (latin1), just a character encoding. It (and UTF16, UCS2, UCS4) is only special in that it was issued by the Unicode consortium and can encode any Unicode character. Wherease ISO-8859-1 (for example), being only 8 bits, can only encode characters used in Western Europe. GB18030, to take another extreme, is a 32-bit encoding endorsed by the Chinese govnerment; being 32-bit, it can encode/represent a lot of Unicode characters, even many non-Chinese ones; it is big enough to potentially encode any Unicode character, if the Chinese government defined how each Unicode code point was mapped into GB18030. In this case, it would be similar in function to UCS4 (I think it is). Internally, we want to work with Unicode strings (where str[4] is the 4th character) instead of UTF-8 encoded text strings (where str[4], being the 4th byte, has little semantic meaning). Bye, -- Bjorn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Do we need a Packages directory in the new repository
Jim Fulton wrote: Historically, we've had Packages, Products, Packages3 and Products3 directories in the CBS repository. I wonder of we need these going forward. Perhaps we should just have top-level projct directories in the new subversion repository. I think having a distinction between Zope 2 and Zope 3 extensions would be a good idea. While for some extensions eventually that can become blurred, we'll cross that bridge when it happens. In the initial conversion, I plan to convert projects pretty selectively. Initially, we'll convert only parts needed for Zope 2, Zope 3 and ZODB. Agreed. Regards, Martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] Decouple Interface and zope.interface (Martijn was right)
On Sunday 25 April 2004 06:57 am, Jim Fulton wrote: If we drop the requirement to make the interfaces interoperate (e.g. extend one another, or be used in the same declarations), then, with a small change that Martijn has suggested, we can use zope.interface in Zope 2 without any other changes. That would be terrific! (Makes my life easier, anyway). Cheers, Terry -- Terry Hancock ( hancock at anansispaceworks.com ) Anansi Spaceworks http://www.anansispaceworks.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Source Code
Does anyone have the source code to the Addressit sample program found in the Zope Bible? If not all of it, I'm mainly looking for the Addressit.py file. Thanks! __ Do you Yahoo!? Yahoo! Photos: High-quality 4x6 digital prints for 25ยข http://photos.yahoo.com/ph/print_splash ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope3-dev] Re: [Zope-dev] Re: The bleak Future of Zope?
[Max M, tests a zip file of .pyd files in Zope3, on Windows] I finally got around to testing this, and it works *exactly* as I hoped. I downloaded Z3 from CVS, and Tim's zip file. I unpacked the zip file into the Z3 directory, and it started the first time. Yippee! That's what I expected, but it's always a pleasant surprise when software works 0.8 wink. I don't know about others, but I don't need anything more than this. If either Chris or Tim would put that structure up at a saner location, I will write a How-to about it. Basically I just need a web adress that stays current to use in the How-to, so that I could point to somewhere like: http://zope.org/Members/tim_one/z3builds Where they could get the latest zip file. http://zope.org/Members/tim_one/Zope3-pyd.zip is in place now. I'll put up a similarly named one for Zope2 this afternoon (EDT). They'll be replaced from time to time (when Zope 2/3's C code changes), but will retain theses names. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Policy for Collector-Issues 545 and 1217?
Maik Jablonski wrote: Sounds cool... I'm not sure if it's easy as you describe, but I hope so...:) Basically you just grep for URL1 in all dtml files, anywhere you see dtml-URL1; or dtml-var URL1 html_quote and change it to dtml-var REQUEST.URL1 html_quote IIRC REQUEST is safe to use. The culprit in this case is likely in manage_tabs.dtml I've never been able to reproduce the content_type bug. -- Jamie Heilman http://audible.transient.net/~jamie/ Most people wouldn't know music if it came up and bit them on the ass. -Frank Zappa ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: The bleak Future of Zope?
Martijn Faassen wrote: Dieter Maurer wrote: Martijn Faassen wrote at 2004-4-24 22:49 +0200: ... In practice right now the picture is 'Use all of the CMF or none of it'. No, not really... We use SkinsTool, ActionsTool and DCWorkflow a lot, MembershipTool sometimes and most other tools not at all. Okay, point taken. :) How much do the tools listed interdepend on each other? See the attached file. Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com [/home/tseaver/projects/CMF/CMF-1.4-head/CMFCore] $ grep getToolByName Action*.py ActionInformation.py:from utils import getToolByName ActionInformation.py:membership = getToolByName(tool, 'portal_membership') # uses 'portal_membership' to compute whether the user is anonymous, and # to look up the user's ID. # uses 'Expression' class heavily; Expression also depends on # 'portal_membership'. [/home/tseaver/projects/CMF/CMF-1.4-head/CMFCore] $ grep getToolByName Skins*.py SkinsTool.py:from utils import UniqueObject, getToolByName, _dtmldir SkinsTool.py:pm = getToolByName(self, 'portal_membership') SkinsTool.py:pu = getToolByName(self, 'portal_url') # Uses 'portal_membership' to look up member skin prefs # Uses 'portal_url' to compute the path of the site object, in order to set #the path on the skin cookie. [/home/tseaver/projects/CMF/CMF-1.4-head/CMFCore] $ grep getToolByName FS*.py FSObject.py:from utils import expandpath, getToolByName FSObject.py:portal_skins = getToolByName(self,'portal_skins') # Uses 'portal_skins' to do customization. [/home/tseaver/projects/CMF/CMF-1.4-head/CMFCore] $ grep getToolByName Member*.py RegistrationTool.py MemberDataTool.py:from utils import UniqueObject, getToolByName, _dtmldir MemberDataTool.py:membertool = getToolByName(self, 'portal_membership') MemberDataTool.py:mtool = getToolByName(self, 'portal_membership') MemberDataTool.py:membertool= getToolByName(self, 'portal_membership') MemberDataTool.py:membership = getToolByName(self, 'portal_membership') MemberDataTool.py:registration = getToolByName(self, 'portal_registration', None) MembershipTool.py:from utils import getToolByName, _dtmldir MembershipTool.py:registration = getToolByName(self, 'portal_registration', None) MembershipTool.py:md = getToolByName(parent, 'portal_memberdata') MembershipTool.py:md = getToolByName( self, 'portal_memberdata' ) from utils import getToolByName membership = getToolByName(self, 'portal_membership') membership = getToolByName(self, 'portal_membership') # Membership, memberdata, and registration are self-contained. [/home/tseaver/projects/CMF/CMF-1.4-head/CMFCore] $ grep getToolByName Workflow*.py WorkflowCore.py:from utils import getToolByName WorkflowCore.py:wf = getToolByName(instance, 'portal_workflow', None) WorkflowTool.py:from utils import getToolByName WorkflowTool.py:types_tool = getToolByName( self, 'portal_types', None )WorkflowTool.py:pt = getToolByName(self, 'portal_types', None) # Uses 'portal_types', *if present*, to compute list of type names, and to #verify type-specific bindings. [/home/tseaver/projects/CMF/CMF-1.4-head/CMFCore] $ cd ../DCWorkflow [/home/tseaver/projects/CMF/CMF-1.4-head/DCWorkflow] $ grep getToolByName *.py DCWorkflow.py:from Products.CMFCore.utils import getToolByName DCWorkflow.py:catalog = getToolByName(self, 'portal_catalog') # Uses 'portal_catalog' to build work lists. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: The bleak Future of Zope?
Tres Seaver wrote at 2004-4-26 11:46 -0400: Martijn Faassen wrote: Dieter Maurer wrote: ... We use SkinsTool, ActionsTool and DCWorkflow a lot, MembershipTool sometimes and most other tools not at all. Okay, point taken. :) How much do the tools listed interdepend on each other? See the attached file. ... ... longer list than written above ... This happens when I write from memory, sorry! We also use portal_types regularly. When you do not use the full functionality from the MembershipTool, you may not need the MemberDataTool nor the RegistrationTool. Our minimal tool usage is probably: Actions, Membership, Skins, Types, Workflow. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Weird couldn't load state on Zope 2.6.2
Chris Withers wrote at 2004-4-25 18:50 +0100:
When I click on the Advanced tab of one of my ZSQL Methods, I see this:
Traceback (innermost last):
...
- __traceback_info__: XXdF
Module ZODB.Connection, line 149, in __getitem__
- __traceback_info__: ('\x00\x00\x00\x00\x00\x08dF',
'(cExtensionClass\nExtensionClass\nq\x01(U\x1dToDoClass_PropertySheetsClassq\x02(cZClasses.ZClass\nPersistentClass\nq\x03cProducts.ZPatterns.PropertySheets\nVirtualSheets\nq\x04cZClasses.Property\nZInstanceSheets\nq\x05tq\x06}q\x07(U\x04iconq\x08U\x00U\n_p_changedq\tK\x01U\x07__doc__q\nU\x19ToDoClass
Property
Sheetsq\x0bU\x11__propset_attrs__q\x0c(U\x05Basicq\rtq\x0eU\x05Basicq\x0f(U\x08\x00\x00\x00\x00\x00\x08d\\q\x10(U\x11ZClasses.Propertyq\x11U\x0eZInstanceSheetq\x12tq\x13tq\x14QU\x06_p_oidq\x15U\x08\x00\x00\x00\x00\x00\x002nq\x16U\n__module__q\x17NU\t_p_serialq\x18U\x08\x038;\xa2\xb7n[\xb3q\x19utq\x1at.N.')
Could not load oid XXdF, pickled data in traceback info may
contain clues.
An incredibly old problem. I wrote at least 2 threads about it
(one with Andreas Tille with about 10 messages).
Please search the mailing list archives ([EMAIL PROTECTED]), for details...
--
Dieter
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] What do we want to bring from CVS to Subversion
I'm working on the cvs to subversion conversion for the ZODB, Zope 2, and Zope 3 projects. I'm currently doing the conversion of the full history with tags and branches. This is taking a long time and creating a huge repository, which is OK, but, do we really need that much history? I see 3 options: 1. Convert the full history with branches. This will create a rather large and complex repository. 2. Convert the mainline history, but leave off the branches. 3. Start with a clean slate and simply import the current head. Note that, for Zope 2 and ZODB, current maintenance branches will remain in CVS. I think that option 2 provides a nice compromise. The main disadvantage of it is that it will leave current development branches high and dry. I'm not sure how big an issue this is. In theory, these could be committed to the subversion heav via patch files. Thoughts? Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] What do we want to bring from CVS to Subversion
On Mon, 2004-04-26 at 15:23, Jim Fulton wrote: 2. Convert the mainline history, but leave off the branches. Would this mean we'd lose the log messages too? If so -1 because often merge messages aren't very helpful. merged foo-bar-branch to head would suck if you couldn't at least see the log messages on that branch. Otherwise, -0. Disk is cheap but history priceless. -Barry ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] What do we want to bring from CVS to Subversion
On Monday 26 April 2004 03:23 pm, Jim Fulton wrote: 2. Convert the mainline history, but leave off the branches. This sounds good to me. -Fred -- Fred L. Drake, Jr. fred at zope.com PythonLabs at Zope Corporation ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Subversion repository layout
The standard subversion repository layout is by project: proj1 /trunk /branches /br1 /br2 ... /tags /tag1 /tag2 ... proj2 /trunk /branches /br1 /br2 ... /tags /tag1 /tag2 ... ... With this layout, when you want to checkout the main development branch (aka head) of ZODB, you do: svn co svn+ssh://svn.zope.org/repos/ZODB/trunk ZODB That is, you need to include /trunk at the end and provide ZODB as the name of the output directory so as not to get a directory named trunk. If you forget the /trunk, you'll get a checkout that includes copies of all of the various tags and branches, which could be huge. I suggest, instead to use the following layout: proj1 proj2 tags /proj1 /tag1 /tag2 ... /proj2 /tag1 /tag2 ... branches /proj1 /br1 /br2 ... /proj2 /br1 /br2 ... With this layout, when you want to checkout the main development branch (aka head) of ZODB, you do: svn co svn+ssh://svn.zope.org/repos/ZODB which seems cleaner and less error prone. The only disadvantage I see in this layout is that we couldn't have projects named tags or branches, but I could live with that. Alternatively, we could have a top-level trunk directory: trunk /proj1 /proj2 tags /proj1 /tag1 /tag2 ... /proj2 /tag1 /tag2 ... branches /proj1 /br1 /br2 ... /proj2 /br1 /br2 ... But that would require inclusion of a /trunk dead chicken in checkouts: svn co svn+ssh://svn.zope.org/repos/trunk/ZODB which seems unnecessary to me. Thoughts? Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] What do we want to bring from CVS to Subversion
Barry Warsaw wrote: On Mon, 2004-04-26 at 15:23, Jim Fulton wrote: 2. Convert the mainline history, but leave off the branches. Would this mean we'd lose the log messages too? Yes, you'd lose the branch log messages. If so -1 because often merge messages aren't very helpful. merged foo-bar-branch to head would suck if you couldn't at least see the log messages on that branch. I hate to see merge messages like this. In fact, when people work on branches, I'd like to see the informative messages in the merge not on the branch work. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] What do we want to bring from CVS to Subversion
Jim Fulton wrote: Barry Warsaw wrote: On Mon, 2004-04-26 at 15:23, Jim Fulton wrote: 2. Convert the mainline history, but leave off the branches. Would this mean we'd lose the log messages too? Yes, you'd lose the branch log messages. If so -1 because often merge messages aren't very helpful. merged foo-bar-branch to head would suck if you couldn't at least see the log messages on that branch. I hate to see merge messages like this. In fact, when people work on branches, I'd like to see the informative messages in the merge not on the branch work. The reason that I hate merge messages like this is that I find it very difficult to find the branch checkin messages in thye log. It's possible, but so difficult that I generally would rather not bother. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] What do we want to bring from CVS to Subversion
On Mon, 2004-04-26 at 15:42, Jim Fulton wrote: The reason that I hate merge messages like this is that I find it very difficult to find the branch checkin messages in thye log. It's possible, but so difficult that I generally would rather not bother. I agree, but history is history, so we have to live with what we've got in there. Is your only objection the disk space it consumes? -Barry ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [ZODB-Dev] Subversion repository layout
On Mon, 2004-04-26 at 15:48, Tim Peters wrote: [Jim] I read some subversion docs over the weekend, and so am sufficiently prepared to live with the oddities of a standard subversion layout. I think that if you make a non-standard layout, then everyone coming to, or going from, Zope from/to other subversion projects will be forever tripping over the uniqueness of Zope's setup. IOW, the existing subversion docs cover the standard layout quite well. If we do something unique, I'm afraid it becomes another piece of folklore that will be impossible to guess and difficult to find out about. The other thing is that -- for me w/cvs at least -- full checkouts are rare. Usually I do a single checkout and then tons of updates, which never require the paths to be given. -Barry ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] What do we want to bring from CVS to Subversion
Barry Warsaw wrote: On Mon, 2004-04-26 at 15:23, Jim Fulton wrote: 2. Convert the mainline history, but leave off the branches. Would this mean we'd lose the log messages too? If so -1 because often merge messages aren't very helpful. merged foo-bar-branch to head would suck if you couldn't at least see the log messages on that branch. Otherwise, -0. Disk is cheap but history priceless. You aren't counting the cost of clutter. Option 3 says history prior to the CVS - SVN migration remains in CVS. We are committed to leaving the existing CVS repository in place *forever* (or until the 2.7 branch dies, whichever comes first ;). -1 on merging *any* history prior to the creation of the 2.7 branch to SVN; -0 on merging the mainline since then. Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] What do we want to bring from CVS to Subversion
Barry Warsaw wrote: On Mon, 2004-04-26 at 15:42, Jim Fulton wrote: The reason that I hate merge messages like this is that I find it very difficult to find the branch checkin messages in thye log. It's possible, but so difficult that I generally would rather not bother. I agree, but history is history, so we have to live with what we've got in there. Is your only objection the disk space it consumes? And the complexity in the resulting repository and the conversion times. I suspect that copying everything will take a couple of days. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: The bleak Future of Zope?
On Mon, Apr 26, 2004 at 07:47:39PM +0200, Dieter Maurer wrote: Our minimal tool usage is probably: Actions, Membership, Skins, Types, Workflow. I'm curious... do you use these with sites that are not in any other way based on CMFCore/CMFDefault ? -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's ! (random hero from isometric.spaceninja.com) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] RE: [ZODB-Dev] Subversion repository layout
Tim Peters wrote: I read some subversion docs over the weekend, and so am sufficiently prepared to live with the oddities of a standard subversion layout. I think that if you make a non-standard layout, then everyone coming to, or going from, Zope from/to other subversion projects will be forever tripping over the uniqueness of Zope's setup. IOW, the existing subversion docs cover the standard layout quite well. If we do something unique, I'm afraid it becomes another piece of folklore that will be impossible to guess and difficult to find out about. I don't know much about subversion, but this strikes me as a good argument for sticking with the standard layout. Regards, Martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Z2 HEAD in rough shape for b/c
[Chris McDonough] ... - The ZODBMountPoint product relies on a method of Connection objects named _getMountedConnection, which apparently no longer exists. Connection objects never had such a method, but they still do wink. This breaks any mounted databases (which breaks dbtab, which breaks sessions, which breaks lots of other things). Hopefully this is simple to fix, I haven't looked yet. I expect Shane will look at this in a week or two (when he settles down from moving) -- but can't know that. There was some horrid patching going on in ZODBMountPoint.py, which reached into various ZODB internals and replaced them. Jeremy refactored that, to make MountConnection a subclass of Connection, and MountConnection._getMountedConnection() is still there. There's still horrid patching going on, in that ZODBMountPoint.py reaches into ZODB.DB.DB and forces it to create MountConnection connections instead of Connection connections. Perhaps this isn't being done soon enough in whatever exactly it is you were trying. It would probably make most sense to move some of this code into ZODB (instead of replacing ZODB guts by magic). I think Shane wants to do that. I wish I could say it's peculiar that the Zope HEAD tests all pass despite the problems you're finding. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Archetypes-devel] Unicode in Zope 2 (ZMI, Archetypes, Plone, Formulator)
Bjorn Stabell wrote: Formulator: * gets charset from manage_page_charset (same as ZMI), but can be overridden * stores field values as encoded text (not Unicode), but lets you specify which encoding to use (confusingly calls this unicode mode) * messages are stored as UTF-8 (hardcoded) While there is no question about the confusingness of the user interface of Formulator pertaining unicode, most of this is not correct (unless there are bugs I don't know about). Formulator has two modes; unicode mode and 'classic mode'. In unicode mode, all strings are stored as Python unicode strings. In classic mode, all strings are stored in 'whatever encoding the user is using'. It's possible to convert from one mode to another, and for this switching behavior an encoding to use can be specified. In unicode mode, that encoding is ignored, however. Classic mode basically exists so as not to break all Formulator forms already in existence. This complicated the design significantly, but I thought this was important. Quite independently from this, fields can also be configured to *deliver* unicode upon validation/conversion. The character set is specified of the page that the form is in can be specified in the form settings. I suggest this way of dealing with Unicode right now in Zope 2: General note: this way sounds good to me, but I know from hard experience how difficult it is to convert an existing application to fully unicode. (1) Let ZPublisher do the encoding/decoding of form input and HTML output: a. Always set a character encoding in a HTTP Content-type request Silva does this (and Formulator too). b. Always append :ustring/utext/ulines/utokens:ENCODING to field names of fields that support Unicode (we may need some library code to make this easier) Formulator won't be able to do 'b' very easily. It'll do its own converting to unicode though for fields that want this. (2) Store Unicode strings directly in the ZODB. The ZODB is perfectly capable of storing strings in Python's internal Unicode format; no need to encode the text to UTF-8 or some other encoding. Silva has been doing this fully since version 0.9.2, released in the summer of last year. Formulator took a while longer to catch up (before it would only interoperate if the form titles etc were only ascii), but is now a first class citizen in a Zope/unicode environment. Its XML serialization is UTF-8 in this mode. (3) Encode/decode yourself when reading from/ writing to other external data sources such as files and other databases. Do it just before you write, or just after you read, so that as much code as possible can be encoding-agnostic. Keep the encoding/decoding as close to the source data as possible. The best way to do it is (in most cases) to specify the encoding on the IO stream, and let Python do the encoding/decoding for you transparently. If possible, get the encoding from the external data source (e.g., the file) instead of relying on a magical global variable. If you have to rely on a global variable, let it be manage_page_charset. (4) [This is really just advice...] Resist patching your code to work with components that doesn't deal with Unicode. Others are likely having the same problem, so to avoid ending up with lots of ugly patches (that are the source of mysterious Unicode problems), fix the problem at its source: the other component. It's really not that difficult to fix (if we agree on how it should be fixed ;) It's actually quite difficult to fix if you care about backwards compatibility. Fixing Formulator was quite complicated. You're definitely making this sound far easier than it is. It's a good thing to do, Silva has it, but the words 'not that difficult' don't fit in this debate. None of the above components handles Unicode in this way, but it seems to be how the Unicode support in Zope 2 was meant to be used. You're actually wrong about Formulator. :) Regards, Martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [Archetypes-devel] Unicode in Zope 2 (ZMI, Archetypes, Plone, Formulator)
David Convent wrote: Hi Bjorn, I always believed that unicode and utf-8 were same encoding, but reading you let me think i was wrong. Can you tell me what the difference is between unicode and utf-8 ? Unicode should not be seen as an encoding as such. While Python internally uses an encoding for unicode strings (which are the strings that if you represent them python will add a 'u' in front of them), you shouldn't care about what that is, and Python can in fact be recompiled to use another. UTF-8 is one particular way to represent unicode data, in this case as 8 bit strings. UTF-8 happens to be popular for two (related) reasons: * since UTF-8 includes ASCII, ASCII is automatically UTF-8 and UTF-8 without a lot of special characters looks like ASCII. * Software that can deal with 8 bit strings can usually deal with UTF-8. Anyway, in my experience most programmers have only a vague grasp of encoding issues. The basics are in Python not that hard to understand, but: * Python is not very educational if you do it wrong; you basically get weird errors * you get weird errors frequently in a different place in the code than where you made them, when some code is trying to combine unicode strings with classic strings. * you can 'hack' your way around it and survive for a long time. You don't notice the problem as it works with the test text which happens to be ascii. Etc. Regards, Martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Z2 HEAD in rough shape for b/c
On Mon, 2004-04-26 at 18:06, Tim Peters wrote: Connection objects never had such a method, but they still do wink. Ya, I figured this out later. ;-) This breaks any mounted databases (which breaks dbtab, which breaks sessions, which breaks lots of other things). Hopefully this is simple to fix, I haven't looked yet. I expect Shane will look at this in a week or two (when he settles down from moving) -- but can't know that. There was some horrid patching going on in ZODBMountPoint.py, which reached into various ZODB internals and replaced them. Jeremy refactored that, to make MountConnection a subclass of Connection, and MountConnection._getMountedConnection() is still there. Right. FWIW, the binding to DB.klass to allow the DB to use the MountConnection happens too late as it stands. But that doesn't matter too much for reasons below. There's still horrid patching going on, in that ZODBMountPoint.py reaches into ZODB.DB.DB and forces it to create MountConnection connections instead of Connection connections. Perhaps this isn't being done soon enough in whatever exactly it is you were trying. Yes, I fixed this by making the binding happen earlier (during startup). Once I do that, Zope in its default configuration indeed starts and appears to work. But any time an attempt is made to access or store an object in a mounted database (the default Zope sessioning configuration makes use of a mounted database), it falls over. The same code works ok against objects in the main database. You can probably replicate the problem by using the Zope Shopping Cart example code (accessible by viewing the Zope default home page without '/manage') For anyone who cares, here's a traceback demonstrating one variation of the symptom: 2004-04-25T22:38:57 ERROR SiteError http://www.plope.com/logged_in Traceback (most recent call last): File /home/chrism/software/Trunk/lib/python/ZPublisher/Publish.py, line 111, in publish request, bind=1) File /home/chrism/software/Trunk/lib/python/ZPublisher/mapply.py, line 88, in mapply if debug is not None: return debug(object,args,context) File /home/chrism/software/Trunk/lib/python/ZPublisher/Publish.py, line 40, in call_object result=apply(object,args) # Type scr to step into published object. File /home/chrism/var/zopemafia.org/Products/CMFCore/FSPythonScript.py, line 107, in __call__ return Script.__call__(self, *args, **kw) File /home/chrism/software/Trunk/lib/python/Shared/DC/Scripts/Bindings.py, line 306, in __call__ return self._bindAndExec(args, kw, None) File /home/chrism/software/Trunk/lib/python/Shared/DC/Scripts/Bindings.py, line 343, in _bindAndExec return self._exec(bound_data, args, kw) File /home/chrism/var/zopemafia.org/Products/CMFCore/FSPythonScript.py, line 162, in _exec result = f(*args, **kw) File Script (Python), line 22, in logged_in File /home/chrism/software/Trunk/lib/python/ZPublisher/HTTPRequest.py, line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File /home/chrism/software/Trunk/lib/python/ZPublisher/HTTPRequest.py, line 1174, in get if callable(v): v = v() File /home/chrism/software/Trunk/lib/python/Products/Sessions/SessionDataManager.py, line 93, in getSessionData return self._getSessionDataObject(key) File /home/chrism/software/Trunk/lib/python/Products/Sessions/SessionDataManager.py, line 180, in _getSessionDataObject ob = container.new_or_existing(key) File /home/chrism/software/Trunk/lib/python/Products/Transience/Transience.py, line 176, in new_or_existing item = self.get(key, notfound) File /home/chrism/software/Trunk/lib/python/Products/Transience/Transience.py, line 817, in get b = index.get(k, notfound) File /home/chrism/software/Trunk/lib/python/ZODB/Connection.py, line 795, in setstate self._setstate(obj) File /home/chrism/software/Trunk/lib/python/ZODB/Connection.py, line 849, in _setstate self._reader.setGhostState(obj, p) File /home/chrism/software/Trunk/lib/python/ZODB/serialize.py, line 401, in setGhostState obj.__setstate__(state) SystemError: new style getargs format but argument is not a tuple I haven't worked up the will to dug into the problem any further, but I intend to do so later this week. It would probably make most sense to move some of this code into ZODB (instead of replacing ZODB guts by magic). I think Shane wants to do that. I wish I could say it's peculiar that the Zope HEAD tests all pass despite the problems you're finding. It's understandable because these are all functional kinda bugs, dealing with the interaction between pieces of the system and/or the operation of new code against old database state. The unit tests could probably be improved to catch some of them, but nothing will ever be is a thorough as meat testing. Most of the time, though, I think I'd prefer that someone else were the meat. ;-) - C
[Zope-dev] RE: [Zope3-dev] RE: [ZODB-Dev] Subversion repository layout
[Tim Peters] ... IOW, the existing subversion docs cover the standard layout quite well. If we do something unique, I'm afraid it becomes another piece of folklore that will be impossible to guess and difficult to find out about. [Martijn Faassen] I don't know much about subversion, but this strikes me as a good argument for sticking with the standard layout. The I don't know part is exactly why I would prefer a standard layout. I didn't know anything about subversion, but the online book looks very good: http://svnbook.red-bean.com/ After reading (parts of) that, I thought I knew what the standard layout looked like, and why. Appendix A is especially for CVS users, and has a giant warning box saying: Warning Since Subversion treats branches and tags as ordinary directories, always remember to check out the trunk http://svn.example.com/repos/calc/trunk/) of your project, and not the project itself (http://svn.example.com/repos/calc/). If you make the mistake of checking out the project itself, you'll wind up with a working copy that contains a copy of your project for every branch and tag you have [That is, providing you don't run out of disk space before your checkout finishes.] The standard layout is assumed, like it is there, all over the book. Now if we change it to something we like better, the people doing the change will understand it completely, but few others will, and those relying on the svnbook docs to get up to speed will discover (probably the hard way) that all the examples in the book are wrong for Zope's unique layout. So I think it would do more harm than good, unless it does a lot more good than just saving me from typing an extra trunk/ now again. In the standard layout, each project has a trunk, branches, and tags subdirectory, containing what you already think they contain -- it's quite logical and elegant. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)
On 11/03/2004, at 7:25 PM, Chris McDonough wrote: On Wed, 2004-03-10 at 23:14, michael wrote: Thanks for your excellent isolation here. I don't get a KeyError thrown until *after* at least one rendering of standard_error_message that includes an access to a session variable. And I'm only getting KeyError's for sessions that have had a chance to expire. And that KeyError will be thrown whenever any access is made for the session. (Previously I thought I was *only* seeing KeyError's being thrown from standard_error_message.) I think after provide transaction isolation for the duration of an error, we will not see this symptom occur again. Okay - I've gone ahead and implemented a flavour of transaction isolation for errors. (See below for patch.) That seems to have done the trick just nicely. I have not seen any more KeyError's thrown from Transience.py. Yay! http://collector.zope.org/Zope/848 has a couple of examples of such KeyErrors thrown from Transience. These KeyError were a very painful user experience as either the server needed to be restarted or the user needed to delete their cookie for that server from their browser cookie jar. However there is another KeyError reported in 848 too. This one is thrown from TempoaryStoarge. After some further tweaking of my test rig I was able to produce such KeyErrors quite frequently. Using Zope 2.7.0 and recent the rewrite of Transience.py: http://cvs.zope.org/*checkout*/Products/Transience/Transience.py? rev=1.32.12.2.2.2only_with_tag=chrism-sessiongeddon errors look like: 2004-04-27T08:24:16 BLATHER(-100) OFS Exception while rendering an error message Traceback (most recent call last): File /Users/michael/builds/zope/Zope/lib/python/OFS/SimpleItem.py, line 204, in raise_standardErrorMessage v = s(**kwargs) File /Users/michael/builds/zope/Zope/lib/python/Shared/DC/Scripts/ Bindings.py, line 306, in __call__ return self._bindAndExec(args, kw, None) File /Users/michael/builds/zope/Zope/lib/python/Shared/DC/Scripts/ Bindings.py, line 343, in _bindAndExec return self._exec(bound_data, args, kw) File /Users/michael/builds/zope/Zope/lib/python/Products/PythonScripts/ PythonScript.py, line 318, in _exec result = f(*args, **kw) File Script (Python), line 12, in standard_error_message File /Users/michael/builds/zope/Zope/lib/python/ZPublisher/HTTPRequest.py, line 1218, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File /Users/michael/builds/zope/Zope/lib/python/ZPublisher/HTTPRequest.py, line 1178, in get if callable(v): v = v() File /Users/michael/builds/zope/Zope/lib/python/Products/Sessions/ SessionDataManager.py, line 93, in getSessionData return self._getSessionDataObject(key) File /Users/michael/builds/zope/Zope/lib/python/Products/Sessions/ SessionDataManager.py, line 180, in _getSessionDataObject ob = container.new_or_existing(key) File /Users/michael/builds/zope/Zope/lib/python/Products/Transience/ Transience.py, line 504, in new_or_existing item = self.get(key, _marker) File /Users/michael/builds/zope/Zope/lib/python/Products/Transience/ Transience.py, line 332, in get item = self._move_item(k, current_ts, _marker) File /Users/michael/builds/zope/Zope/lib/python/Products/Transience/ Transience.py, line 208, in _move_item self._gc(current_ts) File /Users/michael/builds/zope/Zope/lib/python/Products/Transience/ Transience.py, line 429, in _gc for key in list(self._data.keys(None, max_ts)): File /Users/michael/builds/zope/Zope/lib/python/ZODB/Connection.py, line 561, in setstate p, serial = self._storage.load(oid, self._version) File /Users/michael/builds/zope/Zope/lib/python/tempstorage/ TemporaryStorage.py, line 94, in load s=self._index[oid] KeyError: '\x00\x00\x00\x00\x00\x00\x08\x82' So far it looks like commenting out the construction of the to_notify list in _gc makes this go away. So _gc looks like: def _gc(self, now=None): if self._timeout_slices == 0: return # dont do gc if there is no timeout if now is None: now = getCurrentTimeslice() # for unit tests max_ts = now - (PERIOD * (self._timeout_slices + 1)) keys = self._data.keys(None, max_ts) #to_notify = [] for key in list(self._data.keys(None, max_ts)): assert(key = max_ts) STRICT and _assert(self._data.has_key(key)) #for v in self._data[key].values(): #to_notify.append(v) del self._data[key] #for v in to_notify: #self.notifyDel(v) Note that the test rig does not set onAdd or onDelete. Along the way there where a few things that I was not too sure about so I tried fiddling with to see what would happen: (1) Use of hasattr in Transience.py. hasattr should be avoided? See http://mail.zope.org/pipermail/zodb-dev/2003-October/005968.html. I did change usage
[Zope-dev] RE: [ZODB-Dev] Subversion repository layout
On Mon, 2004-04-26 at 15:48, Tim Peters wrote: [Jim] The standard subversion repository layout is by project: proj1 /trunk /branches /br1 /br2 ... /tags /tag1 /tag2 ... ... With this layout, when you want to checkout the main development branch (aka head) of ZODB, you do: svn co svn+ssh://svn.zope.org/repos/ZODB/trunk ZODB That is, you need to include /trunk at the end and provide ZODB as the name of the output directory so as not to get a directory named trunk. If you forget the /trunk, you'll get a checkout that includes copies of all of the various tags and branches, which could be huge. I suggest, instead to use the following layout: ... Thoughts? I read some subversion docs over the weekend, and so am sufficiently prepared to live with the oddities of a standard subversion layout. I think that if you make a non-standard layout, then everyone coming to, or going from, Zope from/to other subversion projects will be forever tripping over the uniqueness of Zope's setup. IOW, the existing subversion docs cover the standard layout quite well. If we do something unique, I'm afraid it becomes another piece of folklore that will be impossible to guess and difficult to find out about. i like the layouts jim's presented (specifically #2 of 3), i think when considering the subversion docs, the important distinctions are made between the directories used for branches and tags, as long as that information is clearly communicated the semantics are exactly the same. the subversion docs themselves outline multiple repository structures (for example the single project layout), although they do recommend a standard structure, the docs go through great lengths to convey a semantic understanding of subversion as a versioned filesystem, not a magic functional notion as is common with cvs. i honestly dont think anyone coming from/to a subversion system will have problems as long as the location of the trunk, tags, and branches directories for a project are clearly identifiable. quoting the svn docs. Lay out your repository in whatever way you see fit. Subversion does not expect or enforce a layout schemain its eyes, a directory is a directory is a directory. Ultimately, you should choose the repository arrangement that meets the needs of the people who work on the projects that live there. in the existing zopeorg cvs repository, most of the existing projects are python packages, multiples of which must be stitched together to make a working dev environment, using jim's layout avoids alot of needless renaming. on the flipside, it only avoids the renaming in the case of using the trunk, and while they are the only things being converted at the moment, from my experience most of the checkouts from zopeorg have been on release branches, afaik, and i believe jim wanted to continue with existing structure such that most of the toplevel projects are not python packages. so the benefits/comparative advantage aren't that strong in this case in terms of breaking with the common expectation of repository layout. two cents, -kapil ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] performance tuning of ZODB
Hi Syver, Please add this issue to the Collector, including the test (prefereably without the twisted bits) On Thu, 2004-04-22 at 12:57, Syver Enstad wrote: [...] I have a strange case here with ReadConflictErrors. I don't know if this is covered already but anyway. I am using ZODB 3.2 so this might be fixed in 3.3 for all I know. Just replace the twisted stuff with the standard lib unittest module to have it work on a computer without twisted installed. -- Leonardo Rochael Almeida [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [Archetypes-devel] Unicode in Zope 2 (ZMI, Archetypes, Plone, Formulator)
None of the above components handles Unicode in this way, but it seems to be how the Unicode support in Zope 2 was meant to be used. Martijn wrote: You're actually wrong about Formulator. :) Apologies. We were using older versions of Formulator before, and I was just doing code inspection of the new version when I concluded the above about Formulator. One less component to worry about :) Bye, -- Bjorn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [ZODB-Dev] Subversion repository layout
[Kapil Thangavelu] ... I like the layouts Jim's presented (specifically #2 of 3), i think when considering the subversion docs, the important distinctions are made between the directories used for branches and tags, as long as that information is clearly communicated the semantics are exactly the same. the subversion docs themselves outline multiple repository structures (for example the single project layout), Sorry, I haven't seen that. The Choosing a Repository Layout section does talk about many ways to organize repositories, like multiple repositories vs one, and putting all projects in a repository at top level vs grouping them into related subtrees. But in all these cases, the only structure for a project they discuss or illustrate has project/trunk, project/branches, and project/tags structure. although they do recommend a standard structure, the docs go through great lengths to convey a semantic understanding of subversion as a versioned filesystem, not a magic functional notion as is common with cvs. i honestly dont think anyone coming from/to a subversion system will have problems as long as the location of the trunk, tags, and branches directories for a project are clearly identifiable. I haven't used svn, and I'm more concerned about people like me wink: the docs assume a specific project (not repository) layout throughout. I'm not interested in svn for its own sake, it's just something I'll need to do my job. The closer the docs match the system I have to work with, the easier it will be to get started. I can't say Jim's suggestions are bad, or good -- I can't judge them since I haven't used svn (you?). Going against the recommendation of the people who designed and coded the system seems a dubious step on the face of it, though. quoting the svn docs. Lay out your repository in whatever way you see fit. Subversion does not expect or enforce a layout schema-in its eyes, a directory is a directory is a directory. Ultimately, you should choose the repository arrangement that meets the needs of the people who work on the projects that live there. That's at the end of the Choosing a Repository Layout section. As above, that section discusses and shows nothing but the standard per-project layout; the layout choices they do discuss in that section are the ones mentioned above (how to organize projects in relation to each other, and in relation to repositories). ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [ZODB-Dev] Subversion repository layout
On Mon, 2004-04-26 at 22:30, Tim Peters wrote: [Kapil Thangavelu] ... I like the layouts Jim's presented (specifically #2 of 3), i think when considering the subversion docs, the important distinctions are made between the directories used for branches and tags, as long as that information is clearly communicated the semantics are exactly the same. the subversion docs themselves outline multiple repository structures (for example the single project layout), Sorry, I haven't seen that. The Choosing a Repository Layout section does talk about many ways to organize repositories, like multiple repositories vs one, and putting all projects in a repository at top level vs grouping them into related subtrees. But in all these cases, the only structure for a project they discuss or illustrate has project/trunk, project/branches, and project/tags structure. there are numerous references throughout the docs to alternative repository structures, like having one with /trunk/projectname http://svnbook.red-bean.com/svnbook/ch03s06.html or several examples w/ pics without trunk, tag, branches at all, http://svnbook.red-bean.com/svnbook/ch02s03.html but regardless its true that the vast majority of the examples assume the /projectname/trunk /branches setup. although they do recommend a standard structure, the docs go through great lengths to convey a semantic understanding of subversion as a versioned filesystem, not a magic functional notion as is common with cvs. i honestly dont think anyone coming from/to a subversion system will have problems as long as the location of the trunk, tags, and branches directories for a project are clearly identifiable. I haven't used svn, and I'm more concerned about people like me wink: the docs assume a specific project (not repository) layout throughout. I'm not interested in svn for its own sake, it's just something I'll need to do my job. The closer the docs match the system I have to work with, the easier it will be to get started. imo, the docs dont assume one layout, they have mixed examples, with a majority of them given with suggested layout, reading past the copy/paste code though and i think it tries to make clear numerous times that its not about the layout its about the semantics and policy assigned to locations. for example here's a quote on creating branches Creating a branch is very simpleyou make a copy of the project in the repository using the svn copy command. Subversion is not only able to copy single files, but whole directories as well. In this case, you want to make a copy of the /calc/trunk directory. Where should the new copy live? Wherever you wishit's a matter of project policy. Let's say that your team has a policy of creating branches in the /calc/branches area of the repository, and you want to name your branch my-calc-branch. You'll want to create a new directory, /calc/branches/my-calc-branch, which begins its life as a copy of /calc/trunk. I can't say Jim's suggestions are bad, or good -- I can't judge them since I haven't used svn (you?). Going against the recommendation of the people who designed and coded the system seems a dubious step on the face of it, though. i've been using svn for almost 2yrs, for client projects the last year, and i'm about to convert over plone and the sf.net collective project (~150 Products) and around 200 committers over to svn. i think jim's proposed layout could be very helpful there, as typically people will be using unfamiliar gui clients (like tortoisesvn) and will be checking out the head of the products and nitting them together to create a working zope instance/dev enviornment, in which case avoiding having to fillin an extra dialog might be nice and pratical. but i'm going to try and solicit feedback on those project lists. quoting the svn docs. Lay out your repository in whatever way you see fit. Subversion does not expect or enforce a layout schema-in its eyes, a directory is a directory is a directory. Ultimately, you should choose the repository arrangement that meets the needs of the people who work on the projects that live there. That's at the end of the Choosing a Repository Layout section. As above, that section discusses and shows nothing but the standard per-project layout; the layout choices they do discuss in that section are the ones mentioned above (how to organize projects in relation to each other, and in relation to repositories). it does shows another layout in the choosing repository layout scheme section, namely the single project etc, which you could say jim's layout is adapted from.. ie. /trunk /branches /tags sigh.. debating over what the book says isn't very productive. my conclusions at the end of my previous email, namely that what this layout will accomplish for the zopeorg repository in terms of avoiding renames of checkouts will likely be fairly limited in
