Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thu, 31 May 2001 10:36:55 -0400 (EDT), Shane Hathaway [EMAIL PROTECTED] wrote: On Thu, 31 May 2001, Toby Dickenson wrote: On Thu, 31 May 2001 10:03:31 -0400 (EDT), Shane Hathaway [EMAIL PROTECTED] wrote: Right now ZCatalog randomly generates ConflictErrors even if there are no conflicts in the data being indexed. It's quite rare, however, and there's machinery that retries the entire transaction up to three times when it happens. If the current mechanism is reliable, and efficient (given that conflict/retries are rare) then I dont understand the motivation The system becomes less reliable as server load increases. Large catalog updates (where every object is reindexed) also generate a lot of conflicts. Is that last bit true? I thought 'Update Catalog' created *new* indexes. There might be a conflict on the root catalog opbject, but inside the index BTrees. Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] mxDateTime-based Zope DateTime module
Hello everyone, after I got finally frustrated enough with the current implementation of the Zope DateTime module, I decided to write a wrapper for mxDateTime. I checked the license, and the eGenix Public License is GPL compatible, so that we can use it for Zope now. For more details, see http://www.lemburg.com/files/python/mxDateTime.html. You can find the first version of the new DateTime module here: http://www.zope.org/Members/srichter/Products/DateTimeNG Currently, it has a problem with ZCatalog. If you create a catalog and then restart Zope, you get a nasty setstate error: 2001-06-07T04:54:40 ERROR(200) ZODB Couldn't load state for '\000\000\000\000\000\000 \015' Traceback (innermost last): File /opt/devel/dev11480/lib/python/ZODB/Connection.py, line 533, in setstate SystemError: Failed to import class localtime from module __main__ (This is abbreviated debug output from the console.) Since I would like to see this new module in the Zope core ASAP (best in Zope 2.4), I wanted to ask you guys, whether you could have a look at this problem, since I do not know much about the Catalog and the entire ZODB pickle/unpickle process. Furthermore, I would love to hear your comments and ideas. With mxDateTime we can do much more now and have many more possibilities. The current version of the new DateTime module handles most of the old API functions correctly. Only some timezone issues are left open, but using the existing time zone mappers this should be no problem. Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Student Web2k - Web Design/Development Technical Project Management ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
My last post didnt make much sense. This time ill include all the words in my sentence Large catalog updates (where every object is reindexed) also generate a lot of conflicts. Is that last bit true? I thought 'Update Catalog' created *new* indexes. There might be a conflict on the root catalog object, but not on the index BTrees. Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A simple dtml-if question...
Thanks Christian, but it didn't work. I went with the long way: dtml-if expr=LoginResults=='Pass' dtml-if expr=PATH_INFO=='/Maintain/Results' Valid response dtml-else False response /dtml-if dtml-else False response /dtml-if dtml-if expr=LoginResults=='Pass' and PATH_INFO=='/Maintain/Results' Valid Response dtml-else Fasle Response /dtml-if cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Request for a Pluggin Index (NameIndex)
- Original Message - From: ender [EMAIL PROTECTED] To: Andreas Jung [EMAIL PROTECTED] Cc: zope-dev [EMAIL PROTECTED] Sent: Wednesday, June 06, 2001 5:30 PM Subject: Re: [Zope-dev] Request for a Pluggin Index (NameIndex) On Monday 04 June 2001 16:55, Andreas Jung wrote: Looks like you should write your own index type. Zope 2.4 comes with an PlugableIndex interface to allow third-party indexes to be integrated into the Catalog. this brings up an interesting question of what is the best way to register a new plugindex thats distributed with a product. Glancing over the cvs logs it looks as though plugin indexes are arranged to be the first product installed in Application.py. Given that what is the suggested method for registering a new plugin index? I think this should be subject of a small How-To. Anyway...to register a plugin index you have to call context.registerClass(...). Take a look at PluginIndexes/__init__.py how Zopes indexes are registered. Other indexes should do it in the same way. The reason why PluginIndexes are installed as first product is that there are some dependencies between PluginIndexes and other Zope Products. Products are usually inialized in alphabetical order. But in this case we made an exception. Andreas ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thursday 07 June 2001 07:43, Toby Dickenson wrote: Large catalog updates (where every object is reindexed) also generate a lot of conflicts. Is that last bit true? I thought 'Update Catalog' created *new* indexes. There might be a conflict on the root catalog object, but not on the index BTrees. I should say large catalog updates tend to generate a conflict. It really doesn't matter how many conflicts there are. Within a single transaction, 1 conflict is as bad as 100. One thing I didn't make clear in the proposal is that I'm interested in repurposing ZCatalog as a general ZODB indexing mechanism and essentially moving it down from the application layer to the database layer. Catalogs would be updated automatically (in a lazy fashion to avoid performance penalties). There would potentially be a large number of catalogs, though, so the number of conflicts would increase significantly. But I think I have a solution for all of the issues in conflict resolution. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
Shane Hathaway wrote: But I think I have a solution for all of the issues in conflict resolution. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Please do this! :-) I'm particularly interested in ZCatalog becoming a standard ZODB feature, and I know there are otehrs on ZODB-dev who are dying for this right now... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] mxDateTime-based Zope DateTime module
We at one point were fiddling with mx from eegenix, for psycopgDA. We found out the hard way that the mx.h headers in the DateTime were being put in a funny place. Perhaps that may give you a clue. I hope you are going to use mxDateTime for a new PostGreSQL DA. This would be for me the MAIN reason to choose it, since I really want DateTime objects returned, not strings. For the Zope DateTime module: Since we are working on the Python level I thought it would be no problem on the C level. Unfortunetely I do not know enough about the Python C API to check this. Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Student Web2k - Web Design/Development Technical Project Management ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Using Threads in Zope-Product
Hi zopistas, I currently work on a PythonProduct called JobManager. The purpose of this Product is, that one can define a hierachical set of PythonScripts (Jobs with SubJobs) that are executed in seperate Threads by call or scheduled. This is intended for use with expensive Database-Update or my ServiceManager (XML-RPC Server/Client) which need to much time to complete, that a user can't / doesn't want to wait for completion. Now there is some trikyness in programming Threads in zope .. because I wanted the thread-instances to be accessible throug zope as long as they are running - ZODB uses the module thread (start_new_thread, aquire/release_lock) - Scheduler (i found it in DC-CVS) is probably not finished and uses thread as well. the thread instance is not accessible from Zope as far as i can see - ZExternalNewManager uses the module threading (Thread(...)) the thread-instance is stored as volatile attribute (_v_threadinstance) Using the threading library would be most convinient but the Objects from threading are not pickable. I need Locks and would like to have Conditions to control the behavior of the JobManager, so I tried to make a copy of threading and added Persistence .. (derived all classes from ZODB.Persistent, but doesn't work) with no success. Is it basically possible .. does it make sense to store the instance of a Thread as an attribute of a ZODB-stored Object ??? If yes .. what changes are neccesary to make threading persistant If no .. has anybody else done such a thing with success ?? any comment welcome thanks Ulrich Eck net-labs ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] mxDateTime-based Zope DateTime module
At 10:28 AM 6/7/01 -0400, Chris McDonough wrote: Actually, it's good that the license is BSDish, that means that it's legally possible to include mxDateTime inside a Zope distribution. Cool, so the license problem is definitely solved. Now, we just have to get rid of the last technical issues. From DC's point of view: Is there a chance, that this port to mxDateTime could make it into Zope 2.4? I think this would be the correct release in anyway, since there will be so many changes. Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Student Web2k - Web Design/Development Technical Project Management ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] mxDateTime-based Zope DateTime module
I think using mxDateTime would be a great idea (it would mean DC would no longer need to maintain our own DateTime module). And since you've done a lot of good work in creating a wrapper, we've got a head start. But I think before it can go in to any release, we need to follow the fishbowl process. This means creating a proposal on dev.zope.org, letting Brian know about it, and then letting Brian choose if and when he wants it to go in (it will subsequently be made into a project). Thanks, - C Stephan Richter wrote: At 10:28 AM 6/7/01 -0400, Chris McDonough wrote: Actually, it's good that the license is BSDish, that means that it's legally possible to include mxDateTime inside a Zope distribution. Cool, so the license problem is definitely solved. Now, we just have to get rid of the last technical issues. From DC's point of view: Is there a chance, that this port to mxDateTime could make it into Zope 2.4? I think this would be the correct release in anyway, since there will be so many changes. Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Student Web2k - Web Design/Development Technical Project Management ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thu, 7 Jun 2001, Shane Hathaway wrote: It really doesn't matter how many conflicts there are. Within a single transaction, 1 conflict is as bad as 100. Why? But I think I have a solution for all of the issues in conflict resolution. Whee! If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Please do this! ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
Me too! Me too! John On Thursday 07 June 2001 08:49, Chris Withers wrote: Shane Hathaway wrote: But I think I have a solution for all of the issues in conflict resolution. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Please do this! :-) I'm particularly interested in ZCatalog becoming a standard ZODB feature, and I know there are otehrs on ZODB-dev who are dying for this right now... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) -- . . . . . . . . . . . . . . . . . . . . . . . . John D. Heintz | Senior Engineer 1016 La Posada Dr. | Suite 240 | Austin TX 78752 T 512.633.1198 | [EMAIL PROTECTED] w w w . d a t a c h a n n e l . c o m ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] mxDateTime-based Zope DateTime module
At 11:21 AM 6/7/01 -0400, Chris McDonough wrote: I think using mxDateTime would be a great idea (it would mean DC would no longer need to maintain our own DateTime module). And since you've done a lot of good work in creating a wrapper, we've got a head start. But I think before it can go in to any release, we need to follow the fishbowl process. This means creating a proposal on dev.zope.org, letting Brian know about it, and then letting Brian choose if and when he wants it to go in (it will subsequently be made into a project). Cool. I will do that. Let's see whether we can get the last Catalog bug out of there, so that we can have a good prototype. I am very excited about that. I think this improvement will also be of advantage to the database adapters and other products that use Dates and Times extensively. Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Student Web2k - Web Design/Development Technical Project Management ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] DBObjects 1.1.0b1 (with volatile support)
Hello everyone, in my development of making SmartObjects a reality, I have implemented the 'isVolatile' attribute into DBObjects. If an object is volatile, it is not saved in the ZODB, but is created on run time. Thanks a lot Jonathan; without LocalFS as my template I could have not done it! ALSO, a lot of thanks to bluedynamics.com, which sponsored this particular development!!! I just put the latest package on the iuveno demo server: http://demo.iuveno-net.de/iuveno/Products/DBObjects I tagged this version 1.1.0 beta 1, since I am sure there are little bugs left due to huge code changes. Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Student Web2k - Web Design/Development Technical Project Management ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
--On Thursday, June 07, 2001 09:34:54 AM -0400 Shane Hathaway [EMAIL PROTECTED] wrote: But I think I have a solution for all of the issues in conflict resolution. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Please do this! It is the sort of thing that can save us and others a lot of time by keeping use of Zope simple and stable. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [Announce] API Documentation Fishbowl Project
--On Wednesday, June 06, 2001 11:50:43 AM -0700 jimbo [EMAIL PROTECTED] wrote: I hope this helps. I wanted to add my feelings on the whole documentation issue. It seems to me that the whole process caters around developers too much. I have to disagree with you **in the context of this thread** for two reasons: 1. DC has done a lot to improve the documentation for non-developers in the last year. The Zope Book(s) should have a major impact when they start to appear on shelves in the next month or so. Developer documentation has lagged behind. This thread is about a proposal to improve developer documentation. It says nothing at all about the other *existing* efforts by DC and others to improve other types of Zope documentation. 2. One of the main points DC made at this years Python conference was that they have tried to focus the Zope core on too many audiences at once. They had to have a clearer focus and chose developers as that focus. Of course I like this choice because I am a developer, but the more important point is that this tighter focus has the potential to make life easier for all of us. For example, the CMF is focused on content managers. It is implemented as a *separate*, *layered* product on top of Zope. This means that it is free to make tradeoffs in favor of content managers that the Zope core isn't. It also means that CMF release schedules are not tied to Zope release schedules. Incremental restructuring of the Zope core and improved Zope developer documentation makes it much easier and more practical for DC and others to create layered Zope products that address other communities. Continuing the current rather inscrutable nature of the Zope core makes all of this harder for everyone involved. Dan Pierson, Control.com, Inc. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Bug in Zope VersionControl
hello again ...
Okay ... I admit using opera and enjoying it.
Problem is, that opera is sooo standardsconform.
See Zope/lib/python/Products/OFSP/Version.py:175
in function enter()
Somebody thats the path for the cookie as SCRIPT_NAME.
This seems that the scope of the versions should be
limited to the subtree where the version object was
instanciated. Nice idea.
But this doesn't work.
First:
Internet Explorer and Netscape ignore the path of the cookie
and assume '/'.
Second:
Opera is conform to the rfc of http 1.1, and this means, that
the cookie is only valid for the version itself, and is not
used in any place out of http://myzope:8080/blaah/myVersion
Proposed solution:
Change the path to '/'. And have the same behaviour on all
browsers.
Or:
Change the path to REQUEST[URL1] (is this the parent folder?)
and have the intended mechanism working at least on opera.
The last is my personal favorite, because you can have different
versions concurrently open on different projects @ one server.
Proposed patch for both solutions comes as attachement.
(both intended for zope 2.4.0a1)
(btw: remember same behaviour on function leave() ...)
--
Christian Theune - [EMAIL PROTECTED]
gocept gmbh co.kg - schalaunische strasse 6 - 06366 koethen/anhalt
tel.+49 3496 3099112 - fax.+49 3496 3099118 mob. - 0178 48 33 981
reduce(lambda x,y:x+y,map(lambda x:chr(ord(x)^42),tuple('zS^BED\nX_FOY\x0b')))
--- Version.py Thu Jan 11 16:43:57 2001
+++ lib/python/Products/OFSP/Version.py Thu Jun 7 20:25:20 2001
@@ -172,7 +172,7 @@
Begin working in a version
RESPONSE.setCookie(
Globals.VersionNameName, self.cookie,
-path=REQUEST['SCRIPT_NAME'],
+path='/',
)
if (REQUEST.has_key('SERVER_SOFTWARE') and
REQUEST['SERVER_SOFTWARE'][:9]=='Microsoft'):
@@ -190,7 +190,7 @@
RESPONSE.setCookie(
Globals.VersionNameName,'No longer active',
expires=Mon, 25-Jan-1999 23:59:59 GMT,
-path=REQUEST['SCRIPT_NAME'],
+path='/',
)
if (REQUEST.has_key('SERVER_SOFTWARE') and
REQUEST['SERVER_SOFTWARE'][:9]=='Microsoft'):
--- Version.py Thu Jan 11 16:43:57 2001
+++ lib/python/Products/OFSP/Version.py Thu Jun 7 20:25:20 2001
@@ -172,7 +172,7 @@
Begin working in a version
RESPONSE.setCookie(
Globals.VersionNameName, self.cookie,
-path=REQUEST['SCRIPT_NAME'],
+path=REQUEST['URL1'],
)
if (REQUEST.has_key('SERVER_SOFTWARE') and
REQUEST['SERVER_SOFTWARE'][:9]=='Microsoft'):
@@ -190,7 +190,7 @@
RESPONSE.setCookie(
Globals.VersionNameName,'No longer active',
expires=Mon, 25-Jan-1999 23:59:59 GMT,
-path=REQUEST['SCRIPT_NAME'],
+path=REQUEST['URL1'],
)
if (REQUEST.has_key('SERVER_SOFTWARE') and
REQUEST['SERVER_SOFTWARE'][:9]=='Microsoft'):
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thursday 07 June 2001 11:51, Erik Enge wrote: On Thu, 7 Jun 2001, Shane Hathaway wrote: It really doesn't matter how many conflicts there are. Within a single transaction, 1 conflict is as bad as 100. Why? Because in Zope it means the whole request is processed again (which can lead to strange behavior if non-transactional data is changed by the request). In other apps it might mean messy exception handling. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Please do this! I should have stated that it also needs to be a priority. Within DC we've been using better project management and it's getting harder to do projects just because we might think they'll be useful for someone. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thursday 07 June 2001 12:17, Phillip J. Eby wrote: At 09:34 AM 6/7/01 -0400, Shane Hathaway wrote: One thing I didn't make clear in the proposal is that I'm interested in repurposing ZCatalog as a general ZODB indexing mechanism and essentially moving it down from the application layer to the database layer. Catalogs would be updated automatically (in a lazy fashion to avoid performance penalties). There would potentially be a large number of catalogs, though, so the number of conflicts would increase significantly. But I think I have a solution for all of the issues in conflict resolution. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Sounds cool. At one time Ty and I thought we might do something like this by adding to a linked list of pending updates which the catalog would use to alter its search results, until the list got too long, at which point it would post the actual updates. That sounds like what I have in mind. The only catch was that this would still produce conflicts at the head end of the linked list. :( Of course, that was in the days before ZODB conflict resolution. Nowadays, you could probably implement it as a simple sequence object with the conflict resolution method implemented. But then there'd be the question of how to resolve conflicts if more than one thread or ZEO client decided to apply the queued changes... a 100 conflicts vs 1 situation. Ugh. I was thinking the queue would only last the duration of a transaction and that the queue would be thread-specific. Anyway, I'd be really interested in seeing a solution for all the issues in conflict resolution. I was thinking that certain types of objects would be committed by the transaction manager before all others. In this case, the catalog (or a special object in the catalog) would be committed first. It would resolve all conflicts in the contained indices before they occur by replaying the changes in the persisted queues from the transaction history, then setting the _p_serial attributes to convince the storage that the conflicts have already been resolved. Will it help with arguments, too? How about world peace? ;) Okay, maybe we haven't solved that. :-) Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
At 02:07 PM 6/7/01 -0400, Shane Hathaway wrote: On Thursday 07 June 2001 12:17, Phillip J. Eby wrote: The only catch was that this would still produce conflicts at the head end of the linked list. :( Of course, that was in the days before ZODB conflict resolution. Nowadays, you could probably implement it as a simple sequence object with the conflict resolution method implemented. But then there'd be the question of how to resolve conflicts if more than one thread or ZEO client decided to apply the queued changes... a 100 conflicts vs 1 situation. Ugh. I was thinking the queue would only last the duration of a transaction and that the queue would be thread-specific. That's not hard to do - you could just toss an object into the transaction queue to do it during transaction commit, similar to what ZPatterns does in a more general way. (Note that ZPatterns users can already get the benefits of deferring catalog changes until the end of a (sub)transaction, by using triggers to implement automatic cataloging). Anyway, I'd be really interested in seeing a solution for all the issues in conflict resolution. I was thinking that certain types of objects would be committed by the transaction manager before all others. In this case, the catalog (or a special object in the catalog) would be committed first. It would resolve all conflicts in the contained indices before they occur by replaying the changes in the persisted queues from the transaction history, then setting the _p_serial attributes to convince the storage that the conflicts have already been resolved. Hm. Sounds to me like what you actually want is for the transaction manager to do this *after* everything else, rather than before. Thus, you would catch any changes which occur *during* transaction commit - such as commit-phase cataloging (as some folks do with ZPatterns currently). That is, in ZPatterns one can specify triggers such as: WHEN OBJECT DELETED, CHANGED CALL someCatalog.manage_uncatalog(self.absolute_url(1)) WHEN OBJECT ADDED, CHANGED CALL someCatalog.manage_catalog(self,self.absolute_url(1)) Which will be executed during transaction commit if the conditions apply. If you have the catalog process its queue *before* other objects commit, it will not have received these calls yet. This will end up making the catalog try to commit itself a second time, which will fail with a conflict error - 100% of the time. (And retry won't help, because the transaction is conflicting with itself.) Of course, this issue could be fixed by minor adjustment to the ZODB API and implementation such that it is permissible to store() an object more than once during a transaction's commit phase without creating a ConflictError. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] xt_objects, and weird problems with properties
i am encountering some rather bizarre problems with 'properties' on xt_objects, which are effectively fancy folderish DTML Documents. the problems are two fold... - i call manage_changeProperties on an xt_object within other xt_objects, and at times, it would change the same property, e.g. xt_options, on all the other xt_objects in the same container! this would be magically rectified when i restart the zope server! - the second problem is of a different nature, it's to do with 'Date properties'. i have several date properties set by dtml when an object is created, e.g. xt_expiration_date. however, when i manually change a date property through manage_propertiesForm, it suddenly stops being a date property, e.g. if set through DTML, i can do dtml-var xt_expiration_date.year() but, i get an attribute error on year, if i change it manually... the code for xt_objects can be found at http://mu2.espnow.com/~zope/xt_object.py and any help would be appreciated. -- best regards, tav [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Property Storage
Hi, If I change one property on, say, a DTML Document, does that store a whole new copy of the document in the ZODB? If not, then how are properties stored? Is so, then how about storing properties in their own mini-class that just subclassed Persistent, so each property got its own pickle jar? What do people think/know? cheers, Chis ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
Phillip J. Eby wrote: That is, in ZPatterns one can specify triggers such as: WHEN OBJECT DELETED, CHANGED CALL someCatalog.manage_uncatalog(self.absolute_url(1)) WHEN OBJECT ADDED, CHANGED CALL someCatalog.manage_catalog(self,self.absolute_url(1)) After I read this again I realized what you were saying. This capability of ZPatterns is very brittle, don't you think? If the catalog is updated manually before the special ZPatterns object is added to the queue, the behavior is undefined AFAIK--either the later changes to the catalog will be ignored, will cause a conflict, or some objects will be written twice in the same transaction. However, if we could specify transaction commit priorities, and the ZPatterns update came first, auto-indexing came second, and everything else followed, I think it would work. Or perhaps ZPatterns should be able to register things that occur *before* the two-phase commit protocol. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [Announce] API Documentation Fishbowl Project
On Wed, 6 Jun 2001, jimbo wrote: I believe that if you are a true developer you will/can figure out the api given the vast information available today. For example the dcworkflow product was just released. I believe the best documentation would be how-to actually use the product. Ah, not really. Or, rather, maybe, with the *new* products whose APIs are being constructed/documented better from the start, but if and probably only if there are either comprehensive examples or the framework docs mentioned in the proposal. IMO the biggest issue here for developers, as others have said, is clarifying/documenting the old Zope interfaces. If we get that, then new products will be even better/better integrated. This includes ZClasses. Half the problem with ZClasses is that they sort of almost work, but they break down partly because the interfaces really aren't documented very well. (I strongly recommend learning python. It's an easy language to learn. Then use real Python classes instead of ZClasses. ZClasses have their place, but if you are doing serious development they tend to get in the way more than they help, IMO). In short enough geek speek. Change the language into something the rest of masses can understand. How can I use zope/API to get PAID! How can I actually make the dcworkflow or the core session or the ZPT do something. Plenty of example uses. I think they might call them case studies or something to that effect. How can I actually make x do something sounds an awful lot like what I think the framework docs portion of the proposal is addressing, when you are talking about components like workflow and coresessiontracking that are used to *develop* applications. I think you will find that good API+framework documentation will either provide a good deal of the info you are looking for, or, for products that are less in the way of infrastructure components and more in the way of end user products, provide the essential foundation upon which more end-user directed documentation can built. Good end user stuff is best written by someone who understands how the product works in detail, and the API/framework docs provide the foundation to acquire that knowledge (assuming the product author himself doesn't go straight on to providing the end user docs). How to get use the API to get paid is, I think, outside of the scope of the proposal grin. As for the proposal itself, the mechanism outlined sounds OK to me. I do also have an issue with the Interface scarecrow in that it does not seem to cover stuff that is not specifically a Class interface. I'm thinking of module level functions in particular, but also the documentation of Protocols mentioned by another poster is of concern. So Interfaces by themselves do not seem to be enough. Where they are enough, though, I should think there would be nothing stoping someone from writing one for an existing Zope internal interface, without modifying the code (ie: no automated verification in that case). Of course, I suspect that anyone doing that grubby job would tend to want to start tinkering with the code to clean up the interface...which come to think of it might be something that should go in the risks section wry grin. On the third hand, there's nothing to stop the community from generating some this is how I think it works documentation that people with inside knowledge can then help tune. I think some of this happened during the original Interfaces Wiki days, and it seems to have produced good results. I also want to say that I really like the fact that this proposal makes it clear that DC understands the long term value of good developer documentation grin. Oh, and one final thought. It seems to me that the Developer's Guide needs to evolve into a meta framework document: a place to learn how the whole system fits together, and how you use the various components to build working systems. I think it's a solid start in its current form. --RDM ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thu, 7 Jun 2001, Phillip J. Eby wrote:
I was thinking that certain types of objects would be committed by the
transaction manager before all others. In this case, the catalog (or a
special object in the catalog) would be committed first. It would
resolve all conflicts in the contained indices before they occur by
replaying the changes in the persisted queues from the transaction
history, then setting the _p_serial attributes to convince the storage
that the conflicts have already been resolved.
Hm. Sounds to me like what you actually want is for the transaction
manager to do this *after* everything else, rather than before. Thus, you
would catch any changes which occur *during* transaction commit - such as
commit-phase cataloging (as some folks do with ZPatterns currently).
Maybe I didn't explain this clearly enough. Let me write some quick
pseudocode:
class Catalog (Persistent):
finished_changes = None # Mapping: {path - (object, adding)}
unfinished_changes = None # Same as above
tid = None# Transaction ID
def catalogObject(self, ob, path):
unf = self.unfinished_changes
if unf is None: self.unfinished_changes = unf = {}
unf[path] = (ob, 1)
def uncatalogObject(self, path):
unf = self.unfinished_changes
if unf is None: self.unfinished_changes = unf = {}
unf[path] = (ob, 0)
def searchResults(self, ...):
self.finishChanges()
# ... Perform search ...
return results
def finishChanges(self):
unf = self.unfinished_changes
if unf is not None:
fin = self.finished_changes
if fin is None or self._p_serial != self.tid:
# Create finished_changes if not yet created
# and clear it if we're in a different transaction
# from the last time finished_changes was changed.
self.finished_changes = fin = {}
self.tid = self._p_serial
for path, (ob, adding) in unf.items():
if adding: self.addToIndexes(ob, path)
else: self.removeFromIndexes(path)
fin[path] = (ob, adding)
self.unfinished_changes = None
def __getstate__(self):
# Called during transaction commit.
self.finishChanges()
return Persistent.__getstate__(self)
def _p_priority(self):
# Causes this object to be added to the *top*
# of the list of objects to commit rather than the
# bottom. (Just an idea.)
return 1
def _p_resolveConflict(self, old, committed, newstate):
'''
Apply the changes in self.finished_changes to
committed and return the result.
'''
This does mean that _p_resolveConflict() might be called frequently, but
(potentially) it would never fail because of conflicts.
Now, this doesn't provide any automatic cataloging, which is what I think
you're suggesting. I think automatic reindexing, a good idea, is mostly
independent of bulletproofing and lazifying the catalog.
To achieve automatic indexing, I was thinking that a special attribute
would be added to cataloged objects. It would contain the OIDs of the
catalogs interested in the object. Transaction.register() would look for
this attribute and invoke catalogObject(). Of course, that wouldn't quite
work because the object might change again within the transaction and the
transaction manager wouldn't be told about the second and further changes.
But I'm sure there's a good way to compensate for this, such as making the
catalog scan for later changes before calling searchResults().
(cPersistence might need to assist.)
Shane
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Property Storage
At 09:29 PM 6/7/01 +0100, Chris Withers wrote: If I change one property on, say, a DTML Document, does that store a whole new copy of the document in the ZODB? It updates the object in the ZODB. Whether that causes a copy to be made, depends on the underlying storage. FileStorage makes copies, BerkeleyStorage (at least Ty's first implementation thereof) does not. Is so, then how about storing properties in their own mini-class that just subclassed Persistent, so each property got its own pickle jar? There are a lot of things you'd have to tinker with to get that behavior. You would probably be better off just using a storage that doesn't make copies, or using ZPatterns to store selected attributes in such a storage (such as a differently-backed ZODB, the filesystem, or an SQL or LDAP database). ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] ANN: ReplacingDateTime proposal
Feel free to review and comment the propsal to replace the current DateTime module of Zope by the mxDateTime module: http://dev.zope.org/Wikis/DevSite/Proposals/ReplacingDateTime Cheers, Andreas Digitial Creations ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
At 07:07 PM 6/7/01 -0400, Shane Hathaway wrote: Phillip J. Eby wrote: That is, in ZPatterns one can specify triggers such as: WHEN OBJECT DELETED, CHANGED CALL someCatalog.manage_uncatalog(self.absolute_url(1)) WHEN OBJECT ADDED, CHANGED CALL someCatalog.manage_catalog(self,self.absolute_url(1)) After I read this again I realized what you were saying. This capability of ZPatterns is very brittle, don't you think? Yep. That's why I've previously described ZPatterns as a hack. :) If the catalog is updated manually before the special ZPatterns object is added to the queue, the behavior is undefined AFAIK--either the later changes to the catalog will be ignored, will cause a conflict, or some objects will be written twice in the same transaction. True. But this behavior is avoidable through the use of subtransaction commits, in the event that someone has to have transactions which update a ZCatalog directly. Usually, when someone is using catalogs with ZPatterns, they use triggers to do all the updates and don't touch the catalog manually. Note that I'm not saying this still isn't a hack. But it's the best I could do without either fixing the multi-commit issue in ZODB, or with some kind of priority scheme. However, if we could specify transaction commit priorities, and the ZPatterns update came first, auto-indexing came second, and everything else followed, I think it would work. Or perhaps ZPatterns should be able to register things that occur *before* the two-phase commit protocol. Yep. One of the last two times I spoke with Jim in person (either the January DC visit or IPC 8, I forget which), he said something about it maybe being a good idea to have some kind of priority system like that. I'd love to see something like it exist, if it would make some of ZPatterns' hackery unnecessary. The implementation could consist of two subscription queues: ruleAgents and indexingAgents. ZCatalog would register in indexingAgents, and ZPatterns objects would register in one or the other, usually ruleAgents. (I can think of some circumstances where it would be nice to use the indexingAgents queue, but right now ZPatterns apps have to work around this by defining their rules in execution priority order.) Upon being told to perform a transaction or subtransaction commit, the transaction would notify all the ruleAgents, and then all the indexingAgents. Objects could still subscribe to either queue while this notifying is taking place. (So that triggered actions could cause indexish objects to register as indexingAgents, not to mention causing updated objects to fire additional triggers.) Once all agents in a queue are notified, that queue should be cleared so that notifications are on a per-subtransaction basis. Once both queues are cleared, normal transaction behavior goes forward. Hm. That's simpler than I thought it was going to be. Shoot, I can even see how to implement it as a runtime patch, that would've been simpler than all the shenanigans ZPatterns goes through to fake out the transaction machinery... and it's a better implementation. Ah well. At the time I wanted to avoid trying to convince Jim to add machinery to transactions just for ZPatterns, given that ZPatterns wasn't a particularly established product at the time. Let me know if you guys put something like this in, though, and I'll definitely look at reworking ZPatterns to use the mechanism. It could potentially cut a lot of code out and improve the robustness at the same time. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] A simple dtml-if question...
Hey thanks. That works. -Original Message- From: Chris Withers [mailto:[EMAIL PROTECTED]] Sent: Thursday, June 07, 2001 4:43 AM To: Jeff Nielsen / UgoFast Cc: Christian Theune; Zope-Dev@Zope. Org Subject: Re: [Zope-dev] A simple dtml-if question... Thanks Christian, but it didn't work. I went with the long way: dtml-if expr=LoginResults=='Pass' dtml-if expr=PATH_INFO=='/Maintain/Results' Valid response dtml-else False response /dtml-if dtml-else False response /dtml-if dtml-if expr=LoginResults=='Pass' and PATH_INFO=='/Maintain/Results' Valid Response dtml-else Fasle Response /dtml-if cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] mxDateTime-based Zope DateTime module
Stephan Richter wrote: Hello everyone, after I got finally frustrated enough with the current implementation of the Zope DateTime module, I decided to write a wrapper for mxDateTime. I checked the license, and the eGenix Public License is GPL compatible, so that we can use it for Zope now. What does being GPL compatible have to do with Zope ? Have I missed some licence changes/discussion ? - Hannu ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] mxDateTime-based Zope DateTime module
What does being GPL compatible have to do with Zope ? Have I missed some licence changes/discussion ? Originally mxDateTime was not completely open. This was earlier on a problem, if we wanted to use it for Zope. But now, we can do this. So I just wanted to make the point that mxDateTime is now free as in beer and freedom! ;-) Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Student Web2k - Web Design/Development Technical Project Management ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] mxDateTime-based Zope DateTime module
Actually, it's good that the license is BSDish, that means that it's legally possible to include mxDateTime inside a Zope distribution. Stephan Richter wrote: What does being GPL compatible have to do with Zope ? Have I missed some licence changes/discussion ? Originally mxDateTime was not completely open. This was earlier on a problem, if we wanted to use it for Zope. But now, we can do this. So I just wanted to make the point that mxDateTime is now free as in beer and freedom! ;-) Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Student Web2k - Web Design/Development Technical Project Management ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
