Re: [Zope-dev] Bulletproof ZCatalog proposal
At 04:58 PM 6/8/01 -0400, Shane Hathaway wrote: On Thursday 07 June 2001 21:28, Phillip J. Eby wrote: Upon being told to perform a transaction or subtransaction commit, the transaction would notify all the ruleAgents, and then all the indexingAgents. Objects could still subscribe to either queue while this notifying is taking place. (So that triggered actions could cause indexish objects to register as indexingAgents, not to mention causing updated objects to fire additional triggers.) Once all agents in a queue are notified, that queue should be cleared so that notifications are on a per-subtransaction basis. Once both queues are cleared, normal transaction behavior goes forward. Would you say this would occur before tpc_begin() messages or just after? Before. I'm saying this would take place immediately at the start of the existing Transaction.commit() method. Hm. That's simpler than I thought it was going to be. Shoot, I can even see how to implement it as a runtime patch, that would've been simpler than all the shenanigans ZPatterns goes through to fake out the transaction machinery... and it's a better implementation. Ah well. At the time I wanted to avoid trying to convince Jim to add machinery to transactions just for ZPatterns, given that ZPatterns wasn't a particularly established product at the time. Let me know if you guys put something like this in, though, and I'll definitely look at reworking ZPatterns to use the mechanism. It could potentially cut a lot of code out and improve the robustness at the same time. I don't foresee us adding this capability right away since we need to understand it better and it only applies to one working product and a hypothetical product. I'd suggest you go ahead with the runtime patch. I think I'll wait until ZPatterns works with Zope 2.4, unless it becomes necessary otherwise. Replacing what I've got now is a pretty significant undertaking, and risk-prone to boot, so I don't want to delay the finishing of ZPatterns' support for Zope 2.3.x. Basically, the runtime patch would be to replace ZODB.Transaction.Transaction with a subclass that implemented the notification queues in the commit() method. There'd be a few other little extensions needed, like clearing the queues on any abort operation, and adding registerIndex() and registerRule() or some such methods. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thu, 31 May 2001 10:36:55 -0400 (EDT), Shane Hathaway [EMAIL PROTECTED] wrote: On Thu, 31 May 2001, Toby Dickenson wrote: On Thu, 31 May 2001 10:03:31 -0400 (EDT), Shane Hathaway [EMAIL PROTECTED] wrote: Right now ZCatalog randomly generates ConflictErrors even if there are no conflicts in the data being indexed. It's quite rare, however, and there's machinery that retries the entire transaction up to three times when it happens. If the current mechanism is reliable, and efficient (given that conflict/retries are rare) then I dont understand the motivation The system becomes less reliable as server load increases. Large catalog updates (where every object is reindexed) also generate a lot of conflicts. Is that last bit true? I thought 'Update Catalog' created *new* indexes. There might be a conflict on the root catalog opbject, but inside the index BTrees. Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
My last post didnt make much sense. This time ill include all the words in my sentence Large catalog updates (where every object is reindexed) also generate a lot of conflicts. Is that last bit true? I thought 'Update Catalog' created *new* indexes. There might be a conflict on the root catalog object, but not on the index BTrees. Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thursday 07 June 2001 07:43, Toby Dickenson wrote: Large catalog updates (where every object is reindexed) also generate a lot of conflicts. Is that last bit true? I thought 'Update Catalog' created *new* indexes. There might be a conflict on the root catalog object, but not on the index BTrees. I should say large catalog updates tend to generate a conflict. It really doesn't matter how many conflicts there are. Within a single transaction, 1 conflict is as bad as 100. One thing I didn't make clear in the proposal is that I'm interested in repurposing ZCatalog as a general ZODB indexing mechanism and essentially moving it down from the application layer to the database layer. Catalogs would be updated automatically (in a lazy fashion to avoid performance penalties). There would potentially be a large number of catalogs, though, so the number of conflicts would increase significantly. But I think I have a solution for all of the issues in conflict resolution. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
Shane Hathaway wrote: But I think I have a solution for all of the issues in conflict resolution. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Please do this! :-) I'm particularly interested in ZCatalog becoming a standard ZODB feature, and I know there are otehrs on ZODB-dev who are dying for this right now... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thu, 7 Jun 2001, Shane Hathaway wrote: It really doesn't matter how many conflicts there are. Within a single transaction, 1 conflict is as bad as 100. Why? But I think I have a solution for all of the issues in conflict resolution. Whee! If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Please do this! ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
Me too! Me too! John On Thursday 07 June 2001 08:49, Chris Withers wrote: Shane Hathaway wrote: But I think I have a solution for all of the issues in conflict resolution. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Please do this! :-) I'm particularly interested in ZCatalog becoming a standard ZODB feature, and I know there are otehrs on ZODB-dev who are dying for this right now... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) -- . . . . . . . . . . . . . . . . . . . . . . . . John D. Heintz | Senior Engineer 1016 La Posada Dr. | Suite 240 | Austin TX 78752 T 512.633.1198 | [EMAIL PROTECTED] w w w . d a t a c h a n n e l . c o m ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
--On Thursday, June 07, 2001 09:34:54 AM -0400 Shane Hathaway [EMAIL PROTECTED] wrote: But I think I have a solution for all of the issues in conflict resolution. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Please do this! It is the sort of thing that can save us and others a lot of time by keeping use of Zope simple and stable. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thursday 07 June 2001 11:51, Erik Enge wrote: On Thu, 7 Jun 2001, Shane Hathaway wrote: It really doesn't matter how many conflicts there are. Within a single transaction, 1 conflict is as bad as 100. Why? Because in Zope it means the whole request is processed again (which can lead to strange behavior if non-transactional data is changed by the request). In other apps it might mean messy exception handling. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Please do this! I should have stated that it also needs to be a priority. Within DC we've been using better project management and it's getting harder to do projects just because we might think they'll be useful for someone. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thursday 07 June 2001 12:17, Phillip J. Eby wrote: At 09:34 AM 6/7/01 -0400, Shane Hathaway wrote: One thing I didn't make clear in the proposal is that I'm interested in repurposing ZCatalog as a general ZODB indexing mechanism and essentially moving it down from the application layer to the database layer. Catalogs would be updated automatically (in a lazy fashion to avoid performance penalties). There would potentially be a large number of catalogs, though, so the number of conflicts would increase significantly. But I think I have a solution for all of the issues in conflict resolution. If you, or anyone else, is also interested in this, please show support (if only by saying please do this!) :-) I can't work on it unless I have a reason to. Sounds cool. At one time Ty and I thought we might do something like this by adding to a linked list of pending updates which the catalog would use to alter its search results, until the list got too long, at which point it would post the actual updates. That sounds like what I have in mind. The only catch was that this would still produce conflicts at the head end of the linked list. :( Of course, that was in the days before ZODB conflict resolution. Nowadays, you could probably implement it as a simple sequence object with the conflict resolution method implemented. But then there'd be the question of how to resolve conflicts if more than one thread or ZEO client decided to apply the queued changes... a 100 conflicts vs 1 situation. Ugh. I was thinking the queue would only last the duration of a transaction and that the queue would be thread-specific. Anyway, I'd be really interested in seeing a solution for all the issues in conflict resolution. I was thinking that certain types of objects would be committed by the transaction manager before all others. In this case, the catalog (or a special object in the catalog) would be committed first. It would resolve all conflicts in the contained indices before they occur by replaying the changes in the persisted queues from the transaction history, then setting the _p_serial attributes to convince the storage that the conflicts have already been resolved. Will it help with arguments, too? How about world peace? ;) Okay, maybe we haven't solved that. :-) Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
At 02:07 PM 6/7/01 -0400, Shane Hathaway wrote: On Thursday 07 June 2001 12:17, Phillip J. Eby wrote: The only catch was that this would still produce conflicts at the head end of the linked list. :( Of course, that was in the days before ZODB conflict resolution. Nowadays, you could probably implement it as a simple sequence object with the conflict resolution method implemented. But then there'd be the question of how to resolve conflicts if more than one thread or ZEO client decided to apply the queued changes... a 100 conflicts vs 1 situation. Ugh. I was thinking the queue would only last the duration of a transaction and that the queue would be thread-specific. That's not hard to do - you could just toss an object into the transaction queue to do it during transaction commit, similar to what ZPatterns does in a more general way. (Note that ZPatterns users can already get the benefits of deferring catalog changes until the end of a (sub)transaction, by using triggers to implement automatic cataloging). Anyway, I'd be really interested in seeing a solution for all the issues in conflict resolution. I was thinking that certain types of objects would be committed by the transaction manager before all others. In this case, the catalog (or a special object in the catalog) would be committed first. It would resolve all conflicts in the contained indices before they occur by replaying the changes in the persisted queues from the transaction history, then setting the _p_serial attributes to convince the storage that the conflicts have already been resolved. Hm. Sounds to me like what you actually want is for the transaction manager to do this *after* everything else, rather than before. Thus, you would catch any changes which occur *during* transaction commit - such as commit-phase cataloging (as some folks do with ZPatterns currently). That is, in ZPatterns one can specify triggers such as: WHEN OBJECT DELETED, CHANGED CALL someCatalog.manage_uncatalog(self.absolute_url(1)) WHEN OBJECT ADDED, CHANGED CALL someCatalog.manage_catalog(self,self.absolute_url(1)) Which will be executed during transaction commit if the conditions apply. If you have the catalog process its queue *before* other objects commit, it will not have received these calls yet. This will end up making the catalog try to commit itself a second time, which will fail with a conflict error - 100% of the time. (And retry won't help, because the transaction is conflicting with itself.) Of course, this issue could be fixed by minor adjustment to the ZODB API and implementation such that it is permissible to store() an object more than once during a transaction's commit phase without creating a ConflictError. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
Phillip J. Eby wrote: That is, in ZPatterns one can specify triggers such as: WHEN OBJECT DELETED, CHANGED CALL someCatalog.manage_uncatalog(self.absolute_url(1)) WHEN OBJECT ADDED, CHANGED CALL someCatalog.manage_catalog(self,self.absolute_url(1)) After I read this again I realized what you were saying. This capability of ZPatterns is very brittle, don't you think? If the catalog is updated manually before the special ZPatterns object is added to the queue, the behavior is undefined AFAIK--either the later changes to the catalog will be ignored, will cause a conflict, or some objects will be written twice in the same transaction. However, if we could specify transaction commit priorities, and the ZPatterns update came first, auto-indexing came second, and everything else followed, I think it would work. Or perhaps ZPatterns should be able to register things that occur *before* the two-phase commit protocol. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thu, 7 Jun 2001, Phillip J. Eby wrote:
I was thinking that certain types of objects would be committed by the
transaction manager before all others. In this case, the catalog (or a
special object in the catalog) would be committed first. It would
resolve all conflicts in the contained indices before they occur by
replaying the changes in the persisted queues from the transaction
history, then setting the _p_serial attributes to convince the storage
that the conflicts have already been resolved.
Hm. Sounds to me like what you actually want is for the transaction
manager to do this *after* everything else, rather than before. Thus, you
would catch any changes which occur *during* transaction commit - such as
commit-phase cataloging (as some folks do with ZPatterns currently).
Maybe I didn't explain this clearly enough. Let me write some quick
pseudocode:
class Catalog (Persistent):
finished_changes = None # Mapping: {path - (object, adding)}
unfinished_changes = None # Same as above
tid = None# Transaction ID
def catalogObject(self, ob, path):
unf = self.unfinished_changes
if unf is None: self.unfinished_changes = unf = {}
unf[path] = (ob, 1)
def uncatalogObject(self, path):
unf = self.unfinished_changes
if unf is None: self.unfinished_changes = unf = {}
unf[path] = (ob, 0)
def searchResults(self, ...):
self.finishChanges()
# ... Perform search ...
return results
def finishChanges(self):
unf = self.unfinished_changes
if unf is not None:
fin = self.finished_changes
if fin is None or self._p_serial != self.tid:
# Create finished_changes if not yet created
# and clear it if we're in a different transaction
# from the last time finished_changes was changed.
self.finished_changes = fin = {}
self.tid = self._p_serial
for path, (ob, adding) in unf.items():
if adding: self.addToIndexes(ob, path)
else: self.removeFromIndexes(path)
fin[path] = (ob, adding)
self.unfinished_changes = None
def __getstate__(self):
# Called during transaction commit.
self.finishChanges()
return Persistent.__getstate__(self)
def _p_priority(self):
# Causes this object to be added to the *top*
# of the list of objects to commit rather than the
# bottom. (Just an idea.)
return 1
def _p_resolveConflict(self, old, committed, newstate):
'''
Apply the changes in self.finished_changes to
committed and return the result.
'''
This does mean that _p_resolveConflict() might be called frequently, but
(potentially) it would never fail because of conflicts.
Now, this doesn't provide any automatic cataloging, which is what I think
you're suggesting. I think automatic reindexing, a good idea, is mostly
independent of bulletproofing and lazifying the catalog.
To achieve automatic indexing, I was thinking that a special attribute
would be added to cataloged objects. It would contain the OIDs of the
catalogs interested in the object. Transaction.register() would look for
this attribute and invoke catalogObject(). Of course, that wouldn't quite
work because the object might change again within the transaction and the
transaction manager wouldn't be told about the second and further changes.
But I'm sure there's a good way to compensate for this, such as making the
catalog scan for later changes before calling searchResults().
(cPersistence might need to assist.)
Shane
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
At 07:07 PM 6/7/01 -0400, Shane Hathaway wrote: Phillip J. Eby wrote: That is, in ZPatterns one can specify triggers such as: WHEN OBJECT DELETED, CHANGED CALL someCatalog.manage_uncatalog(self.absolute_url(1)) WHEN OBJECT ADDED, CHANGED CALL someCatalog.manage_catalog(self,self.absolute_url(1)) After I read this again I realized what you were saying. This capability of ZPatterns is very brittle, don't you think? Yep. That's why I've previously described ZPatterns as a hack. :) If the catalog is updated manually before the special ZPatterns object is added to the queue, the behavior is undefined AFAIK--either the later changes to the catalog will be ignored, will cause a conflict, or some objects will be written twice in the same transaction. True. But this behavior is avoidable through the use of subtransaction commits, in the event that someone has to have transactions which update a ZCatalog directly. Usually, when someone is using catalogs with ZPatterns, they use triggers to do all the updates and don't touch the catalog manually. Note that I'm not saying this still isn't a hack. But it's the best I could do without either fixing the multi-commit issue in ZODB, or with some kind of priority scheme. However, if we could specify transaction commit priorities, and the ZPatterns update came first, auto-indexing came second, and everything else followed, I think it would work. Or perhaps ZPatterns should be able to register things that occur *before* the two-phase commit protocol. Yep. One of the last two times I spoke with Jim in person (either the January DC visit or IPC 8, I forget which), he said something about it maybe being a good idea to have some kind of priority system like that. I'd love to see something like it exist, if it would make some of ZPatterns' hackery unnecessary. The implementation could consist of two subscription queues: ruleAgents and indexingAgents. ZCatalog would register in indexingAgents, and ZPatterns objects would register in one or the other, usually ruleAgents. (I can think of some circumstances where it would be nice to use the indexingAgents queue, but right now ZPatterns apps have to work around this by defining their rules in execution priority order.) Upon being told to perform a transaction or subtransaction commit, the transaction would notify all the ruleAgents, and then all the indexingAgents. Objects could still subscribe to either queue while this notifying is taking place. (So that triggered actions could cause indexish objects to register as indexingAgents, not to mention causing updated objects to fire additional triggers.) Once all agents in a queue are notified, that queue should be cleared so that notifications are on a per-subtransaction basis. Once both queues are cleared, normal transaction behavior goes forward. Hm. That's simpler than I thought it was going to be. Shoot, I can even see how to implement it as a runtime patch, that would've been simpler than all the shenanigans ZPatterns goes through to fake out the transaction machinery... and it's a better implementation. Ah well. At the time I wanted to avoid trying to convince Jim to add machinery to transactions just for ZPatterns, given that ZPatterns wasn't a particularly established product at the time. Let me know if you guys put something like this in, though, and I'll definitely look at reworking ZPatterns to use the mechanism. It could potentially cut a lot of code out and improve the robustness at the same time. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Wed, 30 May 2001, Shane Hathaway wrote: I have written a proposal. So, if I understand correctly, when this is implemented, will the ZCatalog always resolve conflicts and never raise ConflictErrors? If there are no ConflictErrors in the main object system? I think I'm missing the point here. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thu, 31 May 2001 10:03:31 -0400 (EDT), Shane Hathaway [EMAIL PROTECTED] wrote: On Thu, 31 May 2001, Erik Enge wrote: On Wed, 30 May 2001, Shane Hathaway wrote: I have written a proposal. So, if I understand correctly, when this is implemented, will the ZCatalog always resolve conflicts and never raise ConflictErrors? If there are no ConflictErrors in the main object system? I think I'm missing the point here. How did you miss the point? That's exactly the point! :-) Right now ZCatalog randomly generates ConflictErrors even if there are no conflicts in the data being indexed. It's quite rare, however, and there's machinery that retries the entire transaction up to three times when it happens. If the current mechanism is reliable, and efficient (given that conflict/retries are rare) then I dont understand the motivation But I want it to be more reliable, especially so that ZCatalog can be used in ZODB applications outside Zope. Is it because other non-Zope applications might not be so proficient at performing retries? Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thu, 31 May 2001, Toby Dickenson wrote: On Thu, 31 May 2001 10:03:31 -0400 (EDT), Shane Hathaway [EMAIL PROTECTED] wrote: Right now ZCatalog randomly generates ConflictErrors even if there are no conflicts in the data being indexed. It's quite rare, however, and there's machinery that retries the entire transaction up to three times when it happens. If the current mechanism is reliable, and efficient (given that conflict/retries are rare) then I dont understand the motivation The system becomes less reliable as server load increases. Large catalog updates (where every object is reindexed) also generate a lot of conflicts. But I want it to be more reliable, especially so that ZCatalog can be used in ZODB applications outside Zope. Is it because other non-Zope applications might not be so proficient at performing retries? Yes. Long-running transactions are especially problematic. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thu, 31 May 2001, Shane Hathaway wrote: How did you miss the point? That's exactly the point! :-) Oh :) Right now ZCatalog randomly generates ConflictErrors even if there are no conflicts in the data being indexed. This is why I think I've missed a/the point: wouldn't it be better to figure out why ZCatalog randomly generates these ConflictErrors instead of making them disappear using other means? ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Bulletproof ZCatalog proposal
On Thu, 31 May 2001, Erik Enge wrote: On Thu, 31 May 2001, Shane Hathaway wrote: How did you miss the point? That's exactly the point! :-) Oh :) Right now ZCatalog randomly generates ConflictErrors even if there are no conflicts in the data being indexed. This is why I think I've missed a/the point: wouldn't it be better to figure out why ZCatalog randomly generates these ConflictErrors instead of making them disappear using other means? We know why they happen. They are only *apparently* random--they actually make perfect sense. (Maybe I didn't make that clear.) If you concurrently modify objects that happen to be placed near enough each other in one of the indexes then a conflict will occur. We have a partial solution in place which tries to merge the buckets that make up the index, but this solution can't solve the whole problem. Think about what SQL databases do. You can add entries to a table simultaneously from different connections. As long as the contents of the entries don't conflict, there will not be any kind of conflict in the indexes either. ZCatalog is to ZODB what indexes are to SQL databases. I'd like ZCatalog to have the same level of reliability. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Bulletproof ZCatalog proposal
I have written a proposal. http://dev.zope.org/Wikis/DevSite/Proposals/ArmoredCatalog Could it be that we're involved in yet another Battle of Fredericksburg and we're trying to save innocent ZCatalogs from a ZODB conflict? Nawww... ;-) Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
