Re: [Zope] "Pausing" ZCatalog and re-indexing later
- Original Message - From: <[EMAIL PROTECTED]> To: Sent: Friday, August 11, 2006 1:10 PM Subject: [Zope] "Pausing" ZCatalog and re-indexing later Hi :-) The system I am working on is using a ZCatalog with some indexes (over 10) for faster searching. Unfortunatelly there are some use cases, where I have to add really many new objects to ZODB. It works, but very slowly. I suspect that this is because when I add an object to Folder (with _setObject) it gets indexed - couple thousands of objects multiplied by over 10 indexes = long. Am I guessing right or am I completely lost? ;-) If this is the probable reason - is there some way I could disable indexing temporarily and after adding all of the objects make it re-index the new ones? I think that even a full re-index would work faster than it works now... I would be very grateful for any suggestions. Another approach is to build/update a new ZCatalog in another .fs file (you can do this offline, on another box, etc), and then just swap in the new .fs file. You can also create a mount point for a folder that contains nothing but the ZCatalog and then just swap in that (instead of swapping the entire Data.fs file). hth Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] "Pausing" ZCatalog and re-indexing later
--On 11. August 2006 17:10:29 + [EMAIL PROTECTED] wrote: Hi :-) The system I am working on is using a ZCatalog with some indexes (over 10) for faster searching. Unfortunatelly there are some use cases, where I have to add really many new objects to ZODB. It works, but very slowly. What kind of objects? I suspect that this is because when I add an object to Folder (with _setObject) it gets indexed - couple thousands of objects multiplied by over 10 indexes = long. Consider using CatalogQueue to defer indexing. Am I guessing right or am I completely lost? ;-) No idea, you might investigate your bottleneck(s) using ZopeProfiler. -aj pgpD2fQSIOfZn.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] "Pausing" ZCatalog and re-indexing later
Hi :-) The system I am working on is using a ZCatalog with some indexes (over 10) for faster searching. Unfortunatelly there are some use cases, where I have to add really many new objects to ZODB. It works, but very slowly. I suspect that this is because when I add an object to Folder (with _setObject) it gets indexed - couple thousands of objects multiplied by over 10 indexes = long. Am I guessing right or am I completely lost? ;-) If this is the probable reason - is there some way I could disable indexing temporarily and after adding all of the objects make it re-index the new ones? I think that even a full re-index would work faster than it works now... I would be very grateful for any suggestions. Best regards, Maciek ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
