[Zope-dev] RE: [ZC] 365/ 2 Reject metadata in Catalog is space inefficient
I specifically stated a FieldIndex, not TextIndex. It is a special case, but when your talking a huge Catalog this could be a lot of extra data. If you want to make it more general, perhaps a PluginIndex could advertise itself as metadata capable. When someone adds a metadata field it checks to see if an existing index supports returning of metadata for an RID. -Original Message- From: Collector: Zope Bugs, Features, and Patches ... [mailto:[EMAIL PROTECTED]] Sent: Thursday, 25 April 2002 6:32 PM To: djay; chrisw; klm; Brian; chrism; Caseman; _Rejected_ recipient Subject: [ZC] 365/ 2 Reject metadata in Catalog is space inefficient Issue #365 Update (Reject) metadata in Catalog is space inefficient Status Rejected, Catalog/bug low To followup, visit: http://collector.zope.org/Zope/365 == = Reject - Entry #2 by chrisw on Apr 25, 2002 4:31 am Status: Pending = Rejected You're making a huge assumption there. In many cases, I'm thinking TextIndex being the primary example here, it's not actually possible, let alone feasible, to retrieve the originally indexed information from the index itself. So your point is kidna invalid ;-) cheers, Chris = Request - Entry #1 by djay on Apr 25, 2002 3:52 am If I create a metadata to be stored so that it is availalbe in my results, and this metadata is the same as a FieldIndex, then the data is effectively being stored twice for no reason. The value of the metadata should just be retrieved from the index to save space == ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RE: [ZC] 365/ 2 Reject metadata in Catalog is space inefficient
On Fri, 26 Apr 2002 09:02:26 +1000, Jay, Dylan [EMAIL PROTECTED] wrote: I specifically stated a FieldIndex, not TextIndex. It is a special case, Yes. KeywordIndex too, although you lose ordering information. but when your talking a huge Catalog this could be a lot of extra data. Under your proposal data would be stored two times (original object, plus index) rather than three times (metadata too). A saving of 2:3, assuming the original object is no larger than the sum of its indexed properties. In most cases the original object is bigger, and it will probably add up to no more than a few percent of disk space. Are there any common scenarios where the margin would be bigger? Note that this scheme may not necessarily give runtime performance benefits. Loading the reverse index data may not be any faster than loading metadata. Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RE: [ZC] 365/ 2 Reject metadata in Catalog is space inefficient
Toby Dickenson wrote: Note that this scheme may not necessarily give runtime performance benefits. Loading the reverse index data may not be any faster than loading metadata. I'm betting in a lot of cases it'll be a damn site slower. MetaData is specifically designed to be real quick to load. For the small extra space usage (how much _does_ disk space, or RAM for that matter, cost nowadays?! ;-), I'm more than happy to take the speed win... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
