Re: [Zope-dev] Sorting for zope.index
Chris McDonough wrote: > Dan Korostelev wrote: >> 2008/12/27 Chris McDonough : >>> Dan Korostelev wrote: I also made an implementation for the FieldIndex that may not be too optimal, but I'm currently most interested in clean and universal IIndexSort definition that any index could efficiently implement. >>> I've done this work too... but outside zope.index... please see (for >>> example): >>> >>> http://svn.repoze.org/repoze.catalog/trunk/repoze/catalog/indexes/field.py >>> >>> It's reasonably optimized. >> Thanks for the point. That's something I was going to write to add >> optimizations for FieldIndex sorting, now I only need to adapt your >> code and that's great! :-) > > There are many tests in there too. Note that the algorithms came mostly from > the Zope 2 Catalog code. The only things I'm less than sure about in there > is: > > - the computation of when to use n-best and lazy (there are constants in there > stolen from Zope 2). > > - The fact that when we need to sort in reverse order we can't be lazy. > I.e. in the branch that reads: > > # If the result set is not much larger than the number > # of documents in this index, or if we need to sort in > # reverse order, use a non-lazy sort. > > BTree values can't be iterated in reverse order, that's why we don't try > the lazy case here if it's a reverse sort. There may be a better way to > do this. > > >> However, the more important thing for now is the IIndexSort interface >> declaration. Is it okay and fits any possible sortable index? ;-) >> > > It looks like we came up with the same interface independently; I think > that's a > good sign. One difference is that the repoze.catalog implementation skips docids that aren't in the index rather than raising a KeyError. I think I like the KeyError better. > > - C > ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Sorting for zope.index
Dan Korostelev wrote: > 2008/12/27 Chris McDonough : >> Dan Korostelev wrote: >>> I also made an implementation for the FieldIndex that may not be too >>> optimal, but I'm currently most interested in clean and universal >>> IIndexSort definition that any index could efficiently implement. >> I've done this work too... but outside zope.index... please see (for >> example): >> >> http://svn.repoze.org/repoze.catalog/trunk/repoze/catalog/indexes/field.py >> >> It's reasonably optimized. > > Thanks for the point. That's something I was going to write to add > optimizations for FieldIndex sorting, now I only need to adapt your > code and that's great! :-) There are many tests in there too. Note that the algorithms came mostly from the Zope 2 Catalog code. The only things I'm less than sure about in there is: - the computation of when to use n-best and lazy (there are constants in there stolen from Zope 2). - The fact that when we need to sort in reverse order we can't be lazy. I.e. in the branch that reads: # If the result set is not much larger than the number # of documents in this index, or if we need to sort in # reverse order, use a non-lazy sort. BTree values can't be iterated in reverse order, that's why we don't try the lazy case here if it's a reverse sort. There may be a better way to do this. > However, the more important thing for now is the IIndexSort interface > declaration. Is it okay and fits any possible sortable index? ;-) > It looks like we came up with the same interface independently; I think that's a good sign. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Sorting for zope.index
2008/12/27 Chris McDonough : > Dan Korostelev wrote: >> I also made an implementation for the FieldIndex that may not be too >> optimal, but I'm currently most interested in clean and universal >> IIndexSort definition that any index could efficiently implement. > > I've done this work too... but outside zope.index... please see (for example): > > http://svn.repoze.org/repoze.catalog/trunk/repoze/catalog/indexes/field.py > > It's reasonably optimized. Thanks for the point. That's something I was going to write to add optimizations for FieldIndex sorting, now I only need to adapt your code and that's great! :-) However, the more important thing for now is the IIndexSort interface declaration. Is it okay and fits any possible sortable index? ;-) -- WBR, Dan Korostelev ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Sorting for zope.index
Dan Korostelev wrote: > Hello, fellow zope developers :) > > I made a "nadako-sorting" branch of zope.index to add sorting support > for zope indexes. I defined a new IIndexSort interface that specifies > a "sort" method that takes document ids and returns an iterable of > those ids, sorted by indexed value. It supports optional "limit" and > "reverse" arguments. It also raises KeyError if you pass a docid that > is not indexed by this index. > > Can someone review the interface definition and make any > suggestions/objections? > > I also made an implementation for the FieldIndex that may not be too > optimal, but I'm currently most interested in clean and universal > IIndexSort definition that any index could efficiently implement. > > If there's no objections, I'd like to merge it to trunk and start > adding sorting features for zope.app.catalog/z3c.indexer, as well as > zc.catalog indexes. > > Thanks in advance! > Hi Dan, I've done this work too... but outside zope.index... please see (for example): http://svn.repoze.org/repoze.catalog/trunk/repoze/catalog/indexes/field.py It's reasonably optimized. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Proposal: Eliminating Globals as an impor t façade
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I have just checked in work on a separate branch which removes all use of 'Globals' as an indirection for imports within the Zope2 core, restoring the module to its original purpose, which was to hold shared data (e.g., the opened database, etc.). My rationale was that the current usage leads to a confusion of the actual dependencies, as well as oddball anti-patterns based around import cycles, etc. I got started on the task while trying to track down whatever import-time side effect it is which causes the current functional test failures on the trunk. I didn't actually *find* that issue, of course ;(, but I did get back to the staus quo ante. I am more comfortable now that finding those side-effects will be possible. I do restore the importable names in the module for benefit of 3rd party code, but that happens only at Zope2 startup time (for now). I would prefer that folks change that code, but want to leave such products workable indefinitely: at the moment, for instance, the CMF produccts work at appserver startup, although their tests fail (because the BBB is added too late). The branch is here: svn+ssh://svn.zope.org/repos/main/Zope/branches/tseaver-no_globals_imports I would like to merge the branch soon, because it touches lots of files and is likely to hit conflicts if I let it go too long. I am still pondering whether to have the BBB imports done earlier: I really don't want to spew warnings about them, which is why I didn't use the "deferred import" mechanism initially. Thoughts? Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJVmF++gerLs4ltQ4RAlWXAKCi4QDmGsXNG0mVz09DUpCGF2Z9rQCgkcKc 8z9rLwRRaKljOGyXiQTbYdg= =0oTI -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Sorting for zope.index
Hello, fellow zope developers :) I made a "nadako-sorting" branch of zope.index to add sorting support for zope indexes. I defined a new IIndexSort interface that specifies a "sort" method that takes document ids and returns an iterable of those ids, sorted by indexed value. It supports optional "limit" and "reverse" arguments. It also raises KeyError if you pass a docid that is not indexed by this index. Can someone review the interface definition and make any suggestions/objections? I also made an implementation for the FieldIndex that may not be too optimal, but I'm currently most interested in clean and universal IIndexSort definition that any index could efficiently implement. If there's no objections, I'd like to merge it to trunk and start adding sorting features for zope.app.catalog/z3c.indexer, as well as zc.catalog indexes. Thanks in advance! -- WBR, Dan Korostelev ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 4 OK, 2 Failed
Summary of messages to the zope-tests list. Period Fri Dec 26 12:00:00 2008 UTC to Sat Dec 27 12:00:00 2008 UTC. There were 6 messages: 6 from Zope Tests. Test failures - Subject: FAILED (failures=1) : Zope-trunk Python-2.4.5 : Linux From: Zope Tests Date: Fri Dec 26 20:39:08 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-December/010727.html Subject: FAILED (failures=1) : Zope-trunk Python-2.5.2 : Linux From: Zope Tests Date: Fri Dec 26 20:40:38 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-December/010728.html Tests passed OK --- Subject: OK : Zope-2.8 Python-2.3.7 : Linux From: Zope Tests Date: Fri Dec 26 20:33:07 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-December/010723.html Subject: OK : Zope-2.9 Python-2.4.5 : Linux From: Zope Tests Date: Fri Dec 26 20:34:37 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-December/010724.html Subject: OK : Zope-2.10 Python-2.4.5 : Linux From: Zope Tests Date: Fri Dec 26 20:36:07 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-December/010725.html Subject: OK : Zope-2.11 Python-2.4.5 : Linux From: Zope Tests Date: Fri Dec 26 20:37:37 EST 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-December/010726.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
