Hello,
Adam Groszer wrote:
I'd like to ask your opinion, your experiences about searching and
filtering in quite large object DBs.
We need to add search and filter functions to our current app, where
the user might be able to create quite _sophisticated_ filter criterias.
(The app is a pure Z3 app, subject is document management)
Currently we're looking at something based on catalog/indexes.
As I checked the most comfortable solution would be based on
hurry.query.
Some questions arose:
- Is it necessary/worth adding indexes on all attributes?
- How does the index perform on modification and retrieval?
The biggest problem is that this will be our first try, so we're
missing experiences and are a bit puzzled about the right solution.
Certain is that moving to RDB is not an option.
I think one of the main limitations of the current catalog (and
hurry.query) is efficient support for sorting and batching the query
results. The Zope 3 catalog returns all matching results, which can then
be sorted and batched. This will stop being scalable for large
collections. A relational database is able to do this internally, and is
potentially able to use optimizations there.
It would be very nice if someone could look into expanding hurry.query
and/or the catalog to support these cases. It would be interesting to
look at what Dieter Maurer has done with AdvancedQuery in Zope 2 in this
regard as well.
Regards,
Martijn
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
