Ian Bicking wrote:
Chris McDonough wrote:
There's been a tension in the opencore stuff with the catalog, mostly that it's easy to setup and use for things, but it doesn't really work for things outside of the ZODB. Or, I guess theoretically you could catalog things not in the ZODB, but it's never happened.


IMO, mostly it's a matter of deciding what "index" and "catalog" means for searching. If you only need full-text search, some indexing systems are totally inappropriate. Likewise, if you only need "field" indexes that match just one particular value, it's sometimes vastly simpler just to come up with your own system based on, e.g. a relational table, than it is to try to use somebody else's "indexer" or "catalog".

They are similar in that they both need to get information about updates, and a way to reindex. Full text search can be a little lazier, as being 100% up-to-date isn't such a big issue.

I'd typically handle the "info about updates" bit by sending an event (which I suspect cabochon is all about).

That said, there's a real need for cross-system indexing of different kinds. There's text search indexes, but other kinds of more strict indexes also make sense. It would be very handy to have an index that wasn't tied to the ZODB, a database, or anything else. (It could be implemented using the ZODB or a database, of course.)

Xapian seems like a good choice too. It has its own persistence mechanism and reasonable Python bindings (although from experience maybe slightly memory-leaky).

Yes, once you get the documents into it. Actually, it's really a many-to-many notification system that's needed. We have one that needs documenting and review (http://www.openplans.org/projects/cabochon/project-home) but while it handles notifications and events (as do several other systems), that doesn't cover reindexing the site. And then you also need a set of useful endpoints, but those can grow over time.

Stuff that I'm not sure about above:

  "many-to-many notification system"

  "reindexing the site"

That said...

I guess what you're saying is that it's hard to send an event that makes it clear what needs to be done as a result of the event being sent.

In the case where you have a content node that holds data, that node would be attched to the event. it's pretty clear that you need to do.. that thing changed, and the indexing code needs to inspect it and reindex whatever indexes you've got sitting around that are willing to index data about those types of nodes.

In the case where you need to deal with the simultaneous case that some row in an RDB was changed, it's not quite as clear. What does it mean and who should do what with that event? It's all policy at that point. I'm not sure any framework helps. It's just block-and-tackle event reception and reaction, "do the needful", isn't it?

- C

_______________________________________________
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Reply via email to