David Pratt wrote:
Hi Jim. This approach is completely new to me. It is interesting and I'll keep an eye on your blog. How expensive is updating indexes with data modification. You are talking about sqlite, does this mean you figure the indexes would be a significant drag on the ZODB due their size or is this idea a performance consideration. Many thanks.

The expense is in recalculating the new factoradic for each stored index, which involves a lot of division and factorial operations. But this might be done in a smart way. Maybe instead of on-change, we could recalculate an index on first access to that index after change.

Sqlite would be mainly for performance, ATM. A lookup is less expensive than calculating a factoradic of a very large number.

Size may also be an issue if the index is stored in the zodb. An integer (long as it might be) is the most compact representation of the factoradic. I'm pretty sure that longs end up as base ten strings in a pickle, but that is still smaller than the pickle of the factoradic list representation. I still need to experiment more, but it's not out of the question that the indexes could end up as sqlite-backed annotations to the container/data source, particularly when sqlite may already be in the mix for memoizing factorial and/or factoradic calculations.

It may still turn out that this is all not worth the effort in practice.

-Jim Washington
Zope3-users mailing list

Reply via email to