seb bacon wrote:
Shane Hathaway wrote:

Chris Withers wrote:

Shane Hathaway wrote:

performance. There needs to be a way for applications that modify the database to tell Zope about the modification, so Zope can reset its caches.

But, IIRC, the last time this was discussed on a mailing list you had some cool ideas to sovle the problem, right?

Yes, but I want to hear other people's ideas first.  What do you think?

Isn't this a different problem for each kind of storage, e.g. MD5 hash for ext2, transaction ID for foo...? Or are you referring to a different aspect of the problem?
I'm thinking about "real-time" updates. When the underlying data changes, you'd like Zope to see the change immediately. If indefinite delays are OK, then AdaptableStorage already does enough: it raises a ConflictError if you try to write changes based on old data.

The idea I like the most for relational databases is to ask the RDBMS what the ID of the last transaction was. If Zope missed a transaction, it should flush all caches. This will work if the database is infrequently changed by external applications, or if Zope is accessed infrequently.

If external applications make a lot of changes, however, and Zope needs good performance at the same time, then both Zope and the external applications need to update a per-object transaction ID. Then, at the beginning of transactions, Zope would invalidate only the recently updated objects. Hmm, perhaps smarter RDBMSs could make it easy to keep transaction IDs updated using triggers. (This solution could also replace both ZEO and ZRS, BTW. ;-) )

On the filesystem, the problem seems much more difficult, since there are no transactions. You'd like the kernel to send Zope a message anytime someone modifies a file in a certain hierarchy, but that would require kernel hacking.

For that case, I'm thinking that requiring external apps to touch a special file somewhere might be the right thing. At the beginning of each transaction, if Zope sees a change to the file, it flushes its cache.

While reading the referenced thread on the subject, I found your description of the product design here:

Could this go in the docs/ directory of the product? The design, while very clean, doesn't lend itself to immediate understanding on a cursory view of the source...
I'm hoping to present a complete tutorial on AdaptableStorage at PyCon DC 2003. I'll integrate those notes. Thanks for pointing them out--I'd forgotten about them. The names are changed somewhat, but the basic design is the same.


Zope-Dev maillist - [EMAIL PROTECTED]
** No cross posts or HTML encoding! **
(Related lists - )

Reply via email to