Hi Geoff,
I'm working on the exact same thing as you are. The specific backend in my case is an XML database that is updated by a different subsystem than cocoon and I'd like to be able to let the the cocoon caching framework know about the changes going on there.
Ah, this is really good - because I'm doing it for fun, and have no guinea pigs! And of course, I'd love help and at least a second set of eyes. I've had this on my brain though for a year or two since implementing a similar facility for Cocoon at a past job.
> I am working on a process for external cache invalidation for > use in Cocoon. To get right to the point, what would the > opinion be of exposing something like SourceValdity[] > getValidities() on AbstractAggregatedValidity? The purpose > would be not to alter the validities, but to examine them. > > If validities are externally invalidated, how do you accomplish this?
Yes, this seems to be a blind spot in the caching component, it delegates the responsibility of handing out a validity strategy to the individual components that make up a pipeline.
Well, what I'm envisioning is a new ExternalValidity extends SourceValidity. So the individual components still have to pick an Impl of this and return it from getValidity().
> In the Cocoon cache (which uses Excalibur store) a Pipeline > result and key are placed in the store. The result holds a > SourceValidity array and exposes getValidities() so that they > can be tested upon retrieval from the cache before use.
... leaving pro-active cache invalidation difficult to implement.
> I don't think an > expiration scheme is a good solution.
Me neither.
> > What I'd like to do instead is have each validity
Just for clarity, you are talking about excalibur.source.SourceValidity? I assume this validity object also has a way of identifying the Source it is a validity object of, like its uri string or something, in order for events referring to the same source to be matched.
Yes, as that is what is used by Cocoon (even though some examples and docs still refer to the old CacheValidity from Cocoon before this code was donated to and then migrated to Excalibur sourceresolve.
> register > it's dependencies and the key under which it is stored with > the event processor upon going into the cache. When an event > is received, the processor would look up those keys and > remove the items from the cache (it could just mark them > invalid, but why bother to keep them around at all?) When > the validity is created, it doesn't know what key it is being > stored under, and probably shouldn't care. So, I think the > place to handle this registration is when the item goes into > the cache.
I agree.
> In cocoon, there is a wrapper around the store implementation > that would be a good place to handle this - I can check for > these special validities and handle them when I have the key > right there, and can unregister when the item is removed, > etc. The hitch I've run into is with aggregated validities > which is where the proposal above comes from.
I think it is very natural for an aggregated validity to expose its composite validities. I'm all for it!
> > Very open to other suggestions and input and sorry for the > long post, Geoff Howard
I think your approach is excellent. My current implementation depends on the individual cacheable components to be aware of a validity cache that receives events from the backend. The obvious problem with this is that it is not very general and requires each individual component to be aware of that validity cache.
The implementation I envision involves declaring external dependencies - probably as key-value pairs. So, for a database it might be "tablename" and "primarykeyvalue" - similar for EJB. For an xml db backend do you think that still works?
The next step for me would then be to extend the Cocoon Cache (which is a wrapper around the excalibur store) to examine each SourceValidity to see if it is an instanceof ExternalValidity, and then call something like getDependencies(). These then need to get stored so that they can be the key to lookup the key under which the cached response is stored in the cache/store. Kind of a reverse index.
Shoot, that doesn't make much sense as written does it, but I need to go. Here's some links to the recent discussions that led to this, but some of my comments there are misguided I now see.
http://marc.theaimsgroup.com/?t=105369753900004&r=1&w=2 http://marc.theaimsgroup.com/?t=105403813400002&r=1&w=2 http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=105409228125661&w=2
As you'll see there, my intention is to provide a jms listener to signal
this external dependency aware cache but one could implement other listening
components to do the same thing, and I'll keep that in mind to facilitate that.
Unfortunately, publishing the events is up to the individual system, but code or examples to facilitate that would be awesome.
Have you committed any of your work to cocoon yet? I would be very interested in seeing the code. If there is anything I can do I would like to help out.
Not yet, but I will. You can see the quick mock up of the basic jms listener posted on that last message but that's it so far. In the end, this other stuff isn't many lines of code - just a lot of thinking and digging to find the right way to do it - or as close to "right" as I can get.
By the way, there was a two email exchange accidentally off list between me
and carsten after his last post - just basically saying ok, I'll submit a patch
for the proposed method and get the cocoon side ready for scratchpad/block.
Geoff
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
