Below is the larger picture I envision for a new kind of cache invaliditation that I've needed in the past and comes up in requests from people using EJB or database driven data that is cacheable. I'd love feedback from anyone who's interested. I've BCC'd a few people who I've talked with about this in the past.
---------------------------------------------------------------------------- I've committed some little things into the cocoon scratchpad - part of some experimental work for external cache invalidation, which as I've thought about it is really "event" based cache invalidation. Why? Because the events need not be external to Cocoon, and when you think about it, all cache validities are "external" in that the Cache validity object needs to consult some external reference - the filesystem, the system time, etc. But event based invalidation has some differences in the way they need to be handled. The other validities can reasonably be expected to check with their external resources (time, filesystem, etc) when retrieved from cache and isValid() is called. But with events, the subsystem which received the events would need to store them all up waiting for the call to isValid() which depending on other factors might never come. It seems to me more fitting with the transient nature of events to act on them when they arrive and then discard them. That leaves only several choices - make the event know about what cache keys it needs to remove. This is the only solution currently available and it has in practice meant hard coding the event-key relationship somewhere and manually maintaining it. Not good IMHO. - search through every cached item on the receipt of an event to see if it is now invalid in light of the current event. Also not good. - Extend the cache to provide support for the cache-event concept. This is the tack I'm taking. Essentially, this solution involves the CacheImpl keeping track of mapping the Event's declared in an EventValidity to the key under which that response is stored in the cache. The "glue" that is missing is the org.apache.cocoon.caching.impl.CacheImpl extension, because it won't compile without the change I made to sourceresolve, which is not yet in cocoon's cvs. For some odd reason I'm having a hard time building the sourceresolve package using its build script. It's also not "done" as noted below - but I'd love others to be able to work on it. Here are the issues the "event aware" CacheImpl would need to take care of: - during store() it gets the SourceValidity[] from the CachedResponse and looks for instanceof EventValidity (recursively for AggregatedValidity). - if found, it calls EventValidity.getEvent() and stores the key-event mapping. - expose a removeByEvent(Event e) method that can be called by the specific event-handling component. This could be a jms listener (as I've orginally envisioned it) or an http/soap based system (as in the ESI patch that was in bugzilla) or even a cocoon Action or Flow, or some combination of all of the above. - When the key is ejected from the cache for other reasons (another pipeline component signalled invalid for example) I think it's necessary to at that moment remove the event-key mapping entry. This introduces a complication in the data structure used to store these mappings as I mention below. I also haven't looked into the effect of the store janitor - if it acts directly on the Store without going through the CacheImpl wrapper, that introduces a wrinkle. Most of the above is accounted for - except for the data structure to store the event-key mappings. As discussed above, it needs to: - allow duplicate keys (each event may uncache multiple pipelines, and each pipeline might be uncached by any of multiple events). So it needs a Bag. - allow lookup of mappings based on either event or key. Jakarta Commons Collections has a DoubleOrderedMap, but not a DoubleOrderedBag. Bummer. - be persistent across shutdown and startup, and needs to recover gracefully when the cache is missing (like when it's been manually deleted) - be efficient I have made an assumption so far that I'd like tested by some sharp minds. When a removeByEvent() is received, the CacheImpl would do something like PipelineCacheKey[] getByEvent() on its datastructure. This would rely on hashCode() and equals() of Event() to locate relevant events. I think this works well for true "equals" type of information: like "table_name" and "primary_key" -- if they are both equal, the event has happened. But there may be some where a "greater than" or "less than" or worse yet, a kind of wild card lookup might need to be supported. Can that be accomodated by a Collections sort of implementation, or does something more flexible need to be invented? As it stands, you might implement hashCode() in a way that will cause intentional collisions and rely on equals() to sort things out. Is that crazy? Geoff > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > Sent: Friday, June 20, 2003 11:36 PM > To: [EMAIL PROTECTED] > Subject: cvs commit: > cocoon-2.1/src/scratchpad/src/org/apache/cocoon/caching/validity > NameValueEvent.java Event.java EventValidity.java NamedEvent.java > > > ghoward 2003/06/20 20:36:15 > > Added: src/scratchpad/src/org/apache/cocoon/caching/validity > NameValueEvent.java Event.java EventValidity.java > NamedEvent.java > Log: > Experiment with external cache invalidation. An EventAwareCacheImpl > can't be committed yet because it relies on the latest sourceresolve > cvs. > > Revision Changes Path > 1.1 > cocoon-2.1/src/scratchpad/src/org/apache/cocoon/caching/validity/N > ameValueEvent.java > > Index: NameValueEvent.java > =================================================================== > /* > > > ================================================================== > ========== > The Apache Software License, Version 1.1 > > ================================================================== > ========== > > Copyright (C) 1999-2003 The Apache Software Foundation. All > rights reserved. > > Redistribution and use in source and binary forms, with or > without modifica- > tion, are permitted provided that the following conditions are met: > > 1. Redistributions of source code must retain the above > copyright notice, > this list of conditions and the following disclaimer. > > 2. Redistributions in binary form must reproduce the above > copyright notice, > this list of conditions and the following disclaimer in the > documentation > and/or other materials provided with the distribution. > > 3. The end-user documentation included with the > redistribution, if any, must > include the following acknowledgment: "This product > includes software > developed by the Apache Software Foundation > (http://www.apache.org/)." > Alternately, this acknowledgment may appear in the > software itself, if > and wherever such third-party acknowledgments normally appear. > > 4. The names "Apache Cocoon" and "Apache Software Foundation" > must not be > used to endorse or promote products derived from this > software without > prior written permission. For written permission, please contact > [EMAIL PROTECTED] > > 5. Products derived from this software may not be called > "Apache", nor may > "Apache" appear in their name, without prior written > permission of the > Apache Software Foundation. > > THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR > IMPLIED WARRANTIES, > INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF > MERCHANTABILITY AND > FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO > EVENT SHALL THE > APACHE SOFTWARE FOUNDATION OR ITS CONTRIBUTORS BE LIABLE > FOR ANY DIRECT, > INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > DAMAGES (INCLU- > DING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR > SERVICES; LOSS > OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER > CAUSED AND ON > ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > LIABILITY, OR TORT > (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > OF THE USE OF > THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > > */ > package org.apache.cocoon.caching.validity; > > /** > * Very experimental start at external cache invalidation. > * Warning - API very unstable. Do not use! In fact, if this > * becomes useful, it would probably move to Excalibur SourceResolve. > * > * An external uncache event that consists of a name/value pair. > * An example might be "table_name", "primary_key" > * > * @author Geoff Howard ([EMAIL PROTECTED]) > * @version $CVS$ > */ > public class NameValueEvent extends Event { > > private String m_name; > private String m_value; > > public NameValueEvent(String name, String value) { > m_name = name; > m_value = value; > } > > private String getName() { > return m_name; > } > > private String getValue() { > return m_value; > } > > public boolean equals(Event e) { > if (e instanceof NameValueEvent) { > NameValueEvent nve = (NameValueEvent)e; > return ( m_name.equals(nve.getName()) && > m_value.equals(nve.getValue()) ); > } > return false; > } > > } > > > > 1.1 > cocoon-2.1/src/scratchpad/src/org/apache/cocoon/caching/validity/E > vent.java > > Index: Event.java > =================================================================== > /* > > > ================================================================== > ========== > The Apache Software License, Version 1.1 > > ================================================================== > ========== > > Copyright (C) 1999-2003 The Apache Software Foundation. All > rights reserved. > > Redistribution and use in source and binary forms, with or > without modifica- > tion, are permitted provided that the following conditions are met: > > 1. Redistributions of source code must retain the above > copyright notice, > this list of conditions and the following disclaimer. > > 2. Redistributions in binary form must reproduce the above > copyright notice, > this list of conditions and the following disclaimer in the > documentation > and/or other materials provided with the distribution. > > 3. The end-user documentation included with the > redistribution, if any, must > include the following acknowledgment: "This product > includes software > developed by the Apache Software Foundation > (http://www.apache.org/)." > Alternately, this acknowledgment may appear in the > software itself, if > and wherever such third-party acknowledgments normally appear. > > 4. The names "Apache Cocoon" and "Apache Software Foundation" > must not be > used to endorse or promote products derived from this > software without > prior written permission. For written permission, please contact > [EMAIL PROTECTED] > > 5. Products derived from this software may not be called > "Apache", nor may > "Apache" appear in their name, without prior written > permission of the > Apache Software Foundation. > > THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR > IMPLIED WARRANTIES, > INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF > MERCHANTABILITY AND > FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO > EVENT SHALL THE > APACHE SOFTWARE FOUNDATION OR ITS CONTRIBUTORS BE LIABLE > FOR ANY DIRECT, > INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > DAMAGES (INCLU- > DING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR > SERVICES; LOSS > OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER > CAUSED AND ON > ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > LIABILITY, OR TORT > (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > OF THE USE OF > THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > > */ > package org.apache.cocoon.caching.validity; > > /** > * Very experimental start at external cache invalidation. > * Warning - API very unstable. Do not use! In fact, if this > * becomes useful, it would probably move to Excalibur SourceResolve. > * > * Base class encapsulating the information about an external > * uncache event. > * > * @author Geoff Howard ([EMAIL PROTECTED]) > * > * @version $CVS$ > */ > public abstract class Event { > > public abstract boolean equals(Event e); > > public boolean equals(Object o) { > if (o instanceof Event) { > return equals((Event)o); > } > return false; > } > > } > > > > 1.1 > cocoon-2.1/src/scratchpad/src/org/apache/cocoon/caching/validity/E > ventValidity.java > > Index: EventValidity.java > =================================================================== > /* > > > ================================================================== > ========== > The Apache Software License, Version 1.1 > > ================================================================== > ========== > > Copyright (C) 1999-2003 The Apache Software Foundation. All > rights reserved. > > Redistribution and use in source and binary forms, with or > without modifica- > tion, are permitted provided that the following conditions are met: > > 1. Redistributions of source code must retain the above > copyright notice, > this list of conditions and the following disclaimer. > > 2. Redistributions in binary form must reproduce the above > copyright notice, > this list of conditions and the following disclaimer in the > documentation > and/or other materials provided with the distribution. > > 3. The end-user documentation included with the > redistribution, if any, must > include the following acknowledgment: "This product > includes software > developed by the Apache Software Foundation > (http://www.apache.org/)." > Alternately, this acknowledgment may appear in the > software itself, if > and wherever such third-party acknowledgments normally appear. > > 4. The names "Apache Cocoon" and "Apache Software Foundation" > must not be > used to endorse or promote products derived from this > software without > prior written permission. For written permission, please contact > [EMAIL PROTECTED] > > 5. Products derived from this software may not be called > "Apache", nor may > "Apache" appear in their name, without prior written > permission of the > Apache Software Foundation. > > THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR > IMPLIED WARRANTIES, > INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF > MERCHANTABILITY AND > FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO > EVENT SHALL THE > APACHE SOFTWARE FOUNDATION OR ITS CONTRIBUTORS BE LIABLE > FOR ANY DIRECT, > INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > DAMAGES (INCLU- > DING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR > SERVICES; LOSS > OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER > CAUSED AND ON > ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > LIABILITY, OR TORT > (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > OF THE USE OF > THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > > */ > package org.apache.cocoon.caching.validity; > > import org.apache.excalibur.source.SourceValidity; > > /** > * Very experimental start at external cache invalidation. > * Warning - API very unstable. Do not use! In fact, if this > * becomes useful, it would probably move to Excalibur SourceResolve. > * > * The SourceValidity object for cache invalidation based on > * external events. > * > * @author Geoff Howard([EMAIL PROTECTED]) > * @version $CVS$ > */ > public class EventValidity implements SourceValidity { > > private Event m_event; > > public EventValidity(Event ev) { > m_event = ev; > } > > public Event getEvent() { > return m_event; > } > > /** Basic implementation is always valid until event signals > * otherwise. May never need other behavior. > */ > public int isValid() { > return VALID; > } > > public int isValid(SourceValidity sv) { > if (sv instanceof EventValidity) { > return VALID; > } > return INVALID; > } > } > > > > 1.1 > cocoon-2.1/src/scratchpad/src/org/apache/cocoon/caching/validity/N > amedEvent.java > > Index: NamedEvent.java > =================================================================== > /* > > > ================================================================== > ========== > The Apache Software License, Version 1.1 > > ================================================================== > ========== > > Copyright (C) 1999-2003 The Apache Software Foundation. All > rights reserved. > > Redistribution and use in source and binary forms, with or > without modifica- > tion, are permitted provided that the following conditions are met: > > 1. Redistributions of source code must retain the above > copyright notice, > this list of conditions and the following disclaimer. > > 2. Redistributions in binary form must reproduce the above > copyright notice, > this list of conditions and the following disclaimer in the > documentation > and/or other materials provided with the distribution. > > 3. The end-user documentation included with the > redistribution, if any, must > include the following acknowledgment: "This product > includes software > developed by the Apache Software Foundation > (http://www.apache.org/)." > Alternately, this acknowledgment may appear in the > software itself, if > and wherever such third-party acknowledgments normally appear. > > 4. The names "Apache Cocoon" and "Apache Software Foundation" > must not be > used to endorse or promote products derived from this > software without > prior written permission. For written permission, please contact > [EMAIL PROTECTED] > > 5. Products derived from this software may not be called > "Apache", nor may > "Apache" appear in their name, without prior written > permission of the > Apache Software Foundation. > > THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR > IMPLIED WARRANTIES, > INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF > MERCHANTABILITY AND > FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO > EVENT SHALL THE > APACHE SOFTWARE FOUNDATION OR ITS CONTRIBUTORS BE LIABLE > FOR ANY DIRECT, > INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > DAMAGES (INCLU- > DING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR > SERVICES; LOSS > OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER > CAUSED AND ON > ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > LIABILITY, OR TORT > (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > OF THE USE OF > THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > > */ > package org.apache.cocoon.caching.validity; > > /** > * Very experimental start at external cache invalidation. > * Warning - API very unstable. Do not use! In fact, if this > * becomes useful, it would probably move to Excalibur SourceResolve. > * > * An External cache event that consists of just a name. Examples > * (not necessarily useful) could include "Easter" or "Shutdown" > * > * @author Geoff Howard ([EMAIL PROTECTED]) > * @version $CVS$ > */ > public class NamedEvent extends Event { > > private String m_name; > > public NamedEvent(String name) { > m_name = name; > } > > private String getName() { > return m_name; > } > > public boolean equals(Event e) { > if (e instanceof NamedEvent) { > return m_name.equals(((NamedEvent)e).getName()); > } > return false; > } > > > } > > > > >
