cziegeler 2002/07/15 01:17:29 Modified: . changes.xml todo.xml src/documentation/xdocs/plan release.xml src/java/org/apache/cocoon/caching ComponentCacheKey.java PipelineCacheKey.java src/java/org/apache/cocoon/components/pipeline AbstractProcessingPipeline.java ProcessingPipeline.java src/java/org/apache/cocoon/components/pipeline/impl CachingProcessingPipeline.java src/java/org/apache/cocoon/components/treeprocessor/sitemap GenerateNode.java SerializeNode.java TransformNode.java src/java/org/apache/cocoon/webapps/portal/components PortalManager.java src/webapp sitemap.xmap Added: src/java/org/apache/cocoon/components/pipeline/impl AbstractCachingProcessingPipeline.java CachingPointProcessingPipeline.java Log: Syncing docs and applying patch 10284 Revision Changes Path 1.213 +4 -1 xml-cocoon2/changes.xml Index: changes.xml =================================================================== RCS file: /home/cvs/xml-cocoon2/changes.xml,v retrieving revision 1.212 retrieving revision 1.213 diff -u -r1.212 -r1.213 --- changes.xml 13 Jul 2002 15:09:21 -0000 1.212 +++ changes.xml 15 Jul 2002 08:17:27 -0000 1.213 @@ -39,6 +39,9 @@ </devs> <release version="@version@" date="@date@"> + <action dev="CZ" type="add" fixes-bug="10284" due-to="Michael Melhem" due-to-email="[EMAIL PROTECTED] "> + Extended pipeline handling for CachingPoint caching algorithm. + </action> <action dev="CZ" type="add"> When Cocoon is built, the build system automatically detects the current jvm and builds a target for this virtual machine. This is eiter JVM 1.4 or JVM 1.2 upto 1.3. 1.38 +11 -49 xml-cocoon2/todo.xml Index: todo.xml =================================================================== RCS file: /home/cvs/xml-cocoon2/todo.xml,v retrieving revision 1.37 retrieving revision 1.38 diff -u -r1.37 -r1.38 --- todo.xml 13 Jul 2002 15:02:13 -0000 1.37 +++ todo.xml 15 Jul 2002 08:17:27 -0000 1.38 @@ -3,6 +3,7 @@ <!DOCTYPE todo PUBLIC "-//APACHE//DTD Todo V1.0//EN" "src/documentation/xdocs/dtd/todo-v10.dtd"> <!-- + History of Cocoon changes $Id$ --> @@ -31,31 +32,27 @@ <actions priority="high"> <action context="code" assigned-to="open"> - Rewrite the FilesystemStore. The current implementations relies on + For 2.1: Rewrite the FilesystemStore. The current implementations relies on a correct implementation of the toString() method of the used keys. A correct version should rely on hashCode() and equals(). (This is actually an Avalon action as the code is in Excalibur) </action> <action context="code" assigned-to="open"> - Implement the smart-caching for the processing pipeline and + For 2.1: Implement the smart-caching for the processing pipeline and iron out bugs. </action> <action context="docs" assigned-to="open"> - Attend to any high+ issues in the + For 2.1: Attend to any high+ issues in the <link href="plan/todo-doc.html">Documentation To Do List</link> </action> <action context="code" assigned-to="open"> - Correct the sitemap DTD which is now broken and make sitemap validated. + For 2.1: Correct the sitemap DTD which is now broken and make sitemap validated. </action> <action context="code" assigned-to="open"> - Make the use of Batik optional to avoid the X-server problem. - </action> - - <action context="code" assigned-to="open"> Complete (means put everything we know of into even if it has to be commented) the cocoon.xconf file and put descriptions into it </action> @@ -70,35 +67,17 @@ </action> <action context="code" assigned-to="open"> - Fix lastModificationDate() method misuse. Instead of modificationDate, - sometimes generated hash value is being returned, and instead of - comparing dates on before/after, they are compared on equality. - See - <link href="http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=99780708222079">http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=99780708222079</link> - and - <link href="http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=99814216923705">http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=99814216923705</link> - for more information on this issue. - </action> - - <action context="code" assigned-to="open"> View must start not from the first encountered label, but from the last one. </action> - <action context="code" assigned-to="open"> - Allow a Reader to give the content length to pipelines. This will allow to correctly - serve static PDFs (well-known IE+Acrobat bug) without requiring to buffer the whole - file in memory to know its content length. - </action> - <action context="code" assigned-to="NKB"> - Make comprehensive esamples with the handle-errors sitemap and real world + For 2.1: Make comprehensive samples with the handle-errors sitemap and real world use cases. Add also specific Selector and a FaqBuilder to be used by the as a NotifyingBuilder. </action> <action context="code" assigned-to="NKB"> - Make all the samples work again, and - <link href="plan/samples.html">finish refactoring</link>. + For 2.1: Make all the samples in WIP (Work In Progress) work again, and finish refactoring. </action> </actions> @@ -106,42 +85,25 @@ <actions priority="medium"> <action context="code"> - Finish the SourceWritingTransformer: for insertion of fragments the serializer - should be used instead of the XMLUtils. - </action> - - <action context="code"> - Rationalise the various copies of the DTDs and entities into one set. - See email discussion 2002-04-21 - </action> - - <action context="code"> Remove all useless blank strings in XSP-generated code that hinder performances. This should be configurable (through an attribute?) to be able to keep them when needed. </action> <action context="code"> - Make a guide on how to upgrade Cocoon,and see how this can be eased. + For 2.1: Make a guide on how to upgrade Cocoon,and see how this can be eased. </action> <action context="code"> - Check how to handle the mixing of output streams when an error inside + For 2.1: Check how to handle the mixing of output streams when an error inside a pipeline occurs. When the pipeline has something written to the output stream and then an error occurs the result is the first written part with the appended exception. One solution could be a configurable intermediate output stream. </action> - <action context="code" assigned-to="CZ"> - Redesign pipeline configuration and management: (1) Move pipeline - definitions from cocoon.xconf into sitemap.xmap, <map:components> - section; and (2) allow usage of different pipeline implementations in - <map:pipelines> section of the sitemap. - </action> - <action context="code"> - Redesign FragmentExtractorGenerator/Transformer so that it works on a clustered + For 2.1: Redesign FragmentExtractorGenerator/Transformer so that it works on a clustered server : store fragments in the session rather than in a local store. <br/> Additionnaly, store the fragments using the XMLCompiler instead of building a DOM. </action> 1.8 +1 -3 xml-cocoon2/src/documentation/xdocs/plan/release.xml Index: release.xml =================================================================== RCS file: /home/cvs/xml-cocoon2/src/documentation/xdocs/plan/release.xml,v retrieving revision 1.7 retrieving revision 1.8 diff -u -r1.7 -r1.8 --- release.xml 2 Jul 2002 10:32:57 -0000 1.7 +++ release.xml 15 Jul 2002 08:17:27 -0000 1.8 @@ -15,14 +15,12 @@ <s1 title="Timeframe"> <p>This is the current time frame for the next releases:</p> <ul> - <li>July/August 2002 : Release date for 2.0.3 (bug fix release)</li> + <li>September 2002 : Release date for 2.0.4 (bug fix release)</li> <li>August/September 2002 : Release date for 2.1 Alpha 1</li> </ul> </s1> <s1 title="ToDo List"> - <p>For 2.0.3:</p> - <ul><li>Solve the JDK 1.4 problems (whatever they are...)</li></ul> <p>For 2.1 Beta 1:</p> <ul> <li>apply patches</li> 1.13 +25 -1 xml-cocoon2/src/java/org/apache/cocoon/caching/ComponentCacheKey.java Index: ComponentCacheKey.java =================================================================== RCS file: /home/cvs/xml-cocoon2/src/java/org/apache/cocoon/caching/ComponentCacheKey.java,v retrieving revision 1.12 retrieving revision 1.13 diff -u -r1.12 -r1.13 --- ComponentCacheKey.java 5 Jun 2002 07:33:24 -0000 1.12 +++ ComponentCacheKey.java 15 Jul 2002 08:17:27 -0000 1.13 @@ -79,6 +79,8 @@ private Serializable key; /** the hash code */ private int hashCode = 0; + /** cachePoint */ + private boolean cachePoint = false; /** * Constructor @@ -93,6 +95,21 @@ } /** + * alternate cachepoint Constructor + */ + public ComponentCacheKey(int componentType, + String componentIdentifier, + Serializable cacheKey, + boolean cachePoint) { + this.type = componentType; + this.identifier = componentIdentifier; + this.key = cacheKey; + /** cachePoint */ + this.cachePoint = cachePoint; + + } + + /** * Compare */ public boolean equals(Object object) { @@ -130,5 +147,12 @@ toString = "CCK:" + this.type + this.identifier + " - " + this.key.toString(); } return toString; + } + + /** + * check if we are a cachepoint + */ + public boolean isCachePoint() { + return cachePoint; } } 1.18 +20 -1 xml-cocoon2/src/java/org/apache/cocoon/caching/PipelineCacheKey.java Index: PipelineCacheKey.java =================================================================== RCS file: /home/cvs/xml-cocoon2/src/java/org/apache/cocoon/caching/PipelineCacheKey.java,v retrieving revision 1.17 retrieving revision 1.18 diff -u -r1.17 -r1.18 --- PipelineCacheKey.java 7 Jun 2002 13:15:09 -0000 1.17 +++ PipelineCacheKey.java 15 Jul 2002 08:17:27 -0000 1.18 @@ -102,6 +102,25 @@ } /** + * Remove unitl cachepoint (including cachePoint) + */ + public void removeUntilCachePoint() { + this.hashCode = 0; + this.toString = null; + int keyCount = this.keys.size(); + + while (keyCount > 0) { + if (((ComponentCacheKey)this.keys.get(keyCount-1)).isCachePoint()) { + this.keys.remove(keyCount-1); + return; + } + this.keys.remove(keyCount-1); + keyCount--; + } + + } + + /** * Return the number of keys */ public int size() { 1.18 +7 -1 xml-cocoon2/src/java/org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.java Index: AbstractProcessingPipeline.java =================================================================== RCS file: /home/cvs/xml-cocoon2/src/java/org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.java,v retrieving revision 1.17 retrieving revision 1.18 diff -u -r1.17 -r1.18 --- AbstractProcessingPipeline.java 4 Jul 2002 11:46:05 -0000 1.17 +++ AbstractProcessingPipeline.java 15 Jul 2002 08:17:28 -0000 1.18 @@ -204,6 +204,12 @@ } /** + * Inform pipeline we have come across a branch point + * Default Behaviour is do nothing + */ + public void informBranchPoint() {} + + /** * Set the generator that will be used as the initial step in the pipeline. * The generator role is given : the actual <code>Generator</code> is fetched * from the latest <code>ComponentManager</code> given by <code>compose()</code> 1.9 +6 -1 xml-cocoon2/src/java/org/apache/cocoon/components/pipeline/ProcessingPipeline.java Index: ProcessingPipeline.java =================================================================== RCS file: /home/cvs/xml-cocoon2/src/java/org/apache/cocoon/components/pipeline/ProcessingPipeline.java,v retrieving revision 1.8 retrieving revision 1.9 diff -u -r1.8 -r1.9 --- ProcessingPipeline.java 28 May 2002 14:03:18 -0000 1.8 +++ ProcessingPipeline.java 15 Jul 2002 08:17:28 -0000 1.9 @@ -120,6 +120,11 @@ Generator getGenerator(); /** + * Inform pipeline we have come across a branch point + */ + public void informBranchPoint(); + + /** * Add a transformer at the end of the pipeline. * The transformer role is given : the actual <code>Transformer</code> is fetched * from the latest <code>ComponentManager</code> given by <code>compose()</code> 1.28 +41 -544 xml-cocoon2/src/java/org/apache/cocoon/components/pipeline/impl/CachingProcessingPipeline.java Index: CachingProcessingPipeline.java =================================================================== RCS file: /home/cvs/xml-cocoon2/src/java/org/apache/cocoon/components/pipeline/impl/CachingProcessingPipeline.java,v retrieving revision 1.27 retrieving revision 1.28 diff -u -r1.27 -r1.28 --- CachingProcessingPipeline.java 4 Jul 2002 11:46:05 -0000 1.27 +++ CachingProcessingPipeline.java 15 Jul 2002 08:17:28 -0000 1.28 @@ -1,36 +1,36 @@ /* - + ============================================================================ The Apache Software License, Version 1.1 ============================================================================ - + Copyright (C) 1999-2002 The Apache Software Foundation. All rights reserved. - + Redistribution and use in source and binary forms, with or without modifica- tion, are permitted provided that the following conditions are met: - + 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. - + 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. - + 3. The end-user documentation included with the redistribution, if any, must include the following acknowledgment: "This product includes software developed by the Apache Software Foundation (http://www.apache.org/)." Alternately, this acknowledgment may appear in the software itself, if and wherever such third-party acknowledgments normally appear. - + 4. The names "Apache Cocoon" and "Apache Software Foundation" must not be used to endorse or promote products derived from this software without prior written permission. For written permission, please contact [EMAIL PROTECTED] - + 5. Products derived from this software may not be called "Apache", nor may "Apache" appear in their name, without prior written permission of the Apache Software Foundation. - + THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE @@ -41,18 +41,16 @@ ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - + This software consists of voluntary contributions made by many individuals on behalf of the Apache Software Foundation and was originally created by Stefano Mazzocchi <[EMAIL PROTECTED]>. For more information on the Apache Software Foundation, please see <http://www.apache.org/>. - + */ package org.apache.cocoon.components.pipeline.impl; -import org.apache.avalon.framework.activity.Disposable; import org.apache.avalon.framework.component.ComponentException; -import org.apache.avalon.framework.component.ComponentManager; import org.apache.avalon.framework.parameters.Parameters; import org.apache.cocoon.ConnectionResetException; import org.apache.cocoon.ProcessingException; @@ -74,12 +72,13 @@ import org.apache.cocoon.xml.XMLConsumer; import org.apache.cocoon.xml.XMLProducer; import org.apache.excalibur.source.SourceValidity; -import java.io.ByteArrayOutputStream; import java.io.OutputStream; import java.io.Serializable; import java.net.SocketException; import java.util.ArrayList; import java.util.Iterator; +import java.util.ListIterator; +import java.util.Map; /** * The CachingProcessingPipeline @@ -89,499 +88,42 @@ * @version CVS $Id$ */ public class CachingProcessingPipeline -extends AbstractProcessingPipeline -implements Disposable { - - /** The Cache storing the cached response (or part of the response) */ - protected Cache cache; - - /** The role name of the generator */ - protected String generatorRole; - - /** The role names of the transfomrers */ - protected ArrayList transformerRoles = new ArrayList(); - - /** The role name of the serializer */ - protected String serializerRole; - - /** The role name of the reader */ - protected String readerRole; - - /** The deserializer */ - protected XMLDeserializer xmlDeserializer; - /** The serializer */ - protected XMLSerializer xmlSerializer; - - /** The cached byte stream */ - protected byte[] cachedResponse; - /** The index indicating the first transformer getting input from the cache */ - protected int firstProcessedTransformerIndex; - /** Complete response is cached */ - protected boolean completeResponseIsCached; - - /** The key to store the generated response */ - protected PipelineCacheKey pipelineCacheKey; - /** The validity objects of the generated response */ - protected SourceValidity[] pipelineValidityObjects; - /** The index indicating to the first transformer which is not cacheable */ - protected int firstNotCacheableTransformerIndex; - /** Cache complete response */ - protected boolean cacheCompleteResponse; - /** Smart caching ? */ - protected boolean doSmartCaching; - - /** - * Composable Interface - */ - public void compose (ComponentManager manager) - throws ComponentException { - super.compose(manager); - this.cache = (Cache)this.manager.lookup(Cache.ROLE); - } - - /** - * Setup this component - */ - public void setup(Parameters params) { - super.setup(params); - this.doSmartCaching = params.getParameterAsBoolean("smart-caching", - this.configuration.getParameterAsBoolean("smart-caching", true)); - // FIXME (CZ) - Implement smart caching - } - - /** - * Set the generator. - */ - public void setGenerator (String role, String source, Parameters param) - throws ProcessingException { - super.setGenerator(role, source, param); - this.generatorRole = role; - } - - /** - * Add a transformer. - */ - public void addTransformer (String role, String source, Parameters param) - throws ProcessingException { - super.addTransformer(role, source, param); - this.transformerRoles.add(role); - } - - /** - * Set the serializer. - */ - public void setSerializer (String role, String source, Parameters param, String mimeType) - throws ProcessingException { - super.setSerializer(role, source, param, mimeType); - this.serializerRole = role; - } + extends AbstractCachingProcessingPipeline { /** - * Set the Reader. - */ - public void setReader (String role, String source, Parameters param, String mimeType) - throws ProcessingException { - super.setReader(role, source, param, mimeType); - this.readerRole = role; - } - - /** - * Process the given <code>Environment</code>, producing the output. - */ - protected boolean processXMLPipeline(Environment environment) - throws ProcessingException { - if (this.pipelineCacheKey == null && this.cachedResponse == null) { - return super.processXMLPipeline( environment ); - } else if (this.cachedResponse != null && this.completeResponseIsCached) { - try { - final OutputStream outputStream = environment.getOutputStream(); - if (this.cachedResponse.length > 0) { - environment.setContentLength(this.cachedResponse.length); - outputStream.write(this.cachedResponse); - } - } catch ( SocketException se ) { - if (se.getMessage().indexOf("reset") > 0 - || se.getMessage().indexOf("aborted") > 0) { - throw new ConnectionResetException("Connection reset by peer", se); - } else { - throw new ProcessingException("Failed to execute reader pipeline.", se); - } - } catch ( Exception e ) { - if (e instanceof ProcessingException) - throw (ProcessingException)e; - throw new ProcessingException("Error executing reader pipeline.",e); - } - } else { - - if (this.getLogger().isDebugEnabled()) { - this.getLogger().debug("Caching content for further requests of '" + environment.getURI() + "'."); - } - try { - OutputStream os = environment.getOutputStream(); - if ( this.cacheCompleteResponse ) { - os = new CachingOutputStream( os ); - } - if (this.serializer.shouldSetContentLength()) { - // set the output stream - ByteArrayOutputStream baos = new ByteArrayOutputStream(); - this.serializer.setOutputStream(baos); - - // execute the pipeline: - if ( this.xmlDeserializer != null ) { - this.xmlDeserializer.deserialize(this.cachedResponse); - } else { - this.generator.generate(); - } - byte[] data = baos.toByteArray(); - environment.setContentLength(data.length); - os.write(data); - } else { - // set the output stream - this.serializer.setOutputStream( os ); - // execute the pipeline: - if ( this.xmlDeserializer != null ) { - this.xmlDeserializer.deserialize(this.cachedResponse); - } else { - this.generator.generate(); - } - } - if (this.pipelineCacheKey != null) { - if ( this.cacheCompleteResponse ) { - CachedResponse response = new CachedResponse(this.pipelineValidityObjects, - ((CachingOutputStream)os).getContent()); - this.cache.store(environment.getObjectModel(), - this.pipelineCacheKey, - response); - } else { - CachedResponse response = new CachedResponse(this.pipelineValidityObjects, - (byte[])this.xmlSerializer.getSAXFragment()); - this.cache.store(environment.getObjectModel(), - this.pipelineCacheKey, - response); - } - } - } catch ( SocketException se ) { - if (se.getMessage().indexOf("reset") > 0 - || se.getMessage().indexOf("aborted") > 0) { - throw new ConnectionResetException("Connection reset by peer", se); - } else { - throw new ProcessingException("Failed to execute reader pipeline.", se); - } - } catch ( ProcessingException e ) { - throw e; - } catch ( Exception e ) { - throw new ProcessingException("Failed to execute pipeline.", e); + * Cache longest cachable key + */ + protected void cacheResults(Environment environment, OutputStream os) throws Exception { + if (this.pipelineCacheKey != null) { + if ( this.cacheCompleteResponse ) { + CachedResponse response = new CachedResponse(this.pipelineValidityObjects, + ((CachingOutputStream)os).getContent()); + this.cache.store(environment.getObjectModel(), + this.pipelineCacheKey, + response); + } else { + CachedResponse response = new CachedResponse(this.pipelineValidityObjects, + (byte[])this.xmlSerializer.getSAXFragment()); + this.cache.store(environment.getObjectModel(), + this.pipelineCacheKey, + response); } - return true; } - return true; } /** - * Setup the evenet pipeline. - * The components of the pipeline are checked if they are - * Cacheable. + * create a new cache key */ - protected void setupPipeline(Environment environment) - throws ProcessingException { - super.setupPipeline( environment ); - - // - // FIXME (VG): This method is 280 lines long. It *must* be refactored... - // - - PipelineCacheKey cachedPipelineKey = null; - PipelineCacheKey processingPipelineKey = null; - - Serializable key = null; - boolean generatorIsCacheableProcessingComponent = false; - boolean serializerIsCacheableProcessingComponent = false; - boolean[] transformerIsCacheableProcessingComponent = new boolean[this.transformers.size()]; - - this.firstNotCacheableTransformerIndex = 0; - this.firstProcessedTransformerIndex = 0; - this.completeResponseIsCached = false; - this.cacheCompleteResponse = false; - - // first step is to generate the key: - // All pipeline components starting with the generator - // are tested if they are either a CacheableProcessingComponent - // or Cacheable (deprecated). The returned keys are chained together - // to build a unique key of the request - - // Is the generator cacheable? - if (super.generator instanceof CacheableProcessingComponent) { - key = ((CacheableProcessingComponent)super.generator).generateKey(); - generatorIsCacheableProcessingComponent = true; - } else if (super.generator instanceof Cacheable) { - key = new Long(((Cacheable)super.generator).generateKey()); - } - - if (key != null) { - processingPipelineKey = new PipelineCacheKey(); - processingPipelineKey.addKey(new ComponentCacheKey(ComponentCacheKey.ComponentType_Generator, - this.generatorRole, - key) - ); - - // Now testing transformers - final int transformerSize = super.transformers.size(); - while (firstNotCacheableTransformerIndex < transformerSize) { - final Transformer trans = - (Transformer)super.transformers.get(firstNotCacheableTransformerIndex); - key = null; - if (trans instanceof CacheableProcessingComponent) { - key = ((CacheableProcessingComponent)trans).generateKey(); - transformerIsCacheableProcessingComponent[firstNotCacheableTransformerIndex] = true; - } else if (trans instanceof Cacheable) { - key = new Long(((Cacheable)trans).generateKey()); - } - if (key != null) { - processingPipelineKey.addKey(new ComponentCacheKey(ComponentCacheKey.ComponentType_Transformer, - (String)this.transformerRoles.get(this.firstNotCacheableTransformerIndex), - key) - ); - this.firstNotCacheableTransformerIndex++; - } else { - break; - } - } - - // All transformers are cacheable => pipeline is cacheable, test serializer - if (this.firstNotCacheableTransformerIndex == transformerSize - && super.serializer == this.lastConsumer) { - - key = null; - if (super.serializer instanceof CacheableProcessingComponent) { - key = ((CacheableProcessingComponent)this.serializer).generateKey(); - serializerIsCacheableProcessingComponent = true; - } else if (this.serializer instanceof Cacheable) { - key = new Long(((Cacheable)this.serializer).generateKey()); - } - if (key != null) { - processingPipelineKey.addKey(new ComponentCacheKey(ComponentCacheKey.ComponentType_Serializer, - (String)this.serializerRole, - key) - ); - this.completeResponseIsCached = true; - this.cacheCompleteResponse = true; - } - } - } - - // Now, we have a key representing all cacheable components up to the - // first non cacheable - if (processingPipelineKey != null) { - cachedPipelineKey = processingPipelineKey.copy(); - this.firstProcessedTransformerIndex = this.firstNotCacheableTransformerIndex; - } - - boolean finished = false; - SourceValidity[] cachedValidityObjects = null; - while (cachedPipelineKey != null && !finished) { - finished = true; - CachedResponse response = this.cache.get( cachedPipelineKey ); - - // Now test validity of the response - if (response != null) { - boolean responseIsValid = true; - boolean responseIsUsable = true; - if (this.getLogger().isDebugEnabled()) { - this.getLogger().debug("Found cached response for '" + environment.getURI() + "'."); - } - this.cachedResponse = response.getResponse(); - SourceValidity[] validities = response.getValidityObjects(); - int i = 0; - while (responseIsValid && i < validities.length) { - // Obtain new validity of the component - SourceValidity validity = null; - if (i == 0) { - // test generator - if (generatorIsCacheableProcessingComponent) { - validity = ((CacheableProcessingComponent)super.generator).generateValidity(); - } else { - validity = CacheValidityToSourceValidity.createValidity(((Cacheable)super.generator).generateValidity()); - } - } else if (i <= this.firstProcessedTransformerIndex) { - // test transformer - final Transformer trans = (Transformer)super.transformers.get(i-1); - if (transformerIsCacheableProcessingComponent[i-1]) { - validity = ((CacheableProcessingComponent)trans).generateValidity(); - } else { - validity = CacheValidityToSourceValidity.createValidity(((Cacheable)trans).generateValidity()); - } - } else { - // test serializer - if (serializerIsCacheableProcessingComponent) { - validity = ((CacheableProcessingComponent)super.serializer).generateValidity(); - } else { - validity = CacheValidityToSourceValidity.createValidity(((Cacheable)super.serializer).generateValidity()); - } - } - - int valid = -1; - // It can be null. After update. - if (validities[i] != null) { - valid = validities[i].isValid(); - } - boolean isValid = false; - if ( valid == 0 ) { - if (validity != null) { - isValid = validities[i].isValid( validity ); - } - } else { - isValid = (valid == 1); - } - if ( !isValid ) { - responseIsValid = false; - // Update validity. Validity can be null. - validities[i] = validity; - if (validity == null) responseIsUsable = false; - } else { - i++; - } - } - if ( responseIsValid ) { - // we are valid, ok that's it - cachedValidityObjects = validities; - if (this.getLogger().isDebugEnabled()) { - this.getLogger().debug("Using valid cached content for '" + environment.getURI() + "'."); - } - } else { - if (this.getLogger().isDebugEnabled()) { - this.getLogger().debug("Cached content is invalid for '" + environment.getURI() + "'."); - } - // we are not valid! - this.completeResponseIsCached = false; - finished = false; - this.cachedResponse = null; - - if (!responseIsUsable) { - // We couldn't compare, because we got no validity - // object, so shorten pipeline key and try again - if (i > 0) { - int deleteCount = validities.length - i; - if (i > 0 && i <= firstNotCacheableTransformerIndex + 1) { - this.firstNotCacheableTransformerIndex = i-1; - } - for(int x=0; x < deleteCount; x++) { - processingPipelineKey.removeLastKey(); - } - finished = false; - } else { - processingPipelineKey = null; - } - this.cacheCompleteResponse = false; - } else { - // the entry is invalid, remove it - this.cache.remove( cachedPipelineKey ); - } - // Try a shorter key - if (i > 0) { - cachedPipelineKey.removeLastKey(); - this.firstProcessedTransformerIndex--; - } else { - cachedPipelineKey = null; - } - } - } else { - if (this.getLogger().isDebugEnabled()) { - this.getLogger().debug("Cached response not found for '" + environment.getURI() + "'."); - } - finished = false; - this.completeResponseIsCached = false; - // try a shorter key - // FIXME (CZ) calculation of PPK - if (cachedPipelineKey.size() > 1) { - cachedPipelineKey.removeLastKey(); - this.firstProcessedTransformerIndex--; - } else { - cachedPipelineKey = null; - } - } - } - - // FIXME (VG): This part below incorrectly caches only part of pipeline when - // it was detected change in one of the pipeline components. It should cache - // whole pipeline (if it is cacheable), even if current response was created - // from partial cached response. - - // now generate validity objects for the new response - if (processingPipelineKey != null) { - if (cachedPipelineKey == null || - cachedPipelineKey.size() < processingPipelineKey.size()) { - this.pipelineCacheKey = processingPipelineKey; - this.pipelineValidityObjects = new SourceValidity[processingPipelineKey.size()]; - int i = 0; - if (cachedPipelineKey != null) { - i = cachedPipelineKey.size(); - for(int j = 0; j < i; j++) { - this.pipelineValidityObjects[j] = cachedValidityObjects[j]; - } - } - int len = this.pipelineValidityObjects.length; - while (i < len) { - final SourceValidity validity; - if (i == 0) { - // test generator - if (generatorIsCacheableProcessingComponent) { - validity = ((CacheableProcessingComponent)super.generator).generateValidity(); - } else { - validity = CacheValidityToSourceValidity.createValidity(((Cacheable)super.generator).generateValidity()); - } - } else if (i <= firstNotCacheableTransformerIndex) { - // test transformer - final Transformer trans = - (Transformer)super.transformers.get(i-1); - if (transformerIsCacheableProcessingComponent[i-1]) { - validity = ((CacheableProcessingComponent)trans).generateValidity(); - } else { - validity = CacheValidityToSourceValidity.createValidity(((Cacheable)trans).generateValidity()); - } - } else { - // test serializer - if (serializerIsCacheableProcessingComponent) { - validity = ((CacheableProcessingComponent)super.serializer).generateValidity(); - } else { - validity = CacheValidityToSourceValidity.createValidity(((Cacheable)super.serializer).generateValidity()); - } - } - if (validity == null) { - if (i > 0 && i > cachedPipelineKey.size()) { - // shorten key - for(int m=i; m < len; m++) { - this.pipelineCacheKey.removeLastKey(); - } - SourceValidity[] copy = new SourceValidity[i]; - System.arraycopy(this.pipelineValidityObjects, 0, - copy, 0, copy.length); - this.pipelineValidityObjects = copy; - len = this.pipelineValidityObjects.length; - } else { - this.pipelineCacheKey = null; - this.pipelineValidityObjects = null; - len = 0; - } - } else { - this.pipelineValidityObjects[i] = validity; - } - i++; - } - } - } + protected ComponentCacheKey newComponentCacheKey(int type, String role,Serializable key) { + return new ComponentCacheKey(type, role, key); } + /** * Connect the pipeline. */ - protected void connectPipeline(Environment environment) + protected void connectCachingPipeline(Environment environment) throws ProcessingException { - if ( this.pipelineCacheKey == null && this.cachedResponse == null) { - super.connectPipeline( environment ); - } else if (this.completeResponseIsCached) { - // do nothing - return; - } else { try { XMLSerializer localXMLSerializer = null; if (!this.cacheCompleteResponse) { @@ -625,7 +167,7 @@ next = (XMLConsumer) itt.next(); if (cacheableTransformerCount >= this.firstProcessedTransformerIndex) { if (localXMLSerializer != null - && cacheableTransformerCount == this.firstNotCacheableTransformerIndex) { + && cacheableTransformerCount == this.firstNotCacheableTransformerIndex) { next = new XMLTeePipe(next, localXMLSerializer); localXMLSerializer = null; } @@ -645,7 +187,6 @@ } catch ( ComponentException e ) { throw new ProcessingException("Could not connect pipeline.", e); } - } } @@ -676,7 +217,7 @@ pcKey.addKey(new ComponentCacheKey(ComponentCacheKey.ComponentType_Reader, this.readerRole, readerKey) - ); + ); // now we have the key to get the cached object CachedResponse cachedObject = (CachedResponse)this.cache.get( pcKey ); @@ -703,7 +244,7 @@ } } if (readerValidity != null) { - valid = cachedValidity.isValid(readerValidity); + valid = cachedValidity.isValid(readerValidity); } } else { valid = (result > 0); @@ -760,13 +301,13 @@ environment.getObjectModel(), pcKey, new CachedResponse( new SourceValidity[] {readerValidity}, - ((CachingOutputStream)outputStream).getContent()) + ((CachingOutputStream)outputStream).getContent()) ); } } } catch ( SocketException se ) { if (se.getMessage().indexOf("reset") > 0 - || se.getMessage().indexOf("aborted") > 0) { + || se.getMessage().indexOf("aborted") > 0) { throw new ConnectionResetException("Connection reset by peer", se); } else { throw new ProcessingException("Failed to execute pipeline.", se); @@ -780,48 +321,4 @@ return true; } - /** - * Return valid validity objects for the event pipeline - * If the "event pipeline" (= the complete pipeline without the - * serializer) is cacheable and valid, return all validity objects. - * Otherwise return <code>null</code> - */ - public SourceValidity[] getValiditiesForEventPipeline() { - if (!this.completeResponseIsCached - && this.firstNotCacheableTransformerIndex == super.transformers.size()) { - return this.pipelineValidityObjects; - } - return null; - } - - /** - * Recyclable Interface - */ - public void recycle() { - super.recycle(); - - this.manager.release( this.xmlDeserializer ); - this.xmlDeserializer = null; - - this.manager.release( this.xmlSerializer ); - this.xmlSerializer = null; - - this.generatorRole = null; - this.transformerRoles.clear(); - this.serializerRole = null; - this.readerRole = null; - - this.pipelineCacheKey = null; - this.pipelineValidityObjects = null; - this.cachedResponse = null; - } - - /** - * Disposable Interface - */ - public void dispose() { - this.manager.release(this.cache); - this.cache = null; - this.manager = null; - } } 1.1 xml-cocoon2/src/java/org/apache/cocoon/components/pipeline/impl/AbstractCachingProcessingPipeline.java Index: AbstractCachingProcessingPipeline.java =================================================================== /* ============================================================================ The Apache Software License, Version 1.1 ============================================================================ Copyright (C) 1999-2002 The Apache Software Foundation. All rights reserved. Redistribution and use in source and binary forms, with or without modifica- tion, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. The end-user documentation included with the redistribution, if any, must include the following acknowledgment: "This product includes software developed by the Apache Software Foundation (http://www.apache.org/)." Alternately, this acknowledgment may appear in the software itself, if and wherever such third-party acknowledgments normally appear. 4. The names "Apache Cocoon" and "Apache Software Foundation" must not be used to endorse or promote products derived from this software without prior written permission. For written permission, please contact [EMAIL PROTECTED] 5. Products derived from this software may not be called "Apache", nor may "Apache" appear in their name, without prior written permission of the Apache Software Foundation. THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLU- DING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. This software consists of voluntary contributions made by many individuals on behalf of the Apache Software Foundation and was originally created by Stefano Mazzocchi <[EMAIL PROTECTED]>. For more information on the Apache Software Foundation, please see <http://www.apache.org/>. */ package org.apache.cocoon.components.pipeline.impl; import org.apache.avalon.framework.activity.Disposable; import org.apache.avalon.framework.component.ComponentException; import org.apache.avalon.framework.component.ComponentManager; import org.apache.avalon.framework.parameters.Parameters; import org.apache.cocoon.ConnectionResetException; import org.apache.cocoon.ProcessingException; import org.apache.cocoon.caching.Cache; import org.apache.cocoon.caching.Cacheable; import org.apache.cocoon.caching.CacheableProcessingComponent; import org.apache.cocoon.caching.CachedResponse; import org.apache.cocoon.caching.CacheValidity; import org.apache.cocoon.caching.CacheValidityToSourceValidity; import org.apache.cocoon.caching.CachingOutputStream; import org.apache.cocoon.caching.ComponentCacheKey; import org.apache.cocoon.caching.PipelineCacheKey; import org.apache.cocoon.components.pipeline.AbstractProcessingPipeline; import org.apache.cocoon.components.sax.XMLDeserializer; import org.apache.cocoon.components.sax.XMLSerializer; import org.apache.cocoon.components.sax.XMLTeePipe; import org.apache.cocoon.environment.Environment; import org.apache.cocoon.transformation.Transformer; import org.apache.cocoon.xml.XMLConsumer; import org.apache.cocoon.xml.XMLProducer; import org.apache.excalibur.source.SourceValidity; import java.io.ByteArrayOutputStream; import java.io.OutputStream; import java.io.Serializable; import java.net.SocketException; import java.util.ArrayList; import java.util.HashMap; import java.util.Map; /** * This is the base class for all caching pipeline implementations. * * * @since 2.1 * @author <a href="mailto:[EMAIL PROTECTED]">Carsten Ziegeler</a> * @author <a href="mailto:[EMAIL PROTECTED]">Michael Melhem</a> * @version CVS $Id: AbstractCachingProcessingPipeline.java,v 1.1 2002/07/15 08:17:28 cziegeler Exp $ */ public abstract class AbstractCachingProcessingPipeline extends AbstractProcessingPipeline implements Disposable { protected Cache cache; /** The role name of the generator */ protected String generatorRole; /** The role names of the transfomrers */ protected ArrayList transformerRoles = new ArrayList(); /** The role name of the serializer */ protected String serializerRole; /** The role name of the reader */ protected String readerRole; /** The deserializer */ protected XMLDeserializer xmlDeserializer; /** The serializer */ protected XMLSerializer xmlSerializer; /** The cached byte stream */ protected byte[] cachedResponse; /** The index indicating the first transformer getting input from the cache */ protected int firstProcessedTransformerIndex; /** Complete response is cached */ protected boolean completeResponseIsCached; /** The key to store the generated response */ protected PipelineCacheKey pipelineCacheKey; /** The validity objects of the generated response */ protected SourceValidity[] pipelineValidityObjects; /** The index indicating to the first transformer which is not cacheable */ protected int firstNotCacheableTransformerIndex; /** Cache complete response */ protected boolean cacheCompleteResponse; /** Smart caching ? */ protected boolean doSmartCaching; /** * Abstract methods defined in subclasses */ protected abstract void cacheResults(Environment environment, OutputStream os) throws Exception; protected abstract ComponentCacheKey newComponentCacheKey(int type, String role,Serializable key); protected abstract void connectCachingPipeline(Environment environment) throws ProcessingException; /** * Composable Interface */ public void compose (ComponentManager manager) throws ComponentException { super.compose(manager); this.cache = (Cache)this.manager.lookup(Cache.ROLE); } /** * Setup this component */ public void setup(Parameters params) { super.setup(params); this.doSmartCaching = params.getParameterAsBoolean("smart-caching", this.configuration.getParameterAsBoolean("smart-caching", true)); // FIXME (CZ) - Implement smart caching } /** * Set the generator. */ public void setGenerator (String role, String source, Parameters param) throws ProcessingException { super.setGenerator(role, source, param); this.generatorRole = role; } /** * Add a transformer. */ public void addTransformer (String role, String source, Parameters param) throws ProcessingException { super.addTransformer(role, source, param); this.transformerRoles.add(role); } /** * Set the serializer. */ public void setSerializer (String role, String source, Parameters param, String mimeType) throws ProcessingException { super.setSerializer(role, source, param, mimeType); this.serializerRole = role; } /** * Set the Reader. */ public void setReader (String role, String source, Parameters param, String mimeType) throws ProcessingException { super.setReader(role, source, param, mimeType); this.readerRole = role; } /** * Process the given <code>Environment</code>, producing the output. */ protected boolean processXMLPipeline(Environment environment) throws ProcessingException { if (this.pipelineCacheKey == null && this.cachedResponse == null) { return super.processXMLPipeline( environment ); } else if (this.cachedResponse != null && this.completeResponseIsCached) { try { final OutputStream outputStream = environment.getOutputStream(); if (this.cachedResponse.length > 0) { environment.setContentLength(this.cachedResponse.length); outputStream.write(this.cachedResponse); } } catch ( SocketException se ) { if (se.getMessage().indexOf("reset") > 0 || se.getMessage().indexOf("aborted") > 0) { throw new ConnectionResetException("Connection reset by peer", se); } else { throw new ProcessingException("Failed to execute reader pipeline.", se); } } catch ( Exception e ) { if (e instanceof ProcessingException) throw (ProcessingException)e; throw new ProcessingException("Error executing reader pipeline.",e); } } else { if (this.getLogger().isDebugEnabled()) { this.getLogger().debug("Caching content for further requests of '" + environment.getURI() + "'."); } try { OutputStream os = environment.getOutputStream(); if ( this.cacheCompleteResponse ) { os = new CachingOutputStream( os ); } if (this.serializer.shouldSetContentLength()) { // set the output stream ByteArrayOutputStream baos = new ByteArrayOutputStream(); this.serializer.setOutputStream(baos); // execute the pipeline: if ( this.xmlDeserializer != null ) { this.xmlDeserializer.deserialize(this.cachedResponse); } else { this.generator.generate(); } byte[] data = baos.toByteArray(); environment.setContentLength(data.length); os.write(data); } else { // set the output stream this.serializer.setOutputStream( os ); // execute the pipeline: if ( this.xmlDeserializer != null ) { this.xmlDeserializer.deserialize(this.cachedResponse); } else { this.generator.generate(); } } // // Now that we have processed the pipeline, // we do the actual caching // this.cacheResults(environment,os); } catch ( SocketException se ) { if (se.getMessage().indexOf("reset") > 0 || se.getMessage().indexOf("aborted") > 0) { throw new ConnectionResetException("Connection reset by peer", se); } else { throw new ProcessingException("Failed to execute reader pipeline.", se); } } catch ( ProcessingException e ) { throw e; } catch ( Exception e ) { throw new ProcessingException("Failed to execute pipeline.", e); } return true; } return true; } /** * Setup the evenet pipeline. * The components of the pipeline are checked if they are * Cacheable. */ protected void setupPipeline(Environment environment) throws ProcessingException { super.setupPipeline( environment ); PipelineCacheKey cachedPipelineKey = null; PipelineCacheKey processingPipelineKey = null; Serializable key = null; boolean generatorIsCacheableProcessingComponent = false; boolean serializerIsCacheableProcessingComponent = false; boolean[] transformerIsCacheableProcessingComponent = new boolean[this.transformers.size()]; this.firstNotCacheableTransformerIndex = 0; this.firstProcessedTransformerIndex = 0; this.completeResponseIsCached = false; this.cacheCompleteResponse = false; // first step is to generate the key: // All pipeline components starting with the generator // are tested if they are either a CacheableProcessingComponent // or Cacheable (deprecated). The returned keys are chained together // to build a unique key of the request // is the generator cacheable? if (super.generator instanceof CacheableProcessingComponent) { key = ((CacheableProcessingComponent)super.generator).generateKey(); generatorIsCacheableProcessingComponent = true; } else if (super.generator instanceof Cacheable) { key = new Long(((Cacheable)super.generator).generateKey()); } if (key != null) { processingPipelineKey = new PipelineCacheKey(); processingPipelineKey.addKey(this.newComponentCacheKey(ComponentCacheKey.ComponentType_Generator, this.generatorRole, key) ); // now testing transformers final int transformerSize = super.transformers.size(); boolean continueTest = true; while (this.firstNotCacheableTransformerIndex < transformerSize && continueTest) { final Transformer trans = (Transformer)super.transformers.get(this.firstNotCacheableTransformerIndex); key = null; if (trans instanceof CacheableProcessingComponent) { key = ((CacheableProcessingComponent)trans).generateKey(); transformerIsCacheableProcessingComponent[this.firstNotCacheableTransformerIndex] = true; } else if (trans instanceof Cacheable) { key = new Long(((Cacheable)trans).generateKey()); } if (key != null) { processingPipelineKey.addKey(this.newComponentCacheKey(ComponentCacheKey.ComponentType_Transformer, (String)this.transformerRoles.get(this.firstNotCacheableTransformerIndex), key)); this.firstNotCacheableTransformerIndex++; } else { continueTest = false; } } // all transformers are cacheable => pipeline is cacheable, test serializer if (this.firstNotCacheableTransformerIndex == transformerSize && super.serializer == this.lastConsumer) { key = null; if (super.serializer instanceof CacheableProcessingComponent) { key = ((CacheableProcessingComponent)this.serializer).generateKey(); serializerIsCacheableProcessingComponent = true; } else if (this.serializer instanceof Cacheable) { key = new Long(((Cacheable)this.serializer).generateKey()); } if (key != null) { processingPipelineKey.addKey(this.newComponentCacheKey(ComponentCacheKey.ComponentType_Serializer, (String)this.serializerRole, key) ); this.completeResponseIsCached = true; this.cacheCompleteResponse = true; } } } // now, we have a key representing all cacheable components up to the // first non cacheable if (processingPipelineKey != null) { cachedPipelineKey = processingPipelineKey.copy(); this.firstProcessedTransformerIndex = this.firstNotCacheableTransformerIndex; } boolean finished = false; SourceValidity[] cachedValidityObjects = null; while (cachedPipelineKey != null && !finished) { finished = true; CachedResponse response = this.cache.get( cachedPipelineKey ); // now test validity if (response != null) { boolean responseIsValid = true; boolean responseIsUsable = true; if (this.getLogger().isDebugEnabled()) { this.getLogger().debug("Found cached response for '" + environment.getURI() + "'."); } this.cachedResponse = response.getResponse(); SourceValidity[] validities = response.getValidityObjects(); int i = 0; while (responseIsValid && i < validities.length) { int valid = validities[i].isValid(); boolean isValid = false; SourceValidity validity = null; if ( valid == 0) { if (i == 0) { // test generator if (generatorIsCacheableProcessingComponent) { validity = ((CacheableProcessingComponent)super.generator).generateValidity(); } else { validity = CacheValidityToSourceValidity.createValidity(((Cacheable)super.generator).generateValidity()); } } else if (i <= this.firstProcessedTransformerIndex) { // test transformer final Transformer trans = (Transformer)super.transformers.get(i-1); if (transformerIsCacheableProcessingComponent[i-1]) { validity = ((CacheableProcessingComponent)trans).generateValidity(); } else { validity = CacheValidityToSourceValidity.createValidity(((Cacheable)trans).generateValidity()); } } else { // test serializer if (serializerIsCacheableProcessingComponent) { validity = ((CacheableProcessingComponent)super.serializer).generateValidity(); } else { validity = CacheValidityToSourceValidity.createValidity(((Cacheable)super.serializer).generateValidity()); } } if (validity != null) { isValid = validities[i].isValid( validity ); } } else { isValid = (valid == 1); } if ( !isValid ) { responseIsValid = false; // update validity validities[i] = validity; if (validity == null) responseIsUsable = false; } else { i++; } } if ( responseIsValid ) { // we are valid, ok that's it cachedValidityObjects = validities; if (this.getLogger().isDebugEnabled()) { this.getLogger().debug("Using valid cached content for '" + environment.getURI() + "'."); } } else { if (this.getLogger().isDebugEnabled()) { this.getLogger().debug("Cached content is invalid for '" + environment.getURI() + "'."); } // we are not valid! this.completeResponseIsCached = false; finished = false; this.cachedResponse = null; if (!responseIsUsable) { // we could compare, because we got no // validity object, so shorten pipeline key if (i > 0) { int deleteCount = validities.length - i; if (i > 0 && i <= firstNotCacheableTransformerIndex + 1) { this.firstNotCacheableTransformerIndex = i-1; } for(int x=0; x < deleteCount; x++) { processingPipelineKey.removeLastKey(); } finished = false; } else { processingPipelineKey = null; } this.cacheCompleteResponse = false; } else { // the entry is invalid, remove it this.cache.remove( cachedPipelineKey ); } // try a shorter key if (i > 0) { cachedPipelineKey.removeLastKey(); this.firstProcessedTransformerIndex--; } else { cachedPipelineKey = null; } } } else { if (this.getLogger().isDebugEnabled()) { this.getLogger().debug("Cached response not found for '" + environment.getURI() + "'."); } finished = false; this.completeResponseIsCached = false; // try a shorter key // FIXME (CZ) calculation of PPK if (cachedPipelineKey.size() > 1) { cachedPipelineKey.removeLastKey(); this.firstProcessedTransformerIndex--; } else { cachedPipelineKey = null; } } } // now generate validity objects for the new response if (processingPipelineKey != null) { if (cachedPipelineKey == null || cachedPipelineKey.size() < processingPipelineKey.size()) { this.pipelineCacheKey = processingPipelineKey; this.pipelineValidityObjects = new SourceValidity[processingPipelineKey.size()]; int start = 0; if (cachedPipelineKey != null) { start = cachedPipelineKey.size(); for(int i=0; i<start;i++) { this.pipelineValidityObjects[i] = cachedValidityObjects[i]; } } int len = this.pipelineValidityObjects.length; int i = start; while (i < len) { final SourceValidity validity; if (i == 0) { // test generator if (generatorIsCacheableProcessingComponent) { validity = ((CacheableProcessingComponent)super.generator).generateValidity(); } else { validity = CacheValidityToSourceValidity.createValidity(((Cacheable)super.generator).generateValidity()); } } else if (i <= firstNotCacheableTransformerIndex) { // test transformer final Transformer trans = (Transformer)super.transformers.get(i-1); if (transformerIsCacheableProcessingComponent[i-1]) { validity = ((CacheableProcessingComponent)trans).generateValidity(); } else { validity = CacheValidityToSourceValidity.createValidity(((Cacheable)trans).generateValidity()); } } else { // test serializer if (serializerIsCacheableProcessingComponent) { validity = ((CacheableProcessingComponent)super.serializer).generateValidity(); } else { validity = CacheValidityToSourceValidity.createValidity(((Cacheable)super.serializer).generateValidity()); } } if (validity == null) { if (i > 0 && i > cachedPipelineKey.size()) { // shorten key for(int m=i; m < this.pipelineValidityObjects.length; m++) { this.pipelineCacheKey.removeLastKey(); } SourceValidity[] copy = new SourceValidity[i]; System.arraycopy(this.pipelineValidityObjects, 0, copy, 0, copy.length); this.pipelineValidityObjects = copy; len = this.pipelineValidityObjects.length; } else { this.pipelineCacheKey = null; this.pipelineValidityObjects = null; len = 0; } } else { this.pipelineValidityObjects[i] = validity; } i++; } } } } /** * Connect the pipeline. */ protected void connectPipeline(Environment environment) throws ProcessingException { if ( this.pipelineCacheKey == null && this.cachedResponse == null) { super.connectPipeline( environment ); return; } else if (this.completeResponseIsCached) { // do nothing return; } else { this.connectCachingPipeline(environment); } } /** * Return valid validity objects for the event pipeline * If the "event pipeline" (= the complete pipeline without the * serializer) is cacheable and valid, return all validity objects. * Otherwise return <code>null</code> */ public SourceValidity[] getValiditiesForEventPipeline() { if (!this.completeResponseIsCached && this.firstNotCacheableTransformerIndex == super.transformers.size()) { return this.pipelineValidityObjects; } return null; } /** * Recyclable Interface */ public void recycle() { super.recycle(); this.manager.release( this.xmlDeserializer ); this.xmlDeserializer = null; this.manager.release( this.xmlSerializer ); this.xmlSerializer = null; this.generatorRole = null; this.transformerRoles.clear(); this.serializerRole = null; this.readerRole = null; this.pipelineCacheKey = null; this.pipelineValidityObjects = null; this.cachedResponse = null; } /** * Disposable Interface */ public void dispose() { this.manager.release(this.cache); this.cache = null; this.manager = null; } } 1.1 xml-cocoon2/src/java/org/apache/cocoon/components/pipeline/impl/CachingPointProcessingPipeline.java Index: CachingPointProcessingPipeline.java =================================================================== /* ============================================================================ The Apache Software License, Version 1.1 ============================================================================ Copyright (C) 1999-2002 The Apache Software Foundation. All rights reserved. Redistribution and use in source and binary forms, with or without modifica- tion, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. The end-user documentation included with the redistribution, if any, must include the following acknowledgment: "This product includes software developed by the Apache Software Foundation (http://www.apache.org/)." Alternately, this acknowledgment may appear in the software itself, if and wherever such third-party acknowledgments normally appear. 4. The names "Apache Cocoon" and "Apache Software Foundation" must not be used to endorse or promote products derived from this software without prior written permission. For written permission, please contact [EMAIL PROTECTED] 5. Products derived from this software may not be called "Apache", nor may "Apache" appear in their name, without prior written permission of the Apache Software Foundation. THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLU- DING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. This software consists of voluntary contributions made by many individuals on behalf of the Apache Software Foundation and was originally created by Stefano Mazzocchi <[EMAIL PROTECTED]>. For more information on the Apache Software Foundation, please see <http://www.apache.org/>. */ package org.apache.cocoon.components.pipeline.impl; import org.apache.avalon.framework.component.ComponentException; import org.apache.avalon.framework.parameters.Parameters; import org.apache.cocoon.ProcessingException; import org.apache.cocoon.caching.Cache; import org.apache.cocoon.caching.Cacheable; import org.apache.cocoon.caching.CacheableProcessingComponent; import org.apache.cocoon.caching.CachedResponse; import org.apache.cocoon.caching.CachingOutputStream; import org.apache.cocoon.caching.ComponentCacheKey; import org.apache.cocoon.caching.PipelineCacheKey; import org.apache.cocoon.components.pipeline.AbstractProcessingPipeline; import org.apache.cocoon.components.sax.XMLDeserializer; import org.apache.cocoon.components.sax.XMLSerializer; import org.apache.cocoon.components.sax.XMLTeePipe; import org.apache.cocoon.environment.Environment; import org.apache.cocoon.transformation.Transformer; import org.apache.cocoon.xml.XMLConsumer; import org.apache.cocoon.xml.XMLProducer; import org.apache.excalibur.source.SourceValidity; import java.io.OutputStream; import java.io.Serializable; import java.util.ArrayList; import java.util.HashMap; import java.util.Iterator; import java.util.ListIterator; import java.util.Map; /** * The CachingPointProcessingPipeline * * @since 2.1 * @author <a href="mailto:[EMAIL PROTECTED]">Michael Melhem</a> * @version CVS $Id: CachingPointProcessingPipeline.java,v 1.1 2002/07/15 08:17:28 cziegeler Exp $ */ public class CachingPointProcessingPipeline extends AbstractCachingProcessingPipeline { protected ArrayList isCachePoint = new ArrayList(); protected ArrayList xmlSerializerArray = new ArrayList(); protected boolean nextIsCachePoint = false; /** * Add a transformer. */ public void addTransformer (String role, String source, Parameters param) throws ProcessingException { super.addTransformer(role, source, param); // add caching point flag // default value is false this.isCachePoint.add(new Boolean(this.nextIsCachePoint)); this.nextIsCachePoint = false; } /** * Inform the pipeline that we have come across * a branch point in the tree */ public void informBranchPoint() { // // Set cachepoint flag whenever we come across a branchpoint // this.nextIsCachePoint = true; if (this.getLogger().isDebugEnabled()) { this.getLogger().debug("Informed Pipeline of branch point"); } } /** * Cache longest cachable path plus cache points. */ protected void cacheResults(Environment environment, OutputStream os) throws Exception { if (this.pipelineCacheKey != null) { if ( this.cacheCompleteResponse ) { if (this.getLogger().isDebugEnabled()) { this.getLogger().debug("Cached: caching complete response; pSisze" + this.pipelineCacheKey.size() + " Key " + this.pipelineCacheKey); } CachedResponse response = new CachedResponse(this.pipelineValidityObjects, ((CachingOutputStream)os).getContent()); this.cache.store(environment.getObjectModel(), this.pipelineCacheKey.copy(), response); // // Scan back along the pipelineCacheKey for // for any cachepoint(s) // this.pipelineCacheKey.removeUntilCachePoint(); // // adjust the validities object // to reflect the new length of the pipeline cache key. // // REVISIT: Is it enough to simply reduce the length of the validities array? // if (this.pipelineCacheKey.size()>0) { SourceValidity[] copy = new SourceValidity[this.pipelineCacheKey.size()]; System.arraycopy(this.pipelineValidityObjects, 0, copy, 0, copy.length); this.pipelineValidityObjects = copy; } } if (this.pipelineCacheKey.size()>0) { ListIterator itt = this.xmlSerializerArray.listIterator(this.xmlSerializerArray.size()); while (itt.hasPrevious()) { XMLSerializer serializer = (XMLSerializer) itt.previous(); CachedResponse response = new CachedResponse(this.pipelineValidityObjects, (byte[])serializer.getSAXFragment()); this.cache.store(environment.getObjectModel(), this.pipelineCacheKey.copy(), response); if (this.getLogger().isDebugEnabled()) { this.getLogger().debug("Caching results for the following key: " + this.pipelineCacheKey ); } // // Check for further cachepoints // pipelineCacheKey.removeUntilCachePoint(); if (this.pipelineCacheKey.size()==0) // no cachePoint found in key break; // // re-calculate validities array // SourceValidity[] copy = new SourceValidity[this.pipelineCacheKey.size()]; System.arraycopy(this.pipelineValidityObjects, 0, copy, 0, copy.length); this.pipelineValidityObjects = copy; } //end serializer loop } } } /** * Create a new ComponentCachekey * ComponentCacheKeys can be flagged as cachepoints */ protected ComponentCacheKey newComponentCacheKey(int type, String role,Serializable key) { boolean cachePoint = false; if (type == ComponentCacheKey.ComponentType_Transformer) { cachePoint = ((Boolean)this.isCachePoint.get(this.firstNotCacheableTransformerIndex)).booleanValue(); } else if (type == ComponentCacheKey.ComponentType_Serializer) { cachePoint = this.nextIsCachePoint; } return new ComponentCacheKey(type, role, key, cachePoint); } /** * Connect the caching point pipeline. */ protected void connectCachingPipeline(Environment environment) throws ProcessingException { try { XMLSerializer localXMLSerializer = null; XMLSerializer cachePointXMLSerializer = null; if (!this.cacheCompleteResponse) { this.xmlSerializer = (XMLSerializer)this.manager.lookup( XMLSerializer.ROLE ); localXMLSerializer = this.xmlSerializer; } if ( this.cachedResponse == null ) { XMLProducer prev = super.generator; XMLConsumer next; int cacheableTransformerCount = this.firstNotCacheableTransformerIndex; int currentTransformerIndex = 0; //start with the first transformer Iterator itt = this.transformers.iterator(); while ( itt.hasNext() ) { next = (XMLConsumer) itt.next(); // if we have cacheable transformers, // check the tranformers for cachepoints if (cacheableTransformerCount > 0) { if ( (this.isCachePoint.get(currentTransformerIndex) != null) && ((Boolean)this.isCachePoint.get(currentTransformerIndex)).booleanValue()) { cachePointXMLSerializer = ((XMLSerializer) this.manager.lookup( XMLSerializer.ROLE )); next = new XMLTeePipe(next, cachePointXMLSerializer); this.xmlSerializerArray.add(cachePointXMLSerializer); } } // Serializer is not cachable, // but we have the longest cachable key. Do default longest key caching if (localXMLSerializer != null) { if (cacheableTransformerCount == 0) { next = new XMLTeePipe(next, localXMLSerializer); this.xmlSerializerArray.add(localXMLSerializer); localXMLSerializer = null; } else { cacheableTransformerCount--; } } this.connect(environment, prev, next); prev = (XMLProducer) next; currentTransformerIndex++; } next = super.lastConsumer; // if the serializer is not cachable, but all the transformers are: // (this is default longest key caching) if (localXMLSerializer != null) { next = new XMLTeePipe(next, localXMLSerializer); this.xmlSerializerArray.add(localXMLSerializer); localXMLSerializer = null; } // else if the serializer is cachable and has cocoon views else if ((currentTransformerIndex == this.firstNotCacheableTransformerIndex) && this.nextIsCachePoint) { cachePointXMLSerializer = ((XMLSerializer)this.manager.lookup( XMLSerializer.ROLE )); next = new XMLTeePipe(next, cachePointXMLSerializer); this.xmlSerializerArray.add(cachePointXMLSerializer); } this.connect(environment, prev, next); } else { // Here the first part of the pipeline has been retrived from cache // we now check if any part of the rest of the pipeline can be cached this.xmlDeserializer = (XMLDeserializer)this.manager.lookup(XMLDeserializer.ROLE); // connect the pipeline: XMLProducer prev = xmlDeserializer; XMLConsumer next; int cacheableTransformerCount = 0; Iterator itt = this.transformers.iterator(); while ( itt.hasNext() ) { next = (XMLConsumer) itt.next(); if (cacheableTransformerCount >= this.firstProcessedTransformerIndex) { // if we have cacheable transformers left, // then check the tranformers for cachepoints if (cacheableTransformerCount < this.firstNotCacheableTransformerIndex) { if ( !(prev instanceof XMLDeserializer) && (this.isCachePoint.get(cacheableTransformerCount) != null) && ((Boolean)this.isCachePoint.get(cacheableTransformerCount)).booleanValue()) { cachePointXMLSerializer = ((XMLSerializer)this.manager.lookup( XMLSerializer.ROLE )); next = new XMLTeePipe(next, cachePointXMLSerializer); this.xmlSerializerArray.add(cachePointXMLSerializer); } } // Serializer is not cachable, // but we have the longest cachable key. Do default longest key caching if (localXMLSerializer != null && !(prev instanceof XMLDeserializer) && cacheableTransformerCount == this.firstNotCacheableTransformerIndex) { next = new XMLTeePipe(next, localXMLSerializer); this.xmlSerializerArray.add(localXMLSerializer); localXMLSerializer = null; } this.connect(environment, prev, next); prev = (XMLProducer)next; } cacheableTransformerCount++; } next = super.lastConsumer; //*all* the transformers are cachable, but the serializer is not!! this is longest key if (localXMLSerializer != null && !(prev instanceof XMLDeserializer)) { next = new XMLTeePipe(next, localXMLSerializer); this.xmlSerializerArray.add(localXMLSerializer); localXMLSerializer = null; } // else the serializer is cachable but has views else if (this.nextIsCachePoint && !(prev instanceof XMLDeserializer) && cacheableTransformerCount == this.firstNotCacheableTransformerIndex) { cachePointXMLSerializer = ((XMLSerializer)this.manager.lookup( XMLSerializer.ROLE )); next = new XMLTeePipe(next, cachePointXMLSerializer); this.xmlSerializerArray.add(cachePointXMLSerializer); } this.connect(environment, prev, next); } } catch ( ComponentException e ) { throw new ProcessingException("Could not connect pipeline.", e); } } /** Process the pipeline using a reader. * @throws ProcessingException if an error occurs */ protected boolean processReader(Environment environment) throws ProcessingException { if (this.getLogger().isDebugEnabled()) { this.getLogger().debug("Readers not yet supported for CachingPoint Pipelines"); } return true; } /** * Recyclable Interface */ public void recycle() { super.recycle(); Iterator itt = this.xmlSerializerArray.iterator(); while (itt.hasNext()) { this.manager.release((XMLSerializer) itt.next()); } this.xmlSerializerArray.clear(); this.nextIsCachePoint = false; } } 1.5 +5 -1 xml-cocoon2/src/java/org/apache/cocoon/components/treeprocessor/sitemap/GenerateNode.java Index: GenerateNode.java =================================================================== RCS file: /home/cvs/xml-cocoon2/src/java/org/apache/cocoon/components/treeprocessor/sitemap/GenerateNode.java,v retrieving revision 1.4 retrieving revision 1.5 diff -u -r1.4 -r1.5 --- GenerateNode.java 24 Jun 2002 20:32:19 -0000 1.4 +++ GenerateNode.java 15 Jul 2002 08:17:29 -0000 1.5 @@ -108,6 +108,10 @@ // Check view if (this.views != null) { + + //inform the pipeline that we have a branch point + context.getProcessingPipeline().informBranchPoint(); + String cocoonView = env.getView(); if (cocoonView != null) { 1.3 +5 -1 xml-cocoon2/src/java/org/apache/cocoon/components/treeprocessor/sitemap/SerializeNode.java Index: SerializeNode.java =================================================================== RCS file: /home/cvs/xml-cocoon2/src/java/org/apache/cocoon/components/treeprocessor/sitemap/SerializeNode.java,v retrieving revision 1.2 retrieving revision 1.3 diff -u -r1.2 -r1.3 --- SerializeNode.java 27 May 2002 13:14:52 -0000 1.2 +++ SerializeNode.java 15 Jul 2002 08:17:29 -0000 1.3 @@ -101,6 +101,10 @@ // Check view if (this.views != null) { + + //inform the pipeline that we have a branch point + context.getProcessingPipeline().informBranchPoint(); + String cocoonView = env.getView(); if (cocoonView != null) { 1.4 +5 -1 xml-cocoon2/src/java/org/apache/cocoon/components/treeprocessor/sitemap/TransformNode.java Index: TransformNode.java =================================================================== RCS file: /home/cvs/xml-cocoon2/src/java/org/apache/cocoon/components/treeprocessor/sitemap/TransformNode.java,v retrieving revision 1.3 retrieving revision 1.4 diff -u -r1.3 -r1.4 --- TransformNode.java 24 Jun 2002 20:32:19 -0000 1.3 +++ TransformNode.java 15 Jul 2002 08:17:29 -0000 1.4 @@ -105,6 +105,10 @@ // Check view if (this.views != null) { + + //inform the pipeline that we have a branch point + context.getProcessingPipeline().informBranchPoint(); + String cocoonView = env.getView(); if (cocoonView != null) { 1.6 +2 -2 xml-cocoon2/src/java/org/apache/cocoon/webapps/portal/components/PortalManager.java Index: PortalManager.java =================================================================== RCS file: /home/cvs/xml-cocoon2/src/java/org/apache/cocoon/webapps/portal/components/PortalManager.java,v retrieving revision 1.5 retrieving revision 1.6 diff -u -r1.5 -r1.6 --- PortalManager.java 1 Jul 2002 08:38:46 -0000 1.5 +++ PortalManager.java 15 Jul 2002 08:17:29 -0000 1.6 @@ -189,13 +189,13 @@ * Avalon Recyclable Interface */ public void recycle() { - super.recycle(); if (this.manager != null) { this.manager.release(this.profileStore); this.manager.release(this.authenticationManager); this.profileStore = null; this.authenticationManager = null; } + super.recycle(); } /** 1.71 +1 -0 xml-cocoon2/src/webapp/sitemap.xmap Index: sitemap.xmap =================================================================== RCS file: /home/cvs/xml-cocoon2/src/webapp/sitemap.xmap,v retrieving revision 1.70 retrieving revision 1.71 diff -u -r1.70 -r1.71 --- sitemap.xmap 10 Jul 2002 08:31:57 -0000 1.70 +++ sitemap.xmap 15 Jul 2002 08:17:29 -0000 1.71 @@ -312,6 +312,7 @@ --> <map:pipelines default="caching"> <map:pipeline name="caching" src="org.apache.cocoon.components.pipeline.impl.CachingProcessingPipeline"/> + <map:pipeline name="cachingpoint" src="org.apache.cocoon.components.pipeline.impl.CachingPointProcessingPipeline"/> <map:pipeline name="noncaching" src="org.apache.cocoon.components.pipeline.impl.NonCachingProcessingPipeline"/> <!-- The following two can be used for profiling: <map:pipeline name="profile-caching" src="org.apache.cocoon.components.profiler.ProfilingCachingProcessingPipeline"/>
---------------------------------------------------------------------- In case of troubles, e-mail: [EMAIL PROTECTED] To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]