On 18/05/2010, at 01:59, Thorsten Scherler wrote: > > On 17/05/2010, at 23:24, Tim Williams wrote: > >> Hi Thorsten, is this working for you locally? I noticed the >> Forrestbot failure just now and I'm also failing locally (with a NPE) >> on a dispatcher sample site. Was thinking it might be related but >> don't see anything obvious in here... > > Hmm, really weird. I did a quick debug and it seems the problem is in > <xsl:include href="cocoon://prepare.contract.html.helper-render-image" /> > If you comment this line then it will work again. > > I ATM not sure what happens and am flying out in a couple of hours, I will > try to have a look again tomorrow but if you want you can revert the commit > or comment the contract in the structurer.
The problem seems to be <xsl:include href="cocoon://prepare.contract.html.element"></xsl:include> However I am not sure, need to go to bed now. salu2 > > salu2 > >> >> --tim >> >> On Mon, May 17, 2010 at 1:42 PM, <thors...@apache.org> wrote: >>> Author: thorsten >>> Date: Mon May 17 17:42:05 2010 >>> New Revision: 945269 >>> >>> URL: http://svn.apache.org/viewvc?rev=945269&view=rev >>> Log: >>> FOR-1194 Fixing utf-8 compability by forcing to use UTF-8 in every step >>> >>> Modified: >>> >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap >>> >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf >>> >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java >>> >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java >>> >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java >>> >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java >>> >>> Modified: >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap >>> URL: >>> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap?rev=945269&r1=945268&r2=945269&view=diff >>> ============================================================================== >>> --- >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap >>> (original) >>> +++ >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap >>> Mon May 17 17:42:05 2010 >>> @@ -22,7 +22,7 @@ xmlns:map="http://apache.org/cocoon/site >>> <map:pipeline id="lm"> >>> <map:match pattern="locationmap.xml"> >>> <map:generate src="locationmap.xml" /> >>> - <map:serialize type="xml" /> >>> + <map:serialize/> >>> </map:match> >>> </map:pipeline> >>> <map:pipeline id="dispatcher"> >>> @@ -61,7 +61,7 @@ xmlns:map="http://apache.org/cocoon/site >>> </map:transform> >>> <map:transform src="lm://hooks-to-fo.xsl" /> >>> <map:transform src="lm://strip-dispatcher-remains-fo.xsl" /> >>> - <map:serialize type="xml" /> >>> + <map:serialize/> >>> </map:match> >>> <map:match pattern="**.prepare.dispatcher.css"> >>> <map:generate src="lm://resolve.structurer.{1}" type="jx"> >>> @@ -93,11 +93,11 @@ xmlns:map="http://apache.org/cocoon/site >>> <map:act type="locale"> >>> <map:match pattern="resolve.structurer.**"> >>> <map:generate src="lm://resolve.structurer.{1}" /> >>> - <map:serialize type="xml" /> >>> + <map:serialize/> >>> </map:match> >>> <map:match pattern="resolve.contract.*.**"> >>> <map:generate src="{lm:resolve.contract.{1}.{2}}" /> >>> - <map:serialize type="xml" /> >>> + <map:serialize/> >>> </map:match> >>> <map:match pattern="prepare.contract.*.**"> >>> <map:generate src="{lm:resolve.contract.{1}.{2}}" /> >>> @@ -105,7 +105,7 @@ xmlns:map="http://apache.org/cocoon/site >>> <map:transform type="i18n"> >>> <map:parameter name="locale" value="{../locale}" /> >>> </map:transform> >>> - <map:serialize type="xml" /> >>> + <map:serialize/> >>> </map:match> >>> </map:act> >>> </map:pipeline> >>> @@ -116,7 +116,7 @@ xmlns:map="http://apache.org/cocoon/site >>> <map:match pattern="prepare.panels.**"> >>> <map:generate src="{lm:resolve.panels.{1}}" /> >>> <map:transform src="{lm:root-strip.xsl}" /> >>> - <map:serialize type="xml" /> >>> + <map:serialize/> >>> </map:match> >>> </map:pipeline> >>> <map:pipeline> >>> @@ -126,7 +126,7 @@ xmlns:map="http://apache.org/cocoon/site >>> <map:parameter name="path" value="{1}.html" /> >>> <map:parameter name="theme" value="{global:dispatcher.theme}" /> >>> </map:transform> >>> - <map:serialize type="xml" /> >>> + <map:serialize/> >>> </map:match> >>> </map:pipeline> >>> <map:pipeline id="resources"> >>> >>> Modified: >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf >>> URL: >>> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf?rev=945269&r1=945268&r2=945269&view=diff >>> ============================================================================== >>> --- >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf >>> (original) >>> +++ >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf >>> Mon May 17 17:42:05 2010 >>> @@ -28,7 +28,7 @@ limitations under the License. >>> <cache-at-startup>true</cache-at-startup> >>> </map:transformer> >>> </map:transformers> >>> - <map:serializers default="xhtml"> >>> + <map:serializers default="xml-utf8"> >>> <map:serializer logger="sitemap.serializer.xhtml" >>> mime-type="text/html; charset=UTF-8" name="xhtml" ppool-max="64" >>> src="org.apache.cocoon.serialization.XMLSerializer"> >>> @@ -57,5 +57,9 @@ limitations under the License. >>> <encoding>UTF-8</encoding> >>> <indent>true</indent> >>> </map:serializer> >>> - </map:serializers> >>> + <map:serializer name="xml-utf8" mime-type="text/xml" >>> + src="org.apache.cocoon.serialization.XMLSerializer"> >>> + <encoding>UTF-8</encoding> >>> + </map:serializer> >>> + </map:serializers> >>> </map:components> >>> >>> Modified: >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java >>> URL: >>> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java?rev=945269&r1=945268&r2=945269&view=diff >>> ============================================================================== >>> --- >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java >>> (original) >>> +++ >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java >>> Mon May 17 17:42:05 2010 >>> @@ -21,6 +21,8 @@ import java.io.BufferedOutputStream; >>> import java.io.ByteArrayInputStream; >>> import java.io.ByteArrayOutputStream; >>> import java.io.InputStream; >>> +import java.io.OutputStreamWriter; >>> +import java.io.UnsupportedEncodingException; >>> import java.util.Map; >>> >>> import javax.xml.stream.XMLStreamException; >>> @@ -105,17 +107,23 @@ public class XSLContract extends Loggabl >>> } >>> } >>> ByteArrayOutputStream out = new ByteArrayOutputStream(); >>> + byte[] bytes = null; >>> + String utf8 = ""; >>> // create a StreamResult and use it for the transformation >>> - Result streamResult = new StreamResult(new BufferedOutputStream(out)); >>> try { >>> - helper.transform(dataStream,streamResult); >>> + OutputStreamWriter writer = new OutputStreamWriter(out,"UTF-8"); >>> + Result streamResult = new StreamResult(writer); >>> + helper.transform(dataStream,streamResult); >>> + utf8 = out.toString("UTF-8"); >>> + log.debug(utf8); >>> + bytes = utf8.getBytes("utf-8"); >>> } catch (Exception e) { >>> String message = "Could not invoke the transformation for " >>> + "the contract \""+name+"\". "+"\n"+ e; >>> throw new ContractException(message); >>> } >>> - log.debug(out.toString()); >>> - return new BufferedInputStream(new >>> ByteArrayInputStream(out.toByteArray())); >>> + >>> + return new BufferedInputStream(new ByteArrayInputStream(bytes)); >>> } >>> >>> /* (non-Javadoc) >>> >>> Modified: >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java >>> URL: >>> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java?rev=945269&r1=945268&r2=945269&view=diff >>> ============================================================================== >>> --- >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java >>> (original) >>> +++ >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java >>> Mon May 17 17:42:05 2010 >>> @@ -20,6 +20,7 @@ import java.io.BufferedInputStream; >>> import java.io.ByteArrayInputStream; >>> import java.io.IOException; >>> import java.io.InputStream; >>> +import java.io.InputStreamReader; >>> import java.io.PrintWriter; >>> import java.util.Iterator; >>> import java.util.Map; >>> @@ -124,6 +125,7 @@ public class XSLContractHelper extends L >>> SAXException, IOException { >>> // prepare transformation >>> transformer = transFact.newTransformer(xslSource); >>> + transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8"); >>> transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes"); >>> transformer.setOutputProperty(OutputKeys.INDENT, "yes"); >>> transformer.setOutputProperty(OutputKeys.METHOD, "xml"); >>> @@ -220,8 +222,8 @@ public class XSLContractHelper extends L >>> } >>> // Setting the necessary attributes of the contract >>> // xsl >>> - contract.setXslSource(new StreamSource(new ByteArrayInputStream(handler >>> - .getBytes()))); >>> + byte[] bytes = handler.getBytes(); >>> + contract.setXslSource(new StreamSource(new >>> ByteArrayInputStream(bytes))); >>> /* DEBUG_CODE: >>> * The following is useful to see what is going on >>> * >>> @@ -254,7 +256,9 @@ public class XSLContractHelper extends L >>> throws ContractException { >>> //Source dataSource = new StreamSource(dataStream); >>> try { >>> - SAXSource saxSource = new SAXSource(xmlReader,new >>> InputSource(dataStream)); >>> + InputSource inputSource = new InputSource(new >>> InputStreamReader(dataStream, "UTF-8")); >>> + inputSource.setEncoding("UTF-8"); >>> + SAXSource saxSource = new SAXSource(xmlReader,inputSource); >>> transformer.transform(saxSource, streamResult); >>> } catch (Exception e) { >>> String message = "The xsl transformation has thrown an exception. for " >>> >>> Modified: >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java >>> URL: >>> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java?rev=945269&r1=945268&r2=945269&view=diff >>> ============================================================================== >>> --- >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java >>> (original) >>> +++ >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java >>> Mon May 17 17:42:05 2010 >>> @@ -20,6 +20,7 @@ import java.io.BufferedInputStream; >>> import java.io.File; >>> import java.io.IOException; >>> import java.io.InputStream; >>> +import java.io.InputStreamReader; >>> import java.io.Serializable; >>> import java.io.StringReader; >>> import java.util.HashMap; >>> @@ -490,7 +491,7 @@ public class DispatcherTransformer exten >>> throw new SAXException(e); >>> } >>> if (null != property) { >>> - localParams.put(currentProperty, property.getBytes()); >>> + localParams.put(currentProperty, property.getBytes("UTF-8")); >>> } >>> } >>> } >>> @@ -695,7 +696,7 @@ public class DispatcherTransformer exten >>> root.serialize(out); >>> } >>> >>> - InputSource is = new InputSource(new StringReader(out.toString())); >>> + InputSource is = new InputSource(new >>> StringReader(out.toString("UTF-8"))); >>> // adding the result to the consumer >>> parser.parse(is, super.xmlConsumer); >>> } catch (Exception e) { >>> @@ -1099,8 +1100,9 @@ public class DispatcherTransformer exten >>> */ >>> private static InputSource getInputSource(final Source source) >>> throws IOException, SourceException { >>> - final InputSource newObject = new InputSource(source.getInputStream()); >>> - newObject.setSystemId(source.getURI()); >>> - return newObject; >>> + final InputSource inputSource = new InputSource(new >>> InputStreamReader(source.getInputStream(), "UTF-8")); >>> + inputSource.setEncoding("UTF-8"); >>> + inputSource.setSystemId(source.getURI()); >>> + return inputSource; >>> } >>> } >>> >>> Modified: >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java >>> URL: >>> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java?rev=945269&r1=945268&r2=945269&view=diff >>> ============================================================================== >>> --- >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java >>> (original) >>> +++ >>> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java >>> Mon May 17 17:42:05 2010 >>> @@ -374,7 +374,7 @@ public class DispatcherWrapperTransforme >>> } >>> // get the result of the structurer as stream >>> InputStream result = structurer.execute(new BufferedInputStream( >>> - new ByteArrayInputStream(document.getBytes())), requestedFormat); >>> + new ByteArrayInputStream(document.getBytes("UTF-8"))), >>> requestedFormat); >>> // requesting a parser >>> parser = (SAXParser) manager.lookup(SAXParser.ROLE); >>> // adding the result to the consumer >>> >>> >>> > > Thorsten Scherler <thorsten.at.apache.org> > Open Source Java <consulting, training and solutions> > Thorsten Scherler <thorsten.at.apache.org> Open Source Java <consulting, training and solutions>
