On 17/05/2010, at 23:24, Tim Williams wrote: > Hi Thorsten, is this working for you locally? I noticed the > Forrestbot failure just now and I'm also failing locally (with a NPE) > on a dispatcher sample site. Was thinking it might be related but > don't see anything obvious in here...
Hmm, really weird. I did a quick debug and it seems the problem is in <xsl:include href="cocoon://prepare.contract.html.helper-render-image" /> If you comment this line then it will work again. I ATM not sure what happens and am flying out in a couple of hours, I will try to have a look again tomorrow but if you want you can revert the commit or comment the contract in the structurer. salu2 > > --tim > > On Mon, May 17, 2010 at 1:42 PM, <thors...@apache.org> wrote: >> Author: thorsten >> Date: Mon May 17 17:42:05 2010 >> New Revision: 945269 >> >> URL: http://svn.apache.org/viewvc?rev=945269&view=rev >> Log: >> FOR-1194 Fixing utf-8 compability by forcing to use UTF-8 in every step >> >> Modified: >> >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap >> >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf >> >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java >> >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java >> >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java >> >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java >> >> Modified: >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap >> URL: >> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap?rev=945269&r1=945268&r2=945269&view=diff >> ============================================================================== >> --- >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap >> (original) >> +++ >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap >> Mon May 17 17:42:05 2010 >> @@ -22,7 +22,7 @@ xmlns:map="http://apache.org/cocoon/site >> <map:pipeline id="lm"> >> <map:match pattern="locationmap.xml"> >> <map:generate src="locationmap.xml" /> >> - <map:serialize type="xml" /> >> + <map:serialize/> >> </map:match> >> </map:pipeline> >> <map:pipeline id="dispatcher"> >> @@ -61,7 +61,7 @@ xmlns:map="http://apache.org/cocoon/site >> </map:transform> >> <map:transform src="lm://hooks-to-fo.xsl" /> >> <map:transform src="lm://strip-dispatcher-remains-fo.xsl" /> >> - <map:serialize type="xml" /> >> + <map:serialize/> >> </map:match> >> <map:match pattern="**.prepare.dispatcher.css"> >> <map:generate src="lm://resolve.structurer.{1}" type="jx"> >> @@ -93,11 +93,11 @@ xmlns:map="http://apache.org/cocoon/site >> <map:act type="locale"> >> <map:match pattern="resolve.structurer.**"> >> <map:generate src="lm://resolve.structurer.{1}" /> >> - <map:serialize type="xml" /> >> + <map:serialize/> >> </map:match> >> <map:match pattern="resolve.contract.*.**"> >> <map:generate src="{lm:resolve.contract.{1}.{2}}" /> >> - <map:serialize type="xml" /> >> + <map:serialize/> >> </map:match> >> <map:match pattern="prepare.contract.*.**"> >> <map:generate src="{lm:resolve.contract.{1}.{2}}" /> >> @@ -105,7 +105,7 @@ xmlns:map="http://apache.org/cocoon/site >> <map:transform type="i18n"> >> <map:parameter name="locale" value="{../locale}" /> >> </map:transform> >> - <map:serialize type="xml" /> >> + <map:serialize/> >> </map:match> >> </map:act> >> </map:pipeline> >> @@ -116,7 +116,7 @@ xmlns:map="http://apache.org/cocoon/site >> <map:match pattern="prepare.panels.**"> >> <map:generate src="{lm:resolve.panels.{1}}" /> >> <map:transform src="{lm:root-strip.xsl}" /> >> - <map:serialize type="xml" /> >> + <map:serialize/> >> </map:match> >> </map:pipeline> >> <map:pipeline> >> @@ -126,7 +126,7 @@ xmlns:map="http://apache.org/cocoon/site >> <map:parameter name="path" value="{1}.html" /> >> <map:parameter name="theme" value="{global:dispatcher.theme}" /> >> </map:transform> >> - <map:serialize type="xml" /> >> + <map:serialize/> >> </map:match> >> </map:pipeline> >> <map:pipeline id="resources"> >> >> Modified: >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf >> URL: >> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf?rev=945269&r1=945268&r2=945269&view=diff >> ============================================================================== >> --- >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf >> (original) >> +++ >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf >> Mon May 17 17:42:05 2010 >> @@ -28,7 +28,7 @@ limitations under the License. >> <cache-at-startup>true</cache-at-startup> >> </map:transformer> >> </map:transformers> >> - <map:serializers default="xhtml"> >> + <map:serializers default="xml-utf8"> >> <map:serializer logger="sitemap.serializer.xhtml" >> mime-type="text/html; charset=UTF-8" name="xhtml" ppool-max="64" >> src="org.apache.cocoon.serialization.XMLSerializer"> >> @@ -57,5 +57,9 @@ limitations under the License. >> <encoding>UTF-8</encoding> >> <indent>true</indent> >> </map:serializer> >> - </map:serializers> >> + <map:serializer name="xml-utf8" mime-type="text/xml" >> + src="org.apache.cocoon.serialization.XMLSerializer"> >> + <encoding>UTF-8</encoding> >> + </map:serializer> >> + </map:serializers> >> </map:components> >> >> Modified: >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java >> URL: >> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java?rev=945269&r1=945268&r2=945269&view=diff >> ============================================================================== >> --- >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java >> (original) >> +++ >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java >> Mon May 17 17:42:05 2010 >> @@ -21,6 +21,8 @@ import java.io.BufferedOutputStream; >> import java.io.ByteArrayInputStream; >> import java.io.ByteArrayOutputStream; >> import java.io.InputStream; >> +import java.io.OutputStreamWriter; >> +import java.io.UnsupportedEncodingException; >> import java.util.Map; >> >> import javax.xml.stream.XMLStreamException; >> @@ -105,17 +107,23 @@ public class XSLContract extends Loggabl >> } >> } >> ByteArrayOutputStream out = new ByteArrayOutputStream(); >> + byte[] bytes = null; >> + String utf8 = ""; >> // create a StreamResult and use it for the transformation >> - Result streamResult = new StreamResult(new BufferedOutputStream(out)); >> try { >> - helper.transform(dataStream,streamResult); >> + OutputStreamWriter writer = new OutputStreamWriter(out,"UTF-8"); >> + Result streamResult = new StreamResult(writer); >> + helper.transform(dataStream,streamResult); >> + utf8 = out.toString("UTF-8"); >> + log.debug(utf8); >> + bytes = utf8.getBytes("utf-8"); >> } catch (Exception e) { >> String message = "Could not invoke the transformation for " >> + "the contract \""+name+"\". "+"\n"+ e; >> throw new ContractException(message); >> } >> - log.debug(out.toString()); >> - return new BufferedInputStream(new >> ByteArrayInputStream(out.toByteArray())); >> + >> + return new BufferedInputStream(new ByteArrayInputStream(bytes)); >> } >> >> /* (non-Javadoc) >> >> Modified: >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java >> URL: >> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java?rev=945269&r1=945268&r2=945269&view=diff >> ============================================================================== >> --- >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java >> (original) >> +++ >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java >> Mon May 17 17:42:05 2010 >> @@ -20,6 +20,7 @@ import java.io.BufferedInputStream; >> import java.io.ByteArrayInputStream; >> import java.io.IOException; >> import java.io.InputStream; >> +import java.io.InputStreamReader; >> import java.io.PrintWriter; >> import java.util.Iterator; >> import java.util.Map; >> @@ -124,6 +125,7 @@ public class XSLContractHelper extends L >> SAXException, IOException { >> // prepare transformation >> transformer = transFact.newTransformer(xslSource); >> + transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8"); >> transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes"); >> transformer.setOutputProperty(OutputKeys.INDENT, "yes"); >> transformer.setOutputProperty(OutputKeys.METHOD, "xml"); >> @@ -220,8 +222,8 @@ public class XSLContractHelper extends L >> } >> // Setting the necessary attributes of the contract >> // xsl >> - contract.setXslSource(new StreamSource(new ByteArrayInputStream(handler >> - .getBytes()))); >> + byte[] bytes = handler.getBytes(); >> + contract.setXslSource(new StreamSource(new >> ByteArrayInputStream(bytes))); >> /* DEBUG_CODE: >> * The following is useful to see what is going on >> * >> @@ -254,7 +256,9 @@ public class XSLContractHelper extends L >> throws ContractException { >> //Source dataSource = new StreamSource(dataStream); >> try { >> - SAXSource saxSource = new SAXSource(xmlReader,new >> InputSource(dataStream)); >> + InputSource inputSource = new InputSource(new >> InputStreamReader(dataStream, "UTF-8")); >> + inputSource.setEncoding("UTF-8"); >> + SAXSource saxSource = new SAXSource(xmlReader,inputSource); >> transformer.transform(saxSource, streamResult); >> } catch (Exception e) { >> String message = "The xsl transformation has thrown an exception. for " >> >> Modified: >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java >> URL: >> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java?rev=945269&r1=945268&r2=945269&view=diff >> ============================================================================== >> --- >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java >> (original) >> +++ >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java >> Mon May 17 17:42:05 2010 >> @@ -20,6 +20,7 @@ import java.io.BufferedInputStream; >> import java.io.File; >> import java.io.IOException; >> import java.io.InputStream; >> +import java.io.InputStreamReader; >> import java.io.Serializable; >> import java.io.StringReader; >> import java.util.HashMap; >> @@ -490,7 +491,7 @@ public class DispatcherTransformer exten >> throw new SAXException(e); >> } >> if (null != property) { >> - localParams.put(currentProperty, property.getBytes()); >> + localParams.put(currentProperty, property.getBytes("UTF-8")); >> } >> } >> } >> @@ -695,7 +696,7 @@ public class DispatcherTransformer exten >> root.serialize(out); >> } >> >> - InputSource is = new InputSource(new StringReader(out.toString())); >> + InputSource is = new InputSource(new >> StringReader(out.toString("UTF-8"))); >> // adding the result to the consumer >> parser.parse(is, super.xmlConsumer); >> } catch (Exception e) { >> @@ -1099,8 +1100,9 @@ public class DispatcherTransformer exten >> */ >> private static InputSource getInputSource(final Source source) >> throws IOException, SourceException { >> - final InputSource newObject = new InputSource(source.getInputStream()); >> - newObject.setSystemId(source.getURI()); >> - return newObject; >> + final InputSource inputSource = new InputSource(new >> InputStreamReader(source.getInputStream(), "UTF-8")); >> + inputSource.setEncoding("UTF-8"); >> + inputSource.setSystemId(source.getURI()); >> + return inputSource; >> } >> } >> >> Modified: >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java >> URL: >> http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java?rev=945269&r1=945268&r2=945269&view=diff >> ============================================================================== >> --- >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java >> (original) >> +++ >> forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java >> Mon May 17 17:42:05 2010 >> @@ -374,7 +374,7 @@ public class DispatcherWrapperTransforme >> } >> // get the result of the structurer as stream >> InputStream result = structurer.execute(new BufferedInputStream( >> - new ByteArrayInputStream(document.getBytes())), requestedFormat); >> + new ByteArrayInputStream(document.getBytes("UTF-8"))), >> requestedFormat); >> // requesting a parser >> parser = (SAXParser) manager.lookup(SAXParser.ROLE); >> // adding the result to the consumer >> >> >> Thorsten Scherler <thorsten.at.apache.org> Open Source Java <consulting, training and solutions>
