RE: distorting special characters

[Josh Canfield]

When your XML document is parsed the ™ entity becomes the actual unicode 
character represented. The ? is generated String encoder trying to encode the 
Unicode character into the windows-1252. 

String content = new String(rawfileOutputStream.toByteArray(),"windows-1252");

If you tell the stylesheet that you are not going to output Unicode by changing 
the output elements encoding attribute to iso-8859-1 (close to windows-1252), 
and that your output method is xml, then characters outside of the iso-8859-1 
charset will be encoded on output. Since you want output escaping also remove 
the disable-output-escaping="yes" attribute from the xsl:value-of element.


<xsl:stylesheet version="1.0"
        xmlns:xsl="http://www.w3.org/1999/XSL/Transform";>

<xsl:output method="xml" encoding="iso-8859-1"/>

<xsl:template match="/product_metadata">

<content>

   <container>

    <xsl:value-of select="category/@name"/>

</container>

</content>

</xsl:template>

</xsl:stylesheet>


-----Original Message-----
From: Pramodh Peddi [mailto:[EMAIL PROTECTED]
Sent: Tuesday, November 04, 2003 8:55 AM
To: [EMAIL PROTECTED]
Subject: distorting special characters


Hi,
I am desparately looking for a solution for my problem.
This problem is occuring ONLY on UNIX machines. It is working fine on
Windows machines.
I am using Java1.4.1's Transformer to transform my xml files, which uses
Xalan-Java internally. The xml file
has &#8482; (TM symbol). And it spits out some funky chars in place of this
TM character. There are many such special chars in the source xml file (like
copyright, registered mark, etc).

In the application, I am trying to preserve the encoding. If I don't do
that, it is spitting out "?" marks in place of special chars.

I would appreciate any help. I am desperate for any advices and suggestions.

Source xml looks like this (i extracted a part of it to make it small):
****************source xml*********************
<content>
    <category name = "MavicaCLIE&#8482;"/>
</content>
*********************************************

This is what i get out of the transformation process:
---------------------------------------------------------------------
<content>
    <container>MavicaCLIEâÂ"¢</container>
</content>
--------------------------------------------------------------------

And the stylesheet looks like following:
=============================================
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform";>

<xsl:output method="text"/>

<xsl:template match="/product_metadata">

<content>

   <container>

    <xsl:value-of select="category/@name" disable-output-escaping="yes"/>

</container>

</content>

</xsl:template>

</xsl:stylesheet>

==============================================

This is what I am doing in the application (source xml file has
"windows-1252" encoding):
**********************************************************************
ByteArrayOutputStream rawfileOutputStream =

new ByteArrayOutputStream();


// get the file

filePath = (String) taxKeyIt.next();

file = (SftpFile) taxFileMap.get(filePath);

filePath = ServiceUtils.getFileNameFromURL(filePath);

if (filePath != null) {

sftp.get(filePath, rawfileOutputStream);

rawfileOutputStream.close();

}


String content = new String(rawfileOutputStream.toByteArray(),
"windows-1252");


log.info("initialContent: " + content);


content = this.removeDTDFromMetadata(content);


// transform the file

TransformerFactory tFactory = TransformerFactory.newInstance();

Transformer transformer = tFactory.newTransformer(new StreamSource(new
URL(this.taxXSLT).openStream()));


ByteArrayInputStream rawfileInputStream = new
ByteArrayInputStream(content.getBytes("windows-1252"));

ByteArrayOutputStream transformedFileOutputStream = new
ByteArrayOutputStream();


File transformedFile = new File("../server/ic/deploy/data.war/" +
this.taxXSLTResult);

FileOutputStream out = new FileOutputStream(transformedFile);


transformer.transform(

new StreamSource(new InputStreamReader(rawfileInputStream)),

new StreamResult(new OutputStreamWriter(out, "UTF-8")));

rawfileInputStream.close();

transformedFileOutputStream.close();

// move file up to data dir

}

****************************************************************************
***

Thanks,

pramodh.