Re: xpathscript utf-8 character encoding problem

[Matthew Smith]

I have a little test script and file to demonstrate this problem - hopefully other perl 5.8.0 users will see this too (otherwise maybe it will help someone see where I've gone wrong).

I have attached three files, the demo XML file in utf-8 encoding, the test script which uses XPath and then does a var Dump on a few different variations of stringification of the XML::XPath::Literal object. I have attached the output I got on my system. The first two tests print a variable that has utf-8 data but doesn't have the utf-8 flag set - this causes the utf-8 to get converted to utf-8 again (mangled). The third test is where the stringification works ok - the utf8 flag is set somehow before printing.

I think this is what's happening in my XPathScript. Changing the binmode in my XPathScript makes no difference either (i have tried :raw :bytes and :utf8. (see test.xps)

matt

#!/usr/bin/perl

use strict;
use warnings;

use XML::XPath;
use Devel::Peek;

binmode(STDOUT, ":utf8");

my $xp = XML::XPath->new(filename => $ARGV[0]);

print "**** Test 1 ****\n";
my $var = $xp->findvalue('article/title');
print $var."\n";
Dump($var);

print "\n\n**** Test 2 ****\n";
my $var2 = "$var Other\n";
print $var2;
Dump($var2);

print "\n\n**** Test 3 ****\n";
my $var3 = "$var"."Other\n";
print $var3."\n";
Dump($var3);
1;

**** Test 1 ****
“What’s with this xml?”
SV = RV(0x83ca3d8) at 0x812cf00
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,ROK,OVERLOAD)
  RV = 0x817d4e4
  SV = PVMG(0x83deeb8) at 0x817d4e4
    REFCNT = 1
    FLAGS = (PADBUSY,PADMY,OBJECT,POK,pPOK,UTF8)
    IV = 0
    NV = 0
    PV = 0x83cec10 "\342\200\234What\342\200\231s with this xml?\342\200\235"\0 [UTF8 
"\x{201c}What\x{2019}s with this xml?\x{201d}"]
    CUR = 29
    LEN = 30
    STASH = 0x817d2d4   "XML::XPath::Literal"


**** Test 2 ****
“What’s with this xml?” Other
SV = PV(0x83c8c4c) at 0x83b6f88
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,POK,pPOK)
  PV = 0x83d0020 "\342\200\234What\342\200\231s with this xml?\342\200\235 Other\12"\0
  CUR = 36
  LEN = 37



**** Test 3 ****
“What’s with this xml?”Other
SV = PV(0x83c8c64) at 0x83b70a8
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,POK,pPOK,UTF8)
  PV = 0x83cefb0 "\342\200\234What\342\200\231s with this xml?\342\200\235Other\12"\0 
[UTF8 "\x{201c}What\x{2019}s with this xml?\x{201d}Other\n"]
  CUR = 35
  LEN = 36



<?xml version="1.0"?>

<?xml-stylesheet href="test.xps" type="application/x-xpathscript"?>

<article type="editorial">

  <title>“What’s with this xml?”</title>
</article>


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]