Hi Oli,
> Anybody here has positive experience with Umlauts and can help out?
Interesting problem, and I am not sure why this happens. Originally we designed
OpenXPKI to be fully UTF8-compliant. I also know that it was used by our
Russian friends to some success, and they also kept a keen eye on keeping the
code UTF8-aware. However, I remember Sergei writing to the list that some
problems occurred with later commits regarding UTF8 support.
It also seems to be an annoying truth that the semantics of handling UTF8 in
Perl has changed significantly, possibly breaking things in OpenXPKI. Sergei
once mentioned that the changes were so deep that fixing the behaviour was not
really possible without breaking backwards compatibility with pre 5.10 releases.
The problem you are seeing with the serialization is very likely the result of
the difference between bytes and characters when encoded in UTF8. I know that
this used to work before.
It might also point into the direction of a bug in Perl (5.10?) itself:
In December 2011 I also encountered severe problems with a superfluous
UTF8-encoding applied to binary data in Oxi with perl 5.10. This is probably
unrelated, but might point in the right direction. Debian Squeeze uses the same
Perl version, possibly it is a side effect.
On the system I deployed (on Solaris 11 using Perl 5.10.1 from OpenCSW) I
observed the following: all data downloaded from the PKI system (e. g. PKCS#12,
CRLs and certificates in DER format) had an additional and superfluous UTF8
encoding applied. When manually decoding the downloaded binary file from UTF8
to latin1 the data was perfectly OK.
After hours of debugging I could pinpoint the exact position in the code where
this happens:
in trunk/perl-modules/core/trunk/OpenXPKI.pm, method read_file() you will find
my $result = do {
open my $HANDLE, "<", $filename;
...
With some telling comments about UTF8 stuff. These comments were inserted a
really long time ago and point to possible problems here. However what I
observed on this Perl 5.10.1 system was that the data in the input file
contained the correct data. However, the variable $result ended up with the
same data plus an additional, unnecessary UTF8 encoding. I literally tried
everything with this code snippet to stop Perl from doing this, including
fiddling with the UTF8 bit of the string, UTF8 mode for the input file,
downgrading the string etc.
I was completely unable to get the clean binary file data in the string.
Interestingly, when I isolated this code in a separate test program everything
worked fine. This tells me that something in the context of the OpenXPKI daemon
must be causing or introducing the problem. But I don't know what and where.
Regards,
Martin
------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
_______________________________________________
OpenXPKI-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/openxpki-devel
