Re: Apache->print() problem with UTF-8 data in Perl 5.8

[Steve Hay]

Steve Hay wrote:

Randy Kobes wrote:

On Tue, 29 Jul 2003, Steve Hay wrote:

Is anybody going to take a serious look at the problem that I 
previously
reported with Apache->print()'s handling of UTF-8 data in Perl 5.8?

The patch that I sent
(http://marc.theaimsgroup.com/?l=apache-modperl&m=105912130001046&w=2)
seems to fix it for me on Windows as long as I've got perl #20203
incorporated.  Does it work on other platforms?
The #ifdef version-checks need a little work: on Windows (and other
platforms [-- which ones?] that rely on Perl's makedef.pl to get 
symbols
exported from the Perl library) you need perl-5.8.1; on other platforms
the test for perl-5.8.0 should be fine.  The brokenness of
Apache->print() under perl-5.8.0 on Windows et al would also need
documenting somewhere since it can't be fixed properly.


mod_perl 2 makes two special cases for handling systems that use
export files - Win32 and aix ($^O eq 'aix'). So these might
be the only two one has to worry about in this regard.
These were the only two that Nicholas Clark mentioned when I asked 
about the missing PerlIO_isutf8() on the perl-xs list, so that may 
well be true.  I will double-check with p5p to see if there are any more. 
Apparently there are a few more -- several listed in Perl's makedef.pl, 
plus VMS which evidently has a completely different build process.

The attached patch (against mod_perl 1.28) sorts out the version-check 
stuff, which was the only remaining issue that I had.  This leaves 
Apache->print() broken on Windows et al under Perl 5.8.0, and fixes it 
under Perl 5.8.0 on other platforms and under Perl 5.8.1+ on all platforms.

Steve
--- Apache.xs.orig      2003-06-06 12:31:10.000000000 +0100
+++ Apache.xs   2003-07-31 09:45:37.000000000 +0100
@@ -51,6 +51,16 @@
 #include "mod_perl.h"
 #include "mod_perl_xs.h"
 
+/* Figure out if we've got PerlIO_isutf8(). This is needed to handle UTF-8
+ * data correctly in Perl 5.8.0+, but was not exported by Perl in 5.8.0 itself
+ * on those OS's where symbols to export have to be explicitly listed. */
+#if !( PERL_REVISION < 5 || \
+      (PERL_REVISION == 5 && PERL_VERSION < 8) || \
+      (PERL_REVISION == 5 && PERL_VERSION == 8 && PERL_SUBVERSION == 0 && \
+       (defined(WIN32) || defined(NETWARE) || defined(OS2) || \
+        defined(_AIX) || defined(MACOS_TRADITIONAL) || defined(VMS))) )
+#define _HAVE_ISUTF8
+#endif
 
 #ifdef USE_SFIO
 #undef send_fd_length    
@@ -1119,11 +1129,27 @@
        SV *sv = sv_newmortal();
        SV *rp = ST(0);
        SV *sendh = perl_get_sv("Apache::__SendHeader", TRUE);
+#ifdef _HAVE_ISUTF8
+       PerlIO *fp = IoOFP(GvIOp(defoutgv));
+#endif
 
        if(items > 2)
            do_join(sv, &sv_no, MARK+1, SP); /* $sv = join '', @_[1..$#_] */
         else
            sv_setsv(sv, ST(1));
+#ifdef _HAVE_ISUTF8
+       if (PerlIO_isutf8(fp)) {
+           if (!SvUTF8(sv))
+               sv_utf8_upgrade(sv = sv_mortalcopy(sv));
+       }
+       else if (DO_UTF8(sv)) {
+           if (!sv_utf8_downgrade((sv = sv_mortalcopy(sv)), TRUE)
+               && ckWARN_d(WARN_UTF8))
+           {
+               Perl_warner(aTHX_ packWARN(WARN_UTF8), "Wide character in print");
+           }
+       }
+#endif
 
        PUSHMARK(sp);
        XPUSHs(rp);
@@ -1176,6 +1202,20 @@
        int sent = 0;
         SV *sv = SvROK(ST(i)) && (SvTYPE(SvRV(ST(i))) == SVt_PV) ?
                  (SV*)SvRV(ST(i)) : ST(i);
+#ifdef _HAVE_ISUTF8
+       PerlIO *fp = IoOFP(GvIOp(defoutgv));
+       if (PerlIO_isutf8(fp)) {
+           if (!SvUTF8(sv))
+               sv_utf8_upgrade(sv = sv_mortalcopy(sv));
+       }
+       else if (DO_UTF8(sv)) {
+           if (!sv_utf8_downgrade((sv = sv_mortalcopy(sv)), TRUE)
+               && ckWARN_d(WARN_UTF8))
+           {
+               Perl_warner(aTHX_ packWARN(WARN_UTF8), "Wide character in print");
+           }
+       }
+#endif
        buffer = SvPV(sv, len);
 #ifdef APACHE_SSL
         while(len > 0) {