[Templates] $tt->error drops utf8 flag

[Bill Moseley]

Template Toolkit version 2.23

This is a bit obscure, but the problem is TT is somehow turning off the
utf8 flag on $tt->error.

This can be a problem if trying to match text against the error returned
or, as in my case, sending the error to Log4perl, which (when configured)
will encode to utf8 as it should.

I'm wondering where and why TT is dropping the utf8 flag on the error
message.   It seems to happen when executing the complied template block.


Below I have simple code to demonstrate.

If the template generates an exception with wide-*character* data (e.g.
utf8 flag is on) the $tt->error string still comes back with the utf8 flag
OFF.

The example code below will generate an error like this -- note that the
utf8 flag is OFF:

Template ERROR [undef error - Chars are [Сарлаг хусуулах цаг - ထို yak
ရိတ်ဖို့အချိန်]
] flag is [OFF]


then sending that to log4perl will essentially run encode_utf8() on the
error message and corrupt it like this, which is what I'm seeing in my logs.


undef error - Chars are [Сарлаг Ñ…ÑƒÑ ÑƒÑƒÐ»Ð°Ñ… цаг - ထို
yak ရိဠ်ဖို့အဠျိန်]



Now, if I remove or comment out the "diesub" and let the template process
normally then I get the template output back with the utf8 flag set, like
one would expect.

Template output [This is my template Сарлаг хусуулах цаг - ထို yak
ရိတ်ဖို့အချိန်.
] flag is [ON]


So, it appears that when the complied template block is executed and an
exception happens the utf8 flag is getting cleared on $tt->error.






use strict;
use warnings;
use Template;
use Encode;

my $octets = 'Сарлаг хусуулах цаг - ထို yak ရိတ်ဖိုအချိန်';
my $char = decode_utf8( $octets );

my $tt = Template->new;


my $output;

$tt->process(
    \*DATA,
    {
        hello => $char,
        diesub => sub { die "Chars are [$char]\n" },
    },
    \$output
) || do {
    my $output = $tt->error;
    die printf "Template ERROR [%s] flag is [%s]\n",
        Encode::is_utf8( $output )
            ? ( encode_utf8( $output), 'ON' )
            : ($output, 'OFF');
};

printf "Template output [%s] flag is [%s]\n",
    Encode::is_utf8( $output )
        ? ( encode_utf8( $output), 'ON' )
        : ($output, 'OFF');


__END__
This is my template [% hello %].
[% diesub() %]



-- 
Bill Moseley
mose...@hank.org

_______________________________________________
templates mailing list
templates@template-toolkit.org
http://mail.template-toolkit.org/mailman/listinfo/templates