Mark,
I sent around a patch long ago which fixes this problem:
http://www.template-toolkit.org/pipermail/templates/2001-April/000810.html
But as I said then, Stephen Adkins solution is much better than mine:
<<<
3. However, the previous messages were *very helpful* in putting me
onto the basic idea of breaking text into chunks of less than 12k
(or 8k). This led me to modify the
( (?: \\. | [^\$] )+ ) expression to
( (?: \\. | [^\$] ){1,4000} ). This limits the chunk of text
matched to between 4000 and 8000 chars (if it were solid escapes).
This solved my problem!
I made the change in "ptest" and the "ptest" script now works.
>>>
Attached is the subclass of Template::Parser that we use, here, which
happily interpolates really, really gigantic files all the time. (It
also makes the interpolation regex more greedy, so you may not want to
use it directly -- the important change is the one Stephen makes above,
to limit the amount of text processed per pass).
Hope this helps,
JP
On Thu, 2002-04-04 at 09:35, Mark Proctor wrote:
> I'm having the issue where a HTML file gets to a certain and you get the
> error:
> "Premature end of script headers"
>
> I read about the issue of 32K size limit (or less) for templates when
> INTERPOLATE option is set.
>
> However I find it has problems with files larger than 7500 - anything
> larger than the file below causes the error.
> -rw-rw-rw- 1 mproctor users 7594 Apr 4 16:29 search.html
> I believe this is only 7.5K
>
> Is there anything i can do about this? I have included my Perl -V info
> below and am using TT2.06
>
> Regards
>
> Mark
>
> Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration:
> Platform:
> osname=linux, osvers=2.4.9, archname=i586-linux
> uname='linux reiser 2.4.9 #1 smp wed sep 19 12:11:44 gmt 2001 i686
> unknown '
> config_args='-ds -e -Dprefix=/usr -Di_db -Di_dbm -Di_ndbm -Di_gdbm'
> hint=recommended, useposix=true, d_sigaction=define
> usethreads=undef use5005threads=undef useithreads=undef
> usemultiplicity=undef
> useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef
> use64bitint=undef use64bitall=undef uselongdouble=undef
> Compiler:
> cc='cc', ccflags ='-fno-strict-aliasing -I/usr/local/include
> -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
> optimize='-O2 -pipe',
> cppflags='-fno-strict-aliasing -I/usr/local/include'
> ccversion='', gccversion='2.95.3 20010315 (SuSE)', gccosandvers=''
> intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
> d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
> ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
> lseeksize=8
> alignbytes=4, usemymalloc=n, prototype=define
> Linker and Libraries:
> ld='cc', ldflags =' -L/usr/local/lib'
> libpth=/usr/local/lib /lib /usr/lib
> libs=-lnsl -ldl -lm -lc -lcrypt -lutil
> perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
> libc=, so=so, useshrplib=false, libperl=libperl.a
> Dynamic Linking:
> dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
> cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'
>
>
> Characteristics of this binary (from libperl):
> Compile-time options: USE_LARGE_FILES
> Built under linux
> Compiled at Sep 20 2001 02:28:43
> @INC:
> /usr/lib/perl5/5.6.1/i586-linux
> /usr/lib/perl5/5.6.1
> /usr/lib/perl5/site_perl/5.6.1/i586-linux
> /usr/lib/perl5/site_perl/5.6.1
> /usr/lib/perl5/site_perl
>
#============================================================
# Playboy::Template::Parser
#
# Our local subclass of TT2's Template::Parser, providing:
# * bugfix for segfault on large interpolations
# * interpolate on by default
# * our tags (<!--% -->) by default
#============================================================
package Playboy::Template::Parser;
use strict;
use Template::Parser;
use base qw(Template::Parser);
# set tag styles to ours
$Template::Parser::TAG_STYLE->{ playboy } = [ quotemeta('<!--%'),
quotemeta('-->') ];
$Template::Parser::TAG_STYLE->{ template } =
$Template::Parser::TAG_STYLE->{ tt2 } =
$Template::Parser::TAG_STYLE->{ default } =
$Template::Parser::TAG_STYLE->{ playboy };
$Template::Parser::DEFAULT_STYLE =
{
START_TAG => $Template::Parser::TAG_STYLE->{ playboy }->[0],
END_TAG => $Template::Parser::TAG_STYLE->{ playboy }->[1],
ANYCASE => 0,
INTERPOLATE => 1,
PRE_CHOMP => 0,
POST_CHOMP => 0,
V1DOLLAR => 1,
EVAL_PERL => 0,
};
#------------------------------------------------------------------------
# interpolate_text($text, $line)
#
# Examines $text looking for any variable references embedded like
# $this or like ${ this }.
#
# We override this function to implement the following fixes:
#
# * to work around the segfault that occurs when sending > (12? 16? 32)
# k of text through a single regex, limit the amount of raw text
# skipped per pass with {1,4000}
# * make the variable interpolation regex more greedy, to catch:
# $var.${key} and $var('param')
# which are legal in tags, so should be legal everywhere
#------------------------------------------------------------------------
sub interpolate_text {
my ($self, $text, $line) = @_;
my @tokens = ();
my ($pre, $var, $dir);
while ($text =~
/
( (?: \\. | [^\$] ){1,4000} ) # escaped or non-'$' character [$1]
|
( \$ # $ directive [$2]
(?:\{\s*)? # optional { and whitespace
( # embedded variable [$3]
(?:
(?: \$ \{ [^\}]* \} ) # ${ reference }
|
(?: \( [^\)]* \) ) # ( params )
|
(?: [\w\.]+ ) # word[.word]
)+
) (?:\s*\})? # optional space and closing }
)
/gx) {
($pre, $var, $dir) = ($1, $3, $2);
# preceding text
if (defined($pre) && length($pre)) {
$line += $pre =~ tr/\n//;
$pre =~ s/\\\$/\$/g;
push(@tokens, 'TEXT', $pre);
}
# $variable reference
if ($var) {
$line += $dir =~ tr/\n/ /;
push(@tokens, [ $dir, $line, $self->tokenise_directive($var) ]);
}
# other '$' reference - treated as text
elsif ($dir) {
$line += $dir =~ tr/\n//;
push(@tokens, 'TEXT', $dir);
}
}
return \@tokens;
}
1;
