Re: Fast template system. Ideas,theorys and tools
I looked at just about every template system on CPAN and came across text::template. Anyone use this one? I'd suggest you read my overview of templating options. It summarizes the top choices for templating tools, and talks about the strengths of weaknesses of Text::Template.. http://perl.apache.org/features/tmpl-cmp.html I implamented some google style timing in the API. It's basicly gets a Time::HiRes timestamp in the beginning and does the math at the very end and posts it in an html comment. You'd be better off with Devel::DProf (or Apache::DProg under mod_perl). My average transaction time is about .08 of a second. That leads me to think that my machine can handle 10 html page generations a second (this isn't an exact science, but close). You are assuming serial execution. You should be able to push much more than that through in a second because of parallel execution. 3rd. my sql queries are not the most optimized and mostly tossed together. DBIx::Profile can help you identify problems in your queries. And follow the optimization advice for DBI in the guide. - Perrin
Re: Fast template system. Ideas,theorys and tools
On Thu, 2002-01-03 at 12:20, Perrin Harkins wrote: snip I implamented some google style timing in the API. It's basicly gets a Time::HiRes timestamp in the beginning and does the math at the very end and posts it in an html comment. You'd be better off with Devel::DProf (or Apache::DProg under mod_perl). My average transaction time is about .08 of a second. That leads me to think that my machine can handle 10 html page generations a second (this isn't an exact science, but close). You are assuming serial execution. You should be able to push much more than that through in a second because of parallel execution. What do you suggest as a good benchmark tool to use that would be 'smart' when testing a whole complete site. -- Jason Czerak
Re: Fast template system. Ideas,theorys and tools
What do you suggest as a good benchmark tool to use that would be 'smart' when testing a whole complete site. For pounding a bunch of URLs, the best are ab, httperf, and http_load. If you need something fancier that tests a complex series of actions and responses, there are several packages on CPAN. They've been discussed on this list, but I haven't tried any of them myself so I can't comment on them. They will not scale as easilly as the first three I mentioned though, so if you need to test hundreds of requests per second with them you will need multiple machines to run the clients on. - Perrin
Re: Fast template system. Ideas,theorys and tools
Perrin Harkins wrote: What do you suggest as a good benchmark tool to use that would be 'smart' when testing a whole complete site. For pounding a bunch of URLs, the best are ab, httperf, and http_load. If you need something fancier that tests a complex series of actions and responses, there are several packages on CPAN. They've been discussed on this list, but I haven't tried any of them myself so I can't comment on them. They will not scale as easilly as the first three I mentioned though, so if you need to test hundreds of requests per second with them you will need multiple machines to run the clients on. Like Perrin says, also see some examples in the guide. http://perl.apache.org/guide/performance.html If you really need to write many benchmarks try my never released Apache::Benchmark and if you find it useful, it should be easy to port it to use HTTPD::Bench::ApacheBench as it currently calls 'ab' and parses the output. Get the package from here: http://stason.org/works/modules/Apache-Benchmark-0.01.tar.gz _ Stas Bekman JAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide http://perl.apache.org/guide mailto:[EMAIL PROTECTED] http://ticketmaster.com http://apacheweek.com http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: Fast template system. Ideas,theorys and tools
On Sun, 2001-12-30 at 19:47, Ryan Thompson wrote:
Mark Maunder wrote to Ryan Thompson:
Ryan Thompson wrote:
There must be a faster way. I have thought about pre-compiling each
HTML file into a Perl module, but there would have to be an automated
(and secure) way to suck these in if the original file changes.
Either that, or maybe someone has written a better parser. My code
looks something like this, to give you an idea of what I need:
Sure there are tons of good template systems out there. I think
someone made a comment about writing a template system being a
right of passage as a perl developer. But it's also more fun to do
it yourself.
:-)
I guess you've tried compiling your regex with the o modifier?
Yep, problem is there are several of them. I've done some work
recently to simplify things, which might have a positive effect.
Also, have you tried caching your HTML in global package variables
instead of shared memory? I think it may be a bit faster than
shared memory segments like Apache::Cache uses. (The first request
for each child will be slower, but after they've each served once,
they'll all be fast). Does your engine stat (access) the html file
on disk for each request? You mentioned you're caching, but
perhaps you're checking for changes to the file. Try to stat as
My caching algorithm uses 2 levels:
When an HTML file is requested, the instance of my template class
checks in its memory cache. If it finds it there, great... everything
is done within that server process.
If it's not in the memory cache, it checks in a central MySQL cache
database on the local machine. These requests are on the order of a
few ms, thanks to an optimized query and Apache::DBI. NOT a big deal.
If it's not in either cache, it takes it's lumps and goes to disk.
In each cache, I use a TTL. (time() + $TTL), which is configurable,
and usually set to something like 5 minutes in production, or 60
seconds during development/bug fixes. (And, for this kind of data, 5
minutes is pretty granular, as templates don't change very often.. but
setting it any higher would, on average, have only a negligible
improvement in performance at the risk of annoying developers :-).
And, with debugging in my template module turned on, it has been
observed that cache misses are VERY infrequent ( 0.1% of all
requests).
In fact, if I use this cache system and disable all parsing (i.e.,
just use it to include straight HTML into mod_perl apps), I can serve
150-200 requests/second on the same system.
With my parsing regexps enabled, it drops to 50-60 requests/second.
First. I would love to know what kinda machine your using here. :)
Second. Im creating a template based shopping cart. Yeah. there are a
ton of carts out there. But they all either under powered/featured/coded
or over powered/features/coded. Plus most of the free ones don't support
multiple domains. And I'm not in the mood duplicated a good single
domain cart 'x' times per client.
Key features were easy intergration into web sites, also,'pure
intergration' for a custom look and feel.
I looked at just about every template system on CPAN and came across
text::template. Anyone use this one? I don't require if/then's within
the template. Infact all I really need are call backs to an API that I
am currently developing. And in the case that I do need if/then's. I can
use pure perl within the template. Even do SQL queries with-in the
template, or not even use the API at all if I wanted to.
There are how ever a few quirks that I need to work out with
text::template. I plan on posting a thread about it later today once I
make sure it's not my coding.
Anyways, getting closer to my point here. How my cart works is that you
have something like this.
html
body
table trtd
{ #display catagory view here
%vars = ( #this modifies the global defaults for html output.
'border' = '1',
'cellpadding' = '2'
);
JC::API::cat_view(/%vars);
''
}
/td
/tr
/body/html
That's basicly it. You make call backs to various parts of the API to
create the look and feel of the site based on what is in the database.
I implamented some google style timing in the API. It's basicly gets a
Time::HiRes timestamp in the beginning and does the math at the very end
and posts it in an html comment. On my dev machine, a dual PIII-750
with a gig of ram and mod_perl running with full caching enabled and
fully persistant code base for the cart. My average transaction time is
about .08 of a second. That leads me to think that my machine can handle
10 html page generations a second (this isn't an exact science, but
close).
Now, As with the bug I think I am having with text::template. I'm using
STDOUT when I compile the template and execute all call backs and sql
queries and the such fixes things for now. (the buffering bit it can use
to compile the output into a
Re: Fast template system. Ideas,theorys and tools
On Tue, Jan 01, 2002 at 03:02:23PM -0500, Jason Czerak wrote: I looked at just about every template system on CPAN and came across text::template. Anyone use this one? I don't require if/then's within ^ A template system should (IMO) do exactly that. The logic shouldn't be embedded in the templates. That's why I settled on CGI::FastTemplate (Yes, yet another templating system). the template. Infact all I really need are call backs to an API that I am currently developing. And in the case that I do need if/then's. I can use pure perl within the template. Even do SQL queries with-in the template, or not even use the API at all if I wanted to. I don't want any code/sql in my templates, and it works out fine that way for whatever project I do. And it's certainly easier for the project's webdesigner to gasp when there's no code they can break by using whatever windows-app they want to use to edit their html. I tried Mason and other of the more featurefull template systems out there but I always ended up going back to CGI::FastTemplate. my $cent = 2; # on templates -- Thomas Eibner http://thomas.eibner.dk/ DnsZone http://dnszone.org/ mod_pointer http://stderr.net/mod_pointer
Re: Fast template system. Ideas,theorys and tools
Jason Czerak wrote to [EMAIL PROTECTED]:
In fact, if I use this cache system and disable all parsing (i.e.,
just use it to include straight HTML into mod_perl apps), I can serve
150-200 requests/second on the same system.
With my parsing regexps enabled, it drops to 50-60 requests/second.
First. I would love to know what kinda machine your using here. :)
It's part of a testing environment. Nothing fancy... Lightly loaded
single CPU PII-400MHz, 512MB RAM, Adaptec Ultra160 SCSI (no RAID),
FreeBSD 4.4-RELEASE, with some in-house kernel patches applied, latest
MySQL, mod_perl, modules.. It's great for testing, because when we
move things to the REAL server (more power, more load), I know they'll
be at least as fast over there and won't crack under pressure :-)
The database this runs off of has several tables. The smallest has
about 15K rows. The largest is pushing 300K rows.
3rd. my sql queries are not the most optimized and mostly tossed
together. But then again my developemnt tables are not that big at
all.
This might be one of your largest areas to improve upon. Thanks to
practical and theoretical experience with RDBMS', I write fairly well
optimized SQL when I throw things together, but I recently rewrote a
JOIN in one of my applications that was fast enough and doubled the
observed performance (oops.. should have noticed that the first time).
Check your table design and indexing, too, as these are areas which
are tremendously important, that many people overlook.
Even a development table that is not that big might be a few
thousand rows. If your query is poorly designed, and your DBMS has to
read every row in the table, you're going to take a noticeable hit
over a number of queries. Worse yet, if, for example, you're taking
the cartesian product by mistake (this is easy enough to do), pretty
quickly you've got hundreds of thousands of rows to deal with. :-)
I guess what I'm saying is, never underestimate the importance of
carefully constructed SQL.
Keep in mind that my times are based also on the time it takes to
make the transmission of the data from server to client as well as
the time it takes to compile the template and do sql queries. Im
not sure if your times are factoring that in. Locally times are
.08/sec but remotely from T1 to cable times are .16/sec to .20/sec
depending on network conditions and size of html file at the time
naturally. Some of the html files can be about 50K in size we
found.
Generally when testing application performance, it is wise to
eliminate highly variable things like network performance. Network
importance might be significant in your application deployment, but
you can test that in isolation with the many great network tools out
there.
Also, pages will do about 3 sql queries on average plus queries to
the Apache::Session state information.
Take advantage of query caching... Instead of
$cursor = $dbh-prepare('SELECT * FROM yourtable WHERE foo = bar');
use
$cursor = $dbh-prepare('SELECT * FROM yourtable WHERE foo = ?');
$cursor-bind_param(1,bar);
This way, your SQL database will cache the queries so that they do not
have to be parsed and examined each time. Note that this does NOT
cache the results of the queries. You can do that, as well, but can
lead to nasty surprises.
Also, try to limit the total number of SQL queries you make per
request. Take advantage of joins, group by, etc, if you can.
Do these numbers sound right? There are places were I could move
from doing SQL to keeping some data in Apache::Session. but it
would be small speed improments unless the catagories are very
deep, I do have recursive SQL queries inplace for some things.
Your talking 50 to 60/sec as slow. So I dunno what is 'fast'.
I optimized a few things wrt. the parsing and am now seeing
110-115/sec pretty consistently. IIRC, this server won't handle much
more than that without SQL.
- Ryan
--
Ryan Thompson [EMAIL PROTECTED]
Network Administrator, Accounts
SaskNow Technologies - http://www.sasknow.com
#106-380 3120 8th St E - Saskatoon, SK - S7H 0W2
Tel: 306-664-3600 Fax: 306-664-1161 Saskatoon
Toll-Free: 877-727-5669 (877-SASKNOW) North America
Re: Fast template system
Ryan Thompson wrote: Mark Maunder wrote to Ryan Thompson: Ryan Thompson wrote: There must be a faster way. I have thought about pre-compiling each HTML file into a Perl module, but there would have to be an automated (and secure) way to suck these in if the original file changes. Either that, or maybe someone has written a better parser. My code looks something like this, to give you an idea of what I need: Sure there are tons of good template systems out there. I think someone made a comment about writing a template system being a right of passage as a perl developer. But it's also more fun to do it yourself. :-) I guess you've tried compiling your regex with the o modifier? Yep, problem is there are several of them. I've done some work recently to simplify things, which might have a positive effect. Also, have you tried caching your HTML in global package variables instead of shared memory? I think it may be a bit faster than shared memory segments like Apache::Cache uses. (The first request for each child will be slower, but after they've each served once, they'll all be fast). Does your engine stat (access) the html file on disk for each request? You mentioned you're caching, but perhaps you're checking for changes to the file. Try to stat as My caching algorithm uses 2 levels: When an HTML file is requested, the instance of my template class checks in its memory cache. If it finds it there, great... everything is done within that server process. If it's not in the memory cache, it checks in a central MySQL cache database on the local machine. These requests are on the order of a few ms, thanks to an optimized query and Apache::DBI. NOT a big deal. If it's not in either cache, it takes it's lumps and goes to disk. If you're using a disk based table, in most cases, mysql would access the disk itself anyway. So whether you're getting the cached data from mysql or a file, it's still coming from disk. (yes mysql caches - especially if you're using InnoDB tables, but you're not gauranteed to save a disk access). Not sure how much html/content you have, but any chance you can stick it all in shared memory, or even better, give each child their own copy in a package global variable (like a hashref)? If it's under a meg (maybe even 2) you might be able to get away with that. In each cache, I use a TTL. (time() + $TTL), which is configurable, and usually set to something like 5 minutes in production, or 60 seconds during development/bug fixes. (And, for this kind of data, 5 minutes is pretty granular, as templates don't change very often.. but setting it any higher would, on average, have only a negligible improvement in performance at the risk of annoying developers :-). And, with debugging in my template module turned on, it has been observed that cache misses are VERY infrequent ( 0.1% of all requests). In fact, if I use this cache system and disable all parsing (i.e., just use it to include straight HTML into mod_perl apps), I can serve 150-200 requests/second on the same system. With my parsing regexps enabled, it drops to 50-60 requests/second. So, to me, it is clear where performance needs to be improved. :-) How about instead of having a cache expiry/TTL, you parse the HTML on the first request only and then always serve from the cache. To refresh the cache, you set a flag in shared memory. Then whenever a child is about to serve from cache, it just checks the flag in shared memory to see if it needs to refresh it's cache. That way you can 'push' out new code by just setting the flag and unsetting it once all running children have read from the cache. You'll also have every request served from the cache except the first. You'll also get the benifit of having each child serve live code for every request by keeping the flag set. That way your developers can code in realtime without a 60 second latency for new HTML to take effect when you want them to.
Re: Fast template system
Yeah, TT is good, all right, especially with the ability to change the
start and end tags to something more compliant, like !-- and --.
Uh ... why aren't you syntax-validating the actual HTML output by your
application?
Also, there are multiple syntax-checkerts that allow you to specify
validation rules.
I'd gladly switch, if someone knows how to get it working on FreeBSD
-STABLE without breaking everything else :-)
Quit using FreeBSD ports. Compile from source. Your problem is almost
certainly due to FreeBSD's weird installation of Perl, specifically its
library paths.
The best solution I've found is to create a new user 'perl' and install
Perl and all libraries etc under that user's home directory.
My system:
nick@world ~uname -a
FreeBSD world.tonkinresolutions.com 4.4-PRERELEASE FreeBSD 4.4-PRERELEASE
#1: Sat Aug 11 18:17:44 PDT 2001
[...]
nick@world ~perl -v
This is perl, version 5.005_03 built for i386-freebsd
[...]
nick@world ~perl -MTemplate -e 'print $Template::VERSION,\n'
2.06
- nick
Nick Tonkin {|8^)
Fast template system
Hey everybody, Some time ago, I coded an HTML template engine that runs under mod_perl that allows a mod_perl application to parse variables, simple conditionals, and nested include files in an HTML file read from disk (or from my module's very quick memory cache). Trouble is, all the regexps I'm using to substitute variables, handle conditionals, etc, are extremely slow. (In fact, they are the bottleneck in several applications that do other expensive things like SQL queries on large tables). I have done some work to optimize the regexps, but my stuff is still only able to handle about 50-60 requests per second. (Without my template module, the same scripts can achieve 120-130 r/s on the same machine, making the same SQL queries). There must be a faster way. I have thought about pre-compiling each HTML file into a Perl module, but there would have to be an automated (and secure) way to suck these in if the original file changes. Either that, or maybe someone has written a better parser. My code looks something like this, to give you an idea of what I need: pMy name is: !-- name --/p !--#ifdef age -- pMy age is: !-- age --/p !--#endif -- To keep things simple and relatively efficient, I don't allow nested IFs. I've used HTML comments so that WYSIWYG editors or HTML validators don't complain. I've looked at some other template systems, and haven't really found much that interests me. Maybe someone can spark my interest again. Any thoughts? - Ryan -- Ryan Thompson [EMAIL PROTECTED] Network Administrator, Accounts SaskNow Technologies - http://www.sasknow.com #106-380 3120 8th St E - Saskatoon, SK - S7H 0W2 Tel: 306-664-3600 Fax: 306-664-1161 Saskatoon Toll-Free: 877-727-5669 (877-SASKNOW) North America
Re: Fast template system
Ryan Thompson [EMAIL PROTECTED] writes: Any thoughts? You really have to ask?!!! * _Dave thinks: Template Toolit. -- David Hodgkinson, Wizard for Hirehttp://www.davehodgkinson.com Editor-in-chief, The Highway Star http://www.deep-purple.com Deep Purple Family Tree news http://www.slashrock.com Interim Technical Director, Web Architecture Consultant for hire
Re: Fast template system
On Sun, 30 Dec 2001, Ryan Thompson wrote: Any thoughts? There's quite a number of HTML template systems on CPAN which have been under development for years and are well supported. Use one of those and save yourself the hassle. I like Mason (but then again, I'm one of the developers ;) but also check out Embperl, HTML::Template, Apache::ASP, and Template Toolkit. Those are, IMHO, the major players in the text templating space, and are the most widely used. -dave
Re: Fast template system
At 02:18 PM 12/30/2001 -0600, Ryan Thompson wrote: Any thoughts? Have a look at this document that compares most of the template engines out there. http://perl.apache.org/features/tmpl-cmp.html I think you'll have an easy time converting your homegrown templates over to Template Toolkit, and, since it's fast, it should meet your needs. here's your example using tt's syntax. pMy name is: [% name %]/p [% IF age %] pMy age is: [% age %]/p [% END %] Good luck, Kenny
Re: Fast template system
Dave Hodgkinson wrote to Ryan Thompson: Ryan Thompson [EMAIL PROTECTED] writes: Any thoughts? You really have to ask?!!! Yes!! :-) I've tried or at least taken a critical look at most of the template systems out there, and they are either far too simple (i.e., variable expansion only), far too complex, or use constructs/syntax that break HTML validation. * _Dave thinks: Template Toolit. I've looked at TT (and have heard it's praises sung), but it requires Perl 5.6.0, which is, unfortunately, not yet stable on all of the production systems my projects are deployed on. The syntax and features look about right, though... So it is something I might try when 5.6.0 is more widely deployed. So far, the only complaint about my system is that it is slower than it needs to be. It works under 5.005, can be installed ANYWHERE, caches content, doesn't break HTML validation, is proven to be easy for non-programmers to use, and has been through about three years of careful development and production testing. So, I'm willing to try something different.. but this is not a decision I take lightly. :-) Thanks, - Ryan -- Ryan Thompson [EMAIL PROTECTED] Network Administrator, Accounts SaskNow Technologies - http://www.sasknow.com #106-380 3120 8th St E - Saskatoon, SK - S7H 0W2 Tel: 306-664-3600 Fax: 306-664-1161 Saskatoon Toll-Free: 877-727-5669 (877-SASKNOW) North America
Re: Fast template system
On Sun, 30 Dec 2001, Kenny Gatdula wrote:
I think you'll have an easy time converting your homegrown templates over
to Template Toolkit, and, since it's fast, it should meet your needs.
here's your example using tt's syntax.
Using Apache::Template (the Apache/mod_perl interface to the Template
Toolkit) you can just set the tag delimiters to use html comments by doing
TT2Tagshtml
In your httpd.conf. Of course, TMTOWTDI, and you can also switch the
comments in the template itself with a [% %] directive at the start, pass
different arguments to Template.pm or it's submodules in your own custom
handlers etc.
Later.
Mark.
--
s'' Mark Fowler London.pm Bath.pm
http://www.twoshortplanks.com/ [EMAIL PROTECTED]
';use Term'Cap;$t=Tgetent Term'Cap{};print$t-Tputs(cl);for$w(split/ +/
){for(0..30){$|=print$t-Tgoto(cm,$_,$y). $w;select$k,$k,$k,.03}$y+=2}
Re: Fast template system
Kenny Gatdula wrote to Ryan Thompson and [EMAIL PROTECTED]: At 02:18 PM 12/30/2001 -0600, Ryan Thompson wrote: Any thoughts? Have a look at this document that compares most of the template engines out there. http://perl.apache.org/features/tmpl-cmp.html Thanks, I wasn't aware of that page. I think you'll have an easy time converting your homegrown templates over to Template Toolkit, and, since it's fast, it should meet your needs. here's your example using tt's syntax. pMy name is: [% name %]/p [% IF age %] [...] Yeah, TT is good, all right, especially with the ability to change the start and end tags to something more compliant, like !-- and --. I'd gladly switch, if someone knows how to get it working on FreeBSD -STABLE without breaking everything else :-) - Ryan -- Ryan Thompson [EMAIL PROTECTED] Network Administrator, Accounts SaskNow Technologies - http://www.sasknow.com #106-380 3120 8th St E - Saskatoon, SK - S7H 0W2 Tel: 306-664-3600 Fax: 306-664-1161 Saskatoon Toll-Free: 877-727-5669 (877-SASKNOW) North America
Re: Fast template system
At 05:05 AM 12/31/2001, Ryan Thompson wrote: I've looked at TT (and have heard it's praises sung), but it requires Perl 5.6.0, which is, unfortunately, not yet stable on all of the production systems my projects are deployed on. The syntax and features look about right, though... So it is something I might try when 5.6.0 is more widely deployed. When did it switch it's position? I am using a version of TT from a few months ago but on 5.00503 quite nicely.
Re: Fast template system
[EMAIL PROTECTED] wrote to Ryan Thompson: Ryan Thompson wrote: Yeah, TT is good, all right, especially with the ability to change the start and end tags to something more compliant, like !-- and --. I'd gladly switch, if someone knows how to get it working on FreeBSD -STABLE without breaking everything else :-) Update your ports tree. Version 2.06 (the latest) is there and it works with the Perl in FreeBSD 4.4 OK... Thanks. Looks like that was committed on Dec 3. The last time I tried to build TT was a few days prior to that, at the time when it was still apparently forbidden. Once I saw it was dependent on 5.6.0, I was not optimistic that a future update to the port would bring in 5.005_03 compatibility. - Ryan -- Ryan Thompson [EMAIL PROTECTED] Network Administrator, Accounts SaskNow Technologies - http://www.sasknow.com #106-380 3120 8th St E - Saskatoon, SK - S7H 0W2 Tel: 306-664-3600 Fax: 306-664-1161 Saskatoon Toll-Free: 877-727-5669 (877-SASKNOW) North America
Re: Fast template system
Ryan Thompson wrote: There must be a faster way. I have thought about pre-compiling each HTML file into a Perl module, but there would have to be an automated (and secure) way to suck these in if the original file changes. Either that, or maybe someone has written a better parser. My code looks something like this, to give you an idea of what I need: Sure there are tons of good template systems out there. I think someone made a comment about writing a template system being a right of passage as a perl developer. But it's also more fun to do it yourself. I guess you've tried compiling your regex with the o modifier? Also, have you tried caching your HTML in global package variables instead of shared memory? I think it may be a bit faster than shared memory segments like Apache::Cache uses. (The first request for each child will be slower, but after they've each served once, they'll all be fast). Does your engine stat (access) the html file on disk for each request? You mentioned you're caching, but perhaps you're checking for changes to the file. Try to stat as infrequently as possible, or if you have to then use the _ special filehandle for multiple stats. Just my 2c. ~mark.
Re: Fast template system
On Sun, 30 Dec 2001, Ryan Thompson wrote: Dave Hodgkinson wrote to Ryan Thompson: Ryan Thompson [EMAIL PROTECTED] writes: Any thoughts? You really have to ask?!!! Yes!! :-) I've tried or at least taken a critical look at most of the template systems out there, and they are either far too simple (i.e., variable expansion only), far too complex, or use constructs/syntax that break HTML validation. You want validation? Use XSLT. -- !-- Matt -- :-Get a smart net/:-
Re: Fast template system
* Ryan Thompson ([EMAIL PROTECTED]) [31 Dec 2001 08:19]: [...] I've tried or at least taken a critical look at most of the template systems out there, and they [...] use constructs/syntax that break HTML validation. Surely you only validate your HTML that's served and not your template system source? I, personally, would be quite annoyed if I had to do things like: if expr=$a $b blah blah else / flarp flarp /if cheers, -- iain. http://eh.org/~koschei/
Re: Fast template system
Mark Maunder wrote to Ryan Thompson: Ryan Thompson wrote: There must be a faster way. I have thought about pre-compiling each HTML file into a Perl module, but there would have to be an automated (and secure) way to suck these in if the original file changes. Either that, or maybe someone has written a better parser. My code looks something like this, to give you an idea of what I need: Sure there are tons of good template systems out there. I think someone made a comment about writing a template system being a right of passage as a perl developer. But it's also more fun to do it yourself. :-) I guess you've tried compiling your regex with the o modifier? Yep, problem is there are several of them. I've done some work recently to simplify things, which might have a positive effect. Also, have you tried caching your HTML in global package variables instead of shared memory? I think it may be a bit faster than shared memory segments like Apache::Cache uses. (The first request for each child will be slower, but after they've each served once, they'll all be fast). Does your engine stat (access) the html file on disk for each request? You mentioned you're caching, but perhaps you're checking for changes to the file. Try to stat as My caching algorithm uses 2 levels: When an HTML file is requested, the instance of my template class checks in its memory cache. If it finds it there, great... everything is done within that server process. If it's not in the memory cache, it checks in a central MySQL cache database on the local machine. These requests are on the order of a few ms, thanks to an optimized query and Apache::DBI. NOT a big deal. If it's not in either cache, it takes it's lumps and goes to disk. In each cache, I use a TTL. (time() + $TTL), which is configurable, and usually set to something like 5 minutes in production, or 60 seconds during development/bug fixes. (And, for this kind of data, 5 minutes is pretty granular, as templates don't change very often.. but setting it any higher would, on average, have only a negligible improvement in performance at the risk of annoying developers :-). And, with debugging in my template module turned on, it has been observed that cache misses are VERY infrequent ( 0.1% of all requests). In fact, if I use this cache system and disable all parsing (i.e., just use it to include straight HTML into mod_perl apps), I can serve 150-200 requests/second on the same system. With my parsing regexps enabled, it drops to 50-60 requests/second. So, to me, it is clear where performance needs to be improved. :-) - Ryan -- Ryan Thompson [EMAIL PROTECTED] Network Administrator, Accounts SaskNow Technologies - http://www.sasknow.com #106-380 3120 8th St E - Saskatoon, SK - S7H 0W2 Tel: 306-664-3600 Fax: 306-664-1161 Saskatoon Toll-Free: 877-727-5669 (877-SASKNOW) North America
Re: Fast template system
Ryan == Ryan Thompson [EMAIL PROTECTED] writes: Ryan I've looked at TT (and have heard it's praises sung), but it requires Ryan Perl 5.6.0, Wrong. I'm running it on 5.5.3 just fine. Where did you see it requires 5.6.0? -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 [EMAIL PROTECTED] URL:http://www.stonehenge.com/merlyn/ Perl/Unix/security consulting, Technical writing, Comedy, etc. etc. See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
