On Sun, 2001-12-30 at 19:47, Ryan Thompson wrote:
> Mark Maunder wrote to Ryan Thompson:
>
> > Ryan Thompson wrote:
> >
> > > There must be a faster way. I have thought about pre-compiling each
> > > HTML file into a Perl module, but there would have to be an automated
> > > (and secure) way to suck these in if the original file changes.
> > >
> > > Either that, or maybe someone has written a better parser. My code
> > > looks something like this, to give you an idea of what I need:
> >
> > Sure there are tons of good template systems out there. I think
> > someone made a comment about writing a template system being a
> > right of passage as a perl developer. But it's also more fun to do
> > it yourself.
>
> :-)
>
>
> > I guess you've tried compiling your regex with the o modifier?
>
> Yep, problem is there are several of them. I've done some work
> recently to simplify things, which might have a positive effect.
>
>
> > Also, have you tried caching your HTML in global package variables
> > instead of shared memory? I think it may be a bit faster than
> > shared memory segments like Apache::Cache uses. (The first request
> > for each child will be slower, but after they've each served once,
> > they'll all be fast). Does your engine stat (access) the html file
> > on disk for each request? You mentioned you're caching, but
> > perhaps you're checking for changes to the file. Try to stat as
>
> My caching algorithm uses 2 levels:
>
> When an HTML file is requested, the instance of my template class
> checks in its memory cache. If it finds it there, great... everything
> is done within that server process.
>
> If it's not in the memory cache, it checks in a central MySQL cache
> database on the local machine. These requests are on the order of a
> few ms, thanks to an optimized query and Apache::DBI. NOT a big deal.
>
> If it's not in either cache, it takes it's lumps and goes to disk.
>
> In each cache, I use a TTL. (time() + $TTL), which is configurable,
> and usually set to something like 5 minutes in production, or 60
> seconds during development/bug fixes. (And, for this kind of data, 5
> minutes is pretty granular, as templates don't change very often.. but
> setting it any higher would, on average, have only a negligible
> improvement in performance at the risk of annoying developers :-).
>
> And, with debugging in my template module turned on, it has been
> observed that cache misses are VERY infrequent (< 0.1% of all
> requests).
>
> In fact, if I use this cache system and disable all parsing (i.e.,
> just use it to include straight HTML into mod_perl apps), I can serve
> 150-200 requests/second on the same system.
>
> With my parsing regexps enabled, it drops to 50-60 requests/second.
>
First. I would love to know what kinda machine your using here. :)
Second. I"m creating a template based shopping cart. Yeah. there are a
ton of carts out there. But they all either under powered/featured/coded
or over powered/features/coded. Plus most of the free ones don't support
multiple domains. And I'm not in the mood duplicated a good single
domain cart 'x' times per client.
Key features were easy intergration into web sites, also,'pure
intergration' for a custom look and feel.
I looked at just about every template system on CPAN and came across
text::template. Anyone use this one? I don't require if/then's within
the template. Infact all I really need are call backs to an API that I
am currently developing. And in the case that I do need if/then's. I can
use pure perl within the template. Even do SQL queries with-in the
template, or not even use the API at all if I wanted to.
There are how ever a few quirks that I need to work out with
text::template. I plan on posting a thread about it later today once I
make sure it's not my coding.
Anyways, getting closer to my point here. How my cart works is that you
have something like this.
<html>
<body>
<table> <tr><td>
{ #display catagory view here
%vars = ( #this modifies the global defaults for html output.
'border' = '1',
'cellpadding' = '2'
);
&JC::API::cat_view(/%vars);
''
}
</td>
</tr>
</body></html>
That's basicly it. You make call backs to various parts of the API to
create the look and feel of the site based on what is in the database.
I implamented some google style timing in the API. It's basicly gets a
Time::HiRes timestamp in the beginning and does the math at the very end
and posts it in an html comment. On my dev machine, a dual PIII-750
with a gig of ram and mod_perl running with full caching enabled and
fully persistant code base for the cart. My average transaction time is
about .08 of a second. That leads me to think that my machine can handle
10 html page generations a second (this isn't an exact science, but
close).
Now, As with the bug I think I am having with text::template. I'm using
STDOUT when I compile the template and execute all call backs and sql
queries and the such fixes things for now. (the buffering bit it can use
to compile the output into a varable is not working, it process the
output of the perl API and prints it in the incorrect spot, the
beginning of the output).
Second feature that I'm not using is caching of output. The caching I'm
wondering how to handle it. I would need this text::template buffering
bit to work better first before I can use the caching. But I found a few
nice mod_perl aware caching tools on CPAN that I'll use to do caching.
3rd. my sql queries are not the most optimized and mostly tossed
together. But then again my developemnt tables are not that big at all.
Keep in mind that my times are based also on the time it takes to make
the transmission of the data from server to client as well as the time
it takes to compile the template and do sql queries. I"m not sure if
your times are factoring that in. Locally times are .08/sec but remotely
from T1 to cable times are .16/sec to .20/sec depending on network
conditions and size of html file at the time naturally. Some of the html
files can be about 50K in size we found.
Also, pages will do about 3 sql queries on average plus queries to the
Apache::Session state information.
Do these numbers sound right? There are places were I could move from
doing SQL to keeping some data in Apache::Session. but it would be small
speed improments unless the catagories are very deep, I do have
recursive SQL queries inplace for some things.
Your talking 50 to 60/sec as slow. So I dunno what is 'fast'.
I do plan on release this cart GPL one of these days once I know it
works. And I do plan on having a web site explaining it in full detail.
If anyone is interested right now in the development of it. Give me a
ring. I'll be glad to get some help with it. (expecially with fedex, ups
shipping calcs :) )
If anyone would like to see a working site using the cart that I am
working on. It's over at http://www2.test.jasnik.net.
--
Jason Czerak
