Serge Knystautas wrote:
On 2/26/06, Jonathan Revusky <[EMAIL PROTECTED]> wrote:
Serge Knystautas wrote:
I'm working on a CMS that uses velocity template extensively (many
thousands of templates), and we're trying to figure out how to avoid
reparsing these templates over and over.
Have you actually determined that this is a bottleneck in your application?
Yes. Why would I ask otherwise? :)
It wouldn't be logical, but in my experience, many people are willing to
spend quite a bit of time and energy speculating about the performance
implications of whatever without any proof that it's a real-world
bottleneck.
Well, you could look at FreeMarker. :-) I think we have a pretty good
MRU template caching implementation. I don't know too much about it, it
was written by Attila Szegedi, not me. It's probably pretty good. Attila
is a pretty thorough guy. This stuff is quite well tested. It's been
part of our stable release for several years.
Heh, if freemarker can parse Velocity templates, then maybe. Like I
said, I've got thousands of temlates (not to mention users) that would
need converting...
Actually, we've got an automatic conversion tool that works fairly well,
reads in VTL and spits out FTL. A real miracle. The modern equivalent of
turning water into wine.
Besides Velocity is extremely flexible and trivial to stick a
memory-based LRU template cache. JDK 1.4 provides an LRU map, so I
can create that with a cap instead of the default hashtable, and I've
got an impl that I would hope is as reliable as what Attila
implemented.
But my real goal is to cache without using memory...
Really though, something like caching the parsed templates (i.e., the
AST nodes) to disk
Well, you're talking about storing the AST in some format on disk, but
there is already a way of storing a VTL AST on disk and that is the way
it is stored already, as a .vtl file. I mean, it is not obvious to me
offhand why re-instantiating the AST from the disk in your scenario is
going to be that much more efficient than just reparsing the template
from its canonical VTL format.
I mean, at the end of the day, you're just reading the template (in some
format) off the disk again. Okay, you could have a binary format for
storage that is a bit more efficient than the VTL text format, but how
much difference would it make?
In terms of overall Velocity development, how much value added could
their be?
Yes, it is a fair point as to whether the serializable API would be
faster, but I would hope so. Given the number of pages, I have to
turn off memory caching, so every page requires parsing.
What you need, I think, is the MRU cache.
My benchmarks calculated 37ms/MB to read a file (just read into a
string), 754ms/MB to compile (create a Template object), and 31ms/MB
to merge it with a simple context. Obviously if I can eat into that
700+ms/MB rate by serializing the ASTNodes or something else, then
that'd be great.
Maybe the question is how much of the overhead in parsing VTL is in the
lexical scanning. From what you're saying it seems that not much of it
is in actually physically reading the file. Offhand (I could be wrong)
it seems the gain you'd get from a binary storage format is that,
instead of scanning ahead for #foreach and all the other tokens they are
stored as a single byte, say.
But probably the solution you'll implement will be some kind of MRU
cache. Or you can just buy a lot more RAM, I guess. :-)
Jonathan Revusky
--
lead developer, FreeMarker project, http://freemarker.org/
And beyond serialization, if Velocity templates could get translated
into bytecode, then we've changed velocity from interpreted to
compiled templating, much like what Resin is doing with PHP
(http://wiki.caucho.com/Quercus). I'm not expecting someone else
write it... Just wondering if it seemed at all feasible and/or
querying for other people who have dealt with this.
--
Serge Knystautas
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
