Miles,
Thanks for the tips. I'll move forward on coding this using STX and post some benchmarking numbers when I finish.
TB
Upayavira wrote:
Miles Elam wrote:
Very useful piece. Would you mind if I put this on the wiki?
Regards, Upayavira
As someone who has used STX, I can recommend it in this situation wholeheartedly. STX looks very much like XSLT but uses a different namespace and doesn't have as many options for transformation.
Unless something drastic has changed lately in the XSLT used by Cocoon, it uses a document table model (like a DOM but tailored toward a read-only view and a transformation source). This is necessary because XSLT allows several passes over the same source document and also allows arbitrary access to any point in the tree (although this is usually quite inefficient). So while XSLT is the preferred method for XML transformation in general, certain circumstances like yours would point toward alternatives.
As far as streaming XSLT results is concerned, it's possible to configure it this way at the expense of overall processing time. But you don't appear to have the memory for even one full transformation let alone many at the same time. STX is your best bet in my opinion. This always streams the output by its very nature.
Also, do NOT put this into a caching pipeline. With such a large source, memory constraints will get worse before they get better. Reprocess each time (or pregenerate on intervals a la cron) to shift the weight from memory to CPU/disk in this case.
Of course, a final option is to write your own custom Cocoon transformer, but I would recommend the STX route as it would likely be almost as fast and a while lot more flexible and maintainable in the long run.
- Miles Elam
On Nov 19, 2004, at 7:07 AM, Tom Bloomfield wrote:
The number of iterations cooresponds to the number of rows returned from the database. There are roughly 46,000 rows present now, so I need at least that many rows in my display. The XSL design enables me to use SAX which should help. The easiest thing would be to limit the number of rows returned to something more reasonable like 10,000 (or up the JVM memory :P), but this is the requirement I'm stuck with.
Help me understand this: If I apply a transformation using XSLT, streaming the xml in, does Cocoon "stream" the results out? IE, does the entire transformation happen in memory and then get flushed to the client, or does Cocoon flush the buffer to the client as xxx bytes are filled? I made an assumption that Cocoon does this automatically.
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
