For transformations of very large documents you may also wish to look into STX (http://www.xml.com/pub/a/2003/02/26/stx.html).
STX isn't as flexible as XSLT (because there isn't a DOM available) but can be significantly more efficient for large documents. Oh, and by the way it is generally regarded as impolite to post to multiple lists. Try posting to one list first, and only posting to a second if you don't get a suitable reply. Regards, Simon On Tue, 2003-12-09 at 11:43, Barbara Samson wrote: > An XSLT processor needs to keep the whole document in memory because the > template for any node can reference nodes before and after it. So there > will always be an upper limit to how large an XML document you can > process with XSLT. If you can limit the references between nodes, you > should be able to process the document as a stream through a SAX > handler. I would use one pass through a SAX handler to extract the > elements of interest, then pass those elements through an XSLT > processor. Apache Cocoon offers a good model for this kind of "pipeline" > processing. > > Barbara > > -----Original Message----- > From: Rajesh Raheja [mailto:[EMAIL PROTECTED] > Sent: Monday, December 08, 2003 1:14 PM > To: xalan-c-users@xml.apache.org; [EMAIL PROTECTED] > Subject: Memory Consumption for Large XML Transformation (50MB to 1GB) > with SAX > > > We are trying to transform very large XML documents (30MB to 1GB in > size) and were planning on using an XSLT engine for it. Our tests > showed that passing in the document in DOM typically crashed with out of > memory errors. > > however, even with passing in SAX events, the memory consumption was > around FIVE times the document size (e.g. the 50MB document input > consumed 250MB of the jvm). > > would appreciate any inputs on any way to improve the memory consumption > and more generally - Is XSLT the way to go for such large documents? > what are the alternatives (btw, we tried asking customer to reduce or > break up the document > - not feasible!)? > > Thanks > Rajesh > > > __________________________________ > Do you Yahoo!? > New Yahoo! Photos - easier uploading and sharing. > http://photos.yahoo.com/ >
