There are applications which serialize Xerces' DOM using Java's object serialization services which rely on these classes being compatible from release to release. Aside from moving around and removing transient fields, it will be difficult to trim the size of the DOM implementation without breaking serialization compatibility. Probably seemed like a good idea at the time but making all the classes implement java.io.Serializable has significantly reduced our ability to make structural changes.
Michael Glavassevich XML Parser Development IBM Toronto Lab E-mail: [EMAIL PROTECTED] E-mail: [EMAIL PROTECTED] Arnaud Le Hors <[EMAIL PROTECTED]> wrote on 11/11/2005 05:35:46 PM: > Hi Ken, > I agree with you. I'm not sure what the motivation was to want to "reuse > StringBuffer" but it was a bad call. I suppose it was done to gain speed > but with today's JVMs it is not even clear that this is the right thing > to do for that matter either. In any case ChildNode being one of the > core classes of this DOM implementation anything affecting its size has > a major impact on the footprint. I have spent a lot of time in the past > triming down the size of those classes to improve the memory footprint. > I'm glad someone else is looking into it. Maybe with better tools and > different eyes you can squeeze some more! > Have fun. > > -- > Arnaud Le Hors - Program Director, Corporate Standards, IBM > > > > Ken Geis wrote: > > > Earlier this year, I was working at a company where we were working > > with some large XML documents. Parsing and transforming a 40M XML > > document was using up all of the memory we had. I thought that it > > would be good to look into how Xerces' footprint could be improved. > > > > Just the other day, I started writing a memory profiling tool that I > > had envisioned. I looked at what is in the DOM objects, and I found > > that one thing I couldn't justify was > > > > StringBuffer fBufferStr; > > > > defined in org.apache.xerces.dom.ChildNode. It is documented simply > > here: > > > > http://svn.apache.org/viewcvs.cgi?rev=319759&view=rev > > > > The reference takes up 4 bytes (in a 32-bit JVM) which ends up being > > about 7% of the footprint of a class like ElementNSImpl or 13% of the > > footprint of CDATASectionImpl. > > > > I've found this attribute used only in two places to implement DOM > > Level 3 functionality, so it seems to me that it punishes everyone who > > doesn't use that. I've done a little benchmarking using XMLBench > > (http://www.sosnoski.com/opensrc/xmlbench/) and found that if I revert > > the patch, it saves somewhere between 1.7% and 3.4% on memory, mostly > > around 2.5%. Not a lot, but a few percent here and there helps. > > > > It gets more interesting though. Hanging on to a StringBuffer like > > this leads to problems that can be illustrated by a pathological > > case. Imagine an XML file with a 1M text node that's 1000 nodes deep > > in the tree. Though this file may only be a little bigger than 1M, > > the referenced StringBuffers would use a gigabyte of memory of you > > were to traverse the tree and call getTextContent() at each node. > > > > I recommend that this change be reverted. If someone wants to send me > > some cases that illustrate the performance improvement from reusing > > the StringBuffer, I would try to implement some compromise between > > memory and CPU usage. At the least, these StringBuffers should be > > held by soft references to keep them from using up all of the memory. > > > > I found it quite amusing that in running XMLBench, it required 211M of > > heap in order to benchmark a 3M log file without getting an > > OutOfMemoryError. So there are clearly some inefficiencies not only > > in DOM representation but in parsing. So I have some other memory > > issues to deal with, but let's start here. > > > > > > Ken Geis > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
