Earlier this year, I was working at a company where we were working
with some large XML documents. Parsing and transforming a 40M XML
document was using up all of the memory we had. I thought that it
would be good to look into how Xerces' footprint could be improved.
Just the other day, I started writing a memory profiling tool that I
had envisioned. I looked at what is in the DOM objects, and I found
that one thing I couldn't justify was
StringBuffer fBufferStr;
defined in org.apache.xerces.dom.ChildNode. It is documented simply
here:
http://svn.apache.org/viewcvs.cgi?rev=319759&view=rev
The reference takes up 4 bytes (in a 32-bit JVM) which ends up being
about 7% of the footprint of a class like ElementNSImpl or 13% of the
footprint of CDATASectionImpl.
I've found this attribute used only in two places to implement DOM
Level 3 functionality, so it seems to me that it punishes everyone who
doesn't use that. I've done a little benchmarking using XMLBench
(http://www.sosnoski.com/opensrc/xmlbench/) and found that if I revert
the patch, it saves somewhere between 1.7% and 3.4% on memory, mostly
around 2.5%. Not a lot, but a few percent here and there helps.
It gets more interesting though. Hanging on to a StringBuffer like
this leads to problems that can be illustrated by a pathological case.
Imagine an XML file with a 1M text node that's 1000 nodes deep in the
tree. Though this file may only be a little bigger than 1M, the
referenced StringBuffers would use a gigabyte of memory of you were to
traverse the tree and call getTextContent() at each node.
I recommend that this change be reverted. If someone wants to send me
some cases that illustrate the performance improvement from reusing the
StringBuffer, I would try to implement some compromise between memory
and CPU usage. At the least, these StringBuffers should be held by
soft references to keep them from using up all of the memory.
I found it quite amusing that in running XMLBench, it required 211M of
heap in order to benchmark a 3M log file without getting an
OutOfMemoryError. So there are clearly some inefficiencies not only in
DOM representation but in parsing. So I have some other memory issues
to deal with, but let's start here.
Ken Geis
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
