String.intern() always returns a unique string object, so you could
override addSymbol like this:
public String addSymbol(char[] buffer, int offset, int length) {
return new String(buffer, offset, length).intern();
}
and cache nothing in the table. This would be slower but you'd save memory
which would have otherwise been used to create a new Entry and a new
character array (that has the same contents as the string). The memory
consumption of String.intern() is another story, but for Xerces to work
correctly it can't be avoided.
Bob Foster <[EMAIL PROTECTED]> wrote on 03/16/2005 10:43:51 PM:
> But it isn't clear what he could do to reduce the memory used by the
> SymbolTable, given that every element name will be referenced at least
> twice. Since the table guarantees that an identical String be returned,
> how do you keep from needing all those Strings in memory? Maybe you
> could expand on your suggestion?
>
> Bob Foster
> http://xmlbuddy.com/
>
> Michael Glavassevich wrote:
> > You could extend the SymbolTable class and then replace the default by
> > setting the http://apache.org/xml/properties/internal/symbol-table
> > property with your own SymbolTable. The parser components assume that
> > symbols returned from the SymbolTable have been internalized with
> > String.intern() so all SymbolTable implementations must return
> > internalized strings in order for the parser to function properly.
> >
> > [EMAIL PROTECTED] wrote on 03/16/2005 11:43:52 AM:
> >
> >
> >>Hello,
> >>
> >>Our application is going out of memory on moderate sized XML files
> >
> > (2-4MB)
> >
> >>containing random XML tags due to the caching nature of the
SymbolTable.
> >
> > The
> >
> >>random tags come from our customers and are embedded as a subtree in
our
> >
> > own
> >
> >>xml document. This is a historical decision and can not be reverted.
The
> >>result is that we get XML documents were we end up with 80000+
different
> >
> > XML
> >
> >>tags. Profiling learnt me that parsing such a file goes up to 20MB of
> >
> > memory
> >
> >>referred to by the SymbolTable. With the possibility of parsing
multiple
> >
> > of
> >
> >>such documents in parallel, we can go OOM very easily.
> >>
> >>Is there anything I can do to circumvent/optimize the internal usage
of
> >
> > the
> >
> >>SymbolTable?
> >>
> >>Ringo
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
Michael Glavassevich
XML Parser Development
IBM Toronto Lab
E-mail: [EMAIL PROTECTED]
E-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
