Our real-world use case is merging into one file a number of XML files
that total approx 3-4Mb in size, maybe more.
A test case I tried is below:
<target name="try-xmltask">
<xmltask dest="xmltask_test.xml">
<insert path="/" >
<![CDATA[
<test/>
]]>
</insert>
</xmltask>
<script language="jython">
result = ",".join([str(i) for i in range(10000)])
project.setProperty("xml.test.list", str(result))
</script>
<echo>${xml.test.list}</echo>
<for param="number" list="${xml.test.list}">
<sequential>
<xmltask source="xmltask_test.xml"
dest="xmltask_test.xml">
<insert path="/test">
<![CDATA[
<element>@{number}</element>
]]>
</insert>
</xmltask>
</sequential>
</for>
</target>
>From the memory profiling I did, it seems to increase from ~80Mb to
~140Mb.
Are buffers used in any use of xmltask? What is persisted there? When
might they get cleaned up?
thanks
paul
________________________________
From: ext Brian Agnew [mailto:[EMAIL PROTECTED]
Sent: Sunday, March 30, 2008 8:26 AM
To: Mackay Paul (Nokia-D-MSW/Vancouver)
Cc: [email protected]
Subject: Re: [Xmltask-users] Memory leaks using xmltask
There's been no explicit memory profiling done.
I suspect that XMLTask doesn't clear up the XML doc it's been
manipulating. I will check that and rectify.
Note that buffers are persisted across XMLTask calls and so
those will remain. XMLTask will read in the document as a DOM, and due
to the random access nature of XPath and the ad hoc manipulation of XML,
I don't think there's an alternative to that.
What size of docs are you dealing with, and what specifically
are you doing ? With that info I can start to investigate further. Can
you forward me your test detailed below ?
Brian
[EMAIL PROTECTED] wrote:
Hi,
Has anyone tried testing memory usage of xmltask? We run
a long build using Ant with large XML data content. Recently we
experienced OOM errors in our builds. I did some profiling and found
that running xmltask takes a portion of heap memory which does not get
garbage collected. We had the max heap size configured to 200Mb.
Increasing that has allowed our build to proceed further, but it would
be helpful to know if there are any dangling objects not being cleaned
up. This may also be an Ant problem, but specifically the memory
increases seemed to be around the use of xmltask. I wrote a specific
test that simply inserted an element 10000 times into an initially empty
document and this also grew the heap memory over time.
Also has there been any examination of the memory
profile of processing large XML documents? Although most of the memory
is garbage collected, reading a large document as DOM can take many Mbs
of memory temporarily.
Any suggestions most welcome.
thanks
paul
________________________________
------------------------------------------------------------------------
-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketp
lace
________________________________
_______________________________________________
Xmltask-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xmltask-users
--
Brian Agnew http://www.oopsconsultancy.com
OOPS Consultancy Ltd
Tel: +44 (0)7720 397526
Fax: +44 (0)20 8682 0012
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Xmltask-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xmltask-users
