Thx for this. I'll run some tests and have a look at what's happening. I
can't say much until I've had a look, but the below is sufficiently simple
that I should be able to determine pretty quickly...
On Wed, April 2, 2008 21:42, [EMAIL PROTECTED] wrote:
> Our real-world use case is merging into one file a number of XML files
> that total approx 3-4Mb in size, maybe more.
>
> A test case I tried is below:
>
> <target name="try-xmltask">
> <xmltask dest="xmltask_test.xml">
> <insert path="/" >
> <![CDATA[
> <test/>
> ]]>
> </insert>
> </xmltask>
> <script language="jython">
> result = ",".join([str(i) for i in range(10000)])
> project.setProperty("xml.test.list", str(result))
> </script>
> <echo>${xml.test.list}</echo>
> <for param="number" list="${xml.test.list}">
> <sequential>
> <xmltask source="xmltask_test.xml"
> dest="xmltask_test.xml">
> <insert path="/test">
> <![CDATA[
> <element>@{number}</element>
> ]]>
> </insert>
> </xmltask>
> </sequential>
> </for>
> </target>
>
>>From the memory profiling I did, it seems to increase from ~80Mb to
> ~140Mb.
>
> Are buffers used in any use of xmltask? What is persisted there? When
> might they get cleaned up?
>
> thanks
>
> paul
>
>
>
>
> ________________________________
>
> From: ext Brian Agnew [mailto:[EMAIL PROTECTED]
> Sent: Sunday, March 30, 2008 8:26 AM
> To: Mackay Paul (Nokia-D-MSW/Vancouver)
> Cc: xmltask-users@lists.sourceforge.net
> Subject: Re: [Xmltask-users] Memory leaks using xmltask
>
>
> There's been no explicit memory profiling done.
>
> I suspect that XMLTask doesn't clear up the XML doc it's been
> manipulating. I will check that and rectify.
>
> Note that buffers are persisted across XMLTask calls and so
> those will remain. XMLTask will read in the document as a DOM, and due
> to the random access nature of XPath and the ad hoc manipulation of XML,
> I don't think there's an alternative to that.
>
> What size of docs are you dealing with, and what specifically
> are you doing ? With that info I can start to investigate further. Can
> you forward me your test detailed below ?
>
> Brian
>
> [EMAIL PROTECTED] wrote:
>
> Hi,
>
> Has anyone tried testing memory usage of xmltask? We run
> a long build using Ant with large XML data content. Recently we
> experienced OOM errors in our builds. I did some profiling and found
> that running xmltask takes a portion of heap memory which does not get
> garbage collected. We had the max heap size configured to 200Mb.
> Increasing that has allowed our build to proceed further, but it would
> be helpful to know if there are any dangling objects not being cleaned
> up. This may also be an Ant problem, but specifically the memory
> increases seemed to be around the use of xmltask. I wrote a specific
> test that simply inserted an element 10000 times into an initially empty
> document and this also grew the heap memory over time.
>
> Also has there been any examination of the memory
> profile of processing large XML documents? Although most of the memory
> is garbage collected, reading a large document as DOM can take many Mbs
> of memory temporarily.
>
> Any suggestions most welcome.
>
> thanks
>
> paul
>
>
> ________________________________
>
>
>
> ------------------------------------------------------------------------
> -
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
>
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketp
> lace
> ________________________________
>
>
> _______________________________________________
> Xmltask-users mailing list
> Xmltask-users@lists.sourceforge.net
>
> https://lists.sourceforge.net/lists/listinfo/xmltask-users
>
>
>
> --
> Brian Agnew http://www.oopsconsultancy.com
> OOPS Consultancy Ltd
> Tel: +44 (0)7720 397526
> Fax: +44 (0)20 8682 0012
>
>
>
--
Brian Agnew http://www.oopsconsultancy.com
OOPS Consultancy Ltd
Tel: +44 (0)7720 397526
Fax: +44 (0)20 8682 0012
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Xmltask-users mailing list
Xmltask-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xmltask-users
