Actually this diff better shows the proposed changes:
https://github.com/robertfeldt/LightXML.jl/commit/238fbb5c41629e254088dfe7553a53c72bd5dffa
Will test and see if it helps.
/Robert
Den måndag 26 januari 2015 kl. 17:36:47 UTC+1 skrev Robert Feldt:
>
> Ok, great. I'm not used to explicit mem management from Julia so
> appreciate feedback on proposed solution:
>
> function free(xelem::XMLElement)
> ccall(xmlFreeNode, Void, (Ptr{Void},), xelem.node.ptr)
> xelem.node.ptr = nullptr
> end
>
> since XMLElement type has a node::XMLNode attribute and the XMLNode type
> has a ptr::Xptr and free for XMLDocument uses xmlFreeDoc like so:
>
> function free(xdoc::XMLDocument)
> ccall(xmlFreeDoc, Void, (Ptr{Void},), xdoc.ptr)
> xdoc.ptr = nullptr
> end
>
> and then add free as finalizer in constructor for XMLElement like so:
>
> function XMLElement(node::XMLNode)
> if !is_elementnode(node)
> throw(ArgumentError("The input node is not an element."))
> end
> xelem = new(node)
> finalize(xelem, free)
> xelem
> end
>
> I will try this and see if it addresses my problems but would be great
> with input on this.
>
> Cheers,
>
> Robert
>
> Den måndag 26 januari 2015 kl. 15:49:14 UTC+1 skrev Isaiah:
>>
>> The underlying XMLNode needs to be freed using xmlFreeNode from libxml2:
>>
>> http://xmlsoft.org/html/libxml-tree.html#xmlFreeNode
>>
>> I don't think this is wrapped yet, but it should be straightforward to
>> add -- ideally, called via a finalizer so that the objects are freed
>> automatically when Julia decides they can be GC'd.
>>
>> On Mon, Jan 26, 2015 at 7:42 AM, Robert Feldt <[email protected]>
>> wrote:
>>
>>> Hi,
>>>
>>> Does anyone now how to free up the memory in a XMLElement object in
>>> LightXML.jl?
>>>
>>> LightXML.jl has a free(xdoc::XMLDocument) but no corresponding
>>> free(xelem::XMLElement). There are functions related to free in clib.jl in
>>> the package but no docs that I can find on how to use it. I tried to create
>>> an XMLDocument from the element and then free the document but mem still
>>> leaks. Help appreciated.
>>>
>>> Background/context/details:
>>>
>>> In another thread we found out that our problems with mem-leaks in a
>>> long-running optimization is probably due to the use of LightXML.jl. The
>>> latter seem to require manual free'ing of resources as of the current
>>> version. Basically our main loop is something like this:
>>>
>>> # Create a large number of XMLElement's (possibly deeply nested) with
>>> construct_element(name::String, content::Array{Any}) = begin
>>> xmlelement = new_element(name)
>>> for item in content
>>> if typeof(item) <: Main.XMLElement
>>> add_child(xmlelement, item)
>>> elseif typeof(item) <: (String,String)
>>> set_attribute(xmlelement, item[1], item[2])
>>> elseif typeof(item) <: String
>>> add_text(xmlelement, item)
>>> else
>>> @assert false
>>> end
>>> end
>>> xmlelement
>>> end
>>>
>>> # What we are optimizing is the length of the xml when printed as string
>>> so we evaluated this by looping over array of xml elements generated
>>> # by construct_element above and calling on each one (the replace is for
>>> taking away at least some of the whitespace which is not important):
>>> fitness(xml) = length(replace(string(xml), r">\s+<", "><"))
>>>
>>> Since the optimization algorithm calls fitness on 100K to several
>>> million xml elements per run when we compare several optimization
>>> algorithms to each other the mem expansion gets up to many gigs and
>>> eventually there is problems. It seems there is no free on XMLElement in
>>> LightXML.jl but there is a free for XMLDocument. We thus tried to change to
>>> this:
>>>
>>> free_xmlelement(xmlelem::XMLElement) = begin
>>> tempdoc = XMLDocument()
>>> set_root(tempdoc, xmlelem)
>>> free(tempdoc)
>>> end
>>> fitness_with_free(xml) = begin
>>> qv = length(replace(string(xml), r">\s+<", "><"))
>>> free_xmlelement(xml)
>>> qv
>>> end
>>>
>>> but it does not seem to help. We would appreciate any help/advice on
>>> this or how we can further debug it. The documentation for xmlFreeDoc in
>>> libxml2 (which is what is what LightXML.free(xdoc::XMLDocument) calls)
>>> says that it will recursively free the document. But maybe we are missing
>>> something.
>>>
>>> Thanks for any advice,
>>>
>>> Robert Feldt
>>>
>>
>>
