Ok, thanks. The finalizer is called and often work but sometimes there is a
malloc error. Ok, I need to check the rest of the code. Might be in rest of
LightXML or in my code, I guess.
Thanks,
Robert
Den tisdag 27 januari 2015 kl. 04:12:36 UTC+1 skrev Isaiah:
>
> I'm not sure where the malloc error is coming from, exactly, but you need
> to call "finalizer", not "finalize". Also it should be run on the XMLNode
> rather than the XMLElement.
>
> The following example might help (at least, it doesn't crash!)
>
> julia> n = new_textnode("s")
> s
> julia> finalizer(n, z::XMLNode -> begin
> #ccall(:puts, Void, (Ptr{Uint8},),
> "called finalizer\n") #uncomment to verify
> ccall(dlsym(LightXML.libxml2,
> :xmlFreeNode), Void, (Ptr{Void},), z.ptr);
> end)
> julia> n = nothing
> julia> gc()
>
>
> On Mon, Jan 26, 2015 at 11:50 AM, Robert Feldt <[email protected]
> <javascript:>> wrote:
>
>> I fixed the typo (should be finalizer and not finalize) and also had to
>> import xmlFreeNode from the c lib but when I run I get:
>>
>> julia(56167,0x7fff7924c300) malloc: *** error for object 0x7f9a33b159c0:
>> pointer being freed was not allocated
>> *** set a breakpoint in malloc_error_break to debug
>>
>> so there is something more at play here. Appreciate ideas.
>>
>> Cheers,
>>
>> Robert
>>
>> Den måndag 26 januari 2015 kl. 17:43:04 UTC+1 skrev Robert Feldt:
>>
>>> Actually this diff better shows the proposed changes:
>>>
>>> https://github.com/robertfeldt/LightXML.jl/commit/
>>> 238fbb5c41629e254088dfe7553a53c72bd5dffa
>>>
>>> Will test and see if it helps.
>>>
>>> /Robert
>>>
>>> Den måndag 26 januari 2015 kl. 17:36:47 UTC+1 skrev Robert Feldt:
>>>>
>>>> Ok, great. I'm not used to explicit mem management from Julia so
>>>> appreciate feedback on proposed solution:
>>>>
>>>> function free(xelem::XMLElement)
>>>> ccall(xmlFreeNode, Void, (Ptr{Void},), xelem.node.ptr)
>>>> xelem.node.ptr = nullptr
>>>> end
>>>>
>>>> since XMLElement type has a node::XMLNode attribute and the XMLNode
>>>> type has a ptr::Xptr and free for XMLDocument uses xmlFreeDoc like so:
>>>>
>>>> function free(xdoc::XMLDocument)
>>>> ccall(xmlFreeDoc, Void, (Ptr{Void},), xdoc.ptr)
>>>> xdoc.ptr = nullptr
>>>> end
>>>>
>>>> and then add free as finalizer in constructor for XMLElement like so:
>>>>
>>>> function XMLElement(node::XMLNode)
>>>> if !is_elementnode(node)
>>>> throw(ArgumentError("The input node is not an element."))
>>>> end
>>>> xelem = new(node)
>>>> finalize(xelem, free)
>>>> xelem
>>>> end
>>>>
>>>> I will try this and see if it addresses my problems but would be great
>>>> with input on this.
>>>>
>>>> Cheers,
>>>>
>>>> Robert
>>>>
>>>> Den måndag 26 januari 2015 kl. 15:49:14 UTC+1 skrev Isaiah:
>>>>>
>>>>> The underlying XMLNode needs to be freed using xmlFreeNode from
>>>>> libxml2:
>>>>>
>>>>> http://xmlsoft.org/html/libxml-tree.html#xmlFreeNode
>>>>>
>>>>> I don't think this is wrapped yet, but it should be straightforward to
>>>>> add -- ideally, called via a finalizer so that the objects are freed
>>>>> automatically when Julia decides they can be GC'd.
>>>>>
>>>>> On Mon, Jan 26, 2015 at 7:42 AM, Robert Feldt <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Does anyone now how to free up the memory in a XMLElement object in
>>>>>> LightXML.jl?
>>>>>>
>>>>>> LightXML.jl has a free(xdoc::XMLDocument) but no corresponding
>>>>>> free(xelem::XMLElement). There are functions related to free in clib.jl
>>>>>> in
>>>>>> the package but no docs that I can find on how to use it. I tried to
>>>>>> create
>>>>>> an XMLDocument from the element and then free the document but mem still
>>>>>> leaks. Help appreciated.
>>>>>>
>>>>>> Background/context/details:
>>>>>>
>>>>>> In another thread we found out that our problems with mem-leaks in a
>>>>>> long-running optimization is probably due to the use of LightXML.jl. The
>>>>>> latter seem to require manual free'ing of resources as of the current
>>>>>> version. Basically our main loop is something like this:
>>>>>>
>>>>>> # Create a large number of XMLElement's (possibly deeply nested) with
>>>>>> construct_element(name::String, content::Array{Any}) = begin
>>>>>> xmlelement = new_element(name)
>>>>>> for item in content
>>>>>> if typeof(item) <: Main.XMLElement
>>>>>> add_child(xmlelement, item)
>>>>>> elseif typeof(item) <: (String,String)
>>>>>> set_attribute(xmlelement, item[1], item[2])
>>>>>> elseif typeof(item) <: String
>>>>>> add_text(xmlelement, item)
>>>>>> else
>>>>>> @assert false
>>>>>> end
>>>>>> end
>>>>>> xmlelement
>>>>>> end
>>>>>>
>>>>>> # What we are optimizing is the length of the xml when printed as
>>>>>> string so we evaluated this by looping over array of xml elements
>>>>>> generated
>>>>>> # by construct_element above and calling on each one (the replace is
>>>>>> for taking away at least some of the whitespace which is not important):
>>>>>> fitness(xml) = length(replace(string(xml), r">\s+<", "><"))
>>>>>>
>>>>>> Since the optimization algorithm calls fitness on 100K to several
>>>>>> million xml elements per run when we compare several optimization
>>>>>> algorithms to each other the mem expansion gets up to many gigs and
>>>>>> eventually there is problems. It seems there is no free on XMLElement in
>>>>>> LightXML.jl but there is a free for XMLDocument. We thus tried to change
>>>>>> to
>>>>>> this:
>>>>>>
>>>>>> free_xmlelement(xmlelem::XMLElement) = begin
>>>>>> tempdoc = XMLDocument()
>>>>>> set_root(tempdoc, xmlelem)
>>>>>> free(tempdoc)
>>>>>> end
>>>>>> fitness_with_free(xml) = begin
>>>>>> qv = length(replace(string(xml), r">\s+<", "><"))
>>>>>> free_xmlelement(xml)
>>>>>> qv
>>>>>> end
>>>>>>
>>>>>> but it does not seem to help. We would appreciate any help/advice on
>>>>>> this or how we can further debug it. The documentation for xmlFreeDoc in
>>>>>> libxml2 (which is what is what LightXML.free(xdoc::XMLDocument) calls)
>>>>>> says that it will recursively free the document. But maybe we are
>>>>>> missing
>>>>>> something.
>>>>>>
>>>>>> Thanks for any advice,
>>>>>>
>>>>>> Robert Feldt
>>>>>>
>>>>>
>>>>>
>
