What a shame. I think we should investigate alternatives, but high
memory usage is better than crashing. :'(
I should point out that we also have high load but we haven't
experienced any segfaulting - so it may be configuration dependent, but
not necessarily....
I'll try and take a look at this when I get the chance to see if we can
find a way to have our cake and eat it too.
- Joe Khoobyar
Charlie Savage wrote:
#0 rxml_attr_mark (xattr=0x0) at ruby_xml_attr.c:41
#1 0xb7ed6a15 in gc_mark_children (ptr=3050895040, lev=1) at gc.c:945
#2 0xb7ed6c49 in mark_locations_array (x=0xbfc32f90, n=39) at gc.c:629
#3 0xb7ed6e17 in garbage_collect () at gc.c:1366
#4 0xb7ed79c5 in ruby_xmalloc (size=48) at gc.c:103
#5 0xb71855fa in xmlNewPropInternal (node=0xa20a190, ns=0x0,
name=0x84983b0 "k", value=0xa20a170 "created_by", eatname=0) at
tree.c:1791
One of my colleagues thinks he has spotted the problem:
"the attribute is being allocated (rxml_attr_alloc), which sets the data
pointer to NULL. almost immediately, the initialise method is called
(rxml_attr_initialize) where the self object has that NULL data
pointer. when it gets to xmlNewProp it triggers the GC which tries to
mark the current object, which hasn't finished initialising yet...
This is an interesting one, and the diagnosis is correct. It is
caused by having libxml use ruby's memory allocator.
1. Create a new ruby attribute object
2. Initialize it
3. Call libxml xmlNewNsProp
4. It asks ruby for memory
5. Ruby runs a gc since it has no memory
6. The newly created, but not initialized attribute has its mark
function called
The reason for that change was it greatly reduced libxml-ruby's memory
usage.
Making rxml_attr_mark() return immediately if xattr is null seems to
have stopped it segving anyway - the mark routine for nodes is
already doing that in fact.
Yes, that would work. But I wonder if there are other cases of
something similar happening.
I think you're seeing it because you probably have a lot higher load
on your server then what we test with. I tried to duplicate the issue
but didn't succeed:
def test_high_allocations
node = XML::Node.new('test')
1.upto(100000) do |i|
name = "attr_#{i}"
XML::Attr.new(node, name, i.to_s)
end
assert(true)
end
Anyway, I added the check and went back to using libxml's internal
memory allocator.
Fixes in 1.1.0 which I just uploaded. Let me know how it goes.
Charlie
------------------------------------------------------------------------
_______________________________________________
libxml-devel mailing list
libxml-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/libxml-devel
_______________________________________________
libxml-devel mailing list
libxml-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/libxml-devel
