On Sun, 22 Jul 2007 22:06:32 +0100, Charlie Savage <[EMAIL PROTECTED]> wrote:
>> there are still a number >> of fundamental problems with libxml-ruby - mostly to do with the fact >> that >> libxml and ruby are constantly fighting over memory management and the >> like. > > Ross - Could you summarize these issues, and exactly what is going wrong > where? It would be extremely helpful as a guide to anyone who has > time to dig into the issue. > > I did a lot of work on SWIG's ruby support, so have a good feeling for > the various memory issues that can arise. But I know nothing about > libxml's api or memory management, or how the ruby bindings interact > with it. > Basically, the problem is with the way we're handling libxml's structs. The best example is with the nodes: When freeing a node (or a document) libxml can free all the children, or not. With the first option we get problems when Ruby-side XML::Nodes have been created, since they free their node when garbage collected. With the second option, we'd have to create Node objects for every node, or otherwise ensure they get freed at the appropriate time - which doesn't work too well with the speed and efficiency thing (esp. with large documents). Currently this is worked around with a sort of pointer-node system - when GCing a node, the libxml struct is only freed if it's a pointer node. IIRC new nodes are created as pointers, and only made into full nodes when they're copied outside of the document, or otherwise made independent of their containing document. This kinda works (with nodes, at least) but is a bit messy. It gets a bit inconsistent when it comes to, for example, attributes though - there are still certain cases that cause things to get freed twice, or not at all. This is fixable, but difficult owing to the way the code's evolved - a case of having the tools, and the time, I guess. There's a fair few comments dotted around the code relating to these issues, though... As an aside, would SWIG make any of this easier? I remember reading a comment (from Sam Ruby I think) that we should junk the current code and rewrite with SWIG. I disagreed at the time, but maybe he had a point? > Also, does the patch from open street map fix the issue? I honestly don't know. > >> The Windows issue matters, too, but since I've been lucky enough to >> avoid >> Windows almost entirely in my professional life for several years now I >> don't really know the platform well enough to handle this, and frankly >> I'm >> unwilling to take the time to learn for the sake of compatibility on one >> project. Another thing that needs to be done. > > I can solve the windows issues. I have a patch to fix up memory > allocation on windows, and with my experience with ruby-prof, I can > easily create a win32 gem. > Cool - that would definitely be a big step forward for the project. To be honest, this was one of the major things that pissed me off before - a lot of people were bemoaning our lack of Win32 gems, but no-one wanted to help with it. I ended up spending far more time that I wanted to messing about with a QEMU windows installation, and in the end admitted defeat since every way I turned seemed to require spending (more) money on Microsoftware I personally don't need. Cheers, Ross -- Ross Bamford - [EMAIL PROTECTED] _______________________________________________ libxml-devel mailing list libxml-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/libxml-devel
