Oliver Tonnhofer <[email protected]> wrote: > > 4th file XML : 1680 MB (1.64 GB) > > ET (DOM) : n/a > > ET (iterparse) : n/a > > xml.sax : 365 seconds (require about 50 MB), 20 Knodes/sec > > > > Note : > > - ET (DOM) broke, not enough memory > > - ET (iterparse) also use all memory (don't understand why, must be a > > bug in my code)
> > iterparse still builds the (Element)Tree. This makes working with it more > convenient but as you noticed the memory grows. You need to clear() each > element and for very flat XML structures (like OSM) also the root element. > See: http://effbot.org/zone/element-iterparse.htm#incremental-parsing I used elem.clear (that's why the 3rd example do not use memory). For the 4th one (very large) i think i miss to clear the root. > Can you make another test with that or can you post your xml.sax example? > I really like to see how they compare. You can see here > http://pastie.org/1843106 how to get the root of iterparse and how to use > clear. I'm making so tests and debug, and i'll post the final code tomorrow. The actual code : https://github.com/padorange/osm-xml-tests -- Pierre-Alain Dorange OSM experiences : <http://www.leretourdelautruche.com/map/> _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
