Re: [XML-SIG] XML processing

2009-02-15 Thread Stefan Behnel
Stefan Behnel wrote: > For minidom, I get about 60MB, where Fredrik got 80MB. That's still about a > factor of 17-23 compared to the serialised XML file, whereas lxml and cET > end up with a factor of 3-5. Your assumption that you can use a system with > 3GB of RAM to parse a 500MB XML file into a

Re: [XML-SIG] XML processing

2009-02-15 Thread Stefan Behnel
Hi, Bill Kinnersley wrote: > Can anyone add any substance to this remark? With today's typical > system RAM of 2GB to 3GB, is it even worth consideration any more that a > document might not fit in memory? At least, it allows you to parse pretty large documents. But think of parallel handling of

Re: [XML-SIG] XML processing

2009-02-14 Thread Bill Kinnersley
John W. Shipman wrote: This document does not discuss the lxml package's toolset for the equivalent of SAX (when the document doesn't fit in memory); Can anyone add any substance to this remark? With today's typical system RAM of 2GB to 3GB, is it even worth consideration any more that a do

Re: [XML-SIG] XML processing

2009-02-14 Thread Stefan Behnel
Hi, Josh English wrote: > The only disadvantage I've run into is my implementation spends a lot > of time parsing and over writing my XML data files, so that overhead > is costly Sounds like you should give lxml a try. It's about as fast as cElementTree on parsing, but several times faster on ser

Re: [XML-SIG] XML processing

2009-02-14 Thread Josh English
I started with DOM processing, but the computer I had back then couldn't handle XML files larger that 60K or so. (Serious memory limitations.) So I learned SAX to extract data from my XML, but had to revert to DOM to put data back in. Since these worked differently, I ended up creating a DOM elemen

Re: [XML-SIG] XML processing

2009-02-13 Thread John W. Shipman
On Fri, 13 Feb 2009, Neil Munro wrote: +-- | I'm a final year university student in the UK and I've chosen | python as my language of choice for implementing my desired solution... | | ...if someone can poin me to tutorials and answer questions on them +-- Here's some documentation for my Python

[XML-SIG] XML processing

2009-02-13 Thread Neil Munro
Hello all I'm a final year university student in the UK and I've chosen python as my language of choice for implementing my desired solution but I'm having issues with getting xml working, the problem is that I've done a lot of reading discovered lots of different libraries and means of