Instead of inventing something from scratch, I'd recommend an
existing, standard, flexible, free format, like NetCDF. It even
has Python bindings.
http://my.unidata.ucar.edu/content/software/netcdf/
If you must invent it yourself, use Python marshal format over
HTTP. Works fine.
XML is wonderful stuff, but it is really unsuitable for large
amounts of data or fast transfer.
SAX should not require reading in the whole document, though.
It is almost always the right choice if you are extracting the
data from a document instead of manipulating it.
wunder
--On Friday, January 14, 2005 10:34:02 AM +0800 Ping Yeh <[EMAIL PROTECTED]>
wrote:
That's true. I'm willing to trade some speed for language neutrality.
But after some study it seems the speed is really toooooooo slow...
One big reason is that with DOM and SAX the whole data tree has to
be built before I can do anything with the data. Pull DOM is my
hope of using smaller memory footprint to reduce memory allocation
overheads in python.
I may be forced to use binary format if I can't find better performance.
I'm reluctant to do that because the type and amount of data my experiment
will produce is not yet known exactly and adding flexibility to binary
format is painful.
Ping
Walter Underwood wrote:
There are some XML speed issues that won't go away with a better parser.
Sending floating point numbers as formatted ASCII is never going to be
really fast.
wunder
--On January 13, 2005 3:33:49 PM +0800 Ping Yeh <[EMAIL PROTECTED]> wrote:
Thanks a lot for the reference pointers! I'm now studying
pull DOM, and will go through other modules later. I'll make
performance comparisons available just in case they might be useful.
cheers,
Ping
Fredrik Lundh wrote:
But I haven't found any. I'm not sure this is possible with current
architecture of parsers. Any advise is highly appreciated.
http://online.effbot.org/2004_12_01_archive.htm#element-generator
http://online.effbot.org/2004_12_01_archive.htm#element-generator-2
also:
http://www-106.ibm.com/developerworks/xml/library/x-tipulldom.html
http://cvs.sourceforge.net/viewcvs.py/splice/kid/pulltree.py?view=markup
</F>
_______________________________________________
XML-SIG maillist - XML-SIG@python.org
http://mail.python.org/mailman/listinfo/xml-sig
_______________________________________________
XML-SIG maillist - XML-SIG@python.org
http://mail.python.org/mailman/listinfo/xml-sig
--
Walter Underwood
Principal Architect, Verity
--
Walter Underwood
Principal Architect
Verity Ultraseek
_______________________________________________
XML-SIG maillist - XML-SIG@python.org
http://mail.python.org/mailman/listinfo/xml-sig
