Pascal Bleser <[EMAIL PROTECTED]> writes: > Ulrich Windl wrote: >> On 23 May 2006 at 14:09, jdd wrote: >> >>> If I follow well the thread it seems this is a meta-data >>> download problem. >> >> If it's a download problem, why would the CPU be at 100%? > > That sounds a lot like XML repository metadata parsing. > libzypp/ZMD is most probably not parsing the data stream as it is > being downloaded, so I presume it's download everything (for one > repository) first, then parse. > > Would be interesting to do some profiling on parse-metadata. > Anything available for Mono ?
parse-metadata is in C++.
>
> What XML parsing model is being used there, SAX, DOM, StAX ?
>
> Most probably not DOM...
libxml2's most appropriate one for this, don't know what it is.
>
> When I look at my ("guru") RPM-MD repository for 10.0 (which is large,
> but a lot smaller than the FTP tree):
> == compressed:
> primary.xml.gz = 1,058,326 (bytes)
> filelists.xml.gz = 1,029,756
> other.xml.gz = 497,642
>
> == uncompressed:
> primary.xml = 6,174,750
> filelists.xml = 11,032,393
> other.xml = 2,950,989
>
> == compression ratio:
> primary.xml = 5.83
> filelists.xml = 10.42
> other.xml = 5.92
>
> Now when I look at SL-10.1/inst-source/suse/repodata:
> == compressed:
> primary.xml.gz = 8,056,136 (bytes)
> filelists.xml.gz = 17,474,199
> other.xml.gz = 53,265,854
>
> BTW, other.xml.gz is *huge* (contains %changelog information) - I
> don't know whether libzypp/zmd download and/or use "other.xml.gz"
> though. smart (http://smartpm.org) doesn't.
It does not use it anymore. That was one of the early fixes ;-)
>
> == uncompressed:
> primary.xml = 47,127,500
> filelists.xml = 211,884,423
> other.xml = 206,534,422
>
> == compression ratio:
> primary.xml = 5.83
> filelists.xml = 12.12
> other.xml = 3.87
>
> Assuming that other.xml is not being used by libzypp/ZMD, I would
> guess the following memory usage with DOM:
> - primary.xml: 47MB on disk => 150-200MB RAM
> - filelists.xml: 210MB on disk => 600-800MB RAM
>
> Hmm.. after all... maybe it _is_ DOM ;)
No, definitely not,
Andreas
--
Andreas Jaeger, [EMAIL PROTECTED], http://www.suse.de/~aj/
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
pgpvti5ADKu79.pgp
Description: PGP signature
