On Mon, Jun 03, 2002, David A. Desrosiers wrote: > There are 26522 straight .html files,
I doubt you will be able to create a Plucker document with that many files. The largest Plucker document I have created has about 1700 records and it was a PITA to create ;-) > The python process currently is using 995 megs of ram and 396 megs of swap, > but the processor load is well under 1%. I don't think it will help to profile the app. To fix this you probably need a major re-design of the parser. Still, I don't understand how it can create this ridiculous amount of objects that will suck up all available memory. One would think that it shouldn't use that much more memory than the size of the actual contents you are trying to convert. Writing intermediate data to files would seem like a possible solution (the awk parser from the pre-historical days of Plucker did that), i.e. don't keep more data in memory than necessary. /Mike
