Thanks, Jim. I cut it lots, and it no longer hangs. However, after 600 pages of the 700 it was looking for, it gave me the "PYTHON caused an invalid page fault in module MSVCRT.DLL at..." again.
I am running a Win98 machine with a clean install (four days ago), Athlon 700 Mhz, 128 MB Ram. Maybe that's too big a job for my system? --- "Jewett, Jim J" <[EMAIL PROTECTED]> wrote: > sd shields: > > > I am trying to grab a huge amount of data (100 > meg?) > > with plkr. The site is very straight forward, so > it > > should import well, when I am getting it from the > > internet, the desktop freezes up after getting > several > > hundred pages. > > Try splitting it into several smaller channels. As > to > why it happens: > > (1) The desktop uses the python parser. The python > > parser keeps everything in memory. In theory, the > dictionaries could be replaced with shelves (disk), > but at the moment - plucking a 100meg site will > require at least 100meg of memory (and probably > several times that). > > (2) Even if you solved the memory problem, the > plucker format is limited to about 64K records > because of PalmOS limitations, and perhaps 16K > in practice. Individual records are limited to > about 32K before they get split. You may be > hitting these limits. > > (3) Even if you do have enough memory and > few enough records, you are probably putting a > severe strain on both that network (which > can cause errors, which are not always > handled gracefully) and on your memory > (which can cause garbage collection, > which looks like a freeze). > > -jJ > _______________________________________________ > plucker-list mailing list > [EMAIL PROTECTED] > http://lists.rubberchicken.org/mailman/listinfo/plucker-list __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com _______________________________________________ plucker-list mailing list [EMAIL PROTECTED] http://lists.rubberchicken.org/mailman/listinfo/plucker-list
