What happens if you first load the whole thing into memory, ie (let Invoices (in ... and then work with the Invoices variable?
On Thu, May 31, 2012 at 9:56 PM, Joe Bogner <joebog...@gmail.com> wrote: > I'd like to do some analysis on a large text file of invoices - for example, > grouping and summing. Is this an efficient way to build the list? > > The file has 4 million rows in it. The first several hundred thousand load > quickly and then I notice the time between my checkpoints taking longer (10 > secs+ per 10,000). I ran it for about 10 minutes and killed it and was > around 2.4M rows > > I played around with other variations of building a list push1, idx, etc and > as expected they all had a more significant time growth as the list grew. > > (make > (in "invoices.txt" > (until (eof) > (setq Line (line) ) > (at (0 . 10000) (prinl Line)) > (link (mapcar pack (split Line "^I"))) ) ) ) > > > My file has: > Customer # > Prod # > Amount > Quantity > > I'm experimenting with loading into a db right now to see if that yields > better results. > > Thanks > Joe -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
