What happens if you first load the whole thing into memory, ie (let
Invoices (in ... and then work with the Invoices variable?
On Thu, May 31, 2012 at 9:56 PM, Joe Bogner <joebog...@gmail.com> wrote:
> I'd like to do some analysis on a large text file of invoices - for example,
> grouping and summing. Is this an efficient way to build the list?
> The file has 4 million rows in it. The first several hundred thousand load
> quickly and then I notice the time between my checkpoints taking longer (10
> secs+ per 10,000). I ran it for about 10 minutes and killed it and was
> around 2.4M rows
> I played around with other variations of building a list push1, idx, etc and
> as expected they all had a more significant time growth as the list grew.
> (in "invoices.txt"
> (until (eof)
> (setq Line (line) )
> (at (0 . 10000) (prinl Line))
> (link (mapcar pack (split Line "^I"))) ) ) )
> My file has:
> Customer #
> Prod #
> I'm experimenting with loading into a db right now to see if that yields
> better results.