From: Craig Cardimon [mailto:[EMAIL PROTECTED]
>
> I'm working with large numbers of files that can vary in size up to
> 4 GB and more. Every time I try tackling them straight on I run out
> of virtual memory. The files are sequentially numbered somewhat like
> SS numbers -- they contain hyphens. The file numbers are unique as
> far as I can tell.
>
> Is there a loop I can use that will process the first 10,000 files,
> then exit; then the next 10,000 files, then exit; and so on until
> all files have been processed? The exiting is so I can run a SQL
> Server job to insert the processed files into SQL Server.
What I would do is create a file that contains the last file number
you worked on. Then the program can read that number, loop across
the next 10,000 and then update the file before exiting.
open LASTNUM, "lastnum.dat";
$lastnum = <LASTNUM>;
close LASTNUM;
$curfile = $lastnum;
for $i (1 .. 10000)
{
$curfile++;
# process file $curfile;
}
open LASTNUM, ">lastnum.dat";
print LASTNUM "$curfile\n";
close LASTNUM;
Of course you'd need to add a bit of logic to handle detecting the end
of the file list, but you get the idea.
Bowie
_______________________________________________
ActivePerl mailing list
[email protected]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
