Craig Cardimon wrote:
> I'm working with large numbers of files that can vary in size up to 4
> GB and more. Every time I try tackling them straight on I run out of
> virtual memory. The files are sequentially numbered somewhat like SS
> numbers -- they contain hyphens. The file numbers are unique as far
> as I can tell.
I don't know what an SS number is, and its probably not relevant. I just
hope that its explanation doesn't invoke Godwin's law. :-)
>
> Is there a loop I can use that will process the first 10,000 files,
> then exit; then the next 10,000 files, then exit; and so on until all
> files have been processed? The exiting is so I can run a SQL Server
> job to insert the processed files into SQL Server.
The loop itself is fairly simple, for example:
my $number_of_files = get_number_of_files_to_process();
for (1..$number_of_files) {
my $file = get_next_filename();
last unless defined $file;
process_file($file);
mark_file_as_processed($file);
}
As long as the list of files to be processed does not change during
processing, then a simple solution involves 2 files. One containing a
the names of files to be processed, and the other containing the name of
the last file processed.
HTH
--
Brian Raven
=================================
Atos Euronext Market Solutions Disclaimer
=================================
The information contained in this e-mail is confidential and solely for the
intended addressee(s). Unauthorised reproduction, disclosure, modification,
and/or distribution of this email may be unlawful.
If you have received this email in error, please notify the sender immediately
and delete it from your system. The views expressed in this message do not
necessarily reflect those of Atos Euronext Market Solutions.
L'information contenue dans cet e-mail est confidentielle et uniquement
destinee a la (aux) personnes a laquelle (auxquelle(s)) elle est adressee.
Toute copie, publication ou diffusion de cet email est interdite. Si cet e-mail
vous parvient par erreur, nous vous prions de bien vouloir prevenir
l'expediteur immediatement et d'effacer le e-mail et annexes jointes de votre
systeme. Le contenu de ce message electronique ne represente pas necessairement
la position ou le point de vue d'Atos Euronext Market Solutions.
_______________________________________________
ActivePerl mailing list
[email protected]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
