This is the classic asynchronous data transfer problem. I boy, have I done a lot of these over the years.
The old-fashioned (meaning old 1980's UNIX way) of doing this is to transfer the data file. Then when complete, transfer a zero-byte flag file of the same name with a *.flag extension. On the receiving end have a watchdog process wake up every now and then (or run from cron; an inittab/restart process with a sleep loop works best), and look for any flag files; if it finds one, process the associated data file.
More recently (mid-1990's) I worked on a project the was truly asynchronous in nature. When the data-source would finish transferring a data file to the target machine, it would then run a remote command to process that file, giving the filename as the argument (using rsh or ssh). The target machine responsible for displaying the data had no cron-job or watchdog at all, just an executable (usually a script) called by the source machine(s) to process data files.
The advantage to the latter is it is more responsive -- processing of the data file takes place as soon as transferred, and there is the possibility of processing more than one file at a time, possibly from multiple sources.
On 7-Apr-2005 15:18 EDT, Mark Freeze wrote:
The way I need my script to read will not allow for picking up part of a file then picking the rest of the file up later, unless by later you guys mean later like a couple of ms later. My script would need to do the following things:
1. Check for the existance of a file. 2. Download the file. 3. Run the file through a parsing program. 4. Import the parsed file into a database. 5. Query the database and email the results to a recipient list. (Quantity and dollar totals of the downloaded file.) 6. Export the file into a Samba directory so my Windows box can pick it up and process it through some canned software then place the result file back into the Samba directory. 7. Use php to convert the result file into seperate pdf images. 8. Place the pdf images into a directory and index the pdf file list into a web-enabled database so users can log into my website and view customers bills (the pdfs) online.
On some small files this process would be almost instanateous. But on larger files the download might take a while so I didnt want my script getting part of the file and starting the next step before the download was complete.
-- Scott G. Hall Raleigh, NC, USA [EMAIL PROTECTED] -- TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug TriLUG Organizational FAQ : http://trilug.org/faq/ TriLUG Member Services FAQ : http://members.trilug.org/services_faq/ TriLUG PGP Keyring : http://trilug.org/~chrish/trilug.asc
