On Apr 7, 2005, at 3:18 PM, Mark Freeze wrote:
The way I need my script to read will not allow for picking up part of a file then picking the rest of the file up later, unless by later you guys mean later like a couple of ms later. My script would need to do the following things:
1. Check for the existance of a file. 2. Download the file. 3. Run the file through a parsing program. 4. Import the parsed file into a database. 5. Query the database and email the results to a recipient list. (Quantity and dollar totals of the downloaded file.) 6. Export the file into a Samba directory so my Windows box can pick it up and process it through some canned software then place the result file back into the Samba directory. 7. Use php to convert the result file into seperate pdf images. 8. Place the pdf images into a directory and index the pdf file list into a web-enabled database so users can log into my website and view customers bills (the pdfs) online.
On some small files this process would be almost instanateous. But on larger files the download might take a while so I didnt want my script getting part of the file and starting the next step before the download was complete. In the past instance I processed a file of 80,000 records after I had only downloaded about 60,000. Would rsync or ncftpget still work in this situation? I am deleteing the files after I download. Downloading a second file behind the first sounds like a good idea. I'd just have to have the script check that condition before continuing. How would I go about checking the file size on the remote machine before download to detect things like failed transfers, etc... ?
Thanks, Mark.
I would suggest that the second file uploaded by the clients is an md5 sum of the just uploaded file. You can use that as a trigger as Jeff suggested and as a check that the file is complete.
John
-- TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug TriLUG Organizational FAQ : http://trilug.org/faq/ TriLUG Member Services FAQ : http://members.trilug.org/services_faq/ TriLUG PGP Keyring : http://trilug.org/~chrish/trilug.asc
