At 10:28 19/02/01 -0600, Evil Twin wrote:
> I have a ship at sea that will have 24/7 network connectivity via
>C-band satellite comm. The bandwidth is 2Mb/sec (I am not sure how
>reliable the C-band comm will be)
Expect a high error rate, and for TCP applications not the greatest
throughput. Make sure your tcp window sizes on both ends are cranked way up
to handle the high latency (to at least 2Mb/s*600ms/(8*1024)=150kB)
Anyway, back to wget :-)
>Data files are appended to until
>the file names are changed, generally at the end of a survey line. I
>would like to mirror some of the collected data to a Linux system
>inhouse but, for those files that are open and being appended to, I
>don't want to resend the entire file, just the portion that I don't
>have. So, the directories that I want to mirror will consist of closed
>files and files that are being appended to.
I suspect wget isn't quite the right tool for this - you don't need its
fancy recursive features nor html parsing and so on. It will do it, no
doubt, but you may be better with something else. The mirror tool might be
better (pure ftp, in pure perl, very nice regexp support if needed), or
possibly even better is rsync, which can (to a very reasonable degree)
transfer just the pieces of the file that have changed, rather than the
whole thing each time. Since you have root on both ends, you can install an
rsync server on the ship, and point it at the appropriate directories.
The only thing that might cause a little grief is rsync/mirror'ing an open
file. If you maintain a lock on the file while writing to it, other apps
may get blocked. If it's purely a write lock it should be Ok (OS
dependent). If it closes the file when not writing to it, and then briefly
opens it to append to it, again you should be Ok most of the time.
HTH.
Cheers,
Markus
Markus Buchhorn, Faculty of Engineering and IT, | Ph: +61 2 61258810
email: [EMAIL PROTECTED], mail: CSIT Bldg #108 |Fax: +61 2 61250010
Australian National University, Canberra 0200, Australia |Mobile: 0417 281429
** Note new address and phone numbers **
