Having had to upload a lot of data in the past 15 years or so, mostly during the last 8 years I think I have learnt to adapt to poverty, poor bandwidth conditions, unstable links et al.
I have used a few of the advanced protocols in the UNIX world to move data around. I also download but I think I am more of an upload guy. In fact my LiveCD and LiveUSB projects themselves contribute to nearly 4.8 GB of upload. I have had physical machines that assist this process in the US and now in the UK. But it is not any high funda stuff. Just plain UNIX command line usage. I thought it would make an interesting story for you. I mostly use a combination of a transfer protocol like scp or ftp or rsync and follow it up with a checksum check using sha1. For really large files the checksumming itself will take minutes. The average size of files I deal with are in the range of 600MB to 2 GB. How do I manage to move them around? Wherever rsync is available I use that. But it is a problem due to using a non standard port, due to the need to have it at both ends, so I use scp, dd and some shell commands. dd(1) has an ability to copy from an offset, either from input or output. The issue happens when transfers get cut or interrupted. To speed up transfers you could use a download accelerator. But we are talking about uploads here. So I never did that. If you have multiple Internet links then you could use a better link aggregation strategy. But I have not done such things yet. Maybe now since I have two links here. There are a few points to be noted about uploading data. And upload may choke downloads in certain cases. So I normally leave it running at night in an at(1) job. And in a DSL case which is most common for us, upload speeds are 1/4th of the download speed. Mostly lesser than that. But BSNL links I have used all my life have been very helpful for me. Rock solid and stable. Even the 256 kbps link at my parents' has been able to fetch me a Debian ISO of 650 MB. So when things go well I don't do any UNIX stuff. But I think I downloaded the ISO using jigdo which itself has built in magic for interrupted transfers. For those of you who understood what I spoke just now and have done these things yourself the specific commands would not be hard. But just in case, $ dd if=foo.iso skip=2000 bs=1024 of=rem.bin will help. -Girish -- Gayatri Hitech http://gayatri-hitech.com _______________________________________________ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
