On 25.11.2014 15:02, net.rs...@io7m.com wrote: > 'Lo. > > I've run into a frustrating issue when trying to synchronize a > directory hierarchy over a reliable (but slow) connection to an > unreliable remote. Basically, I have the following: > > http://mvn-repository.io7m.com/com/io7m/ > > This is a set of nested directories containing binaries and sources for > projects I develop/maintain. Every time a new release is made, I deploy > the binaries and sources to an exact copy of the above hierarchy on my > local machine, and then rsync that (over SSH) to > mvn-repository.io7m.com. > > $ rsync -avz --delete --progress local/ > io7m.com:/home/io7m/mvn-repository.io7m.com/ > > The problem: > > The latest project produces one .jar file that's about 80mb. > Evidently, the hosting provider I use for io7m.com is using some sort > of process tracking system that kills processes that have been running > for too long (I think it just measures CPU time). The result of this is > that I get about 50% of the way through copying that > (comparatively) large file, and then the remote rsync process is > suddenly killed because it has been running for too long. > > This would be fine, except that it seems that rsync is utterly refusing > all efforts to continue copying that file from wherever it left off. It > always restarts copying of the file from nothing and tries to copy the > full 80mb, resulting it being killed halfway through and causing much > grinding of teeth. > > The documentation for --partial states that "Using the --partial option > tells rsync to keep the partial file which should make a subsequent > transfer of the rest of the file much faster.". Well, for whatever > reason, it doesn't (or it at least fails to continue using it). > > I've tried --partial-dir, passing it an absolute path to the temporary > directory in my home directory. It created a file in there the first time, > but after being killed by the remote side and restarting, it ignored > that file and instead created a new temporary file (with a random suffix) > in the destination directory! Am I doing something wrong? > > $ rsync -avz --delete --progress --partial-dir=/home/io7m/tmp/rsync > io7m.com:/home/io7m/mvn-repository.io7m.com/ > > I'm at a loss. How can I reliably get this directory hierarchy up onto > the server? I don't care if I have to retry the command multiple times > until the copy has fully succeeded, but I obviously can't do that if > rsync keeps restarting the failed file from scratch every time.
--inplace --partial The later prevents rsync from discarting partial files, the former makes rsync "reuse" already transfered parts and overwriting/appending as needed. You only have to be careful if you use hardlinks, --inplace can have "bad interaction" with hardlinks. -- Matthias -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html