On Thu, Sep 22, 2011 at 10:49 PM, Duncan <[email protected]> wrote:
> James Colby posted on Thu, 22 Sep 2011 15:03:57 -0400 as excerpted: > > > On Thu, Sep 22, 2011 at 2:30 PM, Nikos Chantziaras <[email protected]> > > wrote: > > > >> On 09/22/2011 09:23 PM, James Colby wrote: > >> > >>> List Members - > >>> > >>> I am trying to use dolphin to copy a rather large directory (approx. > >>> 22 Gb. 2000 files, 500 directories) from my laptop to a server using > >>> the fish:// protocol. When I first attempted the copy it ran for a > >>> few hours and then died due to a network disconnect. Now when I try > >>> to do the copy again, I am getting an error saying that the directory > >>> already exists, which is true, as it looks like dolphin created the > >>> directory structure first, and then starting copying the files. Does > >>> anyone know of a way to resume this copy or is it possible to tell > >>> dolphin to just skip files and directories that already exist at the > >>> destination? If this is not possible with dolphin does anyone have a > >>> suggestion as to a better way to do this copy? > >>> > >>> > >> Use rsync. It was made for exactly this kind of job. > >> > >> > > Sounds like we have a consensus. :) I've always been a little > > intimidated by rsync but I guess now is the time to man up (man rsync) > > </end bad pun> > > Yet another vote for rsync. Note that once you have ssh setup, as it > appears you already do, rsync over ssh is quite easy indeed. > > My personal usage? I run gentoo on both my 64-bit workstation and my 32- > bit-only netbook. Gentoo is of course build-from-source, but since my > workstation is 64-bit while my netbook is 32-bit, I can't just build once > for both, I have to build separately for each. And the netbook being a > netbook, I'm not particularly eager to do my building on it. So I have a > 32-bit build-image chroot on my 64-bit machine, setup specifically to > handle the building and maintenance for the 32-bit netbook. > > Originally, I copied the image to USB thumbdrive (while it was plugged > into the workstation, of course), booted the thumbdrive on the netbook, > then copied from it to the netbook's harddrive so I could native-boot > from the netbook's own drive. > > To that point I had never used ssh before, as I had always had only the > single machine, so after getting the netbook installed and running in > general, I learned how to setup sshd securely and did so, on the netbook > (while still using the thumbdrive to sync between the workstation and the > netbook, rsyncing twice per transfer, once to the thumbdrive from the > workstation, once from the thumbdrive to the netbook). Then I learned > the ssh client side setup and did that on the workstation. > > It was then only a matter of a couple adjustments to the rsync scripts > that had been syncing to and from the thumbdrive, to use direct rsync on > the LAN, instead. > > Learning how to handle ssh for both server and client was *MUCH* harder > than learning, then scripting, the rsync solution, first to/from the > thumbdrive, then script tweaked only slightly to use direct rsync via ssh > between the hosts themselves, instead of the thumbdrive "sneakernet". > > So if you already have ssh setup, one end server, one end client, as it > sounds like you do, learning to handle rsync over the existing ssh setup > should be no sweat at all, comparatively. > > FWIW the rsync manpage is a bit of a monster, but most of it is dealing > with exclusion file formats, etc, that you can probably do without, at > least at first. There are a few command line switches that are useful > for handling symlinks if you're dealing with them (as I was), but other > than that, it's pretty basic. > > The one hint that I WILL leave you with is to *ALWAYS* use the --dry-run > switch the first time around. In my scripts, I actually made that the > default, and then go back and run the command again, adding a LIVE > parameter, to actually run it. That way, if you screwed up the command, > you can catch the error from the output, before you actually do something > stupid like switch the source and destination, or forget to mount the > source so it copies an empty dir, or some such. Dry-run first will SEEM > to take a lot of time and might SEEM a waste, because of all the drive > seeking it does, but in reality it's caching all those files so the > system doesn't have to work nearly as hard doing the compares the second > time around as the data is already cached, so you'd be spending most of > that time that it can now skip the second time around anyway, if you > hadn't done the dry run. (Tho you may wish to do subdirs or other chunks > of around the same sice as your memory on the source machine, rather than > the whole 22 gig at once, unless you have 22+ gig of RAM for it to cache > to, of course.) Using dry-run has SAVED MY BACON a few times, when I fat- > fingered or fat-neuroned something, so I'd definitely recommend ALWAYS > using it first. Far better safe than sorry, as they say. > > -- > Duncan - List replies preferred. No HTML msgs. > "Every nonfree program has a lord, a master -- > and if you use the program, he is your master." Richard Stallman > > ___________________________________________________ > This message is from the kde mailing list. > Account management: https://mail.kde.org/mailman/listinfo/kde. > Archives: http://lists.kde.org/. > More info: http://www.kde.org/faq.html. > Duncan - Thanks for your answer. I ran my first rsync session over night last night (rsync -av <src_dir> <dest_host>:<dest_dir>) and it seemed to work OK. I was a little disappointed that it only copied about 2 Gb, over 6 hours. I am not sure if that is due to network congestion or rsync overhead, as the day before I was able to copy approx. 7 Gb. over the same 6 hours. Thanks for the --dry-run tip, I'll be sure to use that in the future. Regards, James
___________________________________________________ This message is from the kde mailing list. Account management: https://mail.kde.org/mailman/listinfo/kde. Archives: http://lists.kde.org/. More info: http://www.kde.org/faq.html.
