Joe Zeff posted on Fri, 26 Aug 2011 21:29:44 -0700 as excerpted: > Right now, I'm house sitting at Chaos Manor (Some of you might know what > that is.) and using my laptop. I have my own domain, and my desktop has > an Internet-visible machine name with dynamic DNS. Before using Pan > here, I tried using rsync to get everything I needed over here with this > command: > > rsync -av [email protected]:~/.pan2 ~/.pan2 > > However, when I ran Pan, it showed every post since the last time I ran > it, earlier this month, as unread and I had to mark them manually. What > did I do wrong? (If it wasn't clear, this is on Linux.)
That rsync command got corrupted on gmane, as evidently it has an @ in it, and gmane took it as an email address and "obfuscated" it. =:^\ As such, this is of necessity a bit hand-wavy. (Added later, right before sending:) I hope the following doesn't come across as mocking. It's not intended to, tho I'm aware it could come across that way, thus this note. Rather, this is an important lesson a lot of users including me have had to learn the hard way. In my case, my lesson on the importance of dry-run and confirming shell substitutions was a script that did an rm -r $variable/, only I typoed $variable so it ended up empty, and did an rm-r /, instead! I made the further mistake of breaking the rules and testing the script "live as root", catching it only after the rm seemed to be taking longer, with way more disk activity, than it should have! If you've never had that sinking feeling of realizing you just did a recursive rm / as root, consider yourself *VERY* lucky! (/bin was gone, /boot, /etc, part of /home... when I hit ^c. Luckily I had a somewhat dated, but workable, backup to restore from.) This experience is why I was SO careful to *CAREFULLY* debug my rsync scripts, making dry-run the default, putting in confirmations, etc. If you've reversed your rsync direction, you've just staled (if you're lucky and didn't sync with an empty source dir!) everything you intended to update from, so rsync's --dry-run is a **VERY** good thing to have and for users to use. (With a reasonable amount of system memory, dry-run isn't not really that bad, either, because the dry-run gets the files into cache so the LIVE run goes much quicker, without the level of disk thrashing the dry-run had, as the files are already cached on both sides.) What I got from gmane's obfuscation: rsync -av user- (user-something @ something) :~/.pan2 ~/.pan2 FWIW, when I use rsync for syncing between my netbook build-image on my main machine, and the netbook itself, I try using --dry-run (-n) first, just to see what it's syncing. In my case, now that I've been using the scripted setup with all the correct switches for sometime, and am thus relatively confident I have /it/ right, sometimes I've changed a few files differently on each machine, between update sessions, and the dry- run allows me to see which ones I might need to manually diff to remember what I changed, before I do the real pre-build-image-update. After I've manually synced them, I do the rsync for real, so both sides are in the same pre-update state, then do the update, then sync again, without making changes to the netbook in the mean time, so the second sync can be one-way without danger of overwriting anything I might have changed on the netbook, since I haven't changed anything on it since the pre-update sync. In your case, however, you'd use dry-run to make sure it's syncing what you intended, not something else. (Before I got in the dry-run-first habit, I once synced without mounting a normally unmounted partition that I should have. It filled up the existing mountpoint partition and errored out. Fortunately, all I had to do was delete all the junk in the otherwise empty mount-point-only dir, but had I been syncing the other way, it would have deleted everything I wanted to sync, since I was syncing with an empty dir! After that close-call, I *ALWAYS* do the dry- run, first!) Note that with --dry-run, you can reverse the sync from what you actually intend to do, as well, to make sure it's syncing the actual files that should be there, not something somewhere entirely unintended, due to a typo or forgetting to mount a partition first, or something. Just be *ABSOLUTELY* *SURE* you have the --dry-run in there first. (That's one benefit of the scripts. If you make --dry-run the default as I did in my scripts, if there's a typo, it does the dry-run and doesn't harm anything. I have to actually set a --LIVE parameter on my script, to get it not to do the dry-run, and the confirmation prompt shows me the command it'll run and specifically notes whether it is LIVE or not, before asking if I'm sure.) The --dry-run first should take care of ensuring that you're doing what you intend to do, but what happened in this case? I don't know for sure, but I can GUESS: If you indeed used ~/ for /home/username/ in both cases, as the (part that I can make out of the) above indicates, that might have been the mistake, since the shell (on the machine you're running the rsync command from) expands that itself -- to the local value. Quoting the bash (other POSIX shells should be similar but may not be identical, there's more to it, but this is the main bit) manpage: Tilde Expansion If a word begins with an unquoted tilde character (`~'), all of the characters preceding the first unquoted slash (or all characters, if there is no unquoted slash) are considered a tilde-prefix. If none of the characters in the tilde-prefix are quoted, the characters in the tilde-prefix following the tilde are treated as a possible login name. If this login name is the null string, the tilde is replaced with the value of the shell parameter HOME. If HOME is unset, the home directory of the user executing the shell is substituted instead. Otherwise, the tilde-prefix is replaced with the home directory associated with the specified login name. Basically, anything after a ~ but before the first / is considered a login name. If it's null, the value of $HOME is substituted. If $HOME is unset or the login name isn't null, the system queryies /etc/passwd (or whatever) for the home dir for that user, substituting the result. Even more basically, under ordinary circumstances, the shell will substitute ~ with the location of the user's home dir, the user in question being the user under which bash is running. My guess is thus that the value the shell substituted for those ~s was the wrong one for either the remote or local side. If you were specifying the rsync user @ host (spaces added to avoid gmane obfuscation) information, it's VERY likely that the local shell's substitution for those ~s was NOT what you intended. Once again, you'd have very likely caught this easily, had you used a --dry-run first, or even had you simply verified the output of what it actually did. However, it would appear you didn't, so you didn't catch it. Meanwhile, check your systems to see what DID get synced. It's very possible you synced something between /root dirs, if you were running as root at the time, or something similarly not what you expected, but doing exactly what rsync was told to do after the shell finished doing its substitutions for those tildes. =:^) Of course, that's another benefit of using a script that shows you the command as it's going to run it, with the same substitutions made, then asks you to confirm that's what you really intend to do, before it does it. I very quickly learned the benefit of this myself, while debugging my own sync scripts, trying to get them setup so that what I was /telling/ rsync to do was exactly what I /intended/ to tell it to do! Had you been using such a script, you'd have seen the substitutions the shell made, and would have almost certainly noted if it was trying to sync the wrong path due to using substitutions other than what you intended. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman _______________________________________________ Pan-users mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/pan-users
