How to mirror only specified directories
I am trying to mirror only select directories from one server to another with rsync through its daemon mode. Server A /export /home /A-do /A-not /A-copy /das /htdocs /docs /em /htdocs /docs /psb /htdocs /docs /X-do /X-not /X-copy Server B /export /home /extranet /das /htdocs /docs /em /htdocs /docs /psb /htdocs /docs I wish to copy everything in the /docs directories under the /das, /em and /psb directories (in this simplified example) moving them from /export/home on server A to /export/home/extranet on server B. This is what I have in a rsyncd.conf on server B. [dprweb_extranet] path = /export/home/extranet comment = California Department of Pesticide Regulation Extranet uid = webuser gid = other read only = no list = yes hosts allow = ww.xxx.yy.zz secrets file = /etc/.password auth users = aaa Here is the command I've tried on server A /usr/local/bin/rsync -n -vvv --stats -Pzrtpl --delete --password-file=/export/home/webuser/.appprod --log-file=/export/home/webuser/logs/rsync-log --include /export/home/[internal|das|em|enf|itb|medtox|pml|psb|reg|whs]/htdocs/docs/* --exclude /* /export/home/xterernal/htdocs/ webu...@appprod::dprweb_extranet This is failing with this message in the output. sending incremental file list rsync: change_dir /export/home/xterernal/htdocs failed: No such file or directory (2) Question A: Why was it looking at the /export/home/xterernal/htdocs directory. The external is one of the many directories I am trying to ignore. Question B: Where is it trying to change to the /export/home/xterernal/htdocs directory? On server B? If so, that is not the desired path, /export/home/extranet/... is the desired destination. Can anybody provide some guidance on how to filter to select directories and mirror them to another server with a different base path? TIA Ian -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: How to mirror only specified directories
On Tue 28 Sep 2010, Ian Skinner wrote: I am trying to mirror only select directories from one server to another with rsync through its daemon mode. Server A /export /home /A-do /A-not /A-copy /das /htdocs /docs /em /htdocs /docs /psb /htdocs /docs /X-do /X-not /X-copy Server B /export /home /extranet /das /htdocs /docs /em /htdocs /docs /psb /htdocs /docs I wish to copy everything in the /docs directories under the /das, /em and /psb directories (in this simplified example) moving them from /export/home on server A to /export/home/extranet on server B. This is what I have in a rsyncd.conf on server B. [dprweb_extranet] path = /export/home/extranet comment = California Department of Pesticide Regulation Extranet uid = webuser gid = other read only = no list = yes hosts allow = ww.xxx.yy.zz secrets file = /etc/.password auth users = aaa Here is the command I've tried on server A /usr/local/bin/rsync -n -vvv --stats -Pzrtpl --delete --password-file=/export/home/webuser/.appprod --log-file=/export/home/webuser/logs/rsync-log --include /export/home/[internal|das|em|enf|itb|medtox|pml|psb|reg|whs]/htdocs/docs/* --exclude /* /export/home/xterernal/htdocs/ webu...@appprod::dprweb_extranet This is failing with this message in the output. sending incremental file list rsync: change_dir /export/home/xterernal/htdocs failed: No such file or directory (2) Question A: Why was it looking at the /export/home/xterernal/htdocs directory. The external is one of the many directories I am trying to ignore. You have explicitly told it to use /export/home/xterernal/htdocs/ as the source directory! Note that you wrote xterernal which is not the same as external. Question B: Where is it trying to change to the /export/home/xterernal/htdocs directory? On server B? If so, that is not the desired path, /export/home/extranet/... is the desired destination. No, on the source, see answer to question B. Can anybody provide some guidance on how to filter to select directories and mirror them to another server with a different base path? You need to know that include/exclude patterns are anchored at the root of the transfer. It's a bit difficult from your message to see what exactly you're trying to transfer and exclude, but if you want to transfer /export/home/external/htdocs/ which contains subdirectories yes and no, to exclude the no directory use this: rsync -aP --exclude /no /export/home/external/htdocs/ remote::whatever/ Also, using [] in patterns is only supported for character ranges as in [0-9], not for lists of words. Paul -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: How to mirror only specified directories
Yes, I just noticed that myself a left over artifact from the source I used as my base example. I'm still struggling to get just the directory(ies) that I want... -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: How to mirror only specified directories
I'm still struggling to get just the directory(ies) that I want... You may find this http://tinyurl.com/rsync-exclude-all-include-some post to the LBackup mailing list helpful. The example listed (link above) revolves around specifying the root directory as the source and then specifying a selection of sub-directories to include in the sync and then finally a rule which will exclude everything else. Please note that each directory within the path must specifically be included for this approach to work. IN addition, this approach utilizes only an excludes file (no include list via the command line). I learnt about this approach initially from a thread on this (rsync) mailing list. I am positive that a similar example will be listed somewhere within the the rsync mailing list archives. Hope this helps. - This email is protected by LBackup, an open source backup solution. Free as in freedom; LBackup is licensed under the GNU http://www.lbackup.org -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: recent discussion regarding 'checksums'
On Mon, 2010-09-27 at 22:33 -0400, Benjamin R. Haskell wrote: But the flip side is that rsync is not a security tool. MD5 is fine for rsync for the same reason SHA-1 (which, as with all hashes, will eventually be broken) is fine for git: This gets a little off topic, but I /do/ want git to use a collision-resistant hash function. I would like to be able to fetch from others without giving them a free pass to poison my repository. I believe that was the original intended semantic of the fetch operation; it's only now eroding as SHA-1 gets studied. But the risk isn't great enough to goad me into action yet. -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: recent discussion regarding 'checksums'
On Tue, 28 Sep 2010, Matt McCutchen wrote: On Mon, 2010-09-27 at 22:33 -0400, Benjamin R. Haskell wrote: But the flip side is that rsync is not a security tool. MD5 is fine for rsync for the same reason SHA-1 (which, as with all hashes, will eventually be broken) is fine for git: This gets a little off topic, but I /do/ want git to use a collision-resistant hash function. I would like to be able to fetch from others without giving them a free pass to poison my repository. I believe that was the original intended semantic of the fetch operation; it's only now eroding as SHA-1 gets studied. But the risk isn't great enough to goad me into action yet. Shoot. I left out the link that was the whole point: it's Linus explaining why SHA-1 is fine (because security happens at a higher level): http://kerneltrap.org/mailarchive/git/2006/8/28/211065 (Boils down to the two points I paraphrased -- but the upshot is that hash collision != free pass to poison) -- Best, Ben -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
