Re: How to mirror only specified directories
I'm still struggling to get just the directory(ies) that I want... You may find this http://tinyurl.com/rsync-exclude-all-include-some post to the LBackup mailing list helpful. The example listed (link above) revolves around specifying the root directory as the source and then specifying a selection of sub-directories to include in the sync and then finally a rule which will exclude everything else. Please note that each directory within the path must specifically be included for this approach to work. IN addition, this approach utilizes only an excludes file (no include list via the command line). Following up with the approach listed above, the excludes file quoted below should provide the results you are after if modified. Some minor modifications should be all that is required. The example below (extract from URL above) is going to copy the directories /private and /home. It will transfer everything recursively within /private/etc/, /private/var/ and /home/ + private + private/etc/*** + private/var/*** + home/*** - * I understand you wanted to only transfer the /das/htdocs/docs/ directory and any contents within this directory recursively with rsync. Provided you are syncing the root directory then the following excludes file example listed below should make this happen : + das + das/htdocs + das/htdocs/docs/*** - * If you have other directories other than /das/htdocs/docs/ to transfer just add them in as required. I think that reiterating the following is worth while : read the man page. ** and *** can be very useful. Quoted from the rsync man page : a trailing dir_name/*** will match both the directory (as if dir_name/ had been specified) and everything in the directory (as if dir_name/** had been specified). This behavior was added in version 2.6.7 I hope this helps. - This email is protected by LBackup, an open source backup solution. Free as in freedom; LBackup is licensed under the GNU http://www.lbackup.org -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync on AS400-PASE and AIX rsync binary installation
We are currently replicating several of our servers to a remote colocation using rsync 3.0.6 on Mac OS X 10.4, 10.5 and 10.6. We also have an AS400 system and would like to replicate some of its files, contained in an IFS file system, to the Mac server running rsync at the colocation facility. The AS400 system is in the hands of specialists within our organization who are well versed in configuring and maintaining the system. My online searches have turned up references to a PASE add-on that allows AIX binaries to be installed and run on an AS400 system. One reference I have found says that rsync needs to be run in daemon mode on the AS400 end but does not provide any details. This reference is from 2002 and I have not found anything more recent. Can anyone point me to more complete information on installing and running rsync on AS400, including the use of PASE and the AIX rsync binary? --Carney Mimms -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Excluding most and including some problems continue.
On Thu, Sep 30, 2010 at 8:27 AM, Ian Skinner iskin...@cdpr.ca.gov wrote: Unfortunately there are some subdirectories in some of these selected */htdocs/docs* directories that are unintentionally being excluded by these rules. I.E. */export/home/enf/htdocs/docs/county/internal/*. It is usually best to anchor your matching terms, unless you want a term to float and match anywhere. In an .rsync-filter file, terms that start with a slash are anchored in that file's directory. You can also use a wildcard for all the subdir excusions. For example: + /das + /em + /enf + /internal + /itb + /medtox + /pml + /psb + /reg + /whs - /* + /*/htdocs - /*/* + /*/htdocs/docs - /*/htdocs/* Another alternative is to sprinkle .rsync-filter files throughout your hierarchy with localized rules for that part of the hierarchy, but the above should do what you want. ..wayne.. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 3653] Reduce the need for the vanished files warning
https://bugzilla.samba.org/show_bug.cgi?id=3653 j...@jwz.org changed: What|Removed |Added CC||j...@jwz.org --- Comment #17 from j...@jwz.org 2010-10-01 11:27 CST --- Me too. Doing nightly backups from cron is an *extremely* common use-case of rsync, and telling each and every user of rsync that they should either A) write their own shell script to filter and discard error messages, or B) play a constant game of whack-a-mole chasing down new --exclude options to add, is just silly. It pushes work downstream that could be done upstream much more efficiently. Why make each user write the same code over and over when the developer could write it just once, and get it right? Option A is bad because everyone gets to introduce differently-subtle bugs in their script (wrong error code? accidentally ignoring too many error messages? who knows. It's fragile.) Option B is bad because I don't *want* to have a bunch of --exclude options for my backup: disk and bandwidth are cheap, so I want my backup disk to be an exact mirror. That way, when the disk holding / dies, I can just drop the backup in and boot it, being confident that everything is exactly the same. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Excluding most and including some problems continue.
On Fri, 1 Oct 2010, Wayne Davison wrote:
On Thu, Sep 30, 2010 at 8:27 AM, Ian Skinner wrote:
Unfortunately there are some subdirectories in some of these selected
*/htdocs/docs* directories that are unintentionally being excluded by
these rules. I.E. */export/home/enf/htdocs/docs/county/internal/*.
It is usually best to anchor your matching terms, unless you want a
term to float and match anywhere. In an .rsync-filter file, terms
that start with a slash are anchored in that file's directory. You
can also use a wildcard for all the subdir excusions. For example:
+ /das
+ /em
+ /enf
+ /internal
+ /itb
+ /medtox
+ /pml
+ /psb
+ /reg
+ /whs
- /*
+ /*/htdocs
- /*/*
+ /*/htdocs/docs
- /*/htdocs/*
Another alternative is to sprinkle .rsync-filter files throughout your
hierarchy with localized rules for that part of the hierarchy, but the
above should do what you want.
I think I sent a variant of the attached Perl script last time someone
was asking something similar. What Wayne suggested is better right now
(that is: while your patterns are very simple -- just a few root-level
directories, each of which should include the /htdocs/docs/ subdir).
But if you start adding more, it could get more annoying to have to
manually fiddle with the rules.
The attached script takes as input lines of the form:
/rooted/path/to/include
and produces what should work as a filter file. It supports
'{one,other}' brace-style expansions (but I think that may be
OS-dependent -- I think it works if your system's 'glob()' function
supports them). So, for example, I produced a working filter file for
your situation from:
{== input.rsync.rules ==}
/{das,em,enf,internal,itb,medtox,pml,psb,reg,whs}/htdocs/docs
{=}
$ perl ./rsync-filter-generate.pl input.rsync.rules
[produces rule file] [1]
$ perl ./rsync-filter-generate.pl input.rsync.rules | rsync --include-from=-
/path/to/root/
[shows what would be transferred]
$ perl ./rsync-filter-generate.pl input.rsync.rules | rsync --include-from=-
/path/to/root/ /path/to/dest/
[does it]
The general strategy:
$ echo /abc/def/ghi | perl ./rsync-filter-generate.pl
+ /abc -- first include each path component
+ /abc/def -- one-at-a-time, for each thing to include
+ /abc/def/ghi
- /abc/def/* -- then exclude everything else at each
- /abc/* -- level of the hierarchy
- /*
--
Best,
Ben
[1] output for your case:
+ /das
+ /das/htdocs
+ /das/htdocs/docs
+ /das/htdocs/other
+ /em
+ /em/htdocs
+ /em/htdocs/docs
+ /enf
+ /enf/htdocs
+ /enf/htdocs/docs
+ /internal
+ /internal/htdocs
+ /internal/htdocs/docs
+ /itb
+ /itb/htdocs
+ /itb/htdocs/docs
+ /medtox
+ /medtox/htdocs
+ /medtox/htdocs/docs
+ /pml
+ /pml/htdocs
+ /pml/htdocs/docs
+ /psb
+ /psb/htdocs
+ /psb/htdocs/docs
+ /reg
+ /reg/htdocs
+ /reg/htdocs/docs
+ /whs
+ /whs/htdocs
+ /whs/htdocs/docs
- /whs/htdocs/*
- /whs/*
- /reg/htdocs/*
- /reg/*
- /psb/htdocs/*
- /psb/*
- /pml/htdocs/*
- /pml/*
- /medtox/htdocs/*
- /medtox/*
- /itb/htdocs/*
- /itb/*
- /internal/htdocs/*
- /internal/*
- /enf/htdocs/*
- /enf/*
- /em/htdocs/*
- /em/*
- /das/htdocs/*
- /das/*
- /*#!/usr/bin/perl
use strict;
use warnings;
# read in all of the paths to include
my @all;
while () {
chomp;
push @all, glob;
}
my (%inc, %exc);
for (@all) {
my @parts = split m{/};
for (1..$#parts) {
# include every path component up to the end
# e.g.
# /abc
# /abc/def
# /abc/def/ghi
$inc{join /, @parts[0..$_]}++;
# exclude every other path component
# e.g.
# /abc/def/*
# /abc/*
# /*
$exc{join /, @parts[0..$_-1]}++;
}
}
# include things from shortest-to-longest path
# exclude things from longest to shortest
print + $_\n for sort keys %inc;
print - $_/*\n for reverse sort keys %exc;
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Excluding most and including some problems continue.
On Fri, 2010-10-01 at 16:28 -0400, Benjamin R. Haskell wrote: I think I sent a variant of the attached Perl script last time someone was asking something similar. What Wayne suggested is better right now (that is: while your patterns are very simple -- just a few root-level directories, each of which should include the /htdocs/docs/ subdir). But if you start adding more, it could get more annoying to have to manually fiddle with the rules. The attached script takes as input lines of the form: /rooted/path/to/include and produces what should work as a filter file. A similar script is distributed with rsync: support/files-to-excludes . -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Excluding most and including some problems continue.
On Friday, October 01, 2010 at 7:54 AM, in message + /das + /em + /enf + /internal + /itb + /medtox + /pml + /psb + /reg + /whs - /* + /*/htdocs - /*/* + /*/htdocs/docs - /*/htdocs/* Thanks Wayne, this worked well and seems simpler and lazier then my original version. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
