Re: downloading only specific directories from directory tree
If you want every openSUSE_Leap_42.2 then just add an include of */ and let --prune-empty-dirs handle the rest. If you want it only in those dirs then you would need something like (I didn't test this): + dir:/ + dir:/**/openSUSE_Leap_42.2/*** On 01/11/2018 11:10 AM, Istvan Gabor via rsync wrote: > Thank you for your answer, Kevin. > > 2. OK, I understand. > > 1. I checked colon dirs, and indeed they don't have openSUSE_Leap_42.2 > dirs directly. But they have different subdirs which have > openSUSE_Leap_42.2 dirs. What would be the correct filter set > to mirror all openSUSE_Leap_42.2 dirs at any level if I don't want > to include all dirs at the root level? At the roor level tehre are > many dirs and I can't mirror everything at a time because of space > limitations. I would like to mirror different groups from the > root level separately. Either by using root level pattern names or > by directly specifying the dirs I want to mirror. Each root level dir > may or may not have an openSUSE_Leap_42.2 dir directly or at a > deeper level. I want to mirror all openSUSE_Leap_42.2 dirs that > belong to the specified root level dirs. To make it clear, for example, > I want to mirror games, games:, graphics, graphics:, hamradio, hamradio:, > hardware, hardware:, only the openSUSE_Leap_42.2 directory containing > paths. > What would be the rules for this? > > Thanks again, > > Istvan > > > > On Wed, 10 Jan 2018 18:27:11 -0500, Kevin Korb via rsync wrote: >> 1. You don't appear to be including anything that is within the colon >> dirs so they are being left out by --prune-empty-dirs. >> >> 2. Includes only override excludes that follow them. Since your debug >> excludes are exceptions to the openSUSE_Leap_42.2/*** include they must >> be before it not after it. >> >> On 01/10/2018 05:46 PM, Istvan Gabor via rsync wrote: >>> Dear Kevin: >>> >>> ~ 1 year ago your answer helped me to solve my problem. >>> This time I would like to do a similar thing but little bit modified. >>> I read again carefully INCLUDE/EXCLUDE PATTERN RULES section of rsync >>> manual but still cannot comprehend every part of it. >>> >>> I understand I have to add include patterns first and exclude patterns >>> second. But it is not clear if I have to add all the include patterns >>> first and all the exclude patterns after them. Or do I have add >>> include/exclude groups? >>> >>> Now I would like to mirror openSUSE_Leap_42.2 dirs from >>> ftp://ftp.halifax.rwth-aachen.de/opensuse/repositories/ >>> but only from specified directories and without files with debuginfo >>> or debugsource patterns in their names. >>> >>> I tried this command: >>> >>> rsync -av --include-from=filters.txt --prune-empty-dirs \ >>> rsync://ftp.halifax.rwth-aachen.de/opensuse/repositories/ \ >>> opensuse/repositories >>> >>> where filters.txt contains: >>> >>> + /filesystems >>> + /filesystems: >>> + /games >>> + /games: >>> + /graphics >>> + /graphics: >>> + /hamradio >>> + /hamradio: >>> + /hardware >>> + /hardware: >>> ... >>> + openSUSE_Leap_42.2/*** >>> - *debuginfo*.rpm >>> - *debugsource*.rpm >>> - * >>> >>> but it doesn't work the way I'd like. >>> >>> 1. The directories ending with : in their names are not mirrored. >>> I tried eg /games: , /games\: and "/games:", none of these worked. >>> >>> 2. The other directories are mirrored but debuginfo and debugsource >>> files are not excluded. >>> >>> I'd appreciate if you explained what the problem is and how to solve it. >>> >>> Thanks in advance, >>> >>> Istvan > > -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., signature.asc Description: OpenPGP digital signature -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: downloading only specific directories from directory tree
Thank you for your answer, Kevin. 2. OK, I understand. 1. I checked colon dirs, and indeed they don't have openSUSE_Leap_42.2 dirs directly. But they have different subdirs which have openSUSE_Leap_42.2 dirs. What would be the correct filter set to mirror all openSUSE_Leap_42.2 dirs at any level if I don't want to include all dirs at the root level? At the roor level tehre are many dirs and I can't mirror everything at a time because of space limitations. I would like to mirror different groups from the root level separately. Either by using root level pattern names or by directly specifying the dirs I want to mirror. Each root level dir may or may not have an openSUSE_Leap_42.2 dir directly or at a deeper level. I want to mirror all openSUSE_Leap_42.2 dirs that belong to the specified root level dirs. To make it clear, for example, I want to mirror games, games:, graphics, graphics:, hamradio, hamradio:, hardware, hardware:, only the openSUSE_Leap_42.2 directory containing paths. What would be the rules for this? Thanks again, Istvan On Wed, 10 Jan 2018 18:27:11 -0500, Kevin Korb via rsync wrote: 1. You don't appear to be including anything that is within the colon dirs so they are being left out by --prune-empty-dirs. 2. Includes only override excludes that follow them. Since your debug excludes are exceptions to the openSUSE_Leap_42.2/*** include they must be before it not after it. On 01/10/2018 05:46 PM, Istvan Gabor via rsync wrote: Dear Kevin: ~ 1 year ago your answer helped me to solve my problem. This time I would like to do a similar thing but little bit modified. I read again carefully INCLUDE/EXCLUDE PATTERN RULES section of rsync manual but still cannot comprehend every part of it. I understand I have to add include patterns first and exclude patterns second. But it is not clear if I have to add all the include patterns first and all the exclude patterns after them. Or do I have add include/exclude groups? Now I would like to mirror openSUSE_Leap_42.2 dirs from ftp://ftp.halifax.rwth-aachen.de/opensuse/repositories/ but only from specified directories and without files with debuginfo or debugsource patterns in their names. I tried this command: rsync -av --include-from=filters.txt --prune-empty-dirs \ rsync://ftp.halifax.rwth-aachen.de/opensuse/repositories/ \ opensuse/repositories where filters.txt contains: + /filesystems + /filesystems: + /games + /games: + /graphics + /graphics: + /hamradio + /hamradio: + /hardware + /hardware: ... + openSUSE_Leap_42.2/*** - *debuginfo*.rpm - *debugsource*.rpm - * but it doesn't work the way I'd like. 1. The directories ending with : in their names are not mirrored. I tried eg /games: , /games\: and "/games:", none of these worked. 2. The other directories are mirrored but debuginfo and debugsource files are not excluded. I'd appreciate if you explained what the problem is and how to solve it. Thanks in advance, Istvan -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: downloading only specific directories from directory tree
1. You don't appear to be including anything that is within the colon dirs so they are being left out by --prune-empty-dirs. 2. Includes only override excludes that follow them. Since your debug excludes are exceptions to the openSUSE_Leap_42.2/*** include they must be before it not after it. On 01/10/2018 05:46 PM, Istvan Gabor via rsync wrote: > Dear Kevin: > > ~ 1 year ago your answer helped me to solve my problem. > This time I would like to do a similar thing but little bit modified. > I read again carefully INCLUDE/EXCLUDE PATTERN RULES section of rsync > manual but still cannot comprehend every part of it. > > I understand I have to add include patterns first and exclude patterns > second. But it is not clear if I have to add all the include patterns > first and all the exclude patterns after them. Or do I have add > include/exclude groups? > > Now I would like to mirror openSUSE_Leap_42.2 dirs from > ftp://ftp.halifax.rwth-aachen.de/opensuse/repositories/ > but only from specified directories and without files with debuginfo > or debugsource patterns in their names. > > I tried this command: > > rsync -av --include-from=filters.txt --prune-empty-dirs \ > rsync://ftp.halifax.rwth-aachen.de/opensuse/repositories/ \ > opensuse/repositories > > where filters.txt contains: > > + /filesystems > + /filesystems: > + /games > + /games: > + /graphics > + /graphics: > + /hamradio > + /hamradio: > + /hardware > + /hardware: > ... > + openSUSE_Leap_42.2/*** > - *debuginfo*.rpm > - *debugsource*.rpm > - * > > but it doesn't work the way I'd like. > > 1. The directories ending with : in their names are not mirrored. > I tried eg /games: , /games\: and "/games:", none of these worked. > > 2. The other directories are mirrored but debuginfo and debugsource > files are not excluded. > > I'd appreciate if you explained what the problem is and how to solve it. > > Thanks in advance, > > Istvan > > > > > > On Fri, 20 Jan 2017 18:15:00 -0500, Kevin Korb wrote: >> An include only overrides an exclude that follows it. So, you would >> need something like this: >> >> + */ >> + openSUSE_13.2/*** >> - * >> >> Then add --prune-empty-dirs to your rsync command line. I left off the >> initial ** because it doesn't really mean anything unless openSUSE_13.2 >> isn't always the entire directory name in which case 1 * would do. >> >> Another way to explain it is that by default everything is included. An >> exclude rule makes exceptions. An include rule makes exceptions to the >> exclude rule. >> >> On 01/20/2017 06:08 PM, Istvan Gabor wrote: >>> Hello: >>> >>> I have read rsync manual and several howtos on how to use rsync, still >>> I don't know if it's doable what I want to do, and if yes, how. >>> >>> The scenario: >>> >>> I would like to make a local copy of openSUSE 13.2 repositories. >>> I use openSUSE linux. >>> >>> The repos are located in a multi-level directory structure, eg: >>> >>> ftp://ftp.halifax.rwth-aachen.de/opensuse/repositories/ >>> >>> Let's call this directory the root dir. >>> >>> Each directory in the root has subdirectories, and one of the >>> subdirectories >>> is named "openSUSE_13.2". The openSUSE_13.2 dir can be one, two, or more >>> level >>> downstairs in the root dir, eg: >>> >>> http://download.opensuse.org/repositories/Apache/openSUSE_13.2/ >>> >>> http://download.opensuse.org/repositories/Apache:/MirrorBrain/openSUSE_13.2/ >>> >>> >>> >>> There are plenty of directories in the root dir, and each >>> subdirectory can >>> have several subdirectories. >>> >>> Is it possible, using rsync, to download only all "openSUSE_13.2" >>> directories with >>> their contents, without specifying directly one by one every >>> subdirectory >>> to be included or excluded? >>> >>> I tried to use filters.txt file with content: >>> >>> + **openSUSE_13.2/*** >>> >>> with this command: >>> >>> rsync -av --include-from=filters.txt >>> rsync://ftp.halifax.rwth-aachen.de/opensuse/repositories/ ./ >>> >>> This downloads every directory. How can I exclude those are not >>> openSUSE_13.2? >>> >>> Thanks, >>> >>> Istvan >>> > > -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., signature.asc Description: OpenPGP digital signature -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: downloading only specific directories from directory tree
Dear Kevin: ~ 1 year ago your answer helped me to solve my problem. This time I would like to do a similar thing but little bit modified. I read again carefully INCLUDE/EXCLUDE PATTERN RULES section of rsync manual but still cannot comprehend every part of it. I understand I have to add include patterns first and exclude patterns second. But it is not clear if I have to add all the include patterns first and all the exclude patterns after them. Or do I have add include/exclude groups? Now I would like to mirror openSUSE_Leap_42.2 dirs from ftp://ftp.halifax.rwth-aachen.de/opensuse/repositories/ but only from specified directories and without files with debuginfo or debugsource patterns in their names. I tried this command: rsync -av --include-from=filters.txt --prune-empty-dirs \ rsync://ftp.halifax.rwth-aachen.de/opensuse/repositories/ \ opensuse/repositories where filters.txt contains: + /filesystems + /filesystems: + /games + /games: + /graphics + /graphics: + /hamradio + /hamradio: + /hardware + /hardware: ... + openSUSE_Leap_42.2/*** - *debuginfo*.rpm - *debugsource*.rpm - * but it doesn't work the way I'd like. 1. The directories ending with : in their names are not mirrored. I tried eg /games: , /games\: and "/games:", none of these worked. 2. The other directories are mirrored but debuginfo and debugsource files are not excluded. I'd appreciate if you explained what the problem is and how to solve it. Thanks in advance, Istvan On Fri, 20 Jan 2017 18:15:00 -0500, Kevin Korb wrote: An include only overrides an exclude that follows it. So, you would need something like this: + */ + openSUSE_13.2/*** - * Then add --prune-empty-dirs to your rsync command line. I left off the initial ** because it doesn't really mean anything unless openSUSE_13.2 isn't always the entire directory name in which case 1 * would do. Another way to explain it is that by default everything is included. An exclude rule makes exceptions. An include rule makes exceptions to the exclude rule. On 01/20/2017 06:08 PM, Istvan Gabor wrote: Hello: I have read rsync manual and several howtos on how to use rsync, still I don't know if it's doable what I want to do, and if yes, how. The scenario: I would like to make a local copy of openSUSE 13.2 repositories. I use openSUSE linux. The repos are located in a multi-level directory structure, eg: ftp://ftp.halifax.rwth-aachen.de/opensuse/repositories/ Let's call this directory the root dir. Each directory in the root has subdirectories, and one of the subdirectories is named "openSUSE_13.2". The openSUSE_13.2 dir can be one, two, or more level downstairs in the root dir, eg: http://download.opensuse.org/repositories/Apache/openSUSE_13.2/ http://download.opensuse.org/repositories/Apache:/MirrorBrain/openSUSE_13.2/ There are plenty of directories in the root dir, and each subdirectory can have several subdirectories. Is it possible, using rsync, to download only all "openSUSE_13.2" directories with their contents, without specifying directly one by one every subdirectory to be included or excluded? I tried to use filters.txt file with content: + **openSUSE_13.2/*** with this command: rsync -av --include-from=filters.txt rsync://ftp.halifax.rwth-aachen.de/opensuse/repositories/ ./ This downloads every directory. How can I exclude those are not openSUSE_13.2? Thanks, Istvan -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: downloading only specific directories from directory tree
The following URL may also be helpful in terms of another example : http://www.lbackup.org/developer/advanced_excludes_file_examples Hope that link is helpful. HTRAX 2013 Revitalised : EGYPTIAN HUMP HTRAX : Direct URL download : http://henri.shustak.org/download/htrax/egyptian-hump-htrax.mp3 "Dr Who Meets B52's" - More reviews : http://www.jessetaylor.com.au/htrax On 21/01/2017, at 12:15 PM, Kevin Korb wrote: > An include only overrides an exclude that follows it. So, you would > need something like this: > > + */ > + openSUSE_13.2/*** > - * > > Then add --prune-empty-dirs to your rsync command line. I left off the > initial ** because it doesn't really mean anything unless openSUSE_13.2 > isn't always the entire directory name in which case 1 * would do. > > Another way to explain it is that by default everything is included. An > exclude rule makes exceptions. An include rule makes exceptions to the > exclude rule. > > On 01/20/2017 06:08 PM, Istvan Gabor wrote: >> Hello: >> >> I have read rsync manual and several howtos on how to use rsync, still >> I don't know if it's doable what I want to do, and if yes, how. >> >> The scenario: >> >> I would like to make a local copy of openSUSE 13.2 repositories. >> I use openSUSE linux. >> >> The repos are located in a multi-level directory structure, eg: >> >> ftp://ftp.halifax.rwth-aachen.de/opensuse/repositories/ >> >> Let's call this directory the root dir. >> >> Each directory in the root has subdirectories, and one of the >> subdirectories >> is named "openSUSE_13.2". The openSUSE_13.2 dir can be one, two, or more >> level >> downstairs in the root dir, eg: >> >> http://download.opensuse.org/repositories/Apache/openSUSE_13.2/ >> http://download.opensuse.org/repositories/Apache:/MirrorBrain/openSUSE_13.2/ >> >> >> There are plenty of directories in the root dir, and each subdirectory can >> have several subdirectories. >> >> Is it possible, using rsync, to download only all "openSUSE_13.2" >> directories with >> their contents, without specifying directly one by one every subdirectory >> to be included or excluded? >> >> I tried to use filters.txt file with content: >> >> + **openSUSE_13.2/*** >> >> with this command: >> >> rsync -av --include-from=filters.txt >> rsync://ftp.halifax.rwth-aachen.de/opensuse/repositories/ ./ >> >> This downloads every directory. How can I exclude those are not >> openSUSE_13.2? >> >> Thanks, >> >> Istvan >> > > -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > Kevin Korb Phone:(407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. ke...@futurequest.net (work) > Orlando, Floridak...@sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: downloading only specific directories from directory tree
On Fri, 20 Jan 2017 18:15:00 -0500, Kevin Korb wrote: An include only overrides an exclude that follows it. So, you would need something like this: + */ + openSUSE_13.2/*** - * Then add --prune-empty-dirs to your rsync command line. I left off the initial ** because it doesn't really mean anything unless openSUSE_13.2 isn't always the entire directory name in which case 1 * would do. Another way to explain it is that by default everything is included. An exclude rule makes exceptions. An include rule makes exceptions to the exclude rule. Thank you a lot. This worked the way I wanted. Istvan -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: downloading only specific directories from directory tree
An include only overrides an exclude that follows it. So, you would need something like this: + */ + openSUSE_13.2/*** - * Then add --prune-empty-dirs to your rsync command line. I left off the initial ** because it doesn't really mean anything unless openSUSE_13.2 isn't always the entire directory name in which case 1 * would do. Another way to explain it is that by default everything is included. An exclude rule makes exceptions. An include rule makes exceptions to the exclude rule. On 01/20/2017 06:08 PM, Istvan Gabor wrote: > Hello: > > I have read rsync manual and several howtos on how to use rsync, still > I don't know if it's doable what I want to do, and if yes, how. > > The scenario: > > I would like to make a local copy of openSUSE 13.2 repositories. > I use openSUSE linux. > > The repos are located in a multi-level directory structure, eg: > > ftp://ftp.halifax.rwth-aachen.de/opensuse/repositories/ > > Let's call this directory the root dir. > > Each directory in the root has subdirectories, and one of the > subdirectories > is named "openSUSE_13.2". The openSUSE_13.2 dir can be one, two, or more > level > downstairs in the root dir, eg: > > http://download.opensuse.org/repositories/Apache/openSUSE_13.2/ > http://download.opensuse.org/repositories/Apache:/MirrorBrain/openSUSE_13.2/ > > > There are plenty of directories in the root dir, and each subdirectory can > have several subdirectories. > > Is it possible, using rsync, to download only all "openSUSE_13.2" > directories with > their contents, without specifying directly one by one every subdirectory > to be included or excluded? > > I tried to use filters.txt file with content: > > + **openSUSE_13.2/*** > > with this command: > > rsync -av --include-from=filters.txt > rsync://ftp.halifax.rwth-aachen.de/opensuse/repositories/ ./ > > This downloads every directory. How can I exclude those are not > openSUSE_13.2? > > Thanks, > > Istvan > -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., signature.asc Description: OpenPGP digital signature -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
downloading only specific directories from directory tree
Hello: I have read rsync manual and several howtos on how to use rsync, still I don't know if it's doable what I want to do, and if yes, how. The scenario: I would like to make a local copy of openSUSE 13.2 repositories. I use openSUSE linux. The repos are located in a multi-level directory structure, eg: ftp://ftp.halifax.rwth-aachen.de/opensuse/repositories/ Let's call this directory the root dir. Each directory in the root has subdirectories, and one of the subdirectories is named "openSUSE_13.2". The openSUSE_13.2 dir can be one, two, or more level downstairs in the root dir, eg: http://download.opensuse.org/repositories/Apache/openSUSE_13.2/ http://download.opensuse.org/repositories/Apache:/MirrorBrain/openSUSE_13.2/ There are plenty of directories in the root dir, and each subdirectory can have several subdirectories. Is it possible, using rsync, to download only all "openSUSE_13.2" directories with their contents, without specifying directly one by one every subdirectory to be included or excluded? I tried to use filters.txt file with content: + **openSUSE_13.2/*** with this command: rsync -av --include-from=filters.txt rsync://ftp.halifax.rwth-aachen.de/opensuse/repositories/ ./ This downloads every directory. How can I exclude those are not openSUSE_13.2? Thanks, Istvan -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html