Re: rsync files from subfolders on source to root of a folder on destination
I think hard-links are the easiest solution. mkdir hardlinks find Music -type f -print0 | xargs -0 cp -al -t hardlinks (You get a warning from cp about doubles, if any) Then you can rsync the 'hardlinks'-directory as usual. When you want to update you should just 'rm -rf hardlinks' then execute the find again. That way even deletions work as expected (Or just put it in a script). Bis denn Thanks for the suggestions. In the end I used this example and will just write a shell script around it to update the hard links when I add/removed music. Really appreciate it. Regards, James -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync files from subfolders on source to root of a folder on destination
I wish to sync a bunch of flac files that reside in various subfolders
to the root of a folder on a destination.
An example of the directory structure on the source is:
source tree Music/
Music/
├── R
│ ├── Radiohead
│ │ └── OK Computer
│ │ ├── 01 - Radiohead - Airbag.flac
│ │ ├── 02 - Radiohead - Paranoid Android.flac
│ └── Red Hot Chilli Peppers
│ └── Greatest Hits
│ ├── 01 - Red Hot Chili Peppers - Under the Bridge.flac
│ ├── 02 - Red Hot Chili Peppers - Give It Away.flac
I am using this command which syncs fine but includes the directory
paths on the destination:
rsync -rltDzvh --delete -e ssh Music/ user@192.168.1.1:/Destination/
--include=*/ --include=*.flac --exclude=*
So on the destination the structure is:
destination tree /Destination/
/Destination/
├── R
│ ├── Radiohead
│ │ └── OK Computer
│ │ ├── 01 - Radiohead - Airbag.flac
│ │ ├── 02 - Radiohead - Paranoid Android.flac
│ └── Red Hot Chilli Peppers
│ └── Greatest Hits
│ ├── 01 - Red Hot Chili Peppers - Under the Bridge.flac
│ ├── 02 - Red Hot Chili Peppers - Give It Away.flac
I want to prune all directories so only the files are placed into the
root of /Destination/ e.g.
destination tree /Destination/
/Destination/
├── 01 - Radiohead - Airbag.flac
├── 02 - Radiohead - Paranoid Android.flac
├── 01 - Red Hot Chili Peppers - Under the Bridge.flac
├── 02 - Red Hot Chili Peppers - Give It Away.flac
All the files have different names so that's ok and I have reviewed
the various options in rsync such as --no-relative but have been
unable to get it working as desired.
I also came up with:
find ./Music/ -name *.flac -exec rsync -ltDzvh {} -e ssh
user@192.168.1.1:/Destination/ \;
But this means using ssh keys and is probably inefficient and if I
want to use the --delete it would likely break everything.
So how would I achieve this whilst still being able to use --delete or
--delete-excluded for times when I add, remove or rename items on my
Music library on the source and have those changes sync to the
destination?
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --files-from= and --delete corrent options
Try this instead: rsync [OPTIONS] --include-from=/tmp/BACKUPFILE --exclude='*' /var/data/shares/vmbackups/ /mnt/usb_backup/SFSYDVS01/daily.0/VM/ This way, rsync will scan the destination directory for deletions (since you have -a without --files-from, which implies -r) and will delete the old files because they are excluded with --delete-excluded. Matt Thanks Matt, I tried this but it doesn't seem to work for me as you describe. I just created a test setup. The commands I'm running are as follows: [EMAIL PROTECTED]:~# rsync --version rsync version 2.6.9 protocol version 29 [EMAIL PROTECTED]:~# ls /tmp/source/ Thursday2008-09-11.bkf Tuesday2008-09-09.bkf Wednesday2008-09-10.bkf [EMAIL PROTECTED]:~# ls -t1 /tmp/source | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# cat /tmp/BACKUPFILE Thursday2008-09-11.bkf [EMAIL PROTECTED]:~# rsync -ah --delete --numeric-ids --stats --delete-excluded -include-from=/tmp/BACKUPFILE --exclude='*' /tmp/source/ /tmp/destination/ .d..t.. ./ Number of files: 1 Number of files transferred: 0 Total file size: 0 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 20 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 42 Total bytes received: 26 sent 42 bytes received 26 bytes 136.00 bytes/sec total size is 0 speedup is 0.00 [EMAIL PROTECTED]:~# ls /tmp/destination/ [EMAIL PROTECTED]:~# -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --files-from= and --delete corrent options
Matt McCutchen wrote: On Wed, 2008-09-24 at 13:00 +1000, James Robertson wrote: I tried this but it doesn't seem to work for me as you describe. I just created a test setup. The commands I'm running are as follows: [EMAIL PROTECTED]:~# rsync --version rsync version 2.6.9 protocol version 29 [EMAIL PROTECTED]:~# ls /tmp/source/ Thursday2008-09-11.bkf Tuesday2008-09-09.bkf Wednesday2008-09-10.bkf [EMAIL PROTECTED]:~# ls -t1 /tmp/source | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# cat /tmp/BACKUPFILE Thursday2008-09-11.bkf [EMAIL PROTECTED]:~# rsync -ah --delete --numeric-ids --stats --delete-excluded -include-from=/tmp/BACKUPFILE --exclude='*' /tmp/source/ /tmp/destination/ .d..t.. ./ Number of files: 1 Number of files transferred: 0 Total file size: 0 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 20 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 42 Total bytes received: 26 sent 42 bytes received 26 bytes 136.00 bytes/sec total size is 0 speedup is 0.00 [EMAIL PROTECTED]:~# ls /tmp/destination/ [EMAIL PROTECTED]:~# The -include-from=/tmp/BACKUPFILE is short a dash, causing it to be parsed as -i -n -c -l -u -d --rsh=-from=/tmp/BACKUPFILE . Change it to --include-from=/tmp/BACKUPFILE . Matt If /tmp/BACKUPFILE contained a directory e.g. [EMAIL PROTECTED]:~# cat /tmp/BACKUPFILE 3/ How would I get it to copy everything under and including /var/data/shares/vmbackups/3/? and delete everything else on the destination? The guys running the backup to /var/data/shares/vmbackups/ are considering copying the files direct rather than sticking it into a *.bkf and have it create folders like 1/, 2/, 3/ with files and folders under that. I came up with running this: [EMAIL PROTECTED]:~# ls /tmp/source/ 1/ 2/ 3/ [EMAIL PROTECTED]:~# ls -t1 /tmp/source/ | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# cat /tmp/BACKUPFILE 3/ [EMAIL PROTECTED]:~# ls -t1 /tmp/source/ | grep -f /tmp/BACKUPFILE -v /tmp/BACKUPEXCLUDE [EMAIL PROTECTED]:~# cat /tmp/BACKUPEXCLUDE 2/ 1/ [EMAIL PROTECTED]:~# rsync -ah --numeric-ids --stats --delete-excluded --include-from=/tmp/BACKUPFILE --exclude-from=/tmp/BACKUPEXCLUDE /tmp/source/ /tmp/destination/ [EMAIL PROTECTED]:~# ls /tmp/destination/ 3/ Which works and copies the subdirectories and files but is there a more elegant solution without the need for the /tmp/BACKUPEXCLUDE file? I think i'll rename /tmp/BACKUPFILE to /tmp/BACKUPDIR :) Apologies as they only just decided this today after I came back with the solution you helped me with. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
