Re: Memory consumption for rsync -axv --delete
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 If you were using --link-dest to make multiple backups you wouldn't need --delete because the target is always a new empty directory (with - --link-dest pointing to the previous backup run). So, you get the benefit of having multiple backups to restore from and rsync doesn't have to --delete. When you run low on space you just rm - -rf some old backups (takes a while but doesn't need much RAM). On 03/25/2016 05:54 AM, John Long wrote: > Hi, > > I have been using rsync for many years and never had any kind of > problem. Lately I am running out of RAM trying to do an incremental > backup to a box that only has 2G of RAM. The entire directory > structure I'm mirroring is about 200G of files. A minority of > subdirectories have many files. > > Is there a way to do an incremental backup with --delete option > that does not use as much memory? Is there a way to tell rsync to > use a tempfile instead of RAM for keeping tracking of whatever it > does? > > And would it be useful to add ignores for the subdirectories I know > have many files and back them up separately? Is --delete safe to > use in this case, as in does --delete with --ignore somedir/ not > delete files in other target dirs that are not in the ignore path? > > Thanks, > > /jl > - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -BEGIN PGP SIGNATURE- Version: GnuPG v2 iEYEARECAAYFAlb1Vl8ACgkQVKC1jlbQAQccMQCfR+5LfpqH9to3D1QDBDScZOBX RIQAn1BB2vFbf7eDgyy7HSS2SITBTWCM =BYru -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Memory consumption for rsync -axv --delete
On Fri, Mar 25, 2016 at 09:54:14AM +, John Long wrote: > Hi, > > I have been using rsync for many years and never had any kind of problem. > Lately I am running out of RAM trying to do an incremental backup to a box > that only has 2G of RAM. The entire directory structure I'm mirroring is > about 200G of files. A minority of subdirectories have many files. > > Is there a way to do an incremental backup with --delete option that does > not use as much memory? Is there a way to tell rsync to use a tempfile > instead of RAM for keeping tracking of whatever it does? > > And would it be useful to add ignores for the subdirectories I know have > many files and back them up separately? Is --delete safe to use in this > case, as in does --delete with --ignore somedir/ not delete files in other > target dirs that are not in the ignore path? I didn't phrase this part very well. Is --delete safe to use with --ignore, meaning will rsync avoid deleting files in the ignore path on the target side? I think the answer is probably yes but since I'm crashing the target box with --delete I don't want to have to try this too many times. Really I'm looking for a workaround to the high memory consumption so I can sync up the file trees without exceeding the small RAM capacity of the target box. Any suggestions appreciated. Thanks, /jl -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Memory consumption for rsync -axv --delete
On Fri 25 Mar 2016, John Long wrote: > > I have been using rsync for many years and never had any kind of problem. > Lately I am running out of RAM trying to do an incremental backup to a box > that only has 2G of RAM. The entire directory structure I'm mirroring is > about 200G of files. A minority of subdirectories have many files. > > Is there a way to do an incremental backup with --delete option that does > not use as much memory? Is there a way to tell rsync to use a tempfile > instead of RAM for keeping tracking of whatever it does? No to the last question; you could consider adding (more) swapspace to the system, which is effectively like using a tempfile. > And would it be useful to add ignores for the subdirectories I know have > many files and back them up separately? Is --delete safe to use in this > case, as in does --delete with --ignore somedir/ not delete files in other > target dirs that are not in the ignore path? There's no --ignore, you probably mean --exclude. I don't really understand what you're asking in your last question... Why should --exclude somedir/ affect what --delete does elsewhere? --delete will still delete stuff elsewhere if necessary. Also look at the description of --delete and --delete-excluded, if you have any questions about what's in the manpage then feel free to ask those here; but for now I get the impression you haven't spent much time reading the manpage. Paul -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Memory consumption for rsync -axv --delete
Hi, I have been using rsync for many years and never had any kind of problem. Lately I am running out of RAM trying to do an incremental backup to a box that only has 2G of RAM. The entire directory structure I'm mirroring is about 200G of files. A minority of subdirectories have many files. Is there a way to do an incremental backup with --delete option that does not use as much memory? Is there a way to tell rsync to use a tempfile instead of RAM for keeping tracking of whatever it does? And would it be useful to add ignores for the subdirectories I know have many files and back them up separately? Is --delete safe to use in this case, as in does --delete with --ignore somedir/ not delete files in other target dirs that are not in the ignore path? Thanks, /jl -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary/ \http://www.mutt.org attachments / \ Code Blue or Go Home! Encrypted email preferred PGP Key 2048R/DA65BC04 -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
