Re: [rdiff-backup-users] Is rdiff-backup outdated?
On Mon, May 13, 2013 at 4:41 PM, Ans Alghamdi ansd...@gmail.com wrote: Hi, I just find it a bit odd that this powerful tool does not have any bug fix (if there are any) since 2009! So is it outdated, or its so powerful that there is no need for any update? I've already posted this quastion on serverfault at: http://serverfault.com/questions/507259/is-rdiff-backup-outdated? (P.S. The reason behind this is that I'm looking for a backup tool. rdiff-backup really suits my needs but the lack of bug fixes/active development is what is keeping me away) You have good timing. rdiff-backup has been without a maintainer for a while. That changed a couple days ago. http://lists.nongnu.org/archive/html/rdiff-backup-users/2013-05/msg8.html fyi: I've been using it since 10 years or so ago. It's a great tool and I haven't really noticed a couple year gap without a maintainer. The first task Edward (the new maintainer) seems to be chasing is cleaning up the website. Greg ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Help Needed to setup 'rdiff-backup' on SuSE !
On Mon, Mar 21, 2011 at 7:33 AM, neomatrix rdiff-backup-fo...@backupcentral.com wrote: Hi All, I need a help to setup 'rdiff-backup' on SLES server, and i require below. (I) Backup will be compressed in the backup server rdiff-backup keeps a normal uncompressed copy of every file. Compression is only used for maintaining the history of the file changes. So you need to put your backup destination on a compressing filesystem. I've never tried that with linux and have no recommendations. (II) Backup must retain the the attributes as original. Metadata is maintained by default. (III) Backups will always create a new file rather than overwriting it rdiff-backup sends a rsync like set of deltas from the source computer to the backup computer. The previous backup is then used to create a new backup by applying those deltas. And a reverse delta is created and maintained that allows the current backup to be reverted to historical versions. I'm not sure if that satisfies your desire or not. I don't think you have any control of the process. I keep a local rdiff-backup of my files and a remote copy. I rsync the local copy to the remote. rsync has lots of options for how that is done. If you want details, I can give you my command line for that. (IV) incremental backup daily. You get a increment every time you call rdiff-backup. So use cron to invoke it daily. If there are no changes between calls, then your backup won't be modified. note: I have a nightly backup script which invokes rdiff-backup. It does several other things, so it is my backup script I call from cron, not rdiff-backup directly. note2: There are some scripts running around that read rdiff-backup logs and statistics and email you a nightly summary. I do that for each of my machines I backup. If you have some doc or a pointer it would be great. Please help me out. Regards, Neo Greg ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] rdiff-backup streaming?
Are you sure the rdiff delta's are encrypted? I know the primary backup is not. We address this by placing our rdiff backups on encrypted filessystems (using encfs). So our process is: 1) Create a LVM snapshot 2) run rdiff-backup sending data to a local encrypted filesystem. 3) use rsync to send the encrypted backup to a remote machine on the cloud. If you don't feel the need for a local and remote copy, you can likely combine 2 and 3 and go directly to the remote site, but I have not looked into that. Greg On Thu, Feb 17, 2011 at 2:01 PM, Dominic Raferd domi...@timedicer.co.uk wrote: I used rdiff.exe briefly before moving to rdiff-backup. Certainly rdiff-backup does your 2-4 'under the hood' and fast. It is highly optimised both for speed of transfer and for storage space - but does require a reliable connection between source and destination. It does not do your 1, you can achieve this best by backing up from an LVM snapshot (if your source machine is Linux) or VSS (if it is Windows). I am pretty sure that rdiff-backup does not use rdiff, instead it uses a python module built from the librsync library (which is also called, I think, by rdiff). Dominic http://www.timedicer.co.uk/ On 17/02/2011 17:41, Mark Price wrote: Question - We use rdiff (not rdiff-backup) to do our incremental file backups. We do: 1. Copy the file to a staging area (so the file won't disappear or be modified while we work on it) 2. Hash the original file, and computes an rdiff signature (used for delta differencing) 3. Comput an rdiff delta difference (if we have no prior version, this step is skipped) 4. Compress encrypt the resulting delta difference Our problem is all of these things happen in separate phases, distinctly one from the other. This means it takes a long time to do its job. What I am wondering is if rdiff-backup does all of these things in one read/write file pass in a streaming manner, or if it simply calls to the standard rdiff.exe (which isn't working for us)? I didn't quite see information concerning this in the docs or wiki. We have considered using xdelta (because it operates in a streaming manner) but the problem with xdelta is that it stores double copies of the deltas and kills storage space. Any help on this matter would be great! ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki -- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer CNN/TruTV Aired Forensic Imaging Demo - http://insession.blogs.cnn.com/2010/03/23/how-computer-evidence-gets-retrieved/ The Norcross Group The Intersection of Evidence Technology http://www.norcrossgroup.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
[rdiff-backup-users] Getting a comprehensive file listing of a directory
All, I've got a directory that seems to be missing some files. I don't recall their names or exactly when they existed. rdiff-backup -l --list-increment-sizes path-to-backup-dir shows me that I have about 10 instances of that directory backup up. Is there some way to get a list of every file that I've ever backed up for that directory tree? Even if I see the same file 10 times, I'm happy. I just want to see all the files. Thanks Greg ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [Fwd: Re: [rdiff-backup-users] Restarting development]
On Wed, Apr 7, 2010 at 11:27 AM, Nicolas Jungers nico...@jungers.net wrote: On 04/06/2010 10:32 PM, Alexander Samad wrote: But I would say on encryption and de duplication - why not leave that to the filesystem - stay focused on what rdiff-backup does best - differential backups, you can get de duplication, compression and encryption file systems why not leave it to them to do that (well atleast for linux and any os that accepts fuse filesystem). I don't know for de-duplication, but for encryption the filesystem solution falls a bit short. Block device encryption doesn't allow to rsync the backup of site and cryptfs doesn't support spare files (nor do rdiff-backup, but that shall be addressed soon, right?). My memory is a bit fuzzy but I think I selected cryptfs because it's the only solutions which (1) allows access to either the crypt or uncrypt version of the files and (2) may leave the metadata uncrypted. So yes, it's usable, just not optimal. Nicolas, My rdiff-backup main repo is in a encfs filesystem. Encfs may not be a fully secure as other choices (I don't know), but it does allow rsync of the encrypted data to a remote site. (I do that). I have done test restores to a third machine and unencrypted my data via encfs on that machine. So if encfs is secure enough for you, it works fine with rdiff-backup. fyi: what is a spare file? Greg ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] native VSS (Shadow Copy) support
On Tue, Apr 6, 2010 at 11:27 AM, Randy Syring rsyr...@inteli-com.com wrote: Josh Nisly wrote: Two things: 1) The rdiff-backup project likely won't accept patches for features that are platform specific. There are exceptions for OS-specific filesystem metadata, but VSS doesn't fall into that category. Now that there is consideration of restarting development, any chance we could reconsider this? Any backup solution has to wrestle with the Windows user base and the key issue on Windows is whether or not a backup solution uses VSS. If rdiff-backup supported VSS, its uptake might be huge. At very best, maybe a plugin system so that VSS can be added easily without needing to touch the core code? I don't post here often, but I strongly agree that snapshot technology should be leveraged by the rdiff-backup environment. I'm not as positive that it needs to be in the tool itself. I can't remember the name of the package, but I believe there is a linux package that wraps rdiff-backup and uses LVM snapshots from which it runs rdiff-backup. As to VSS, it is supported in Windows XP SP3, 2003, Vista, 2008, etc. So it is a long term technology that it here to stay. To me the only thing to really discuss is how vss support should be integrated into a windows rdiff-backup environment, not if it should be. 2) I've written a python extension module in C++ to implement VSS. Is that something that you'd be interested in? Very, please share. Have you integrated this with rdiff-backup at all? The great thing about C code, etc. (as opposed to just scripts calling the Vista provided CLI commands) is that VSS CLI commands are not included in XP SP3 iirc. And also with XP SP3 you only have 20 or 30 seconds to mount the VSS snapshot after you initialize it. (again, iirc) So having it in a real program as opposed to just a script makes it easier to have the error / retry logic built in. === And for those that don't know much about VSS snapshots, one really cool feature is that services like Exchange, MS-SQL, etc. have vss knowledge integrated into them. They register themselves with the vss service and whenever a snapshot is created, the registered services are notified to quiesce themselves prior to the snapshot being made. I don't know if people are backing up that class of service via rdiff-backup, but if they are it is pretty critical that vss snapshots be used in order to ensure you have quiesced data. vss also solves the open file issue that is so problematic with windows backups. Thanks for reading Greg ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Are my backups too corrupted to recover?
Andrew, any info for me. Even, I'm too busy to look at it for a few weeks would be useful. I need to get this fixed, or give up and restart my backups from scratch. Thanks Greg On Tue, Apr 7, 2009 at 6:36 PM, Greg Freemyer greg.freem...@gmail.com wrote: Andrew, Anything else I can send you on this. Do you think your going to get a fix in the next week or so? If not, I may just wipe out my backup repository and restart. Been going without backups for a few weeks now. (Mostly because I was too busy to look into why they were failing.) Greg On Fri, Apr 3, 2009 at 4:59 PM, Greg Freemyer greg.freem...@gmail.com wrote: All, I sent the listings directly to Andrew and Josh. The rdiff-backup -v8 output is inline here: == Using rdiff-backup version 1.2.6 Unable to import module xattr. Extended attributes not supported on filesystem at /data Unable to import module posix1e from pylibacl package. POSIX ACLs not supported on filesystem at /data Unable to import win32security module. Windows ACLs not supported by filesystem at /data escape_dos_devices not required by filesystem at /data - Detected abilities for source (read only) file system: Access control lists Off Extended attributes Off Windows access control lists Off Case sensitivity On Escape DOS devices Off Escape trailing spaces Off Mac OS X style resource forks Off Mac OS X Finder information Off - Making directory /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0 Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/5-_ a.snapshot.gz Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/5-_ a.snapshot.gz Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/uniᄉ Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/uniᄉ Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/:\ Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/:\ Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/A Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/A Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/foo Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/foo Making directory /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/hl Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/hardlinked_file1 Hard linking /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/hl/hardlinked_file2 to /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/hardlinked_file1 Unable to import module xattr. Extended attributes not supported on filesystem at /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0 Unable to import module posix1e from pylibacl package. POSIX ACLs not supported on filesystem at /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0 Unable to import win32security module. Windows ACLs not supported by filesystem at /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0 Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/dir_inc_check Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/dir_inc_check Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/regfile Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/regfile Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/high_perms_file Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/high_perms_dir Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/high_perms_file Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/high_perms_dir Touching /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/symlinked_file1 Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/symlinked_file2 Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0/symlinked_file1 escape_dos_devices not required by filesystem at /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0 Deleting /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0 Removing directory /backup/data-rdiff/rdiff-backup-data/rdiff-backup.tmp.0 - Detected abilities for destination (read/write) file system: Ownership changing On Hard linking On fsync() directories On Directory inc permissions On High-bit permissions On Symlink permissions Off Extended filenames Previous backup seems to have failed, regressing destination now. Exception 'Bad
Re: [rdiff-backup-users] Are my backups too corrupted to recover?
, line
497, in backup_final_init
checkdest_if_necessary(rpout)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
905, in checkdest_if_necessary
dest_rp.conn.regress.Regress(dest_rp)
File /usr/lib/python2.6/site-packages/rdiff_backup/regress.py,
line 71, in Regress
for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
File /usr/lib/python2.6/site-packages/rdiff_backup/rorpiter.py,
line 275, in __call__
assert 0, Bad index order: %s = %s % (self.index, index)
On
Windows reserved filenames Off
Access control lists Off
Extended attributes Off
Windows access control lists Off
Case sensitivity On
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forksOff
Mac OS X Finder information Off
-
Backup: must_escape_dos_devices = 0
Regressing to Wed Mar 4 19:36:32 2009
Restoring with increment base 1 for file Common Files/Copy of
Goldmine/MailBox/Attach/bhw/myel patentee desecrater adore suffolk
immediate vineyard sycophantictomb twirly bloody clara carbohydrate
procrustes greece stanza drophead demitting plaintive solicitation
aspersion callous .gif
Traceback (most recent call last):
File /usr/bin/rdiff-backup, line 30, in module
rdiff_backup.Main.error_check_Main(sys.argv[1:])
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
304, in error_check_Main
try: Main(arglist)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
324, in Main
take_action(rps)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
280, in take_action
elif action == backup: Backup(rps[0], rps[1])
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
337, in Backup
backup_final_init(rpout)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
497, in backup_final_init
checkdest_if_necessary(rpout)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
905, in checkdest_if_necessary
dest_rp.conn.regress.Regress(dest_rp)
File /usr/lib/python2.6/site-packages/rdiff_backup/regress.py,
line 71, in Regress
for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
File /usr/lib/python2.6/site-packages/rdiff_backup/rorpiter.py,
line 275, in __call__
assert 0, Bad index order: %s = %s % (self.index, index)
AssertionError: Bad index order: ('long_filename_data', '1') =
('Common Files', 'Copy of Goldmine', 'MailBox', 'Attach', 'bhw',
'norcrossgroupnotepad(1).pdf')
==
Greg
On Fri, Apr 3, 2009 at 4:52 PM, Greg Freemyer greg.freem...@gmail.com wrote:
Andrew/Josh,
They are attached. About 3MB total. I did not compress it, hope
you've got a fast link.
You both seemed interested. No need send something this big to the whole
list.
Greg
On Wed, Apr 1, 2009 at 1:44 PM, Andrew Ferguson adfergu...@gmail.com wrote:
On Apr 1, 2009, at 1:07 PM, Greg Freemyer wrote:
I was shuffling things around and may have broke my backup directory.
I would like to recover it if it can be done relatively easily.
Any clue how to restore functionality short of destroying my backup
directory and recreating it?
See results below
# rdiff-backup --version
rdiff-backup 1.2.6
+ rdiff-backup -v5 --print-statistics --exclude /data/EDD-Batches
/data //backup/data-rdiff
Previous backup seems to have failed, regressing destination now.
Exception 'Bad index order: ('long_filename_data', '1') = ('Common
Files', 'Copy of Goldmine', 'MailBox', 'Attach', 'bhw',
'norcrossgroupnotepad(1).pdf')' raised of class 'type
'exceptions.AssertionError'':
Hmm, interesting. I was just thinking about this bug the other day and
hoping to figure it out.
Can you send me:
The complete output of:
$ rdiff-backup -v8 --print-statistics --exclude /data/EDD-Batches /data
//backup/data-rdiff
Recursive directory listing of:
/data/Common Files/Copy of Goldmine/MailBox/Attach/
/backup/data-rdiff/Common Files/Copy of Goldmine/Mailbox/Attach/
plus the listing of /backup/data-rdiff/ and
/backup/data-rdiff/rdiff-backup-data/
Thanks!
Andrew
--
Greg Freemyer
Head of EDD Tape Extraction and Processing team
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf
The Norcross Group
The Intersection of Evidence Technology
http://www.norcrossgroup.com
--
Greg Freemyer
Head of EDD Tape Extraction and Processing team
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf
The Norcross Group
The Intersection of Evidence
[rdiff-backup-users] Are my backups too corrupted to recover?
I was shuffling things around and may have broke my backup directory.
I would like to recover it if it can be done relatively easily.
Any clue how to restore functionality short of destroying my backup
directory and recreating it?
See results below
# rdiff-backup --version
rdiff-backup 1.2.6
+ rdiff-backup -v5 --print-statistics --exclude /data/EDD-Batches
/data //backup/data-rdiff
Previous backup seems to have failed, regressing destination now.
Exception 'Bad index order: ('long_filename_data', '1') = ('Common
Files', 'Copy of Goldmine', 'MailBox', 'Attach', 'bhw',
'norcrossgroupnotepad(1).pdf')' raised of class 'type
'exceptions.AssertionError'':
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
304, in error_check_Main
try: Main(arglist)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
324, in Main
take_action(rps)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
280, in take_action
elif action == backup: Backup(rps[0], rps[1])
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
337, in Backup
backup_final_init(rpout)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
497, in backup_final_init
checkdest_if_necessary(rpout)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
905, in checkdest_if_necessary
dest_rp.conn.regress.Regress(dest_rp)
File /usr/lib/python2.6/site-packages/rdiff_backup/regress.py,
line 71, in Regress
for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
File /usr/lib/python2.6/site-packages/rdiff_backup/rorpiter.py,
line 275, in __call__
assert 0, Bad index order: %s = %s % (self.index, index)
Traceback (most recent call last):
File /usr/bin/rdiff-backup, line 30, in module
rdiff_backup.Main.error_check_Main(sys.argv[1:])
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
304, in error_check_Main
try: Main(arglist)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
324, in Main
take_action(rps)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
280, in take_action
elif action == backup: Backup(rps[0], rps[1])
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
337, in Backup
backup_final_init(rpout)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
497, in backup_final_init
checkdest_if_necessary(rpout)
File /usr/lib/python2.6/site-packages/rdiff_backup/Main.py, line
905, in checkdest_if_necessary
dest_rp.conn.regress.Regress(dest_rp)
File /usr/lib/python2.6/site-packages/rdiff_backup/regress.py,
line 71, in Regress
for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
File /usr/lib/python2.6/site-packages/rdiff_backup/rorpiter.py,
line 275, in __call__
assert 0, Bad index order: %s = %s % (self.index, index)
AssertionError: Bad index order: ('long_filename_data', '1') =
('Common Files', 'Copy of Goldmine', 'MailBox', 'Attach', 'bhw',
'norcrossgroupnotepad(1).pdf')
Greg
--
Greg Freemyer
Head of EDD Tape Extraction and Processing team
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf
The Norcross Group
The Intersection of Evidence Technology
http://www.norcrossgroup.com
___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
[rdiff-backup-users] OpenSuse 11.1 using rdiff-backup-1.2.1
All, How old is that version? I just upgraded one of my server to OS 11.1 and I'm getting a couple errors when rdiff-backup runs. I suspect I need a newer rdiff-backup version, but I prefer to get this type of software via OpenSUSE's software management system. Greg -- Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence Technology http://www.norcrossgroup.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Graphical User Interfaces (GUIs) for rdiff-backup - Later as Wiki-Entry - Request for comments now
On Tue, Jan 6, 2009 at 6:58 AM, Oliver Mulatz sira...@arcor.de wrote: Hi *, most will use rdiff-backup via commandline, but some users don't feel comfortable with/are used to graphical interfaces, so the below list might help. Especially pyBackPack looks interesting, although it is not found in a Google-search for rdiff-backup GUI. ## pyBackPack - sponsored 2005 Google Summer of Code project ## ## PythonGTK2+ GUI for RDB ## http://andrewprice.me.uk/projects/pybackpack/ http://projects.sucs.org/projects/pybackpack/ http://andrewprice.me.uk/weblog/all/pybackpack Should work in CygWin under windows, too: http://lists.sucs.org/pipermail/pybackpack/2007-May/33.html A graphical backup utility based on rdiff-backup. The latest version is 0.5.6, released on September 25 2008. ## FlyBack ## http://code.google.com/p/flyback/ http://code.google.com/p/flyback/downloads/list + PyGTK, so mostly cross-platform - only several releases in Nov 2007 ## rdiffweb ## http://www.rdiffweb.org/wiki/index.php?title=Main_Page + requires only Python, PySQLite, CherryPy (for local webserver) ## rdiffbackupweb ## http://rdiffbackupweb.sourceforge.net/ - requires php + mysql backend ## Keep ## - only working in Linux KDE: http://jr.falleri.free.fr/keep/wiki/Home http://jr.falleri.free.fr/keep/wiki/Screenshots --- If I have forgotten other working + maintained GUIs, please post to the list, so I can create a RDB Wiki-Page for GUIs. Since I do not require a GUI, I haven't tested any of the above, so feedback on this is also welcome. Cheers, Oliver Oliver, I have not used any of the GUI tools either. Personally I run rdiff-backup on a few different fileservers. It would be nice to give the users a HTTP interface to the backups, etc. So if you do document whats out there, please organize them by user interface: X / KDE / HTTP / Windows native / etc. Thanks Greg -- Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence Technology http://www.norcrossgroup.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
[rdiff-backup-users] Does rdiff implement kernel specific optimizations?
Devels, I just came across tee() and splice(). They were introduced in 2.6.17, so relatively new, but not that new. http://linux.die.net/man/2/tee http://linux.die.net/man/2/splice They allow data to be copied from one file descriptor to another without the data coming into userspace. If I was still an active developer, I think about modifying rdiff-backup to use them if there available. I doubt it would really impact the speed of rdiff-backup, but it seems like a fun project and I always liked learning new system calls. Greg -- Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence Technology http://www.norcrossgroup.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] different versions
On Mon, Jan 5, 2009 at 10:32 AM, Chris Wilson ch...@qwirx.com wrote: Hi Peter, On Mon, 5 Jan 2009, Peter wrote: Being a novice at Linux and still learning, I use a few different versions. My main mail server is Fedora, My desktop is Kubuntu, andother is another flavoour etc. On the Ubuntu unit I use apt-get install rdiff-backup and it installed version 1.15.something On my fedora I had version 1.05 or something. and offcourse they didn't work when my fedora tried to backup to my ubuntu. What's the easiest way for me to get and install the same version on all boxes? The easiest way I've found is to download the source code from the rdiff-backup website (rdiff-backup.nongnu.org) and build from source on each computer. Ignoring windows installs, rdiff-backup is not even compiled iirc, so installing it from scratch is one of the most trivial installs from source you can do. Do read down to the requirements section of the project page and make sure you have those or else things won't work. I don't think the requirements have changed in a long while, so you should not have anything you need to change, but do check. Greg -- Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence Technology http://www.norcrossgroup.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Data Privacy from system administrator withrdiff-backup
On Mon, Dec 29, 2008 at 7:49 AM, Dominic d...@edendevelopments.co.uk wrote: Greg Freemyer wrote: I use rdiff-backup to a local encfs directory. Then I do a rsync of the encrypted version of the encfs directory to a third party location. It is working fine so far. Admittedly my only restores from the remote site have been tests. ie. It is for disaster recovery purposes only. I use the local rdiff-backup copy for normal data recovery needs. That sounds clever. But I don't understand why it is not secure to use encfs directly on the third party remote server (assuming that it is available of course)? Something like this (sorry this is from a Windows client hence use of plink and unusual escapes): Ignoring security, there is also a bandwidth issue. I'm backing up about 300GB across a T-1. Not very active, so I'm sending less that a GB per night most times. A T-1 basically does about .5 GB/hr., so the nightly activity is no big deal. But 600 hrs to backup / restore the entire 300GB dataset is not very practical for routine use. ie. a few weeks to do the restore. So I treat that as a true disaster recovery backup (fire / natural disaster / etc.). If I were to lose my production raid array, I would restore from the local backup. Greg -- Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence Technology http://www.norcrossgroup.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Data Privacy from system administrator withrdiff-backup
On Sun, Dec 28, 2008 at 8:04 AM, Andreas Olsson andr...@arrakis.se wrote: On Sun, 28 Dec 2008 11:11:05 +, Dominic d...@edendevelopments.co.uk wrote: Is it possible with rdiff-backup to create backups on a (linux) backup server which cannot then be accessed by the system administrator? I think that Duplicity was created to provide this privacy, but in other respects rdiff-backup seems a much more polished solution and I wondered if this problem can be overcome while sticking with rdiff-backup. I guess you could always combine a SSHFS- and a EncFS-mount. http://wiki.rdiff-backup.org/wiki/index.php/BackupToEncfsAcrossSshfs Myself I would probably prefer using Duplicity. // Andreas I use rdiff-backup to a local encfs directory. Then I do a rsync of the encrypted version of the encfs directory to a third party location. It is working fine so far. Admittedly my only restores from the remote site have been tests. ie. It is for disaster recovery purposes only. I use the local rdiff-backup copy for normal data recovery needs. Greg -- Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence Technology http://www.norcrossgroup.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] long file path support for windows
Ryan, Good theory, but it does not always work in practice. If you run a file server and you share D:\home\John.Doe as a private share, then John Doe gets 255 chars + strlen(\home\John.Doe) to work with from his local workstation. So to backup a windows file server properly, you basically have to address the 255 char issue. Greg On Mon, Oct 27, 2008 at 7:40 PM, Ryan How [EMAIL PROTECTED] wrote: Thanks for your input. It all makes a lot of sense. So, seems it would save more headaches to just not use long file paths :) Ryan Greg Freemyer wrote: I have never used rdiff-backup from Windows, but I am a little familiar with the long path issue. History (as I understand it): NTFS has supported long paths from day 1 Way back when (the 90's), the windows kernel only had one API for accessing files. That API was restricted to 255 chars (or so). When Windows 2000 was released a new alternate Unicode API was included. With that API you can in theory access paths up to 32K. So in a sense Windows has supported long paths for a long time now, BUT few if any applications used the new API for years after that original release. In particular the 3 most fundamental tools for accessing files are: Windows File Explorer, Open File Dialog Box, and the Command Prompt. For Win2000/Win2003/XP all 3 use the original restricted file API. When VISTA/Win2008 was released a new default File Explorer and Open File Dialog Box were provided that uses the new API and thus you can access long paths via them. The Command Prompt still does NOT provide access to long paths. Microsoft apparently still thinks creating files with long paths is a bad idea, so the new file explorer will not let you do that, nor does the Save As dialog box. HTH Greg On Fri, Oct 24, 2008 at 10:34 PM, Ryan How [EMAIL PROTECTED] wrote: Hmmm... maybe this isn't a bug at all? I was just trying to create long path in explorer and it has a limit also (round about the 260 it would seem). I used the UNC path and I couldn't even navigate to the deepest level getting an error. I used the subst command to go beyond the path limit. When going back to the original path you simply can't access the files from explorer! I just tried it on vista too!. Doesn't work either!. It seems long paths on windows are stuffed anyway. If you create a file you can't access it anyway. (I know if rdiff-backup worked with long paths then it could access it, but I like the idea of manually being able to get to the backup files if needs be) So the rules would just be, don't backup to a folder which is too deep, and don't backup from a folder that goes too deep! (because the added rdiff-backup-data folder will add extra onto the path and cause the error) Ok... so... can we just change rdiff-backup to instead of crashing to print out a warning that the path is too long and skip the file? The error would be originating in os.makedirs ? So if we can try catch (does python have that?) any calls then skip the file. Thanks for listening to my thinking out loud. Ryan How wrote: Hmm.. The current cygwin head has unicode and long file path support I believe. I wonder about going down that path again until windows python catches up... According to http://bugs.python.org/issue542314 the long file path issue is resolved in python 2.5 if using UNC paths ? I'm gonna do some investigating :) And I don't know about long file paths on Windows happening in the 1.2.x branch I think you're going to need to wait for Python 3 and an updated WinPython with Unicode operations. :-/ As a possible workaround, from the Windows CLI you can try to use subst to map a new drive letter into the middle of you longpath. ie. if you know you have long paths within c:\documents and settings\administrator\documents you should be able to do: subst x: c:\documents and settings\administrator\documents rdiff-backup x: And gain 40 or so chars. ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki __ Information from ESET NOD32 Antivirus, version of virus signature database 3553 (20081024) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki -- Greg Freemyer
Re: [rdiff-backup-users] New release: 1.2.2
For OpenSUSE users, they may be interested that stable versions of rdiff-backup are available in their build service: http://software.opensuse.org/search?baseproject=ALLq=rdiff-backup I don't see 1.2.2 yet, but assume it will be there in a few days. FYI: I know the build service will support numerous distros, I don't know how an additional distro is added to their proces for any given package. See the search pull-down for what is theoretically supported. Greg On Sun, Oct 19, 2008 at 1:02 PM, Andrew Ferguson [EMAIL PROTECTED] wrote: Hi all, As promised, a new rdiff-backup stable release. ChangeLog included below. Direct download links: http://savannah.nongnu.org/download/rdiff-backup/rdiff-backup-1.2.2.tar.gz http://savannah.nongnu.org/download/rdiff-backup/rdiff-backup-1.2.2-win32.zip GPG signatures: http://savannah.nongnu.org/download/rdiff-backup/rdiff-backup-1.2.2.tar.gz.asc http://savannah.nongnu.org/download/rdiff-backup/rdiff-backup-1.2.2-win32.zip.asc (I actually learned how to do detached-signatures this time! :-) My thanks to everyone who contributed patches, bug reports, and suggestions for this release. Andrew New in v1.2.2 (2008/10/19) --- Automatically resume after a failed initial backup. (Patch from Josh Nisly) Improve compatibility between Unix and remote native Windows client. It is now possible to use SSH daemons other than Putty on Windows. (Andrew Ferguson) Print a more informative error message if the user's remote shell prints extraneous information before rdiff-backup runs. (Andrew Ferguson) Don't backup Windows ACLs if the --no-acls option is specified. Thanks to Richard Metzger for reporting the issue. (Andrew Ferguson) Add error handling and logging to Windows ACL support; fixes Windows backup to SMB share. Improve test in fs_abilities to determine if Windows ACLs are supported. (Andrew Ferguson) Add a warning message if extended attributes support is broken by the filesystem (such as with older EncFS versions). (Andrew Ferguson) Improve handling of Windows ACLs by switching to API functions which understand inherited ACEs; fixes support for Windows 2000. (Andrew Ferguson) Support extended attributes on symbolic links. (Andrew Ferguson) On Mac OS X, read the com.apple.FinderInfo extended attribute since it is the only storage location for the 'busy' (Z) Finder attribute. (Andrew Ferguson) Properly fix AttributeError: RPath instance has no attribute 'inc_compressed' bug. Fix in 1.1.12 was in correct place, but wrong solution. (Andrew Ferguson) Improve support for Python 2.5, which refactored the built-in exceptions so that SystemExit and KeyboardInterrupt no longer derive from Exception. Closes support request #106504. (Andrew Ferguson) Adjust --exclude-if-present option to support directories, symlinks, device files, etc. Closes bug #24192. Thanks to Vadim Zeitlin for the suggestion. ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki -- Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence Technology http://www.norcrossgroup.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Any plans for Amazon S3 support?
On Mon, Aug 25, 2008 at 7:21 PM, Chris Wilson [EMAIL PROTECTED] wrote: Hi Greg, On Mon, 25 Aug 2008, Greg Freemyer wrote: I took a look at S3's pricing today and it is pretty good. ($0.15/GB/month) http://aws.amazon.com/s3 The trouble from a rdiff-backup perspective is that it has a custom API. I'd like to see rdiff-backup have support for backing up to S3, but that may be much difficult than I expect. So count this as a vote for S3 support. Have you looked at Amazon EBS? With this, you should be able to back up to an ECC instance running rdiff-backup, and save the resulting filesystem images on S3. Cheers, Chris. Hey Chris, I've been deleting the articles about most of the Amazon Cloud instead of reading them, so a couple questions if you happen to know. I gather the ECC is basically a virtual server with a full OS installation? (I currently have one on those at a provider that I run our company website on.) If so, could I consolidate my current webserver virtual Server and then add on the EBS storage to backup our fileservers to it? Not sure how the ECC is priced, but EBS includes per i/o pricing. Would that be a lot for a rdiff-backup backend server? Also, is anyone already doing something like this? Personally, I use rdiff-backup to a local drive, then rsync the whole repository offsite to a online storage vendor. I'm currently paying about $75/month for 250GB of repository. Thanks Greg -- Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence Technology http://www.norcrossgroup.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] external hard disk backups - best practice
On Mon, Mar 17, 2008 at 5:02 PM, Chris Wilson [EMAIL PROTECTED] wrote: Hi Warren and all, On Mon, 17 Mar 2008, Warren Guy wrote: While this won't help you get your data back, perhaps a point that can be made from your unfortunate story is the importance of maintaining more than one set of backups. A second backup set would potentially allow recovery of data in this and other circumstances where recovery from another backup disk fails for whatever reason. Perhaps others can add other best practice tips they have, for the benefit of current and future readers of the list? I rdiff-backup to two locations, one onsite and one offsite. Even if I lose my onsite backup through some unfortunate accident, I won't lose the offsite one. And it being offsite would discourage me from pulling the disk out and sticking it into the machine to be restored :-) Note that this would work _much_ better if rdiff-backup was not so inefficient in its use of network bandwidth. It transfers about 1GB of metadata every day over the network, for about 300GB of backed-up files, and takes 4-12 hours to do so. Cheers, Chris. Chris, I also maintain a local and offsite rdiff backup copy. But I only have rdiff create the first copy. Then I rsync that to my offsite location. I have about 150GB of data I backup. rdiff + rsync takes about 45 minutes most nights if there is not a significant change to my dataset. (FYI: I have a number of spindles involved via raid, so I'm not slowed by disk i/o as much as one might expect.) The bad news is that if my first set gets corrupted, then rsync will relay the corruption out to the offsite copy. My reason for two copies is disaster recovery, not backup repository corruption. May need to rethink that based on this discussion. Greg -- Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence Technology http://www.norcrossgroup.com ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] rdiff-backup 1.1.10 released
Dean, Once enough time has gone by to move this to stable on the front page please post another e-mail. FYI: I tried to get OpenSUSE factory to update to 1.1.10, but they said no because it was a devel/unstable release. Thanks Greg On 5/13/07, dean gaudet [EMAIL PROTECTED] wrote: i realised i'd moved my rdiff-backup mail folder after linux-kernel in my tab list... and hadn't got past linux-kernel in months. oops. better late than never. as before, i haven't tested it (i'm still using 1.0.5), i'm just releasing it because there was a request for a new release. -dean New in v1.1.10 (2007/05/12) --- New --exclude-if-present option (i.e. --exclude-if-present .nobackup). (Jeff Strunk). Use signal 0 rather than signal.NSIG when testing if another rdiff-backup is still running. (Patch from Sébastien Maret) Sockets don't have extended attributes -- don't try to access them. (Patch from Andrew Ferguson.) Fix restore from read-only bug -- rx perms on a repository directory are enough, no need for write perms when restoring. (patch from Andrew Price) Fix --list-increments bug in set_must_escape_dos_devices. (Marc Dyksterhouse) ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki -- Greg Freemyer The Norcross Group Forensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Ownership not preserved
Did you try a restore?
I believe the ownership and other metadata is (can be) maintained
separately from the file, so you really can't tell much by just
looking at the ownership etc. of the backup file itself.
Greg
On 5/8/07, Stefan Wimmer [EMAIL PROTECTED] wrote:
Hello,
I set up rdiff-backup for backing up my hosts following this howto:
http://arctic.org/~dean/rdiff-backup/unattended.html
The problem is that *all* files in my backup directory are now owned by
backup:backup regardless whether it was host:/root or host:/etc :-{
If I run rdiff-backup from the hosts I want to backup as root the
ownership is preserved but I wanted to avoid this because there is a lot
of trouble with cronjobs as root ...
Is there anything I miss or I could do to solve this problem?
Thanks in advance
Stefan
___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
--
Greg Freemyer
The Norcross Group
Forensics for the 21st Century
___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Re: offsite service providers?
On 3/16/07, Richard Steven Hack [EMAIL PROTECTED] wrote: Greg Freemyer wrote: With a Dedicated Server or VPS (virtual private server) I would feel confident enough, but with a truly shared server at a minimum the local admin could access the files. If your box isn't physically under your control, ANYBODY who DOES have physical access can get in. That is the purpose of a encrypted filesystem. I have not done any research recently, but I assume that I could set one up via FUSE. Then anytime the server / VPS rebooted I would have to login via SSH and manually do the mount. For me at least that would be acceptable. It's that simple, dedicated server or no. That's a basic security concept. So the risk isn't any higher than ANY Web hosting situation, in my view. Again, with normal web hosting I doubt if I have a way to setup an encrypted FS. (I may be wrong about that. I'll do a little more research now that I think about it more. ie. Dreamhost provides ssh access, but no root access. It is possible that via fuse I can do this as a normal user, not just as root. If you have files detailing your plans to fly planes into a tall building, I wouldn't store them there. But if you're willing to let your customers credit card numbers be stored there, why not any other files - as long as they're legal? Actually it is not our companies files I'm worried about. It is our clients data that I have a stronger requirement to secure. Specifically we do some work with the US Fed. Gov. and the contracts call for us to ensure Business Secrecy. I'm not sure what that really means, but I talked internally and we're not comfortable sending that class of data out of our control unless it is encrypted. Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Re: offsite service providers?
Okay, I think I have a plan for doing this. Anyone care to comment? === Use EncFS to expose an encrypted and non-encrypted FS on my local machine. ie. /backup and /backup-raw. /backup-raw would be the encrypted version. /backup is the unencrypted version. rdiff-backup would be configured to write to /backup. Each night after rdiff-backup runs I would use standard rsync to duplicate the full /backup-raw to the remote site so all the files offsite would be encrypted. That is sounding fairly safe and reasonably bandwidth efficient. Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Re: offsite service providers?
On 3/16/07, Dimi Paun [EMAIL PROTECTED] wrote: On Fri, March 16, 2007 17:29, Greg Freemyer wrote: That is sounding fairly safe and reasonably bandwidth efficient. That's true, if you don't have large files that change in incremental ways. The encrytion _should_ render these useless for differential upload. In fact, that mey be the case even for files that don't change, depending on how EncFS works. I'm not knowledgeable on EncFS, but I believe in general filesystems that encrypt do it one block at a time in such a way to have the encrypted block occupy exactly the same space on disk as the unencrypted block would. So assume it is a 4k block, then if you update any data in that block it must: read (4k block) - unencrypt - modify - encrypt - write (4k block). Otherwise writes to the middle of the file would be highly inefficient. So changes in the middle of a large file should not impact the rest of file outside of the block you updated. This should allow rsync to work well. Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] offsite service providers?
On 3/15/07, Jonathan Sefton [EMAIL PROTECTED] wrote: On 14 Mar 2007, at 16:50, Greg Freemyer wrote: Or know of another cost-effective way to get 200-500 GB of offsite storage for rdiff backups? (I tend to have very small delta's so I'm won't need much bandwidth after the initial pass.) I've just started using rsync.net for this - http://rsync.net/ - they currently support rdiff-backup 1.0.4. Standard rate is $1.60 per GB per month, but this is discounted to $0.80 per GB per month for 400-999 GB. So you'd be looking at maybe $400 per month - does that count as cost effective? (There are no bandwidth charges). Excellent service so far. It does look like a good product and I like the RAID-6 storage. I'll think about the $400/month for 500GB. Not outrageous, but more than I was hoping for. Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Re: offsite service providers?
Dreamhost looks very intriguing. (They have a deal right now $19.95/month for their 338GB plan and they say their filers all use RAID of some sort and have hotswap capability.) They offer shell access and have python installed. I know I need the password-less ssh connection. Anything else I should verify before I sign-up. It does not look like much of a risk. Greg On 3/15/07, Richard Steven Hack [EMAIL PROTECTED] wrote: The Dreamhost ISP is offering a file hosting option for about $2.50 per GB ONE-TIME payment to host the files FOREVER. That's about as cheap as it gets for file hosting. Those guys tend to be crazy, but the price is right. I don't know if they have a way to set up with rdiff-backup to their storage, but you might contact them and see if they can do a deal. At the moment, I think it's only open to Dreamhost Web hosting customers, but at $9.95 for their basic hosting plan (which BTW gives you a couple hundred GB of Web space in itself which some people use to store files, plus a terabyte or more of download bandwidth), basically it would cost you $10/month to use their Files Forever service plus the one-time payment per GB. They are marketing this as a service for content providers - they provide a mechanism for loaning and selling access to the files to other people. But they also say it can be used for archive storage of private files. Check it out here: https://files.dreamhost.com/ ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki -- Greg Freemyer The Norcross Group Forensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Re: offsite service providers?
Good question. With a Dedicated Server or VPS (virtual private server) I would feel confident enough, but with a truly shared server at a minimum the local admin could access the files. Does rdiff-backup itself have any sort of solution for when the remote end is not itself as private as we would like? Thanks Greg On 3/15/07, Jeff Strunk [EMAIL PROTECTED] wrote: How secure are will your private files be once you put them on a shared host? Will other customers be able to access them? Etc. Jeff On Thursday 15 March 2007 3:34 pm, Greg Freemyer wrote: Dreamhost looks very intriguing. (They have a deal right now $19.95/month for their 338GB plan and they say their filers all use RAID of some sort and have hotswap capability.) They offer shell access and have python installed. I know I need the password-less ssh connection. Anything else I should verify before I sign-up. It does not look like much of a risk. Greg On 3/15/07, Richard Steven Hack [EMAIL PROTECTED] wrote: The Dreamhost ISP is offering a file hosting option for about $2.50 per GB ONE-TIME payment to host the files FOREVER. That's about as cheap as it gets for file hosting. Those guys tend to be crazy, but the price is right. I don't know if they have a way to set up with rdiff-backup to their storage, but you might contact them and see if they can do a deal. At the moment, I think it's only open to Dreamhost Web hosting customers, but at $9.95 for their basic hosting plan (which BTW gives you a couple hundred GB of Web space in itself which some people use to store files, plus a terabyte or more of download bandwidth), basically it would cost you $10/month to use their Files Forever service plus the one-time payment per GB. They are marketing this as a service for content providers - they provide a mechanism for loaning and selling access to the files to other people. But they also say it can be used for archive storage of private files. Check it out here: https://files.dreamhost.com/ ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki -- Greg Freemyer The Norcross Group Forensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
[rdiff-backup-users] offsite service providers?
All, I've been using rdiff-backup internally for a while, but now we want to ensure the data is really getting offsite. I know there are a bunch of linux virtual server providers out there that could easily be configured to be a rdiff-backup server. Is anyone doing this? Or know of another cost-effective way to get 200-500 GB of offsite storage for rdiff backups? (I tend to have very small delta's so I'm won't need much bandwidth after the initial pass.) Thanks Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Simple nightly script?
message.message += Error running parsing script\n
message.message += parser.error
message.message += \n\n
end
end
if parsers.size == 0
message.subject = 'Backup(s) unknown'
message.message = 'Status unknown, did not get a valid log.'
elsif ok_backups == parsers.size
message.subject = 'Backup(s) OK'
else
message.subject = 'Backup(s) failed'
end
message.subject += (#{ok_backups}/#{parsers.size})
message.log = input
message.send
end
--
Config file:
-
cat /etc/parse-rdiff-backup.conf
message:
main_contact: [EMAIL PROTECTED]
cc_contact: [EMAIL PROTECTED]
parser:
statistics:
Direction: Starting mirror|increment\ operation (.*)
Backup started: ^StartTime [\d\.]+ \((.*)\)$
#EndTime: ^EndTime (.*)$
#ElapsedTime: ^ElapsedTime (.*)$
Source files: ^SourceFiles (.*)$
Source files size: ^SourceFileSize (.*)$
#MirrorFiles: ^MirrorFiles (.*)$
#MirrorFileSize: ^MirrorFileSize (.*)$
#NewFiles: ^NewFiles (.*)$
New files size: ^NewFileSize (.*)$
#DeletedFiles: ^DeletedFiles (.*)$
Deleted files size: ^DeletedFileSize (.*)$
#ChangedFiles: ^ChangedFiles (.*)$
#Changed files size: ^ChangedFileSize (.*)$
#ChangedMirrorSize: ^ChangedMirrorSize (.*)$
#IncrementFiles: ^IncrementFiles (.*)$
#IncrementFileSize: ^IncrementFileSize (.*)$
Destination size change: ^TotalDestinationSizeChange \d+ \((.*)\)$
Errors: ^Errors (.*)$
Script is licensed under the GNU GPLv2 and Author is Christian Marie
while working for Solutions First.
thanks
dave
Greg Freemyer wrote:
All,
I've looked at the examples at
http://www.nongnu.org/rdiff-backup/examples.html, but none of them seem
to address automated nightly scripts and error handling.
Are there some more complex examples available?
=== Details
I haven't used rdiff-backup for a couple of years and I never did have
it integrated into a nightly cron script for production use.
My needs have changed and I want to give it another shot.
I'm backing up to a local directory on a dedicated backup disk so it is
easy enough to add rdiff-backup /src /backup to my backup script.
But what about catching errors?
Seems like I should be sending output to a log file, grepping thru it
and e-mailing it to myself if anything goes wrong.
I've looked at the examples at
http://www.nongnu.org/rdiff-backup/examples.html, but none of them seem
to address this common need.
Thanks
Greg
--
Greg Freemyer
The Norcross Group
Forensics for the 21st Century
___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
--
Greg Freemyer
The Norcross Group
Forensics for the 21st Century
___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] 4GB limit fixed in FC5 -- no longer affects rdiff-backup
On 12/3/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: if librsync is unmaintained now, why isn`t anybody taking over maintenance then? librsync code includes major league mathematical magic. Very few people would feel comfortable taking over maintenance. I thought the maintainer was hired by Google in the Spring and he said he was going to use some/all of his 20% time working on librsync. I guess that did not happen. Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] Simple nightly script?
: statistics: Direction: Starting mirror|increment\ operation (.*) Backup started: ^StartTime [\d\.]+ \((.*)\)$ #EndTime: ^EndTime (.*)$ #ElapsedTime: ^ElapsedTime (.*)$ Source files: ^SourceFiles (.*)$ Source files size: ^SourceFileSize (.*)$ #MirrorFiles: ^MirrorFiles (.*)$ #MirrorFileSize: ^MirrorFileSize (.*)$ #NewFiles: ^NewFiles (.*)$ New files size: ^NewFileSize (.*)$ #DeletedFiles: ^DeletedFiles (.*)$ Deleted files size: ^DeletedFileSize (.*)$ #ChangedFiles: ^ChangedFiles (.*)$ #Changed files size: ^ChangedFileSize (.*)$ #ChangedMirrorSize: ^ChangedMirrorSize (.*)$ #IncrementFiles: ^IncrementFiles (.*)$ #IncrementFileSize: ^IncrementFileSize (.*)$ Destination size change: ^TotalDestinationSizeChange \d+ \((.*)\)$ Errors: ^Errors (.*)$Script is licensed under the GNU GPLv2 and Author is Christian Mariewhile working for Solutions First.thanksdaveGreg Freemyer wrote: All, I've looked at the examples at http://www.nongnu.org/rdiff-backup/examples.html, but none of them seem to address automated nightly scripts and error handling. Are there some more complex examples available? === Details I haven't used rdiff-backup for a couple of years and I never did have it integrated into a nightly cron script for production use. My needs have changed and I want to give it another shot. I'm backing up to a local directory on a dedicated backup disk so it is easy enough to add rdiff-backup /src /backup to my backup script. But what about catching errors? Seems like I should be sending output to a log file, grepping thru it and e-mailing it to myself if anything goes wrong. I've looked at the examples at http://www.nongnu.org/rdiff-backup/examples.html, but none of them seem to address this common need. Thanks Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki -- Greg FreemyerThe Norcross GroupForensics for the 21st Century ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
