Re: [rdiff-backup-users] More patches to get rdiff-backup working under cygwin/windows
Dean, Actually, an extra os.close was already in the except block so I don't think there is an fd leak here. ASCII 128 thru 255 are valid in FAT-32 filenames...and common outside the US. I did add 127 (0177) to the quote list since that is not a valid char. This is now in fs_abilities.py.2.patch. The new patches (fs_abilities.py.2.patch and rpath.py.patch) should work against the head of cvs. These patches haven't changed and you already committed them: http://www.visiwave.com/download/rdiff_backup/FilenameMapping.py.patch http://www.visiwave.com/download/rdiff_backup/Security.py.patch http://www.visiwave.com/download/rdiff_backup/fs_abilities.py.patch These two still need to be applied: http://www.visiwave.com/download/rdiff_backup/fs_abilities.py.2.patch http://www.visiwave.com/download/rdiff_backup/rpath.py.patch Sorry this got so confusing. I was actually trying to keep it simple. For those using v1.1.7 with cygwin, just apply all 5 patches in the above order. marc dean gaudet wrote: On Mon, 29 Jan 2007, Marc Dyksterhouse wrote: Dean, I looked into what was happening with fsync under cygwin. Turns out fsync returns EACCES for any file so I added that to the except clause to prevent the exception from being re-raised. After fixing that, a few more exceptions where happening because the 'data' array is not filled in under cygwin (not sure why). I put some checks around some of the places that data['xxx'] is accessed to prevent the exceptions. The result is a new rpath.py patch (link below) that seems to work and should be cross platform. I'm also submitting a second fs_abilities.py patch. This one addresses your concern about which chars are quoted. The problem with the original regex is that it didn't quote * or . My new regex only quotes the chars that are not valid in FAT-32 plus the semicolon since that's the escape char. New regex: '\000-\037\*/:?|;' Complete list: http://www.visiwave.com/download/rdiff_backup/rpath.py.patch i'd prefer a specific try/except around the os.fsync to catch the EACCESS -- otherwise you'll be skipping the os.close... so i'm guessing you have an fd leak in this patch. http://www.visiwave.com/download/rdiff_backup/Security.py.patch http://www.visiwave.com/download/rdiff_backup/fs_abilities.py.patch http://www.visiwave.com/download/rdiff_backup/fs_abilities.py.2.patch hmm... do you need to escape \200-\377 as well... ? also ... any chance i could convince you to rebase all of these against the CVS HEAD, since i've already commited most of it? http://savannah.nongnu.org/cvs/?group=rdiff-backup thanks a bunch -dean ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] More patches to get rdiff-backup working under cygwin/windows
On Mon, 29 Jan 2007, Marc Dyksterhouse wrote: These two still need to be applied: http://www.visiwave.com/download/rdiff_backup/fs_abilities.py.2.patch http://www.visiwave.com/download/rdiff_backup/rpath.py.patch both are now committed... thanks! -dean ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] More patches to get rdiff-backup working under cygwin/windows
David, Yes to ssh. No to CIFS. marc David Kempe wrote: Hi Mark does this mean you are using ssh as the transport? have you happened to test these on unix to cifs(on windows)? dave Marc Dyksterhouse wrote: Note that these fixes are only needed when backing up unix to windows. Windows to windows or windows to unix shouldn't need this. Well, actually, the Security.py patch is probably needed in all these cases. ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] More patches to get rdiff-backup working under cygwin/windows
On Mon, 29 Jan 2007, Marc Dyksterhouse wrote: Dean, I looked into what was happening with fsync under cygwin. Turns out fsync returns EACCES for any file so I added that to the except clause to prevent the exception from being re-raised. After fixing that, a few more exceptions where happening because the 'data' array is not filled in under cygwin (not sure why). I put some checks around some of the places that data['xxx'] is accessed to prevent the exceptions. The result is a new rpath.py patch (link below) that seems to work and should be cross platform. I'm also submitting a second fs_abilities.py patch. This one addresses your concern about which chars are quoted. The problem with the original regex is that it didn't quote * or . My new regex only quotes the chars that are not valid in FAT-32 plus the semicolon since that's the escape char. New regex: '\000-\037\*/:?|;' Complete list: http://www.visiwave.com/download/rdiff_backup/rpath.py.patch i'd prefer a specific try/except around the os.fsync to catch the EACCESS -- otherwise you'll be skipping the os.close... so i'm guessing you have an fd leak in this patch. http://www.visiwave.com/download/rdiff_backup/Security.py.patch http://www.visiwave.com/download/rdiff_backup/fs_abilities.py.patch http://www.visiwave.com/download/rdiff_backup/fs_abilities.py.2.patch hmm... do you need to escape \200-\377 as well... ? also ... any chance i could convince you to rebase all of these against the CVS HEAD, since i've already commited most of it? http://savannah.nongnu.org/cvs/?group=rdiff-backup thanks a bunch -dean ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] More patches to get rdiff-backup working under cygwin/windows
On Fri, 26 Jan 2007, Marc Dyksterhouse wrote: http://www.visiwave.com/download/rdiff_backup/rpath.py.patch can you provide more information on why this is necessary? i'm assuming it's because cygwin/windows can't do an fsync in some situation... would it be possible to put another try/except around the os.fsync to catch that case instead of just disabling the fsync entirely? i don't think i want to commit this patch as is... unless that fsync really isn't necessary. http://www.visiwave.com/download/rdiff_backup/Security.py.patch committed to cvs HEAD http://www.visiwave.com/download/rdiff_backup/FilenameMapping.py.patch committed to cvs HEAD http://www.visiwave.com/download/rdiff_backup/fs_abilities.py.patch hmm i'm committing this anyhow because i didn't notice the previous patch depends on it... next time send them in order please :) but -- can you expand on this chunk: - else: return ^a-z0-9_ -. # quote everything but basic chars + else: return ^a-z0-9_ .- # quote everything but basic chars if self.dest_fsa.extended_filenames: return # Don't quote anything - else: return ^A-Za-z0-9_ -. + else: return ^A-Za-z0-9_ .- -. is a valid range... so this change will start escaping those things except for space dot dash... this was intentional? oh and... i tested none of this. i encourage folks to grab the cvs head and report back if it's broken or not. -dean ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
[rdiff-backup-users] More patches to get rdiff-backup working under cygwin/windows
Hi all, I had some issues backing up a linux machine to a Windows box using rdiff-backup. I got passed all my (known) issues today so I wanted to submit the fixes so others could take advantage of them. I started with version 1.1.7. Note that these fixes are only needed when backing up unix to windows. Windows to windows or windows to unix shouldn't need this. Well, actually, the Security.py patch is probably needed in all these cases. My issues: o The now famous fsync issue: I just included the same patch here that is already being used. o Files with asterisks (*) and other bad characters: I had a file named *.xxx that was causing the backup to fail. Turns out there's a pretty bad bug in the get_ctq_from_fsas logic where the regex used wants to quote all characters except a-z, underscore, space, period, and dash. But the faulty regex also includes all ASCII chars BETWEEN a space and a period. Bad regex: ^a-z0-9_ .- Good regex: ^a-z0-9_ .-. Putting the dash at the end makes it not be a range special character. This appears twice in fs_abilities.py. The second one includes A-Z (uppercase chars) too. o DOS device files (aux, prn, etc) caused backup to fail. Any file on linux that was named aux or aux.XXX or prn[.XXX] or con[.XXX] or nul[.XXX] or com[0-9][.XXX] or lpt[12][.XXX] would cause the backup to fail. I created a patch that tests the file system abilities and escapes these names if necessary by quoting the first character of the name. So, aux is converted to ;097ux. This was the biggest change. I've only tested it when backing up unix on to windows. Restore works too. o Security.py exception FilenameMapping.set_init_quote_vals was missing from the list of methods that can be called remotely. The four patches can be downloaded here: http://www.visiwave.com/download/rdiff_backup/rpath.py.patch http://www.visiwave.com/download/rdiff_backup/Security.py.patch http://www.visiwave.com/download/rdiff_backup/FilenameMapping.py.patch http://www.visiwave.com/download/rdiff_backup/fs_abilities.py.patch Regards, marc ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Re: [rdiff-backup-users] More patches to get rdiff-backup working under cygwin/windows
Hi Mark does this mean you are using ssh as the transport? have you happened to test these on unix to cifs(on windows)? dave Marc Dyksterhouse wrote: Note that these fixes are only needed when backing up unix to windows. Windows to windows or windows to unix shouldn't need this. Well, actually, the Security.py patch is probably needed in all these cases. ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
