unsubscribe rdiff-backup-users Carter J. Castor
On Sat, Aug 15, 2015 at 3:01 AM, <rdiff-backup-users-requ...@nongnu.org> wrote: > Send rdiff-backup-users mailing list submissions to > rdiff-backup-users@nongnu.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users > or, via email, send a message with subject or body 'help' to > rdiff-backup-users-requ...@nongnu.org > > You can reach the person managing the list at > rdiff-backup-users-ow...@nongnu.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of rdiff-backup-users digest..." > > > Today's Topics: > > 1. Re: State of the rdiff-backup project (Robert Nichols) > 2. Re: State of the rdiff-backup project (Tobias Leupold) > 3. Re: State of the rdiff-backup project (Claus-Justus Heine) > 4. Re: State of the rdiff-backup project (Tobias Leupold) > 5. Re: State of the rdiff-backup project (Dominic Raferd) > 6. Re: State of the rdiff-backup project (Claus-Justus Heine) > 7. Re: State of the rdiff-backup project (Claus-Justus Heine) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 14 Aug 2015 12:12:18 -0500 > From: Robert Nichols <rnicholsnos...@comcast.net> > To: rdiff-backup-users@nongnu.org > Subject: Re: [rdiff-backup-users] State of the rdiff-backup project > Message-ID: <mql7hj$gre$1...@ger.gmane.org> > Content-Type: text/plain; charset=windows-1252; format=flowed > > On 08/13/2015 02:16 PM, Claus-Justus Heine wrote: >> I'm really no python guy. But: I really >> would like to have the "regression takes ages" and "crowded directories >> take ages" being resolved. OR OR OR: would like to understand this >> issue. Cannot claim that I will be able to fix it. But I would very much >> like to understand what the heck is going on there. Just doesn't feel sa >> ne. > > I doubt that the "regression takes ages" problem can be fixed within > rdiff-backup. It's inherently a complex operation that requires > searching throughout the archive for things that aren't consistent with > the previous state. Remember that you can't trust that the latest > metadata files are consistent with the current state of the mirror and > increments. In large part it's due to use of the filesystem as a > database, with bits of information scattered in file names in the > increments directory and various metadata files. You're not going to > change that without a major rewrite. > > I suppose one solution to the regression issue is to store the archive > in a filesystem or LVM volume that supports snapshots. Rather than let > rdiff-backup do the regression, stop it and restore the snapshot. I > suspect the penalty in space (transient, until the snapshot is deleted) > and performance for the backup would be serious. And that still leaves > the issue of regressing more than the last level. > > Like you, I'm no Python guy. Every time I try to study it, I end up as a > lump in a snake's belly. I think it's because there are some things > about the language that I hate (starting with the use of whitespace as a > syntax element) and the incompatibility of major versions. And then > there is the tendency of Python programmers to believe that stack > backtraces are an acceptable substitute for meaningful error messages. > It all leaves a bad taste that I just can't get around. > > -- > Bob Nichols "NOSPAM" is really part of my email address. > Do NOT delete it. > > > > > ------------------------------ > > Message: 2 > Date: Fri, 14 Aug 2015 19:30:49 +0200 > From: Tobias Leupold <tobias.leup...@web.de> > To: rdiff-backup-users@nongnu.org > Subject: Re: [rdiff-backup-users] State of the rdiff-backup project > Message-ID: <1924852.jOvLUf9yh1@skoni> > Content-Type: text/plain; charset="us-ascii" > >> You're not going to change that without a major rewrite. > >> Like you, I'm no Python guy. > > I like Python. And I like rdiff-backup. But perhaps, some skilled programmer > has the energy to take the "chance" of the questionable state of rdiff-backup > by only continuing the concept of rdiff-backup and rewriting some similar > program from scratch, e. g. in C++. Perhaps with some metadata database to > solve the regression issue (which really sucks). And so on. > > Not that I would be remotely skilled enough in C++ to do so ... I'm only > thinking about what could be done ... > > > > ------------------------------ > > Message: 3 > Date: Fri, 14 Aug 2015 20:43:18 +0200 > From: Claus-Justus Heine <hims...@claus-justus-heine.de> > To: rdiff-backup-users@nongnu.org > Subject: Re: [rdiff-backup-users] State of the rdiff-backup project > Message-ID: <55ce36c6.5030...@claus-justus-heine.de> > Content-Type: text/plain; charset=utf-8 > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Am 14.08.2015 um 19:30 schrieb Tobias Leupold: >>> You're not going to change that without a major rewrite. >> >>> Like you, I'm no Python guy. > > I do not have any serious objections against Python. Its widely used, > and the limiting time factor is IO efficiency, and not the question > about the fasted for-loop ever possible. > > Concerning a "major rewrite": for me this is out of scope. I was rather > thinking about a code review. I first want to figure out whether there > are maybe places which can be optimized significantly, e.g. excessive > readir() stuff or something like this, or any other duplicate file IO > which doesn't hurt on small backup sets. Also before a major code > rewrite could be tackled at all, one would first have to understand what > rdiff-backup is doing beneath its skin. Otherwise it would not be clear > what the precise goal of such a major rewrite should be. Ok: to produce > something not worse than rdiff-backup. But to clarify this, you first > have to understand the existing program. > > Cheers, > > Claus > >> >> I like Python. And I like rdiff-backup. But perhaps, some skilled prog > rammer >> has the energy to take the "chance" of the questionable state of rdiff > - -backup >> by only continuing the concept of rdiff-backup and rewriting some simi > lar >> program from scratch, e. g. in C++. Perhaps with some metadata databas > e to >> solve the regression issue (which really sucks). And so on. >> >> Not that I would be remotely skilled enough in C++ to do so ... I'm on > ly >> thinking about what could be done ... >> >> _______________________________________________ >> rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org >> https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users >> Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBac > kupWiki >> > > > - -- > Claus-Justus Heine hims...@claus-justus-heine.de > > http://www.claus-justus-heine.de/ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iQIcBAEBCAAGBQJVzjbEAAoJEF8rbXubOwvK7sAP/iyuj+1CGjOBt1VMUAf1Tr88 > aM3/a1VevcwjLQ/KjyrExMYlzYI5tOU9Dx+JcB2pJzvEIcvb9vBl32hO0CUNUHvt > k+yz6D8C7nkXwZoZWDZH0Ct3yMPYfMCX/pa5B1OqrPLHaRrG6QIWNubDRlf8pKJL > e+lnQyyhfdOY1OBLH2GGQAuH4dNsTAwZYkXrvw1n0QA1mXU6cqbacZdv6mFi4wCb > O7gmzbA96agxmh9Z0zSg520IS9FPt0e2uwMzgTTKdOUX4zqD09v9/wWTGJSGjhhp > NQ3kQGylQRiPZAhnDzyeWjLy3/3nivj2F3aBcymhNZOqpVmtqgxEYgg9Ck6AokQM > dHmimEG6GR9VtxpU/MfwSAPXhNrr5IktofdyLebc+GYDmq/RlZmCoFl2WM70Prhw > 6PU39kvC2Oa44whmJIK4aAyVUr9q92T5hv3VT+dI5vRh0rBSfDW9r2ypnMMu1+n5 > GpfBU0ndOAOboqsufcyDBm5xakTthzFMS1mxFJq4dJjNdO/17naZNm3DG5O4d7hE > CDoCZ6tb+o3OipxF7AOoX0ZoxMs+lGlEOOHZx0pn3YW1oiAod9E6LYm9ba84dxAK > 7NJfgqE9NGzuh8AP2GV099l8Ces32qNHI17W2n3IfMVwlt7VMXGvKidlR+vsVcwv > d1QGXv7KFo9Q+0y0+kLz > =qJ/a > -----END PGP SIGNATURE----- > > > > ------------------------------ > > Message: 4 > Date: Fri, 14 Aug 2015 20:50:56 +0200 > From: Tobias Leupold <tobias.leup...@web.de> > To: rdiff-backup-users@nongnu.org > Subject: Re: [rdiff-backup-users] State of the rdiff-backup project > Message-ID: <11802680.JxqrsxpRjb@skoni> > Content-Type: text/plain; charset="us-ascii" > >> But to clarify this, you first have to understand the existing program. > > Of course! And speaking of me, I don't ... I was just thinking about the whole > situation. > > > > ------------------------------ > > Message: 5 > Date: Fri, 14 Aug 2015 19:53:42 +0100 > From: Dominic Raferd <domi...@timedicer.co.uk> > To: rdiff-backup-users@nongnu.org > Subject: Re: [rdiff-backup-users] State of the rdiff-backup project > Message-ID: <55ce3936.3030...@timedicer.co.uk> > Content-Type: text/plain; charset=utf-8; format=flowed > > My 2p... > > Rdiff-backup has limitations and it would be *really* good if someone > who understood python could step up and maintain the code, but I don't > see the problem with regressions. Yes they are slow but they should be > emergency operations reserved for rare circumstances. If you are > frequently experiencing broken backup session then I think you should > look at why that is happening. We only use rdiff-backup within our lan > and backups always complete. I have only needed regressions when we have > inadventently backed up a lot of extraneous data, when I use them to > overcome bloating of the repository. For offsite backup (of the entire > set of repositories) we use rsync. I don't find backup speeds with > rdiff-backup particularly slow BTW, but we run the backups at a quiet > time when speed is not critical. > > v1.2.8 is the official stable version, that's why Debian uses it. v1.3.3 > is officially 'unstable' although I haven't heard of any problems with > it (and haven't used it). > > I keep repositories on ext4 - I tried btrfs but found it very slow (on > our vm) and the big wins that I sought (compression, deduplication) made > it slower still - and btrfs deduplication is still buggy. > > Dominic > http://www.timedicer.co.uk > > > > ------------------------------ > > Message: 6 > Date: Fri, 14 Aug 2015 20:55:45 +0200 > From: Claus-Justus Heine <hims...@claus-justus-heine.de> > To: rdiff-backup-users@nongnu.org > Subject: Re: [rdiff-backup-users] State of the rdiff-backup project > Message-ID: <55ce39b1.20...@claus-justus-heine.de> > Content-Type: text/plain; charset=utf-8 > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Am 14.08.2015 um 20:50 schrieb Tobias Leupold: >>> But to clarify this, you first have to understand the existing progr > am. >> >> Of course! And speaking of me, I don't ... I was just thinking about t > he whole >> situation. > > I tried to give it a round a couple of years ago, and gave up on it for > lack of time. And I do not know how things will be going on this time. > We will see. > > Cheers, > > Claus > >> >> _______________________________________________ >> rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org >> https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users >> Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBac > kupWiki >> > > > - -- > Claus-Justus Heine hims...@claus-justus-heine.de > > http://www.claus-justus-heine.de/ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iQIcBAEBCAAGBQJVzjmuAAoJEF8rbXubOwvKhoEP/21Or6fBcSaSesR8jJn/l+A0 > F1wUOu74ogcy2WzXDP1sBv2cSu3gevVeYBbQJOGrOafeBk7oXbkagGH/u6JKjCGy > 6EhMzjjwLaehniU0hdlQB7XsjxMFE7Js8bJlnSMDgHHWvMpgsadOtPrl+qJ189nY > b181rTSdRmHSpFYxjCgjf5jcckwS7M9XkLpmz7/BJRzXh6pp43iIdJH8STd/4+1b > w80ApGs7eYbbi+Fh87y/dNNnVN4LpvZjZSB6qpPQR5f6TQ93EHoJRu3t4cHphzqZ > RN5iwK3jOygb6noCUiu3D0dRtCGccgP5JXZNrBLkyBDBzkdWcF/6Bd1XKwq1PmEH > 9bjZld2HgdIA/3eUQUodJ1X2Hl27EZfSy8xrARbdLhJkK5nf1L7IemkUfYkCWTq3 > Dl7sQ5X56KlyAgIDTxcPoh4vx7U7/IUXI1BILF/f0/vU3wgNnrImihBns380Udye > sm1TZsqKiQ94HViU1rW5VMx3ITk3dzAZiZ+Osml+r0LGzLMslCR8YFDTw6vZsA25 > 3oh/9pp29ZXNoRAgVWyCc8roCg/P2ko72DYf78qenGrWoU0YpMsxmA6mUKIcKw6k > La/1XUJZBqSsH2wmX/0BPrbsgIz5QWlob6bGafzDs2l4LoP9x9fsv4OtP5emrg7Q > X5L1lxju197veLHUQuyM > =n9Sq > -----END PGP SIGNATURE----- > > > > ------------------------------ > > Message: 7 > Date: Fri, 14 Aug 2015 21:00:54 +0200 > From: Claus-Justus Heine <hims...@claus-justus-heine.de> > To: rdiff-backup-users@nongnu.org > Subject: Re: [rdiff-backup-users] State of the rdiff-backup project > Message-ID: <55ce3ae6.5080...@claus-justus-heine.de> > Content-Type: text/plain; charset=utf-8 > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Am 14.08.2015 um 20:53 schrieb Dominic Raferd: >> My 2p... >> >> Rdiff-backup has limitations and it would be *really* good if someone >> who understood python could step up and maintain the code, but I don't >> see the problem with regressions. Yes they are slow but they should be >> emergency operations reserved for rare circumstances. If you are >> frequently experiencing broken backup session then I think you should >> look at why that is happening. We only use rdiff-backup within our lan >> and backups always complete. I have only needed regressions when we ha > ve >> inadventently backed up a lot of extraneous data, when I use them to >> overcome bloating of the repository. For offsite backup (of the entire >> set of repositories) we use rsync. I don't find backup speeds with >> rdiff-backup particularly slow BTW, but we run the backups at a quiet >> time when speed is not critical. > > Well, I'am actually in progress of figuring things out on my side. > However, I experience slowness at times. Also: even if regressions are > extra-ordinary events, it still does not feel right that they take so > long. At least I want to try to understand what is going on there. > >> >> v1.2.8 is the official stable version, that's why Debian uses it. v1.3 > .3 >> is officially 'unstable' although I haven't heard of any problems with >> it (and haven't used it). > > Silly me. You are right. Maybe I should volunteer as maintainer and as > first act copy v1.3.3 tp v1.4.0 and declare it stable ;) > >> I keep repositories on ext4 - I tried btrfs but found it very slow (on >> our vm) and the big wins that I sought (compression, deduplication) ma > de >> it slower still - and btrfs deduplication is still buggy. > > For my "slowness" experience this may very well be the problem, at the > moment I'm running btrfs. However, as I'm right now replacing the backup > hardware this would be the time to change something. We will see. > > Many thanks for your feed-back, > > Claus > > > - -- > Claus-Justus Heine hims...@claus-justus-heine.de > > http://www.claus-justus-heine.de/ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iQIcBAEBCAAGBQJVzjrkAAoJEF8rbXubOwvKlZIQALXxUAme+GPIMxxMGUThpMsg > P2EjSTCZeJXRWMHg3obGLWFg7eVf3L2NNuPd/i94rEyHTVQFNpXYC5+aTH3I9eKM > 8GnapWqlZWF2ldfw4GLe8RGzm45e+WkeVF3MkiwIaMTQJ1jtOIF/kEtmeNgMWpeo > Mt8H/J3Rzk3Iz+zRbI+nUKRLq0ZQ9LDpiL+ob+IdnLSHSKT6LKiQz4o6VKHfimf6 > EcfuaecD6J6HRoaPOTcepk73r3IgjFHMx5yYdGveav7TbloBiNJqycaxS3XXfNLf > I3QBJt0AqXJ4dnXSv77XrpnyysMCpDcLFQEwncU5q7iOBEB9gpeOWt5TGpteawXP > 4W+Mw5hqnOK0f37EwwdKEVU6KUU87l/jOkMpO3uPYEjdX9ukykeLqwGq3gZakSry > 0tGrzOvUuzXouIFW0bFNyBjQv1WBGHupzEm/R3sShoLe1Yn7G2viUt8dgLioNvAq > IvoD80q18bRElzWHfY1iHgno3lIkqIw4XKeEMw7SLydW74warYjBimVxnJ7Mf7jp > uafhW1hrqUVkKspWEmfLJ64wrEEJsecfTV7MGaJsybZ8zTg2OhS39H08gqPXQcc3 > DxWmzyK+ZMSx3fV3puXT7IBQdBq280Sd/DxnLDmf0P05PM8A2zPheuvr/h/CpZma > 88qGbv58T6MN2rNkKwKA > =0HCf > -----END PGP SIGNATURE----- > > > > ------------------------------ > > _______________________________________________ > rdiff-backup-users mailing list > rdiff-backup-users@nongnu.org > https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users > > > End of rdiff-backup-users Digest, Vol 153, Issue 2 > ************************************************** _______________________________________________ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
