[rdiff-backup-users] Python 3 migration: considering non-UTF-8 conform filenames

[Eric L.]

Hi,

as I worked on migrating to Python 3, one of the "fanciest" aspects was
the change from str/unicode to bytes/str "character chains" types.

Without going into the technical details (python savvy persons will know
what I mean), it means among other things that the codeset of file names
becomes relevant and must be UTF-8. Files with a name which isn't
compliant with UTF-8 aren't backed up.

The warnings look something like:

Sat Aug  3 10:51:51 2019  Warning: unable to read ACL from 'very
complicated filename': 'utf-8' codec can't encode character '\udcb1' in
position 54: surrogates not allowed
Sat Aug  3 10:51:51 2019  Warning: ignoring file 'very complicated
filename' with wrong encoding: 'utf-8' codec can't encode character
'\udcb1' in position 54: surrogates not allowed

I don't see much options because only str (i.e. codeset-aware) can be
matched against regex, bytes can't (filenames could still be read as bytes).

Few consequences:

1. such files can't get backed-up anymore.
2. old backup repos which contain such files are seen as broken - as
long as the last version doesn't contain such files, only in increments,
it'll be usable though.

This said, non-UTF-8-compatible file systems are uncommon since many
years, so that the impact should be very limited (in my case, old
Windows files lying around since 2010).

I'm mostly concerned about the Asian room, because I've heard (but have
no experience whatsoever) that they might use other rich encodings than
Unicode. The original code was IMHO already not very clean in this
regard, the migration to UTF-8 hasn't improved things, strings are
encoded/decoded sometimes explicitly with UTF-8 sometimes without
explicit UTF-8 encoding.

If the users on this list could comment on their experience and
expectations it would be great. Doing tests with old backup repos on my
PR [1] would be even greater.

Don't expect miracles though, currently I don't see any viable
alternative to the decision I've taken. I mostly wanted to make sure
it's taken transparently.

Thanks, Eric

[1] https://github.com/sol1/rdiff-backup/pull/40

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: [rdiff-backup-users] Python 3 migration: considering non-UTF-8 conform filenames

[Eric L.]

Hi,

On 03/08/2019 14:50, Robert Nichols wrote:
>> The way to work around this in rdiffweb at least it's to manage path as
>> bytes. That is how rdiffweb 1.2.8 is working. Path are bytes. That is
>> also
>> how most filesystem are working too. Paths are bytes and those are
>> decoded
>> to be displayed to the user.
> 
> Indeed. Paths are bytes, and there is no requirement that they be printable
> strings. The forward slash is the separator, and the individual path
> components can contain any byte except 0x00 (ASCII NUL). rdiff-backup
> should accept that.

I never challenged that, I was just highlighting the difficulties.

OK, I've created a sub-branch ericzolf-py2to3-bytes and will work on
this aspect, moving away from str for paths to bytes.

KR, Eric

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: [rdiff-backup-users] Future of rdiff-backup: Python 3 migration and project maintainership in general

[Eric L.]

On 04/08/2019 14:46, rhkra...@gmail.com wrote:
> I added the  below --was it intended that there be some content in 
> this message?  If so, I don't see it.
> 
> (The reason I bring this up is that, over on the Debian user list, someone 
> was 
> sort of constructing there own emails (iiuc), and unintentionally inserting a 
> null (0x00) in the content.  My email client (an older version of kmail) 
> truncates the content on encountering a null).)

Dave's e-mail was also empty for me... :-)

KR, Eric

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: [rdiff-backup-users] Future of rdiff-backup: Python 3 migration and project maintainership in general

[Eric L.]

Hello again (in the morning for me),

in more length and with a fresh mind, and after having gone through all
thread answers, let me give a lengthy position:

0. I'm the EricZolf referenced elsewhere, who has a branch finished for
Linux with the migration to Python 3. I'll post a note after this e-mail
into the PR 40 to prove it.

1. it's great to see that there is still a community of users, I didn't
realise, else I'd have communicated earlier. I'm now on the mailing list
so all is good.

2. I started the migration effort because I didn't want to lose my
backup tool once Python 2 is out of support, else I'm an IT guy with
quite a lot of Ansible background (Python!), one wife, 2 children, a
consulting job and little time, but making the best out of it.

3. Initially, I didn't want to create my own definitive fork but wanted
to give sol1 a chance to become active and take their job as maintainer
seriously. As Otto noticed, I wasn't very successful till now. I would
have given them the Summer to react and then I'd have gone my own way,
without a clear idea how to create a community.

4. Knowing now that there is still such a community alive (thanks to
Otto!), I'd suggest following approach:

a. I'll ping a last time sol1 and ask for their position.
b. In the meantime, review my PR, it's huge, no chance to merge anything
else before it's merged back into master.
c. I merge back into my master based on your feedback.
d. A last task is required before others can start and I would ask your
patience a last time: I want the whole code to be PEP8 conform before
others contribute to it, and I think (but open to discussion) that it's
best done if one person does it in one go.
e. Once this is done, I would second Patrick's suggestion and create an
rdiff-backup project, open it to the community and push my repository to
there for further common work (I wouldn't like to lose my repository
because I have 30+ issues I've created as I went through the code).

A few more side notes:

A. my PR isn't tested against Windows and Mac, feel free to test and
push fix PR against my branch on my repo and I'll merge (it should work,
never tried, else I'll merge manually). Please focus on regression bugs
that we get quickly this huge branch merged.
B. I'm fully with Patrick regarding CI/CD, if you know tox, you'll see
that I have a good start and one of my next moves would probably have
been to integrate tox with GitHub's pipeline.
C. This and anything else like web page, a mailing list we own, release
process, and pending issues, we can discuss together once we've agreed
on the big plan.

Let the discussion roll, happy to be here, happy to hear there are
others who care about rdiff-backup, thanks to Otto for kicking this!
Eric

On 27/07/2019 01:17, Eric L. wrote:
> Hi,
> 
> I've just finished the migration of rdiff-backup to Python 3 after months of 
> work, improving at the same time the test framework. Anybody can check and 
> feedback at https://github.com/sol1/rdiff-backup/pull/40 without paying money
> 
> The quality seems equal to the version 1.2.8 packaged under Fedora, Windows 
> and Mac support wasn't a priority though. 
> 
> Feel free to save the Debian package, there is enough work for more people, 
> but we should avoid useless work and forks.
> 
> KR, Eric(Zolf)
> 
> On July 26, 2019 4:36:24 PM UTC, "Otto Kekäläinen"  wrote:
>> Hello!
>>
>> There has not been any new releases of rdiff-backup since 2009. If the
>> original maintainer does not intend to work on this project, could I
>> please be allowed to take over?
>>
>> I am a Debian Developer and active in multiple open source projects.
>> Our company supports many open source projects (seravo.com/opensource)
>> and since we also use rdiff-backup, I could get some funding and man
>> power to for example complete the Python3 migration. I know Python
>> well and have recently contributed Python code to AppArmor upstream,
>> so I think I am technically competent. With 20 years of open source
>> experience I believe I can be a good steward this project.
>>
>> Rdiff-backup is marked for autoremoval from Debian on August 8th. I
>> hope we could get some responses and activity on this soon so I have a
>> chance to save rdiff-backup in Debian.
>> https://tracker.debian.org/pkg/rdiff-backup
>>
>> That do you think?
>>
>> If you are in favor of this please let me know by starring
>> https://github.com/Seravo/rdiff-backup
>>
>> If I get more than 5 stars I will begin the Python 3 migration and
>> also pulling in the best commits from the existing forks that have had
>> most activity:
>> - https://github.com/ericzolf/rdiff-backup
>> - https://github.com/ardovm/rdiff-backup
>> - https://github.com/hosting90/rdiff-backup
>> - https://github.

Re: [rdiff-backup-users] Future of rdiff-backup: Python 3 migration and project maintainership in general

[Eric L.]

Hi Graeme, me (?), others,

On 27/07/2019 10:52, Graeme Robinson wrote:
> I echo the goodwill from users of rdiff-backup like myself.  I would
> be lost without it!  Its still the primary backup tool for 90% of my
> servers.
> I too would be happy to help with testing.

I've just added a few notes to get you started testing
https://github.com/sol1/rdiff-backup/pull/40#issuecomment-515669441

Feedback please directly as comment to the PR, it's easier to track.

Thanks, Eric

> 
> On Sat, Jul 27, 2019 at 5:35 PM me via rdiff-backup-users
>  wrote:
>>
>>
>>
>> Thanks all good faith to give this nice project another life !
>>
>> Sorry not to be able to help coding, could only test if needed.
>>
>> Greetings from South Pacific !

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: [rdiff-backup-users] Future of rdiff-backup: Python 3 migration and project maintainership in general

[Eric L.]

Hi,

I've just finished the migration of rdiff-backup to Python 3 after months of 
work, improving at the same time the test framework. Anybody can check and 
feedback at https://github.com/sol1/rdiff-backup/pull/40 without paying money

The quality seems equal to the version 1.2.8 packaged under Fedora, Windows and 
Mac support wasn't a priority though. 

Feel free to save the Debian package, there is enough work for more people, but 
we should avoid useless work and forks.

KR, Eric(Zolf)

On July 26, 2019 4:36:24 PM UTC, "Otto Kekäläinen"  wrote:
>Hello!
>
>There has not been any new releases of rdiff-backup since 2009. If the
>original maintainer does not intend to work on this project, could I
>please be allowed to take over?
>
>I am a Debian Developer and active in multiple open source projects.
>Our company supports many open source projects (seravo.com/opensource)
>and since we also use rdiff-backup, I could get some funding and man
>power to for example complete the Python3 migration. I know Python
>well and have recently contributed Python code to AppArmor upstream,
>so I think I am technically competent. With 20 years of open source
>experience I believe I can be a good steward this project.
>
>Rdiff-backup is marked for autoremoval from Debian on August 8th. I
>hope we could get some responses and activity on this soon so I have a
>chance to save rdiff-backup in Debian.
>https://tracker.debian.org/pkg/rdiff-backup
>
>That do you think?
>
>If you are in favor of this please let me know by starring
>https://github.com/Seravo/rdiff-backup
>
>If I get more than 5 stars I will begin the Python 3 migration and
>also pulling in the best commits from the existing forks that have had
>most activity:
>- https://github.com/ericzolf/rdiff-backup
>- https://github.com/ardovm/rdiff-backup
>- https://github.com/hosting90/rdiff-backup
>- https://github.com/orangenschalen/rdiff-backup
>(see https://github.com/sol1/rdiff-backup/network)
>
>
>- Otto

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: [rdiff-backup-users] Python 3 migration: considering non-UTF-8 conform filenames

[Eric L.]

Hi,

On 04/08/2019 08:59, Eric L. wrote:
> OK, I've created a sub-branch ericzolf-py2to3-bytes and will work on
> this aspect, moving away from str for paths to bytes.

If someone is interested, there is a working version at

https://github.com/ericzolf/rdiff-backup/tree/ericzolf-py2to3-bytes

You can backup, also incrementally, not sure about recovering, and none
of the tests is working. The repository looks good, rdiff-backup 1.2.8
accepts it without moaning.

So, all in all, we're back on track and unicode isn't a pre-requisite
anymore. Time to have dinner and go to bed...

You might want to decide if you prefer to review the sub-branch before I
merge it into the main branch ericzolf-py2to3, or if it's preferable to
merge it into it before general review. Both is possible, I don't really
care.

KR, Eric

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: Pre-release v1.9.1b0 - 3rd beta release before 2.0.0

[Eric L.]

Hi again,

one last word: I'd like to shoot for a release in two weeks, meaning 8th
of March, unless there is a (very) good reason to slip it further.
Hope this fits everybody's timeline.

KR, Eric

On 23/02/2020 10:33, Eric L. wrote:
> Hi,
> 
> sorry, I forgot to say but the version is also available from pypi.org:
> 
> https://pypi.org/project/rdiff-backup/1.9.1b0/
> resp. `pip install rdiff-backup==1.9.1b0`
> 
> We expect this to be the last beta version before final release, hence
> high noon for testers!
> 
> Happy testing, Eric
> 
> On 23/02/2020 10:30, Eric L. wrote:
>> Hi,
>>
>> 3rd beta release before 2.0.0
>> <https://github.com/rdiff-backup/rdiff-backup/releases/tag/v1.9.1b0>
>>
>> Repository: rdiff-backup/rdiff-backup
>> <https://github.com/rdiff-backup/rdiff-backup> · Tag: v1.9.1b0
>> <https://github.com/rdiff-backup/rdiff-backup/tree/v1.9.1b0> · Commit:
>> 3f2b56a
>> <https://github.com/rdiff-backup/rdiff-backup/commit/3f2b56af77819bde54c04edc4ced11a27053569f>
>> · Released by: rdiff-backup-admin <https://github.com/rdiff-backup-admin>
>>
>> The details of what has changed is documented in the change log
>>  but we've roughly
>> fixed a few more regressions around Windows and hard-links, improved the
>> handling of sparse files, and improved and documented
>>  our release
>> process.
>>
>> The installation instructions are available in the read me file
>>  but don't hesitate
>> to open an issue
>> <https://github.com/rdiff-backup/rdiff-backup/issues/new> if you need
>> more help or if you find issues while testing.
>>
>> Happy testing hence!
>>
>> Eric
>>
>> —
>>
>> This release has 25 assets:
>>
>>   * rdiff-backup-1.9.1b0.tar.gz
>>   * rdiff-backup-1.9.1b0.win32exe.zip
>>   * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux1_i686.whl
>>   * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux1_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux2010_i686.whl
>>   * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux2010_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux2014_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux1_i686.whl
>>   * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux1_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux2010_i686.whl
>>   * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux2010_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux2014_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux1_i686.whl
>>   * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux1_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux2010_i686.whl
>>   * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux2010_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux2014_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp37-cp37m-win32.whl
>>   * rdiff_backup-1.9.1b0-cp38-cp38-manylinux1_i686.whl
>>   * rdiff_backup-1.9.1b0-cp38-cp38-manylinux1_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp38-cp38-manylinux2010_i686.whl
>>   * rdiff_backup-1.9.1b0-cp38-cp38-manylinux2010_x86_64.whl
>>   * rdiff_backup-1.9.1b0-cp38-cp38-manylinux2014_x86_64.whl
>>   * Source code (zip)
>>   * Source code (tar.gz)
>>
>> Visit the release page
>> <https://github.com/rdiff-backup/rdiff-backup/releases/tag/v1.9.1b0> to
>> download them.
>>
> 

-- 
please do _not_ put me in copy on rdiff-backup-users,
it just confuses my filters. Thanks.

Re: Backward compatibility of next beta

[Eric L.]

Hi,

On 04/02/2020 14:58, Patrik Dufresne wrote:
> I'm definitely looking toward a similar solution where it's seamless.
> The more I'm thinking of it, we may need to change the code in
> rdiff-backup to help us. What would really help is having rdiff-backup
> calling a different executable in the remote schema. Instead of calling
> "rdiff-backup" it should call rdiff-backup2.0.0. And rdiff-backup could
> be a symbolic link to the default version, either 1.2.8 or 2.0.0. That
> would really simplify the migration process for everyone.
> 
> 
> @EricZolf  Do you think it's feasible ?
> It would mitigate all the migration issue and API incompatibilities. We
> could also call "rdiff-backup2" (with only the major version) and bump
> the major version only when an API incompatibility get introduced in the
> code.

The issue is that wheels don't support links, at least not by default
(it seems it's possible to fiddle them in, but I haven't tried and it
sounds again ugly). There also isn't anything you can't configure /
tweak yourself as part of the installation you anyway need to do (as
1.2/1.3 and 1.9/2.0 use different versions of Python, only the binary
itself need attention, the libraries are anyway in different places) and
using --remote-schema.

Also, so late in the release cycle, I'm reluctant to change even more
things.

Hope you can live with it,
Eric

rdiff-backup pre-release v1.9.0b0 - Second beta before 2.0.0

[Eric L.]

Hello everybody,

We've fixed a few bugs under Windows, worked a lot on the documentation
and our build process, offering new formats to install. Please try it
and give us feedback on the mailing list.

The README should explain how to install which format, if not, let us know.

Enjoy, Eric

—

This release has 32 assets:

  * rdiff-backup-1.9.0b0.tar.gz
  * rdiff-backup-1.9.0b0.win32.zip
  * rdiff-backup-1.9.0b0.win32exe.zip
  * rdiff-backup-dbgsym_1.4.0b1_amd64.ddeb
  * rdiff-backup_1.4.0b1.dsc
  * rdiff-backup_1.4.0b1.tar.xz
  * rdiff-backup_1.4.0b1_amd64.build
  * rdiff-backup_1.4.0b1_amd64.buildinfo
  * rdiff-backup_1.4.0b1_amd64.changes
  * rdiff-backup_1.4.0b1_amd64.deb
  * rdiff_backup-1.9.0b0-cp35-cp35m-manylinux1_i686.whl
  * rdiff_backup-1.9.0b0-cp35-cp35m-manylinux1_x86_64.whl
  * rdiff_backup-1.9.0b0-cp35-cp35m-manylinux2010_i686.whl
  * rdiff_backup-1.9.0b0-cp35-cp35m-manylinux2010_x86_64.whl
  * rdiff_backup-1.9.0b0-cp35-cp35m-manylinux2014_x86_64.whl
  * rdiff_backup-1.9.0b0-cp36-cp36m-manylinux1_i686.whl
  * rdiff_backup-1.9.0b0-cp36-cp36m-manylinux1_x86_64.whl
  * rdiff_backup-1.9.0b0-cp36-cp36m-manylinux2010_i686.whl
  * rdiff_backup-1.9.0b0-cp36-cp36m-manylinux2010_x86_64.whl
  * rdiff_backup-1.9.0b0-cp36-cp36m-manylinux2014_x86_64.whl
  * rdiff_backup-1.9.0b0-cp37-cp37m-manylinux1_i686.whl
  * rdiff_backup-1.9.0b0-cp37-cp37m-manylinux1_x86_64.whl
  * rdiff_backup-1.9.0b0-cp37-cp37m-manylinux2010_i686.whl
  * rdiff_backup-1.9.0b0-cp37-cp37m-manylinux2010_x86_64.whl
  * rdiff_backup-1.9.0b0-cp37-cp37m-manylinux2014_x86_64.whl
  * rdiff_backup-1.9.0b0-cp38-cp38-manylinux1_i686.whl
  * rdiff_backup-1.9.0b0-cp38-cp38-manylinux1_x86_64.whl
  * rdiff_backup-1.9.0b0-cp38-cp38-manylinux2010_i686.whl
  * rdiff_backup-1.9.0b0-cp38-cp38-manylinux2010_x86_64.whl
  * rdiff_backup-1.9.0b0-cp38-cp38-manylinux2014_x86_64.whl
  * Source code (zip)
  * Source code (tar.gz)

Visit the release page
 to
download them.

Re: using --remove-older-than takes a very long time?

[Eric L.]

Hi Derek,

On 07/02/2020 14:46, Derek Atkins wrote:
> HI,
> 
> I've been running backups on my servers for a long time using
> rdiff-backup.  My backup server has been offline for about a week and I
> just put it back online yesterday.  The server does the following:
> 
> foreach host (list of machines to backup); do
>   rdiff-backup host local-storage-for-host
>   rdiff-backup --force --remove-older-than 1Y local-storage-for-host
> end
> 
> This process starts at 1am.  I noticed today (it's not 8am) that the
> remove process for the first server started at 4:09am and is still
> running.  In other words, the "clear history" process is taking longer
> than the backup did!
> 
> Is this expected behavior?

I don't think so but it probably depends on how many backups it has to
remove.

> 
> While I keep track of how long it takes to run the backup process for
> each server, I didnt separate out the backup vs cleanup stages of the
> process.  If cleanup is going to take so long, I might separate it out
> into a different stage!  Although it would be nice if cleanup could be
> sped up!
> 
> Is there any way to profile the code to see where it is spending its
> time?

A profiling as such is possible at python level but I would start with
just calling the removal option with `-v9`, it should give you/us a
first hint as the date/times for each action are logged.

KR, Eric

> 
> Thanks,
> 
> -derek
> 
> PS: I should mention this is with 1.2.8, not a more recent version.

I wouldn't expect a big difference but you never know...

Re: Last call before release of rdiff-backup 2.0.0

[Eric L.]

Hi,

as a further side note, test.pypi.org hold now the betas as well and as
wheel (ha!), but also as source. rdiff-backup is now one pip command
away (with or without --user):

pip install -i https://test.pypi.org/simple/ rdiff-backup==1.9.0b1.dev8

See https://test.pypi.org/project/rdiff-backup/1.9.0b1.dev8/ for details.

Further betas and the final release will be available from the
"production" PyPI server so that you won't need the `-i` option to install.

KR, Eric

On 08/02/2020 17:48, EricZolf wrote:
> Hi,
> 
> as I write these lines, the last MR closing the last issue marked for
> milestone 2.0.0 is being tested (it'll deploy automatically the release
> to PyPI, yeah!).
> 
> Before this can become reality, a few things need to happen:
> 
> 1. someone needs to review my 4 PRs at [1] - I can merge myself but I
> can't review myself (well, I don't want to)
> 2. @zjw - you have 2 MRs waiting for updates after review - I'd like to
> take them with this release. When are you able to address them?
> 3. @ottok - similar question for [2] - we currently have inconsistently
> versioned Debian packages.
> 4. anybody needs to test, test, test! At this stage, we'll address only
> critical bugs not already present in 1.2.8.
> 
> Thanks everybody for your support, we're almost there,
> Eric
> 
> [1] https://github.com/rdiff-backup/rdiff-backup/pulls
> [2] https://github.com/rdiff-backup/rdiff-backup/issues/252
> 

-- 
please do _not_ put me in copy on rdiff-backup-users,
it just confuses my filters. Thanks.

Pre-release v1.9.1b0 - 3rd beta release before 2.0.0

[Eric L.]

Hi,

3rd beta release before 2.0.0


Repository: rdiff-backup/rdiff-backup
 · Tag: v1.9.1b0
 · Commit:
3f2b56a

· Released by: rdiff-backup-admin 

The details of what has changed is documented in the change log
 but we've roughly
fixed a few more regressions around Windows and hard-links, improved the
handling of sparse files, and improved and documented
 our release
process.

The installation instructions are available in the read me file
 but don't hesitate
to open an issue
 if you need
more help or if you find issues while testing.

Happy testing hence!

Eric

—

This release has 25 assets:

  * rdiff-backup-1.9.1b0.tar.gz
  * rdiff-backup-1.9.1b0.win32exe.zip
  * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux1_i686.whl
  * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux1_x86_64.whl
  * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux2010_i686.whl
  * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux2010_x86_64.whl
  * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux2014_x86_64.whl
  * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux1_i686.whl
  * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux1_x86_64.whl
  * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux2010_i686.whl
  * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux2010_x86_64.whl
  * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux2014_x86_64.whl
  * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux1_i686.whl
  * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux1_x86_64.whl
  * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux2010_i686.whl
  * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux2010_x86_64.whl
  * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux2014_x86_64.whl
  * rdiff_backup-1.9.1b0-cp37-cp37m-win32.whl
  * rdiff_backup-1.9.1b0-cp38-cp38-manylinux1_i686.whl
  * rdiff_backup-1.9.1b0-cp38-cp38-manylinux1_x86_64.whl
  * rdiff_backup-1.9.1b0-cp38-cp38-manylinux2010_i686.whl
  * rdiff_backup-1.9.1b0-cp38-cp38-manylinux2010_x86_64.whl
  * rdiff_backup-1.9.1b0-cp38-cp38-manylinux2014_x86_64.whl
  * Source code (zip)
  * Source code (tar.gz)

Visit the release page
 to
download them.

Re: Pre-release v1.9.1b0 - 3rd beta release before 2.0.0

[Eric L.]

Hi,

sorry, I forgot to say but the version is also available from pypi.org:

https://pypi.org/project/rdiff-backup/1.9.1b0/
resp. `pip install rdiff-backup==1.9.1b0`

We expect this to be the last beta version before final release, hence
high noon for testers!

Happy testing, Eric

On 23/02/2020 10:30, Eric L. wrote:
> Hi,
> 
> 3rd beta release before 2.0.0
> <https://github.com/rdiff-backup/rdiff-backup/releases/tag/v1.9.1b0>
> 
> Repository: rdiff-backup/rdiff-backup
> <https://github.com/rdiff-backup/rdiff-backup> · Tag: v1.9.1b0
> <https://github.com/rdiff-backup/rdiff-backup/tree/v1.9.1b0> · Commit:
> 3f2b56a
> <https://github.com/rdiff-backup/rdiff-backup/commit/3f2b56af77819bde54c04edc4ced11a27053569f>
> · Released by: rdiff-backup-admin <https://github.com/rdiff-backup-admin>
> 
> The details of what has changed is documented in the change log
>  but we've roughly
> fixed a few more regressions around Windows and hard-links, improved the
> handling of sparse files, and improved and documented
>  our release
> process.
> 
> The installation instructions are available in the read me file
>  but don't hesitate
> to open an issue
> <https://github.com/rdiff-backup/rdiff-backup/issues/new> if you need
> more help or if you find issues while testing.
> 
> Happy testing hence!
> 
> Eric
> 
> —
> 
> This release has 25 assets:
> 
>   * rdiff-backup-1.9.1b0.tar.gz
>   * rdiff-backup-1.9.1b0.win32exe.zip
>   * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux1_i686.whl
>   * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux1_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux2010_i686.whl
>   * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux2010_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp35-cp35m-manylinux2014_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux1_i686.whl
>   * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux1_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux2010_i686.whl
>   * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux2010_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp36-cp36m-manylinux2014_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux1_i686.whl
>   * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux1_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux2010_i686.whl
>   * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux2010_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp37-cp37m-manylinux2014_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp37-cp37m-win32.whl
>   * rdiff_backup-1.9.1b0-cp38-cp38-manylinux1_i686.whl
>   * rdiff_backup-1.9.1b0-cp38-cp38-manylinux1_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp38-cp38-manylinux2010_i686.whl
>   * rdiff_backup-1.9.1b0-cp38-cp38-manylinux2010_x86_64.whl
>   * rdiff_backup-1.9.1b0-cp38-cp38-manylinux2014_x86_64.whl
>   * Source code (zip)
>   * Source code (tar.gz)
> 
> Visit the release page
> <https://github.com/rdiff-backup/rdiff-backup/releases/tag/v1.9.1b0> to
> download them.
> 

-- 
please do _not_ put me in copy on rdiff-backup-users,
it just confuses my filters. Thanks.

Re: Exception ''<' not supported between instances of 'RPath' and 'RPath''

[Eric L.]

Hi Miroslav,

the issue is known and addressed 
https://github.com/rdiff-backup/rdiff-backup/issues/322


If you are able to test the release candidate 2.0.1rc0 the issue should 
be gone.

https://github.com/rdiff-backup/rdiff-backup/releases/tag/v2.0.1rc0

KR, Eric

On 14.05.2020 10:02, Miroslav Geisselreiter wrote:

Hi,

after upgrading rdiff-backup from 1.2.8 to 2.0 on CentOS 7 I have this 
error:


Previous backup seems to have failed, regressing destination now.
Exception ''<' not supported between instances of 'RPath' and 'RPath''
raised of class '':

[...]


This is fatal because rdiff-backup has ended without any action.

I use command:
rdiff-backup --preserve-numerical-ids --print-statistics /home/ 
/backup/home/

Source is 164569M large, target is 587348M

Now I run backup to new destination /backup/home2 for some time - it
looks it will do backup without error.

Merac

Re: Issue with the man page

[Eric L.]

Hi,

difficult to say without knowing what fish is looking for, and man pages 
are not meant to do automated auto-completion.


Anyway, I can only say that the `--allow-duplicate-timestamps` option is 
new with 2.0.5 and might be the reason for the issue.
This said, I don't see anything suspicious and the man page renders 
properly.


Feel free to place a Merge Request to fix the issue though, once you've 
found the culprit.


If fish also allows to create its autocompletion script from a 
bash-autocompletion script, it would possibly be a better option.


KR, Eric

On 25.10.2020 16:10, David Croll wrote:

Hi,


after updating my Ubuntu to 20.10, I have rdiff-backup 2.0.5 now.

There is a curious issue when using the fish console. It has a powerful
autocompletion feature, both regarding file paths and command options.

We can hit

fish_update_completions

and then the files containing the command completions (suggestions like
--list-increment-size, --no-fsync) are being generated from the man 
pages.


But then, fish hits a snag. The completion file generated that way is
truncated:

david@nitro ~/.l/s/f/generated_completions> cat rdiff-backup.fish
# rdiff-backup
# Autogenerated from man page /usr/share/man/man1/rdiff-backup.1.gz
complete -c rdiff-backup -l allow-duplicate-timestamps --description
'This option is only to be used if you encounter the issue of metadata
mirrors…'

Did man page change in any way that fish_update_completions cannot
parse? Other man pages are parsed correctly, and fish generates a file
with all the options.



With the best regards,


David

Weekly release of rdiff-backup

[Eric L.]

Hi,

I just wanted to announce that we have now weekly releases.

They are available always under the same tag at 
https://github.com/rdiff-backup/rdiff-backup/releases/tag/weekly


And they are (re-)generated each night from Sunday to Monday at 22:22 
UTC.


They are not meant for production use. You can use them for testing and 
reporting bugs (that would be nice) and/or verify that recent evolutions 
fulfill your expectations.


KR, Eric

Re: Backups not working after running out of space

[Eric L.]

Hi,

too late for Peter's issue, but I created an example on how to avoid the 
issue (or at least reduce the risk):

https://github.com/rdiff-backup/rdiff-backup/pull/947

Review welcome.

BTW, check also 
https://rdiff-backup.net/FAQ.html#how_to_handle_a_no_space_left_on_device_message_while_doing_a_backup


KR, Eric

On 15.11.2023 20:52, qx6uwum...@liamekaens.com wrote:

Anyone have any suggestions?

If nothing else, is there some way I could remove appropriate files
from the rdiff-backup-data directory to get my backups working again?

thanks,
Peter

On 2023-11-13 13:01, Peter Canning wrote:
Here's the command I've been using, with -v 6 added, followed by the 
output it produced:
C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe -v 6 backup 
--print-statistics `

    --exclude "H:/Lightroom/Processing Catalog/Backups" `
    --exclude "H:/Lightroom/Processing Catalog/Processing 
Catalog Previews.lrdata" `
    --exclude "H:/Lightroom/Processing Catalog/Processing 
Catalog-v12 Previews.lrdata" `

    "H:/Lightroom/Processing Catalog" `
\\nas4free.local\lightroom-catalogs-backups\Lightroom-Processing-Catalog.rdiff-backup


C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe : WARNING: 
Previous backup seems to have failed, regressing destination now

At line:1 char:1
+ C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe -v 6 
backup  ...
+ 
~
    + CategoryInfo  : NotSpecified: (WARNING: 
Previo...destination now:String) [], RemoteException

    + FullyQualifiedErrorId : NativeCommandError

WARNING: Could not find mirror metadata file. Metadata will be read 
from filesystem instead
Fatal Error: No metadata for time Thu Nov  9 02:35:42 2023 
(1699526142) found, cannot regress
* Using repository 
'//nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup'

* Hardlinks disabled by default on Windows
* Unable to import module (py)xattr. Extended attributes not 
supported on filesystem at path H:/Lightroom/Processing Catalog
* Unable to import module posix1e from pylibacl package. POSIX ACLs 
not supported on filesystem at path H:/Lightroom/Processing Catalog

* -
Detected abilities for source (read only) file system:
  Access control lists Off
  Extended attributes  Off
  Windows access control lists On
  Case sensitivity Off
  Escape DOS devices   On
  Escape trailing spaces   On
  Mac OS X style resource forks    Off
  Mac OS X Finder information  Off
-
* Directories on file system at path 
//nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-data/rdiff-backup.tmp.0 
are not fsyncable. Assuming it's unnecessary.
* Unable to import module (py)xattr. Extended attributes not 
supported on filesystem at path 
//nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-data/rdiff-ba

ckup.tmp.0
* Unable to import module posix1e from pylibacl package. POSIX ACLs 
not supported on filesystem at path 
//nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-da

ta/rdiff-backup.tmp.0
* -
Detected abilities for destination (read/write) file system:
  Ownership changing   Off
  Hard linking On
  fsync() directories  Off
  Directory inc permissions    Off
  High-bit permissions On
  Symlink permissions  Off
  Extended filenames   On
  Windows reserved filenames   On
  Access control lists Off
  Extended attributes  Off
  Windows access control lists On
  Case sensitivity Off
  Escape DOS devices   On
  Escape trailing spaces   On
  Mac OS X style resource forks    Off
  Mac OS X Finder information  Off
-
* Backup: escape_dos_devices = False
* Backup: escape_trailing_spaces = False
* Enabled use_compatible_timestamps
NOTE: Symbolic links excluded by default on Windows
NOTE: Regressing to date/time Thu Nov  9 02:35:42 2023
* Cleaning up



On 2023-11-13 04:57, Patrik Dufresne patrik-at-ikus-soft.com 
|rdiff-backup-users| wrote:

Hello Peter,

I apologize for the inconvenience you've experienced in recovering 
your
backup after encountering space issues. Typically, rdiff-backup can 
handle

Re: Doubled dates in old repositories

[Eric L. Zolf]

Hi Dominic,

On 20/04/2020 08:40, Dominic Raferd wrote:
> I have checked our 108 repositories which go back a long way (I am still
> using v1.2.8). I find this issue in one repository for some 27 dates
> (mostly but not all consecutive) in January and February 2009. They are
> almost (but not quite) the oldest entries in this repo. How do you
> recommend I delete - using --remove-older-than? By the way, verification of

Correct, taking the date of the increment just after the one duplicated.

> all our repositories fails earlier than 2013 but I have left the earlier
> data in place because it might still be somewhat recoverable - I am
> wondering if I should remove it all before upgrading to v2.

How do you mean, it fails? A repository not touched since 2013 fails on
--verify, or all your repositories fail on `--verify-at-time
`, or even something else?

Nothing happened in the code between 2009 and 2019, so the date 2013 is
strange...

KR, Eric

Doubled dates in old repositories

[Eric L. Zolf]

Hi,

we've got a report for an issue [322] where we're not sure how it
happened and if it happens often. As I know that some of you have many
and long standing repos, you might want to have a look before you
upgrade to 2.0.0. Also I'd like to know if the problem is unique (manual
intervention could have caused the issue) or generalized.

Basically, there are two entries with exactly the same date and time in
the repository, this shouldn't be and this makes rdiff-backup 2.0.0
choke on it (rdiff-backup 1.2/1.3 -actually Python 2- doesn't choke but
its reaction is unpredictable, which isn't really better).

How to detect (under Linux): run `cd MY_BACKUP_REPO; ls -1
rdiff-backup-data/mirror_metadata.* | sed -e 's/^.*mirror_metadata\.//'
-e 's/\.[a-z]*.gz$//' | uniq -cd` -> if the output is NOT empty, you
have the issue.

How to fix: delete with 1.2/1.3 the older entries in your repo(s) up and
including to the dates given by the previous command.

As said, small feedback to understand the extension of the issue would
be great. I'll work around the issue to make rdiff-backup 2 more robust
in this regard.

Thanks, Eric

[322] https://github.com/rdiff-backup/rdiff-backup/issues/322

Re: Doubled dates in old repositories

[Eric L. Zolf]

Hi,

On 21/04/2020 13:46, Dominic Raferd wrote:
> Despite the failed verifications, I have successfully recovered a largish
> (>150MB) file from December 2008 in the impacted repository. As there are
> 2806 later versions of this file (with changes in this file occurring
> between most of these versions), the recovery presumably involved applying
> 2806 reverse-diffs to the current file - pretty impressive work by
> rdiff-backup! (Sorry, still on v1.2.8)

As only the metadata are impacted, I'm not too surprised but I also
can't rule out that it might go wrong.

KR, Eric

Re: rdiff-backup 2.0.0 crashed the first time I ran it

[Eric L. Zolf]

Hi Walt,

no issue, we all have our terse days ;-)

The error is known and already fixed in the repo:
https://github.com/rdiff-backup/rdiff-backup/issues/310

Next release (no due date yet) will have the fix.

KR, Eric

On 29/04/2020 21:45, Walt Mankowski wrote:
> Hi everyone,
> 
> First I'd like to apologize for the terseness of my original post. It
> was early Sunday morning when I wore up and discovered that my backup
> had failed, and as a result I wasn't at my best.
> 
> I did some more poking at the problem and here's what I discovered (I
> also posted this at the launchpad URL).
> 
> I modified get_indexpath() in log.py to catch a TypeError exception
> and return the string "TypeError" instead of aborting. It printed 4
> errors:
> 
> ListError: 'TypeError' [Errno 13] Permission denied: b'/run/user/1000/doc'
> ListError: 'TypeError' [Errno 13] Permission denied: b'/run/user/1000/gvfs'
> ListError: 'TypeError' [Errno 13] Permission denied: 
> b'/run/user/1000/keybase/kbfs'
> ListError: 'TypeError' [Errno 13] Permission denied: b'/run/user/141/gvfs'
> 
> I'm not really sure what's wrong with the first 3. Maybe they're odd
> because they're in /run? The final file does really look like it might
> be corrupted.
> 
> I ran the code in the debugger and was able to confirm that the data
> in the rpath was bytes instead of strings. I wasn't able to find where
> they were getting loaded.
> 
> At any rate it looks like the easy fix on my end is to exclude /run
> from my backups. I did that last night and rdiff-backup ran without
> errors.
> 
> Is there any other information you'd like me to provide to help track
> down what's causing the problem?
> 
> Walt
> 
> On Sun, Apr 26, 2020 at 07:33:35AM -0400, Walt Mankowski wrote:
>> Hi,
>>
>> I updated to Ubuntu 20.04 yesterday, which includes rdiff-backup
>> 2.0.0-1. It crashed when it ran last night. I posted the bug to
>> Launchpad but thought I'd crosspost it here too.
>>
>> https://bugs.launchpad.net/ubuntu/+source/rdiff-backup/+bug/1875163
>>
>> Walt
>>
> 

Pre-release v2.0.1rc0

[Eric L. Zolf]

Hello everybody,

We're planning a fix-only release and this is the first release
candidate to allow anybody to test this release, all details in the
change log . Please
focus your tests on the interdependency (with 2.0.0) and remote aspects,
which are especially difficult to test automatically.

The installation, described in details in the read me
, has the quick
version:

  * Linux, trust on your distro to provide the latest version of
rdiff-backup "soon-ish"
  o On Fedora/CentOS/RHEL, you may use Frank's COPR repo


  o On Ubuntu, you may use Otto's PPA repo


  * Windows, download the asset |rdiff-backup-2.0.1rc0.win32exe.zip|
from the tagged release

and unpack it somewhere in your |PATH|.
  * Any supported Platform, use |pip install rdiff-backup --pre| (or
download and install the proper asset from the tagged release
).

If you encounter issues, contact us on the rdiff-backup-users mailing
list 
and/or report an issue
.

Kind Regards, Eric for the team

—

This release has 25 assets:

  * rdiff-backup-2.0.1rc0.tar.gz
  * rdiff-backup-2.0.1rc0.win32exe.zip
  * rdiff_backup-2.0.1rc0-cp35-cp35m-manylinux1_i686.whl
  * rdiff_backup-2.0.1rc0-cp35-cp35m-manylinux1_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp35-cp35m-manylinux2010_i686.whl
  * rdiff_backup-2.0.1rc0-cp35-cp35m-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp35-cp35m-manylinux2014_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp36-cp36m-manylinux1_i686.whl
  * rdiff_backup-2.0.1rc0-cp36-cp36m-manylinux1_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp36-cp36m-manylinux2010_i686.whl
  * rdiff_backup-2.0.1rc0-cp36-cp36m-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp36-cp36m-manylinux2014_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp37-cp37m-manylinux1_i686.whl
  * rdiff_backup-2.0.1rc0-cp37-cp37m-manylinux1_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp37-cp37m-manylinux2010_i686.whl
  * rdiff_backup-2.0.1rc0-cp37-cp37m-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp37-cp37m-manylinux2014_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp37-cp37m-win32.whl
  * rdiff_backup-2.0.1rc0-cp38-cp38-manylinux1_i686.whl
  * rdiff_backup-2.0.1rc0-cp38-cp38-manylinux1_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp38-cp38-manylinux2010_i686.whl
  * rdiff_backup-2.0.1rc0-cp38-cp38-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.1rc0-cp38-cp38-manylinux2014_x86_64.whl
  * Source code (zip)
  * Source code (tar.gz)

Visit the release page
 to
download them.

Re: We need more Pull Requests reviewer

[Eric L. Zolf]

Hi Patrik,

be assured that my e-mail wasn't meant as any kind of complaint about
individual person, and explicitly not you. It wasn't actually a
complaint but just a statement, which you confirmed, that we are lacking
reviewers, and a call for help.

It's an open source project and each participant can only do what they
feel able to do, and they should mostly have fun doing it.

Sorry if my statement was badly worded that it could be misunderstood,
we just need more people reviewing, that the individual ones can do less
work and/or focus on what they prefer to do.

KR, Eric

On 17/08/2020 19:43, Patrik Dufresne wrote:
> Hello Eric,
> 
> I was on vacation the last two weeks and that explains why I did not
> take a look at your Merge Request.
> 
> Your work on rdiff-backup is greatly appreciated and I usually take time
> to review the PR, but I have the feeling you are moving a bit too fast
> for the rest of the developers. Talking for me here, I can't keep up
> with your changes. During the "free" time I have for rdiff-backup, I am
> reviewing your changes instead of fixing bugs. I barely have a couple of
> hours per week to work on the project and I can't afford to review the
> PR in a timely manner all the time. I understand it can be frustrating
> for you and I am open to suggestions.
> 
> For the time being, I will continue reviewing the PR when I have a chance.
> 
> 
> On Thu, Aug 6, 2020 at 1:47 AM Eric L. Zolf  <mailto:ewl%2brdiffbac...@lavar.de>> wrote:
> 
> Hi,
> 
> as the subject says, pull requests can stay for days without review, and
> I really don't like merging my own PRs without someone having looked
> over it.
> 
> Reviewing documentation doesn't require any specific knowledge even if
> being a native English speaker might be an advantage :-) Just tell the
> author if you understand what he wrote, and if the language is correct.
> 
> Reviewing code is more engaged but requires IMHO only some Python coding
> knowledge and the ability to detect bad coding style, and challenge the
> author to make sure nothing has been overlooked and everything is
> properly commented. It doesn't require deep knowledge of the code but
> can be a great way to gain this knowledge.
> 
> You only need to review the PR which are _not_ WIP and where the
> pipeline job was successful (green tick).
> 
> Thanks, Eric
> 
> 
> 
> -- 
> IKUS Software inc.
> https://www.ikus-soft.com/
> 514-971-6442
> 130 rue Doris
> St-Colomban, QC J5K 1T9
> vcs

Re: We need more Pull Requests reviewer

[Eric L. Zolf]

Hi,

On 17/08/2020 20:02, rhkra...@gmail.com wrote:
> On Monday, August 17, 2020 01:43:51 PM Patrik Dufresne wrote:
>>> as the subject says, pull requests can stay for days without review, and
>>> I really don't like merging my own PRs without someone having looked
>>> over it.
>>>
>>> Reviewing documentation doesn't require any specific knowledge even if
>>> being a native English speaker might be an advantage :-) Just tell the
>>> author if you understand what he wrote, and if the language is correct.
> 
> I might take a stab at reviewing some of the documentation if you tell me 
> where to find it, and some specifics on which parts need review.

There is no documentation PR left to review right now, but my request
was more to have a look from time to time at the pull requests
available, and give your feedback if nobody has done yet.

In short:

1. check opened PRs at
https://github.com/rdiff-backup/rdiff-backup/pulls from time to time
2. ignore the ones which are failed (red cross), on-going (orange dot),
WIP (work in progress) or draft
3. you can either check the "files changed" (last tab to the right),
this is generally the easiest, or if you want to see the changes in full
context, you can click on the (dark blue on light blue) branch _from_
which the PR commits are coming (just below the title of the PR).
4. in the "files changed" view, you can then add comments to each line,
clicking on the line's number, start a review.
5. once you're finished, under "review changes" (top right), submit your
review, approving or requesting changes.
6. the author will then react (it doesn't mean they will accept all
change requests, but they should address them all), you might need to
agree or argue, a discussion can take place. At the end the author is
responsible for their PR so their opinion is prevalent; in doubt, I
decide (always with arguments though [1]).

That's it.

Thanks, Eric

[1] https://en.wikipedia.org/wiki/Benevolent_dictator_for_life - without
the "for life" though :-) I quote: « [Eric S.] Raymond elaborates on how
the nature of open source forces the "dictatorship" to keep itself
benevolent, since a strong disagreement can lead to the forking of the
project under the rule of new leaders »

API versioning and backward compatibility discussion

[Eric L. Zolf]

Hello,

Patrik and I are discussing the new API versioning scheme we're trying
to put in place to make sure we don't introduce incompatibilities in the
client/server connection in an uncontrolled manner, like we did between
v1.x and v2.x.

If you are interested, the discussion is taking place in the PR #458,
feel free to review and comment the changes proposed, and join the
discussion.

KR, Eric

PR #458: https://github.com/rdiff-backup/rdiff-backup/pull/458

We need more Pull Requests reviewer

[Eric L. Zolf]

Hi,

as the subject says, pull requests can stay for days without review, and
I really don't like merging my own PRs without someone having looked
over it.

Reviewing documentation doesn't require any specific knowledge even if
being a native English speaker might be an advantage :-) Just tell the
author if you understand what he wrote, and if the language is correct.

Reviewing code is more engaged but requires IMHO only some Python coding
knowledge and the ability to detect bad coding style, and challenge the
author to make sure nothing has been overlooked and everything is
properly commented. It doesn't require deep knowledge of the code but
can be a great way to gain this knowledge.

You only need to review the PR which are _not_ WIP and where the
pipeline job was successful (green tick).

Thanks, Eric

Pre-release v2.0.4rc0 - Bug-fix release candidate 2.0.4rc0

[Eric L. Zolf]

Hello everybody,

as described below, here is the first step to the last bug fix release
for the 2.0.0 branch (as far as I can foresee it), all other bug fixes
require first a bit more work on the overall code, hence they'll happen
later.

The bits can also simply be installed from
https://pypi.org/project/rdiff-backup/2.0.4rc0/

KR, Eric


  Bug-fix release candidate 2.0.4rc0
  

Repository: rdiff-backup/rdiff-backup
 · Tag: v2.0.4rc0
 · Commit:
b2cacf4

· Released by: rdiff-backup-admin 

This is a pre-release for the last expected bug-fix release before we
start seriously with re-factoring the code. Please validate that nothing
is wrong before we release it, focus especially on non-Linux testing,
remote activities and cross-versions tests, as those aspects are not so
well covered by our pipeline.

—

This release has 25 assets:

  * rdiff-backup-2.0.4rc0.tar.gz
  * rdiff-backup-2.0.4rc0.win32exe.zip
  * rdiff_backup-2.0.4rc0-cp35-cp35m-manylinux1_i686.whl
  * rdiff_backup-2.0.4rc0-cp35-cp35m-manylinux1_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp35-cp35m-manylinux2010_i686.whl
  * rdiff_backup-2.0.4rc0-cp35-cp35m-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp35-cp35m-manylinux2014_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp36-cp36m-manylinux1_i686.whl
  * rdiff_backup-2.0.4rc0-cp36-cp36m-manylinux1_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp36-cp36m-manylinux2010_i686.whl
  * rdiff_backup-2.0.4rc0-cp36-cp36m-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp36-cp36m-manylinux2014_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp37-cp37m-manylinux1_i686.whl
  * rdiff_backup-2.0.4rc0-cp37-cp37m-manylinux1_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp37-cp37m-manylinux2010_i686.whl
  * rdiff_backup-2.0.4rc0-cp37-cp37m-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp37-cp37m-manylinux2014_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp37-cp37m-win32.whl
  * rdiff_backup-2.0.4rc0-cp38-cp38-manylinux1_i686.whl
  * rdiff_backup-2.0.4rc0-cp38-cp38-manylinux1_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp38-cp38-manylinux2010_i686.whl
  * rdiff_backup-2.0.4rc0-cp38-cp38-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.4rc0-cp38-cp38-manylinux2014_x86_64.whl
  * Source code (zip)
  * Source code (tar.gz)

Visit the release page
 to
download them.

Question around of remote description

[Eric L. Zolf]

Hi,

in the course of the review of PR #404 we came to challenge the quite
complex quoting rules allowed when you backup to/from a remote location.

Our question is if anybody is relying on those (too) complex quoting
rules to make their backup work? Else it'll allow us to radically
simplify the code.

More details: if you backup to/from a remote site, the description is of
the form `host::path` (e.g. `rdiff-backup /home
myuser@myhost::/srv/backup/homebak`), and the current quoting allows to
have the double colons (::) somewhere in the `host` or `path` part of
the remote description. And the question is if anybody knows of a
concrete use case for this possibility?

For even more gory details, the man page states the quoting rules in the
REMOTE OPERATION section.

Thanks, Eric

PS: and yes, double colons are a valid POSIX path part (`touch ::`
works), but who really needs it?

https://github.com/rdiff-backup/rdiff-backup/pull/404

Re: Question around of remote description

[Eric L. Zolf]

Hi,

once again thanks for the all the good feedback. As said, we keep the
complicated pattern matching for the description, allowing to escape colons.

Thanks, Eric

On 18/06/2020 05:20, Robert Nichols wrote:
> On 6/17/20 1:15 AM, Eric L. Zolf wrote:
>> Hi,
>>
>> in the course of the review of PR #404 we came to challenge the quite
>> complex quoting rules allowed when you backup to/from a remote location.
>>
>> Our question is if anybody is relying on those (too) complex quoting
>> rules to make their backup work? Else it'll allow us to radically
>> simplify the code.
>>
>> More details: if you backup to/from a remote site, the description is of
>> the form `host::path` (e.g. `rdiff-backup /home
>> myuser@myhost::/srv/backup/homebak`), and the current quoting allows to
>> have the double colons (::) somewhere in the `host` or `path` part of
>> the remote description. And the question is if anybody knows of a
>> concrete use case for this possibility?
> 
> On my CentOS 6 system, "locate :: | wc -l" returns 2544, all of which
> are under /usr/share/man. While it is inconceivable that I would ever
> have one of those names in a backup source or destination, it is quite
> possible that I might want to restore one of those files.
> 

Re: Discussion about file format for the future

[Eric L. Zolf]

Hi,

to close this discussion, I've created an enhancement request #399 but
don't hold your breath, it's not yet on the priority list.

KR, Eric

https://github.com/rdiff-backup/rdiff-backup/issues/399

On 09/06/2020 22:51, rhkra...@gmail.com wrote:
> On Tuesday, June 09, 2020 12:30:24 PM Robert Nichols wrote:
>> On 6/9/20 9:44 AM, rhkra...@gmail.com wrote:
>>> In the case of rdiff-back, it wouldn't surprise me that diffs (deltas)
>>> are stored as forward deltas, and, in removing old deltas, a new "base"
>>> must be created before deleting the deltas.  (My words probably aren't
>>> exactly correct, I hope they are correct enough to explain my point.)
>>
>> That's not how rdiff-backup works. The diffs are _reverse_ diffs, working
>> back from the current state. 
> 
> Ahh, ok, thanks!  (Now I know something about rdiff-back ;-)
> 
>> Any metadata or increments file with a date
>> stamp older than the cutoff date can be simply deleted. I suspect that the
>> problem is the time spent parsing all of the file names in the _huge_
>> increments directory tree and comparing the date code.
>>
>> Speaking the the huge increments tree, a pet peeve of mine is that changes
>> that do not affect file content cause a "zero diff" file to be generated.
>> These can be changes to permissions and ownership, or even just a change
>> to the hard link count. The files are typically compressed empty files,
>> and are not empty themselves (there is at least a gzip header). I run an
>> audit to get rid of most of these at the end of each day, after all
>> clients have completed their backups. The first time I ran it, the count
>> was well into the millions.
> 

Re: Obsolescence Python 3.5 support plan

[Eric L. Zolf]

Hi Derek,

On 09/06/2020 16:24, Derek Atkins wrote:
> EricZolf  writes:
> 
>> Actually, while writing this e-mail, I checked the code and noticed that
>> it's enforcing pickle version 1 so the issue isn't the pickle protocol,
>> the issue is solely the bytes vs. str vs. unicode change between python
>> 2 and 3. There is not much we could have done about it IMHO.
> 
> I am not a python guru, but I am a long-time programmer in multiple
> languages and lived through the ISO-8859 -> UTF-8 changes a couple
> decades ago and remember all the hoops we had to jump through to keep
> compatibility.  Could you explain what the "bytes vs str vs unicode"
> issue is/was and why there is (was?) no way to perform a conversion?

Nothing is impossible, the question is rather how much effort you put
into it.

You can find zillions of explanations on the web about what happened
between Python 2 and Python 3 (py2 and py3 for short), but my view in short:

- despite the same name, str in py2 is not str in py3 but rather bytes
(but not quite, sometimes they're the same conceptually, but still
different objects)
- unicode in py2 disappeared in py3 and became a sub-case of str in py3
- so unicode + str in py2 became str + bytes in py3 with an imperfect
overlap, and no way to absolutely decide how to convert.
- i.e. the context needs to be considered to decide if a py2-str needs
to become a py3-str or a py3-bytes, this would need analyzing the code
line by line and deciding based on the context. Effort goes through the
rough!
- to make things even more complicated, as they are not the same objects
even if they are called the same, pickle interprets them differently:

* Python 3:
>>> pickle.dumps('xxx',1)
b'X\x03\x00\x00\x00xxxq\x00.'
>>> pickle.dumps(b'xxx',1)
b'c_codecs\nencode\nq\x00(X\x03\x00\x00\x00xxxq\x01X\x06\x00\x00\x00latin1q\x02tq\x03Rq\x04.'
>>> pickle.dumps(u'xxx',1)
b'X\x03\x00\x00\x00xxxq\x00.'

* Python 2:
>>> pickle.dumps('xxx',1)
'U\x03xxxq\x00.'
>>> pickle.dumps(b'xxx',1)
'U\x03xxxq\x00.'
>>> pickle.dumps(u'xxx',1)
'X\x03\x00\x00\x00xxxq\x00.'

And then there was the requirement to also support files with broken
encoding, which came up during the migration, and the misery was
complete, as this forced usage of bytes, completely unknown to py2...

Hope this helps understand the complexity of what you are/were asking.

KR, Eric

> 
> -derek
> 

Re: Discussion about file format for the future

[Eric L. Zolf]

Hi,

that's an interesting idea, I was more in the optic of keeping a simple
file based structure but if there are requirements, we can think about
alternatives, it could be a SQL or even a noSQL DB, or a keystore.

Regarding the terabytes of data, I take the point, I never foresaw to
convert the data itself, only the metadata, whereas I haven't yet a
clear view on a strategy forward: either a converting tool to "upgrade"
a repo, or convert transparently within rdiff-backup itself, or keep old
data and only use the new format for new metadata (I don't really like
this last one, but if it's the only solution...).

scanning the metadata files is a nightmare -> from which point of view?
Performance and/or complexity of code?

KR, Eric

On 04/06/2020 18:43, Patrik Dufresne wrote:
> Hi Eric,
> 
> My priority in that regard would be to make sure all of this backward 
> compatible with the previous repository. I mean, I have terabytes of data 
> and I don't foresee a way to convert all these metadata files to a new 
> format. And I'm probably not alone with this situation.
> We should probably restructure de code to put the read and write of 
> metadata into a single library. Last time I checked, all those metadata are 
> not that different. Then we could make this metadata read write library 
> evolve and allow us to read or write the old format or the new format.
> 
> If we stick with a file base approach yaml is good for me.
> 
> But two cent on the subject is, should we really keep this filebase ? For 
> rdiffweb, scanning the metadata files is a nightmare. When I just need a 
> subset of the data to be displayed to the user. I always thought a database 
> could be better fit for the job. Something like a key store or similar.
> 
> 
> On Thu, Jun 4, 2020 at 9:52 AM  wrote:
> 
>> Thanks!
>>
>> On Thursday, June 04, 2020 08:37:24 AM EricZolf wrote:
>>> Correct, configuration files and "metadata" files, but also potentially
>>> to exchange information between client and server (e.g. the version
>>> string could be enriched to exchange more information than just
>>> version).
>>
>>
> 

Release v2.0.5 - Last bug-fix before code clean-up

[Eric L. Zolf]

Hello everybody,

without further comments, let's party!

KR, Eric


Last bug-fix before code clean-up
  

Repository: rdiff-backup/rdiff-backup
 · Tag: v2.0.5
 · Commit:
040f122

· Released by: rdiff-backup-admin 


Main changes

In this version we've fixed 10 issues reported by users (thanks!), some
of them quite long standing. We've also improved our development and
testing infrastructure, adding test coverage and profiling instructions.
Detailed release notes are at the top of the changelog
.


Install

The installation, described in details in the read me
, has the quick version:

  * Linux, trust on your distro to provide the latest version of
rdiff-backup "soon-ish" or:
  o On Fedora/CentOS/RHEL, you may use Frank's COPR repo


  o On Ubuntu, you may use Otto's PPA repo


  * Windows, download the asset |rdiff-backup-2.0.5.win32exe.zip| from
the tagged release

and unpack it somewhere in your |PATH|.
  * Any supported Platform, and a few unsupported like BSD and MacOSX,
use |pip install rdiff-backup| (or download and install the proper
asset from the tagged release
).


Notes

It's probably the last bug-fix release of this 2.0 branch because we now
need to focus on improving/modernizing/simplifying the code, and this
will take a little while. This doesn't mean that there aren't known bugs

but they require the code to be cleaner before they can be properly
addressed. This will surely require some months so be patient or help us
to be faster.

This is also most probably the last version of rdiff-backup officially
supporting Python 3.5, which is due end-of-life in September 2020, and
also because we want to support the upcoming Python 3.9, due one month
later, and can't support more than 4 versions of Python in parallel.
In other words, if you're still using Python 3.5, it's time to think
about upgrading; your security will thank you anyway.


Help!

If you encounter issues, contact us on the rdiff-backup-users mailing
list 
and/or report an issue
.

We could also use some help, so don't hesitate to raise your hand on the
mailing list if you'd like to help your favourite backup tool!

—

This release has 25 assets:

  * rdiff-backup-2.0.5.tar.gz
  * rdiff-backup-2.0.5.win32exe.zip
  * rdiff_backup-2.0.5-cp35-cp35m-manylinux1_i686.whl
  * rdiff_backup-2.0.5-cp35-cp35m-manylinux1_x86_64.whl
  * rdiff_backup-2.0.5-cp35-cp35m-manylinux2010_i686.whl
  * rdiff_backup-2.0.5-cp35-cp35m-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.5-cp35-cp35m-manylinux2014_x86_64.whl
  * rdiff_backup-2.0.5-cp36-cp36m-manylinux1_i686.whl
  * rdiff_backup-2.0.5-cp36-cp36m-manylinux1_x86_64.whl
  * rdiff_backup-2.0.5-cp36-cp36m-manylinux2010_i686.whl
  * rdiff_backup-2.0.5-cp36-cp36m-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.5-cp36-cp36m-manylinux2014_x86_64.whl
  * rdiff_backup-2.0.5-cp37-cp37m-manylinux1_i686.whl
  * rdiff_backup-2.0.5-cp37-cp37m-manylinux1_x86_64.whl
  * rdiff_backup-2.0.5-cp37-cp37m-manylinux2010_i686.whl
  * rdiff_backup-2.0.5-cp37-cp37m-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.5-cp37-cp37m-manylinux2014_x86_64.whl
  * rdiff_backup-2.0.5-cp37-cp37m-win32.whl
  * rdiff_backup-2.0.5-cp38-cp38-manylinux1_i686.whl
  * rdiff_backup-2.0.5-cp38-cp38-manylinux1_x86_64.whl
  * rdiff_backup-2.0.5-cp38-cp38-manylinux2010_i686.whl
  * rdiff_backup-2.0.5-cp38-cp38-manylinux2010_x86_64.whl
  * rdiff_backup-2.0.5-cp38-cp38-manylinux2014_x86_64.whl
  * Source code (zip)
  * Source code (tar.gz)

Visit the release page
 to
download them.

Ansible collection to automatically install rdiff-backup

[Eric L. Zolf]

Hi,

as some of you have signaled that they have large deployments of 
rdiff-backup, an automated way to deploy rdiff-backup might be useful. 
It can also hide some of the complexity of the installation, once it's 
been fine tuned.


The collection can be grabbed from:

https://galaxy.ansible.com/rdiffbackup/rdiff_backup

And is maintained at:

https://github.com/rdiff-backup/rdiff-backup-ansible

If someone with Ansible knowledge, or willing to learn it, wants to lead 
the maintenance, I would be more than happy to support him or her.


For now, the installation works on the recent versions of the mainstream 
Linux distros, for which rdiff-backup is packaged by this team, and for 
Windows 2019/10.


Introducing some fall-back installation approaches using the package 
and/or pip modules would be the next logical step. 
Installing/Configuring an SSH client on Windows could also be an 
interesting add-on. [1]


If we don't speak until then and if it represents something to you, 
Merry Christmas and Happy New Gregorian Year.


KR, Eric

[1] https://github.com/rdiff-backup/rdiff-backup-ansible/issues

Re: no-hard-link for a simple mv wrapper

[Eric L. Zolf]

Hi,

the man page is misleading: without `--no-hard-links`, rdiff-backup does 
its best to keep hard links (i.e. stores them under same inode), but 
_with_ this option, it actually treats them as two different files, and 
stores them under different inodes, actually duplicating the disk space 
required.


The "desired effect" is what you get by default from rdiff-backup, 
unless you're working under Windows.


Based on this, I've clarified the man page in 
https://github.com/rdiff-backup/rdiff-backup/pull/483 and realized that 
we should be able to support hard links under Windows, hence issue 
https://github.com/rdiff-backup/rdiff-backup/issues/484 created.


KR, Eric

On 03/11/2020 18:47, Dan Y wrote:

I saw today the rdiff-backup command... I wanted to know:  I see in
the man page the
--no-hard-links option: "don't replicate hard links". Is the meaning
that it creates hard
links instead of to "replicate" them as separate inodes? or does it
simply exclude
new hard links from the mirror?

The reason that I ask this - I wanted to create a simple wrapper for mv that
would first cp -la to a new hard link instead of to create a new inode,
then run rdiff-backup, which would register the new hard link (or not?
that's why I ask)
and then rm the original file...

the desired effect: that rdiff-backup would register the new hard link
in the backup
instead of to create a new inode, then when the original
file is deleted, rdiff-backup would then remove the original hard link
in future backups.

Thanks!
Dan

Feedback required - new arguments parsing for rdiff-backup

[Eric L. Zolf]

Hello everybody,

first, let me wish you a happy new year, health and good luck.

I'm currently working on using argparse to improve the way command line 
arguments are parsed, in a way compatible with the old handling while 
developing a new, hopefully more logical and easier to extend, approach.


The attached script is a proof-of-concept, faking the interface and 
printing the result (without any real action), for which I'd like to get 
feedback, especially regarding your daily use of rdiff-backup:


1. is the compatible interface really similar to the old command line?
2. is the output using the old and new interfaces the same? (ignoring 
"null" default values)

3. how do you like the new interface? any feedback is welcome.

Few more notes to get you started:

`python3 arguments.py --help` gives you the old interface
`python3 arguments.py --new --help` gives you the new one
the new interface makes more clearly the difference between actions 
(backup, restore, compare, etc...) and options, e.g. the old 
`rdiff-backup -b dir1 dir2` becomes `rdiff-backup backup dir1 dir2`.
A neat trick of the new interface is the ability to create arguments 
files, e.g. restore.txt with one argument per line:


restore
--at
3B
repository_dir
target_dir

and use it on the command line with `python3 arguments.py @restore.txt`

I find that filtering away the null values helps to understand the 
output, e.g. with `| grep -v null`.


Hope you like it, let me know in both cases :-)

Thanks, Eric


arguments.py.gz
Description: application/gzip

Re: How to show errors?

[Eric L. Zolf]

Hi Brian,

normally, each repository, under rdiff-backup-data, contains a 
backup.log and dated error_log files, where you can find details, 
depending on the verbosity you've chosen.


Hope this helps,
Eric

On 02/02/2021 13:55, Brian Bouterse wrote:

I read the man pages and faqs but I didn't see an answer so I'll ask here.

How can I see more details about errors that occur when backup statistics
say `Errors 30` for example? Also how can I see error details about
previous backups also?

Thanks!
Brian

Re: too many lstat() syscalls, therefore too many IOPS

[Eric L. Zolf]

Hi,

first, I don't see anything surprising in what you describe, so all 
normal AFAICJ.


Second, rdiff-backup needs to check each source file/directory and each 
target, compare them and then copy (or not), so if you have some 2300 
files to backup, that would sound about right. If the target or the 
source file doesn't exist, it would give an error.


If the files are small or don't have changes, the lstat happen a lot and 
nothing much else; this is typical random access. It gives a much 
different access pattern than the copying of bigger files, where more

sequential is typically done to read/write the file's data.

There is no real way to improve the situation, rdiff-backup goes as fast 
as it can and I personally don't know an I/O-equivalent of "nice" (and 
if you limit the I/O, the backup will be even slower).


You could try the --no-fsync option to improve speed:

  --fsync, --no-fsync   [opt] do (or not) often sync the file system 
(_not_ doing it is faster but can be dangerous)


And, yes, the `rdiff-backup-data/increments` directory is used by 
rdiff-backup to keep track of file and directory changes.


Hope this helps,
Eric

On 12/05/2021 07:10, Andrei Enshin via Any discussion of rdiff-backup wrote:


Hi rdiff-backup folks,
  
Since recent, during backing up I can see spike in IOPS up to 500 which exhaust limit of a VM. Therefore backup process takes very long. I've straced a bit and what I can see is: many failed  lstat() syscalls:

% time seconds  usecs/call callserrors syscall
-- --- --- - - 
  42.710.040247   9  4608  1420 lstat
  35.410.033370  12  2860   getdents
   9.410.008865   6  1431   open
   4.630.004363   3  1430   close
   4.030.003797   3  1431   fstat
   3.750.003536   2  1417   getuid
   0.040.39  39 1   unlink
   0.010.13   1 9   read
-- --- --- - - 
100.000.094230 13187  1420 total
Seems rdiff-backup checks existence of some file/dir:
10:13:16 lstat("/some/path/rdiff-backup-data/increments/foo/bar", 0x7ffd832fa810) = 
-1 ENOENT (No such file or directory) <0.20>
After backup is done, there is still no such file.
Seems the part in path -  /rdiff-backup-data/increments/ - is some "config" for 
rdiff-backup and probably it tryies to find something but can't?

What might be wrong in my setup? What would you recommend to check to solve the 
issue if it is issue at all?
  
---

Best Regards,
Andrei Enshin

Re: progress bar?

[Eric L. Zolf]

Hi,

On 14/05/2021 09:29, griffin tucker wrote:

i can try drafting some pseudo-code, but by the looks of things, it'd
take some major restructuring - not impossible, though!


I never said that it's impossible, but "major restructuring" is indeed 
in progress, and "not very difficult" is something else...



i'll have a better look through the source to see if it's feasible -
another option i'd want is to compress the last backup, not just the
diffs, as well as choice of compressor (as optional features) but
these features could make an already somewhat complex script more
complex


Feel free to plug enhancement requests, but it will have to wait for the 
restructuring to be finished (and it can take a while). Choice of 
compressor is indeed something I have in mind, there are faster/better 
compressor than gzip nowadays.


KR, Eric

Re: too many lstat() syscalls, therefore too many IOPS

[Eric L. Zolf]

Hi,

first, I don't see anything surprising in what you describe, so all 
normal AFAICJ.


Second, rdiff-backup needs to check each source file/directory and each 
target, compare them and then copy (or not), so if you have some 2300 
files to backup, that would sound about right. If the target or the 
source file doesn't exist, it would give an error.


If the files are small or don't have changes, the lstat happen a lot and 
nothing much else; this is typical random access. It gives a much 
different access pattern than the copying of bigger files, where more

sequential is typically done to read/write the file's data.

There is no real way to improve the situation, rdiff-backup goes as fast 
as it can and I personally don't know an I/O-equivalent of "nice" (and 
if you limit the I/O, the backup will be even slower).


You could try the --no-fsync option to improve speed:

  --fsync, --no-fsync   [opt] do (or not) often sync the file system 
(_not_ doing it is faster but can be dangerous)


And, yes, the `rdiff-backup-data/increments` directory is used by 
rdiff-backup to keep track of file and directory changes.


Hope this helps,
Eric

On 12/05/2021 07:10, Andrei Enshin via Any discussion of rdiff-backup wrote:


Hi rdiff-backup folks,
  
Since recent, during backing up I can see spike in IOPS up to 500 which exhaust limit of a VM. Therefore backup process takes very long. I've straced a bit and what I can see is: many failed  lstat() syscalls:

% time seconds  usecs/call callserrors syscall
-- --- --- - - 
  42.710.040247   9  4608  1420 lstat
  35.410.033370  12  2860   getdents
   9.410.008865   6  1431   open
   4.630.004363   3  1430   close
   4.030.003797   3  1431   fstat
   3.750.003536   2  1417   getuid
   0.040.39  39 1   unlink
   0.010.13   1 9   read
-- --- --- - - 
100.000.094230 13187  1420 total
Seems rdiff-backup checks existence of some file/dir:
10:13:16 lstat("/some/path/rdiff-backup-data/increments/foo/bar", 0x7ffd832fa810) = 
-1 ENOENT (No such file or directory) <0.20>
After backup is done, there is still no such file.
Seems the part in path -  /rdiff-backup-data/increments/ - is some "config" for 
rdiff-backup and probably it tryies to find something but can't?

What might be wrong in my setup? What would you recommend to check to solve the 
issue if it is issue at all?
  
---

Best Regards,
Andrei Enshin

Re: too many lstat() syscalls, therefore too many IOPS

[Eric L. Zolf]

Ups, somehow I sent my e-mail twice... But at least now I know about 
ionice; never needed it, but it's even installed by default (it's part 
of util-linux, together with mount, fdisk, etc...).


I'm now browsing through the other utilities of this package, full of 
small jewels. You never stop learning...


Thanks, Eric

On 12/05/2021 23:59, griffin tucker wrote:

there is a tool called ionice that comes with the util-linux package,
which should help other read/write processes if you are multitasking.

ionice shouldn't slow down rdiff-backup if there are no other io
processes going on.

On Thu, 13 May 2021 at 03:08, Eric L. Zolf  wrote:


Hi,

first, I don't see anything surprising in what you describe, so all
normal AFAICJ.

Second, rdiff-backup needs to check each source file/directory and each
target, compare them and then copy (or not), so if you have some 2300
files to backup, that would sound about right. If the target or the
source file doesn't exist, it would give an error.

If the files are small or don't have changes, the lstat happen a lot and
nothing much else; this is typical random access. It gives a much
different access pattern than the copying of bigger files, where more
sequential is typically done to read/write the file's data.

There is no real way to improve the situation, rdiff-backup goes as fast
as it can and I personally don't know an I/O-equivalent of "nice" (and
if you limit the I/O, the backup will be even slower).

You could try the --no-fsync option to improve speed:

--fsync, --no-fsync   [opt] do (or not) often sync the file system
(_not_ doing it is faster but can be dangerous)

And, yes, the `rdiff-backup-data/increments` directory is used by
rdiff-backup to keep track of file and directory changes.

Hope this helps,
Eric

On 12/05/2021 07:10, Andrei Enshin via Any discussion of rdiff-backup wrote:


Hi rdiff-backup folks,

Since recent, during backing up I can see spike in IOPS up to 500 which exhaust 
limit of a VM. Therefore backup process takes very long. I've straced a bit and 
what I can see is: many failed  lstat() syscalls:
% time seconds  usecs/call callserrors syscall
-- --- --- - - 
   42.710.040247   9  4608  1420 lstat
   35.410.033370  12  2860   getdents
9.410.008865   6  1431   open
4.630.004363   3  1430   close
4.030.003797   3  1431   fstat
3.750.003536   2  1417   getuid
0.040.39  39 1   unlink
0.010.13   1 9   read
-- --- --- - - 
100.000.094230 13187  1420 total
Seems rdiff-backup checks existence of some file/dir:
10:13:16 lstat("/some/path/rdiff-backup-data/increments/foo/bar", 0x7ffd832fa810) = 
-1 ENOENT (No such file or directory) <0.20>
After backup is done, there is still no such file.
Seems the part in path -  /rdiff-backup-data/increments/ - is some "config" for 
rdiff-backup and probably it tryies to find something but can't?

What might be wrong in my setup? What would you recommend to check to solve the 
issue if it is issue at all?

---
Best Regards,
Andrei Enshin

Re: what to do if --check-destination-dir ends in traceback?

[Eric L. Zolf]

Hi,

the version 2.0.5 won't help here IMHO, there are two issues:

1. the "Unknown field" errors hint at the fact that the file system 
issue has corrupted the metadata of the repository. This will be 
difficult to impossible to fix, more on this later.
2. the traceback is actually due to an error in the error handling, 
still present in v2.0.5 (but fixed for the next version).


My recommendation would be to throw away the backup repository and start 
a new one, preferably on a different disk drive if you don't know why 
the file system was corrupted in the first place. You don't want to keep 
your backup on hardware you can't trust, do you?


If you want to try to fix the repository, it's a purely manual thing and 
there is a high chance to fail; it really depends how much your file 
system got corrupted, and what.


First, you should fix the 2nd issue in order to get a more meaningful 
error message:

1. edit /usr/lib/python2.7/dist-packages/rdiff_backup/rpath.py (as root)
2. go to line 121
3. replace `rpin.path` with `(rpin.index,)` (don't forget the comma)

You can then try again to fix the repository with 
--check-destination-dir -v9.
This might allow you to more easily identify which 
rdiff-backup-data/mirror_metadata.[...] file is corrupted, you might 
even be lucky and rdiff-backup finishes the regression with only warnings.
If it isn't the case, you can try to identify and patch the faulty 
mirror_metadata file by replacing the strange looking fields with 
meaningful values (or remove them altogether); the mirror_metadata will 
need to be uncompressed and re-compressed. The issue is that if it's a 
not so obvious field (e.g. the size), it'll become difficult to find the 
correct value (I would look at the mirror and assume the file hasn't 
changed since then, but it might not help).


As you see, it's not easy, and it might only be the first corruption in 
a row of other corruptions, some might not get detected immediately.
If you succeed to rollback, you should also run a --verify. But IMHO 
you'll never be 100% sure that your backup repo is fully correct.


Hence my strong personal recommendation is to forget about it and start 
a new repo, you can still keep the old one if you think you might need a 
file from it at some point in time.


Nothing is worse than a backup you aren't sure you can trust.

KR, Eric

On 06/07/2021 01:46, griffin tucker wrote:

On Tue, 6 Jul 2021 at 07:47, Gregor Zattler  wrote:


Dear rdiff-backup users and developers,

lately I had to do a fsck.ext4 -y on the filesystem which
hosts my rdiff-backup.  Then I had tracebacks when I
attempted the next backup.  What to do in such a case?

Is there any hope to revive this rdiff-backup?

This is rdiff-backup 1.2.8 on Debian 10.10 (buster):

the latest version is v2.0.5 - debian is slow with stable builds so
buster doesn't include v2.0.5 with apt-get

you could install v2.0.5 from source on debian 10 (recommended) or
otherwise install debian bullseye (NOT recommended) but either of
these options won't help you recover your data

sorry i couldn't be of more help, can anyone else help recover data?



$ sudo rdiff-backup --check-destination-dir 
/mnt/usb-backup/rdiff-backup/durable/
Unknown field in line '9e r1ons000'
Unknown field in line '9e 10a595828229'
Exception 'RORPath instance has no attribute 'path'' raised of class '':
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in 
error_check_Main
 try: Main(arglist)
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in 
Main
 take_action(rps)
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 282, in 
take_action
 elif action == "check-destination-dir": CheckDest(rps[0])
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 872, in 
CheckDest
 dest_rp.conn.regress.Regress(dest_rp)
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 71, in 
Regress
 for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/rorpiter.py", line 281, 
in __call__
 last_branch.fast_process(*args)
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/regress.py", line 274, 
in fast_process
 else: rpath.copy_with_attribs(rf.metadata_rorp, rf.mirror_rp)
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/rpath.py", line 243, in 
copy_with_attribs
 copy(rpin, rpout, compress)
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/rpath.py", line 121, in 
copy
 else: raise RPathException("File %s has unknown type" % rpin.path)

Traceback (most recent call last):
   File "/usr/bin/rdiff-backup", line 30, in 
 rdiff_backup.Main.error_check_Main(sys.argv[1:])
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in 
error_check_Main
 try: Main(arglist)
   File "/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 324, in 
Main
 take_action(rps)
   File 

Re: Two more documentation pull requests to review

[Eric L. Zolf]

Already one week passed, and not one single review...

Open Source is a bit like Democracy in this that it doesn't work if 
people don't participate. Nobody should be surprised if development 
stalls...


Thanks for your support,
Eric

On 26/02/2021 09:58, EricZolf wrote:

Hi,

as the subject says:

https://github.com/rdiff-backup/rdiff-backup/pull/536 (Clarify selection
principles)

https://github.com/rdiff-backup/rdiff-backup/pull/538 (Add side by side
comparaison of old and new CLI)

Especially the 2nd one is important to learn how the next version will
look like.

Thanks, Eric

Re: Please help the rdiff-backup project by reviewing pull requests (not a hard thing to do!)

[Eric L. Zolf]

Hi,

so, if someone wants to help and isn't yet subscribed, there is a new 
Pull Request to review:


https://github.com/rdiff-backup/rdiff-backup/pull/535

It's half (architecture) documentation and half code, with the focus of 
the clarity of the new action plugin interface.


Thanks, Eric

On 21/02/2021 22:45, EricZolf wrote:

Sorry, Brian, the PRs have been reviewed and merged in the mean time, but new 
ones will come, stay tuned!

Thanks for taking care,
Eric

On February 21, 2021 8:56:09 PM UTC, Brian Bouterse  wrote:

I went to look, but the two PRs that are out there are WIP.

On Thu, Feb 18, 2021 at 7:31 AM Eric Lavarde  wrote:


Hello everybody,

thanks, Otto, for this necessary call to action!

Let me re-enforce the message and also explain what I expect when
someone reviews one of my PRs. The good news is that you don't need

to

be afraid, I don't bite (especially not in Corona times) and I stay
friendly even if I disagree with your observations, which is very

normal.


The other good news is that you don't need to be a specialist of the
rdiff-backup code, just have some knowledge of Python, so that you

can

review the code and ask some more or less stupid questions. It's

really

fine if they are not really stupid but just genuine: if you don't
understand what I'm trying to do, it might mean that my code,

comments

or documentation aren't as clear as I think they are.

A few rules of thumb to make it clearer:

- do your homework, if it's basic Python knowledge missing, read the
docs before you comment, but I'm fine if your understanding is only
theoretical
- if my code is too complex, fair remark. Lack of comment might be

the

issue.
- be ready to see me disagree: even if I don't agree with your

proposed

changes, I'll explain (gently) why, and you'll learn something, and
it'll help me clarify my thoughts and make sure that I really took

the

right decision (or possibly not).

So, don't hesitate, it's a cross check that I don't oversee something
obvious.

Thanks, Eric

On 17/02/2021 07:41, Otto Kekäläinen wrote:

Hello!

Please go to https://github.com/rdiff-backup/rdiff-backup/pulls and
check out the 3 most recent PRs.

The project is currently somewhat stalled by the lack of PR

reviewers.

Eric has made a couple of PRs but I personally don't have time

right

now to review them. The project needs more reviewers – at minimum

you

only need to skim the changes (documentation changes, code changes)
and comment something so that Eric does not have to do everything

by

himself.

A big thanks to Eric for being the main developer and reviving
rdiff-backup and getting 2.0 released. If there isn't enough

support

from the subscribers of this mailing list and users of

rdiff-backup,

Eric might loose interest in maintaining the code.

So please step up those who can.

Thanks,

Otto

Re: Does anybody have any use of error_log?

[Eric L. Zolf]

Hi,

On 07/04/2021 14:05, Patrik Dufresne wrote:

Hello Eric,

Thanks for asking !

To give you a recent example. A new customer of mine has an error with 
it's backup and the following lines get added to error_log.


ListError 
Data/Projets2019/Photos_Maison_Halle/Photos_Maison_Halle/Espaces_au-dessus_des_portes/Poutrelle_01.jpg 
[Errno 13] Permission denied: 
'D:/Data/Projets2019/Photos_Maison_Halle/Photos_Maison_Halle/Espaces_au-dessus_des_portes/Poutrelle_01.jpg'


In rdiffweb, these errors get displayed when the file contains something.
image.png
This could certainly be improved to get more error printed into this 
file as I've discover many are not getting written in that file.


Exactly my point, the `backup.log` contains more (all) information, I 
would like to have:


- only one file with all messages
- easier to parse than today:
* one unique line for each message
* information like type (ERROR, WARNING, DEBUG, etc) and date/time available

I don't think I care too much if it's a dated (like error_log.) or 
an undated file (like backup.log), but I guess it's slightly easier to 
handle if it's dated?


What do you think?

KR, Eric

PS: no need to put me in copy, I'm on the mailing list...






On Sun, Apr 4, 2021 at 1:37 AM EricZolf > wrote:


Hi,

On 03/04/2021 14:42, Patrik Dufresne wrote:
 > Hello Éric,
 >
 > Ok using this file to verify if the backup run without error. In
 > rdiffweb and minarca, this is used to report the status of the
 > repository. When ever the file contains something, this mean
something
 > wrong happen. E.g. lately, a customer backup is complaining about
 > permissions denied on windows.
 >
 > This is a neat way to spot those kind of problem and get them fixed.

I'm wondering because from my analysis of the code, not all errors land
in the error_log file, where they all land in backup.log resp.
restore.log, so I don't think the approach is reliable.

 > So, I would say don't get rid of it.

That's why I ask :-)

KR, Eric



--
IKUS Software inc.
https://www.ikus-soft.com /
514-971-6442
130 rue Doris
St-Colomban, QC J5K 1T9

Re: win_acls changes trigger creation of increments

[Eric L. Zolf]

Hi,

On 22/04/2021 05:15, Robert Nichols wrote:

On 4/21/21 12:08 PM, Patrik Dufresne wrote:
[snip]

1. I'm not aware of any changes in the logic between 1.2.8 and 2.0.5
regarding Win_acls. So I'm expecting this to be an issue in 2.0.5. Is 
it ?

2. Would we consider this normal behavior to create increments (empty
increment) when the only changes are metadata changes ?


It's been that way since forever, and it's not unique to Windows ACLs.
Changes that affect only file metadata cause these "zero-diff" files to
be created, and since they are compressed, they are not empty files but
actually occupy a disk block. Places like /usr/src/kernels and
/var/lib/yum/yumdb, which make extensive use of hard links, cause
thousands of these files to be generated every time there is an update.
I wrote a rather messy audit to get rid of them, and the first time I
ran it resulted in over 4,000,000 files being deleted.


What Robert just wrote. We could possibly optimize the recognition that 
a file content didn't change, avoiding some data churning (and perhaps 
even empty compressed files) but that would be an enhancement.


KR, Eric

Re: Pall Requests waiting for review

[Eric L. Zolf]

Hi,

a friend of mine did review the easiest PRs, but there is still one big
PR pending review, blocking me:

https://github.com/rdiff-backup/rdiff-backup/pull/614

Thanks, Eric

On 06/09/2021 22:07, EricZolf wrote:
> Hi,
> 
> I don't want to sound like a broken record but there are 4 pull requests 
> waiting for a review since one month or more. Open Source works best if many 
> people participate a little, so it would be nice if more of our users could 
> involve themselves in the review process, instead of poor Patrik doing all of 
> it.
> 
> Thanks, Eric
> 

Lack of reviewers (again) and decision

[Eric L. Zolf]

Hi,

I don't want to sound like a broken record but I have again a pull
request [1] waiting for review and nobody takes care.

Our contribution guide states that "Ideally each pull request gets some
feedback within 24 hours from it having been filed" and we're far from
it. I'm also tired of begging for review.

For this reason, I'll stop begging and will merge my own PRs after a
little while (24h-48h, perhaps more if the PR isn't blocking me). Not
exactly development best practice, but anybody disagreeing is free to
commit to provide timely feedback to PRs, and I'll happily support them
in this endeavor.

Thanks for your attention, Eric

[1] https://github.com/rdiff-backup/rdiff-backup/pull/632

Re: cross-platform backup tool Lack of reviewers (again) and decision

[Eric L. Zolf]

Hi Jonas,

you're more than welcome, and don't hesitate to ask questions if you're
struggling with something.

I would recommend to focus on what's in `src/rdiffbackup` (new code)
rather than what's in `src/rdiff_backup` (old code), even though
currently the two are entangled.

And, of course, there is documentation, even though it's neither
complete nor perfect.

KR, Eric

On 28/11/2021 17:42, Jonas Schöpf wrote:
> Hi Eric!
> 
> First of all, thank you for all the work you put into rdiff-backup! I
> love this tool!
> 
> Last time you wrote an email about lack of reviewers, I took a look at
> the PRs, but couldn't decide anything about them as I lack knowledge of
> the implementation.
> 
> I "watch" the repo now and try to find some time to look at the code
> such that in the future I might be somehow more helpful for this project :)
> 
> Cheers,
> Jonas
> 
> 
> On 11/10/21 06:50, Eric L. Zolf wrote:
>> Hi,
>>
>> I don't want to sound like a broken record but I have again a pull
>> request [1] waiting for review and nobody takes care.
>>
>> Our contribution guide states that "Ideally each pull request gets some
>> feedback within 24 hours from it having been filed" and we're far from
>> it. I'm also tired of begging for review.
>>
>> For this reason, I'll stop begging and will merge my own PRs after a
>> little while (24h-48h, perhaps more if the PR isn't blocking me). Not
>> exactly development best practice, but anybody disagreeing is free to
>> commit to provide timely feedback to PRs, and I'll happily support them
>> in this endeavor.
>>
>> Thanks for your attention, Eric
>>
>> [1] https://github.com/rdiff-backup/rdiff-backup/pull/632
>>
> 

Re: Trouble running rdiff-backup after Windows updates

[Eric L. Zolf]

Hi Michael,

it's difficult to tell without a more concrete error message. Running
the command in debug mode (-v 9) might help to understand where the
script hangs.

What happens if you call `ssh ${BACKUP_USER}@${TARGETHOST}
rdiff-backup.bat ${BACKUPTO}` like the cron job? Or even something like
`ssh ${BACKUP_USER}@${TARGETHOST} C:\Windows\System32\OpenSSH\ssh.exe
r...@datasrv2.intra.hoec rdiff-backup --server`

But as you said that the issue appeared after an update of Windows, I
can currently only imagine that something changed with SSH. I would look
if perhaps something changed in the SSHD _and_ SSH configuration files
between 1909 and 21H1, something which makes a difference between
connecting interactively or not.

What you could also try is to call the
KR, Eric

On 01/11/2021 16:21, Michael Crider - HOEC wrote:
> We have used rdiff-backup for well over 10 years for our Linux servers
> and workstations, and until recently to back up Windows servers and
> workstations we mounted their drives locally on the backup server and
> ran rdiff-backup against the mount. When our first Windows 10
> workstations arrived with v1909, we started having trouble reliably
> mounting the drives, so we recently put the Windows executable of 2.0.5
> on our Windows workstations and tried the same method we use for Linux -
> a cron job on the backup server would ssh to the Windows workstation
> (running the built-in OpenSSH server) and call a batch file that would
> run rdiff-backup, sending the backup back to the server. Keys were in
> place to allow passwordless connections. This way we could schedule
> several workstations to run in sequence without having multiples overlap
> in time (as might happen if we did the scheduling at the workstation).
> That was very stable on workstations running Windows 10 1909 and below.
> Then we upgraded all of our workstations to 21H1. Now we can manually
> ssh to the Windows workstation and run the batch file and it runs just
> fine. But if we run the cron job script that does the ssh and runs the
> batch file, everything works (setting environment variables and updating
> specific local files from network masters using scp) until the
> rdiff-backup command. We don't have "echo off" on the batch file so we
> can see the command issued. It makes the connection to the backup server
> and starts rdiff-backup on it, but then it hangs and doesn't progress
> any further, and no files are updated in the backup destination. We
> started with rdiff-backup 2.05 using SysNative in the remote-schema,
> then moved to 2.1.0a1-64 to see if that helped (changing the
> remote-schema to System32 but leaving everything else the same). but
> nothing changed except for an added warning about the server not
> understanding the API. If we are running the cron script in a terminal
> we can press Ctrl-C and exit. If it runs from a cron schedule we have to
> kill rdiff-backup on either the workstation or the server. Has anyone
> else seen something similar?
> 
> The command in the cron job is: ssh ${BACKUP_USER}@${TARGETHOST}
> rdiff-backup.bat ${BACKUPTO}; RDIFF_RESULT=$?
> 
> The command in the batch file is:
> C:\rdiff-backup-2.1.0a1-64\rdiff-backup.exe -v 6
> --exclude-symbolic-links --remote-schema
> "C:\Windows\System32\OpenSSH\ssh.exe %%s rdiff-backup --server"
> --include "C:/FuturaData" --include "C:/Program Files (x86)/Futura
> Systems/FuturaGIS/Staking Client" --exclude
> "C:/Users/%BACKUP_USER%/AppData/Local/Temp" --exclude
> "C:/Users/%BACKUP_USER%/AppData/Local/ESRI/Local Caches" --include
> "C:/Users/%BACKUP_USER%" --exclude "C:/**" C:/
> r...@datasrv2.intra.hoec::%BACKUPTO%
> 
> Both use environment variables set on the command line of the script or
> earlier in the script.

Re: cross-platform backup tool Usage of the

[Eric L. Zolf]

Hi,

On 11/12/2021 17:33, Patrik Dufresne wrote:
> I would recommend to remove the ability to change the quoting, but would
> move the feature into a migration action that would update the quoting on
> existing repository. Similar to the deletion operation.

I like very much the idea. This would allow to add more easily other
features, up to moving a repository from one location to the next.

> But it's clearly not a priority for me anyway.

Yep, and it's also not as simple as it sounds.

As everybody has answered that they didn't know about the feature or
didn't use it, I think we have a decision.

Thanks, Eric

cross-platform backup tool Usage of the

[Eric L. Zolf]

Hi,

while refactoring the code, I stumbled upon the (theoretical)
possibility to change the quoting used in the repo with the option
--override-chars-to-quote (i.e. replace the characters not allowed on
the target filesystem of the repo with some code).

According to my analysis, the option never worked as intended and I'm
not surprised, it's quite a complex thing to do, so I would remove it,
but I'd like to make sure I don't break someones daily job.

Hence the survey under:
https://webapp.oulu.fi/framadate/studs.php?poll=4eKjK3yF7t6ZtrnQ

Answer asap please, so that I can take a decision.

To be very clear, it's not about removing the ability to quote forbidden
characters, it's about removing the possibility to change the quoting
once the repo has been created.

Thanks, Eric

PS: I might re-add the feature later on, but currently it's a bummer.

Re: BackupFriend - new desktop application that uses rdiff-backup, all open source

[Eric L. Zolf]

Hi,

On 21/07/2021 04:07, Guy Sheffer wrote:

BTW two feature requests I got were encryption and no history, only
rsync-like behavior.
I wonder if building something that handles multiple backends might be a
more general solution (as much as I like rdiff-backup).


As much as I like rdiff-backup :-) it doesn't make sense to use it if 
you don't need the delta/history feature. In this case, rsync is much 
more efficient, but it's not exactly a backup tool either.


KR, Eric