Re: [rdiff-backup-users] corrupted files in destination while source is good and no errors when running rdiff-backup

2017-03-21 Thread Patrik Dufresne 
Hello, I'm compare the files.

The files create by rdiff-backup has the right size, but it filled with
zeros. So know the question remain. Why is rdiff-backup is creating a file
with only zeros.

--
Patrik Dufresne


On Tue, Mar 21, 2017 at 9:27 AM, Jelle de Jong 
wrote:

> Beste Patrik,
>
> Thank you!
>
> http://paste.debian.net/plainh/bfbacd37
>
> [hidden]
>
> I added all the information in the above.
>
> Thank you,
>
> Kind regards,
>
> Jelle de Jong
>
>
>
> On 21/03/17 12:18, Patrik Dufresne wrote:
>
>> Can you pick two non personal file and share it with us? I would like to
>> compare them at binary level.
>>
>> Also may you provide more details about your setup?
>> Os version, bitness, Python version, rdiff-backup version, file system,
>> etc on both ends ?
>> Also what is the command line used. You are using a mount point or via
>> SSH?
>>
>> On Mar 21, 2017 5:53 AM, "Jelle de Jong" > > wrote:
>>
>> Anybody?
>>
>> I will remove the whole backup and start over again, but I would
>> love some advice on how to have fixed the current backup (I saved
>> the backups for debugging).
>>
>> Kind regards,
>>
>> Jelle de Jong
>>
>> On 17/03/17 12:54, Jelle de Jong wrote:
>>
>> root@backup:~# md5sum
>> /srv/storage/backups/libvirt-filesystems/sr7-sdb2//DSC
>> _1319.JPG
>> 8cbb1482783c05a1604987532bdc6540
>> /srv/storage/backups/libvirt-filesystems/sr7-sdb2//DSC
>> _1319.JPG
>> root@backup:~# md5sum /mnt/sr7-sdb2//DSC_1319.JPG
>> 913ee9c4832f69cc60a99dbc7b9e57c9  /mnt/sr7-sdb2//DSC_1319.
>> JPG
>>
>> root@backup:~# ls -hal
>> /srv/storage/backups/libvirt-filesystems/sr7-sdb2//DSC
>> _1319.JPG
>> -r-xr-xr-x 1 root root 2.4M Feb  5  2014
>> /srv/storage/backups/libvirt-filesystems/sr7-sdb2//DSC
>> _1319.JPG
>> root@backup:~# ls -hal /mnt/sr7-sdb2//DSC_1319.JPG
>> -r-xr-xr-x 1 root root 2.4M Feb  5  2014
>> /mnt/sr7-sdb2//DSC_1319.JPG
>>
>>
>> Kind regards,
>>
>> Jelle de Jong
>>
>>
>> On 16/03/17 19:12, Joe Steele wrote:
>>
>> This is just a guess, but if your de-duplication process
>> involves the
>> use of hard links which rdiff-backup is backing up, then
>> your problem
>> may relate to the following bug (which affects hard links):
>>
>> http://savannah.nongnu.org/bugs/?26848
>> 
>>
>> The bug can generate the type of errors shown in your verify
>> log.
>>
>> See if your version of rdiff-backup includes the patches
>> attached to the
>> bug report.  If not, apply the patches and see if that fixes
>> any of your
>> problems.
>>
>> --Joe
>>
>>
>> On 3/16/2017 9:43 AM, Jelle de Jong wrote:
>>
>> Hello everybody,
>>
>> Thank you for rdiff-backup it has been very useful for
>> me for many
>> years.
>>
>> I am making a new solution to be able to use
>> rdiff-backup with ntfs3
>> with a new dedup plugin, this seem to be working now.
>>
>> I ran rdiff-backup on the same volume, some of these
>> runs have been done
>> on currupt files while fine tuning the de-duplication
>> read plugin.
>>
>> root@backup:~# rdiff-backup --list-increments
>> /srv/storage/backups/libvirt-filesystems/sr7-sdb2/
>> Found 9 increments:
>> increments.2017-02-09T18:52:53+01:00.dir   Thu Feb
>> 9 18:52:53 2017
>> increments.2017-02-12T03:57:18+01:00.dir   Sun Feb
>> 12 03:57:18 2017
>> increments.2017-02-24T12:42:23+01:00.dir   Fri Feb
>> 24 12:42:23 2017
>> increments.2017-02-27T13:45:11+01:00.dir   Mon Feb
>> 27 13:45:11 2017
>> increments.2017-02-27T17:03:21+01:00.dir   Mon Feb
>> 27 17:03:21 2017
>> increments.2017-02-28T14:56:43+01:00.dir   Tue Feb
>> 28 14:56:43 2017
>> increments.2017-03-05T04:40:29+01:00.dir   Sun Mar
>> 5 04:40:29 2017
>> increments.2017-03-09T19:24:12+01:00.dir   Thu Mar
>> 9 19:24:12 2017
>> increments.2017-03-12T03:30:37+01:00.dir   Sun Mar
>> 12 03:30:37 2017
>> Current mirror: Tue Mar 14 19:26:57 2017
>>
>> We fixed all the errors and I can read all the documents
>> from the source
>> directory fine , but rdiff is having corrupt documents
>>

Re: [rdiff-backup-users] corrupted files in destination while source is good and no errors when running rdiff-backup

2017-03-17 Thread Jelle de Jong 
root@backup:~# md5sum 
/srv/storage/backups/libvirt-filesystems/sr7-sdb2//DSC_1319.JPG
8cbb1482783c05a1604987532bdc6540 
/srv/storage/backups/libvirt-filesystems/sr7-sdb2//DSC_1319.JPG

root@backup:~# md5sum /mnt/sr7-sdb2//DSC_1319.JPG
913ee9c4832f69cc60a99dbc7b9e57c9  /mnt/sr7-sdb2//DSC_1319.JPG

root@backup:~# ls -hal 
/srv/storage/backups/libvirt-filesystems/sr7-sdb2//DSC_1319.JPG
-r-xr-xr-x 1 root root 2.4M Feb  5  2014 
/srv/storage/backups/libvirt-filesystems/sr7-sdb2//DSC_1319.JPG

root@backup:~# ls -hal /mnt/sr7-sdb2//DSC_1319.JPG
-r-xr-xr-x 1 root root 2.4M Feb  5  2014 /mnt/sr7-sdb2//DSC_1319.JPG


Kind regards,

Jelle de Jong


On 16/03/17 19:12, Joe Steele wrote:

This is just a guess, but if your de-duplication process involves the
use of hard links which rdiff-backup is backing up, then your problem
may relate to the following bug (which affects hard links):

http://savannah.nongnu.org/bugs/?26848

The bug can generate the type of errors shown in your verify log.

See if your version of rdiff-backup includes the patches attached to the
bug report.  If not, apply the patches and see if that fixes any of your
problems.

--Joe


On 3/16/2017 9:43 AM, Jelle de Jong wrote:

Hello everybody,

Thank you for rdiff-backup it has been very useful for me for many years.

I am making a new solution to be able to use rdiff-backup with ntfs3
with a new dedup plugin, this seem to be working now.

I ran rdiff-backup on the same volume, some of these runs have been done
on currupt files while fine tuning the de-duplication read plugin.

root@backup:~# rdiff-backup --list-increments
/srv/storage/backups/libvirt-filesystems/sr7-sdb2/
Found 9 increments:
increments.2017-02-09T18:52:53+01:00.dir   Thu Feb  9 18:52:53 2017
increments.2017-02-12T03:57:18+01:00.dir   Sun Feb 12 03:57:18 2017
increments.2017-02-24T12:42:23+01:00.dir   Fri Feb 24 12:42:23 2017
increments.2017-02-27T13:45:11+01:00.dir   Mon Feb 27 13:45:11 2017
increments.2017-02-27T17:03:21+01:00.dir   Mon Feb 27 17:03:21 2017
increments.2017-02-28T14:56:43+01:00.dir   Tue Feb 28 14:56:43 2017
increments.2017-03-05T04:40:29+01:00.dir   Sun Mar  5 04:40:29 2017
increments.2017-03-09T19:24:12+01:00.dir   Thu Mar  9 19:24:12 2017
increments.2017-03-12T03:30:37+01:00.dir   Sun Mar 12 03:30:37 2017
Current mirror: Tue Mar 14 19:26:57 2017

We fixed all the errors and I can read all the documents from the source
directory fine , but rdiff is having corrupt documents in the backup
destination.

When testing I found documents with different md5sums in my backup and
the source destination.

After some investigating --verify outputs the bad documents:

http://paste.debian.net/hidden/d311cfcf/

Why is rdiff-backup running fine over the directory but still has
corrupted documents in the destination and how do I fix this without
removing the full backup.

Kind regards,

Jelle de Jong

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL:
http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki




___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: 
http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki


___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: [rdiff-backup-users] corrupted files in destination while source is good and no errors when running rdiff-backup

2017-03-17 Thread Jelle de Jong 
The files where no hard links, on the ntfs source directory they show up 
as real files.


Kind regards,

Jelle de Jong


On 16/03/17 19:12, Joe Steele wrote:

This is just a guess, but if your de-duplication process involves the
use of hard links which rdiff-backup is backing up, then your problem
may relate to the following bug (which affects hard links):

http://savannah.nongnu.org/bugs/?26848

The bug can generate the type of errors shown in your verify log.

See if your version of rdiff-backup includes the patches attached to the
bug report.  If not, apply the patches and see if that fixes any of your
problems.

--Joe


On 3/16/2017 9:43 AM, Jelle de Jong wrote:

Hello everybody,

Thank you for rdiff-backup it has been very useful for me for many years.

I am making a new solution to be able to use rdiff-backup with ntfs3
with a new dedup plugin, this seem to be working now.

I ran rdiff-backup on the same volume, some of these runs have been done
on currupt files while fine tuning the de-duplication read plugin.

root@backup:~# rdiff-backup --list-increments
/srv/storage/backups/libvirt-filesystems/sr7-sdb2/
Found 9 increments:
increments.2017-02-09T18:52:53+01:00.dir   Thu Feb  9 18:52:53 2017
increments.2017-02-12T03:57:18+01:00.dir   Sun Feb 12 03:57:18 2017
increments.2017-02-24T12:42:23+01:00.dir   Fri Feb 24 12:42:23 2017
increments.2017-02-27T13:45:11+01:00.dir   Mon Feb 27 13:45:11 2017
increments.2017-02-27T17:03:21+01:00.dir   Mon Feb 27 17:03:21 2017
increments.2017-02-28T14:56:43+01:00.dir   Tue Feb 28 14:56:43 2017
increments.2017-03-05T04:40:29+01:00.dir   Sun Mar  5 04:40:29 2017
increments.2017-03-09T19:24:12+01:00.dir   Thu Mar  9 19:24:12 2017
increments.2017-03-12T03:30:37+01:00.dir   Sun Mar 12 03:30:37 2017
Current mirror: Tue Mar 14 19:26:57 2017

We fixed all the errors and I can read all the documents from the source
directory fine , but rdiff is having corrupt documents in the backup
destination.

When testing I found documents with different md5sums in my backup and
the source destination.

After some investigating --verify outputs the bad documents:

http://paste.debian.net/hidden/d311cfcf/

Why is rdiff-backup running fine over the directory but still has
corrupted documents in the destination and how do I fix this without
removing the full backup.

Kind regards,

Jelle de Jong

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL:
http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki




___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: [rdiff-backup-users] corrupted files in destination while source is good and no errors when running rdiff-backup

2017-03-16 Thread Joe Steele 
This is just a guess, but if your de-duplication process involves the 
use of hard links which rdiff-backup is backing up, then your problem 
may relate to the following bug (which affects hard links):


http://savannah.nongnu.org/bugs/?26848

The bug can generate the type of errors shown in your verify log.

See if your version of rdiff-backup includes the patches attached to the 
bug report.  If not, apply the patches and see if that fixes any of your 
problems.


--Joe


On 3/16/2017 9:43 AM, Jelle de Jong wrote:

Hello everybody,

Thank you for rdiff-backup it has been very useful for me for many years.

I am making a new solution to be able to use rdiff-backup with ntfs3
with a new dedup plugin, this seem to be working now.

I ran rdiff-backup on the same volume, some of these runs have been done
on currupt files while fine tuning the de-duplication read plugin.

root@backup:~# rdiff-backup --list-increments
/srv/storage/backups/libvirt-filesystems/sr7-sdb2/
Found 9 increments:
increments.2017-02-09T18:52:53+01:00.dir   Thu Feb  9 18:52:53 2017
increments.2017-02-12T03:57:18+01:00.dir   Sun Feb 12 03:57:18 2017
increments.2017-02-24T12:42:23+01:00.dir   Fri Feb 24 12:42:23 2017
increments.2017-02-27T13:45:11+01:00.dir   Mon Feb 27 13:45:11 2017
increments.2017-02-27T17:03:21+01:00.dir   Mon Feb 27 17:03:21 2017
increments.2017-02-28T14:56:43+01:00.dir   Tue Feb 28 14:56:43 2017
increments.2017-03-05T04:40:29+01:00.dir   Sun Mar  5 04:40:29 2017
increments.2017-03-09T19:24:12+01:00.dir   Thu Mar  9 19:24:12 2017
increments.2017-03-12T03:30:37+01:00.dir   Sun Mar 12 03:30:37 2017
Current mirror: Tue Mar 14 19:26:57 2017

We fixed all the errors and I can read all the documents from the source
directory fine , but rdiff is having corrupt documents in the backup
destination.

When testing I found documents with different md5sums in my backup and
the source destination.

After some investigating --verify outputs the bad documents:

http://paste.debian.net/hidden/d311cfcf/

Why is rdiff-backup running fine over the directory but still has
corrupted documents in the destination and how do I fix this without
removing the full backup.

Kind regards,

Jelle de Jong

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL:
http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki




___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki