Dan thanks for your response.
On Fri, Apr 03, 2009 at 05:16:21PM -0400, Dan Pritts wrote:
> > On Wed, Apr 01, 2009 at 05:38:57PM +0000, John Rouillard wrote:
> > > I am backing up a system using rsync and it has a (source) file that
> > > is 40GB in size. Dring the backup it grows to 200GB in size.
>
> your db file is probably a sparse file.
That's what I though initally, but it's not AFAICT. At least the
apparent size returned by du is a bit smaller than the disk usage.
sudo du /data/bak/fedora-ds/old/userRoot/id2entry.db4
39621000 /data/bak/fedora-ds/old/userRoot/id2entry.db4
sudo du --apparent-size /data/bak/fedora-ds/old/userRoot/id2entry.db4
39582296 /data/bak/fedora-ds/old/userRoot/id2entry.db4
In most sparse files it is larger.
Also I have a little script I use to locate sparse files and it
doesn't flag that file. It uses stat to find the size of the file
(including unallocated blocks) and calculates the actual allocated
blocks times the block size.
For the file in question stat fills in:
if [ 40532271104 -gt `expr 79242000 \* 512` ]; then echo
/data/bak/fedora-ds/old/userRoot/id2entry.db4; fi
where 40532271104 is the ls -l size and 79242000 is the number of
allocated blocks times a 512 byte block size. The allocated size is a
little larger due to partly writing the block, indirect block etc.
I am running on an ext3 filesystem and not zfs, so no compression or
other expanding stuff is going on.
Also a fast dump of the file doesn't show a block of nulls anywhere in
the file.
So given the above do you still think it's a sparseness issue?
Even if it is a sparse file, it should never be larger that the actual
size reported by ls -l right? I am seeing the copy be 5 times the ls
-l size. Since the compression is gzip basically, and I tested
compressing the file with gzip -3 and it reduced (rather than
expanded) in size, that seems to rule out the compression in backuppc
doing something really wrong.
> rsync supports sparse files, but it's not part of the defaults. Not sure
> if backuppc's rsync module handles sparse; based on a quick look at the
> args getting passed to the remote rsync, i don't think so.
I would doubt it is supported as well.
> if backuppc doesn't do sparse, as a workaround you can run a job on the
> backup client to make a compressed local copy of the file, and back that
> up instead.
>
> with moderate-sized sparse files it's just a little overhead but in your
> case it is clearly a big problem :)
Well what I am wondering about is the issue reported in:
http://www.mail-archive.com/backuppc-devel@lists.sourceforge.net/msg00226.html
Which was a corrupted size attribute generated somehow by RsyncP. If
it really thought that that file was somehow larger than it was ....
I never saw a response from the developers on that.
Also I have seen wackyness in the incremental backups that was cleared
by running a full backup:
http://www.mail-archive.com/backuppc-us...@lists.sourceforge.net/msg12913.html
but I haven't tried running a level 0 of the ever expanding file yet
so that I don't destroy state that may be useful in debugging the
problem.
--
-- rouilj
John Rouillard System Administrator
Renesys Corporation 603-244-9084 (cell) 603-643-9300 x 111
------------------------------------------------------------------------------
_______________________________________________
BackupPC-devel mailing list
BackupPC-devel@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-devel
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
