Hi,
I'm using rsync to keep two local 30G disk partitions,
in sync. This *use to* work fine, but now rsync stops
after 500 - 800 files. Some environmental info follows,
some of this may be superfluous, but maybe not ..
** Warning - long lines ahead
OS : FreeBSD 4.1-RELEASE
ad2: 29188MB <ST330630A> [59303/16/63] at ata1-master using UDMA66
ad3: 29188MB <ST330630A> [59303/16/63] at ata1-slave using UDMA66
(a) rsync --version
-------------------------------
rsync version 2.4.6 protocol version 24
Written by Andrew Tridgell and Paul Mackerras
(b) rsync - start of output
---------------------------------
cmd= machine= user= path=/mnt/ad3s1e
cmd=. /mnt/ad3s1e
local_version=24 remote_version=24
local_version=24 remote_version=24
building file list ... make_file(4,protected)
make_file(4,protected/home)
make_file(4,protected/home/samba)
(c) rsync - last lines output before it pauses
-------------------------------------------
make_file(4,protected/home/simon/ccc/ccc-sql/lcrt.sql)
make_file(4,protected/home/simon/ccc/ccc-sql/schedule.sql)
make_file(4,protected/home/simon/ccc/ccc-sql/lcr.sql)
make_file(4,protected/home/simon/ccc/ccc-sql/groups.sql)
make_file(4,protected/home/simon/ccc/db_access.H)
make_file(4,protected/home/simon/ccc/html2)
make_file(4,protected/home/simon/ccc/html2/Backup_of_logo6.CDR)
make_file(4,protected/home/simon/ccc/html2/logo2.CDR)
(d) ls -l logo2.CDR
----------------------------------------
-rwxr--r-- 1 simon simon 12794 Oct 11 16:01 logo2.CDR
(e) df -ki the partitions in question
------------------------------------
Filesystem 1K-blocks Used Avail Capacity iused ifree %iused Mounted on
/dev/ad2s1e 29719055 2685409 26736456 9% 27981 437937 6% /mnt/ad2s1e
/dev/ad3s1e 29719055 2683507 26738358 9% 31956 433962 7% /mnt/ad3s1e
(f) swapinfo - during rsync
---------------------------
Device 1K-blocks Used Avail Capacity Type
/dev/ad2s1b 102272 0 102272 0% Interleaved
/dev/ad3s1b 102272 0 102272 0% Interleaved
(g) mount says
-------------
/dev/ad2s1e on /mnt/ad2s1e (ufs, local, soft-updates, writes: sync 14 async 58687,
reads: sync 42285 async 164438)
/dev/ad3s1e on /mnt/ad3s1e (ufs, local, soft-updates, writes: sync 3 async 0, reads:
sync 1 async 0)
(h) rsync command line
------------------------
/usr/local/bin/rsync -v -v -v -v -v -v -v -v --progress --stats --timeout=5
--recursive --perms --owner --group --times --sparse --one-file-system --delete
/mnt/ad2s1e/* /mnt/ad3s1e
(i) ps auxw | grep rsync
-----------------------
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 3898 0.0 1.2 1128 744 p0 I+ 12:56AM 0:00.18 /usr/local/bin/rsync
-v -v -v -v -v -v -v -v --progress --stats --r
root 3899 0.0 0.9 1020 568 p0 I+ 12:56AM 0:00.01 /usr/local/bin/rsync
-v -v -v -v -v -v -v -v --progress --stats --r
Other observations -
1. Disk activity during the first run is obvious, but
there is little if any activity during subsequent runs
when rsync is collecting its information - all that
filesystem metadata seems to be cached.
2. Without the --timeout option rsync just sits there.
With the --timeout option it does exit but I don't
see any additional info.
3. There is no change in the "ps auxww | grep rsync" output
once rsync has paused for 20 seconds or so. Before that
its status is "S+" - sleeping for less than 20 seconds,
then it goes to "I+" - sleeping for longer than 20
seconds.
4. It doesn't seem to be related to a particular file - I
can delete files in the source and it just stops at
a different file.
5. It does not seem to be related to the number of files -
one log file has it stopping after 584 files another
has it stopping after 841 files. At one point I deleted
200 files in a directory and rsync went further on
but still stopped.
6. I manually placed files in a directory that it has
not yet come across and these have not been duplicated
at the destination. This is just to check that things
aren't actually happening in the background.
7. I have done some testing on the drives in question, copying
5 Gig files back and forth and then checksumming with
MD5 and all is well.
8. The filesystem(s) in question passes fsck checks.
9. I can successfully tar/untar the ccc directory that
it stops at in (c) above.
10. If I don't touch either of the file systems, rsync
always stops at the same file. If I add a file into
a directory that rsync does scan, then it stops
one file short of where it did prior to the addition
of that file.
Hopefully there is something bleedingly obvious I've missed out,
if so I would be happy to feel like a fool and have this work
with an easy fix. I sort of hoped that by going through
some of these points and writing this down something would
raise a flag, but no.
I thought there might be a pattern in 10., but 5. doesn't
support this. In (c) rsync stops after 837 lines.
Any help appreciated.
regs
simon
