On Wed, Jul 17, 2002 at 01:52:15PM -0400, Dave North wrote:
> We have an interesting quandry here. When I'm rsync'ing my directory
> tree (100 directories each containing 1000 files) I see some strange
> results:
>
> All of these machines are on their own network segment (100t) in our QA
> lab
>
> Solaris->Solaris - time taken: 11m32s
> Solaris->RH Linux 7.2 - time taken: 206s
> RH Linux->Rn Linux - time taken 1m59s
>
> In each instance I have rsync running as a server on the target machine
> (Linux in each case). I just can't explain why the performance is just
> so AMAZING on Linux. Any ideas?
I can immediately see three factors that may be affecting
this.
1. Cache.
I'm not certain of Solaris but linux uses almost all
free memory for caching so there may be much less
disk I/O in your situation on the linux boxes.
This will be affected by kernel version, amount of
memory, other activity on system.
2. I/O and network subsystems.
I can't remember the name but you may recall a
notorious benchmark pitting Linux against NT a
couple of years ago. Linux has been undergoing
focused effort to improve performance both disk I/O
(esp 2.5) and networking(2.3). Solaris has not been
held under the cross-hairs and if you talk to an SGI
fan he'll tell you Solaris has poor network
performance. 1+1=2
Rsync is a poor choice for unit and system
benchmarks. Other tools would be better for
checking whether I/O or network are affecting things.
3. Filesystem design.
You don't say what filesystem types are involved.
I'm assuming ext2 on Linux and UFS on Solaris.
This might not be a major factor but could be. The
design of a filesystem involving directory layout,
block allocation and tracking schemes can affect the
time taken to add, delete and grow files. Witness
early XFS slowness in mass file deletion.
I can immediately see FOUR factors.
4. Filesystem build options.
Block and fragment size affect not only allocation
frequency but the presence of indirect blocks
affects speed of file access. Less important for
speed would be bytes per inode and cylinders per
group. Historically Solaris defaults have been
piss-poor for performance so i haven't used defaults
on that platform in more than 10 years.
I can immediately see three, no five factors... Damn, come
in again.
5. Synchronous directory updates.
UNIX has traditionally done directory ops
synchronously. Linux, on the other hand, does them
asynchronously. In English what this means is that
on UNIX the system calls unlink(2), rename(2) or
open(2) will not return until any change to the
directory file has been written to disk; but linux
will will return from these system calls as soon as
the in-memory structures are updated.
Rsync is pretty heavy into making directory changes.
Every file updated causes two directory changes. On
UNIX this means that you have to pause for the disk
I/O to complete twice for every updated file.
This may be what accounts for the big jump when you
change the destination from Solaris to Linux.
Mount options may affect significantly.
A fairly good description of this can be found at:
http://marc.theaimsgroup.com/?l=linux-kernel&m=102676438229862&w=2
--
________________________________________________________________
J.W. Schultz Pegasystems Technologies
email address: [EMAIL PROTECTED]
Remember Cernan and Schmitt
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
