numbers speak louder than words: rsyncing the db server (gigabit)
Number of files: 29021 Number of files transferred: 338 Total file size: 8.16G bytes Total transferred file size: 2.36G bytes Literal data: 2.36G bytes Matched data: 0 bytes File list size: 639379 File list generation time: 0.176 seconds File list transfer time: 0.000 seconds Total bytes sent: 7.82K Total bytes received: 414.13M sent 7.82K bytes received 414.13M bytes 2.28M bytes/sec total size is 8.16G speedup is 19.70 rsyncing the app (& everything else) server (gigabit) Number of files: 792298 Number of files transferred: 4348 Total file size: 426.83G bytes Total transferred file size: 18.43G bytes Literal data: 18.46G bytes Matched data: 0 bytes File list size: 16875888 File list generation time: 393.532 seconds File list transfer time: 0.000 seconds Total bytes sent: 97.80K Total bytes received: 8.28G sent 97.80K bytes received 8.28G bytes 5.06M bytes/sec total size is 426.83G speedup is 51.55 rsyncing the distant cafeteria server (100mbps) Number of files: 685398 Number of files transferred: 1402 Total file size: 25.32G bytes Total transferred file size: 1.34G bytes Literal data: 1.34G bytes Matched data: 0 bytes File list size: 20902098 Total bytes sent: 22.51K Total bytes received: 236.33M sent 22.51K bytes received 236.33M bytes 498.11K bytes/sec total size is 25.32G speedup is 107.11 On Sun, Nov 2, 2008 at 6:14 PM, Winelfred G. Pasamba <[EMAIL PROTECTED]> wrote: > I rsync all my servers to some local disk. It takes a long time the > first time of course, as all the files have to be transferred. But > the next time is faster. The bottleneck used to the the 100mbps lan > (10mB/s), but when i switched to 1gbs (100mB/s) the bottleneck became > the disks. Encryption and compression are not an issue these days, as > core2quads are cheap and fast. > > On Fri, Oct 31, 2008 at 3:56 PM, Ludwig Isaac Lim <[EMAIL PROTECTED]> wrote: >> >> Hi: >> >> Does anyone here have experience in rsync'ing large amount of data >> (nearly 40+ GB)? I would >> like to know the limitation of rsync before doing it. >> >> The current setup is this: >> The data (38+ GB) is in a HP-UX server. The goal is to rsync it to a NFS >> drive (which is a SAN >> [Storage Area Network]) running under Linux. >> >> The SAN can accommodate 2TB so storage area of the target machine is not >> a problem. The data >> consist mostly of schematics in PDF format; so the files are not really >> large, but there is a very >> large number files to be rsync'ed. >> >> Any other things to consider? What is a better way of doing it: >> a) rsync it over NFS? >> b) rsync it via ssh to the other machine? >> >> Regards, >> Ludwig Lim >> >> >> >> _________________________________________________ >> Philippine Linux Users' Group (PLUG) Mailing List >> http://lists.linux.org.ph/mailman/listinfo/plug >> Searchable Archives: http://archives.free.net.ph >> > > > > -- > Seek ye first the kingdom of God and all these things shall be added unto you. > > Winelfred G. Pasamba > Adventist University of the Philippines Online Information Systems > -- Seek ye first the kingdom of God and all these things shall be added unto you. Winelfred G. Pasamba Adventist University of the Philippines Online Information Systems _________________________________________________ Philippine Linux Users' Group (PLUG) Mailing List http://lists.linux.org.ph/mailman/listinfo/plug Searchable Archives: http://archives.free.net.ph
