Re: faster rsync of huge directories

2010-04-14 Thread Shachar Shemesh
guy keren wrote: as well as sys admins/kernel developers - the initrd file on (some?) linux distributions is a gziped cpio file (at least on RHEL 5.X) Initrd can come in one of two formats. These are either some (any) file system (you usually use some read only file system, most common of

Re: faster rsync of huge directories

2010-04-13 Thread Tom Rosenfeld
On Mon, Apr 12, 2010 at 5:02 PM, Nadav Har'El n...@math.technion.ac.ilwrote: On Mon, Apr 12, 2010, Tom Rosenfeld wrote about Re: faster rsync of huge directories: I realized that in my case I did not really need rsync since it is a local disk to disk copy. I could have used a tar and pipe

Re: faster rsync of huge directories

2010-04-13 Thread Nadav Har'El
On Tue, Apr 13, 2010, Tom Rosenfeld wrote about Re: faster rsync of huge directories: By the way, while cpio -p is indeed a good historic tool, nowadays there is little reason to use it, because GNU's cp make it easier to do almost everything that cpio -p did: The -a option to cp

Re: faster rsync of huge directories

2010-04-13 Thread guy keren
Nadav Har'El wrote: On Tue, Apr 13, 2010, Tom Rosenfeld wrote about Re: faster rsync of huge directories: By the way, while cpio -p is indeed a good historic tool, nowadays there is little reason to use it, because GNU's cp make it easier to do almost everything that cpio -p did: The -a option

Re: faster rsync of huge directories

2010-04-12 Thread Vitaly
2010/4/12 Tom Rosenfeld tro...@bezeqint.net Hi, I am a great fan of rsync for copying filesystems. However I now have a filesystem which is several hundred gigabytes and apparently has a lot of small files. I have been running rsync all night and it still did not start copying as it is

Re: faster rsync of huge directories

2010-04-12 Thread Shachar Shemesh
Tom Rosenfeld wrote: Hi, I am a great fan of rsync for copying filesystems. However I now have a filesystem which is several hundred gigabytes and apparently has a lot of small files. I have been running rsync all night and it still did not start copying as it is still building the file

Re: faster rsync of huge directories

2010-04-12 Thread Nadav Har'El
On Mon, Apr 12, 2010, Shachar Shemesh wrote about Re: faster rsync of huge directories: Upgrade both ends to rsync version 3 or later. That version starts the transfer even before the file list is completely built. Maybe I'm missing something, but how does this help? It may find the first

Re: faster rsync of huge directories

2010-04-12 Thread Shachar Shemesh
Nadav Har'El wrote: On Mon, Apr 12, 2010, Shachar Shemesh wrote about Re: faster rsync of huge directories: Upgrade both ends to rsync version 3 or later. That version starts the transfer even before the file list is completely built. Maybe I'm missing something, but how does

Re: faster rsync of huge directories

2010-04-12 Thread Tom Rosenfeld
On Mon, Apr 12, 2010 at 9:41 AM, Tom Rosenfeld tro...@bezeqint.net wrote: Hi, I am a great fan of rsync for copying filesystems. However I now have a filesystem which is several hundred gigabytes and apparently has a lot of small files. I have been running rsync all night and it still did

Re: faster rsync of huge directories

2010-04-12 Thread Shachar Shemesh
Tom Rosenfeld wrote: On Mon, Apr 12, 2010 at 9:41 AM, Tom Rosenfeld tro...@bezeqint.net mailto:tro...@bezeqint.net wrote: Hi, I am a great fan of rsync for copying filesystems. However I now have a filesystem which is several hundred gigabytes and apparently has a lot of

Re: faster rsync of huge directories

2010-04-12 Thread Nadav Har'El
On Mon, Apr 12, 2010, Tom Rosenfeld wrote about Re: faster rsync of huge directories: I realized that in my case I did not really need rsync since it is a local disk to disk copy. I could have used a tar and pipe, but I like cpio: Is this quicker? If it is, then the reason of rsync's extreme

Re: faster rsync of huge directories

2010-04-12 Thread Constantine Shulyupin
By default cpio also will not overwrite files if the source is not newer. Consider cp -ur rsync also can --delete extraneous files from dest dirs -- Constantine Shulyupin Embedded Linux Expert TI DaVinci Expert Tel-Aviv Israel http://www.LinuxDriver.co.il/

RE: faster rsync of huge directories

2010-04-12 Thread Boaz Yahav(berber)
Check out Repliweb From: linux-il-boun...@cs.huji.ac.il [mailto:linux-il-boun...@cs.huji.ac.il] On Behalf Of Tom Rosenfeld Sent: Monday, April 12, 2010 9:41 AM To: linux-il@cs.huji.ac.il Subject: faster rsync of huge directories Hi, I am a great fan of rsync for copying filesystems.

Re: faster rsync of huge directories

2010-04-12 Thread Tom Rosenfeld
On Mon, Apr 12, 2010 at 10:04 AM, Vitaly li...@karasik.org wrote: 2010/4/12 Tom Rosenfeld tro...@bezeqint.net Hi, I am a great fan of rsync for copying filesystems. However I now have a filesystem which is several hundred gigabytes and apparently has a lot of small files. I have been