rsync 2 million files

2008-02-12 Thread Martin Strand
I'm trying to use rsync to get a live backup of 2 million files, about 50  
gb, max depth of 5 directory levels.
I'm on a gigabit lan so I'm passing -W, but it's still incredibly slow.  
What else can I do to speed things up?
Perhaps there's a good way to filter out files older than X so only newer  
files are checked?.

Will rsync 3.0.0 make a big difference for large trees?

Thanks for any tips,
Martin
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


rsync cvs

2008-02-12 Thread PREM KUMAR
Hi
I am in new in rsync cvs. I want to know how to configure the rsync cvs. we
have two site we want sync on both site . If i do sync one site all the copy
come to overwrite. I am using this command'

rsync -az --delete --exclude '#cvs.*' --exclude 'CVSROOT/config' --exclude
'CVSROOT/history' --exclude 'CVSROOT/updatelog' [EMAIL PROTECTED]:
/home/cmadmin/

Please let me know how resolve.

Thanks and Regards

Prem
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: rsync 2 million files

2008-02-12 Thread Giuliano Gavazzi


On 12Feb, 2008, at 09:53 , Martin Strand wrote:

I'm trying to use rsync to get a live backup of 2 million files,  
about 50 gb, max depth of 5 directory levels.
I'm on a gigabit lan so I'm passing -W, but it's still incredibly  
slow. What else can I do to speed things up?
Perhaps there's a good way to filter out files older than X so only  
newer files are checked?.

Will rsync 3.0.0 make a big difference for large trees?


I say yes, although you did not quantify your slow.
The following was not on a gigabit, but on a firewire (tcp/ip over  
firewire), and I suspect that it would have worked better on a gigabit  
(better drivers). It was reported some time ago using rsync-3.0.0pre7:


a new tranfer of about 30GB worth of data in about 2 hours:


Number of files: 407317

Total file size: 34860439736 bytes
Total transferred file size: 34860261591 bytes
Literal data: 34860261591 bytes
Matched data: 0 bytes
File list size: 10391700
File list generation time: 0.003 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 35084947192
Total bytes received: 7332354

sent 35084947192 bytes  received 7332354 bytes  4927652.82 bytes/sec

A subsequent re-rsync took about 5 minutes (little data changes).

I would say that you can look at about the same sort of figures, or  
even a bit less, with better hardware. This was after all just a  
PowerBook G4 backed up to an iMac Intel.


Giuliano
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: backup via an intermediate drive

2008-02-12 Thread Cavan Fyans


On 12 Feb 2008, at 00:01, Matt McCutchen wrote:


On Mon, 2008-02-11 at 23:58 +, Cavan Fyans wrote:

Thanks for the quick help guys!  The external backup server will be
online, just not with a large enough connection to realistically
backup via the connection, hence the transportation system!

Since it will be online then presumably i can use the write-batch
option as the in house server will be able to see the backup server  
to

decide what needs transferring, and then just do the transporting of
the batch on the intermediate drive?


Yes, that should work well.

Matt





Great, thanks for all the help!  One other problem now, one of the  
studio servers is an XServe and therefore running off an OS x  
machine.  It has an older version of RSync installed on it (2.6.3)  
which does not have the 'only-write-batch' option - I presume this was  
just added in the newer versions and its not something that is  
excluded in the build for the OS x system?  I tried updating to the  
latest RSync version (2.6.9) on this machine but after installing (./ 
configure, make, make install) it has clearly not replaced the  
previous version of Rsync.  Am I missing something blindingly obvious  
here that i need to do to update the RSync version on the OS x machine?


Thanks.
Cav.
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: Rsync to a Read Only file system

2008-02-12 Thread Douglas Wade Needham
The one thing I can see is that perhaps part of what is being sync'ed
is on a read-only filesystem, while portions of the underlying tree
are not.  For example, it is common to mount the root file system
read-only, while areas such as /var are read-write.

On the flip side, there is also the problem where the top portions of
a tree being rsync'ed would be in a read-write area, and areas under
it would potentially be in a read-only partition.  I have seen this a
number of times, where developers would have multiple areas mounted
under their home directories onto a build server.

On the performance improvements... I am sure that there are some which
can be made... always is, always will be.  About the best I can see in
this would be perhaps test and set/clear a flag every time one directory is
traversed.  This likely could be done by looking at the st_dev member
of the stat structure, IIRC.

- Doug

Quoting [EMAIL PROTECTED] ([EMAIL PROTECTED]):
 I think your product is awesome, but I am experiencing an unexpected  
 behaviour.
 
 $ rsync -avviPH /Users/alan/Desktop/rsync_test\ Folder/   
 [EMAIL PROTECTED]::Downloads
 
 opening tcp connection to slug port 873
 sending daemon args: --server -vvlHogDtpre30.16i --log-format=%i -- 
 partial . Downloads
 sending incremental file list
 .d..t..g... ./
 rsync: failed to write xattr user.rsync.%stat for /. (in  
 Downloads): Read-only file system (30)
 rsync: failed to set times on /. (in Downloads): Read-only file  
 system (30)
 f.st..g... .DS_Store
 6148 100%4.76MB/s0:00:00 (xfer#1, to-check=12/14)
 cd+ RSYNC_TEST/
 rsync: recv_generator: mkdir /RSYNC_TEST (in Downloads) failed:  
 Read-only file system (30)
 *** Skipping any contents from this failed directory ***
 rsync: mkstemp /..DS_Store.iiOfRf (in Downloads) failed: Read-only  
 file system (30)
 total: matches=6  hash_hits=8  false_alarms=0 data=1948
 
 sent 2414 bytes  received 144 bytes  1705.33 bytes/sec
 total size is 146404  speedup is 57.23
 rsync error: some files could not be transferred (code 23) at main.c 
 (1058) [sender=3.0.0pre8]
 
 The destination file system is  Read Only, but RSYNC takes the time  
 and looks as if the transfer is happening, but of course it is not.
 I would like RSYNC to terminate as soon as it knows its destination  
 is Read only.
 
 The transfer is from a  Mac OSX to Debian Linux HFSPlus file system.
 
 OSX
  rsync  version 3.0.0pre8  protocol version 30.PR16
  Capabilities:
 64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints,
 socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
 append, ACLs, xattrs, iconv, file-flags
 
 Linux
 rsync  version 3.0.0pre8  protocol version 30.PR16
 Capabilities:
 64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
 socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
 append, ACLs, xattrs, iconv
 
 Thanks
 
 
 -- 
 To unsubscribe or change options: 
 https://lists.samba.org/mailman/listinfo/rsync
 Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

-- 
Douglas Wade Needham - KA8ZRTUN*X Consultant  UW/BSD kernel programmer
Email:  cinnion @ ka8zrt . com   http://www.ka8zrt.com
Disclaimer: My opinions are my own.  Since I don't want them, why
should my employer, or anybody else for that matter! 
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: Rsync to a Read Only file system

2008-02-12 Thread Paul Slootman
On Tue 12 Feb 2008, [EMAIL PROTECTED] wrote:

 The destination file system is  Read Only, but RSYNC takes the time and 
 looks as if the transfer is happening, but of course it is not.
 I would like RSYNC to terminate as soon as it knows its destination is Read 
 only.

Rsync doesn't know that the whole destination is read-only...


Paul Slootman
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: Making rsync compile under Mac OS X 10.3.9 with extended attributes

2008-02-12 Thread Matt McCutchen
On Tue, 2008-02-12 at 15:24 +0100, Vitorio Machado wrote: 
 I'm trying to figure out how to modify the makefile to say to compile  
 sysxattrs.c with the -I /Developer/SDKs/MacOSX10.3.9.sdk/Developer/ 
 Headers/CFMCarbon/
 in the case of HAVE_PRE_TIGER_OSX_ATTRS is true.
 Is somebody skilled on makefile to give me some help?

I did some work on top of your rsync10.3xattr_support080211 files.  I
added code to configure.in and Makefile.in that should get the
appropriate -I option passed for lib/sysxattrs.c in the case of
HAVE_PRE_TIGER_OSX_ATTRS.  You can modify the option in configure.in if
necessary.  I put * instead of 3.9 to try to make the option work on
pre-Tiger systems other than 10.3.9.

I also think I figured out why the configure script was refusing to
enable HAVE_PRE_TIGER_OSX_ATTRS.  First, every time you change
configure.in, you need to run autoconf -o configure.sh to update
configure.sh (the underlying configure script for which ./configure is a
simple wrapper).  The ./configure wrapper will invoke autoconf to
generate configure.sh if it doesn't exist, but once it exists, you have
to regenerate it yourself as necessary.  Second, the [] characters in
darwin[4-7]* in configure.in are recognized by autoconf as quoting
characters and stripped out, so configure.sh says darwin4-7* , which
obviously won't match.  I changed configure.in to say darwin[[4-7]]* ;
autoconf strips only the outer pair to leave the desired darwin[4-7]*
in configure.sh.

I also neatened up the code in other places and finished your strncpy
call.  A patch with my proposed changes on top of your files is
attached.

Matt
diff --git a/Makefile.in b/Makefile.in
index 4716aba..0145f59 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -61,6 +61,12 @@ CHECK_OBJS=tls.o getgroups.o getfsdev.o t_stub.o t_unsafe.o trimslash.o wildtest
 
 all: conf_stop make_stop rsync$(EXEEXT) @MAKE_MAN@
 
+# Separate rule to accommodate extra options for lib/sysxattrs.o on pre-Tiger OS X
+lib/sysxattrs.o: lib/sysxattrs.c
[EMAIL PROTECTED]@
+	$(CC) -I. -I$(srcdir) @SYSXATTRS_CFLAGS@ $(CFLAGS) $(CPPFLAGS) -c $ @CC_SHOBJ_FLAG@
[EMAIL PROTECTED]@
+
 install: all
 	-mkdir -p ${DESTDIR}${bindir}
 	${INSTALLCMD} ${INSTALL_STRIP} -m 755 rsync$(EXEEXT) ${DESTDIR}${bindir}
diff --git a/configure.in b/configure.in
index c6bc065..f629815 100644
--- a/configure.in
+++ b/configure.in
@@ -932,10 +932,14 @@ else
 	AC_DEFINE(HAVE_LINUX_XATTRS, 1, [True if you have Linux xattrs])
 	AC_DEFINE(SUPPORT_XATTRS, 1)
 	;;
-	darwin[4567]*)
+# The square brackets in the next line are doubled in configure.in
+# because autoconf strips out one pair as quoting symbols.
+darwin[[4-7]].*)
 	AC_MSG_RESULT(Using OS X PreTiger attrs)
+	AC_DEFINE(HAVE_OSX_XATTRS, 1, [True if you have Mac OS X xattrs])
 	AC_DEFINE(HAVE_PRE_TIGER_OSX_ATTRS, 1, [True if you have Mac OS X PreTiger attrs])
 	AC_DEFINE(SUPPORT_XATTRS, 1)
+	SYSXATTRS_CFLAGS=-I /Developer/SDKs/MacOSX10.*.sdk/Developer/Headers/CFMCarbon/
 	;;
 darwin*)
 	AC_MSG_RESULT(Using OS X xattrs)
@@ -956,6 +960,7 @@ else
 	;;
 esac
 fi
+AC_SUBST(SYSXATTRS_CFLAGS)
 
 AC_CONFIG_FILES([Makefile lib/dummy zlib/dummy popt/dummy shconfig])
 AC_OUTPUT
diff --git a/lib/sysxattrs.c b/lib/sysxattrs.c
index fae677f..bf267c7 100644
--- a/lib/sysxattrs.c
+++ b/lib/sysxattrs.c
@@ -54,6 +54,14 @@ ssize_t sys_llistxattr(const char *path, char *list, size_t size)
 
 #ifdef HAVE_PRE_TIGER_OSX_ATTRS
 
+#include sys/attr.h
+#include unistd.h
+#include string.h
+#include CarbonCore/Files.h
+
+#define FINDER_INFO_XATTR_NAME com.apple.FinderInfo
+#define RESOURCE_FORK_XATTR_NAME com.apple.ResourceFork
+
 ssize_t sys_lgetxattr(const char *path, const char *name, void *value, size_t size)
 {
 	struct attrlist attrList; //List of attributes
@@ -68,7 +76,7 @@ ssize_t sys_lgetxattr(const char *path, const char *name, void *value, size_t si
 	attrList.forkattr=0;
 	
 	/* Uses getattrlist to get FinderInfo */
-	if(strcmp(name, com.apple.FinderInfo)==0)
+	if(strcmp(name, FINDER_INFO_XATTR_NAME)==0)
 	{
 		attrList.commonattr+=ATTR_CMN_FNDRINFO;
 		if(getattrlist(path, attrList, value, size, FSOPT_NOFOLLOW)==0)
@@ -77,7 +85,7 @@ ssize_t sys_lgetxattr(const char *path, const char *name, void *value, size_t si
 			return -1; /* Errno is setup by getattrlist */
 	}
 	/* Uses carbon to get ResourceFork */
-	else if(strcmp(name, com.apple.ResourceFork)==0)
+	else if(strcmp(name, RESOURCE_FORK_XATTR_NAME)==0)
 	{
 		char path_with_name[PATH_MAX]; /* POSIX name of the file */
 		FSRef ref; /* A Carbon reference to the file (analogue to a POSIX Path) */
@@ -180,14 +188,14 @@ int sys_lsetxattr(const char *path, const char *name, const void *value, size_t
 	attrList.forkattr=0;
 	
 	/* Uses getattrlist to get FinderInfo */
-	if(strcmp(name, com.apple.FinderInfo)==0)
+	if(strcmp(name, FINDER_INFO_XATTR_NAME)==0)
 	{
 		attrList.commonattr+=ATTR_CMN_FNDRINFO;
 		return setattrlist(path, attrList, value, size, FSOPT_NOFOLLOW);
 	}
 	
 	/* Uses carbon to set 

Re: Rsync 2.6.9 does not skip any files based on modification time

2008-02-12 Thread Matt McCutchen
On Wed, 2008-02-13 at 03:59 +0100, Stefan Malte Schumacher wrote:
 Hmm, it seems I need help again. I am calling again rsync with the following 
 options :
 rsync -rvtO --del --files-from=/cygdrive/C/Programme/cwRsync/rsync.txt 
 --modify-window=3 [EMAIL PROTECTED]::Backup 
 
 rsync.txt just contains one single line :
 /cygdrive/D/Temp
 
 rsync then outputs a list of the command line options and the following:
 rsync error: syntax or usage error (code 1) at options.c(1418) [client=2.6.9] 

With --files-from, you still have to specify a master source directory
relative to which the --files-from entries are interpreted.  In your
case, you should just use / .  Your command should be:

rsync -rvtO --del --files-from=/cygdrive/C/Programme/cwRsync/rsync.txt
\   --modify-window=3 / [EMAIL PROTECTED]::Backup

 How should I list E:\Bücher  Comics in rsync.txt ?
 As /cygdrive/E/Bücher  Comics or should I just use single or double
 quotes ? 

As you said.  rsync.txt should contain these two lines:

/cygdrive/D/Temp
/cygdrive/E/Bücher  Comics

--files-from implies --relative, so by default, rsync will use the
entire source paths in the file-list and thus recreate cygdrive/D/Temp
and /cygdrive/E/Bücher  Comics under the destination.  You can
override this by putting a ./ in each --files-from entry; then, only the
path components after the ./ will be used.

Matt

-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: Rsync 2.6.9 does not skip any files based on modification time

2008-02-12 Thread Stefan Malte Schumacher


 Good.  Then my guess is that there's some weirdness in the hacked
 Cygwin's handling of command-line arguments.  As a workaround, you could
 try using --files-from instead of putting the path on the command line.


Hmm, it seems I need help again. I am calling again rsync with the following
options :
rsync -rvtO --del --files-from=/cygdrive/C/Programme/cwRsync/rsync.txt
--modify-window=3 [EMAIL PROTECTED]::Backup

rsync.txt just contains one single line :
/cygdrive/D/Temp

rsync then outputs a list of the command line options and the following:
rsync error: syntax or usage error (code 1) at options.c(1418) [client=2.6.9
]

How should I list E:\Bücher  Comics in rsync.txt ? As /cygdrive/E/Bücher 
Comics or should I just use single or double quotes ?

Bye
Stefan
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Rsync to a Read Only file system

2008-02-12 Thread Matt McCutchen
On Tue, 2008-02-12 at 20:31 +0100, [EMAIL PROTECTED] wrote:
 The destination file system is  Read Only, but RSYNC takes the time  
 and looks as if the transfer is happening, but of course it is not.
 I would like RSYNC to terminate as soon as it knows its destination  
 is Read only.

I'm not convinced that it's worth adding code to rsync to second-guess a
filesystem misconfiguration.  If the filesystem backing
[EMAIL PROTECTED]::Downloads is always mounted read-only, you should just set
read only = yes for the module in the daemon configuration; then
pushes will abort immediately.

Matt

-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Rsync to a Read Only file system

2008-02-12 Thread smokin . jake
I think your product is awesome, but I am experiencing an unexpected  
behaviour.


$ rsync -avviPH /Users/alan/Desktop/rsync_test\ Folder/   
[EMAIL PROTECTED]::Downloads


opening tcp connection to slug port 873
sending daemon args: --server -vvlHogDtpre30.16i --log-format=%i -- 
partial . Downloads

sending incremental file list
.d..t..g... ./
rsync: failed to write xattr user.rsync.%stat for /. (in  
Downloads): Read-only file system (30)
rsync: failed to set times on /. (in Downloads): Read-only file  
system (30)

f.st..g... .DS_Store
6148 100%4.76MB/s0:00:00 (xfer#1, to-check=12/14)
cd+ RSYNC_TEST/
rsync: recv_generator: mkdir /RSYNC_TEST (in Downloads) failed:  
Read-only file system (30)

*** Skipping any contents from this failed directory ***
rsync: mkstemp /..DS_Store.iiOfRf (in Downloads) failed: Read-only  
file system (30)

total: matches=6  hash_hits=8  false_alarms=0 data=1948

sent 2414 bytes  received 144 bytes  1705.33 bytes/sec
total size is 146404  speedup is 57.23
rsync error: some files could not be transferred (code 23) at main.c 
(1058) [sender=3.0.0pre8]


The destination file system is  Read Only, but RSYNC takes the time  
and looks as if the transfer is happening, but of course it is not.
I would like RSYNC to terminate as soon as it knows its destination  
is Read only.


The transfer is from a  Mac OSX to Debian Linux HFSPlus file system.

OSX
 rsync  version 3.0.0pre8  protocol version 30.PR16
 Capabilities:
64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, file-flags

Linux
rsync  version 3.0.0pre8  protocol version 30.PR16
Capabilities:
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv

Thanks


--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


[PATCH] Make `make distclean' remove a few more files.

2008-02-12 Thread Matt McCutchen
---
 Makefile.in |3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index 98486f4..fee32b7 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -171,11 +171,14 @@ cleantests:
 # the source directory.
 distclean: clean
rm -f Makefile config.h config.status
+   rm -f lib/dummy popt/dummy zlib/dummy
rm -f $(srcdir)/Makefile $(srcdir)/config.h $(srcdir)/config.status
+   rm -f $(srcdir)/lib/dummy $(srcdir)/popt/dummy $(srcdir)/zlib/dummy
rm -f config.cache config.log
rm -f $(srcdir)/config.cache $(srcdir)/config.log
rm -f shconfig $(srcdir)/shconfig
rm -f $(GENFILES)
+   rm -rf autom4te.cache
 
 # this target is really just for my use. It only works on a limited
 # range of machines and is used to produce a list of potentially
-- 
1.5.4.rc3.15.g4bbc

-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: backup via an intermediate drive

2008-02-12 Thread Matt McCutchen
On Tue, 2008-02-12 at 17:57 +, Cavan Fyans wrote:
 One other problem now, one of the  
 studio servers is an XServe and therefore running off an OS x  
 machine.  It has an older version of RSync installed on it (2.6.3)  
 which does not have the 'only-write-batch' option - I presume this was  
 just added in the newer versions and its not something that is  
 excluded in the build for the OS x system?

Correct.  According to OLDNEWS, --only-write-batch was added in rsync
2.6.5.

 I tried updating to the  
 latest RSync version (2.6.9) on this machine but after installing (./ 
 configure, make, make install) it has clearly not replaced the  
 previous version of Rsync.  Am I missing something blindingly obvious  
 here that i need to do to update the RSync version on the OS x machine?

The source package's make install puts the executable
at /usr/local/bin/rsync by default; it won't replace any rsync
executables you might have elsewhere on the machine.  If the wrong
version of rsync is being invoked, try using the full
path /usr/local/bin/rsync .

Matt

-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: Need ideas for project

2008-02-12 Thread Bob Bagwill
On Mon, 11 Feb 2008 09:02:14 -0500, Matt McCutchen [EMAIL PROTECTED] wrote:

 A while ago, Wayne Davison and Martin Pool came up with
 some ideas for superlifter, a next-generation file copying tool that
 would overcome many of the architectural limitations of rsync, but no
 work has been done on superlifter for a few years.

FWIW, the BEEP peer-to-peer protocol [http://www.beepcore.org/] might be a good 
foundation
for an rsyncNG.

-- 
Bob Bagwill
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: rsync 2 million files

2008-02-12 Thread Martin Strand
On Tue, 12 Feb 2008 10:52:00 +0100, Giuliano Gavazzi [EMAIL PROTECTED]  
wrote:




On 12Feb, 2008, at 09:53 , Martin Strand wrote:

I'm trying to use rsync to get a live backup of 2 million files, about  
50 gb, max depth of 5 directory levels.
I'm on a gigabit lan so I'm passing -W, but it's still incredibly slow.  
What else can I do to speed things up?
Perhaps there's a good way to filter out files older than X so only  
newer files are checked?.

Will rsync 3.0.0 make a big difference for large trees?


I say yes, although you did not quantify your slow.
The following was not on a gigabit, but on a firewire (tcp/ip over  
firewire), and I suspect that it would have worked better on a gigabit  
(better drivers). It was reported some time ago using rsync-3.0.0pre7:


a new tranfer of about 30GB worth of data in about 2 hours:


Number of files: 407317

Total file size: 34860439736 bytes
Total transferred file size: 34860261591 bytes
Literal data: 34860261591 bytes
Matched data: 0 bytes
File list size: 10391700
File list generation time: 0.003 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 35084947192
Total bytes received: 7332354

sent 35084947192 bytes  received 7332354 bytes  4927652.82 bytes/sec

A subsequent re-rsync took about 5 minutes (little data changes).

I would say that you can look at about the same sort of figures, or even  
a bit less, with better hardware. This was after all just a PowerBook G4  
backed up to an iMac Intel.


Giuliano


I just tried it with rsync 3.0.0pre8 and it was significantly faster.
Thanks!
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: Making rsync compile under Mac OS X 10.3.9 with extended attributes

2008-02-12 Thread Vitorio Machado
I'm trying to figure out how to modify the makefile to say to compile  
sysxattrs.c with the -I /Developer/SDKs/MacOSX10.3.9.sdk/Developer/ 
Headers/CFMCarbon/

in the case of HAVE_PRE_TIGER_OSX_ATTRS is true.
Is somebody skilled on makefile to give me some help?

Le 12 févr. 08 à 12:29, Vitorio Machado a écrit :

It seems that this problem is generated by the inclusion of the  
Carbon API that have conflicting headers with the *NIX ones...


I don't know if doing export C_INCLUDE_PATH=/Developer/SDKs/ 
MacOSX10.4.0.sdk/Developer/Headers/CFMCarbon/ is the correct way to  
link with the CarbonCore.



--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: rsync cvs

2008-02-12 Thread Matt McCutchen
On Tue, 2008-02-12 at 14:41 +0530, PREM KUMAR wrote:
 I am in new in rsync cvs. I want to know how to configure the rsync
 cvs. we have two site we want sync on both site . If i do sync one
 site all the copy come to overwrite. I am using this command'
 rsync -az --delete --exclude '#cvs.*' --exclude 'CVSROOT/config'
 --exclude 'CVSROOT/history' --exclude 'CVSROOT/updatelog'
 [EMAIL PROTECTED]: /home/cmadmin/
 
 Please let me know how resolve.

Rsync is designed for one-way copying; for two-way synchronization, use 
Unison ( http://www.cis.upenn.edu/~bcpierce/unison/ ) instead.  Even
so, Unison does file-level synchronization without any regard for the
special structure of CVS repositories, so it may corrupt the
repositories, especially if a change to the same file is committed on
both sites.  You may be able to find a specialized solution for
synchronizing CVS repositories; see this message:

http://readlist.com/lists/nongnu.org/info-cvs/1/5785.html

Or you could switch to a distributed version control system, which has
built-in support for repository synchronization.  My personal favorite
is git ( http://git.or.cz/ ).

Matt

-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: Making rsync compile under Mac OS X 10.3.9 with extended attributes

2008-02-12 Thread Vitorio Machado
It seems that this problem is generated by the inclusion of the  
Carbon API that have conflicting headers with the *NIX ones...


I don't know if doing export C_INCLUDE_PATH=/Developer/SDKs/ 
MacOSX10.4.0.sdk/Developer/Headers/CFMCarbon/ is the correct way to  
link with the CarbonCore.


I'm trying some things unsuccessfully mainly because I'm really lost  
with all this complexity of configure, automake, linking etc. If  
somebody more skilled on compiling can help me on this, would be great!


Thanks,

Vitorio - a little lost in all those automatically generated code lines

Le 11 févr. 08 à 23:35, Vitorio Machado a écrit :


iMac:/Volumes/Nomade/Input/rsync-3.0.1 lag$ make
perl ./mkproto.pl ./*.c ./lib/compat.c
gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt  -c  
flist.c -o flist.o

In file included from rsync.h:351,
 from flist.c:23:
/usr/include/sys/file.h:81: error: parse error before 'caddr_t'
In file included from rsync.h:916,
 from flist.c:23:
proto.h:301: error: parse error before 'u_long'
flist.c: In function 'send_file_entry':
flist.c:429: warning: implicit declaration of function 'major'
flist.c:433: warning: implicit declaration of function 'minor'
flist.c:437: warning: implicit declaration of function 'makedev'
make: *** [flist.o] Error 1


--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: Rsync to a Read Only file system

2008-02-12 Thread Alan Snelgrove


On 13 Feb 2008, at 03:11, Matt McCutchen wrote:


On Tue, 2008-02-12 at 20:31 +0100, [EMAIL PROTECTED] wrote:

The destination file system is  Read Only, but RSYNC takes the time
and looks as if the transfer is happening, but of course it is not.
I would like RSYNC to terminate as soon as it knows its destination
is Read only.


I'm not convinced that it's worth adding code to rsync to second- 
guess a

filesystem misconfiguration.  If the filesystem backing
[EMAIL PROTECTED]::Downloads is always mounted read-only, you should just set
read only = yes for the module in the daemon configuration; then
pushes will abort immediately.

Matt



The problem is that I am syncing to a HFSPlus volume that  
occasionally decides to go read-only, so the share Download is  
unintentially on a Read only Volume. The  -i option also indicates  
the file is copied when it is not.




--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html