Re: Logging of attribute changes when no data is transferred?

2005-09-15 Thread Wayne Davison
On Wed, Sep 14, 2005 at 06:53:46PM -0400, Dave Mielke wrote:
 How can I get rsync to write a log record (like --log-format does for
 data transfer) when no content needs to be transferred but attributes
 (owner, group, mode, ...) are changed?

Use -i or put %i into your --log-format.  See the 2.6.6 manpage for details.

..wayne..
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: Partial files cause: error in rsync protocol data stream (code 12) at io.c(584)

2005-09-15 Thread Wayne Davison
On Wed, Sep 14, 2005 at 08:51:41PM -0700, Ian Webber wrote:
 rsync: writefd_unbuffered failed to write 4 bytes: phase unknown [sender]: 
 Connection reset by peer (104)

See the issues and debugging page of the website for instructions on
how to go about debugging a failure on the remote end of the rsync
connection (e.g. enabling core files to check for a crash, or logging
system calls to see what is failing).

..wayne..
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: RSYNC make problems with HP UX 11.0 GCC 3.3.2

2005-09-15 Thread Wayne Davison
On Wed, Sep 14, 2005 at 03:27:13PM -0400, Poe, David wrote:
 /usr/ccs/bin/ld: Unsatisfied symbols:
_U_Qfneg (first referenced in lib/snprintf.o) (code)

That's a function from HPUX's quadlib.o library (it negates a long
double value).  It should be a part of your standard C library.  I
can't help you beyond that, since I don't know enough about HPUX.

..wayne..
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: How to do server-side logging with rsync running through a SSH tunnel?

2005-09-15 Thread Wayne Davison
On Wed, Sep 14, 2005 at 04:22:01PM -0300, Igor Hjelmstrom Vinhas Ribeiro wrote:
 rsync --config=/home/igor/rsyncd.conf --daemon --server --sender -vr . 
 ./somefolder/somewhere

A modern rsync would reject that command-line as invalid (since --sender
and -r aren't daemon options, and paths aren't sent to a daemon on the
command-line -- such client-to-server items are sent to the daemon via
the internal protocol).

I gave a basic answer to this question of server-side logging using a
daemon-over-ssh setup back on August 24th (it's pretty terse, though):

http://lists.samba.org/archive/rsync/2005-August/013359.html

The important bit is to make sure that the client command is using
daemon-style host syntax (it sounds like you're not if your rsync_shell
script is checking path names -- there's no need to check paths with
daemon mode since the modules set their own path, nor is it possible
without parsing the start of the incoming data).

Also, make sure you're using a modern enough rsync that supports
daemon-mode over ssh (it was added in 2.5.6) and make sure that you're
sending rsync a valid daemon command-line.

..wayne..
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: probbable generator hang bug in rsync

2005-09-15 Thread Wayne Davison
On Mon, Sep 12, 2005 at 06:04:16PM -0500, Steve Sether wrote:
 Could this be a problem with Linux?  I've tried killing the
 stunnel process (as well as the parent stunnel process) on
 the server, and the rsync process persists.

Wow, that's unexpected.  I had thought that the connection from the
stunnel process to the rsync process would have been a normal (albeit
local) socket connection.  Perhaps it is not?  Or perhaps there is a bug
that got triggered by this setup.  The way rsync works, it has two
separate processes accessing one socket (one reading, one writing), and
both need to be notified of the EOF.  It sounds like the receiver got an
EOF when reading (and terminated) but that the generator did not get
told about the EOF when waiting around to write.

..wayne..
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: probbable generator hang bug in rsync

2005-09-15 Thread Steve Sether
 Wow, that's unexpected.  I had thought that the connection from the
 stunnel process to the rsync process would have been a normal (albeit
 local) socket connection.  Perhaps it is not?  Or perhaps there is a bug
 that got triggered by this setup.  The way rsync works, it has two
 separate processes accessing one socket (one reading, one writing), and
 both need to be notified of the EOF.  It sounds like the receiver got an
 EOF when reading (and terminated) but that the generator did not get
 told about the EOF when waiting around to write.
 
 ..wayne..

The connection between stunnel and rsync is just a normal socket connection.  
I've also ran an stunnel server process on a different machine that then 
connects to rsync on the server machine in an attempt to isolate any 
strangeness from 
local connections.  I get the same results of a hung generator process.

for clarity, the above test looks like this:

[rsync client]-localhost-[stunnel process on client]--local lan--[stunnel 
process on server 1]-internet-[rsync process on server 2]


Any suggestions on how to find out more of what's going on?
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: Bug#324200: rsync: man page contradictory on use of ssh

2005-09-15 Thread Paul Slootman
Hi,

I got the following report from a Debian user:

On Sat 20 Aug 2005, Ross Boylan wrote:
 
 The man page says a modern rsync uses ssh for its communications
 (while noting it may have been configure otherwise, but then goes on
 to say (about remote shells) One common substitute is to use ssh,
 which makes it sound as if ssh is not the default.
 
 Either I'm missing something, or the man page needs to be clarified.
 Or perhaps both.  I *think* its using ssh since I've been transferring
 between systems that don't allow rsh without problem.

I think he's got a point there.

The following patch should suffice, it looks like that sentence is a
remnant from when rsh was the default.


Paul

--- rsync.yo.orig   2005-07-29 11:24:21.0 +0200
+++ rsync.yo2005-09-15 11:39:54.0 +0200
@@ -76,9 +76,6 @@
 You can also specify any remote shell you like, either by using the bf(-e)
 command line option, or by setting the RSYNC_RSH environment variable.
 
-One common substitute is to use ssh, which offers a high degree of
-security.
-
 Note that rsync must be installed on both the source and destination
 machines.
 
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


--backup leaves window where file doesn't exist.

2005-09-15 Thread Dave Mielke
When using --backup, the sequence (as monitored by strace) is:

   rename(/path/to/file, backup-dir/filebackup-suffix)
   rename(/path/to/.file.xx, /path/to/file)

This, of course, leaves a momentary window wherein file can't be found. Might 
it be possible to replace the first rename() with link() instead? This, of
course, could only be done when the backup directory is on the same volume.

-- 
Dave Mielke   | 2213 Fox Crescent | I believe that the Bible is the
Phone: 1-613-726-0014 | Ottawa, Ontario   | Word of God. Please contact me
EMail: [EMAIL PROTECTED] | Canada  K2A 1H7   | if you're concerned about Hell.
http://FamilyRadio.com/   | http://Mielke.cc/bible/
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: --backup leaves window where file doesn't exist.

2005-09-15 Thread Chris Shoemaker
On Thu, Sep 15, 2005 at 08:45:00AM -0400, Dave Mielke wrote:
 When using --backup, the sequence (as monitored by strace) is:
 
rename(/path/to/file, backup-dir/filebackup-suffix)
rename(/path/to/.file.xx, /path/to/file)
 
 This, of course, leaves a momentary window wherein file can't be found. 
 Might 
 it be possible to replace the first rename() with link() instead? This, of
 course, could only be done when the backup directory is on the same volume.

Good catch.  But what about when backup-dir is not on same filesystem
as original?  link will give EXDEV, I think.

-chris

 
 -- 
 Dave Mielke   | 2213 Fox Crescent | I believe that the Bible is the
 Phone: 1-613-726-0014 | Ottawa, Ontario   | Word of God. Please contact me
 EMail: [EMAIL PROTECTED] | Canada  K2A 1H7   | if you're concerned about Hell.
 http://FamilyRadio.com/   | http://Mielke.cc/bible/
 -- 
 To unsubscribe or change options: 
 https://lists.samba.org/mailman/listinfo/rsync
 Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: --backup leaves window where file doesn't exist.

2005-09-15 Thread Dave Mielke
[quoted lines by Chris Shoemaker on 2005/09/15 at 09:59 -0400]

what about when backup-dir is not on same filesystem
as original?  link will give EXDEV, I think.

Then, as I see it, rsync should go back to using rename but generate a warning.

-- 
Dave Mielke   | 2213 Fox Crescent | I believe that the Bible is the
Phone: 1-613-726-0014 | Ottawa, Ontario   | Word of God. Please contact me
EMail: [EMAIL PROTECTED] | Canada  K2A 1H7   | if you're concerned about Hell.
http://FamilyRadio.com/   | http://Mielke.cc/bible/
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: --backup leaves window where file doesn't exist.

2005-09-15 Thread Dave Mielke
[quoted lines by Dave Mielke on 2005/09/15 at 10:09 -0400]

[quoted lines by Chris Shoemaker on 2005/09/15 at 09:59 -0400]

what about when backup-dir is not on same filesystem
as original?  link will give EXDEV, I think.

Then, as I see it, rsync should go back to using rename but generate a warning.

I should, of course, have thought a bit more about my answer before presenting
it. The rename() system call itself doesn't support crossing a file system
boundary. What rsync does if the backup-dir is on a different file system is
copy the file over. In that case, therefore, it's already safe. The only
change, therefore, should be to replace the first rename() with link(). Since
both yield EXDEV, I doubt any more complex change is necessary.

-- 
Dave Mielke   | 2213 Fox Crescent | I believe that the Bible is the
Phone: 1-613-726-0014 | Ottawa, Ontario   | Word of God. Please contact me
EMail: [EMAIL PROTECTED] | Canada  K2A 1H7   | if you're concerned about Hell.
http://FamilyRadio.com/   | http://Mielke.cc/bible/
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: --backup leaves window where file doesn't exist.

2005-09-15 Thread Wayne Davison
On Thu, Sep 15, 2005 at 10:38:04AM -0400, Dave Mielke wrote:
 The only change, therefore, should be to replace the first rename()
 with link(). Since both yield EXDEV, I doubt any more complex change
 is necessary.

It's more complex than that.  If the temp file is not in a position
where it can be renamed into place, it is copied into place, which would
overwrite the backup without additional logic to ensure that the
newly-hard-linked destination file was removed first.  Also, the backup
calls are used for both backing up deleted files and updated files, so
the code would also need to be changed to have the delete callers remove
the destination file (as it does in the non-backup-calling case).
Finally, the code would need to handle systems where hard links don't
work by continuing to call rename() (since it is more efficient than
forcing all backups to be done with a file copy).  (At least, that's all
the side-effects I can think of off the top of my head.)

..wayne..
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: --backup leaves window where file doesn't exist.

2005-09-15 Thread Wayne Davison
On Thu, Sep 15, 2005 at 08:49:33AM -0700, Wayne Davison wrote:
 (At least, that's all the side-effects I can think of off the top of
 my head.)

At least one more was that backing up a directory without a --backup-dir
still requires using rename().

Attached is a patch that I think effects the change of using link()
instead of rename() when possible.  It is very minimally tested -- i.e.
it passes make test.  If you choose to try it out, run make proto
after applying the patch.

..wayne..
--- backup.c10 Jun 2005 17:57:18 -  1.45
+++ backup.c15 Sep 2005 16:55:56 -
@@ -52,43 +52,6 @@ char *get_backup_name(char *fname)
return NULL;
 }
 
-/* simple backup creates a backup with a suffix in the same directory */
-static int make_simple_backup(char *fname)
-{
-   int rename_errno;
-   char *fnamebak = get_backup_name(fname);
-
-   if (!fnamebak)
-   return 0;
-
-   while (1) {
-   if (do_rename(fname, fnamebak) == 0) {
-   if (verbose  1) {
-   rprintf(FINFO, backed up %s to %s\n,
-   safe_fname(fname),
-   safe_fname(fnamebak));
-   }
-   break;
-   }
-   /* cygwin (at least version b19) reports EINVAL */
-   if (errno == ENOENT || errno == EINVAL)
-   break;
-
-   rename_errno = errno;
-   if (errno == EISDIR  do_rmdir(fnamebak) == 0)
-   continue;
-   if (errno == ENOTDIR  do_unlink(fnamebak) == 0)
-   continue;
-
-   rsyserr(FERROR, rename_errno, rename %s to backup %s,
-   safe_fname(fname), safe_fname(fnamebak));
-   errno = rename_errno;
-   return 0;
-   }
-
-   return 1;
-}
-
 
 /
 Create a directory given an absolute path, perms based upon another directory
@@ -157,19 +120,75 @@ failure:
return -1;
 }
 
-/* robustly move a file, creating new directory structures if necessary */
-static int robust_move(char *src, char *dst)
+static int do_backup(char *src, char *dst, int use_rename,
+int make_missing_dirs, int nlinks)
 {
-   if (robust_rename(src, dst, 0755)  0  (errno != ENOENT
-   || make_bak_dir(dst)  0 || robust_rename(src, dst, 0755)  0))
+   int backup_errno;
+
+#ifndef SUPPORT_LINKS
+   use_rename = 1;
+#endif
+
+   while (1) {
+   /* CAUTION: the calling logic must take this into account! */
+   if (use_rename) {
+   if (do_rename(src, dst) == 0) {
+   if (nlinks  1) {
+   /* If file is hard-linked into backup 
dir,
+* rename() might succeed but do 
nothing! */
+   robust_unlink(src); /* Just in case... 
*/
+   }
+   break;
+   }
+   } else {
+#ifdef SUPPORT_LINKS
+   robust_unlink(dst);
+   if (do_link(src, dst) == 0)
+   break;
+#endif
+   }
+
+   backup_errno = errno;
+
+   if (make_missing_dirs  backup_errno == ENOENT
+ make_bak_dir(dst) == 0)
+   continue;
+
+   /* cygwin (at least version b19) reports EINVAL */
+   if (backup_errno == ENOENT || backup_errno == EINVAL)
+   break;
+
+   if (backup_errno == EISDIR  do_rmdir(src) == 0)
+   continue;
+   if (backup_errno == ENOTDIR  do_unlink(src) == 0)
+   continue;
+
+   rsyserr(FERROR, backup_errno, backup of %s to %s,
+   safe_fname(src), safe_fname(dst));
+   errno = backup_errno;
return -1;
+   }
+
+   if (verbose  1) {
+   rprintf(FINFO, backed up %s to %s\n,
+   safe_fname(src), safe_fname(dst));
+   }
return 0;
 }
 
+/* simple backup creates a backup with a suffix in the same directory */
+static int make_simple_backup(char *fname, int use_rename)
+{
+   char *fnamebak = get_backup_name(fname);
+
+   if (fnamebak  do_backup(fname, fnamebak, use_rename, 0, 0) == 0)
+   return 1;
+   return 0;
+}
 
 /* If we have a --backup-dir, then we get here from make_backup().
- * We will move the file to be deleted into a parallel directory tree. */
-static int keep_backup(char *fname)
+ * We will move/link the file into a parallel directory tree. */
+static int keep_backup(char *fname, int use_rename)
 {
STRUCT_STAT st;
struct file_struct *file;
@@ 

Re: --backup leaves window where file doesn't exist.

2005-09-15 Thread Wayne Davison
On Thu, Sep 15, 2005 at 10:15:51AM -0700, Wayne Davison wrote:
 Attached is a patch that I think effects the change of using link()
 instead of rename() when possible.

That patch had a couple problems with dealing with boundary conditions
(such as cross-device cases and a filesystem that doesn't support hard
links on an OS that does).  Here's a link to a better one:

http://opencoder.net/more-linking.patch

..wayne..
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


writefd_unbuffered failed to write 32768 bytes: phase unknown

2005-09-15 Thread Dag Wieers
Hi,

I receive this error when rsyncing. I suspect this is because of network 
problems. But the 'Invalid argument' and 'phase unknown' makes it very 
unobvious.

...
packages/scribus/scribus-1.2.3-0.rf.src.rpm
Read from remote host rsync.sw.be: Invalid argument
rsync: writefd_unbuffered failed to write 32768 bytes: phase unknown 
[sender]: Broken pipe (32)
rsync: connection unexpectedly closed (796308 bytes received so far) 
[sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(434)

Kind regards,
--   dag wieers,  [EMAIL PROTECTED],  http://dag.wieers.com/   --
[all I want is a warm bed and a kind word and unlimited power]
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: [rsync] NetEx HyperIP

2005-09-15 Thread webster
Yes, I have received quite a bit of information on the subject.
Thanks.


Gary R. Webster




Pheneger, Steve [EMAIL PROTECTED]
Sent by: [EMAIL PROTECTED]
09/15/05 01:05 AM

 
To: rsync@lists.samba.org
cc: 
Subject:[rsync] NetEx HyperIP

Hello!

I'm looking to see if Webster received an answer to his question with
supporting rsync data replication over a distance of 10,000 miles.

Thank you,

Steve


Steve Pheneger

Program Manager

NetEx

Toll Free: 888-604-5563

Direct: 763-694-4309

FAX: 763-694-0669

mailto:[EMAIL PROTECTED] [EMAIL PROTECTED]

http://www.netex.com/ www.netex.com

HyperIP = Data Replication Optimization (DRO)


--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


rsync file corruption

2005-09-15 Thread Andy Wong

Hi,

I encountered a weird file corruption problem with rsync.
I have a perl script that generates and writes a data file to disk, then 
rsyncs the file to a remote machine. A perl script running on the remote 
machine periodically reads in the data file.


However, occasionally the remote script will barf while processing the 
data file, and I tracked down the problem to file corruption.


I suspected rsync was the problem, so I changed the generation script to 
rsync to a different file name, then remotely rename/ overwrite the file 
with the new data file (using mv over ssh). And the file corruption 
problem is gone.


Now from what I understand, rsync supposedly do the same thing: rsyncs 
to a temporary file, then renames it. So what went wrong?


Thanks,
Andrew
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


[Bug 3099] Please parallelize filesystem scan

2005-09-15 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=3099





--- Additional Comments From [EMAIL PROTECTED]  2005-09-15 13:49 ---
Pardon me for being dense, but how could it possibly require a change to the
rsync protocol for the second host in the sequence to pre-scan its filesystem,
so that that data is available when needed?

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


[Bug 3099] Please parallelize filesystem scan

2005-09-15 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=3099





--- Additional Comments From [EMAIL PROTECTED]  2005-09-15 16:23 ---
Created an attachment (id=1448)
 -- (https://bugzilla.samba.org/attachment.cgi?id=1448action=view)
One possible way to reorder the checksum computation.

 how could it possibly require a change to the rsync protocol for the
 second host in the sequence to pre-scan its filesystem, so that that
 data is available when needed?

The only way to know what to scan is to look at the file list from the sender
(since the receiver usually doesn't know anything other than the destination
directory, and options such as -R, --exclude, and --files-from can radically
limit what files need to be scanned).

I suppose it would be possible for the receiver to compute the full-file
checksums as the file list is arriving from the sender (yes, the sender sends
the list incrementally as it is created), but the code currently doesn't know
if the destination spec is a file or a directory until after it receives the
file list, so the code would need to be made to attempt a chdir to the
destination arg and to skip the pre-caching if that doesn't work.

One bad thing about this solution is that we really should be making the
sending side not pre-compute the checksums before the start of the transfer
phase (to be like the generator, which computes the checksums while looking for
files to transfer). Computing them during the transfer makes it more likley
that the file's data in the disk cache will be able to be re-used when a file
needs to be updated. Thus, changing the receiving side to pre-compute the
checksums before starting the transfer seems to be going in the wrong direction
(though it might speed up a large transfer where few files were different, it
might also slow down a large transfer where many files were changed).

The attached patch implements a simple pre-scan that works with basic options.
It could be improved to handle things like --compare-dest better, but I think
it basically works.  If you'd care to run some speed tests, maybe you could
persuade me that this kluge would be worth looking at further (I'm not
considering it at the moment).

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: [Bug 3099] Please parallelize filesystem scan

2005-09-15 Thread Chris Shoemaker
On Thu, Sep 15, 2005 at 04:23:24PM -0700, [EMAIL PROTECTED] wrote:
 https://bugzilla.samba.org/show_bug.cgi?id=3099
 
 
 
 
 
 --- Additional Comments From [EMAIL PROTECTED]  2005-09-15 16:23 ---
 Created an attachment (id=1448)
  -- (https://bugzilla.samba.org/attachment.cgi?id=1448action=view)
 One possible way to reorder the checksum computation.
 
  how could it possibly require a change to the rsync protocol for the
  second host in the sequence to pre-scan its filesystem, so that that
  data is available when needed?
 
 The only way to know what to scan is to look at the file list from the sender
 (since the receiver usually doesn't know anything other than the destination
 directory, and options such as -R, --exclude, and --files-from can radically
 limit what files need to be scanned).
 
 I suppose it would be possible for the receiver to compute the full-file
 checksums as the file list is arriving from the sender (yes, the sender sends
 the list incrementally as it is created), but the code currently doesn't know
 if the destination spec is a file or a directory until after it receives the
 file list, so the code would need to be made to attempt a chdir to the
 destination arg and to skip the pre-caching if that doesn't work.
 
 One bad thing about this solution is that we really should be making the
 sending side not pre-compute the checksums before the start of the transfer
 phase (to be like the generator, which computes the checksums while looking 
 for
 files to transfer). Computing them during the transfer makes it more likley
 that the file's data in the disk cache will be able to be re-used when a file
 needs to be updated. Thus, changing the receiving side to pre-compute the
 checksums before starting the transfer seems to be going in the wrong 
 direction
 (though it might speed up a large transfer where few files were different, it
 might also slow down a large transfer where many files were changed).

IMHO, in general, optimizing for the few-changes (small delta) case
is the right thing to do.  Rsync's utility diminishes anyway as delta
increases, so there's no reason not to make efficiency increase with
increasing delta.

-chris

 
 The attached patch implements a simple pre-scan that works with basic options.
 It could be improved to handle things like --compare-dest better, but I think
 it basically works.  If you'd care to run some speed tests, maybe you could
 persuade me that this kluge would be worth looking at further (I'm not
 considering it at the moment).
 
 -- 
 Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
 --- You are receiving this mail because: ---
 You are the QA contact for the bug, or are watching the QA contact.
 -- 
 To unsubscribe or change options: 
 https://lists.samba.org/mailman/listinfo/rsync
 Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Re: writefd_unbuffered failed to write 32768 bytes: phase

2005-09-15 Thread Ray Harder
Dag Wieers dag at wieers.com writes:

 
 Hi,
 
 I receive this error when rsyncing. I suspect this is because of network 
 problems. But the 'Invalid argument' and 'phase unknown' makes it very 
 unobvious.
 
   ...
   packages/scribus/scribus-1.2.3-0.rf.src.rpm
   Read from remote host rsync.sw.be: Invalid argument
   rsync: writefd_unbuffered failed to write 32768 bytes: phase unknown
[sender]: Broken pipe (32)
   rsync: connection unexpectedly closed (796308 bytes received so far) 
 [sender]
   rsync error: error in rsync protocol data stream (code 12) at io.c(434)
 
 Kind regards,


Dag:

I am totally ignorant of what this could be or how to help, but I have exactly
the  same problem while booted from an internal Mac PowerBookG4 and rsyncing
from one local firewire volume to another local firewire volume daisy chained to
it, so I think that definitely rules out a network issue, no?

I have been really stumped on this one.

If your figure it out, please share.

Here is the output, which as you can see is fundamentally the same as yours.

Terminal output

sudo rsync -axEHuvv --delete --inplace --whole-file /Volumes/MacXHD/
/Volumes/MacintoshxHD

building file list ... 
[sender] expand file_list to 131072 bytes, did move
[sender] expand file_list to 262144 bytes, did move
[sender] expand file_list to 524288 bytes, did move
[sender] expand file_list to 1048576 bytes, did move
[sender] expand file_list to 2097152 bytes, did move
[sender] expand file_list to 4194304 bytes, did move
done
rsync: writefd_unbuffered failed to write 86 bytes: phase unknown [receiver]:
Broken pipe (32)
rsync error: error in rsync protocol data stream (code 12) at
/SourceCache/rsync/rsync-20/rsync/io.c(909)
Bus error

_End Terminal output___

Ideas?

Thanks
Ray


-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


CVS update: rsync

2005-09-15 Thread Wayne Davison

Date:   Thu Sep 15 15:36:27 2005
Author: wayned

Update of /data/cvs/rsync
In directory dp.samba.org:/tmp/cvs-serv6648

Modified Files:
rsync.yo rsync.1 
Log Message:
Removed conflicting ssh citation in the SETUP section.


Revisions:
rsync.yo1.299 = 1.300
http://www.samba.org/cgi-bin/cvsweb/rsync/rsync.yo?r1=1.299r2=1.300
rsync.1 1.310 = 1.311
http://www.samba.org/cgi-bin/cvsweb/rsync/rsync.1?r1=1.310r2=1.311
___
rsync-cvs mailing list
rsync-cvs@lists.samba.org
https://lists.samba.org/mailman/listinfo/rsync-cvs


CVS update: rsync

2005-09-15 Thread Wayne Davison

Date:   Thu Sep 15 17:19:30 2005
Author: wayned

Update of /data/cvs/rsync
In directory dp.samba.org:/tmp/cvs-serv4745

Modified Files:
NEWS 
Log Message:
Mention doc changes.


Revisions:
NEWS1.324 = 1.325
http://www.samba.org/cgi-bin/cvsweb/rsync/NEWS?r1=1.324r2=1.325
___
rsync-cvs mailing list
rsync-cvs@lists.samba.org
https://lists.samba.org/mailman/listinfo/rsync-cvs


CVS update: rsync

2005-09-15 Thread Wayne Davison

Date:   Thu Sep 15 22:25:55 2005
Author: wayned

Update of /data/cvs/rsync
In directory dp.samba.org:/tmp/cvs-serv25508

Modified Files:
receiver.c 
Log Message:
Added a comment.


Revisions:
receiver.c  1.164 = 1.165
http://www.samba.org/cgi-bin/cvsweb/rsync/receiver.c?r1=1.164r2=1.165
___
rsync-cvs mailing list
rsync-cvs@lists.samba.org
https://lists.samba.org/mailman/listinfo/rsync-cvs


CVS update: rsync

2005-09-15 Thread Wayne Davison

Date:   Thu Sep 15 22:27:14 2005
Author: wayned

Update of /data/cvs/rsync
In directory dp.samba.org:/tmp/cvs-serv26323

Modified Files:
rsync.yo rsync.1 
Log Message:
Mention that the dir used with --delay-updates cannot be on a
different filesystem from the destination files.


Revisions:
rsync.yo1.300 = 1.301
http://www.samba.org/cgi-bin/cvsweb/rsync/rsync.yo?r1=1.300r2=1.301
rsync.1 1.311 = 1.312
http://www.samba.org/cgi-bin/cvsweb/rsync/rsync.1?r1=1.311r2=1.312
___
rsync-cvs mailing list
rsync-cvs@lists.samba.org
https://lists.samba.org/mailman/listinfo/rsync-cvs