[Q] multicasting product ?
Hi everybody! Now that RSYNC has RSYNC+ included a good usage would be to use RSYNC+ to gather update-date, then multicast that on your hosts and process it. So my question is: does anyone know of a product which does reliable multicasting? (source available would be preferred) Simple pointers are appreciated; if noone has one I'm thinking about writing one myself. Thanks for all help! Phil - This message is RSA-encrypted: n=33389, e=257
Problems with rsync on Win2K to Solaris
Hi, I am having trouble getting rsync (v2.5.0 with the latest version of cygwin) to work between my Win2K (SP2) and my Solaris 2.6 server. Both machines are running the same version of rsync. Each time I run rsync, I get the following error: - C:\rsyncrsync server1:/tmp/file . Terminal read: The parameter is incorrect. rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(139) rsync: read error: Connection reset by peer rsync error: error in rsync protocol data stream (code 12) at io.c(151) Debugging stuff I tried: --- * Ensured the rsh command works properly when executed from the Win2K box to the Solaris server (rsh server1 ls - produces correct output). * Ensured the Administrator user is valid on the Solaris server * Tried every combination of quotes, slashes, etc * Tried to rsync in both directions Also, when I specify the location of rsh on the Solaris box, I get the following error: C:\rsyncrsync --rsh='/bin/rsh' server1:/tmp/file . Failed to exec /bin/rsh : No such file or directory rsync error: error in IPC code (code 14) at util.c(137) rsync: read error: Connection reset by peer rsync error: error in rsync protocol data stream (code 12) at io.c(151) I have tried three Win2K boxes and two Solaris boxes - all end up with the same problem. I have *not* tried to install rsync as a daemon on the Win2K box because I don't think that is necessary. Any ideas? I am really getting frustrated :-) -Ron
Re: [Q] multicasting product ?
So my question is: does anyone know of a product which does reliable multicasting? (source available would be preferred) At our company, we have a mrsync running for a couple of months now. mrsync is to transfer files to many machines at the same time using UDP and multicast. I attached at the end of this message the excerpt from the docs of mrync. If this is what you need, we can contribute this program to the openSource. HP +-+ | MRSYNC vs RSYNC | +-+ mrsync is a utility that transfers a bunch of files from a master machine to multiple target machines simultaneously by using the multicasting capability in the UNIX system. The name 'mrsync' is inspired by the popular utility 'rsync' for synchronizing files between two machines. However, beyond this similarity in the functionality, mrsync is fundamentally different from rsync in two areas. (1) rsync uses TCP while mrsync needs UDP in order to use the multicasting part of UNIX's socket communication. The former limits the data commuinication to one-to-one-machine whereas the latter allows one-to-many. UDP has no built in flow control. As a result, the major part of mrsync (more precisely, the multicaster and multicatcher), is devoted to synchronizing the data flow. (2) For a given file, rsync transfers (optionally) only those parts in the file that are different in the two versions on the master and the target machine. This saves time and is accomplished by using a rolling checksum algorithm by Andrew Trigell. mrsync, in contrast, transfers the whole content of a file to all targets in one time. +---+ | HISTORY OF MRSYNC | + --+ The project of mrsync stemmed from the prospective necessity to transfer many files to hundreds of machines running Linux at Renaissance Technologies Corp. Looking into the Open Source Community, we found a preliminary utility codes of multicasting written by Aaron Hillegass. Many unsuccessful test-runs on a huge amount of data files, however, led us to embark on an overhaul on the codes. Most of the following items were inherited and bug-fixed from the original codes. * The low level functions that interact with UNIX's multicasting sockets. * Meta_data -- the essential info about a file which the master machine will first transmit to the target machines. * Division of a file into many 'pages'. * The idea of maintaining a missing page flag. * The idea of a multicaster and multicatcher loop -- In this mrsync, we develop two new critical elements: flow-control message communication conducted by the multicaster, and a four-state page reader (processor) in the multicatcher. The former is to synchronize the task each machine is performing. For example, the master will not start sending the pages of a file unless all machines have acknowledged the completion of openning the disk i/o for the file. In order to accomodate these elements, the codes have been changed significantly from the original version. For example, the multicatcher now never asks for slowing down. And multicaster sends data on a file-by-file basis. The file integrity is achieved by orchestrating the data flow which is closely monitored and conducted by the master machine. As of today, mrsync has been in full use at Renaissance on a daily basis. +--+ | TYPICAL RUNNING TIME | +--+ 25 minutes for a group of files whose total size amounts to 4.6Gb. (This data is obtained from running on 5 SUN machines with Solaris 8 on an Ethernet LAN whose bandwidth is 1Gbits/sec.)
Re: [Q] multicasting product ?
On Wed, Dec 19, 2001 at 08:51:46AM -0500, HP Wei wrote: So my question is: does anyone know of a product which does reliable multicasting? (source available would be preferred) At our company, we have a mrsync running for a couple of months now. mrsync is to transfer files to many machines at the same time using UDP and multicast. I attached at the end of this message the excerpt from the docs of mrync. If this is what you need, we can contribute this program to the openSource. I, for one, have been looking for a long time for an open source multicast rsync like application to handle distributing updated applications (such as Perl) to many machines. Mrsync sounds like it just might do the trick. -- Jim Littlefield I was in the grocery store. I saw a sign that said 'pet supplies.' So I did. Then I went outside and saw a sign that said 'compact cars'... - Steven Wright
Re: rsync server over SSH [includes code patches]
On Tue, 18 Dec 2001, Martin Pool wrote:
On 4 Dec 2001, JD Paul [EMAIL PROTECTED] wrote:
Here's a new version of my rsync-server-over-remote-shell patch:
This looks good. My main reservation is that it makes it even harder
to explain how rsync works, but I think the increase in flexibility
justifies it.
I certainly understand that concern! (I'm sure I'm not the only one
who sometimes thinks that it would be better to split rsync into
several different programs [even if they were the same binary] with
different semantics to make things clearer -- all using the same rsync
engine under the hood.)
I'd like to get some of the smaller patches in to stabilize 2.5 before
we do this.
Sounds good to me.
P.S. Question -- is the enum { } of all the OPT_ variables in
options.c necessary any more? It looks like it's not.
Why? They're used to link from long_options to the cases in
parse_arguments(). Some of them might be obsolete now, but it doesn't
seem worthwhile to take them out.
I don't know how I missed that. I must have been writing late at
night. :-)
JD
Re: reverse name lookup failed
Looks like nobody has answered this yet. I haven't tested it or looked at the code, but according to my reading of the rsyncd.conf man page, that's not a supported syntax for hosts allow. If you specify a /netmask, the first portion has to be an IP address and not a hostname. - Dave Dykstra On Fri, Dec 14, 2001 at 03:46:29PM -0500, Kornmesser, Craig wrote: Any time I try and setup a hosts allow option in the modules I get the following errors: I'm running this from the client. rsync -rv ws2743::test3 /pw/prog/BOS/stuff @ERROR: access denied to test3 from unknown (::3) rsync: connection unexpectedly closed (62 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(139) I get the following from the log file on the rsync server: 2001/12/14 14:50:25 [5121] rsync denied on module test1 from unknown (::3) 2001/12/14 14:56:05 [5203] reverse name lookup failed 2001/12/14 14:56:05 [5203] rsync: forward name lookup for failed: host/servname not known 2001/12/14 14:56:05 [5203] rsync denied on module test2 from unknown (::3) My /etc/rsync.conf file looks like the following: log file = /pw/prog/BOS/log/rsyncd.log pid file = /pw/prog/BOS/log/rsyncd.pid [test] path = /pw/prog/BOS/stuff comment = My Very Own Rsync Server list = yes [test1] path = /pw/prog/BOS/stuff comment = My Very Own Rsync Server list = yes hosts allow = ws2743/255.255.0.0 [test2] path = /pw/prog/BOS/stuff comment = My Very Own Rsync Server list = yes hosts allow = ws3835/255.255.0.0 If I run with module test that doesn't have the hosts allow option I'm able to transfer files. Any ideas?
Re: [Q] multicasting product ?
On Wed, Dec 19, 2001 at 12:11:46PM +0100, Ph. Marek wrote: Now that RSYNC has RSYNC+ included a good usage would be to use RSYNC+ to gather update-date, then multicast that on your hosts and process it. Note that the rsync+ functionality as currently present in CVS does not work; I sent a final patch to Martin Pool for review that makes it work properly (my colleague says he's had no problems so far). Hopefully it will make it into the next stable release. mrsync looks very interesting, too. It's always nice to have more options to choose from. -- Jos Backus _/ _/_/_/Santa Clara, CA _/ _/ _/ _/ _/_/_/ _/ _/ _/_/ [EMAIL PROTECTED] _/_/ _/_/_/use Std::Disclaimer;
Re: rsync-2.5.0 getaddrinfo in AIX4.3?
On Wed, Dec 19, 2001 at 10:20:52AM +, Terry Raggett wrote:
On Dec 18, 22:37, [EMAIL PROTECTED] wrote:
Subject: Re: rsync-2.5.0 getaddrinfo in AIX4.3?
On Tue, Dec 18, 2001 at 03:53:07PM +, Terry Raggett wrote:
Anyone have problems with getaddrinfo under AIX4.3? I have built 2.5.0
under
linux, HP and VPP without too much in the way of problems. However, AIX4.3
version builds OK but doesn't execute getaddrinfo correctly.
What error do you get? I just build the latest CVS on AIX 4.3.2 and
Solaris 8/SPARC And did a transfer from AIX-Solaris with no problems.
I get the following errors whether I use host or IP addressing. I'm
waiting to set-up a test system to play around with a full debug version.
Rsync-2.4.6 works OK but this has a very different socket.c
metis{~syr/rsync}198 %rsync_AIX -q -W --address=metis --timeout=1 --port=10873
catenv frodo::myftp/
rsync: getaddrinfo: frodo 10873: Host not found
rsync error: error in socket IO (code 10) at clientserver.c(84)
metis{~syr/rsync}199 %ypcat hosts | grep frodo
136.156.194.129 frodo
metis{~syr/rsync}200 %rsync_AIX -q -W --address=metis --timeout=1 --port=10873
catenv 136.156.194.129::myftp/
rsync: getaddrinfo: 136.156.194.129 10873: Host not found
rsync error: error in socket IO (code 10) at clientserver.c(84)
metis{~syr/rsync}201 %
Is this with the latest CVS? If not, please try with that and report
back.
--
albert chin ([EMAIL PROTECTED])
Re: rsync *Still* Copying All Files?
On Wed, Dec 19, 2001 at 04:36:21PM -0500, Mack, Daemian wrote: Hi, all. This is my first post to the list, so please forgive me if this is an old issue. I scoured the mailing list archives by hand and could find no mention of it, and the official FAQ-o-matic mentions it, but doesn't offer a fix that works for me. I've set up an rsync daemon on Windows NT4 by installing the latest cygwin. Using the cygwin1.dll and rsync.exe, I'm able to connect successfully from other NT4 and Windows 2000 machines. I can get the rsync server to let me pull files over the net to my local data to synchronize. However, I've noticed that on the clients, *all* files are being copied over. Here's the command line I'm issuing: C:\temprsync --verbose --progress --stats --compress --recursive --times 10.8.1.57::nmap /cygdrive/c/temp/nmap I'm using a source distribution of nmap just to test with. When I started noticing this error, I placed a 37MB file inside the rsyncable nmap area to emphasize the amount of time spent syncing. The progress switch allows me to watch a realtime progress counter as the files are processed. It takes several minutes for that 37MB file to come over, and the stats at the end hold up the theory that ~40MB of data has been transferred. While troubleshooting, I added the times switch to see if preserving timestamps on the files would solve anything; it didn't. I've made sure both client and server are set to the same time. The only thing I've noticed that could be a possible clue is that the files are different *sizes* on each disk (because of NTFS cluster size hijinks, etc.), so, just to be paranoid, I tried using checksum to find file diffs. Again, no dice. No matter what I try, rsyncing causes all files to be copied over the wire onto the local box. I can't figure this out. I don't know if rsync-for-cygwin's algorithm for determining file differences is lacking somehow, or if something about this data is tricking rsync into thinking changes have occurred, or if I'm just doing something dumb. Any ideas? Try --modify-window 2. I'm not sure that will work because it sounds like you're copying between two filesystem types that are the same, but the option was added to copy between PC filesystems that have a timestamp granularity of 2 seconds and Unix filesystems that have a granularity of 1 second. - Dave Dykstra
RE: rsync *Still* Copying All Files?
Try --modify-window 2. I'm not sure that will work because it sounds like you're copying between two filesystem types that are the same, but the option was added to copy between PC filesystems that have a timestamp granularity of 2 seconds and Unix filesystems that have a granularity of 1 second. W! That did it! Wouldn't you know it'd be the one switch I didn't think to try? Thanks! Daemian Mack
Re: rsync *Still* Copying All Files?
On Wed, Dec 19, 2001 at 05:03:51PM -0500, Mack, Daemian wrote: Try --modify-window 2. I'm not sure that will work because it sounds like you're copying between two filesystem types that are the same, but the option was added to copy between PC filesystems that have a timestamp granularity of 2 seconds and Unix filesystems that have a granularity of 1 second. W! That did it! Wouldn't you know it'd be the one switch I didn't think to try? The question is, why does it work? Are you indeed copying between two NTFS filesystems, with rsync running under Windows cygwin on both sides? I would have thought that would result in matching timestamps granularity on both sides so rsync would always end up comparing the same values. - Dave Dykstra
excluding a file.
This is what I have tried to use to back a usr partition to a backup drive. rsync copied most then terminates with a mknod var/run/log: Invalid argument rsync error: partial transfer (code 23) at main.c (537) I have tried different combination if var/run/log (with and with the trailing \) and other combinations. I want to remove this file from the rsync list to copy. /var is linked to /usr/var Also happens with pre1, pre2, pre3 versions. rsync-2.5.0/rsync -axW --exclude-from=exclude.txt /usr/ /backup/usr/ exclude.txt: /usr/src /usr/var/run/log/ -- Ian Kettleborough [EMAIL PROTECTED]
