Re: RSYNC server setup
On 16 Feb 2001, John Poltorak [EMAIL PROTECTED] wrote:
On OS/2 there is a PSTAT command as well a number of other third-party
apps which mimic ps. Using one of these shows no sign of RSYNC.
I guess 'netstat -s' would also provide some evidence of its presence
if port 873 was in use but it isn't.
If an error occurs, it will be written to the system logs. On Unix,
this is something like /var/log/daemon. I'm not sure if syslog is
supported on OS/2. If it's not, you should try something like this in
the rsyncd.conf
log file = /log/rsync.log
Also, it may be that the program is not correctly finding the
configuration file, and perhaps you should start it with
rsync --config /etc/rsyncd.conf --daemon
What would happen if I took the 'exit' out of the following code from
socket.c ?
/
become a daemon, discarding the controlling terminal
/
void become_daemon(void)
{
int i;
if (fork()) {
_exit(0);
}
What normally happens is that when running as a daemon, rsync starts a
background task, and then the task attached to the terminal. This is
the normal Unix method of going into the background. So, on Unix,
this would mean that your terminal would stay blocked, but otherwise
nothing else would change. There are also some consequences about the
daemon remaining in the session group of the shell, which means (I
think) that signals will kill it.
On OS/2, I don't know. I would be surprised if it fixes the problem,
but then I know the OS/2 process model is very different.
If you have an inetd equivalent on your system, then the simplest
solution may be to run rsync from that, as described in the manual.
--
Martin Pool, Human Resource
Linuxcare. Inc. +61 2 6262 8990
[EMAIL PROTECTED], http://linuxcare.com.au/
Linuxcare. Putting Open Source to work.
jrsync at sourceforge
Hi, I have managed to get a project setup for jrsync at sourceforge.net. If anybody is interested in particapting in the project, you are very much welcome to do so. The web page is: https://sourceforge.net/projects/jrsync/ There isn't anything there yet, though hopefully that will change as I manage to find to time and as more people participate in the development of this project. At this stage what is important is the porting of the core algorithms and adaption of the architecture to an object orientated language. Andre
RE: Peer reset
Sorry, it appears I was chasing my tail. After a day of configuration, testing, and looking through code to understand it, I was simply editing out of my conciousness a read only error on the client side. However, it may be a good idea to put some of those errors on the server side, at least on a command line flagged basis. I'll try to take a look through the code... Thanks, Alex Peyser -Original Message- From: Alex Peyser [mailto:[EMAIL PROTECTED]] Sent: Thursday, March 15, 2001 7:24 PM To: '[EMAIL PROTECTED]' Subject: Peer reset I've got rsync 2.4.6 protocol version 24 running as an inetd server under solaris 2.7. I've got set the modules set up, authentication goes ok. Then I get a reverse dns warning... which is fine with me, the security isn't running through rsync anyhow (and in the rsync source code that appears to be a warning that doesn't cause a failure). Then my client get a peer reset error. That happens whether or not I'm coming in over a wan, lan, or through localhost. I get no error in syslog on the server error, just the read error in read_timeout on the client side. Anyone have any idea ? Thanks, Alex Peyser
RE: auth failed on module testshare
Have you checked your syslog output ? I had the same thing with permissions on /etc/rsyncd.secrets, where it was world readable. Try strict modes = false, and if that works you've got permission problems (or simple ls -l). The syslog will also put out comments that rsync is ignoring rsycd.secrets Alex Peyser -Original Message- From: Oliver Pitterling [mailto:[EMAIL PROTECTED]] Sent: Friday, March 16, 2001 1:56 PM To: [EMAIL PROTECTED] Subject: auth failed on module testshare Hi there, I have a RH7 testsystem setup with rsync -daemon. Now, I am trying to login via a FreeBSD 4.1 system by issuing: export RSYNC_PASSWORD=opi8 rsync -avz opi@felix::testshare /home/opi However, i receive following response: @ERROR: auth failed on module testshare All rsync'ing works fine, if I remove the password options in /etc/rsyncd.conf (i.e. auth and secrets lines). Here is my rsyncd.conf: ## uid = nobody gid = nobody use chroot = no max connections = 1 syslog facility = local5 pid file = /etc/rsyncd.pid [testshare] path = /home/opi/rsync comment = TestShare Repository auth users = opi secrets file = /etc/rsyncd.secrets ## and my /etc/rsyncd.secrets file: tridge:mypass susan:herpass opi:opi8 ## Any ideas? I am running the rsync command in a SSH2 Telnet session via the SecureCRT client app. Thanks, Oliver [EMAIL PROTECTED]
Re: LFS support on AIX 4.3.3
On Fri, Mar 16, 2001 at 04:39:18PM +1100, Martin Pool wrote: On 15 Mar 2001, "Willeat, Todd" [EMAIL PROTECTED] wrote: http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/aixprggd/genprogc/prg_lr_files.htm explains how to enable large file support on AIX. The fillowing line was added to the config.h file: #define _LARGE_FILES 1 I committed this to rsync.h (v1.98), so it should be in 2.4.7. It's unconditionally defined. If somebody on AIX could build from anoncvs and confirm that it works that would be great. (Mail me if you would like help with CVS.) ICK! Please undo your change. Defining _LARGE_FILES as is done in rsync.h means everyone will be impacted (Solaris, HP-UX, IRIX, Linux, FreeBSD, Tru64 UNIX, ...). Discussion has already occurred on this list about LFS support: http://lists.samba.org/pipermail/rsync/2000-May/002182.html http://lists.samba.org/pipermail/rsync/2000-May/002199.html If the current configure test is broken for AIX, the autoconf test should be fixed. Also, Andrew is probably going to move away from the explicit use of 64-bit data types (like off64_t). Paul Eggert has written an autoconf macro, AC_SYS_LARGEFILE, which is now a *standard* part of autoconf in CVS, that detects what to add to CFLAGS and LDFLAGS to get LFS support on your OS. I think this is the direction we should head. -- albert chin ([EMAIL PROTECTED])
RE: LFS support on AIX 4.3.3
I agree that it would be better to fix the autoconf rather than risk breaking things on other OSes. I have recompiled a few more times with the -D_LARGE_FILES option and I think everything is working correctly now. The dry run gave no errors and so far the transfer is working fine. My compiler did produce an informational message about increasing MAXMEM to a value greater than 2048, but if my current transfer finishes OK, I'm not going to mess with it... -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, March 16, 2001 3:24 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: LFS support on AIX 4.3.3 On Fri, Mar 16, 2001 at 04:39:18PM +1100, Martin Pool wrote: On 15 Mar 2001, "Willeat, Todd" [EMAIL PROTECTED] wrote: http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/aixprggd/genprogc/prg_lr_ files.htm explains how to enable large file support on AIX. The fillowing line was added to the config.h file: #define _LARGE_FILES 1 I committed this to rsync.h (v1.98), so it should be in 2.4.7. It's unconditionally defined. If somebody on AIX could build from anoncvs and confirm that it works that would be great. (Mail me if you would like help with CVS.) ICK! Please undo your change. Defining _LARGE_FILES as is done in rsync.h means everyone will be impacted (Solaris, HP-UX, IRIX, Linux, FreeBSD, Tru64 UNIX, ...). Discussion has already occurred on this list about LFS support: http://lists.samba.org/pipermail/rsync/2000-May/002182.html http://lists.samba.org/pipermail/rsync/2000-May/002199.html If the current configure test is broken for AIX, the autoconf test should be fixed. Also, Andrew is probably going to move away from the explicit use of 64-bit data types (like off64_t). Paul Eggert has written an autoconf macro, AC_SYS_LARGEFILE, which is now a *standard* part of autoconf in CVS, that detects what to add to CFLAGS and LDFLAGS to get LFS support on your OS. I think this is the direction we should head. -- albert chin ([EMAIL PROTECTED])
re: files can only be read when world-readable
On 15 Mar 2001, pete lindsay [EMAIL PROTECTED] wrote: Hi, Thanks very much for your reply. I am running rsync daemon on the server. I notice that the owner of rsync is root. rsync daemon is running on the server as owned by the root user. The files I want are all owned by the users tags. I note that unless the files have global read permissions, my rsync run wont get them. If rsyncd is running as root, then by default it changes to nobody/nobody before accepting a connection. This explains why only your files need to be publicly readable to be retrieved. What you probably want to do is change the rsyncd.conf to specify the user and group who should be used for that module. For security reasons this should not be root unless it's absolutely necessary. For example [mymodule] path = whatever uid = tags gid = staff You can set different users for each module. Note that the parameter name is 'uid' even if you're actually specifying a username rather than a uid. Hope that helps, -- Martin Pool, Human Resource Linuxcare. Inc. +61 2 6262 8990 [EMAIL PROTECTED], http://linuxcare.com.au/ Linuxcare. Putting Open Source to work.
Re: LFS support on AIX 4.3.3
On 16 Mar 2001, [EMAIL PROTECTED] wrote: ICK! Please undo your change. Defining _LARGE_FILES as is done in rsync.h means everyone will be impacted (Solaris, HP-UX, IRIX, Linux, FreeBSD, Tru64 UNIX, ...). Discussion has already occurred on this list about LFS support: OK, I'll take it out. Thankyou for catching this. If the current configure test is broken for AIX, the autoconf test should be fixed. Also, Andrew is probably going to move away from the explicit use of 64-bit data types (like off64_t). Paul Eggert has written an autoconf macro, AC_SYS_LARGEFILE, which is now a *standard* part of autoconf in CVS, that detects what to add to CFLAGS and LDFLAGS to get LFS support on your OS. I think this is the direction we should head. That sounds like a more clean solution. So, that will just automatically make off_t be as long as possible? -- Martin Pool, Human Resource Linuxcare. Inc. +61 2 6262 8990 [EMAIL PROTECTED], http://linuxcare.com.au/ Linuxcare. Putting Open Source to work.
Re: LFS support on AIX 4.3.3
Speaking of this, does anyone know a portable way to get printf to
handle off_t values when they may be larger that a long? Is there
none?
I normally write
off_t foo = ;
printf("seek to %ld\n", (long) foo);
accepting that for long values on some platforms it will just be
wrong. Is there a better solution that's not gcc or glibc-specific?
From what I've seen of the standards, there is no standard format
longer than %ld, but off_t can be longer than long.
I guess perhaps I could use autoconf to work out appropriate format
strings and typecasts:
#define OFF_T_PRINT_FORMAT "%lld"
#define OFF_T_PRINT_CAST(long long)
printf("seek to " OFF_T_PRINT_FORMAT, OFF_T_PRINT_CAST foo);
I don't care so much about this in rsync, but librsync trace and error
messages do this all the time, and it's bothered me that they might
give the wrong values.
--
Martin Pool, Human Resource
Linuxcare. Inc. +61 2 6262 8990
[EMAIL PROTECTED], http://linuxcare.com.au/
Linuxcare. Putting Open Source to work.
Re: Linux 2.2.16 through 2.2.18preX TCP hang bug triggered by rsync
On 28 Feb 2001, Martin Pool [EMAIL PROTECTED] wrote: What I don't see is how we could recode this to avoid the zero window without losing a lot of the pipelining advantage we have now. Going to a more traditional request/response model in rsync would certainly make TCP like us but would incur a huge penalty in latency. As we are currently working on the design of rsync 3.0 it would be good to get suggestions now on brilliant ways of solving this problem. (please don't suggest opening a second control socket!) Here's my current idea: it may be naïve about the TCP bugs, but I hope it will be more reliable and still keep both directions on the network maximally full. I don't think we really need multiplexed streams: all error messages relate back to a particular operation which has failed, although at the moment they're not construed that way in the code. So, I'd like to change to a request-response protocol like POP3, HTTP/1.1 or SMTP, where the client sends commands and gets response codes which contain a status, possibly body data, and possibly an error message. Perhaps we can allow the body data to be sent in several chunks so that responses that fail halfway through (e.g. file truncated or IO error) can be truncated. If there's an error on the server, then it should result in an error response. I'm imagining a fairly concise encoding format, perhaps ASCII, with commands like * authentication conversations modelled on PAM * choose rsync module * mkdir * set permissions/acl * get rsync signature * apply patch * list directory * make link * turn on gzip At first in rsync3, we can just do this in blocking mode, like most HTTP or POP implementations where the client sends the command, then does blocking reads to get the response. I think we could then just have the client start pipelining requests, by using select() on the socket and a work queue to keep track of outstanding requests. In some cases where there are dependencies between requests, such as creating a directory and then the files inside it, we'll have to compose the protocol so that all the requests can issue before the first one completes. Also, for example, we might have to keep issuing requests for signatures until the first one arrives back, and then start sending its delta. The server of course doesn't have to be nonblocking, but can just handle requests as they arrive. So, I think this will keep the send and receive sides full as much of the time as possible, while still allowing TCP to keep the two sides in loose synchronization. To me, this seems like a fairly conservative use of sockets, and so I hope it won't encounter more of these kinds of bugs. Is it going to be a problem (with or without SSH) to have e.g. the -server side of the socket full while the server is trying to write, or vice versa, if the client doesn't fork? -- Martin Pool, Human Resource Linuxcare. Inc. +61 2 6262 8990 [EMAIL PROTECTED], http://linuxcare.com.au/ Linuxcare. Putting Open Source to work.
rsync3 design (was Re: Linux 2.2.16 through 2.2.18preX TCP hang bug triggered by rsync)
On 16 Mar 2001, Rich Salz [EMAIL PROTECTED] wrote: Here's my current idea: it may be na?ve about the TCP bugs, but I hope it will be more reliable and still keep both directions on the network maximally full. You owe it to yourself to look at BEEP and the TCP mapping. It was designed by folks who really know how to optimize ietf-style protocols. Sliding windows, sequence numbers, all that good stuff. I know it's a good design, and I've read the spec already. I'm just not sure that we need a whole other set of sliding windows etc on top of TCP if TCP is already sufficient. I believe they're RFC's now. Also read the draft-mrose-beep-design Internet-Draft. The draft-mrose-beep-design document is a very nice analysis. I'm reading it now. I wonder if adding two-way communications rather than request/response would be a good thing? It might take rsync into an interesting space for push-replication. -- Martin Pool, Human Resource Linuxcare. Inc. +61 2 6262 8990 [EMAIL PROTECTED], http://linuxcare.com.au/ Linuxcare. Putting Open Source to work.
Re: rsync3 design (was Re: Linux 2.2.16 through 2.2.18preX TCP hang bug triggered by rsync)
Rich wrote: You owe it to yourself to look at BEEP and the TCP mapping. It was designed by folks who really know how to optimize ietf-style protocols. Sliding windows, sequence numbers, all that good stuff. It's a good paper, and an interesting design. If we don't actually use it we should certainly learn from it. Their comments about framing and reply codes are good, and I certainly like SASL. (I was thinking of that before when I said PAM.) I'm certainly open to the idea that all these protocols should build on a common framework, but I wonder if the subset of problems they're trying to solve really includes rsync or not. If you don't mind being their advocate for a little longer, I'll list some concerns: * There seems to be no C implementation yet. Much as I like Java and Python, I think rsync still needs to be done in C, and building this would be a nontrivial part of the new implementation. * I hope the new library will use things like sendfile(2) when available, and I'm not sure if that will fit well into a BXXP layer. * There are already messaging layers that seem more mature, including PSMQDI, COSNotification, Gale, Java Messaging, ... Why not use one of them? * Although there's a lot of potential for interesting two-way mirroring projects, I think they're probably not the core requirement. I think I'd rather provide a library that allows people to do rsync-in-BXXP just as they will do rsync-in-HTTP. * A concise protocol (within reason) is important for rsync, but the ID explicitly says that it's not a goal for BXXP. I really don't think we should use XML or explicit content types on every request. * I don't think we need multiple streams inside a single connection, although perhaps this is a failure of imagination on my part. * If we don't need multiple streams, then we don't need asynchronous responses. Allowing for responses to come back out of order will only make the code complex without actually achieving better performance or features. If you can think of a specific way in which async replies would help, by all means say so. * Although BXXP is making noise, it's not really running any major projects yet that I know of. Unless it seems like a really good fit, I don't see why rsync should be the first. If we design a clean request-response protocol we can always switch later. * It might be simpler to go to request/response first, and then switch to BXXP later when we've worked out the kinks. Asynchronous responses or channels might be interesting for rsync-over-UDP. But I think there is already enough to learn in mapping a single specific protocol to UDP, without having another abstraction layer. I guess in summary I think BXXP is too complex if we just want to get to a cleaner, extensible, and more reliable version of the current rsync tool. If we wanted to turn it into something really different, such as allowing a whole group of machines to talk on a channel to stay synchronized, then it could be good. At the moment it doesn't seem justified. I really like modularity, and I'd kind of like to be persuaded not to write our own network layer. But at the moment this doesn't seem like the one to me. -- Martin Pool, Human Resource Linuxcare. Inc. +61 2 6262 8990 [EMAIL PROTECTED], http://linuxcare.com.au/ Linuxcare. Putting Open Source to work.
Re: LFS support on AIX 4.3.3
On Sat, Mar 17, 2001 at 11:22:31AM +1100, Martin Pool wrote: On 16 Mar 2001, [EMAIL PROTECTED] wrote: ICK! Please undo your change. Defining _LARGE_FILES as is done in rsync.h means everyone will be impacted (Solaris, HP-UX, IRIX, Linux, FreeBSD, Tru64 UNIX, ...). Discussion has already occurred on this list about LFS support: OK, I'll take it out. Thankyou for catching this. Great! If the current configure test is broken for AIX, the autoconf test should be fixed. Also, Andrew is probably going to move away from the explicit use of 64-bit data types (like off64_t). Paul Eggert has written an autoconf macro, AC_SYS_LARGEFILE, which is now a *standard* part of autoconf in CVS, that detects what to add to CFLAGS and LDFLAGS to get LFS support on your OS. I think this is the direction we should head. That sounds like a more clean solution. So, that will just automatically make off_t be as long as possible? As long as possible to support LFS, which might not be "as long as possible". -- albert chin ([EMAIL PROTECTED])
Re: LFS support on AIX 4.3.3
Martin Pool writes:
I normally write
off_t foo = ;
printf("seek to %ld\n", (long) foo);
accepting that for long values on some platforms it will just be
wrong. Is there a better solution that's not gcc or glibc-specific?
From what I've seen of the standards, there is no standard format
longer than %ld, but off_t can be longer than long.
Hmmm. The following
printf("seek to %jd\n", (intmax_t) foo);
might do what you want. Its not in SUSv2 but is in the Austin drafts
so I'm not sure how widely implemented it is.
Chris.
--
[EMAIL PROTECTED]
Support Open Source Ice-Cream
