Re: RSync exclusion
Daniel O'Callaghan: It uses rsync, but manages a directory tree with hard links to unchanged files. It would solve your problem, but in a different way. Thanks for this - the issue is solved, but I will certainly have a look to that one as well. Jos ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: RSync exclusion
Must have had a temporary brain damage, sorry... thanks Jos Paul Macdonald: --exclude /files/photos ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: RSync exclusion
Hi, On Thu, 18 Apr 2013 13:30:45 +0200 Jos Chrispijn wrote: > I have a local folder called /files/ > In my daily backup event I create as per day a backkup folder that > contains this /files folder including all its sub folders > > --- cut --- > > rsync -avrz -e ssh /files/ backupr@x.x.x.x:/vol1/FreeBSD/$DATE/ > > In this /files folder, amongst others, I have another folder called > photos: /files/photos > > What I now would like to do is sync the /files folder with an > exclusion on the /files/photos folder I use the following option: --exclude '/data/Mail/*' /data/Mail is rooted in the directory I want to copy. Erich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: RSync exclusion
In this /files folder, amongst others, I have another folder called photos: /files/photos What I now would like to do is sync the /files folder with an exclusion on the /files/photos folder Reason for that is that this /photos subfolder contains 12 gb on photos, which I don't want to have in a daily archive (takes too much disk space on a monthly basis). I will syn them on a daily basis to a fixed remote folder (where only the updates will be appended). Can someone tell me how I can do that on one command line? have you tried --exclude /files/photos Paul. -- - Paul Macdonald IFDNRG Ltd Web and video hosting - t: 0131 5548070 m: 07970339546 e: p...@ifdnrg.com w: http://www.ifdnrg.com - IFDNRG 40 Maritime Street Edinburgh EH6 6SA High Specification Dedicated Servers from £100.00pm ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: RSync exclusion
On 18/04/2013 9:30 PM, Jos Chrispijn wrote: rsync -avrz -e ssh /files/ backupr@x.x.x.x:/vol1/FreeBSD/$DATE/ Just a thought, but have you looked at rsnapshot? http://www.rsnapshot.org/ http://www.freshports.org/sysutils/rsnapshot/ It uses rsync, but manages a directory tree with hard links to unchanged files. It would solve your problem, but in a different way. regards, Danny ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Rsync and Preservation of Ownership and Permissions
Michael Sierchio writes: > Does the same user exist on the remote system, with the same uid, etc.? Yes. > If you're using rsync with ssh as the transport, and connecting to the > remote machine as the backups user, that's who will own the files on > its local filesystem... I thought rsync had some encoding it might slip in to the tree that another rsync run as root on the recovering system could use to figure out all those thousands of ownerships and get them all straight, but this makes perfect sense. > You've written a lot of narrative, but show us precisely what commands > you're running. Why would you run the command as root, and ssh as > backups, when you want them to be owned by "normal" ? Because root is the only user who can "see" files from all other users so root starts the process. Here is what I tried. Remember, folks, this will not work! This tries to backup a system named z. ##!/bin/sh #rsync --delete -alHvq --exclude "/proc" // back...@backup-server.okstate.edu:z > You can run the command as root, and use restricted ssh keys (use > authorized_keys to restrict it to executing a specific rsync > command) you can run rsync as a regular user to that user's > account on the remote system... per...@pluto.rain.com writes: > Perhaps you could have rsync log in to a jail on the backup server, > where it could safely be granted root permission. Hmm. It's all rather clear, now. A jailed environment that looks like root is about the only thing that could work. Thank you. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Rsync and Preservation of Ownership and Permissions
Martin McCormick wrote: > Rsync is a great utility, but is there a way to preserve > ownership and permissions if rsync remotely logs in to a backup > server as a normal user? AFAIK, no, because only root may change the ownership of a file -- see chown(2). > Any ideas are greatly appreciated. Perhaps you could have rsync log in to a jail on the backup server, where it could safely be granted root permission. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Rsync and Preservation of Ownership and Permissions
On Wed, Nov 23, 2011 at 9:01 PM, Martin McCormick wrote: > Rsync is a great utility, but is there a way to preserve > ownership and permissions if rsync remotely logs in to a backup > server as a normal user? Does the same user exist on the remote system, with the same uid, etc.? If you're using rsync with ssh as the transport, and connecting to the remote machine as the backups user, that's who will own the files on its local filesystem... You've written a lot of narrative, but show us precisely what commands you're running. Why would you run the command as root, and ssh as backups, when you want them to be owned by "normal" ? You can run the command as root, and use restricted ssh keys (use authorized_keys to restrict it to executing a specific rsync command) you can run rsync as a regular user to that user's account on the remote system... > The recovery process is run by root but copies all the > files from the backup server as a normal user and uses its root > capabilities to restore them. > > What happens now is that all the files end up owned by > and in the group of the user ID that copied the information from > the client to the server. That's obviously not too useful so I > suspect there is a better way than trying to make a remote login > to root from another system. > > Basically, cron starts a backup as root on system A. > System A makes a remote ssh connection using the -e flag to > backups@server. The system trying to recover the files starts a > rsync process as root which remotely connects to backups@server > to retrieve the files. > > In practice, the files come across but every last one of > them is owned by and in the group of user backups. > > Any ideas are greatly appreciated. > > Thank you. > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org" > ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync and the ports tree
Sounds like a well-thought out backup strategy. I've started to use your methods here, and I'm building ports I need at the same time, but the wireless here is not password protected so is incredibly slow as there are lots of leaches on the system. But while I'm plodding along, I have a few more questions: On Wed, Oct 26, 2011 at 8:07 AM, C. P. Ghost wrote: > On Wed, Oct 26, 2011 at 12:39 PM, Peter Kryszkiewicz > wrote: > > I have several machines installed in my temporary location and only my > > laptop gets the internet through wireless. So far I've been building > ports > > on the other machines by rsync'ing the distfiles from the laptop as I > need > > them (all machines have the same FreeBSD 8.2 installed). > > > > The problem comes after I did a 'portupgrade -a' on the laptop. To ensure > > the other ports trees are in sync, can I rsync the /usr/ports directory > to > > the other machines? Since some of them are different architectures (amd64 > > multicore for instance) I ran into situations where the distfiles are > > different (for gcc for example). > > First of all, rsync is working perfectly if you want to > distribute /usr/ports/distfiles, /usr/ports to your internal > machines, even when they are not of the same architecture. > I'm doing this with a BIG farm of servers running i386, amd64, > and sparc64 for a long, long time. > > You only need to make sure to rsync the *union* of your > /usr/ports/distfiles directories, or else it won't work. > > Say, on amd64 you have > /usr/ports/distfiles/some-distfile-for-amd64-only.tar.bz2 > and on i386 you have > /usr/ports/distfiles/some-distfile-for-i386-only.tar.bz2 > > Yes, that happens every now and then. > > So you have to rsync both ways, so that you end up with > /usr/ports/distfiles/some-distfile-for-amd64-only.tar.bz2 > /usr/ports/distfiles/some-distfile-for-i386-only.tar.bz2 > on both i386 and amd64 machines. > I've done that, it works well especially for the architecture differences. gcc requires an additional distfile for the amd64 build). > > The catch is: look out for rsync's --delete flag! When some > port managers delete old/stale distfiles, they may also delete > distfiles for the *other* arches because they (rightly) think > they are not needed here... and when you then rsync with --delete, > that would (wrongly) propagate such deletes to those arches, > and you end up with missing distfiles on the targets. > > Since I have more than just two arches, I use a slightly different > 2-layer workflow: > > 0. I have 3 servers that are allowed to fetch files from the outside: > i386-master, amd64-master, sparc64-master. > and a whole bunch of i386-slave-NNN, amd64-slave-NNN and > sparc64-slave-NNN machines that would duplicate from their > relative masters via rsync. > > On all -master(s), I keep $DISTFILES outside of /usr/ports > (on /usr/local/distfiles, with a symbolic link in /usr/ports > /usr/ports/distfiles -> /usr/local/distfiles) > > Initial update of i386-master, as usual: > > 1. On i386-master, csup /usr/ports. > Run portmaster as usual to upgrade everything. > This may delete old stale distfiles and non-i386-distfiles. > This may fetch additional generic and i386-specific distfiles. > > Copy the new /usr/ports (without distfiles) to the other > arch masters: > > 2. rsync -av --delete i386-master:/usr/ports to amd64-master > and sparc64-master. CAUTION: Use --delete is okay, but only > because distfiles are not under /usr/ports, so as not to nuke > non-i386-specific distfiles of the other arches. > > Copy i386-master's NEW distfiles to the other arch masters: > > 3. rsync -av i386-master:/usr/local/distfiles to amd64-master > and sparc64-master. BEWARE: Don't use --delete here! > Do this to copy new generic distfiles (and i386) from > the i386-master build to amd64-master and sparc64-master. > > Update amd64-master and sparc64-master's ports as usual: > > 4. On amd64-master, run portmaster as usual to upgrade everything. > This may delete old stale distfiles and non-amd64-distfiles. > This may fetch additional (generic and) amd64-specific-distfiles. > > 5. On sparc64-master, run portmaster as usual to upgrade everything. > This may delete old stale distfiles and non-sparc64-distfiles. > This may fetch additional (generic and) sparc64-specific-distfiles. > > At this point, i386-master, amd64-master and sparc64-master are > fully updated, and their /usr/local/distfiles directories are up > to date w.r.t. their specific architectures. Now, copy everything > from the masters to the slaves: > > 6. On every i386-slave-NNN, rsync -av --delete: > /usr/ports, /usr/local (including /usr/local/distfiles), > /var/db/pkg, /var/db/ports > from i386-master. > > 7. On every amd64-slave-NNN, rsync -av --delete: > /usr/ports, /usr/local (including /usr/local/distfiles) > /var/db/pkg, /var/db/ports > from amd64-master. > I can see the need to sync /var/db/ports, but isn't /var
Re: rsync and the ports tree
On Wed, Oct 26, 2011 at 12:39 PM, Peter Kryszkiewicz wrote: > I have several machines installed in my temporary location and only my > laptop gets the internet through wireless. So far I've been building ports > on the other machines by rsync'ing the distfiles from the laptop as I need > them (all machines have the same FreeBSD 8.2 installed). > > The problem comes after I did a 'portupgrade -a' on the laptop. To ensure > the other ports trees are in sync, can I rsync the /usr/ports directory to > the other machines? Since some of them are different architectures (amd64 > multicore for instance) I ran into situations where the distfiles are > different (for gcc for example). First of all, rsync is working perfectly if you want to distribute /usr/ports/distfiles, /usr/ports to your internal machines, even when they are not of the same architecture. I'm doing this with a BIG farm of servers running i386, amd64, and sparc64 for a long, long time. You only need to make sure to rsync the *union* of your /usr/ports/distfiles directories, or else it won't work. Say, on amd64 you have /usr/ports/distfiles/some-distfile-for-amd64-only.tar.bz2 and on i386 you have /usr/ports/distfiles/some-distfile-for-i386-only.tar.bz2 Yes, that happens every now and then. So you have to rsync both ways, so that you end up with /usr/ports/distfiles/some-distfile-for-amd64-only.tar.bz2 /usr/ports/distfiles/some-distfile-for-i386-only.tar.bz2 on both i386 and amd64 machines. The catch is: look out for rsync's --delete flag! When some port managers delete old/stale distfiles, they may also delete distfiles for the *other* arches because they (rightly) think they are not needed here... and when you then rsync with --delete, that would (wrongly) propagate such deletes to those arches, and you end up with missing distfiles on the targets. Since I have more than just two arches, I use a slightly different 2-layer workflow: 0. I have 3 servers that are allowed to fetch files from the outside: i386-master, amd64-master, sparc64-master. and a whole bunch of i386-slave-NNN, amd64-slave-NNN and sparc64-slave-NNN machines that would duplicate from their relative masters via rsync. On all -master(s), I keep $DISTFILES outside of /usr/ports (on /usr/local/distfiles, with a symbolic link in /usr/ports /usr/ports/distfiles -> /usr/local/distfiles) Initial update of i386-master, as usual: 1. On i386-master, csup /usr/ports. Run portmaster as usual to upgrade everything. This may delete old stale distfiles and non-i386-distfiles. This may fetch additional generic and i386-specific distfiles. Copy the new /usr/ports (without distfiles) to the other arch masters: 2. rsync -av --delete i386-master:/usr/ports to amd64-master and sparc64-master. CAUTION: Use --delete is okay, but only because distfiles are not under /usr/ports, so as not to nuke non-i386-specific distfiles of the other arches. Copy i386-master's NEW distfiles to the other arch masters: 3. rsync -av i386-master:/usr/local/distfiles to amd64-master and sparc64-master. BEWARE: Don't use --delete here! Do this to copy new generic distfiles (and i386) from the i386-master build to amd64-master and sparc64-master. Update amd64-master and sparc64-master's ports as usual: 4. On amd64-master, run portmaster as usual to upgrade everything. This may delete old stale distfiles and non-amd64-distfiles. This may fetch additional (generic and) amd64-specific-distfiles. 5. On sparc64-master, run portmaster as usual to upgrade everything. This may delete old stale distfiles and non-sparc64-distfiles. This may fetch additional (generic and) sparc64-specific-distfiles. At this point, i386-master, amd64-master and sparc64-master are fully updated, and their /usr/local/distfiles directories are up to date w.r.t. their specific architectures. Now, copy everything from the masters to the slaves: 6. On every i386-slave-NNN, rsync -av --delete: /usr/ports, /usr/local (including /usr/local/distfiles), /var/db/pkg, /var/db/ports from i386-master. 7. On every amd64-slave-NNN, rsync -av --delete: /usr/ports, /usr/local (including /usr/local/distfiles) /var/db/pkg, /var/db/ports from amd64-master. 8. On every sparc64-slave-NNN, rsync -av --delete: /usr/ports, /usr/local (including /usr/local/distfiles) /var/db/pkg, /var/db/ports from sparc64-master. You may also need to update entries in /etc and /usr/local/etc on the slaves. > If not rsync, what is the best way to keep multiple ports trees on different > hardware in sync, assuming everything runs FreeBSD 8.2? > > regards, > Peter Kryszkiewicz -cpghost. -- Cordula's Web. http://www.cordula.ws/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync and the ports tree
On Wed, Oct 26, 2011 at 12:39 PM, Peter Kryszkiewicz wrote: > I have several machines installed in my temporary location and only my > laptop gets the internet through wireless. So far I've been building ports > on the other machines by rsync'ing the distfiles from the laptop as I need > them (all machines have the same FreeBSD 8.2 installed). > > The problem comes after I did a 'portupgrade -a' on the laptop. To ensure > the other ports trees are in sync, can I rsync the /usr/ports directory to > the other machines? Since some of them are different architectures (amd64 > multicore for instance) I ran into situations where the distfiles are > different (for gcc for example). > The distfiles are not different between architectures. Rsyncing /usr/ports works fine. But if you will bump into problems if you also sync /usr/ports/packages and you have different archs (i386 vs amd64 for instance). -- chs, ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync over nfs or rsync protocol
On Sun, Sep 25, 2011 at 09:10:22AM -0700, Jason C. Wells wrote: > On 09/23/11 14:11, Jerry McAllister wrote: > >Why would you interject NFS in the middle of it? jerry > There would be no middle. I would run rsyncd or nfsd, but not both. Ah, I get it. In that case, I think rsync is probably more useful than NFS because it can check and only copy modified files. Alternatively, if you are doing backups to recover from system failures - such as a disk crash, you would probably prefer dump(8)/restore(8) They can write to a file on the other machine, can do "change dumps" and they preserve all the needed UNIX attributes. I actually do a dump piped to a restore on another disk as a convenient backup to handle my all too frequent cases of fumble fingers and sleep deprived bad thinking where I need to quickly get back a file I mangled, deleted or need to start over on. Restore can easily pull single files or directory trees from a dump file as well. But having it already pre-restored makes it easier -- and only doubles my disk use - disk is cheap isn't it. jerry > > Jason ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync over nfs or rsync protocol
On 09/23/11 14:15, Chuck Swiger wrote: Lots. The handbook has a chapter on backups which is worth reading, also Regards, Ah the handbook. I forgot all about it. Thanks. Later, Jason ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync over nfs or rsync protocol
On 09/23/11 14:11, Jerry McAllister wrote: Why would you interject NFS in the middle of it? jerry There would be no middle. I would run rsyncd or nfsd, but not both. Jason ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync over nfs or rsync protocol
On Fri, Sep 23, 2011 at 2:15 PM, Chuck Swiger wrote: > Lots. The handbook has a chapter on backups which is worth reading, also True. Personally, I like dump/restore for disaster-recovery backups on FreeBSD. However, if you frequently need individual file recovery (e.g., Joe User deletes an important file and wants it back) or if you need to back up systems running something other than FreeBSD, BackupPC is hard to beat for disk-to-disk backup. If you're going to tape you might want something like Amanda, instead. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync over nfs or rsync protocol
On Fri, Sep 23, 2011, Jason C. Wells wrote: > I am looking into finally setting up a backup solution that's a little > more sophisticated than a bunch of DVD-RWs. I have two servers. I'd > like to make each a backup server for the other. I'm considering using > rsync. > > Is rsync a good choice for a backup tool? > Should I use the rsyncd or should I use NFS? I'm using 100 mbps ethernet. > What's the better solution I haven't considered? We use rsync extensively for backups on local LANs as well as off-site via the 'Net. My experience using NFS for this was not good, but it's been many years since I last tried this so I don't remember the details. Bill -- INTERNET: b...@celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax:(206) 232-9186 Skype: jwccsllc (206) 855-5792 Windows is a computer virus with a user interface!! ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync over nfs or rsync protocol
On Sep 23, 2011, at 2:04 PM, Jason C. Wells wrote: > Is rsync a good choice for a backup tool? It's OK. A versioned backup system (dump/restore, Legato Networker, Amanda, Retrospect, etc) is more efficient at using backup storage. > Should I use the rsyncd or should I use NFS? I'm using 100 mbps ethernet. If it's local and you already have NFS in place, that would be fine. If you're backing up over a WAN, rsyncd is probably a better call. > What's the better solution I haven't considered? Lots. The handbook has a chapter on backups which is worth reading, also Regards, -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync over nfs or rsync protocol
On Fri, Sep 23, 2011 at 2:04 PM, Jason C. Wells wrote: > I am looking into finally setting up a backup solution that's a little more > sophisticated than a bunch of DVD-RWs. I have two servers. I'd like to > make each a backup server for the other. I'm considering using rsync. > > Is rsync a good choice for a backup tool? rsync is good for mirroring one file tree to another. If you want a complete backup system centering around rsync, check out BackupPC. NFS will let you mount one server's files from another, but you'll still need to use something to copy the files over; either rsync or something else. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync over nfs or rsync protocol
On Fri, Sep 23, 2011 at 02:04:03PM -0700, Jason C. Wells wrote: > I am looking into finally setting up a backup solution that's a little > more sophisticated than a bunch of DVD-RWs. I have two servers. I'd > like to make each a backup server for the other. I'm considering using > rsync. > Is rsync a good choice for a backup tool? rsync can be used effectively. The optimum solution really depends on how much total disk and how much changes between each run. > Should I use the rsyncd or should I use NFS? I'm using 100 mbps ethernet. > What's the better solution I haven't considered? Why would you interject NFS in the middle of it? jerry > > Thanks, > Jason > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
RE: rsync over nfs or rsync protocol
I'm sure you'll get a TON of responses on this. Maybe some script that combines tar, g[b]zip, and rsync? I wouldn't use NFS just for this, but if it's already there it may have a place. -Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd-questi...@freebsd.org] On Behalf Of Jason C. Wells Sent: Friday, September 23, 2011 4:04 PM To: freebsd general questions Subject: rsync over nfs or rsync protocol I am looking into finally setting up a backup solution that's a little more sophisticated than a bunch of DVD-RWs. I have two servers. I'd like to make each a backup server for the other. I'm considering using rsync. Is rsync a good choice for a backup tool? Should I use the rsyncd or should I use NFS? I'm using 100 mbps ethernet. What's the better solution I haven't considered? Thanks, Jason ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org" "This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync: failed to set permissions on "": Function not implemented (38)
On Wed, 27 Oct 2010, Noah wrote: No response on the rsync mail list so I am trying here. I am trying to figure out how I can get my rsync client and server to coexist better. I am running into the following error: rsync: failed to set permissions on "": Function not implemented (38) What is the target filesystem? That kind of error can happen when filesystems don't implement all the features rsync expects, like FAT. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync include and exclude
Thanks. That took care of the problem. Jay On Oct 7, 2009, at 3:02 PM, krad wrote: 2009/10/7 Jay Hall Ladies and Gentlemen, I am using rsync to backup some information and I am having some problem with including and excluding directories. I want to include everything in the user's mail directory and everything in the user's documents directory. Everything else should be excluded. And, it is possible to have multiple users on a single computer. Following is what I am using for my include/exclude patterns. + /Users/*/Library/Mail/ + /Users/*/Documents/ - /Users/*/Documents/* - /Users/*/Library/* This gives me close to the desired result. However, there are some extra files included which I do not need to backup which are stored in /Users/username directory. If I add -/Users/*, no files are backed up. After reading the man pages and several examples, I thought I had a handle on this. Any suggestions would be greatly appreciated. Thanks, Jay ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org " try adding this line to the top + /Users/* then this to the bottom - /Users/*/* I use lists like this at work a lot and you have to be careful. The ordering of the rules is very important. The common on for people to get wrong is the mysql rules.. We generally dont copy the binary db files as they would never be consistent, so we dump the db else where and exclude the files. Except we need to capture the my.cnf file. The following ruleset does it + /var/db/mysql + /var/db/mysql/my.cnf - /var/db/mysql/** If you deviate from this things dont tend to work. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync include and exclude
2009/10/7 Jay Hall > Ladies and Gentlemen, > > I am using rsync to backup some information and I am having some problem > with including and excluding directories. > > I want to include everything in the user's mail directory and everything in > the user's documents directory. Everything else should be excluded. And, > it is possible to have multiple users on a single computer. > > Following is what I am using for my include/exclude patterns. > > + /Users/*/Library/Mail/ > + /Users/*/Documents/ > - /Users/*/Documents/* > - /Users/*/Library/* > > This gives me close to the desired result. However, there are some extra > files included which I do not need to backup which are stored in > /Users/username directory. > > If I add -/Users/*, no files are backed up. > > After reading the man pages and several examples, I thought I had a handle > on this. Any suggestions would be greatly appreciated. > > Thanks, > > > Jay > > > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to " > freebsd-questions-unsubscr...@freebsd.org" > try adding this line to the top + /Users/* then this to the bottom - /Users/*/* I use lists like this at work a lot and you have to be careful. The ordering of the rules is very important. The common on for people to get wrong is the mysql rules. We generally dont copy the binary db files as they would never be consistent, so we dump the db else where and exclude the files. Except we need to capture the my.cnf file. The following ruleset does it + /var/db/mysql + /var/db/mysql/my.cnf - /var/db/mysql/** If you deviate from this things dont tend to work. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync colon in filename to MSWin fails
Here's a good workaround. This works for me from Linux to Win32. In your rsync script, before the rsync, do either find ./ -name '*:*' | xargs tar -czvf colon_files.tgz or (if you're doing the whole box as an rsync backup, like I am) do updatedb locate : | xargs tar -czvf colon_files.tgz Make sure that colon_files.tgz is in the path that's getting backed up by rsync. Then you can do the --exclude '*:*' in your rsync job to skip the files you just tarred. I tested this with the locate version above and it works. Also, updatedb & locate should go much faster than find if you want to do the whole box. I'm not sure if there's a way to use locate to just search a subdirectory, though. Best, Dave Anton Shterenlikht wrote: > > I cannot trasfer a file with a colon via rsync to a Win box. > I've rsync-3.0.4 on the FBSD (sending side) and rsync-2.6.9 > under cygwin on Win (receiving side). I'm not sure what the > error message means: > > % rsync ./http:__en.wikipedia.org_favicon.ico me...@xxx.xxx.xxx.xxx: > rsync: rename "/cygdrive/c/Documents and Settings/mexas/My > Documents/work/.http:__en.wikipedia.org_favicon.ico.8dBX2K" -> > "http:__en.wikipedia.org_favicon.ico": No such file or directory (2) > rsync error: some files/attrs were not transferred (see previous errors) > (code 23) at main.c(1040) [sender=3.0.4] > > Note the extra dot before the file name, and extra suffix at the > end. Is this the expected behaviour? > > However, to another FBSD box transfer is fine: > > % rsync ./http:__en.wikipedia.org_favicon.ico me...@zzz.zzz.zzz.zzz: > % > > Is this something to do with Windows, or old rsync version on cygwin? > > Even more bizzare, I get the same error even if I "--exclude" all > such files from transfer. > > many thanks > anton > > -- > Anton Shterenlikht > Room 2.6, Queen's Building > Mech Eng Dept > Bristol University > University Walk, Bristol BS8 1TR, UK > Tel: +44 (0)117 928 8233 > Fax: +44 (0)117 929 4423 > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "freebsd-questions-unsubscr...@freebsd.org" > > -- View this message in context: http://www.nabble.com/rsync-colon-in-filename-to-MSWin-fails-tp19571614p23925088.html Sent from the freebsd-questions mailing list archive at Nabble.com. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync approach
>> On Wed, 27 May 2009 15:03:30 -0700,
>> prad said:
P> We are thinking of rsync to duplicate 1st [box] > 2nd [box] (with the
P> exception of rc.conf and a few other files of course because we don't
P> want them to be absolutely identical).
P> we plan to allow root login and have disabled all password access so
P> that rsync can preserve permissions. is this a good way to accomplish
P> the bkp job?
If you're going to use root login, I'd suggest access control for ssh
via either daemontools or tcpwrappers, and add some extra security
by putting 'from="hostname"' in root's entry in the "authorized_keys2"
file:
from="1st.box.com" ssh-dss B3NzaC1MtH[...]WDXDrq03pE= r...@1st.box.com
It's not strictly necessary to allow root connections if you want
to keep permissions intact. I use an unprivileged account ("bkup")
to copy gzipped cpio archives between systems. On the 1st box, root
can use pax or cpio to create the archive, and then run something as
user "bkup" to do the copy to the 2nd box:
root# cd /some/where
root# find . -print | pax -x cpio -wd | gzip -1c > /tmp/arch.pax.gz
root# su bkup -c "scp -c arcfour -i /bkup/.ssh/backuphost_dsa \
/tmp/arch.pax.gz 2nd.box.com:/someplace/bkup/can/write"
The arcfour cipher will probably give you better throughput. To unpack
the files on 2nd.box.com:
root# cd /some/where/else
root# gunzip -c /someplace/bkup/can/write/arch.pax.gz | pax -rd -pe
root# rm /someplace/bkup/can/write/arch.pax.gz
If the files you're syncing are huge, you're better off using root login
plus rsync.
--
Karl Vogel I don't speak for the USAF or my company
SUVs are gross because they're the solution to a gross problem:
how to make minivans look more masculine. --Paul Graham
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync approach
we have 2 static ip addresses with a machine running 7.2 connected to each. one is the primary server, while the other does only dns and receives bkp dumps from the first. we want to set things up so the 2nd can be brought on line at a moment's notice. therefore, we are thinking of rsync to duplicate 1st > 2nd (with the exception of rc.conf and a few other files of course because we don't want them to be absolutely identical). we plan to allow root login and have disabled all password access so that rsync can preserve permissions. i don't catch why disabling password access will allow rsync to preserve permission. It will preserve just when you give proper option is this a good way to accomplish the bkp job? yes it is. There is another way too - having both adventage and disadventage. 1) make an option in FreeBSD loader menu to run ramdisk-freebsd (ramdisk from file). Put on that cutdown ramdisk system only startup of ggated with a disk 2) on main machine run ggatec and gmirror. you will get network mirrored hard disk. make this procedure conditional so it runs only on first machine (for eg check MAC address of your network interface) in case of machine 1 fail, you just run second with normal, instead of ramdisk mode. It has adventage of full realtime replication, but it's disadventage in the same time. For example if you run rsync once per 2 hours, and you by accident delete a lots of things, you can recover. with gmirror way it is instantly replicated so you can't recover ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync approach
On 27 May 2009, at 15:03, prad wrote: we have 2 static ip addresses with a machine running 7.2 connected to each. one is the primary server, while the other does only dns and receives bkp dumps from the first. we want to set things up so the 2nd can be brought on line at a moment's notice. therefore, we are thinking of rsync to duplicate 1st > 2nd (with the exception of rc.conf and a few other files of course because we don't want them to be absolutely identical). we plan to allow root login and have disabled all password access so that rsync can preserve permissions. is this a good way to accomplish the bkp job? It might be, but its difficult to say without knowing a lot more about what going on in the system. For example, rsync of large files that change real often can be an issue. You may get an inconsistent copy of the file if it is changing during the update of that file. Rsync does one file at a time. Hence if you have multiple files that need to be consistent between them it might not happen. If transactions changing a file occur between the rsync of the first file and the rsync of the second file, they will not be consistent. I use rsync for basically the same thing you are considering. However, in my situation, the real dynamic files are database files. I don't rsync them. Those are exported every evening and those files are rsync'd. During the day, every application that updates a database file also adds a copy of the new record to the end of a log file. That log file is also rsync'd about every 5 minutes. In this way I can recover to within 5 minutes quite easily. The last 5 minutes might be a bit more work, but the information would be available from the original sources at that point. Allowing root login is generally not a great approach. Rsync will retain permissions (use -p) if the user id's and group ids are the same on both systems. You may have to modify some of the pam files to permit rsync to function easily. The easy test is to rsh date If that works and you get the date, then rsync will function properly. If not you need to track down why in the various log files. I had to make the following change: pam.d/rsh: -auth requiredpam_rhosts.so no_warn +auth required pam_rhosts.so no_warnallow_root This was first done quite a few years ago and has been propagated to the current systems so I don't know if its still required or not. Our systems only have 3 users (the administrators) so we have passwords working fine. It has not interfered with rsync. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Rsync | Push script
> > On Sat, Feb 21, 2009 at 02:40:21PM +0100, Jos Chrispijn wrote: > > > > Just having made a backup script that should take care of nocturnal > > backup of my mySQL data from one server to my backup server. > > > > cd /backup > > DATE=`date "+%d%m%y"` > > DIR=backup.$DATE > > > > /letc/rc.d/mysql-server stop > > > rsync -avpog /var/db/mysql//r...@10.10.10.50:123/usr/backup/$DATE/ > > /letc/rc.d/mysql-server start > > > > > It goes wrong when I run the rsync line; I run my backup thru port 123 > > (can be any portnumber). > > 10.10.10.50 is backup server on which I want to logon as root; during > > script run I will fill out root password myself. > > > > Can someone hint me in the right direction? rsync deamon is running. > > > > Jos Chrispijn > Doesn't rsync work via ssh by default? The default configuration of sshd on the backup server probably prevents external root users from accessing the server. Are you sure you want to run the backups as root? Andrew Gould ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Rsync | Push script
On Sat, Feb 21, 2009 at 02:40:21PM +0100, Jos Chrispijn wrote: > > Just having made a backup script that should take care of nocturnal > backup of my mySQL data from one server to my backup server. > > cd /backup > DATE=`date "+%d%m%y"` > DIR=backup.$DATE > > /letc/rc.d/mysql-server stop > rsync -avpog /var/db/mysql//r...@10.10.10.50:123/usr/backup/$DATE/ > /letc/rc.d/mysql-server start > > It goes wrong when I run the rsync line; I run my backup thru port 123 > (can be any portnumber). > 10.10.10.50 is backup server on which I want to logon as root; during > script run I will fill out root password myself. > > Can someone hint me in the right direction? rsync deamon is running. > > Jos Chrispijn Your script looks like it's got some typos in it (marked above). Don't think that has anything to do with your problems by the sounds of it. Regards, -- Frank Contact info: http://www.shute.org.uk/misc/contact.html ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Rsync | Push script
Just a small note which has nothing to do with the actual rsync problem: On Sat, 21 Feb 2009 14:40:21 +0100, Jos Chrispijn wrote: > DATE=`date "+%d%m%y"` In order to be able to sourt your backups by date, you could use the form DATE=`date "+%y%m%d"` or DATE=`date "+%Y%m%d"` to get a date signature that can be sorted. Just a suggestion - I've had the best experiences with the form DATE=`date "+%Y-%m-%d"` attached to the backup's "subject". -- Polytropon >From Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Rsync | Push script
On Sat, Feb 21, 2009 at 06:14:27PM +0100, Jos Chrispijn wrote: > Sorry for the confusion; the problem is in the rsync line: > > rsync -avpog /var/db/mysql //r...@10.10.10.50:123/usr/backup/ > > Running this line causes rsync to say: > Unexpected remote arg: r...@10.10.10.50:123 > rsync error: syntax or usage error (code 1) at main.c(1202) [sender=3.0.5] > > On the server that I use this push script, I want to connect as root > to the backup server; thru port 123 (can be any port). With rsync > running I can't get this to work. Can someone tell me what I do wrong > here? You got the syntax for the rsync command wrong. Either remove the '//' in front of the remote address or use 'rsync://'. See the rsync(1) manual page. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpL5uR37bpx6.pgp Description: PGP signature
Re: Rsync | Push script
Sorry for the confusion; the problem is in the rsync line: rsync -avpog /var/db/mysql //r...@10.10.10.50:123/usr/backup/ Running this line causes rsync to say: Unexpected remote arg: r...@10.10.10.50:123 rsync error: syntax or usage error (code 1) at main.c(1202) [sender=3.0.5] On the server that I use this push script, I want to connect as root to the backup server; thru port 123 (can be any port). With rsync running I can't get this to work. Can someone tell me what I do wrong here? Jos Chrispijn ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Rsync | Push script
Jos Chrispijn wrote: > > Just having made a backup script that should take care of nocturnal > backup of my mySQL data from one server to my backup server. > > cd /backup > DATE=`date "+%d%m%y"` > DIR=backup.$DATE > > /letc/rc.d/mysql-server stop > rsync -avpog /var/db/mysql//r...@10.10.10.50:123/usr/backup/$DATE/ > /letc/rc.d/mysql-server start > > It goes wrong when I run the rsync line; I run my backup thru port 123 > (can be any portnumber). > 10.10.10.50 is backup server on which I want to logon as root; during > script run I will fill out root password myself. > > Can someone hint me in the right direction? rsync deamon is running. Based on your description, I have no idea what your actual problem is. I mean, you're basically saying, "it doesn't work." Which doesn't give folks any way to help you, really. What error message do you see? What's in the logs? You're running the rsync client with -v, which should be pretty verbose, have you tried running the rsync server with the logging amped up? -- Bill Moran http://www.potentialtech.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: rsync throwing odd error
/usr/local/bin/rsync -az -e 'ssh ' data.cdb [EMAIL PROTECTED]:/usr/local/etc/tinydns/root/data.cdb channel 1: open failed: administratively prohibited: open failed The rsync does work. That is, the file is copied over. So this is actually a warning, I guess. On the other server, the one the file is being copied TO, the following is printed in /var/log/auth.log Nov 21 12:43:38 qu sshd[4604]: Address 67.111.0.194 maps to on.example.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! well it's as easy as reading messages. you exactly posted answer to your question! make your reverse DNS and forward DNS match. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync throwing odd error
A... a reverse DNS problem! Nope... wasn't that. Reverse DNS was working fine. I just didn't know how to check it properly. Well, that was a good idea. Time to find another one! - John ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync throwing odd error
On Nov 21, 2008, at 12:50 PM, John Almberg wrote: This is the week for strange problems... I use rsync to copy tinydns data files to backup name servers. This has been working for about a year with no problem. Suddenly, I am getting odd errors: /usr/local/bin/rsync -az -e 'ssh ' data.cdb [EMAIL PROTECTED]:/ usr/local/etc/tinydns/root/data.cdb channel 1: open failed: administratively prohibited: open failed The rsync does work. That is, the file is copied over. So this is actually a warning, I guess. On the other server, the one the file is being copied TO, the following is printed in /var/log/auth.log Nov 21 12:43:38 qu sshd[4604]: Address 67.111.0.194 maps to on.example.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! Nov 21 12:43:38 qu sshd[4604]: Accepted publickey for root from 67.111.0.194 port 55777 ssh2 I'm guessing this is some sort of DNS mis-match, but I don't quite grasp what the problem could be. A... a reverse DNS problem! Sorry for the dumb question, but it's amazingly helpful to just write down the question clear enough for the group. It frequently clarifies the problem to the point where the answer becomes obvious, even to a newbie like me. DNS had been delegated to this server, but now that seems to be no longer working, so the reverse DNS look up is all wrong. That makes sense... an external change by the colo guys must have triggered this. Will get on to them, and that should short this problem out. -- John ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync or even scp questions....
> How, may I ask, does this work? If you search the bash man file you can find this and lots of other useful constructs, search for 'Parameter Expansion' - I'm not sure how much of this relates to other Bourne Shell derivatives, but I don't imagine it would be difficult to test it out. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync or even scp questions....
On Sun, Oct 12, 2008 at 01:49:31AM -0700, Jeremy Chadwick wrote:
> On Sun, Oct 12, 2008 at 09:42:38AM +0100, Matthew Seaman wrote:
> > mdh wrote:
> >> --- On Sat, 10/11/08, Gary Kline <[EMAIL PROTECTED]> wrote:
> >>> On the Ubuntu computer I am /home/kline; on my main
> >>> computer,
> >>> my home is /usr/home/kline. The following sh script
> >>> worked
> >>> perfected when my home on "tao" [FBSD] was
> >>> /home/kline:
> >>>
> >>> P
> >>> #!/bin/sh
> >>>
> >>> PWD=`pwd`;
> >>> echo "This directory is [${PWD}]";
> >>>
> >>> scp -qrp ${PWD}/* ethos:/${PWD}
> >>> ###/usr/bin/scp -rqp -i /home/kline/.ssh/zeropasswd-id
> >>> ${PWD}/* \ klin
> >>> [EMAIL PROTECTED]:/${PWD}
> >>>
> >>> Question #1: is there any /bin/sh method of getting rid of
> >>> the
> >>> "/usr"? I switch off between my two computers
> >>> especially when
> >>> get mucked up, as with my upgrade to kde4. (Otherwise, I
> >>> do
> >>> backups of ~kline as well as other critical directories.)
> >>>
> >>> Is there a way of automatically using rsync rather that my
> >>> kwik-and-dirty /bin/shell script?
> >>>
> >>> thanks, people,
> >>>
> >>> gary
> >>
> >> If what you wish to do is simply get rid of /usr in a string, you can use
> >> sed like so:
> >> varWithoutUsr=`echo ${varWithUsr} |sed -e 's/\/usr//'`
> >> After running this, where $varWithUsr is the variable containing a
> >> string like "/usr/home/blah", the variable $varWithoutUsr will be equal
> >> to "/home/blah". I create simple scripts like this all the time to
> >> rename batches of files, for example.
> >>
> >> The easier way is probably just to not specify a dir to scp's remote
> >> path though, since it defaults to the user's home directory.
> >
> > Or, in anything resembling Bourne shell:
> >
> > varWithoutUsr=${varWithUsr#/usr}
>
> And I'll take a moment to recommend Matthew's method, since it does not
> involve fork()ing an additional process.
>
> When writing shell scripts in general, it's best if you can avoid
> spawning external processes for things which can be done easily
> (keyword: easily!) within Bourne natively. There's no harm in doing it
> for more complex things, but fork() is somewhat expensive, and try to
> imagine what will happen to those scripts if the system lacks process
> table space, etc... :-) Best to try and make everything
> "self-contained" if possible.
right; esp'ly since i'll be running at least two scripts daily--
at a min. besides, the simpler /bin/sh script is something i use
to save code or prose just in case the sky falls!
ah, no wonder this is the best list in the {known} universe
>
> --
> | Jeremy Chadwickjdc at parodius.com |
> | Parodius Networking http://www.parodius.com/ |
> | UNIX Systems Administrator Mountain View, CA, USA |
> | Making life hard for others since 1977. PGP: 4BD6C0CB |
>
--
Gary Kline [EMAIL PROTECTED] http://www.thought.org Public Service Unix
http://jottings.thought.org http://transfinite.thought.org
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync or even scp questions....
On Sun, Oct 12, 2008 at 09:42:38AM +0100, Matthew Seaman wrote:
> mdh wrote:
> >--- On Sat, 10/11/08, Gary Kline <[EMAIL PROTECTED]> wrote:
> >>On the Ubuntu computer I am /home/kline; on my main
> >>computer,
> >>my home is /usr/home/kline. The following sh script
> >>worked
> >>perfected when my home on "tao" [FBSD] was
> >>/home/kline:
> >>
> >>P
> >>#!/bin/sh
> >>
> >>PWD=`pwd`;
> >>echo "This directory is [${PWD}]";
> >>
> >>scp -qrp ${PWD}/* ethos:/${PWD}
> >>###/usr/bin/scp -rqp -i /home/kline/.ssh/zeropasswd-id
> >>${PWD}/* \ klin
> >>[EMAIL PROTECTED]:/${PWD}
> >>
> >>Question #1: is there any /bin/sh method of getting rid of
> >>the
> >>"/usr"? I switch off between my two computers
> >>especially when
> >>get mucked up, as with my upgrade to kde4. (Otherwise, I
> >>do
> >>backups of ~kline as well as other critical directories.)
> >>
> >>Is there a way of automatically using rsync rather that my
> >>kwik-and-dirty /bin/shell script?
> >>
> >>thanks, people,
> >>
> >>gary
> >
> >If what you wish to do is simply get rid of /usr in a string, you can use
> >sed like so:
> >varWithoutUsr=`echo ${varWithUsr} |sed -e 's/\/usr//'`
> >After running this, where $varWithUsr is the variable containing a string
> >like "/usr/home/blah", the variable $varWithoutUsr will be equal to
> >"/home/blah". I create simple scripts like this all the time to rename
> >batches of files, for example.
> >The easier way is probably just to not specify a dir to scp's remote path
> >though, since it defaults to the user's home directory.
>
> Or, in anything resembling Bourne shell:
>
> varWithoutUsr=${varWithUsr#/usr}
I'll be damrned! It works--I've used the zsh for almost 20
years; it's a ksh clone++. How, may I ask, does this work?
(I've seen ksh chopping from the RHS; I wrote a short C util to
axe any part of a string, but have never seen *this* voodoo.
LOL++)
In any event, merci infiniement!
gary
PS: this will save my rsync scripts too.
>
> Cheers,
>
> Matthew
>
> --
> Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
> Flat 3
> PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
> Kent, CT11 9PW
>
--
Gary Kline [EMAIL PROTECTED] http://www.thought.org Public Service Unix
http://jottings.thought.org http://transfinite.thought.org
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync or even scp questions....
On Sun, Oct 12, 2008 at 09:42:38AM +0100, Matthew Seaman wrote:
> mdh wrote:
>> --- On Sat, 10/11/08, Gary Kline <[EMAIL PROTECTED]> wrote:
>>> On the Ubuntu computer I am /home/kline; on my main
>>> computer,
>>> my home is /usr/home/kline. The following sh script
>>> worked
>>> perfected when my home on "tao" [FBSD] was
>>> /home/kline:
>>>
>>> P
>>> #!/bin/sh
>>>
>>> PWD=`pwd`;
>>> echo "This directory is [${PWD}]";
>>>
>>> scp -qrp ${PWD}/* ethos:/${PWD}
>>> ###/usr/bin/scp -rqp -i /home/kline/.ssh/zeropasswd-id
>>> ${PWD}/* \ klin
>>> [EMAIL PROTECTED]:/${PWD}
>>>
>>> Question #1: is there any /bin/sh method of getting rid of
>>> the
>>> "/usr"? I switch off between my two computers
>>> especially when
>>> get mucked up, as with my upgrade to kde4. (Otherwise, I
>>> do
>>> backups of ~kline as well as other critical directories.)
>>>
>>> Is there a way of automatically using rsync rather that my
>>> kwik-and-dirty /bin/shell script?
>>>
>>> thanks, people,
>>>
>>> gary
>>
>> If what you wish to do is simply get rid of /usr in a string, you can use
>> sed like so:
>> varWithoutUsr=`echo ${varWithUsr} |sed -e 's/\/usr//'`
>> After running this, where $varWithUsr is the variable containing a
>> string like "/usr/home/blah", the variable $varWithoutUsr will be equal
>> to "/home/blah". I create simple scripts like this all the time to
>> rename batches of files, for example.
>>
>> The easier way is probably just to not specify a dir to scp's remote
>> path though, since it defaults to the user's home directory.
>
> Or, in anything resembling Bourne shell:
>
> varWithoutUsr=${varWithUsr#/usr}
And I'll take a moment to recommend Matthew's method, since it does not
involve fork()ing an additional process.
When writing shell scripts in general, it's best if you can avoid
spawning external processes for things which can be done easily
(keyword: easily!) within Bourne natively. There's no harm in doing it
for more complex things, but fork() is somewhat expensive, and try to
imagine what will happen to those scripts if the system lacks process
table space, etc... :-) Best to try and make everything
"self-contained" if possible.
--
| Jeremy Chadwickjdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync or even scp questions....
mdh wrote:
--- On Sat, 10/11/08, Gary Kline <[EMAIL PROTECTED]> wrote:
On the Ubuntu computer I am /home/kline; on my main
computer,
my home is /usr/home/kline. The following sh script
worked
perfected when my home on "tao" [FBSD] was
/home/kline:
P
#!/bin/sh
PWD=`pwd`;
echo "This directory is [${PWD}]";
scp -qrp ${PWD}/* ethos:/${PWD}
###/usr/bin/scp -rqp -i /home/kline/.ssh/zeropasswd-id
${PWD}/* \ klin
[EMAIL PROTECTED]:/${PWD}
Question #1: is there any /bin/sh method of getting rid of
the
"/usr"? I switch off between my two computers
especially when
get mucked up, as with my upgrade to kde4. (Otherwise, I
do
backups of ~kline as well as other critical directories.)
Is there a way of automatically using rsync rather that my
kwik-and-dirty /bin/shell script?
thanks, people,
gary
If what you wish to do is simply get rid of /usr in a string, you can use sed
like so:
varWithoutUsr=`echo ${varWithUsr} |sed -e 's/\/usr//'`
After running this, where $varWithUsr is the variable containing a string like "/usr/home/blah", the variable $varWithoutUsr will be equal to "/home/blah". I create simple scripts like this all the time to rename batches of files, for example.
The easier way is probably just to not specify a dir to scp's remote path though, since it defaults to the user's home directory.
Or, in anything resembling Bourne shell:
varWithoutUsr=${varWithUsr#/usr}
Cheers,
Matthew
--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
Kent, CT11 9PW
signature.asc
Description: OpenPGP digital signature
Re: rsync or even scp questions....
--- On Sat, 10/11/08, Gary Kline <[EMAIL PROTECTED]> wrote:
> On the Ubuntu computer I am /home/kline; on my main
> computer,
> my home is /usr/home/kline. The following sh script
> worked
> perfected when my home on "tao" [FBSD] was
> /home/kline:
>
> P
> #!/bin/sh
>
> PWD=`pwd`;
> echo "This directory is [${PWD}]";
>
> scp -qrp ${PWD}/* ethos:/${PWD}
> ###/usr/bin/scp -rqp -i /home/kline/.ssh/zeropasswd-id
> ${PWD}/* \ klin
> [EMAIL PROTECTED]:/${PWD}
>
> Question #1: is there any /bin/sh method of getting rid of
> the
> "/usr"? I switch off between my two computers
> especially when
> get mucked up, as with my upgrade to kde4. (Otherwise, I
> do
> backups of ~kline as well as other critical directories.)
>
> Is there a way of automatically using rsync rather that my
> kwik-and-dirty /bin/shell script?
>
> thanks, people,
>
> gary
If what you wish to do is simply get rid of /usr in a string, you can use sed
like so:
varWithoutUsr=`echo ${varWithUsr} |sed -e 's/\/usr//'`
After running this, where $varWithUsr is the variable containing a string like
"/usr/home/blah", the variable $varWithoutUsr will be equal to "/home/blah". I
create simple scripts like this all the time to rename batches of files, for
example.
The easier way is probably just to not specify a dir to scp's remote path
though, since it defaults to the user's home directory.
- mdh
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync colon in filename to MSWin fails
On Fri, 19 Sep 2008 13:37:50 +0100, Anton Shterenlikht <[EMAIL PROTECTED]> wrote: > I cannot trasfer a file with a colon via rsync to a Win box. > I've rsync-3.0.4 on the FBSD (sending side) and rsync-2.6.9 > under cygwin on Win (receiving side). I'm not sure what the > error message means: > > % rsync ./http:__en.wikipedia.org_favicon.ico [EMAIL PROTECTED]: > rsync: rename "/cygdrive/c/Documents and Settings/mexas/My > Documents/work/.http:__en.wikipedia.org_favicon.ico.8dBX2K" -> > "http:__en.wikipedia.org_favicon.ico": No such file or directory (2) > rsync error: some files/attrs were not transferred (see previous errors) > (code 23) at main.c(1040) [sender=3.0.4] > > Note the extra dot before the file name, and extra suffix at the > end. Is this the expected behaviour? Yes, this is the expected broken behavior of Windows. The ':' character cannot be used in a normal filename in Windows. The bug goes way back, when DOS developers chose to use ':' to denote 'drive names'. > However, to another FBSD box transfer is fine: > > % rsync ./http:__en.wikipedia.org_favicon.ico [EMAIL PROTECTED]: > % UNIX only reserves '/' as the path separator (and in some cases the '\0' character for the end-of-fiename string marker). So the ':' character is valid for file names or directory names. > Is this something to do with Windows, or old rsync version on cygwin? Yes, it has to do with Windows. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Rsync
On Sun, Aug 10, 2008 at 09:01:36PM +0200, Jos Chrispijn wrote: > I have server1 of which I would like to backup some slices/directories > to a backup server2 that in fact is a mirror of the one that I want to > backup. > I don't allow root logon from prompt on Server2. Can someone tell me how > I can force rsync to backup server1 without having a root login > permission on Server2? You could run rsync in daemon mode on server2, then use rsync on server1 to push the data to server2. See rsync(1) and rsyncd.conf(5). Assuming you have a module [backups] in rsyncd.conf on server2, you can do something like this on server1 (where part1 is a partition/directory you want to back up): rsync -acvx --delete /part1/ server2::backups/part1 Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgplJL6ggj2or.pgp Description: PGP signature
Re: Rsync
Thanks for sharing, I will look at the suggest solutions! regards, Jos ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Rsync
Jos Chrispijn wrote: Matthew Seaman wrote: Use anonymous rsync? There's a section on rsyncd in the rsync(1) man page, but most of the meat is in the rsyncd.conf(5) man page. The downside is you'll lose information about user and group ownership of files. Oh, and obviously be careful about limiting where people can access the rsyncd server from, or your precious data may go on an unplanned walkies... I don't want to loose any user and group ownership of files. Would there be another solution without rsync then or does the 'forbidden root login' affect all backup solutions here? If you're going to expand the backup sets on the mirror box back into a second copy of the filesystem, then you definitely need root access on the client (to read any file irrespective of permissions) and on the server (in order to set the ownership and permissions on the files). You can NFS mount the filesystem onto the second server and copy the files locally that way -- but watch out for the way root-owned files are changed to nobody:nobody ownership by default. You can use ggated(8) and ggatec(8) to share the filesystem at low-level between the two machines. It's even possible to combine that with a local filesystem using gmirror(8) to have instantaneous synchronisation of both copies of the data on the two machines, although I wouldn't trust that for anything your livelihood depends on. You can do a similar trick using iSCSI -- you'll need the net/iscsi-target port installed on the server machine and to use the iscsi_initiator(4) driver on the client machines. See also iscontrol(8) However, if you're willing to store a tarball or other archive format as your backup, then you don't need root access on the backup server, although you will still need it on the client. In this case, you can use just about anything: dump(8), tar(1), cpio(1) -- these all give you the option of 'writing to a remote device' which can just be a regular file on your second machine. Usually network writes are over ssh(1), although you will possibly be required to set some variables in the environment to force that to be the case. So all you need is a non-root account on the server that lets root on the clients log into it. That can be arranged using key-based auth quite nicely. Depending on how much stuff you have, and the likelyhood that you'll need to restore it, you could use a full-blown backup system like bacula. It's pretty easy to get bacula to write backup sets to disk, and you get a not bad at all command interface via bconsole to manipulate all that from either the backup client or the backup server host. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: Rsync
Matthew Seaman wrote: Use anonymous rsync? There's a section on rsyncd in the rsync(1) man page, but most of the meat is in the rsyncd.conf(5) man page. The downside is you'll lose information about user and group ownership of files. Oh, and obviously be careful about limiting where people can access the rsyncd server from, or your precious data may go on an unplanned walkies... I don't want to loose any user and group ownership of files. Would there be another solution without rsync then or does the 'forbidden root login' affect all backup solutions here? thanks for sharing, Jos ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Rsync
Jos Chrispijn wrote: I have server1 of which I would like to backup some slices/directories to a backup server2 that in fact is a mirror of the one that I want to backup. I don't allow root logon from prompt on Server2. Can someone tell me how I can force rsync to backup server1 without having a root login permission on Server2? Use anonymous rsync? There's a section on rsyncd in the rsync(1) man page, but most of the meat is in the rsyncd.conf(5) man page. The downside is you'll lose information about user and group ownership of files. Oh, and obviously be careful about limiting where people can access the rsyncd server from, or your precious data may go on an unplanned walkies... Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: Rsync From FreeBSD To Windows
Tim Daneliuk wrote: > I have an smbfs share that mounts at boot time on a 6.3-STABLE system. > I want to rsync from a FreeBSD directory to the Windows share. For > reasons I seem to not be able to discover, rsync insists on copying > every file, every time. The exact command is: > > rsync -va /FreeBSD-dir-tree /windows-mount > > I have tried -O -no-p without success. > > I'm guessing this is a problem mapping the filesystem semantics from > FreeBSD to SMB so that rsync always thinks the files on the > destination SMB share are out of date, but I cannot seem to find the > right magic to overcome this. > > > Ideas? (And TIA), Inevitably, as soon as I posted this, I finally discovered the problem, which is worth describing here to save other people the same suffering: 1) The problem occurs when the SMB mount is a FAT formatted drive. In this case, it was a USB drive plugged into the WinXP machine being used as removable backup medium. 2) The problem occurs because the FAT file entry has insufficient resolution to maintain exactly the same timestamp as FreeBSD. i.e., FreeBSD (and I presume Linux or other Unix variants) have a finer timestamp resolution than does FAT. 3) The fix is to tell rsync to not be so fussy about exact timestamp matches: rsync -va --modify-window=1 src dest 4) This assumes that the FreeBSD server and the Windows machine hosting the share are more-or-less synchronized to the correct absolute time. If they are not, the --modify-window= parameter may have to be larger to accommodate the difference in what each machine thinks the "correct time" is. -- Tim Daneliuk [EMAIL PROTECTED] PGP Key: http://www.tundraware.com/PGP/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync as root for mail servers?
In response to Chris Maness <[EMAIL PROTECTED]>: > I plan on cutting over a server to new hardware, and I was wondering if > I can add cert based login for root (how do I do this)? This is so that > I can use rsync as root to sync the mail spool and home directories. > Will this work? I am using sendmail and wu-imapd. Follow the instructions for setting up ssh keys (there are dozens of howtos all over the 'net) then tweak /etc/ssh/sshd_config to allow root login (the config paramter is pretty obvious). Ensure that you either have a very strong root password, or that password auth for root is disabled first. Then, rsync you stuff using ssh as the connection mechanism (there's a command line switch for this in rsync, don't remember the details) Once you've got things rsynced the first time (while running) shut down all the services on the active machine and rsync again. This second rsync should be very fast so you'll have little downtime. -- Bill Moran http://www.potentialtech.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
RE: rsync as root for mail servers?
You can use the SSH key then change it for security reasons. For directories is working fine, I never tried on the e-mail spool thou. Best Regards Catalin Miclaus -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris Maness Sent: Monday, May 05, 2008 5:31 AM To: freebsd-questions@FreeBSD.org Subject: rsync as root for mail servers? I plan on cutting over a server to new hardware, and I was wondering if I can add cert based login for root (how do I do this)? This is so that I can use rsync as root to sync the mail spool and home directories. Will this work? I am using sendmail and wu-imapd. Thanks, Chris Maness ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync
On 13:36:56 Mar 01, Gary Kline wrote: > What does the "--delete" do?! I want to make a complete copy of, say, > /usr/home/kline/* from here [tao] to my Ubuntu server, ethos, which has > a "/home" mountpoint--- in otheer words: /home/kline/*. If the > "--delete /usr/home* /home" syntax will let rsync rewrite /usr/home to /home, > then fine. I started using rsync with -avzp but then realized that it would not delete the directories/files I deleted at the source *after* the backup. That is why I added the --delete hoping that it would replicate in such a way that both copies look identical. -Girish -- "unix soi qui mal y pense" UNIX to him who evil thinks ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync
On Saturday 01 March 2008 14:04:17 Roger Olofsson wrote: > Gary Kline skrev: > > On Friday 29 February 2008 19:25:50 Girish Venkatachalam wrote: > >> On 18:18:06 Feb 29, Gary Kline wrote: > >>> Is there a flag I can set to use with rsync to backup every file of > >>> /usr/home/* [here on my FBSD desktop] to my Ubuntu desktop that has > >>> only /home/*?? > >>> > >>> Of course, I could always create /usr/home on my other computers. ... > >> > >> Your question is not clear to me. > >> > >> I use > >> > >> $ rsync -avzp --delete /usr/home/* /home > >> > >> if that is what you are looking for. > > > > What does the "--delete" do?! I want to make a complete copy of, say, > > /usr/home/kline/* from here [tao] to my Ubuntu server, ethos, which has > > a "/home" mountpoint--- in otheer words: /home/kline/*. If the > > "--delete /usr/home* /home" syntax will let rsync rewrite /usr/home to > > /home, then fine. > > > > gary > > > >> -Girish > >> ___ > >> freebsd-questions@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-questions > >> To unsubscribe, send any mail to > >> "[EMAIL PROTECTED]" > > --delete makes sure that if you delete a file on the source, it's also > deleted on the mirror. > > (Tip for those that use rsync to backup Windows machines to FreeBSD - > You will most likely have to use --ignore-errors if you are syncing > to/from windows machines with --delete.) > > Just my nickels worth. > > /R Good point. Because in addition to copying files across flavors of 'Nix, I do want to delete the old changes. Especially as my projects evolve. -- Gary Kline [EMAIL PROTECTED] www.thought.org Public Service Unix http://jottings.thought.org http://transfinite.thought.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync
Gary Kline skrev: On Friday 29 February 2008 19:25:50 Girish Venkatachalam wrote: On 18:18:06 Feb 29, Gary Kline wrote: Is there a flag I can set to use with rsync to backup every file of /usr/home/* [here on my FBSD desktop] to my Ubuntu desktop that has only /home/*?? Of course, I could always create /usr/home on my other computers. ... Your question is not clear to me. I use $ rsync -avzp --delete /usr/home/* /home if that is what you are looking for. What does the "--delete" do?! I want to make a complete copy of, say, /usr/home/kline/* from here [tao] to my Ubuntu server, ethos, which has a "/home" mountpoint--- in otheer words: /home/kline/*. If the "--delete /usr/home* /home" syntax will let rsync rewrite /usr/home to /home, then fine. gary -Girish ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" --delete makes sure that if you delete a file on the source, it's also deleted on the mirror. (Tip for those that use rsync to backup Windows machines to FreeBSD - You will most likely have to use --ignore-errors if you are syncing to/from windows machines with --delete.) Just my nickels worth. /R ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync
On Friday 29 February 2008 19:25:50 Girish Venkatachalam wrote: > On 18:18:06 Feb 29, Gary Kline wrote: > > Is there a flag I can set to use with rsync to backup every file of > > /usr/home/* [here on my FBSD desktop] to my Ubuntu desktop that has > > only /home/*?? > > > > Of course, I could always create /usr/home on my other computers. ... > > Your question is not clear to me. > > I use > > $ rsync -avzp --delete /usr/home/* /home > > if that is what you are looking for. What does the "--delete" do?! I want to make a complete copy of, say, /usr/home/kline/* from here [tao] to my Ubuntu server, ethos, which has a "/home" mountpoint--- in otheer words: /home/kline/*. If the "--delete /usr/home* /home" syntax will let rsync rewrite /usr/home to /home, then fine. gary > > -Girish > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "[EMAIL PROTECTED]" -- Gary Kline [EMAIL PROTECTED] www.thought.org Public Service Unix http://jottings.thought.org http://transfinite.thought.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync
On 18:18:06 Feb 29, Gary Kline wrote: > > Is there a flag I can set to use with rsync to backup every file of > /usr/home/* [here on my FBSD desktop] to my Ubuntu desktop that has > only /home/*?? > > Of course, I could always create /usr/home on my other computers. ... > Your question is not clear to me. I use $ rsync -avzp --delete /usr/home/* /home if that is what you are looking for. -Girish ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync
On Fri, Feb 29, 2008 at 07:43:38PM -0700, James wrote: > > On Fri, 2008-02-29 at 18:18 -0800, Gary Kline wrote: > > Is there a flag I can set to use with rsync to backup every file of > > /usr/home/* [here on my FBSD desktop] to my Ubuntu desktop that has > > only /home/*?? > > > > Of course, I could always create /usr/home on my other computers. ... > > > > tia, > > > > gary > > > > > > rsync -av /usr/home/* [EMAIL PROTECTED]:/home/backup > Well, problem solved! :-) -- Gary Kline [EMAIL PROTECTED] www.thought.org Public Service Unix http://jottings.thought.org http://transfinite.thought.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync
On Fri, 2008-02-29 at 18:18 -0800, Gary Kline wrote: > Is there a flag I can set to use with rsync to backup every file of > /usr/home/* [here on my FBSD desktop] to my Ubuntu desktop that has > only /home/*?? > > Of course, I could always create /usr/home on my other computers. ... > > tia, > > gary > > rsync -av /usr/home/* [EMAIL PROTECTED]:/home/backup ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync to lacie disk (via samba?)
Jerahmy Pocott wrote: > Hello, > > I'm wanting to use rsync from FreeBSD to some lacie ethernet disks, > they have a number of access options including ftp and windows > file sharing.. > > Would mounting the shares with samba then using rsync on the mounted > samba share as though it was syncing between two local disks work properly? I've had problems with it: 1) There may be a loss of precision between the local unix-like file system and the remote file system 2) Attributes are only simulated on smbfs: a chmod call will silently fail as it doesn't have an equivalent on Windows file systems. In any case, rsync would repeatedly find the remote files have changed when they have not. > The other option would be to build the OS on the lacie with rsync and ssh, > as it is linux based OS under GPL but I would probably rather not have to > mess with it.. If you can do it, it would probably be the safest (as in: data integrity) way. signature.asc Description: OpenPGP digital signature
Re: rsync and perms...
Jerry McAllister skrev: On Sat, Jun 16, 2007 at 04:20:42PM -0300, Agus wrote: 2007/6/16, Bill Campbell <[EMAIL PROTECTED]>: On Sat, Jun 16, 2007, Agus wrote: Hi all, Trying to use rsync to update my home page in my freebsd, i am not able to keep the owner,group and perms of the files... I do all the updates in my ubntu. when everything is ready I run from my ubuntu: rsync -uav --delete --rsh=ssh /home/user/web/ [EMAIL PROTECTED]:/root/web/ I tried also with -pog and same results... Files are updated but the perms and owner and group changes to the ones in my ubuntu I want to keep the bsd existing perms and users...How can i do that?? If the users and groups are the same on the system, the -a option to rsync should preserve owner, group, and permissions. One can also use rsync modules in the rsyncd.conf file to specify user and group. Using rsync modules also has the advantages of restricting access to the directory specified in the module, and can also restrict access based on IP addresses. Bill -- INTERNET: [EMAIL PROTECTED] Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676 http://www.celestial.com/ You need only reflect that one of the best ways to get yourself a reputation as a dangerous citizen these days is to go about repeating the very phrases which our founding fathers used in the struggle for independence. -- Charles A. Beard ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to " [EMAIL PROTECTED]" OK...so i should first create a group an user with same ids on both machines? or only the names will suffice? And about the modules, i should be running the rsunc daemon which i am not, so i think i ll do the first... I believe the numerical UID and GID is what matters and has to be the same rather than the names. But correct me if I am wrong. from man rsync: --numeric-ids don't map uid/gid values by user/group name jerry Thanks a lot for your help... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync and perms...
On Sat, Jun 16, 2007 at 04:20:42PM -0300, Agus wrote: > 2007/6/16, Bill Campbell <[EMAIL PROTECTED]>: > > > >On Sat, Jun 16, 2007, Agus wrote: > >>Hi all, > >>Trying to use rsync to update my home page in my freebsd, i am not able > >to > >>keep the owner,group and perms of the files... > >>I do all the updates in my ubntu. when everything is ready I run from my > >>ubuntu: > >>rsync -uav --delete --rsh=ssh /home/user/web/ [EMAIL PROTECTED]:/root/web/ > >> > >>I tried also with -pog and same results... > >> > >>Files are updated but the perms and owner and group changes to the ones > >in > >>my ubuntu > >> > >>I want to keep the bsd existing perms and users...How can i do that?? > > > >If the users and groups are the same on the system, the -a option > >to rsync should preserve owner, group, and permissions. > > > >One can also use rsync modules in the rsyncd.conf file to specify > >user and group. Using rsync modules also has the advantages of > >restricting access to the directory specified in the module, and > >can also restrict access based on IP addresses. > > > >Bill > >-- > >INTERNET: [EMAIL PROTECTED] Bill Campbell; Celestial Software LLC > >URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way > >FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) > >236-1676 > >http://www.celestial.com/ > > > >You need only reflect that one of the best ways to get yourself a > >reputation as a dangerous citizen these days is to go about repeating > >the very phrases which our founding fathers used in the struggle for > >independence. -- Charles A. Beard > >___ > >freebsd-questions@freebsd.org mailing list > >http://lists.freebsd.org/mailman/listinfo/freebsd-questions > >To unsubscribe, send any mail to " > >[EMAIL PROTECTED]" > > > > OK...so i should first create a group an user with same ids on both > machines? or only the names will suffice? > And about the modules, i should be running the rsunc daemon which i am not, > so i think i ll do the first... I believe the numerical UID and GID is what matters and has to be the same rather than the names. But correct me if I am wrong. jerry > > Thanks a lot for your help... > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync and perms...
2007/6/16, Bill Campbell <[EMAIL PROTECTED]>: On Sat, Jun 16, 2007, Agus wrote: >Hi all, >Trying to use rsync to update my home page in my freebsd, i am not able to >keep the owner,group and perms of the files... >I do all the updates in my ubntu. when everything is ready I run from my >ubuntu: >rsync -uav --delete --rsh=ssh /home/user/web/ [EMAIL PROTECTED]:/root/web/ > >I tried also with -pog and same results... > >Files are updated but the perms and owner and group changes to the ones in >my ubuntu > >I want to keep the bsd existing perms and users...How can i do that?? If the users and groups are the same on the system, the -a option to rsync should preserve owner, group, and permissions. One can also use rsync modules in the rsyncd.conf file to specify user and group. Using rsync modules also has the advantages of restricting access to the directory specified in the module, and can also restrict access based on IP addresses. Bill -- INTERNET: [EMAIL PROTECTED] Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676 http://www.celestial.com/ You need only reflect that one of the best ways to get yourself a reputation as a dangerous citizen these days is to go about repeating the very phrases which our founding fathers used in the struggle for independence. -- Charles A. Beard ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to " [EMAIL PROTECTED]" OK...so i should first create a group an user with same ids on both machines? or only the names will suffice? And about the modules, i should be running the rsunc daemon which i am not, so i think i ll do the first... Thanks a lot for your help... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync and perms...
On Sat, Jun 16, 2007, Agus wrote: >Hi all, >Trying to use rsync to update my home page in my freebsd, i am not able to >keep the owner,group and perms of the files... >I do all the updates in my ubntu. when everything is ready I run from my >ubuntu: >rsync -uav --delete --rsh=ssh /home/user/web/ [EMAIL PROTECTED]:/root/web/ > >I tried also with -pog and same results... > >Files are updated but the perms and owner and group changes to the ones in >my ubuntu > >I want to keep the bsd existing perms and users...How can i do that?? If the users and groups are the same on the system, the -a option to rsync should preserve owner, group, and permissions. One can also use rsync modules in the rsyncd.conf file to specify user and group. Using rsync modules also has the advantages of restricting access to the directory specified in the module, and can also restrict access based on IP addresses. Bill -- INTERNET: [EMAIL PROTECTED] Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676 http://www.celestial.com/ You need only reflect that one of the best ways to get yourself a reputation as a dangerous citizen these days is to go about repeating the very phrases which our founding fathers used in the struggle for independence. -- Charles A. Beard ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync install md5 error
Great...thanks a lot.. Greetz 2007/6/8, Beech Rintoul <[EMAIL PROTECTED]>: > 2007/6/8, Beech Rintoul <[EMAIL PROTECTED]>: > > On Friday 08 June 2007, Agus said: > > > Hi all, > > > I am trying to install /usr/ports/net/rsync and i am getting > > > MD5 checksum mismatch for rsync-2.6.6.tar.gz > > > > > > then it says if u are sure u want to override this check, type > > > make NO_CHECKSUM=yes > > > > > > What does it mean that the MD5 and sha256 checksums are wrong? > > > > > > How can i solve it?? > > > > Try "make distclean" then "make install clean". > > > > Cheers, > > > > Beech >On Friday 08 June 2007, Agus said: > I see that in samba.org the version is 2.6.9...so my bsd tries to > download it from another place > is there a way to make install version 2.6.9 instead of the default > that is trying?? 2.6.9 is the current version. You need to update your ports tree. See the handbook for details. Beech -- --- Beech Rintoul - FreeBSD Developer - [EMAIL PROTECTED] /"\ ASCII Ribbon Campaign | FreeBSD Since 4.x \ / - NO HTML/RTF in e-mail | http://www.freebsd.org X - NO Word docs in e-mail | Latest Release: / \ - http://www.freebsd.org/releases/6.2R/announce.html --- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync install md5 error
> 2007/6/8, Beech Rintoul <[EMAIL PROTECTED]>: > > On Friday 08 June 2007, Agus said: > > > Hi all, > > > I am trying to install /usr/ports/net/rsync and i am getting > > > MD5 checksum mismatch for rsync-2.6.6.tar.gz > > > > > > then it says if u are sure u want to override this check, type > > > make NO_CHECKSUM=yes > > > > > > What does it mean that the MD5 and sha256 checksums are wrong? > > > > > > How can i solve it?? > > > > Try "make distclean" then "make install clean". > > > > Cheers, > > > > Beech >On Friday 08 June 2007, Agus said: > I see that in samba.org the version is 2.6.9...so my bsd tries to > download it from another place > is there a way to make install version 2.6.9 instead of the default > that is trying?? 2.6.9 is the current version. You need to update your ports tree. See the handbook for details. Beech -- --- Beech Rintoul - FreeBSD Developer - [EMAIL PROTECTED] /"\ ASCII Ribbon Campaign | FreeBSD Since 4.x \ / - NO HTML/RTF in e-mail | http://www.freebsd.org X - NO Word docs in e-mail | Latest Release: / \ - http://www.freebsd.org/releases/6.2R/announce.html --- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync install md5 error
I see that in samba.org the version is 2.6.9...so my bsd tries to download it from another place is there a way to make install version 2.6.9 instead of the default that is trying?? thanksss 2007/6/8, Beech Rintoul <[EMAIL PROTECTED]>: On Friday 08 June 2007, Agus said: > Hi all, > I am trying to install /usr/ports/net/rsync and i am getting MD5 > checksum mismatch for rsync-2.6.6.tar.gz > > then it says if u are sure u want to override this check, type make > NO_CHECKSUM=yes > > What does it mean that the MD5 and sha256 checksums are wrong? > > How can i solve it?? Try "make distclean" then "make install clean". Cheers, Beech -- --- Beech Rintoul - FreeBSD Developer - [EMAIL PROTECTED] /"\ ASCII Ribbon Campaign | FreeBSD Since 4.x \ / - NO HTML/RTF in e-mail | http://www.freebsd.org X - NO Word docs in e-mail | Latest Release: / \ - http://www.freebsd.org/releases/6.2R/announce.html --- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync install md5 error
On Friday 08 June 2007, Agus said: > Hi all, > I am trying to install /usr/ports/net/rsync and i am getting MD5 > checksum mismatch for rsync-2.6.6.tar.gz > > then it says if u are sure u want to override this check, type make > NO_CHECKSUM=yes > > What does it mean that the MD5 and sha256 checksums are wrong? > > How can i solve it?? Try "make distclean" then "make install clean". Cheers, Beech -- --- Beech Rintoul - FreeBSD Developer - [EMAIL PROTECTED] /"\ ASCII Ribbon Campaign | FreeBSD Since 4.x \ / - NO HTML/RTF in e-mail | http://www.freebsd.org X - NO Word docs in e-mail | Latest Release: / \ - http://www.freebsd.org/releases/6.2R/announce.html --- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync install md5 error
On 6/9/07, Agus <[EMAIL PROTECTED]> wrote: Hi all, I am trying to install /usr/ports/net/rsync and i am getting MD5 checksum mismatch for rsync-2.6.6.tar.gz then it says if u are sure u want to override this check, type make NO_CHECKSUM=yes What does it mean that the MD5 and sha256 checksums are wrong? How can i solve it?? thankss ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to " [EMAIL PROTECTED]" It basically means the file that it downloaded is bad, try deleting the rsync-2.6.6.tar.gz from /usr/ports/distfiles/ and if that doesn't work try downloading that same file manually. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Rsync w/ Windows
At 09:31 Fri 16 Mar 2007, Chris Maness wrote: I need to sync a directory with my freebsd box. In linux/bsd I use the command rsync -vaur [EMAIL PROTECTED]:/home/chris/beer /home/chris/beer and this works perfectly for me. However, using the cwrsync package in windows with this syntax does not work. Do you get an error message when you run the command above? Since it appears you are able sync to the FreeBSD box from other machines, it sounds like this question is maybe a little OT here. You might have better luck asking the cwrsync folks. That having been said, here's what I know. At work we use cwrsync to backup some Windows boxes to a NAS sharing directories over samba. Our rsync looks more or less like this (cutting some switches for clarity): rsync -rvt --delete "/cygdrive/c/Documents and Settings/lcapps" \ //SNAP203762/backups/staff/lcapps/pc_bak A couple of things -- are you sure you're specifying the "cygdrive" correctly? How are you accessing the remote box from Windows? SSH? (I don't have any experience using rsync that way.) Our Windows machines are already connected to the shared directory before rsync runs. Good luck, -- Lee Capps Technology Specialist CTE Resource Center ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Rsync w/ Windows
Roger Olofsson wrote: Hello Chris, May I suggest that you take a peek at http://www.itefix.no/phpws/index.php?module=pagemaster&PAGE_user_op=view_page&PAGE_id=6&MMN_position=23:23 As example I have the following syntax in my cwrsync.cmd (In this example I backup documents and settings from pc to FreeBSD by using the script cwrsync.cmd included in the cwrsync distribution. The script can be run with the windows scheduler) rsync --exclude-from=excludefiles.txt -avz --delete -e "ssh -i bin/.ssh/identify" "/cygdrive/c/documents and settings/username" [EMAIL PROTECTED]:/path/on/freebsd/server You will need to create a folder named .ssh in the bin folder of cwrsync and in that folder create the necessary ssh keys. Good luck! Thank you. This works awesome. -- Chris Maness (909) 223-9179 http://www.chrismaness.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Rsync w/ Windows
Hello Chris, May I suggest that you take a peek at http://www.itefix.no/phpws/index.php?module=pagemaster&PAGE_user_op=view_page&PAGE_id=6&MMN_position=23:23 As example I have the following syntax in my cwrsync.cmd (In this example I backup documents and settings from pc to FreeBSD by using the script cwrsync.cmd included in the cwrsync distribution. The script can be run with the windows scheduler) rsync --exclude-from=excludefiles.txt -avz --delete -e "ssh -i bin/.ssh/identify" "/cygdrive/c/documents and settings/username" [EMAIL PROTECTED]:/path/on/freebsd/server You will need to create a folder named .ssh in the bin folder of cwrsync and in that folder create the necessary ssh keys. Good luck! Chris Maness skrev: I need to sync a directory with my freebsd box. In linux/bsd I use the command rsync -vaur [EMAIL PROTECTED]:/home/chris/beer /home/chris/beer and this works perfectly for me. However, using the cwrsync package in windows with this syntax does not work. Pleas help! Thanks... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Rsync w/ Windows
On 3/16/07, Chris Maness <[EMAIL PROTECTED]> wrote: I need to sync a directory with my freebsd box. In linux/bsd I use the command rsync -vaur [EMAIL PROTECTED]:/home/chris/beer /home/chris/beer and this works perfectly for me. However, using the cwrsync package in windows with this syntax does not work. Pleas help! Thanks... -- Chris Maness (909) 223-9179 http://www.chrismaness.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] You have to tell windows the full path including drive letter. This is what i use to sync Xp laptop to freebsd server: rsync.exe -r /cygdrive/c/z-test/ [EMAIL PROTECTED]:/home/jeremy/backup/ thanks, jeremy ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync and trailing '/'
From: Noah <[EMAIL PROTECTED]> Subject: rsync and trailing '/' To: User Questions Message-ID: <[EMAIL PROTECTED]> Content-Type: text/plain; charset=ISO-8859-1; format=flowed rsync version 2.6.9 protocol version 29 I am running rsync, and I wonder if having the trailing '/' at the end of the directory is proper configuration? --- snip --- /usr/local/bin/rsync -avz -e ssh [EMAIL PROTECTED]:/var/cron/ /var/cron/ --- snip --- Cheers, Noah Whoa, Noah duuude! That trailing slash makes mondo difference. Use with caution FBSD dudes! -- Yudhvir Singh Sidhu Mars or Bust ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync and trailing '/'
Noah wrote: > rsync version 2.6.9 protocol version 29 > > I am running rsync, and I wonder if having the trailing '/' at the end > of the directory is proper configuration? > > --- snip --- > > /usr/local/bin/rsync -avz -e ssh [EMAIL PROTECTED]:/var/cron/ /var/cron/ > Presence of absence of the final '/' on the rsync *source* directory means different things. If you say: rsync -a foo/ bar/ directory bar will end up with a copy of the contents of foo. So a file foo/wibble.txt will end up as bar/wibble.txt. If on the other hand you say: rsync -a foo bar/ then you end up with a directory structure 'bar/foo/...', and wibble.txt ends up as bar/foo/wibble.txt Other than that, the presence or absence of the trailing slash on the *target* directory doesn't make a great deal of difference. I usually try and work things so that the rsync command line always has trailing slashes on both the source and the destination directories simply for consistencies' sake. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. Flat 3 7 Priory Courtyard PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW, UK signature.asc Description: OpenPGP digital signature
Re: rsync issues
just to go through what i'd normally check: 1) make sure the source host is up at 100full 2) make sure the port the source host is plugged into is 100full 3) make sure the destination host is up at 100full 4) make sure the port the destination host is plugged into is 100full given all of that, if you still have trouble you might want to start doing some packet dumps and see if one host or the other are terminating the connection: $ tcpdump -n -vvv -s 1518 -g On 23/01/07, Peter Pluta <[EMAIL PROTECTED]> wrote: I've gotten a few replies and i've tried almost all of the suggestions but the problem still resides. I re-installed rsync on both box's and it still does this. Peter Pluta wrote: > Anyone? > > Peter Pluta wrote: >> I have a win2k3 server running as my rsync server. I also have a >> freebsd web server being the rsync client. A shell script runs every >> night at 5am (it's below). >> >> Shell script: >> >> #!/bin/sh >> . `dirname $0`/settings.inc >> destination=**.***.***.***::backup >> if [ "$TERM" ]; then verbose=-v; fi >> rsync $verbose -azR --delete-after /usr/local/etc/ $destination >> rsync $verbose -azR --delete-after /usr/local/lib/sasl2/ $destination >> rsync $verbose -azR --delete-after /var/cron/$destination >> rsync $verbose -azR --delete-after /root/$destination >> rsync $verbose -azR --delete-after /etc/ $destination >> rsync $verbose -azR --delete-after --exclude httpd-*.log $wwwDir/ >> $destination >> >> After it runs for 5 minutes it throws this: >> rsync: writefd_unbuffered failed to write 16385 bytes [sender]: >> Broken pipe (32) >> rsync: read error: Connection reset by peer (54) >> rsync error: error in rsync protocol data stream (code 12) at >> io.c(613) [sender=2.6.9] >> >> Dmesg on the box only shows this: >> em0: promiscuous mode enabled >> em0: promiscuous mode disabled >> >> But that is probably pretty old. >> >> What can the problem be? backups are really important to me and they >> don't currently work as the transfer times out after the first few >> files. >> >> Anyone got an idea? Any feedback or suggestions would be greatly >> appreciated. >> >> Thanks, >> Peter >> ___ >> freebsd-questions@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-questions >> To unsubscribe, send any mail to >> "[EMAIL PROTECTED]" > > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" -- Greg Albrecht ([EMAIL PROTECTED]) An Indie, Hip Hop and IDM Podcast: The Letter G http://theletterg.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync issues
Hi, > I've gotten a few replies and i've tried almost all of the suggestions > but the problem still resides. I re-installed rsync on both box's and it > still does this. Most likely won't help, but do you have any sort of traffic shaping between the two boxes (not sure if they are on the same LAN or remote), but it is something to check - give appropriate bandwidth for the operation. If they are remotely localted, how about trying to rsync locally (picking up a similar machine) and see if you get the same error. I am not a technical guy. I am just suggesting to check QoS. Warm regards, -- Zbigniew Szalbot ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync issues
I've gotten a few replies and i've tried almost all of the suggestions but the problem still resides. I re-installed rsync on both box's and it still does this. Peter Pluta wrote: Anyone? Peter Pluta wrote: I have a win2k3 server running as my rsync server. I also have a freebsd web server being the rsync client. A shell script runs every night at 5am (it's below). Shell script: #!/bin/sh . `dirname $0`/settings.inc destination=**.***.***.***::backup if [ "$TERM" ]; then verbose=-v; fi rsync $verbose -azR --delete-after /usr/local/etc/ $destination rsync $verbose -azR --delete-after /usr/local/lib/sasl2/ $destination rsync $verbose -azR --delete-after /var/cron/$destination rsync $verbose -azR --delete-after /root/$destination rsync $verbose -azR --delete-after /etc/ $destination rsync $verbose -azR --delete-after --exclude httpd-*.log $wwwDir/ $destination After it runs for 5 minutes it throws this: rsync: writefd_unbuffered failed to write 16385 bytes [sender]: Broken pipe (32) rsync: read error: Connection reset by peer (54) rsync error: error in rsync protocol data stream (code 12) at io.c(613) [sender=2.6.9] Dmesg on the box only shows this: em0: promiscuous mode enabled em0: promiscuous mode disabled But that is probably pretty old. What can the problem be? backups are really important to me and they don't currently work as the transfer times out after the first few files. Anyone got an idea? Any feedback or suggestions would be greatly appreciated. Thanks, Peter ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync issues
On 21/01/07, Peter Pluta <[EMAIL PROTECTED]> wrote: After it runs for 5 minutes it throws this: rsync: writefd_unbuffered failed to write 16385 bytes [sender]: Broken pipe (32) rsync: read error: Connection reset by peer (54) rsync error: error in rsync protocol data stream (code 12) at io.c(613) [sender=2.6.9] i had a similar problem and the fix ended up being pretty simple: make sure the windows box has the right duplex setting, eg: 100full -g -- Greg Albrecht ([EMAIL PROTECTED]) An Indie, Hip Hop and IDM Podcast: The Letter G http://theletterg.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync issues
Peter Pluta wrote: Anyone? Peter Pluta wrote: I have a win2k3 server running as my rsync server. I also have a freebsd web server being the rsync client. A shell script runs every night at 5am (it's below). Shell script: #!/bin/sh . `dirname $0`/settings.inc destination=**.***.***.***::backup if [ "$TERM" ]; then verbose=-v; fi rsync $verbose -azR --delete-after /usr/local/etc/ $destination rsync $verbose -azR --delete-after /usr/local/lib/sasl2/ $destination rsync $verbose -azR --delete-after /var/cron/$destination rsync $verbose -azR --delete-after /root/$destination rsync $verbose -azR --delete-after /etc/ $destination rsync $verbose -azR --delete-after --exclude httpd-*.log $wwwDir/ $destination After it runs for 5 minutes it throws this: rsync: writefd_unbuffered failed to write 16385 bytes [sender]: Broken pipe (32) rsync: read error: Connection reset by peer (54) rsync error: error in rsync protocol data stream (code 12) at io.c(613) [sender=2.6.9] Dmesg on the box only shows this: em0: promiscuous mode enabled em0: promiscuous mode disabled But that is probably pretty old. What can the problem be? backups are really important to me and they don't currently work as the transfer times out after the first few files. Anyone got an idea? Any feedback or suggestions would be greatly appreciated. Thanks, Peter ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" Looks like the 2k3 box is resetting the connection for whatever reason. -- Jay Chandler Network Administrator, Chapman University 714.628.7249 / [EMAIL PROTECTED] Today's Excuse: excess surge protection ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync issues
In response to Peter Pluta <[EMAIL PROTECTED]>: > > Peter Pluta wrote: > > I have a win2k3 server running as my rsync server. I also have a > > freebsd web server being the rsync client. A shell script runs every > > night at 5am (it's below). > > > > Shell script: > > > > #!/bin/sh > > . `dirname $0`/settings.inc > > destination=**.***.***.***::backup > > if [ "$TERM" ]; then verbose=-v; fi > > rsync $verbose -azR --delete-after /usr/local/etc/ $destination > > rsync $verbose -azR --delete-after /usr/local/lib/sasl2/ $destination > > rsync $verbose -azR --delete-after /var/cron/$destination > > rsync $verbose -azR --delete-after /root/$destination > > rsync $verbose -azR --delete-after /etc/ $destination > > rsync $verbose -azR --delete-after --exclude httpd-*.log $wwwDir/ > > $destination > > > > After it runs for 5 minutes it throws this: > > rsync: writefd_unbuffered failed to write 16385 bytes [sender]: Broken > > pipe (32) > > rsync: read error: Connection reset by peer (54) > > rsync error: error in rsync protocol data stream (code 12) at > > io.c(613) [sender=2.6.9] > > > > Dmesg on the box only shows this: > > em0: promiscuous mode enabled > > em0: promiscuous mode disabled > > > > But that is probably pretty old. > > > > What can the problem be? backups are really important to me and they > > don't currently work as the transfer times out after the first few files. > > > > Anyone got an idea? Any feedback or suggestions would be greatly > > appreciated. I don't know what your problem is, but I can make some recommendations on debugging it. *) Are you running it verbosely when this happens? Crank the verbosity up as high as it will go on both the client and the server and see if anything shows up. Is the a DEBUG option available if you recompile? *) Got any network monitoring stuff available? Heavy packet loss? *) Try ktracing the process while it's running. Should narrow down the cause a good bit. Or maybe attach gdb to it. *) Try rsycing to a local directory to see if it still happens. That should narrow the problem down to either network or not. *) fsck your disks? Hope some of this is helpful. Generally, when I have mystery errors, I start with ktrace. If you're not familiar with it, ktrace can be a bit overwhelming, but it's got lotsa useful information. Same can be said for gdb. -- Bill Moran Collaborative Fusion Inc. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync issues
Anyone? Peter Pluta wrote: I have a win2k3 server running as my rsync server. I also have a freebsd web server being the rsync client. A shell script runs every night at 5am (it's below). Shell script: #!/bin/sh . `dirname $0`/settings.inc destination=**.***.***.***::backup if [ "$TERM" ]; then verbose=-v; fi rsync $verbose -azR --delete-after /usr/local/etc/ $destination rsync $verbose -azR --delete-after /usr/local/lib/sasl2/ $destination rsync $verbose -azR --delete-after /var/cron/$destination rsync $verbose -azR --delete-after /root/$destination rsync $verbose -azR --delete-after /etc/ $destination rsync $verbose -azR --delete-after --exclude httpd-*.log $wwwDir/ $destination After it runs for 5 minutes it throws this: rsync: writefd_unbuffered failed to write 16385 bytes [sender]: Broken pipe (32) rsync: read error: Connection reset by peer (54) rsync error: error in rsync protocol data stream (code 12) at io.c(613) [sender=2.6.9] Dmesg on the box only shows this: em0: promiscuous mode enabled em0: promiscuous mode disabled But that is probably pretty old. What can the problem be? backups are really important to me and they don't currently work as the transfer times out after the first few files. Anyone got an idea? Any feedback or suggestions would be greatly appreciated. Thanks, Peter ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync does not --exclude: (??)
On Sat, 28 Oct 2006 11:39 am, Lars Kristiansen wrote: > Gary Kline skrev: > > People, > > > > Weeks ago I tried this *without* the --exclude flags and > > overwrote DOT file that I didn't want to. Iust rechecked > > this use of rsync, saving important files from my work > > server elseswhere, and see that some files or directories > > are being listed and copied over. IDoes anything obvious > > jump out here? > > > > gary > > > > > > > > # > > ## test2ethos.sh > > # > > > >rsync -Cptuvaz -n -e "ssh -i > > /home/kline/.ssh/tao_XXXYYYZZZ-id" \ --exclude > > /home/kline/.zlogin \ > >--exclude /home/kline/.zshrc \ > >--exclude /home/kline/.zhistory.* \ > >--exclude /home/kline/.ctwrm \ > >--exclude /home/kline/.mozilla \ > >--exclude /home/kline/.evolution \ > >--exclude /home/kline/4zip/ \ > >--exclude '/home/kline/www/' \ > >/home/kline [EMAIL PROTECTED]:/home/kline > > Hope this might help. > From man rsync: > "global include/exclude patterns are anchored at the "root of > the transfer" in man rsync look for: "INCLUDE/EXCLUDE PATTERN > RULES" and "ANCHORING INCLUDE/EXCLUDE PATTERNS" > Yes, it looks as though the anchoring of exclude arguments is more complex (or at least different) than I imagined. Never-the-less ambiguous arguments will still need to be quoted. Malcolm > Regards, Lars > > >if [ $? = 0 ] > >then > > echo "rsync transfer went okay: tao to ethos" > >else > > echo "rsync failed to ethos from /home/kline"|mail > > [EMAIL PROTECTED] fi > > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync does not --exclude: (??)
Gary Kline skrev: People, Weeks ago I tried this *without* the --exclude flags and overwrote DOT file that I didn't want to. Iust rechecked this use of rsync, saving important files from my work server elseswhere, and see that some files or directories are being listed and copied over. IDoes anything obvious jump out here? gary # ## test2ethos.sh # rsync -Cptuvaz -n -e "ssh -i /home/kline/.ssh/tao_XXXYYYZZZ-id" \ --exclude /home/kline/.zlogin \ --exclude /home/kline/.zshrc \ --exclude /home/kline/.zhistory.* \ --exclude /home/kline/.ctwrm \ --exclude /home/kline/.mozilla \ --exclude /home/kline/.evolution \ --exclude /home/kline/4zip/ \ --exclude '/home/kline/www/' \ /home/kline [EMAIL PROTECTED]:/home/kline Hope this might help. From man rsync: "global include/exclude patterns are anchored at the "root of the transfer" in man rsync look for: "INCLUDE/EXCLUDE PATTERN RULES" and "ANCHORING INCLUDE/EXCLUDE PATTERNS" Regards, Lars if [ $? = 0 ] then echo "rsync transfer went okay: tao to ethos" else echo "rsync failed to ethos from /home/kline"|mail [EMAIL PROTECTED] fi ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync does not --exclude: (??)
On Sat, Oct 28, 2006 at 06:18:44AM +0930, Malcolm Kay wrote: > On Sat, 28 Oct 2006 03:15 am, Gary Kline wrote: > > People, > > > > Weeks ago I tried this *without* the --exclude flags and > > overwrote DOT file that I didn't want to. Iust rechecked > > this use of rsync, saving important files from my work server > > elseswhere, and see that some files or directories are being > > listed and copied over. IDoes anything obvious jump out here? > > > > gary > > > > > > > > # > > ## test2ethos.sh > > # > > > >rsync -Cptuvaz -n -e "ssh -i > > /home/kline/.ssh/tao_XXXYYYZZZ-id" \ --exclude > > /home/kline/.zlogin \ > >--exclude /home/kline/.zshrc \ > >--exclude /home/kline/.zhistory.* \ > > An ambiguous argument to the --exclude > option /home/kline/.zhistory.* will need to be quoted . . . > --exclude '/home/kline/.zhistory.*' \ > > otherwise it is expanded by the shell before reaching rsync > and then only the first referenced file is preceded by --exclude. > So instead of being excluded the transfer of the rest of the > expansion is emphasised. I don't know how rsync will then deal > with subsequent options. Hm, I didn't think I saw .ctwm.. Or .mozilla. Thanks much; I'll play around and see. gary > > >--exclude /home/kline/.ctwrm \ > >--exclude /home/kline/.mozilla \ > >--exclude /home/kline/.evolution \ > >--exclude /home/kline/4zip/ \ > >--exclude '/home/kline/www/' \ > >/home/kline [EMAIL PROTECTED]:/home/kline > >if [ $? = 0 ] > >then > > echo "rsync transfer went okay: tao to ethos" > >else > > echo "rsync failed to ethos from /home/kline"|mail > > [EMAIL PROTECTED] fi > > Malcolm > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "[EMAIL PROTECTED]" -- Gary Kline Seattle BSD Users' Group (seabug) | [EMAIL PROTECTED] Thought Unlimited Org's Alternate Email Site http://www.magnesium.net/~kline To live is not a necessity; but to live honorably...is a necessity. -Kant ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync does not --exclude: (??)
On Sat, 28 Oct 2006 03:15 am, Gary Kline wrote: > People, > > Weeks ago I tried this *without* the --exclude flags and > overwrote DOT file that I didn't want to. Iust rechecked > this use of rsync, saving important files from my work server > elseswhere, and see that some files or directories are being > listed and copied over. IDoes anything obvious jump out here? > > gary > > > > # > ## test2ethos.sh > # > >rsync -Cptuvaz -n -e "ssh -i > /home/kline/.ssh/tao_XXXYYYZZZ-id" \ --exclude > /home/kline/.zlogin \ >--exclude /home/kline/.zshrc \ >--exclude /home/kline/.zhistory.* \ An ambiguous argument to the --exclude option /home/kline/.zhistory.* will need to be quoted . . . --exclude '/home/kline/.zhistory.*' \ otherwise it is expanded by the shell before reaching rsync and then only the first referenced file is preceded by --exclude. So instead of being excluded the transfer of the rest of the expansion is emphasised. I don't know how rsync will then deal with subsequent options. >--exclude /home/kline/.ctwrm \ >--exclude /home/kline/.mozilla \ >--exclude /home/kline/.evolution \ >--exclude /home/kline/4zip/ \ >--exclude '/home/kline/www/' \ >/home/kline [EMAIL PROTECTED]:/home/kline >if [ $? = 0 ] >then > echo "rsync transfer went okay: tao to ethos" >else > echo "rsync failed to ethos from /home/kline"|mail > [EMAIL PROTECTED] fi Malcolm ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync on Freebsd 5.3
Lisa Casey skrev: Hi, I am trying to transfer all files in /var/mail from one Freebsd 5.3 machine (oldfreebsd.com) to another (newfreebsd.com). I decided to use rsync for this since it would preserve ownership, permissions, etc. I made sure rsync was installed on both machines. Neither machine had an /etc/rsyncd.conf or /etc/rsyncd.motd file. On newfreebsd.com I created /etc/rsyncd.conf thusly: motd file = /etc/rsyncd.motd max connections = 1 hosts allow = 208.44.xx.xx (The IP address is that of oldfreebsd.com) On newfreebsd.com, I created rsyncd.motd with the words Authorized Users Only in it. as a test, I'm just trying to rsync /var/mail/lisa from oldfreebsd.com to newfreebsd.com. When that works I'll do all of /var/mail/ on oldfreebsd.com I cd /var/mail, then type: rsync lisa newfreebsd.com:/var/mail/ and get this: connect to host newfreebsd.com port 22: Connection refused rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(348) Anyone know what might be wrong? Or can anyone suggest a better way of moving these mail boxes? Hi! About rsync: 1. Are you trying to do rsync over ssh as root? Freebsd default is to refuse root to login over ssh. 2. Your command syntax is for rsync over ssh. No need for server. You only need rsync installed in path on both sides. You may want to consider some options like -navP, see man rsync. 3. The configuration files are for setting up rsync as server. That uses port 873. And may provide anonymous login. To connect to see what modules are shared from a server: rsync example.com:: -- Regard from Lars Thanks, Lisa Casey ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync on Freebsd 5.3
Lisa Casey wrote: Hi, I am trying to transfer all files in /var/mail from one Freebsd 5.3 machine (oldfreebsd.com) to another (newfreebsd.com). I decided to use rsync for this since it would preserve ownership, permissions, etc. I made sure rsync was installed on both machines. Neither machine had an /etc/rsyncd.conf or /etc/rsyncd.motd file. On newfreebsd.com I created /etc/rsyncd.conf thusly: motd file = /etc/rsyncd.motd max connections = 1 hosts allow = 208.44.xx.xx (The IP address is that of oldfreebsd.com) On newfreebsd.com, I created rsyncd.motd with the words Authorized Users Only in it. as a test, I'm just trying to rsync /var/mail/lisa from oldfreebsd.com to newfreebsd.com. When that works I'll do all of /var/mail/ on oldfreebsd.com I cd /var/mail, then type: rsync lisa newfreebsd.com:/var/mail/ and get this: connect to host newfreebsd.com port 22: Connection refused rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(348) Anyone know what might be wrong? Or can anyone suggest a better way of moving these mail boxes? man rsync "Modern" rsync wants to use SSH transport, not rsyncd. That's why it's trying for a connect to port 22 of the remote host, and the connection refusal is due to either an intermediate firewall block, TCP wrappers, etc. or else sshd is simply not running. In this case, newfreebsd.com must be running sshd on accessible on port 22. Otherwise, you'll need to actually get rsyncd running and alter your invocation to use it (on port 873). Simplest, most secure fix is to make sure that newfreebsd.com is running sshd, and that should do it. Put sshd_enable="YES" in /etc/rc.conf, tune /etc/ssh/sshd_config as needed, then start sshd by reboot or manually invoking its startup script. -- Greg Barniskis, Computer Systems Integrator South Central Library System (SCLS) Library Interchange Network (LINK) , (608) 266-6348 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync on Freebsd 5.3
I've never needed to do any rsync specific configuration, just install
from ports on both machines. This sounds more like a ssh
configuration issue ("connect to host newfreebsd.com port 22:
Connection refused") on newfreebsd.com. Or maybe you're not allowing
ssh connections thru the firewall on newfreebsd.com?
Kevin
On 8/18/06, Lisa Casey <[EMAIL PROTECTED]> wrote:
Hi,
I am trying to transfer all files in /var/mail from one Freebsd 5.3 machine
(oldfreebsd.com) to another (newfreebsd.com). I decided to use rsync for
this since it would preserve ownership, permissions, etc. I made sure rsync
was installed on both machines. Neither machine had an /etc/rsyncd.conf or
/etc/rsyncd.motd file. On newfreebsd.com I created /etc/rsyncd.conf thusly:
motd file = /etc/rsyncd.motd
max connections = 1
hosts allow = 208.44.xx.xx
(The IP address is that of oldfreebsd.com)
On newfreebsd.com, I created rsyncd.motd with the words Authorized Users
Only in it.
as a test, I'm just trying to rsync /var/mail/lisa from oldfreebsd.com to
newfreebsd.com. When that works I'll do all of /var/mail/
on oldfreebsd.com I cd /var/mail, then type: rsync lisa
newfreebsd.com:/var/mail/
and get this:
connect to host newfreebsd.com port 22: Connection refused
rsync: connection unexpectedly closed (0 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(348)
Anyone know what might be wrong? Or can anyone suggest a better way of
moving these mail boxes?
Thanks,
Lisa Casey
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync on Freebsd 5.3
On Aug 18, 2006, at 10:40 AM, Lisa Casey wrote: Anyone know what might be wrong? Or can anyone suggest a better way of moving these mail boxes? I don't know what might be wrong with rsync but can't you just tar up the mail, move the tar file, and untar at the new place? Simple fast etc. That is how I do these sorts of things Chad --- Chad Leigh -- Shire.Net LLC Your Web App and Email hosting provider chad at shire.net ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync errors
On May 28, 2006, at 11:41 PM, Philip Hallstrom wrote: I have rsync installed on 2 6.1-RELEASE boxes on my home lan. I want to sync the webserver (curly) to the backup server (moe). When I run rsync on moe, I get the following: $ rsync -avz curly:/usr/local/etc/apache22/httpd.conf /usr/local/ etc/apache22 Password: receiving file list ... done rsync: mkstemp "/usr/local/etc/apache22/.httpd.conf.p9eLuI" failed: Permission denied (13) inflate returned -3 (0 bytes) rsync error: error in rsync protocol data stream (code 12) at token.c(421) [receiver=2.6.8] rsync: connection unexpectedly closed (46 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c (472) [generator=2.6.8] Where can I look to correct this? Do you have write access to moe:/usr/local/etc/apache22? What happens if you try and create a file in that directory... does it work? It was a permissions problem, fixed now, thanks very much. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync errors
On Sun, 28 May 2006 23:34:33 -0500 Charles Howse <[EMAIL PROTECTED]> wrote: > $ rsync -avz curly:/usr/local/etc/apache22/httpd.conf /usr/local/etc/ > apache22 > Password: > receiving file list ... done > rsync: mkstemp "/usr/local/etc/apache22/.httpd.conf.p9eLuI" failed: > Permission denied (13) the user you are executing as on your local machine doesnt have rights to create files (temp file) in /usr/local/etc/apache22. You can set the temp path to use as an option to rsync man rsync Beto ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync errors
I have rsync installed on 2 6.1-RELEASE boxes on my home lan. I want to sync the webserver (curly) to the backup server (moe). When I run rsync on moe, I get the following: $ rsync -avz curly:/usr/local/etc/apache22/httpd.conf /usr/local/etc/apache22 Password: receiving file list ... done rsync: mkstemp "/usr/local/etc/apache22/.httpd.conf.p9eLuI" failed: Permission denied (13) inflate returned -3 (0 bytes) rsync error: error in rsync protocol data stream (code 12) at token.c(421) [receiver=2.6.8] rsync: connection unexpectedly closed (46 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(472) [generator=2.6.8] Where can I look to correct this? Do you have write access to moe:/usr/local/etc/apache22? What happens if you try and create a file in that directory... does it work? -philip ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync script not excluding dirs
Parv wrote:
in message <[EMAIL PROTECTED]>,
wrote Pat Maddox thusly...
I have a backup script that runs nightly, and I want it to exclude
certain dirs (ports, obj, etc). However when I run the script it
doesn't exclude anything, leaving me with pretty massive backups.
...
/, /var, /usr, and /backup are all on different partitions. The key
part is at the bottom where it calls rsync and excludes dirs. Can
someone tell me what's wrong with the script?
...
PRE="/usr/local/bin/rsync"
${PRE} -bapoguLxSRC --exclude=*.core --exclude=*~* / --exclude=/dev
--exclude=/backup /backup/${DAY1}/
${PRE} -bapoguLxSRC --exclude=*.core --exclude=*~* /var /backup/${DAY1}/
${PRE} -bapoguLxSRC --exclude=*.core --exclude=*~* --exclude=/usr/src
--exclude=/usr/ports --exclude=/usr/obj /usr /backup/${DAY1}/
Your script seems to have wrapped by your mail client.
Anyway, in rsync(1) man page, see "INCLUDE/EXCLUDE PATTERN RULES"
section, point 2 ...
o if the pattern ends with a / then it will only match a direc-
tory, not a file, link, or device.
In other words, none of your exclude patterns for directories end in
'/' , thus the backup, src, ports, etc. directories are not
excluded.
I'm not sure that's true. It says a pattern ending in slash only
matches a directory, it doesn't say that a pattern not ending in slash
won't match a directory.
However, the patterns are anchored wrongly. Absolute patterns are still
relative to to tree being transferred. So --exclude=/usr/obj when
transferring /usr would try to match /usr/usr/obj, which is wrong. Take
the name of the filesystem being rsynced off those patterns and you
should find them excluded as you want.
The rsync man page does try to explain this: it is a bit long and can
take a few reads, but look at the FILTER RULES and ANCHORED PATTERNS...
sections.
--Alex
PS Your flags are way over the top. -a already includes -rlptgoD so you
don't need them again. Do you really want -R and I think that knocks
the top-level directory off the files which are unpacked? And if this
is a backup, then why -u? That's for use when you are changing files on
the destination and don't want those changes overwritten, which doesn't
sound like what you are doing. And if it is a backup, then you might
also want -H to preserve hard links.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync script not excluding dirs
in message <[EMAIL PROTECTED]>,
wrote Pat Maddox thusly...
>
> I have a backup script that runs nightly, and I want it to exclude
> certain dirs (ports, obj, etc). However when I run the script it
> doesn't exclude anything, leaving me with pretty massive backups.
...
> /, /var, /usr, and /backup are all on different partitions. The key
> part is at the bottom where it calls rsync and excludes dirs. Can
> someone tell me what's wrong with the script?
...
> PRE="/usr/local/bin/rsync"
> ${PRE} -bapoguLxSRC --exclude=*.core --exclude=*~* / --exclude=/dev
> --exclude=/backup /backup/${DAY1}/
> ${PRE} -bapoguLxSRC --exclude=*.core --exclude=*~* /var /backup/${DAY1}/
> ${PRE} -bapoguLxSRC --exclude=*.core --exclude=*~* --exclude=/usr/src
> --exclude=/usr/ports --exclude=/usr/obj /usr /backup/${DAY1}/
Your script seems to have wrapped by your mail client.
Anyway, in rsync(1) man page, see "INCLUDE/EXCLUDE PATTERN RULES"
section, point 2 ...
o if the pattern ends with a / then it will only match a direc-
tory, not a file, link, or device.
In other words, none of your exclude patterns for directories end in
'/' , thus the backup, src, ports, etc. directories are not
excluded.
- Parv
--
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: two quick conceptual questions RE: rsync (and rsyncing snapshots)
user wrote: Chuck - thank you... Sure. On Tue, 1 Nov 2005, Chuck Swiger wrote: rsync complains when the filesystem changes underneath it, but it will continue to run. On the other hand, rsync is not going to safely maintain the referential integrity of a complex file like a live database, but it's okay for most other things including mbox's. Does it simply complain, or does that somehow alter (lessen) the integrity of the sync that is going on ? The files which it notices are either copied or noticed as being missing. Any new files which get created after rsync does it's first scan are not going to be picked up later on by rsync. rsync'ing a snapshot is a fine idea. Ok - because _nothing_ would change, and thus rsync would not complain, etc. My gut is that while rsync performance might be increased, filesystem performance would be about the same, since all destructions and changes on the underlying filesystem are still being committed to the snapshot ... can you think of a reason why this would not only increase the rsync performance, but the overall FS performance while rsyncing ? No. The point of using snapshots is to address the integrity concern above, they don't do anything in particular to change the performance. If disk I/O is a significant concern to you, add more spindles, use RAID-1 or RAID-10 configurations, or some combination of the two. Finally, am I correct that there are _only two_ rsync comparison methods - the default checksum method, and the --size-only method ? Am I correct that rsync _always_ looks at the timestamp first, and then applies either checksum or size comparison ONLY IF the timestamps are different ? No, rsync checks both timestamp and size or checksum. So you are saying even if the timestamps are identical, rsync will _still_ do either a size or checksum comparison ? That seems ... inefficient ? Is there a way to tell it "if the timestamps are identical, just move on" ? What happens if a program appends some more data during the same second? rsync has to fstat() the file anyway which potentially involves a disk operation, once it's done so, comparing both timestamp and size doesn't take a significant amount longer to do. -c, --checksum skip based on checksum, not mod-time & size -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: two quick conceptual questions RE: rsync (and rsyncing snapshots)
Chuck - thank you... On Tue, 1 Nov 2005, Chuck Swiger wrote: > rsync complains when the filesystem changes underneath it, but it will > continue > to run. On the other hand, rsync is not going to safely maintain the > referential integrity of a complex file like a live database, but it's okay > for > most other things including mbox's. Does it simply complain, or does that somehow alter (lessen) the integrity of the sync that is going on ? > > Related: it occurs to me that perhaps it would be better to snapshot the > > filesystem, mount the snapshot, and then rsync the snapshot. On the other > > hand, the filesystem is continuously altering the snapshot as files are > > destroyed or changed ... so perhaps this does not gain anything. Is > > rsyncing a snapshot of a busy filesystem always, ever or never easier than > > rsyncing the busy filesystem itself ? > > rsync'ing a snapshot is a fine idea. Ok - because _nothing_ would change, and thus rsync would not complain, etc. My gut is that while rsync performance might be increased, filesystem performance would be about the same, since all destructions and changes on the underlying filesystem are still being committed to the snapshot ... can you think of a reason why this would not only increase the rsync performance, but the overall FS performance while rsyncing ? > > Finally, am I correct that there are _only two_ rsync comparison methods - > > the default checksum method, and the --size-only method ? Am I correct > > that rsync _always_ looks at the timestamp first, and then applies either > > checksum or size comparison ONLY IF the timestamps are different ? > > No, rsync checks both timestamp and size or checksum. So you are saying even if the timestamps are identical, rsync will _still_ do either a size or checksum comparison ? That seems ... inefficient ? Is there a way to tell it "if the timestamps are identical, just move on" ? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: two quick conceptual questions RE: rsync (and rsyncing snapshots)
user wrote: First, how does rsync respond to, and perform, when the source filesystem is under very heavy change ? If I have a filesystem that I want to rsync up to a backup server, but that filesystem is _very busy_ with the creation, destruction and changing of files, how well does rsync perform, and how much does it interfere with the performance of the underlying filesystem that it is sending up to the backup server ? rsync complains when the filesystem changes underneath it, but it will continue to run. On the other hand, rsync is not going to safely maintain the referential integrity of a complex file like a live database, but it's okay for most other things including mbox's. Rsync imposes a significant workload if you are syncronizing a large tree of stuff which changes a lot, but it's efficient considering the size of the task. Related: it occurs to me that perhaps it would be better to snapshot the filesystem, mount the snapshot, and then rsync the snapshot. On the other hand, the filesystem is continuously altering the snapshot as files are destroyed or changed ... so perhaps this does not gain anything. Is rsyncing a snapshot of a busy filesystem always, ever or never easier than rsyncing the busy filesystem itself ? rsync'ing a snapshot is a fine idea. Finally, am I correct that there are _only two_ rsync comparison methods - the default checksum method, and the --size-only method ? Am I correct that rsync _always_ looks at the timestamp first, and then applies either checksum or size comparison ONLY IF the timestamps are different ? No, rsync checks both timestamp and size or checksum. -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
two quick conceptual questions RE: rsync (and rsyncing snapshots)
Hello, I have started using rsync somewhat extensively and had two questions regarding its operation. First, how does rsync respond to, and perform, when the source filesystem is under very heavy change ? If I have a filesystem that I want to rsync up to a backup server, but that filesystem is _very busy_ with the creation, destruction and changing of files, how well does rsync perform, and how much does it interfere with the performance of the underlying filesystem that it is sending up to the backup server ? Related: it occurs to me that perhaps it would be better to snapshot the filesystem, mount the snapshot, and then rsync the snapshot. On the other hand, the filesystem is continuously altering the snapshot as files are destroyed or changed ... so perhaps this does not gain anything. Is rsyncing a snapshot of a busy filesystem always, ever or never easier than rsyncing the busy filesystem itself ? Finally, am I correct that there are _only two_ rsync comparison methods - the default checksum method, and the --size-only method ? Am I correct that rsync _always_ looks at the timestamp first, and then applies either checksum or size comparison ONLY IF the timestamps are different ? Thanks a lot. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync unable to sync suid files
Rajarajan Rajamani wrote: I am running Release 5.4 with 2 disks and am using rsync to sync between the two. On installing the second disk I used dump/restore to mirror them and am since using rsync for incremental changes. However I have a problem that rsync is unable to copy some files and I suspect it is has something to do with the suid files. Am I missing some switch to rsync ? This is the output # rsync --archive --times --verbose --delete --links --hard-links /usr/ /backup/usr [...] rsync: rename "/backup/usr/bin/.login.afaGPu" -> "bin/login": Operation not permitted (1) It's a problem with the schg flag (and sunlnk might be similar). See man chflags. It just so happens that suid files have been made schg to stop them being tampered with, but otherwise suid is just a coincidence. % ls -lsaFko /usr/bin/login 18 -r-sr-xr-x 1 root wheel schg 17192 Aug 8 23:06 /usr/bin/login* The only solutions I could think of were 1) fix rsync to be flag aware (hard) 2) implement something based on mtree which parsed your source tree, chflags -R on your dest tree, did the rsync, then ran mtree on the dest tree to fix the flags back. I haven't done either yet :-( so if anyone has a better solution I'd love to know. 2) won't work if you run at higher securelevel since you can't un-schg files (because it's not secure :-)), IIRC. --Alex ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rsync and moving files [Re: backup w/ snapshots]
* Garance A Drosihn [2005-08-30 12:50 -0400] > Fwiw, I understand the problem you're trying to describe. And the > basic issue is that rsync keeps no information between separate > runs of it. It has no way of knowing that a given file on the > source volume used to be at a different location. It does not even > know that the destination volume was sync'ed by a previous run of > rsync, so it does not even know that the file at the old location > on the destination is the same as the file at the old location on > the source. It knows nothing more than the information it has at > the moment of any given run of rsync. > > You could kinda fudge that information for rsync by creating a lot > of hard links, but that is probably going to create more of a mess > than it will solve. > > So, you're left with doing something else outside of rsync. The > script you are suggesting would probably be fairly easy to write > in something like ruby, perl, or python. Use a key made up of the > inode number + lastchange date, or maybe inode number + file size. > Then save away the key-to-filename(s) mapping for every file. On > the next run of rsync, see which files have moved on the source > directory. If the destination volume has a file at the old location > which matches the file-size or lastchange date (depending on which > key you used...), then move it to the new location on the destination > volume. Thanks! I think I will try to implement this, then! ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
