Offtopic: experimental perl backup script
Hi, I am writing an experimental implementation in perl to do backup/replication. experimental implementation is availible at www.mboot.org/ftp/amrta its an experiment in its very early stages, currently running on W2K with activestate 5.8.0. I can do backups and restores at the moment. some docs are in the doc section , pictures are openoffice draw Maarten -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Exclude files bigger than X
Hello List! I want to rsync a few homedirectories which also contain big *.tar or *.bin files. Does rsync have a option where it will NOT transfer files which are bigger than X MB/GB. I did have a look in man rsync, but could not really find what i am lookig for. If rsync does not have that feature, is there a workaround? Cheers, Mario -- GMX Weihnachts-Special: Seychellen-Traumreise zu gewinnen! Rentier entlaufen. Finden Sie Rudolph! Als Belohnung winken tolle Preise. http://www.gmx.net/de/cgi/specialmail/ +++ GMX - die erste Adresse für Mail, Message, More! +++ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: how rsync works
On Sun, 26 Oct 2003, jw schultz [EMAIL PROTECTED] wrote: On Tue, Sep 16, 2003 at 03:49:45AM -0700, jw schultz wrote: Aside from numerous other weaknesses that have crept into the manpage i do note that there doesn't seem to be any point where it is mentioned that rsync replaces destination files rather than updating them in-place. I'm not sure where it would go in the current manpage. I'm no writer but i have been toying with the idea of a practical how the rsync utility works for the non-mathematician document. This wouldn't be a how-to but would instead describe in broad terms what the rsync internals are doing, the three processes involved and a general sketch of the protocol. Something that wouldn't be obsoleted by enhancements. OK. Having receive tepid encouragement i have produced a first-draft of the document. As i said I'm no writer, this document no doubt has structural defects and gaping holes as well as being worded strangely. I invite constructive comments and patches (on list) but if someone else wants to pick up the ball and run with it i would be pleased to hand it off. I have formatted it in HTML so if you care it can be found at http://www.pegasys.ws/how-rsync-works.html I think a document such as this would be very helpful. I looked at it soon after it was announced, but now it's not there anymore. Where'd it go? -- John Van Essen Univ of MN Alumnus [EMAIL PROTECTED] -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsync Samba
If anyone has time to look at this problem I would appreciate it. I think I am looking for a way to increase the timeout in samba (in the smb.conf file) for reporting a down link or can't read xyz file. However I am writing to you folks because this problem has come up while using rsync my problem == We have a wan in which links from a central linux machine to a nodes which are xp machines from 1 to 5 minutes a night. not very common, but it happens. we are using samba to mount directories on the remote linux machines to the local linux server. (we use samba because we also have xp machines in our network it is just easier to use this for now). We use rsync nightly to backup remote directories to the local linux server. When one of the outages happes during a backup (rsync operation) either - if I have not set the --timout value in rsync the operation just continues. Rysnc seems not to notice that samba is reporting errors and just copies over I assume a bunch of . Of course the file is corrupted. - if I have set the --timeout value in rsync, in the case of failure rsync just drops out. Of course if i set the --timeout value to something like 2 minutes rsync seems to ignore the samba errors and continues. What I would like to do is set a timeout value in samba for reporting errors from what seems to be about 30 seconds to something like 6 minutes. Then I could set the --timeout value in rsync to something like 5 minutes. I can't find in man smb.conf anywhere were you can set something like if you can't connect in 30 seconds then report an error. Perhaps I should be using an rysnc server or some other method. Any help would be appreciated. _ Say goodbye to busy signals and slow downloads with a high-speed Internet connection! Prices start at less than $1 a day average. https://broadband.msn.com (Prices may vary by service area.) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: how rsync works
On Mon, Nov 24, 2003 at 09:14:56AM -0600, John Van Essen wrote: On Sun, 26 Oct 2003, jw schultz [EMAIL PROTECTED]?wrote: ?On Tue, Sep 16, 2003 at 03:49:45AM -0700, jw schultz wrote: ? ?Aside from numerous other weaknesses that have crept into ?the manpage i do note that there doesn't seem to be any ?point where it is mentioned that rsync replaces destination ?files rather than updating them in-place. ?I'm not sure ?where it would go in the current manpage. ? ?I'm no writer but i have been toying with the idea of a ?practical how the rsync utility works for the ?non-mathematician document. ?This wouldn't be a how-to but ?would instead describe in broad terms what the rsync ?internals are doing, the three processes involved and a ?general sketch of the protocol. ?Something that wouldn't be ?obsoleted by enhancements. ? ?OK. ?Having receive tepid encouragement i have produced a ?first-draft of the document. ? ?As i said I'm no writer, this document no doubt has ?structural defects and gaping holes as well as being worded ?strangely. ?I invite constructive comments and patches (on ?list) but if someone else wants to pick up the ball and run ?with it i would be pleased to hand it off. ? ?I have formatted it in HTML so if you care it can ?be found at http://www.pegasys.ws/how-rsync-works.html I think a document such as this would be very helpful. I looked at it soon after it was announced, but now it's not there anymore. Where'd it go? I had thought so too but now i'm less sure. I put it up on October 26th -- for discussion. In the next 14 days there were hits from 107 unique IPs, 73 of which were in the first 4 days. Then it settled down to a slow, declining, trickle that could have been naught but robots. But no one said anything online or off. So on the 8th of November i took it down. Now over two weeks after i took it down and four since putting it up you are the first person to even acknowledge it. The intent was to, if there is interest, get feedback to polish the document for placement on the web site. Or to let someone who is perhaps a better writer take it over. Not to self-publish the thing. Perhaps i had to high of expectations but i expected at least one or two would say something, even if it was just yuck or its a start before knowledge of it subsided into archives. I'll put it back up if i hear more than just a whimper. -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re[2]: how rsync works
On Monday, November 24, 2003 at 7:04 PM, jw wrote: snipped The intent was to, if there is interest, get feedback to polish the document for placement on the web site. Or to let someone who is perhaps a better writer take it over. Not to self-publish the thing. Perhaps i had to high of expectations but i expected at least one or two would say something, even if it was just yuck or its a start before knowledge of it subsided into archives. I'll put it back up if i hear more than just a whimper. I just joined the list and I, for one, would be very interested in reading the document and giving you feedback. I think such a document would be very useful. I've spent the last couple of days reading all things rsync on the Internet, and I still have questions that would, most likely, be answered by such a document. So please consider this as more than just a whimper. -- Terry -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: minimalist rsync on windows
Hi, My ssh can't create the directory /home/user/.ssh. I realize this is because I am using the ssh outside of the Cygwin environment, however it seems I should be able through a init script with a variable or creating my own /etc/password file, that I could fool it. Is there a way to do this? I have created a key and the key works, but it always asks me about if the server is ok to authentica (yes/no):, if I say yes, then I am logged in with no password. Is there another way to avoid this question for automatic rsync-ing? Brian -- ContagiousDesign! web . design . software . photo Brian Rose . programmer Stacey Rose . designer (604)-588-8827 . contagiousdesignATshawDOTca -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
unable to open configuration file rsyncd.conf
I'm getting an error trying to pull a file with ssh/rsync: $ rsync -e 'ssh -i .ssh/id_rsa' [EMAIL PROTECTED]::www/index.html index.html rsync: unable to open configuration file rsyncd.conf: No such file or directory rsync error: syntax or usage error (code 1) at clientserver.c(502) rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) The /etc/rsyncd.conf file on server.com is indeed there: log file = /var/log/rsyncd [www] path = /home/www/public_html If I use root it works: $ rsync -e ssh [EMAIL PROTECTED]::www/index.html index.html [EMAIL PROTECTED]'s password: $ What's the problem? Thanks, Mike -- A program should be written to model the concepts of the task it performs rather than the physical world or a process because this maximizes the potential for it to be applied to tasks that are conceptually similar and, more important, to tasks that have not yet been conceived. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Exclude files bigger than X
Hi, On Mon, Nov 24, 2003 at 02:18:13PM +0100, Mario Ohnewald wrote: I want to rsync a few homedirectories which also contain big *.tar or *.bin files. Does rsync have a option where it will NOT transfer files which are bigger than X MB/GB. How about something like this: find [src] -size +1000k | rsync [opts...] --exclude-from=- [src] [dst] Regards, Michael -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: minimalist rsync on windows
Hi, I get a Local: Bad packet length 1349676916 when I try to use rsync-minimalist. I tried with the provided cygwin and with the latest cygwin. -- ContagiousDesign! web . design . software . photo Brian Rose . programmer Stacey Rose . designer (604)-588-8827 . contagiousdesignATshawDOTca -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: minimalist rsync on windows
Brian wrote: Hi, I get a Local: Bad packet length 1349676916 when I try to use rsync-minimalist. I tried with the provided cygwin and with the latest cygwin. Used options, debug info, and/or other infos? -- Lapo 'Raist' Luchini [EMAIL PROTECTED] (PGP X.509 keys available) http://www.lapo.it (ICQ UIN: 529796) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync-bugs and unclear semantics when copying multiple source-dirs to one target
On 24 Nov 2003, Dirk Pape [EMAIL PROTECTED] wrote:
Dear Martin Pool,
I tried to ask via the rsync-mailing list but never got an answer. So I
contact you directly.
I refer to the rsync syntax
rsync [OPTION]... SRC [SRC]... DEST
with more than one SRC, which is mentioned in the man-pages.
We use this form to overlay a target directory tree from more than one
sources (class, group1, group2, ..., machine) to yield a costomized
cloned directory.
There are some glitches and bugs when using this form of rsync commands,
one of which I have described in the here attached mail to the rsync
mailing list. This is a platform specific bug.
The heart of the problem is that you are trying to write the same file
from several different source directories. I think this just will not
work predictably in the current design of rsync, because it builds a
single list of all files at the start of the transfer. Furthermore
the order in which files are transferred is rather strange, for
reasons of historical compatibility.
I think we do not make any guarantees about what happens if the same
relative path occurs in several source directories the behaviour is
undefined.
I agree that it would be nice if it processed the source directories
in the order they are given, but that is not how it works.
At the moment your options are:
Fix rsync to support this behaviour.
Transfer the directories one at a time to build up the destination.
This has several problems, one being that there may be many
redundant transfers and another that the state will be inconsistent
for longer.
Make a single source directory that has the state you want.
Ditto, but use union bind mounts to synthesize it from several
directories, assuming that your OS supports that.
Use some other tool.
Do several rsync transfers using exclude/include options to pick the
right directories from each overlay.
The last is possibly the most promising. You could even write a
little Perl script to build the exactly correct include lists.
There is another glitch, which I will describe here:
if you have the following directory structure (- is softlink)
./dir1/dir/a
./dir2/dir - ../dir3/dir
./dir3/dir/b
and do
rsync -av --delete dir1/ dir2/ target
you get
./target/dir - ../dir3/dir
./dir3/dir/a
./dir3/dir/b
I would expect either
Variant 1:
./target/dir - ../dir3/dir
./dir3/dir/b
(contents of /dir1/dir is ignored because dir ist overlayed with a
symlink in dir2)
or
Variant 2:
./target/dir - ../dir3/dir
./dir3/dir/b
(./dir1/dir/a is copied following the overlayed symlink *but* the --delete
then also has to follow the symlink)
I would prefer strongly to see variant 1 or a new option to protect target
directories from changing contents by linking in o them.
For your motivation:
Our more complex scenario is like that: We have
class/usr/share/bugzilla/some_files
machine/usr/share/bugzilla - /local/usr/share/bugzilla
and we do something like
rsync -av --delete --exclude local class/ machine/ targethost:/
the --exclude local protects files in targethost:/local from being
deleted but not from being overwritten with files which are present in
class/usr/share/bugzilla/ on the scr-host.
I would like to see an option (or standard semantics) to simply killing a
directories sub-filelist when the directory is overlayed by a symlink in
a source directory given later in the command line. May be it would suffice
to do that only if the symlink points to a directory, which is outside
all source dirs or element of an exclude list.
I hope you understand and can help me.
Thanks,
Dirk Pape.
From: Dirk Pape [EMAIL PROTECTED]
Subject: bug (filelist) for platforms solaris and darwin (macosx) and *not*
linuxi386
Date: Sun, 28 Sep 2003 13:19:45 +0200
To: [EMAIL PROTECTED]
X-Mailer: Mulberry/3.1.0b7 (Mac OS X)
I have found a nasty bug when a file, which is in some of many sources,
shall be copied to a target.
The linux-Version works well but rsync 2.5.{2|5|6} under solaris9 (gcc
2.95.3) and darwin (gcc 3.1) do not. The decision which file (out of which
src) shall be copied depends on the number of src dirs given on the command
line.
This bug bytes us very hard, because we decided to rely on rsync to build
local directories by overlaying different directories from a server and
need to be sure to have a consistent semantics in what version of the file
appears in the local directory.
I stripped our sitation down to a (yet fairly complex) test archive, so you
can reproduce the situation.
Here is the script, which is also in the archive:
#!/bin/bash
rsyncpath=rsync
$rsyncpath -av --delete dir1/ dir2/ merged12
$rsyncpath -av --delete dir1/ dir2/ dir3/ merged123
# as dir3 only consists of an empty dir subdir we expect
# that merged12 and merged123 have identical files in them
# but merged*/subdir/s0/LOOKATTHIS differ as
Need help with exclude
Hi all, I am having *massive* problems trying to exclude a single directory from an rsync. I have serv1 and serv2. I am trying to rsync /foo/test from serv1 to /foo on serv2 I want to exclude the directory /foo/test/dir1 So I try: rsync -av --exclude-from=/foo/rsync.excludes /foo/test serv2:/foo rsync.excludes contains: /foo/test/dir1/ This is not working. I also try: rsync -av --exclude=/foo/test/dir1/ /foo/test serv2:/foo and the exclude still does not work. I have also tried other variations like: rsync -av --exclude=/dir1/ /foo/test serv2:/foo also to no avail. I have spent many hours reading the manpage and searching google. I see many examples on exluding but nothing on how to exclude one simple directory as I am trying to do. I am really glad you added the EXCLUDE PATTERNS section in the manpage, but it is still not very helpful. I think that something as basic as excluding one single directory should be straightforward and simple to execute and not require such extensive difficulty. I have used rsync for a few years now. My opinion is that it is a great tool but its potential usefulness is limited by some extreme quirkiness as to how paths get specified and its extensive switch options many of which should just be included by default. For example, maybe having a meta-switch such as --mirror which contains a set of popular switches used in most mirroring situations. In this regard, I think the --backup option is a bad idea and instead --backup should be a meta-switch that includes options useful for backups. Most people I think are usually trying to mirror or to backup. -- | Simulated Windows reboot | | | | CrossOver is simulating a Windows reboot. | | Please Wait...| - message window that appears while installing MS Office under CrossOver Office -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
