Re: time to come clean... .

[pauls]

--On September 3, 2006 10:02:45 PM -0700 Gary Kline 
<[EMAIL PROTECTED]> wrote:


yeah, the default IS ssh, rsh-no-mo.  what i want is to cron stuff
exactly like Mathews's ideas.   then at least, i'll have VERY recent
synchronization   iow:help me get this right; please.

i have spent hours reading about rsync.  my printserver (tao) is busted;
the man pages are horrible, &c.


FWIW, this webapge explains the process quite well:


Paul Schmehl ([EMAIL PROTECTED])
Adjunct Information Security Officer
The University of Texas at Dallas
http://www.utdallas.edu/ir/security/

Re: time to come clean... .

[Erik Norgaard]

Gary Kline wrote:


I've just installed/reinstaled rsync here on ns1.thought.org (aka
"sage") and on zen.thought.org.  I've fiddled with the rsyncd.conf on
both FBSD systems.  What I don't understand is how rsync, using
ssh, gets past the secret password.  If, say, I want to
copy all of my www files from sage to zen, what do I put
into /usr/local/etc/rsyncd.secrets? Let's say that rsyncd.secrets
had:

# User : pw
root : abcd
kline: wxyz


I'd use ssh keys, check the man page on how to specify keys for use with 
rsync/ssh.



rsync --verbose  --progress --stats --compress --rsh=/usr/local/bin/ssh
  --recursive --times --perms --links --delete \
  --exclude "*bak" --exclude "*~" \
  /usr/local/www/* zen.thought.org:/usr/local/www


Careful with wildcards, they may be interpreted different than you expect.

I made this script, the script assumes that paths are the same on source 
and destination:


#!/bin/sh
# RSYNC_USER is set as an environment variable or defaults to $USER
RSYNC_USER=${RSYNC_USER:-$USER}

# Exit if RSYNC_HOST not defined, there is no good default value.
if [ -z $RSYNC_HOST ]; then
  echo "RSYNC_HOST undefined, no host to syncronize with.";
  exit;
fi

# RSYNC_PATH sets the path to be syncronized, defaults to $HOME
# would be neat to check if path is absolute or else assume relative
# to $HOME or set RSYNC_PATH as environment/command line variable
if [ -z $1 ]; then
  RSYNC_PATH=$HOME;
else
  RSYNC_PATH=$HOME/$1
fi
# Syncronize folders
echo "Syncing $RSYNC_PATH..."

# Exclude patterns may be stored in .rsync in the home directory or
# the sub directory being syncronized
if [ -f $RSYNC_PATH/.rsync ]; then
  rsync -Cptuvaz --rsh="ssh" --exclude-from=$RSYNC_PATH/.rsync \
$RSYNC_PATH/ [EMAIL PROTECTED]:$RSYNC_PATH;
else
  rsync -Cptuvaz --rsh="ssh" \
$RSYNC_PATH/ [EMAIL PROTECTED]:$RSYNC_PATH;
fi

exit;

You put your exclude list in a file, .rsync (see the man-page), what to 
exclude may depend on the directory you're rsyncing. If you're 
automating this as a cron-job, then you may not have the environment 
variables set.


I think that rsync defaults to ssh so the --rsh is really obsolete, but 
I like to make it explicit.


Cheers, Erik
--
Ph: +34.666334818  web: http://www.locolomo.org
X.509 Certificate: http://www.locolomo.org/crt/8D03551FFCE04F0C.crt
Key ID: 69:79:B8:2C:E3:8F:E7:BE:5D:C3:C3:B1:74:62:B8:3F:9F:1F:69:B9
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: time to come clean... .

[Howard Jones]

On 4 Sep 2006, at 05:35, Gary Kline wrote:


How can I automate the backup via rsync to other servers?


If what you want is a backup to the disk on a remote system (rather  
than a tape drive or whatever), then have a look at rdiff-backup:


http://www.howtoforge.com/linux_rdiff_backup

It uses the rsync protocol for sending only changed blocks, but saves  
the sent data to one side as well to make a 'transaction log'. You  
end up with a full copy of the current data (as you would with rsync)  
and a command to construct any previous version by applying the  
changes in reverse as necessary. I run this on a couple of dozen  
machines, mostly freebsd, with no problems. For things like web  
servers and DNS, the increments are pretty much that day's logs, so  
it is quite effective at saving space/bandwidth. The actual software  
is in ports (sysutils/rdiff-backup, I think). It also has the benefit  
that no special actions are needed to restore a single file - you  
just grab what you want.


Even if you don't fancy the software, the article also has the info  
you need to get a reasonably safe root SSH (limited by IP, and the  
command that can be run, AND the public key) going with public key  
authentication, which will help for rsync as well.


Howie
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: time to come clean... .

[Gary Kline]

On Sun, Sep 03, 2006 at 09:44:53PM -0700, David King wrote:
> >It's time to come clean and admit that parts/most of rsync are
> >lost on me.  [...]
> >How can I automate the backup via rsync to other servers?
> 
> Depending on the backup strategy that you want, I highly recommend  
> rsnapshot (/usr/ports/sysutils/rsnapshot <http://www.rsnapshot.org/ 
> >). It handles most of the management of retaining past backups up  
> to X days, X weeks, etc, and uses hardlinks to save space between the  
> backups. So because it uses rsync, it uses the bandwidth of an  
> incremental backup, but because it uses directory trees of hardlinks,  
> each backup is completely restorable like a full backup.
> 

One problem may be semantics.  I'm not certain if I want directory
/etc/* synchronized on servers A and B, or if I just want a 
100%-guaranteed backup ... .  Since I do 99% of stuff on
tao, I want every other (possible) server to sync up my ~/* files
on other machines. The build and config files I just want tar'd up
and moved to, say, /usr/tmp/tao, /usr/tmp/sage, usr/tmp/zen, and
so on.  This stuff is what I would like done at least daily.

I'll look at rsnapshot.  A very big (*)++plus is that Dru
wrote it.  That mean it's thoroughly first rate.

Around 10 hrs sleep in three days just don' cut it.

-- 
   Gary Kline [EMAIL PROTECTED]   www.thought.org Public service Unix

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: time to come clean... .

[Gary Kline]

On Sun, Sep 03, 2006 at 10:39:40PM -0600, Chad Leigh -- Shire.Net LLC wrote:
> 
> On Sep 3, 2006, at 10:35 PM, Gary Kline wrote:
> 
> >
> > Folks,
> > How can I automate the backup via rsync to other servers?
> 
> I believe the default for rsync is to use ssh as a transport.  That  
> is how I am using it with ssh certificates.  Read up on rsync and if  
> you want to create some cron jobs that would be an easy automation.
> 

yeah, the default IS ssh, rsh-no-mo.  what i want is to cron stuff 
exactly like Mathews's ideas.   then at least, i'll have VERY recent
synchronization   iow:help me get this right; please.

i have spent hours reading about rsync.  my printserver (tao) is busted;
the man pages are horrible, &c.

gary

> Chad
> 
> ---
> Chad Leigh -- Shire.Net LLC
> Your Web App and Email hosting provider
> chad at shire.net
> 
> 
> 
> ___
> freebsd-questions@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "[EMAIL PROTECTED]"

-- 
   Gary Kline [EMAIL PROTECTED]   www.thought.org Public service Unix

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: time to come clean... .

[David King]

It's time to come clean and admit that parts/most of rsync are
lost on my.  [...]
How can I automate the backup via rsync to other servers?


Depending on the backup strategy that you want, I highly recommend  
rsnapshot (/usr/ports/sysutils/rsnapshot <http://www.rsnapshot.org/ 
>). It handles most of the management of retaining past backups up  
to X days, X weeks, etc, and uses hardlinks to save space between the  
backups. So because it uses rsync, it uses the bandwidth of an  
incremental backup, but because it uses directory trees of hardlinks,  
each backup is completely restorable like a full backup.



I have
a couple boxen out there miles remote; the rest are meters apart.
If 'expect the unexpected' is any guide, now I *am* expecting.
If it means a few days and 8 hours of typing, then that's just
the price.  (That I had a recent TaoEtc.tbz is a good sign; that
it was in /usr/tmp _on_ tao is not a good sign... .)

I've just installed/reinstaled rsync here on ns1.thought.org (aka
"sage") and on zen.thought.org.  I've fiddled with the rsyncd.conf on
both FBSD systems.  What I don't understand is how rsync, using
ssh, gets past the secret password.  If, say, I want to
copy all of my www files from sage to zen, what do I put
into /usr/local/etc/rsyncd.secrets? Let's say that rsyncd.secrets
had:

# User : pw
root : abcd
kline: wxyz

Would this old snippet work as a starting place, exec'd by root:



rsync --verbose  --progress --stats --compress --rsh=/usr/local/bin/ 
ssh

  --recursive --times --perms --links --delete \
  --exclude "*bak" --exclude "*~" \
  /usr/local/www/* zen.thought.org:/usr/local/www

  ??

  Thanks for any clues.  I have a lot of stuff tarballed, but it's
  time to do things right!!

  gary







--
   Gary Kline [EMAIL PROTECTED]   www.thought.org Public  
service Unix


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions- 
[EMAIL PROTECTED]"


--
David King
Computer Programmer
Ketralnis Systems


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: time to come clean... .

[Chad Leigh -- Shire.Net LLC]

On Sep 3, 2006, at 10:35 PM, Gary Kline wrote:



Folks,
How can I automate the backup via rsync to other servers?


I believe the default for rsync is to use ssh as a transport.  That  
is how I am using it with ssh certificates.  Read up on rsync and if  
you want to create some cron jobs that would be an easy automation.


Chad

---
Chad Leigh -- Shire.Net LLC
Your Web App and Email hosting provider
chad at shire.net



___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

time to come clean... .

[Gary Kline]

Folks,

It's time to come clean and admit that parts/most of rsync are 
lost on my.  The good-old-days when everybody trusted everybody
on the net--when it was mostly just we geeks are way gone.
Crackers, prob'ly 12-year-olds using DOS scripts, tried to
crack my ns1 server.  After that I upgraded to bind-9 and read
[and enjoyed] DNS/BIND, *and* began using ssh.  So a lot of my
automatic backup programs stopped working.  As thought.org grew
from one machine to four, it became that much harder to tar and
backup stuff.  (I have automated some backups, but crudely.   
Now, after Friday's "trap 18",  my first fatal ever, it's time 
to follow the backup system that Matthew suggested.)

How can I automate the backup via rsync to other servers?  I have 
a couple boxen out there miles remote; the rest are meters apart.  
If 'expect the unexpected' is any guide, now I *am* expecting.   
If it means a few days and 8 hours of typing, then that's just 
the price.  (That I had a recent TaoEtc.tbz is a good sign; that 
it was in /usr/tmp _on_ tao is not a good sign... .)

I've just installed/reinstaled rsync here on ns1.thought.org (aka
"sage") and on zen.thought.org.  I've fiddled with the rsyncd.conf on
both FBSD systems.  What I don't understand is how rsync, using
ssh, gets past the secret password.  If, say, I want to
copy all of my www files from sage to zen, what do I put
into /usr/local/etc/rsyncd.secrets? Let's say that rsyncd.secrets
had:

# User : pw
root : abcd
kline: wxyz

Would this old snippet work as a starting place, exec'd by root:



rsync --verbose  --progress --stats --compress --rsh=/usr/local/bin/ssh
  --recursive --times --perms --links --delete \
  --exclude "*bak" --exclude "*~" \
  /usr/local/www/* zen.thought.org:/usr/local/www

  ??

  Thanks for any clues.  I have a lot of stuff tarballed, but it's
  time to do things right!!

  gary







-- 
   Gary Kline [EMAIL PROTECTED]   www.thought.org Public service Unix

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"