Re: [SLUG] Tar over SSH
On Tue, Oct 23, 2001 at 07:33:13AM +1000, Grant Parnell wrote: Hmmm consider writing a little program to insert in the pipe that filters all output until it gets a marker. ssh root@martel 'echo MARKER; tar cvf - /home/mikel/testing' | filter.pl | dd /dev/rmt0 Nice idea but I am backing up a few Gigabytes of data from a few home directories. That would involve an if comparison for every line which would be a stupendous bottle neck even if written in assembler :-) I shall prob just remove any lines from root's .bashrc, .profile that echo as the script will always be run from root. Just thought that surely someone in slug does *streaming* backup to another machine using *tar over ssh* and must have examples of how the do it. # This is what will run when it all works :-) #ssh root@martel 'tar cvf - /home' | dd /dev/rmt0 Question: How do I stop the login stuff from prepending to the tape.? Is there a better way to pipe tar through ssh or use scp? so Mike -- panic: can't find / Michael Lake, University of Technology, Sydney Work: [EMAIL PROTECTED] Ph: 02 9514 1724 Fx: 02 9514 1628 Home: http://www.speleonics.com.au Linux enthusiast, active caver and interested in anything technical. *** -- SLUG - Sydney Linux User Group Mailing List - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Tar over SSH
This one time, at band camp, Mike Lake wrote: Just thought that surely someone in slug does *streaming* backup to another machine using *tar over ssh* and must have examples of how the do it. Yeah, and it's exactly like everyone's said, the caveat is that there is no output from roots .profile, etc. -- [EMAIL PROTECTED]http://spacepants.org/jaq.gpg Balial This port may thing it's fortified, butt I seem to be mounting a pretty good assault -- SLUG - Sydney Linux User Group Mailing List - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Tar over SSH
Jamie Wilkinson wrote: This one time, at band camp, Mike Lake wrote: The problem is that when ssh logs into root .bashrc .profile gets run and there is some output from the login shell. This is prepended to the tar file so its not a tar file anymore. I can see the lines of the login if I run /test | more. 'touch .hushlogin' will tell the login shell not to print the motd and so on, you can make sure that the rest of .profile doesn't output anything on login. But that only turns off the /etc/motd message. Not anything output by .bashrc. I have a few things echoed to the screen by .bashrc I can turn them off but there must be a way to ignore output of ssh before root runs the tar program. Mike -- Michael Lake Active caver, Linux enthusiast and interested in anything technical. Safety Convenor, Australian Speleological Federation Owner, Speleonics (Australia) -- SLUG - Sydney Linux User Group Mailing List - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Tar over SSH
Howard Lowndes wrote: Have you considered doing rsync across machines. it is considerably faster as it only transfers the changes. Ah that that would be sensible to run in between full tar backups. Wow I am right at this moment doing an apt-get install rsync and will browse that info but I would still like to work out how to stream a tar backup over ssh correctly. This is the script that runs: --- root:# cat test #!/bin/bash # This is what will run when it all works :-) #ssh root@martel 'tar cvf - /home' | dd /dev/rmt0 # This is here for testing, testing dir contains a few files ssh root@martel 'tar cvf - /home/mikel/testing' --- The problem is that when ssh logs into root .bashrc .profile gets run and there is some output from the login shell. This is prepended to the tar file so its not a tar file anymore. I can see the lines of the login if I run /test | more. Question: How do I stop the login stuff from prepending to the tape.? Is there a better way to pipe tar through ssh or use scp? so that it streams from the Alpha to the tape machine. Mike -- Michael Lake Active caver, Linux enthusiast and interested in anything technical. Safety Convenor, Australian Speleological Federation Owner, Speleonics (Australia) -- SLUG - Sydney Linux User Group Mailing List - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Tar over SSH
On Sun, Oct 21, 2001 at 04:50:40PM +1000, Michael Lake wrote: Howard Lowndes wrote: Have you considered doing rsync across machines. it is considerably faster as it only transfers the changes. Ah that that would be sensible to run in between full tar backups. Wow I am right at this moment doing an apt-get install rsync and will browse that info but I would still like to work out how to stream a tar backup over ssh correctly. I use rsync to backup my laptop, which works great. (I only found just _how_ great it was when my laptop HD decided to die ...). One tip if you're planning to use rsync to shovel across gigantor amounts of data is to use rsyncd, don't run rsync over ssh if you don't need the encryption. rsync backing up to rsyncd runs _much_ faster than it tunneling over ssh, and you won't run into problems caused by the ssh tunnel either. -- #ozone/algorithm [EMAIL PROTECTED] - trust.in.love.to.save -- SLUG - Sydney Linux User Group Mailing List - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Tar over SSH
This one time, at band camp, Mike Lake wrote: The problem is that when ssh logs into root .bashrc .profile gets run and there is some output from the login shell. This is prepended to the tar file so its not a tar file anymore. I can see the lines of the login if I run /test | more. 'touch .hushlogin' will tell the login shell not to print the motd and so on, you can make sure that the rest of .profile doesn't output anything on login. -- [EMAIL PROTECTED]http://spacepants.org/jaq.gpg Balial This port may thing it's fortified, butt I seem to be mounting a pretty good assault -- SLUG - Sydney Linux User Group Mailing List - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Tar over SSH
Have you considered doing rsync across machines. it is considerably faster
as it only transfers the changes.
On Sun, 21 Oct 2001, Mike Lake wrote:
Hi guys,
Persuant to a suggestion that I should not be trying to backup machines
using tar over NFS as it can lead to probs (such as root on the remote
machine not being able to read the machine to backup) I have been trying
to get tar over ssh going.
This was the problem I had.
Date: Sun, 7 Oct 2001 09:29:00 +1000
\begin{Mike Lake}
I am having a problem getting tar to backup one machine (martel) which
has it's home directories NFS exported to the machine with the tape
drive on it (tazieff). The prob is basically that root cannot read
some
files as it seems the uid/gid numbers don't match between the
machines.
Angus wrote
tar has support for reading over rsh (or ssh).
since NFS authentication is IP-based, rsh should be no less secure. ssh
would obviously be the best alternative.
it'll be much nicer on the network, avoid permission issues, etc if you
use that method instead of NFS.
so I am now using ssh. I did a search on google and it suggested a few
ways. Most were just ssh'ing an already created tar file across to the
backup machine. What I want is a stream of data from the machine to
backup to the machine with the tape drive on it. That way I won't run out
of diskspace sometime.
I have authorized keys on both machines so root can login using ssh with
no passwd and I have changed the /etc/ssh/sshd_config file to allow root
logins via PermitRootLogin yes (slug archives helped there !)
This is the script that runs:
---
root:# cat test
#!/bin/bash
# This is what will run when it all works :-)
#ssh root@martel 'tar cvf - /home' | dd /dev/rmt0
# This is here for testing, testing dir contains a few files
ssh root@martel 'tar cvf - /home/mikel/testing'
---
The problem is that when ssh logs into root .bashrc .profile gets run
and there is some output from the login shell. This is prepended to the
tar file so its not a tar file anymore. I can see the lines of the login
if I run /test | more.
This is what I want to do: Backup all home directories on the Alpha
by streaming the contents via ssh or scp to the backup machine with
the tape drive. The script would be started FROM the backup machine with
the tape drive on it.
Question:
How do I stop the login stuff from prepending to the tape.?
Is there a better way to pipe tar through ssh or use scp? so
that it streams from the Alpha to the tape machine.
Mike
--
Howard.
LANNet Computing Associates - Your Linux people
Contact detail at http://www.lannetlinux.com
--
SLUG - Sydney Linux User Group Mailing List - http://slug.org.au/
More Info: http://lists.slug.org.au/listinfo/slug
