Re: I don't think Amanda is going to work for my environment...
Michael Kahle wrote:
EXCLUDE doc, "The GNU version of tar, (gnutar or gtar), reads its
data at a file system, (or higher), level and does include the option
to exclude specific files and/or directories. It should be mentioned
here that tar will change the access times no files. Tar has the
ability to preserve the access times however, doing so effectively
disables incremental backups since resetting the access time alters
the inode change time, which in turn causes the file to look like it
needs to be archived again." So, now this ads another problem to the
mix. I cannot do incremental backups on files that I decide to
backup with gnutar, which I am using to solve my first problem. This
is unacceptable.
The problem with access times is not as bad as it looks.
Some people like to preserve the access time of files, to have
a hint which files were used recently; or better, which were *not* used
recently (candidates to throw away or to archive).
While this is a valid issue for some of us, it is not a very reliable
indication. The access time of each file, garbage or not is easily
changed by a command like:
grep -R 'From: paul.bijnens' .
But, the good news is that you *can* preserve the atime, without
modifying the ctime, if you believe this is very important. See below.
If I am understanding this correctly, Amanda has no way to
incrementally backup a disk that also uses a exclude list to prevent
it from backing up certain files. This seams to me to be an
important feature included in a backup package.
You don't understand this correctly. The ability to do incremental
backups has nothing to do with exclude lists. Backups with
excludes work fine, full and incremental, using gnutar.
Some versions of dump do alow to specify a subdirectory instead
of a complete filesystem. But in that case they cannot make
incremental backups of this. This is only for dump, not for gnutar.
2. Not an intuitive way to think about backing things up. This rant
is somewhat related to the above rant. :) The exclude files need to
be written on the client? Again, from the EXCLUDE documentation, "An
...
The include/exclude directives can be on the server (in the
dumptype) or on the client, or a mix of both. A disklist
entry could look like:
my.server.com /my/dir {
comp-user-tar
exclude list append "/var/opt/amanda/exclude.gtar"
exclude list append "./.exclude.gtar"
exclude append "./scratch"
exclude append "*.mp3"
dumpcycle 7
}
It has a dumptype adapted on the fly by the braces syntax.
The first exclude points to a general exclude file on the
client. The second one is a file relative to the directory
"/my/dir", also on the client. The last two are just strings,
sent to the client. (and "comp-user-tar" could have some
exclude directives too). The client merges all the files,
and directives in one file (because of the "append" keyword),
which you can find in the /tmp/amanda/*exclude* .
environments, it just doesn't make the grade. Are there other backup
solutions out there that are free software that might better address
my needs?
Not a user, but hear good comments about bacula:
http://www.bacula.org/
Now, how to preserve the atime without changing the ctime, and, at
the same time avoiding a lot of other backup problems, like active
files etc?
Modern Linux and Solaris versions (and others probably too), have
volumemanagers or filesystems capable of doing snapshots.
Taking a snapshot takes usually less than a minute, during which
databases can be shut (or put in hot backup mode). After the
snapshot is taken, the filesystem can be used as before. You can
then mount that snapshot somewhere in the filesystem in readonly mode
and make a backup of that snapshot.
Advantages:
- the downtime is really minimal.
- the atime is not changed.
- no more complaints about "file changed while reading...",
and very consistent backup images.
- you can leave the snapshot mounted for the next day, letting
users do their own restores from the "oops" mistakes, like "rm * .bak"
Disadvantages:
- you have to reserve a little bit diskspace (but much less than
twice the total diskusage)
- slightly more complicated (my scripts are about 20 lines).
Amanda has a disklist syntax useful for it:
my.server.com /my/dir /snap/my/dir {
comp-user-tar
exclude append "core"
} 1
this means that the logical directory is referred to as /my/dir,
but the backup is made from the device (directory) /snap/my/dir.
People who like to have a look at the scripts for Linux and Solaris
may contact me.
--
Paul Bijnens, XplanationTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit, ZZ, :q, :q!, M
Re: I don't think Amanda is going to work for my environment...
On Thu, 2004-03-18 at 14:31, Joshua Baker-LePain wrote: > It is true that it takes some wedging to make amanda work in a 'doze heavy > environment -- that's simply not what it was designed for. As for advice > on commercial solutions, this isn't exactly the best place to ask. ;) If you're not on a shoestring budget, I'd say knock yourself out, there are some great (but usually expensive) packages out there that are very easy to set up and maintain, though this list is not the best place to get that sort of information. Veritas, Arkeia, and Legato are three packages I would definitely check out if money was not an object. You could, however, consider a sort of hybrid approach to handle the 'doze machines, which is what I am thinking about right now. In a nutshell, you would use some other mechanism to make the 'doze machines back themselves up to a share, and then you would use amanda to backup that share periodically. Windows XP Professional has a built-in backup/restore function, which is one option that I plan to try. I am using a 1 TB Snap Appliance 4500 NAS which came bundled with Backup Express for Guardian OS and an unlimited number of Windows clients, so that is another option that I plan to try. The Snap itself runs Guardian OS, which is some flavor of Linux, and I have it NFS mounted on my amanda server, and also use it as my holding disk for amanda backups. Then I can use GNUTAR with amanda to backup the directory where the 'doze machines have taken a dump. A lot of Windows people I have talked to lately are using a 2nd hard drive on USB or Firewire to do backups, or in a cold-swap cartridge. Norton Ghost is a popular application to do that, also XP Pro backup/restore, and even robocopy (is that Win2K Pro?) with varying degrees of automation. DVD+R with 4.7 GB capacity is popular for longer-term storage. The latest Norton Suite 2004 Pro looks like makes it pretty easy to automate Ghost backup to a 2nd disk or to DVD+R (I have it on the Windows partition of my home computer) but I mainly use Windows for games and don't care enough about losing that data to have put the effort into automating backups. --jonathan
Re: I don't think Amanda is going to work for my environment...
Hello! > 1. Does not handle NT Servers well with smbclient. The biggest problem I > have is the limited ability to exclude files from the backups. I have a > server, ntFileServer, that is running Windows NT Server. There are a ton of > files on the server that are of type mp3, mpeg, avi, etc that I do not want > to backup. Because there is no way for me to build a smart exclude list > that will exclude *.mp3, *.mpg, *.avi etc. So lets say I kill this server > (which I plan to soon) and install in it's place a linux server running > samba and share out all of these directories to users on the network, just > as it is currently, but with a linux server instead. Now the only way to > backup this server, with an exclude list, is to use gnutar. From the > EXCLUDE doc, "The GNU version of tar, (gnutar or gtar), reads its data at a > file system, (or higher), level and does include the option to exclude > specific files and/or directories. It should be mentioned here that tar > will change the access times no files. Tar has the ability to preserve the > access times however, doing so effectively disables incremental backups > since resetting the access time alters the inode change time, which in turn > causes the file to look like it needs to be archived again." So, now this > ads another problem to the mix. I cannot do incremental backups on files > that I decide to backup with gnutar, which I am using to solve my first > problem. This is unacceptable. It's quite a late in the evening here, so maybe I am not getting something, but is there a reason you need to preserve *access* times of files on the server? I'm backing up a server using GNU tar, I use exclude lists on some of the DLEs and I've never thought of that as a problem. Incremental backups work just fine. You are aware, aren't you, that access time is not the same as creation time or modification time on Unix file systems? > If I am understanding this correctly, Amanda has no way to incrementally > backup a disk that also uses a exclude list to prevent it from backing up > certain files. As I said above, incremental backups + exclude lists work together fine here. > 2. Not an intuitive way to think about backing things up. This rant is > somewhat related to the above rant. :) The exclude files need to be > written on the client? Again, from the EXCLUDE documentation, "An exclude > list is a file that resides on the CLIENT machine and contains paths to be > excluded, one per line. This file can be in any location on the CLIENT so > long as the same path is specified in the dumptype." Why should it be > necessary for the CLIENT to have this information reside on it. Because it is the CLIENT who actually runs the backup program (GNU tar) and the exclude list needs to be passed to GNU tar as a command line argument '--exclude-list'. > Perhaps I am just not "getting" the backup paradigm utilized by Amanda. Can > you clear some of this up for me? Or am I dead on and should perhaps go > shopping for a backup solution that would better fit my environment? So far I see nothing that would prevent you using Amanda (that is, once you get rid of the NT server ). > Amanda also has a lack of support for other areas that I need to have a > backup solution for. I currently have in my environment 5 NT servers, 1 > runs Exchange 5.5 and one that runs MSSQL Server 2000. I am able to run a > backup job in MSSQL that dumps the database to a file and I have amanda > backup that file. That's pretty standard approach with any database and any backup software (although some more sophisticated techniques do exist for some combinations of database + backup program) > but for the Exchange server I am left to > backup using BackupExec. :( Now that one I can't help you with :-( -- Toomas Aas | [EMAIL PROTECTED] | http://www.raad.tartu.ee/~toomas/ * Do infants enjoy infancy as much as adults enjoy adultery?
Re: I don't think Amanda is going to work for my environment...
On Thu, 18 Mar 2004 at 12:58pm, Michael Kahle wrote > If I am understanding this correctly, Amanda has no way to incrementally > backup a disk that also uses a exclude list to prevent it from backing up > certain files. This seams to me to be an important feature included in a > backup package. This is incorrect. I use exclude lists and tar extensively, and it Just Works, including (obviously) incrementals. > 2. Not an intuitive way to think about backing things up. This rant is > somewhat related to the above rant. :) The exclude files need to be > written on the client? Again, from the EXCLUDE documentation, "An exclude > list is a file that resides on the CLIENT machine and contains paths to be > excluded, one per line. This file can be in any location on the CLIENT so > long as the same path is specified in the dumptype." Why should it be > necessary for the CLIENT to have this information reside on it. If one > thinks about backups using a CLIENT-SERVER model, I would think it would > only be necessary to have a client piece accept instructions from the server > and do the backups based on those instructions. The server should contain > information necessary to backup all clients connected to the system. Not > some client specific information on the server and other client information > on the client. With recent versions of amanda, you *can* do all the exlucing and including in the disklist. Look in the amanda(8) man page for 'include append' and 'exclude append'. I personally think it makes sense having the exclude list on the client. I put it in the top level of the directory it applies to (named .exclude). This makes it very easy to look at and figure out what's being excluded, what needs to be, and whether anything is being missed. It also allows me to have one dumptype that does excludes, and apply that to as many disklist entries as I need to. > Amanda also has a lack of support for other areas that I need to have a > backup solution for. I currently have in my environment 5 NT servers, 1 > runs Exchange 5.5 and one that runs MSSQL Server 2000. I am able to run a > backup job in MSSQL that dumps the database to a file and I have amanda > backup that file. It works fine, but for the Exchange server I am left to > backup using BackupExec. :( I touched on this on a post I made yesterday, > sort of, when I asked about a native client for MS products using the Backup > API. Are these API's free to link to? That is, could one write software > using gnu compilers to use these libraries and write a robust interface to > backup Microsoft products like Exchange, MSSQL, the registry, file system, > etc? Has this road every been crossed? How do other backup solutions > attack this problem? As was mentioned recently, amanda will compile and run under cygwin, but this (obviously) doesn't use the MS backup API. There was (IIRC) an amanda-win32 project at one point that may have been using those, but I think it went under. It is true that it takes some wedging to make amanda work in a 'doze heavy environment -- that's simply not what it was designed for. As for advice on commercial solutions, this isn't exactly the best place to ask. ;) -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
