Re: What tapecycle value to use?
On Thu, Oct 10, 2002 at 02:00:22PM +0200, Toralf Lund wrote: --On Wednesday, October 09, 2002 16:53:55 +0200 Toralf Lund [EMAIL PROTECTED] wrote: ... . You can take certain tapes out the sequence for permanent archiving by the command amadmin yourconfig no-reuse thetapelabel. What exactly does that mean? That no tape labelled thetapelabel will be accepted in the future, or just that the one that currently has the label won't be reused? How do you envision amanda devining that a tape labeled blue is not the same tape that was labeled blue yesterday? I was thinking more in terms of checking whether the tape has been written. Amanda is good, but not that good. Thus when you say don't use anymore, anything labeled blue, it won't. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: What tapecycle value to use?
On Thu, Oct 10, 2002 at 01:38:18PM +0200, Toralf Lund wrote: Forgot to mention this earlier: I'm not using incrementals at all. Tapes from the same week will contain full backups of different directories, and a given file is backed up (only) once a week. OK, I'll bite... How hard did you have to twist amanda's arm to convince her to do that? dumpcycle 1 week skip-incr Not a lot harder than that. Well, maybe there is no guarantee against getting the same file backed up twice if there's room for it on a tape after everything has been backed up once, but I don't care about that. And there isn't a lot if it (additional tape space when everything has been backed up.) Which is also why incrementals are fairly meaningless. And you do realize that incrementals tend to be a good deal smaller than fulls, yes? Like I said, we work with LARGE files... I'd be willing to bet that you'd be better off letting amanda figure out for herself what to back up, when, and at what level instead of trying to force this 'one full per dump cycle and no incrementals' policy on her. I'm getting the impression that not only have you decided to do things in a way that amanda really isn't designed to handle, but that you'd be able to accomplish it much more easily by just using cron, dump, and gzip. No. With dump, and also many high level backup applications, I would have to split the data into backup sets or whatever you call it, based on what will fit on a tape, which is a significant amount of work.
Multimedia Design at 5$ per hour
Title: Dear Friend Dear Friend, We would like to introduce ourselves Creativeskulls, We are a new media concern having a setup of 11 high end workstations. We are looking for business partners/clients who are interested in offloading their work. Our rate is only 5 US$ for an hour of working. We have 2 master level flash certificate holders and 3 multimedia Ex. Faculty on our rolls. Our clients include Daewoo, LG India, NIIT, Modi Group (India) and now we are planning a foray in the foreign market. Please read economicaldesign.creativeskulls.com/faq.htm on the process that we follow to ensure quality, timely deliveries and economical pricing. Please visit www.creativeskulls.com/skullworks.html to see our portfolio. For further information regarding this offer please write us at [EMAIL PROTECTED] Here are some of our recent creations. Please click on the image to see a detailed view. One of our most interesting works. The photograph is of Aishwarya Rai (Miss World 1994) one of the most beautiful woman in the world. The picture that you see is made up of Approximately 10,000 individual images. Flash work is what we love to do. We have master level certificate holder from 'Brainbench' on our rolls. Our expertise in flash speaks for itself in our works. Scripting happens to be our strength. Having two ex multimedia faculty on our team gives us that edge in 3d. The samples of our work speak for themselves. We have special interest and expertise in character animation. Websites, 3d Work, Print Graphics, Corporate Presentations/Video, Flash Work, CD Rom Presentations all done at an incredible rate and incredible quality. The sample work speaks for itself. This is not a spam mail. It is a one time offer, sent to only those people, who were listed in Graphic Design sites. Just reply to us with the subject line "remove" and you will never be sent another offer mail.We did not purchase your Email id's nor do we sell any Email id's.
backing up postgres / informix ?
Hi, I was wondering if anyone had solutions for backing up live databases. I've some (smallish) databases - around 2GB maybe, that I'd like to backup. I was wondering how amanda could be configured to do this. Is it possible to do something like getting the amanda user to run a script shutting down the database, while it runs tar/dump on the database filesystems ? (or, as in the case of informix, the raw partitions) ? Or, am I better off having a huge disk on the database boxes, and doing a dump of the databases as SQL to disk every night, before the backup is run ? john
questions about proposed amanda hardware solution
Folks -- I've been using amanda for a couple of years now on Sun hardware (DLT8000-based L9 tape library, with stctl as the tape changer controller). It's worked great, but I'm nearing capacity with the current system and trying to decide whether to expand (another L9 and a load of DLT-IV and Sun-bought holding disk) or switch to another hardware platform and tape format. I was hoping folks could take a look at my suggested new setup, and offer advice, warnings, horror stories, etc. The amanda control host would be a smallish Linux rackmount job with attached SCSI+RAID disk enclosure -- currently I'm considering a Dell PowerEdge 1650 (1U, PIII-based) with a Dell PowerVault 220S (3U, 14 x 36 GB Ultra160 SCSI disks + PERC/3 RAID controller). Doing a RAID-5 + hot spare across the fourteen disks would give me 400 real GB (1024^3) of holding disk. I'd use RedHat Linux 7.3 for the OS. Main reason for picking Dell and RedHat over other PC/Linux options is just that they're the standard for my group these days. For a tape library controller, I'm hoping that either the mtx or scsi-changer drivers will be able to control the library hardware (see below). My standard backup depth has been to try to have around two weeks of dailies, with a short dumpcycle to get a high proportion of level zeroes onto the tapes (more for convenience of restores than anything else). I've also got a separate offsite rotation of all level zeroes, run once a week, with about four weeks of depth, in case the data center ever burns down. With my current backups, I wind up having the operators flip the DLT tapes in the L9 every day, which causes lots of dropped tapes and more wear and tear on the library. I'd like to get away from that, and wind up with a library capacity that lets me keep the daily rotation loaded all the time, with an extra slot so I can load the offsite tape for the week on Monday morning and then forget about it until the next Monday. After poking around, the AIT3 tape format from Sony feels like a nice direction in which to move (alternatives being sticking with DLT, or moving to SDLT or Ultrium). I don't know much about hardware vendors for AIT3 libraries, but two of my candidates are the QualStar CLS-4216 (2U, with 1 AIT3 drive and 16 tape slots) or the Overland LibraryPro (4U-ish, with 1 AIT3 drive and 19 tape slots). Either of these would let me load two weeks worth of tapes and a weekly offsite. I'm assuming I'd wind up with two libraries and two amanda rotations with about 200+ GB of data in each. I could go with software 2:1 compression to save on holding disk, or dump to disk uncompressed and then try to get the mythic 2.6:1 hardware compression advertised for the AIT3 format. Does the above sound reasonable? terrible? Have I missed any compatibility issues, and does anyone use a setup similar to this one? Thanks for any and all advice, -mgs
Sony DLTtape III XT Tapetype
If anyone could help me with the said tapetype it would be greatly apprecaited. I've already got the drive up and ready to roll with amanda. Tks, Steve Bertrand Northumberland Network Services
exclude list problems (PEBKAC?)
I've checked the docs/FAQ/google/archives and I must be missing
something really simple. :\
I have an amanda client that I'm backing having problems backing up.
We're using tar for /usr on this client:
SNIP disklist snippet
client /usr no-mirror-tar
SNIP
Here's the def for no-mirror-tar:
SNIP amanda.conf snippet
define dumptype no-mirror-tar {
global
program GNUTAR
compress client fast
comment usr partitions dumped with tar
exclude list .amanda-exclude.gtar
priority medium
}
SNIP
The global config only has a comment and 'index yes.'
The exclude list (on the client) has this:
SNIP
$ cat /usr/.amanda-exclude.gtar
./local/mirror
./local/www/logs
SNIP
The runtar.debug looks like this:
SNIP
gtar: version 2.4.2
running: /usr/local/bin/tar: gtar --create --directory /usr
--listed-incremental /usr/local/var/amanda/gnutar-lists/client_usr_0.new
--sparse --one-file-system --ignore-failed-read --totals --file -
--exclude-from /usr/.amanda-exclude.gtar .
SNIP
And the backup of this partition errors out with this:
SNIP
/-- client /usr lev 0 FAILED [/usr/local/bin/tar returned 2]
sendbackup: start [client:/usr level 0]
sendbackup: info BACKUP=/usr/local/bin/tar
sendbackup: info RECOVER_CMD=/usr/local/bin/gzip -dc |/usr/local/bin/tar
-f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
? gtar: Cannot add file ./local/mirror/pub/redhat/i386
.
.
.
| Total bytes written: 8484853760
? gtar: Error exit delayed from previous errors
sendbackup: error [/usr/local/bin/tar returned 2]
\
SNIP
There's also a 'file changed as we read it' error, but all of these
errors come from ./local (/usr/local) in places that are excluded (see
above).
Suggestions?
Thanks,
Mike
Re: What tapecycle value to use?
--On Friday, October 11, 2002 09:56:46 +0200 Toralf Lund [EMAIL PROTECTED] wrote: On Thu, Oct 10, 2002 at 01:38:18PM +0200, Toralf Lund wrote: On Wed, Oct 09, 2002 at 04:53:55PM +0200, Toralf Lund wrote: - As was mentioned earlier, your tapes won't stay synchronized to the calendar. If Monday is a holiday and the tape doesn't get switched, amanda will ask for the Monday tape on Tuesday. Doesn't really matter. The day names are just meant to make it easier for the person managing the tapes (not me) to know which one to bring. I thought about using labels of the form week no.run no or simple sequencing, but I decided that would be more confusing. Your confusion level may vary from mine and from the person managing the tapes. I would certainly be confused when amanda needs a tape labeled Monday on Wednesday. Well, I may be wrong, but I'm assuming that would be the case only if an exceptional event that would have to be addressed anyway, occured. It doesn't happen often, but sooner or later you will have a tape failure, power failure, wrong tape loaded, unexpected reboot, or whatever that will get your names out of sync. What happens when your data grows enough that you need to increase runtapes to 2 ? Trust the collective experience of the list, they will get out of your planned sequence. I still find it hard to convince myself that a simple sequencing number, like most people suggest, would be better. OK, but don't say we didn't warn you ;-). Frank -- Frank Smith[EMAIL PROTECTED] Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: amanda clinet in RH6.2 giving error
--On Friday, October 11, 2002 15:26:45 +0530 Vijay [EMAIL PROTECTED] wrote: On Friday 11 October 2002 12:18 pm, John Koenig wrote: you need to post your log files... more data, etc... The amandad log file generated in the client is appended below. It shows that authentication of the remote machine as amanda fails. but for no reason. because rsh amanda works very fine passwordless as .rhosts has been set. mail me if anything else has to be checked. --vijay amandad.log file generated in amanda client machine --- amandad: debug 1 pid 12131 ruid 1079 euid 1079: start at Sat Sep 21 11:10:24 2002 amandad: version 2.4.3b4-20020917 amandad: build: VERSION=Amanda-2.4.3b4-20020917 amandad:BUILT_DATE=Thu Sep 19 10:00:14 IST 2002 amandad:BUILT_MACH=Linux darter 2.4.7-10 #1 Thu Sep 6 17:27:27 EDT 2001 i686 unknown amandad:CC=gcc amandad:CONFIGURE_COMMAND='./configure' '--with-user=amanda' '--with-group=disk' '--without-amandahosts' amandad: paths: bindir=/usr/local/bin sbindir=/usr/local/sbin amandad:libexecdir=/usr/local/libexec mandir=/usr/local/man amandad:AMANDA_TMPDIR=/tmp/amanda AMANDA_DBGDIR=/tmp/amanda amandad:CONFIG_DIR=/usr/local/etc/amanda DEV_PREFIX=/dev/ amandad:RDEV_PREFIX=/dev/ DUMP=/sbin/dump amandad:RESTORE=/sbin/restore SAMBA_CLIENT=/usr/bin/smbclient amandad:GNUTAR=/bin/gtar COMPRESS_PATH=/usr/bin/gzip amandad:UNCOMPRESS_PATH=/usr/bin/gzip MAILER=/usr/bin/Mail amandad:listed_incr_dir=/usr/local/var/amanda/gnutar-lists amandad: defs: DEFAULT_SERVER=darter DEFAULT_CONFIG=DailySet1 amandad:DEFAULT_TAPE_SERVER=darter amandad:DEFAULT_TAPE_DEVICE=/dev/null HAVE_MMAP HAVE_SYSVSHM amandad:LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE amandad:AMANDA_DEBUG_DAYS=4 BSD_SECURITY CLIENT_LOGIN=amanda amandad:FORCE_USERID HAVE_GZIP COMPRESS_SUFFIX=.gz amandad:COMPRESS_FAST_OPT=--fast COMPRESS_BEST_OPT=--best amandad:UNCOMPRESS_OPT=-dc amandad: time 0.000: got packet: Amanda 2.4 REQ HANDLE 000-B0670808 SEQ 1032586800 SECURITY USER amanda SERVICE selfcheck OPTIONS features=feff9f00;maxdumps=1;hostname=darter; GNUTAR /home/kuruvi2/cygnet 0 OPTIONS | ;auth=bsd;index;exclude-list=/usr/local/etc/amanda/cygnetbkup/homeexclude; GNUTAR /var/lib/mysql 0 OPTIONS | ;auth=bsd;index;exclude-list=/usr/local/etc/amanda/cygnetbkup/rootexclude; amandad: time 0.000: sending ack: Amanda 2.4 ACK HANDLE 000-B0670808 SEQ 1032586800 amandad: time 0.003: bsd security: remote host kuruvi.ooty.tenet.res.in user amanda local user amanda amandad: time 20.103: bsd security check to amanda from [EMAIL PROTECTED] passed Looks like authentication worked OK ^ amandad: time 20.104: running service /usr/local/libexec/selfcheck amandad: time 20.173: sending REP packet: Amanda 2.4 REP HANDLE 000-B0670808 SEQ 1032586800 OPTIONS features=feff9f00; OK /home/kuruvi2/cygnet OK /home/kuruvi2/cygnet OK /home/kuruvi2/cygnet OK /var/lib/mysql OK /var/lib/mysql OK /var/lib/mysql ERROR [can not execute /usr/local/libexec/runtar: Permission denied] This seems to be your real problem ^^ Frank OK /bin/gtar executable OK /etc/amandates read/writable OK /usr/local/var/amanda/gnutar-lists/. read/writable OK /usr/bin/gzip executable OK /dev/null read/writable OK /tmp/amanda has more than 64 KB available. OK /tmp/amanda has more than 64 KB available. OK /etc has more than 64 KB available. amandad: time 20.175: got packet: Amanda 2.4 ACK HANDLE 000-B0670808 SEQ 1032586800 amandad: time 20.175: pid 12131 finish time Sat Sep 21 11:10:44 2002 -- Frank Smith[EMAIL PROTECTED] Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: What tapecycle value to use?
On Fri, Oct 11, 2002 at 09:15:34AM +0200, Toralf Lund wrote: On Thu, Oct 10, 2002 at 01:38:18PM +0200, Toralf Lund wrote: Forgot to mention this earlier: I'm not using incrementals at all. Tapes from the same week will contain full backups of different directories, and a given file is backed up (only) once a week. OK, I'll bite... How hard did you have to twist amanda's arm to convince her to do that? dumpcycle 1 week skip-incr Tofalf's scheme raises a question in my mind about how does amanda handle this scheme. Maybe someone else has experience with something similar. Suppose I have lots of disklist entries. Further, suppose I use dumpcycle7 days runspercycle 7 skip-incr Will amanda still try for balancing of the nightly dumpsize, simply ignoring the incrementals? Seems like it should. Anyone doing it? If so, Toralf's scheme, given his requirements may be quite reasonable. One thing Toralf; in other postings you mention backing up files. And how it would be difficult to break things up to fit into tape size chunks. Be aware that amanda backs up disklist entries, either file systems or directory trees. You don't tell it to backup a system and it picks and chooses which files to backup today. It picks and chooses which disklist entries to backup. Thus you still have to be concerned with your disklist entries fitting on a single tape. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: exclude list problems (PEBKAC?)
--On Friday, October 11, 2002 10:26:58 -0500 pointer [EMAIL PROTECTED] wrote:
I've checked the docs/FAQ/google/archives and I must be missing
something really simple. :\
I have an amanda client that I'm backing having problems backing up.
We're using tar for /usr on this client:
SNIP disklist snippet
client /usr no-mirror-tar
SNIP
Here's the def for no-mirror-tar:
SNIP amanda.conf snippet
define dumptype no-mirror-tar {
global
program GNUTAR
compress client fast
comment usr partitions dumped with tar
exclude list .amanda-exclude.gtar
priority medium
}
SNIP
The global config only has a comment and 'index yes.'
The exclude list (on the client) has this:
SNIP
$ cat /usr/.amanda-exclude.gtar
./local/mirror
./local/www/logs
SNIP
The runtar.debug looks like this:
SNIP
gtar: version 2.4.2
running: /usr/local/bin/tar: gtar --create --directory /usr
--listed-incremental /usr/local/var/amanda/gnutar-lists/client_usr_0.new
--sparse --one-file-system --ignore-failed-read --totals --file -
--exclude-from /usr/.amanda-exclude.gtar .
SNIP
And the backup of this partition errors out with this:
SNIP
/-- client /usr lev 0 FAILED [/usr/local/bin/tar returned 2]
sendbackup: start [client:/usr level 0]
sendbackup: info BACKUP=/usr/local/bin/tar
sendbackup: info RECOVER_CMD=/usr/local/bin/gzip -dc |/usr/local/bin/tar
-f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
? gtar: Cannot add file ./local/mirror/pub/redhat/i386
.
.
.
| Total bytes written: 8484853760
? gtar: Error exit delayed from previous errors
sendbackup: error [/usr/local/bin/tar returned 2]
\
SNIP
There's also a 'file changed as we read it' error, but all of these
errors come from ./local (/usr/local) in places that are excluded (see
above).
Suggestions?
Thanks,
Mike
Your config looks correct to me. What does '/usr/local/bin/tar --version'
show? Can your backup user read the exclude file?
Frank
--
Frank Smith[EMAIL PROTECTED]
Systems Administrator Voice: 512-374-4673
Hoover's Online Fax: 512-374-4501
Re: backing up postgres / informix ?
back that up. Of course, if you use tar I believe some folks have written wrapper scripts for databade backups. This is what I do for my Oracle database. I have a shell script that sets the database into backup mode and copies all the files to a separate section of the disk. Amanda then comes along and backs up those files. There is a version of the script for Informix, though I have not used it. You can find it here: http://www.storagemountain.com/free-backup-software1.html Good Luck! Anthony Valentine -- UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
Re: What tapecycle value to use?
On Friday 11 October 2002 03:56, Toralf Lund wrote: the person managing the tapes (not me) to know which one to bring. I thought about using labels of the form week no.run no or simple sequencing, but I decided that would be more confusing. In which case have amanda do a printout of whats on each tape, with will I believe also contain the date, and keep that printout with that tape until such time as that tape is over-written. The paper copy effectively becomeing your label. Your confusion level may vary from mine and from the person managing the tapes. I would certainly be confused when amanda needs a tape labeled Monday on Wednesday. Well, I may be wrong, but I'm assuming that would be the case only if an exceptional event that would have to be addressed anyway, occured. Trust the collective experience of the list, they will get out of your planned sequence. I still find it hard to convince myself that a simple sequencing number, like most people suggest, would be better. Forgot to mention this earlier: I'm not using incrementals at all. Tapes from the same week will contain full backups of different directories, and a given file is backed up (only) once a week. When you figure out how to get amanda to do this please let the list know. It is very atypical amanda usage. Is it really? I'm merely using skip-incr, and the total amount backed up is a little smaller than the space on runspercycle tapes. I don't see but one problem with that, and it comes in terms of the maximum filesize vs the size of one tape. Because amanda cannot, and likely never will for dependability reasons, span a single disklist entry across 2 or more tapes, instead restarting the failed dump on the next tape when EOT is encountered, it makes sense to have the disklist entry's broken up into pieces smaller than a tape. This translates to less wasted tape when amanda has to move on to the next tape in the current invocation. It seems to me that: 1. Under these conditions, Amanda won't try to do a 2nd full dump of any of the data, even though the config allows it to do that (?) Also, if it does, I don't really care. 2. Trying to run incremental dumps would be meanlingless as there is hardly any room for the data, although some of it might fit in at the holes left when there is more room on a tape, but not enough to allow a full dump of a dirlist entry to be written. - Toralf Two questions come to mind, the answers to which may lead us to a better suggestion box filler here. 1. How big are the largest of these files expressed as a percentage of a tapes capacity? 2. Do their names regularly change? And a subquestion might be asked, how compressable are these files? -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.17% setiathome rank, not too shabby for a WV hillbilly
Re: exclude list problems (PEBKAC?)
On Fri, Oct 11, 2002 at 10:26:58AM -0500, pointer wrote: I've checked the docs/FAQ/google/archives and I must be missing something really simple. :\ I have an amanda client that I'm backing having problems backing up. We're using tar for /usr on this client: SNIP /-- client /usr lev 0 FAILED [/usr/local/bin/tar returned 2] sendbackup: start [client:/usr level 0] sendbackup: info BACKUP=/usr/local/bin/tar sendbackup: info RECOVER_CMD=/usr/local/bin/gzip -dc |/usr/local/bin/tar -f... - sendbackup: info COMPRESS_SUFFIX=.gz sendbackup: info end ? gtar: Cannot add file ./local/mirror/pub/redhat/i386 . . . | Total bytes written: 8484853760 ? gtar: Error exit delayed from previous errors sendbackup: error [/usr/local/bin/tar returned 2] \ SNIP Even when gnutar ignores failed reads, it exits with an error message and an error status (the returned 2 above). You can compile amanda to consider this exit message and status as normal by placing in config.h a line like #define IGNORE_TAR_ERRORS 1 -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: exclude list problems (PEBKAC?)
Frank, On Fri, 2002-10-11 at 10:50, Frank Smith wrote: Your config looks correct to me. What does '/usr/local/bin/tar --version' show? SNIP $ /usr/local/bin/tar --version tar (GNU tar) 1.13 Copyright (C) 1988, 92,93,94,95,96,97,98, 1999 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by John Gilmore and Jay Fenlason. SNIP Can your backup user read the exclude file? Yes. SNIP $ ls -al /usr/.amanda-exclude.gtar -rw-r--r-- 1 root other 32 Oct 10 10:22 /usr/.amanda-exclude.gtar SNIP Cheers, Mike
Re: backing up postgres / informix ?
On Friday 11 October 2002 05:07, John P. Looney wrote: Hi, I was wondering if anyone had solutions for backing up live databases. I've some (smallish) databases - around 2GB maybe, that I'd like to backup. I was wondering how amanda could be configured to do this. Is it possible to do something like getting the amanda user to run a script shutting down the database, while it runs tar/dump on the database filesystems ? (or, as in the case of informix, the raw partitions) ? Or, am I better off having a huge disk on the database boxes, and doing a dump of the databases as SQL to disk every night, before the backup is run ? Generally, I think most have recommended that some sort of a snapshot be done, and the snapshot then backed up. This is the essence of your latter idea. However, I'd see no real problem in my limited experience in attempting to write a script wrapper around amanda that did the former. That would still take only one crontab entry. For error catching the next morning I think I'd want to do it in something resembling this pseudocode: - #!/bin/bash do whatever shuts down the database \ run amanda \ restart database - The output of that simplied script should be redirected to a logfile of some kind to be inspected the next morning, and any corrective action required can then be taken. The \ stuff means that the script will exit on any error before the next line has a chance to cover up the error with its excessive verbiage. Lots easier to troubleshoot that way, and simple enough to complete by hand once the error is investigated. Database experts please feel free to shoot me down. :-) --- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.17% setiathome rank, not too shabby for a WV hillbilly
RE: tape errors
FWIW: We had similar symptoms happening here with an external HP DLT1 drive off a motherboard-internal Adaptec Ultra2Wide controller. The tape was running U2W in single-ended mode and was connected using the HP provided LVD/SE cable. It appears the SCSI bus was too long with that cable. After switching to a shorter external cable (only about 30cm instead of 150cm) all SCSI problems/tape errors disappeared. Bernhard -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Joshua Baker-LePain Sent: Friday, October 11, 2002 5:04 AM To: Jerry Cc: [EMAIL PROTECTED] Subject: Re: tape errors On Thu, 10 Oct 2002 at 5:30pm, Jerry wrote Solaris Exabyte EZ17 Autoloader w/m2 drive 225m AME self clean tapes Using dump Once it's busted can't even tar to the tape, either gets stuck in the middle or just starts freaking out and sounds like the tape is lost seeking around. As far as syslog, I got a bunch of scsi errors but it's hard to decipher which ones were related since it ended up powering off the drive and on to stop it from resetting and clanking the tape back and forth. The drive has never really been used and the tapes shouldn't be that old, considering the admin before me never even got it to control the changer. All I can say is, ouch. As others have said, start taking a good look at your hardware -- the changer, the cables, termination, the HBA... This sounds like a multi-goat problem. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: HI ALL
On Friday 11 October 2002 06:22, O-Zone wrote:
Hi All,
i'm a new user Amanda user. I've got installed 2.4.3 and i'm
unable to get it working correctly. I've got a SCSI TAPE STREAMER
Sony SDT-9000 with Sequential-Access attacched to scsi tape
device st0. There's no disk tape changer. So i'll try to
configure Amanda,removing all unused files in
/usr/local/etc/amanda/Daily, in that way:
-- amanda.conf
# MOON.TDSIENA.IT
org Daily# your organization name for reports
mailto [EMAIL PROTECTED] # space separated list of operators
at your site
dumpuser backup # the user to run dumps under
inparallel 4 # maximum dumpers that will run in parallel (max 63)
# this maximum can be increased at compile-time,
# modifying MAX_DUMPERS in server-src/driverio.h
dumporder sssS # specify the priority order of each dumper
# s - smallest size
# S - biggest size
# t - smallest time
# T - biggest time
# b - smallest bandwitdh
# B - biggest bandwitdh
# try BTBTBTBTBTBT if you are not holding
# disk constrained
netusage 600 Kbps # maximum net bandwidth for Amanda, in KB per
sec
dumpcycle 1 weeks # the number of days in the normal dump cycle
runspercycle 5 # the number of amdump runs in dumpcycle
days # (4 weeks * 5 amdump runs per week -- just weekdays)
tapecycle 5 tapes # the number of tapes in rotation
# 4 weeks (dumpcycle) times 5 tapes per week (just
# the weekdays) plus a few to handle errors that
# need amflush and so we do not overwrite the full
# backups performed at the beginning of the previous
# cycle
###
The above setup, while it will work, is considered to be dangerous
to your data because you will at any one time, be over-writing your
only copy of last weeks work with the current work. If you were to
have a power outage in the middle of writing a tape, the whole
thing is blown. Its considered to be good practice to have at
least 2x or more runspercycle tapes in tapecycle. These tapes are
quite inexpensive so get another 10 pack so you'll have spares,
putting say 7 or 8 of them into the tapecycle. By doing this, you
are assured of haveing a full backup of the system at least 2
dumpcycles deep.
bumpsize 20 Mb # minimum savings (threshold) to bump level 1 - 2
bumpdays 1 # minimum days at each level
bumpmult 4 # threshold = bumpsize * bumpmult^(level-1)
etimeout 300 # number of seconds per filesystem for estimates.
dtimeout 1800 # number of idle seconds before a dump is aborted.
ctimeout 30# maximum number of seconds that amcheck waits
# for each client host
tapebufs 20
This can be run up a ways if you have plenty of ram, I use 60 here
with half a gig of ram.
runtapes 1 # number of tapes to be used in a single run of amdump
Your tapetype is only a 2gigger, is this big enough?
tpchanger chg-manual # the tape-changer glue script
if runtapes is 1 then comment the above line out, else if it needs
more than one tape in runtapes, use chg-manual but that will
require your presence at change time.
tapedev /dev/st0 # the no-rewind tape device to be used
But you have specified the rewinding device here, it should be
/dev/nst0. Using st0 is a big non-no.
-
you don't need any of this from here
rawtapedev /dev/nftape # the raw device to be used (ftape only)
changerfile /usr/local/etc/amanda/Daily/changer.conf
changerdev /dev/null
to here
--
tapetype HP-DDS# what kind of tape it is (see tapetypes below)
labelstr ^Daily[0-9][0-9]*$ # label constraint regex: all tapes
must match
holdingdisk hd1 {
comment main holding disk
directory /home/amanda # where the holding disk is
use 640 Mb # how much space can we use on it
# a non-positive value means:
#use all space but that value
chunksize 1Gb # size of chunk if you want big dump to be
# dumped on multiple files on holding disks
# N Kb/Mb/Gb split images in chunks of size N
# The maximum value should be
# (MAX_FILE_SIZE - 1Mb)
# 0 same as INT_MAX bytes
}
reserve 30 # percent
# This means save at least 30% of the holding disk space for
degraded # mode backups.
Re: exclude list problems (PEBKAC?)
On 11 Oct 2002 at 11:58am, pointer wrote On Fri, 2002-10-11 at 10:50, Frank Smith wrote: Your config looks correct to me. What does '/usr/local/bin/tar --version' show? SNIP $ /usr/local/bin/tar --version tar (GNU tar) 1.13 Bad. Bad bad bad. If you're using indexing, they're broken. amrecover won't work. Run, don't walk, to download 1.13.25 from ftp://alpha.gnu.org. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: exclude list problems (PEBKAC?)
--On Friday, October 11, 2002 11:58:27 -0500 pointer [EMAIL PROTECTED] wrote: Frank, On Fri, 2002-10-11 at 10:50, Frank Smith wrote: Your config looks correct to me. What does '/usr/local/bin/tar --version' show? SNIP $ /usr/local/bin/tar --version tar (GNU tar) 1.13 Copyright (C) 1988, 92,93,94,95,96,97,98, 1999 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by John Gilmore and Jay Fenlason. SNIP Can your backup user read the exclude file? Yes. SNIP $ ls -al /usr/.amanda-exclude.gtar -rw-r--r-- 1 root other 32 Oct 10 10:22 /usr/.amanda-exclude.gtar SNIP Cheers, Mike I'm stumped on why your excludes aren't working. Possible unrelated problem for you: does anyone on the list remember if tar 1.13 was one of the versions with the bad index file problem (the infamous 'big numbers')? Frank -- Frank Smith[EMAIL PROTECTED] Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: Sony DLTtape III XT Tapetype
On Friday 11 October 2002 10:30, Steve Bertrand wrote: If anyone could help me with the said tapetype it would be greatly apprecaited. I've already got the drive up and ready to roll with amanda. Tks, Steve Bertrand Northumberland Network Services Go into the tape-src's directory and type make tapetype That will make you a drive definer that you can then run against that drive. Be sure and turn any hardware compression off before starting as that can make the /dev/urandom data grow a bit and give a falsely small size. Then leave it off... Forever. Use software which takes time, but compresses much better. -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.17% setiathome rank, not too shabby for a WV hillbilly
Re: questions about proposed amanda hardware solution
On Friday 11 October 2002 10:00, Mike Simpson wrote: Folks -- I've been using amanda for a couple of years now on Sun hardware (DLT8000-based L9 tape library, with stctl as the tape changer controller). It's worked great, but I'm nearing capacity with the current system and trying to decide whether to expand (another L9 and a load of DLT-IV and Sun-bought holding disk) or switch to another hardware platform and tape format. I was hoping folks could take a look at my suggested new setup, and offer advice, warnings, horror stories, etc. The amanda control host would be a smallish Linux rackmount job with attached SCSI+RAID disk enclosure -- currently I'm considering a Dell PowerEdge 1650 (1U, PIII-based) with a Dell PowerVault 220S (3U, 14 x 36 GB Ultra160 SCSI disks + PERC/3 RAID controller). Doing a RAID-5 + hot spare across the fourteen disks would give me 400 real GB (1024^3) of holding disk. I'd use RedHat Linux 7.3 for the OS. Main reason for picking Dell and RedHat over other PC/Linux options is just that they're the standard for my group these days. For a tape library controller, I'm hoping that either the mtx or scsi-changer drivers will be able to control the library hardware (see below). My standard backup depth has been to try to have around two weeks of dailies, with a short dumpcycle to get a high proportion of level zeroes onto the tapes (more for convenience of restores than anything else). I've also got a separate offsite rotation of all level zeroes, run once a week, with about four weeks of depth, in case the data center ever burns down. With my current backups, I wind up having the operators flip the DLT tapes in the L9 every day, which causes lots of dropped tapes and more wear and tear on the library. I'd like to get away from that, and wind up with a library capacity that lets me keep the daily rotation loaded all the time, with an extra slot so I can load the offsite tape for the week on Monday morning and then forget about it until the next Monday. After poking around, the AIT3 tape format from Sony feels like a nice direction in which to move (alternatives being sticking with DLT, or moving to SDLT or Ultrium). I don't know much about hardware vendors for AIT3 libraries, but two of my candidates are the QualStar CLS-4216 (2U, with 1 AIT3 drive and 16 tape slots) or the Overland LibraryPro (4U-ish, with 1 AIT3 drive and 19 tape slots). Either of these would let me load two weeks worth of tapes and a weekly offsite. I'm assuming I'd wind up with two libraries and two amanda rotations with about 200+ GB of data in each. I could go with software 2:1 compression to save on holding disk, or dump to disk uncompressed and then try to get the mythic 2.6:1 hardware compression advertised for the AIT3 format. Does the above sound reasonable? terrible? Have I missed any compatibility issues, and does anyone use a setup similar to this one? Thanks for any and all advice, -mgs About the only comment I'd make is related to compression, particularly hardware compression thats done in the drive. Shut it off because its useage hides the true tape capacity from amanda and may cause EOT mistakes, and use client best for those parts of it that are compressable. A directory full of rpms and other archives won't compress any more, and may in fact grow some. Server would be nice, but would be done sequentially taking a longer time, whereas the clients can be paralleled even if they are slower, and of course compressed data uses less bandwidth to move it to the server. Here, I rsync the client an hour ahead of time, so its all local to this disk(singular) and my 'server best' gets me a tape thats about 30% of the raw data size on the avrerage run. On this machine, the runtimes of the compressor seem to about equal the time to write the tape on average, so its tolerable. The drive is a 46 gigger, the tapes are 4gb DDS2's. -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.17% setiathome rank, not too shabby for a WV hillbilly
Re: exclude list problems (PEBKAC?)
Joshua, On Fri, 2002-10-11 at 11:50, Joshua Baker-LePain wrote: Bad. Bad bad bad. If you're using indexing, they're broken. amrecover won't work. Run, don't walk, to download 1.13.25 from ftp://alpha.gnu.org. Thanks for the heads-up. I'd seen this, but hadn't gotten around to updating it. Yes, bad, bad, bad! Anyone know when a new version is coming out to fix CAN-2002-0399? Did I misunderstand the vuln announcement, or is this really only exploitable when a superuser extracts files from a tarball without looking at the contents...? SNIP $ tar --version tar (GNU tar) 1.13.25 Copyright (C) 2001 Free Software Foundation, Inc. This program comes with NO WARRANTY, to the extent permitted by law. You may redistribute it under the terms of the GNU General Public License; see the file named COPYING for details. Written by John Gilmore and Jay Fenlason. SNIP Cheers, Mike
Re: Sony DLTtape III XT Tapetype
You may also want to make sure that you have the latest version of tapetype. I seem to recall seeing a message a few days ago about a new and better version of tapetype that didn't make it into 2.4.3. On Fri, 2002-10-11 at 09:08, Gene Heskett wrote: On Friday 11 October 2002 10:30, Steve Bertrand wrote: If anyone could help me with the said tapetype it would be greatly apprecaited. I've already got the drive up and ready to roll with amanda. Tks, Steve Bertrand Northumberland Network Services Go into the tape-src's directory and type make tapetype That will make you a drive definer that you can then run against that drive. Be sure and turn any hardware compression off before starting as that can make the /dev/urandom data grow a bit and give a falsely small size. Then leave it off... Forever. Use software which takes time, but compresses much better. -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.17% setiathome rank, not too shabby for a WV hillbilly -- UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
Re: tape errors
On Friday 11 October 2002 12:34, Bernhard Beck wrote: FWIW: We had similar symptoms happening here with an external HP DLT1 drive off a motherboard-internal Adaptec Ultra2Wide controller. The tape was running U2W in single-ended mode and was connected using the HP provided LVD/SE cable. It appears the SCSI bus was too long with that cable. After switching to a shorter external cable (only about 30cm instead of 150cm) all SCSI problems/tape errors disappeared. Bernhard That would be pretty prima-faci evidence to me of a boogered termination someplace. The max length spec for the LVD cabling is ISTR several times longer than for std single-ended cableing, which is by itself about 40 meters! By going down to a 30cm cable, the echos are now so close that the circuitry isn't fast enough and doesn't see them. One other effect that I've no experience with might be that the LDV/SE cable might be one of the newer miniature cabling styles, resembling the very finely spaced ata100 cables. Its characteristic impedance is an unknown to me, whereas the regular, much wider cable is about 120 ohms. I'd think the smaller cables characteristic impedance might be a bit lower unless the wire size shrank in unison with the square laws involved in calculating impedances. And thats a huge guage shrink... Not practical to do in an IDC assembly style IMO. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Joshua Baker-LePain Sent: Friday, October 11, 2002 5:04 AM To: Jerry Cc: [EMAIL PROTECTED] Subject: Re: tape errors On Thu, 10 Oct 2002 at 5:30pm, Jerry wrote Solaris Exabyte EZ17 Autoloader w/m2 drive 225m AME self clean tapes Using dump Once it's busted can't even tar to the tape, either gets stuck in the middle or just starts freaking out and sounds like the tape is lost seeking around. As far as syslog, I got a bunch of scsi errors but it's hard to decipher which ones were related since it ended up powering off the drive and on to stop it from resetting and clanking the tape back and forth. The drive has never really been used and the tapes shouldn't be that old, considering the admin before me never even got it to control the changer. All I can say is, ouch. As others have said, start taking a good look at your hardware -- the changer, the cables, termination, the HBA... This sounds like a multi-goat problem. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.17% setiathome rank, not too shabby for a WV hillbilly
Re: exclude list problems (PEBKAC?)
On Friday 11 October 2002 12:58, Frank Smith wrote: --On Friday, October 11, 2002 11:58:27 -0500 pointer [EMAIL PROTECTED] wrote: Frank, On Fri, 2002-10-11 at 10:50, Frank Smith wrote: Your config looks correct to me. What does '/usr/local/bin/tar --version' show? SNIP $ /usr/local/bin/tar --version tar (GNU tar) 1.13 Copyright (C) 1988, 92,93,94,95,96,97,98, 1999 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by John Gilmore and Jay Fenlason. SNIP Can your backup user read the exclude file? Yes. SNIP $ ls -al /usr/.amanda-exclude.gtar -rw-r--r-- 1 root other 32 Oct 10 10:22 /usr/.amanda-exclude.gtar SNIP Cheers, Mike I'm stumped on why your excludes aren't working. Possible unrelated problem for you: does anyone on the list remember if tar 1.13 was one of the versions with the bad index file problem (the infamous 'big numbers')? Frank That was officially fixed with 1.13-19 Frank, it and 1.13-25 both seem to work just fine. What burns me a bit is that the usual tar --version, doesn't eject the minor number, so thery are all 1.13's. One must forcibly remove any tar thats there, and reinstall a known good one to be sure. -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.17% setiathome rank, not too shabby for a WV hillbilly
Re: questions about proposed amanda hardware solution
The amanda control host would be a smallish Linux rackmount job with attached SCSI+RAID disk enclosure -- currently I'm considering a Dell PowerEdge 1650 (1U, PIII-based) with a Dell PowerVault 220S (3U, 14 x 36 GB Ultra160 SCSI disks + PERC/3 RAID controller). Doing a RAID-5 + hot spare across the fourteen disks would give me 400 real GB (1024^3) of holding disk. One comment... When I was pricing holding disk storage about 6 months ago, I came to the conclusion that I would rather pay a buck or $1.25 per GB (ATA 100) than the $6-$6.50 per GB premium for Ultra 160... Though this may alter your vertical height requirements as I am not aware of any high-end rackmount IDE/ATA storage solutions...but I presume they exist... as it would be a business opportunity due to the cost per GB.
Re: exclude list problems (PEBKAC?)
On Friday 11 October 2002 14:04, pointer wrote: Joshua, On Fri, 2002-10-11 at 11:50, Joshua Baker-LePain wrote: Bad. Bad bad bad. If you're using indexing, they're broken. amrecover won't work. Run, don't walk, to download 1.13.25 from ftp://alpha.gnu.org. Thanks for the heads-up. I'd seen this, but hadn't gotten around to updating it. Yes, bad, bad, bad! Anyone know when a new version is coming out to fix CAN-2002-0399? Did I misunderstand the vuln announcement, or is this really only exploitable when a superuser extracts files from a tarball without looking at the contents...? Thats the way I read that announcement. SNIP $ tar --version tar (GNU tar) 1.13.25 Copyright (C) 2001 Free Software Foundation, Inc. This program comes with NO WARRANTY, to the extent permitted by law. You may redistribute it under the terms of the GNU General Public License; see the file named COPYING for details. Written by John Gilmore and Jay Fenlason. SNIP this is the good one AFAIK. -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.17% setiathome rank, not too shabby for a WV hillbilly
Re: exclude list problems (PEBKAC?)
--On Friday, October 11, 2002 15:11:00 -0400 Gene Heskett [EMAIL PROTECTED] wrote: On Friday 11 October 2002 12:58, Frank Smith wrote: Possible unrelated problem for you: does anyone on the list remember if tar 1.13 was one of the versions with the bad index file problem (the infamous 'big numbers')? Frank That was officially fixed with 1.13-19 Frank, it and 1.13-25 both seem to work just fine. What burns me a bit is that the usual tar --version, doesn't eject the minor number, so thery are all 1.13's. One must forcibly remove any tar thats there, and reinstall a known good one to be sure. Hmm, on my Debian linux server here I get: # /usr/local/bin/tar --version tar (GNU tar) 1.13.19 and on one of my Solaris boxes: # /usr/local/bin/tar --version tar (GNU tar) 1.13.25 Both were compiled from source, but even on a RedHat 7.2 box I see: #/bin/tar --version tar (GNU tar) 1.13.19 Maybe they started including the minor number in the version string somewhere along the line. Frank -- Frank Smith [EMAIL PROTECTED] Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: due vs dumpcycle
I was doing something I didn't realize was a problem.
Perhaps other may benefit from my goof:
global file:
define dumptype AAA {}
ERRONEOUS config file:
dumpcycle 1 week
define dumptype BBB {AAA; then more stuff}
ERRONEOUS diskfile:
node disk BBB
#this is fine; uses dumpcycle 7 days = 1week
node2disk2 AAA
# NOT ok; uses some generic default dumpcycle of 10 days
The FIX:
config file:
dumpcycle 1 week
define dumptype BBB {AAA; more stuff}
define dumptype CCC {AAA}
Use BBB or CCC but do not ever use AAA directly in a disklist.
This wasn't obvious to me!
Deb Baddorf
---
Deb Baddorf [EMAIL PROTECTED] 840-2289
You can't help getting older, but you don't have to get old.
- George Burns IXOYE
Re: questions about proposed amanda hardware solution
--On Friday, October 11, 2002 12:13:46 -0700 John Koenig [EMAIL PROTECTED] wrote: The amanda control host would be a smallish Linux rackmount job with attached SCSI+RAID disk enclosure -- currently I'm considering a Dell PowerEdge 1650 (1U, PIII-based) with a Dell PowerVault 220S (3U, 14 x 36 GB Ultra160 SCSI disks + PERC/3 RAID controller). Doing a RAID-5 + hot spare across the fourteen disks would give me 400 real GB (1024^3) of holding disk. One comment... When I was pricing holding disk storage about 6 months ago, I came to the conclusion that I would rather pay a buck or $1.25 per GB (ATA 100) than the $6-$6.50 per GB premium for Ultra 160... Though this may alter your vertical height requirements as I am not aware of any high-end rackmount IDE/ATA storage solutions...but I presume they exist... as it would be a business opportunity due to the cost per GB. If rack space is an issue, the are servers like the QSOL Q416r that are 4u with 16 hot-swappable IDE drives. Now that several vendors are making reasonably priced IDE RAID controllers it's no longer a major expense to create terabytes of storage. Frank -- Frank Smith[EMAIL PROTECTED] Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
RE: tape errors
On Friday 11 October 2002 12:34, Bernhard Beck wrote: FWIW: We had similar symptoms happening here with an external HP DLT1 drive off a motherboard-internal Adaptec Ultra2Wide controller. The tape was running U2W in single-ended mode and was connected using the HP provided LVD/SE cable. It appears the SCSI bus was too long with that cable. After switching to a shorter external cable (only about 30cm instead of 150cm) all SCSI problems/tape errors disappeared. Bernhard That would be pretty prima-faci evidence to me of a boogered termination someplace. The max length spec for the LVD cabling is ISTR several times longer than for std single-ended cableing, I thought of the termination, too. The HBA (AIC7896 Ultra2Wide) is terminating its end, the tape drive has an external LVD/SE termination, all connectors and cables are standard 68pin wide. The tape drive is the only device on this channel. The holding disks are on the second channel of that controller. The only explanation I could come up with to make sense from this behaviour was that the bus (for whatever reason) drops to UltraWide single-ended, which restricts electrical cable length to 1.5m. which is by itself about 40 meters! By going down to a 30cm cable, the echos are now so close that the circuitry isn't fast enough and doesn't see them. Don't know that much about the electrical part, but seems to make sense to me as well. Either way, the short external cable fixed or at least avoids the problem for me and the box has been stable over the last few days. Since this is a production box, I can't play around with it as much as I'd like to get this resolved. Maybe on a week-end some time. Bernhard One other effect that I've no experience with might be that the LDV/SE cable might be one of the newer miniature cabling styles, resembling the very finely spaced ata100 cables. Its characteristic impedance is an unknown to me, whereas the regular, much wider cable is about 120 ohms. I'd think the smaller cables characteristic impedance might be a bit lower unless the wire size shrank in unison with the square laws involved in calculating impedances. And thats a huge guage shrink... Not practical to do in an IDC assembly style IMO.
Re: questions about proposed amanda hardware solution
On Friday 11 October 2002 15:13, John Koenig wrote: The amanda control host would be a smallish Linux rackmount job with attached SCSI+RAID disk enclosure -- currently I'm considering a Dell PowerEdge 1650 (1U, PIII-based) with a Dell PowerVault 220S (3U, 14 x 36 GB Ultra160 SCSI disks + PERC/3 RAID controller). Doing a RAID-5 + hot spare across the fourteen disks would give me 400 real GB (1024^3) of holding disk. One comment... When I was pricing holding disk storage about 6 months ago, I came to the conclusion that I would rather pay a buck or $1.25 per GB (ATA 100) than the $6-$6.50 per GB premium for Ultra 160... Though this may alter your vertical height requirements as I am not aware of any high-end rackmount IDE/ATA storage solutions...but I presume they exist... as it would be a business opportunity due to the cost per GB. It is in fact being done, with a 4 drive cache of 160gb drives on promise raid cards, using not amanda, but rsync. Working great. -- Cheers, Gene AMD K6-III@500mhz 320M Athlon1600XP@1400mhz 512M 99.17% setiathome rank, not too shabby for a WV hillbilly
HI ALL
Hi All,
i'm a new user Amanda user. I've got installed 2.4.3 and i'm unable to get it
working correctly. I've got a SCSI TAPE STREAMER Sony SDT-9000 with
Sequential-Access attacched to scsi tape device st0. There's no disk tape
changer. So i'll try to configure Amanda,removing all unused files in
/usr/local/etc/amanda/Daily, in that way:
-- amanda.conf
# MOON.TDSIENA.IT
org Daily # your organization name for reports
mailto [EMAIL PROTECTED] # space separated list of operators at your
site
dumpuser backup # the user to run dumps under
inparallel 4# maximum dumpers that will run in parallel (max 63)
# this maximum can be increased at compile-time,
# modifying MAX_DUMPERS in server-src/driverio.h
dumporder sssS# specify the priority order of each dumper
# s - smallest size
# S - biggest size
# t - smallest time
# T - biggest time
# b - smallest bandwitdh
# B - biggest bandwitdh
# try BTBTBTBTBTBT if you are not holding
# disk constrained
netusage 600 Kbps # maximum net bandwidth for Amanda, in KB per sec
dumpcycle 1 weeks # the number of days in the normal dump cycle
runspercycle 5 # the number of amdump runs in dumpcycle days
# (4 weeks * 5 amdump runs per week -- just weekdays)
tapecycle 5 tapes # the number of tapes in rotation
# 4 weeks (dumpcycle) times 5 tapes per week (just
# the weekdays) plus a few to handle errors that
# need amflush and so we do not overwrite the full
# backups performed at the beginning of the previous
# cycle
bumpsize 20 Mb # minimum savings (threshold) to bump level 1 - 2
bumpdays 1 # minimum days at each level
bumpmult 4 # threshold = bumpsize * bumpmult^(level-1)
etimeout 300# number of seconds per filesystem for estimates.
dtimeout 1800 # number of idle seconds before a dump is aborted.
ctimeout 30 # maximum number of seconds that amcheck waits
# for each client host
tapebufs 20
runtapes 1 # number of tapes to be used in a single run of amdump
tpchanger chg-manual # the tape-changer glue script
tapedev /dev/st0 # the no-rewind tape device to be used
rawtapedev /dev/nftape# the raw device to be used (ftape only)
changerfile /usr/local/etc/amanda/Daily/changer.conf
changerdev /dev/null
tapetype HP-DDS # what kind of tape it is (see tapetypes below)
labelstr ^Daily[0-9][0-9]*$ # label constraint regex: all tapes must match
holdingdisk hd1 {
comment main holding disk
directory /home/amanda# where the holding disk is
use 640 Mb # how much space can we use on it
# a non-positive value means:
#use all space but that value
chunksize 1Gb # size of chunk if you want big dump to be
# dumped on multiple files on holding disks
# N Kb/Mb/Gb split images in chunks of size N
# The maximum value should be
# (MAX_FILE_SIZE - 1Mb)
# 0 same as INT_MAX bytes
}
reserve 30 # percent
# This means save at least 30% of the holding disk space for degraded
# mode backups.
autoflush no #
infofile /usr/adm/amanda/Daily/curinfo# database DIRECTORY
logdir /usr/adm/amanda/Daily# log directory
indexdir /usr/adm/amanda/Daily/index # index directory
define tapetype HP-DDS {
comment DAT HP-DDS tape drives
length 11700 mbytes
filemark 230 kbytes
speed 1024 kbytes
}
# dumptypes
define dumptype global {
comment Global definitions
# This is quite useful for setting global parameters, so you don't have
# to type them everywhere. All dumptype definitions in this sample file
# do include these definitions, either directly or indirectly.
# There's nothing special about the name `global'; if you create any
# dumptype that does not contain the word `global' or the name of any
# other dumptype that contains it, these definitions won't apply.
# Note that these definitions may be overridden in other
# dumptypes, if the redefinitions appear *after* the `global'
# dumptype name.
# You may want to use this for globally enabling or disabling
# indexing, recording, etc. Some examples:
# index yes
# record no
}
define dumptype always-full {
global
comment Full dump of this filesystem always
compress none
priority high
Re: tape errors
On Thu, 10 Oct 2002 at 5:30pm, Jerry wrote Solaris Exabyte EZ17 Autoloader w/m2 drive 225m AME self clean tapes Using dump Once it's busted can't even tar to the tape, either gets stuck in the middle or just starts freaking out and sounds like the tape is lost seeking around. As far as syslog, I got a bunch of scsi errors but it's hard to decipher which ones were related since it ended up powering off the drive and on to stop it from resetting and clanking the tape back and forth. The drive has never really been used and the tapes shouldn't be that old, considering the admin before me never even got it to control the changer. All I can say is, ouch. As others have said, start taking a good look at your hardware -- the changer, the cables, termination, the HBA... This sounds like a multi-goat problem. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: backing up postgres / informix ?
John P. Looney wrote: Hi, I was wondering if anyone had solutions for backing up live databases. I've some (smallish) databases - around 2GB maybe, that I'd like to backup. I was wondering how amanda could be configured to do this. Is it possible to do something like getting the amanda user to run a script shutting down the database, while it runs tar/dump on the database filesystems ? (or, as in the case of informix, the raw partitions) ? This is on the amanda todo list Or, am I better off having a huge disk on the database boxes, and doing a dump of the databases as SQL to disk every night, before the backup is run ? Prety sure this is your best option. Just dump the DB and have amanda back that up. Of course, if you use tar I believe some folks have written wrapper scripts for databade backups. =G=
Re: questions about proposed amanda hardware solution (fwd)
I originally sent this off list. But, as the conversation seems to have come around this way... -- Joshua Baker-LePain Department of Biomedical Engineering Duke University -- Forwarded message -- Date: Fri, 11 Oct 2002 10:56:11 -0400 (EDT) From: Joshua Baker-LePain [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: questions about proposed amanda hardware solution On Fri, 11 Oct 2002 at 9:00am, Mike Simpson wrote The amanda control host would be a smallish Linux rackmount job with attached SCSI+RAID disk enclosure -- currently I'm considering a Dell PowerEdge 1650 (1U, PIII-based) with a Dell PowerVault 220S (3U, 14 x 36 GB Ultra160 SCSI disks + PERC/3 RAID controller). Doing a RAID-5 + hot spare across the fourteen disks would give me 400 real GB (1024^3) of holding disk. I'd use RedHat Linux 7.3 for the OS. Main reason for picking Dell and RedHat over other PC/Linux options is just that they're the standard for my group these days. For a tape library How much is cost vs. storage space an issue? If it matters a fair bit, you may want to think about going with an IDE solution (the crowd gasps). I recently bought one of these: http://www.siliconmechanics.com/sm-416r.php Mine is actually based on dual Xeons (rather than PIII). It's maxed out with 16 x 160GB 5400RPM Maxtor disks, 2 3ware 7500-8 controllers, and 4GB of memory, which all cost $11K back in August. Doing a RAID5 w/ hot spare on each controller, that gets me 1.92TB of available disk space, amounting to $5.72/MB. The SCSI solution above (based on the pricing I get from Dell) costs $19.32/MB. You could easily scale back the IDE solution -- PIIIs rather than Xeons, less memory, smaller disks -- and get the absolute cost under the SCSI system and still get far more storage. Or, max it out and use the storage for other stuff as well. Note that the 3ware controllers and the chassis do support hot swap. And, if you're worried about speed, my system gets (doing a software RAID0 stripe across the two controllers) 105MB/s writing and 330MB/s reading (bonnie++). :) I sent this off list since it has less to do with amanda than with disk storage. Unfortunately, I don't have any experience with libraries, although given the system above, another 1TB system for another group, and our first .56TB RAID, I'm looking at AIT3 based libraries as well (I have an AIT1 drive and it's worked rather well). If you didn't want to hear this, sorry for the unsolicited, unrelated advice. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Fwd: Re: Fwd: help for compiling amanda to support xfsdump in RH 7.3
Hi,
Haven't got a solution for this issue...help...!!!
Any feedback would be really appreciated!!!
Thanks a lot,
Zhen Liu
Via Webmail
[EMAIL PROTECTED]
-- Forwarded message ---
From: Zhen Liu [EMAIL PROTECTED]
To: Joshua Baker-LePain [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Date: Thu, 10 Oct 2002 10:26:39 -0500
Hi, Joshua:
Thanks for your quick response...
Here is my ls -l /sbin/xfsdump:
-rwxr-xr-x1 root root 259020 Apr 16 10:17 /sbin/xfsdump
And when I compile the amanda client, my ./configure command are as
follows: ./config --with-user=amanda --with-group=disk
--with-amandahosts --with-config=daily --with-gnutar=/bin/tar
--with-tape-server=ourbackupserver.edu --with-configdir=/etc/amanda
--without-server --with-index-server=ourbackupserver.edu
So, the output of ./configure is below:
loading cache ./config.cache
checking host system type... i686-pc-linux-gnu
checking target system type... i686-pc-linux-gnu
checking build system type... i686-pc-linux-gnu
checking cached system tuple... ok
checking for a BSD compatible install... (cached) /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... (cached) gawk
checking whether make sets ${MAKE}... (cached) yes
checking for non-rewinding tape device... /dev/null
checking for raw ftape device... /dev/null
checking for Kerberos and Amanda kerberos4 bits... no
checking for gcc... (cached) gcc
checking whether the C compiler (gcc ) works... yes
checking whether the C compiler (gcc ) is a cross-compiler... no
checking whether we are using GNU C... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for style of include used by make... GNU
checking dependency style of gcc... (cached) gcc3
checking for object suffix... (cached) o
checking for Cygwin environment... (cached) no
checking for mingw32 environment... (cached) no
checking for executable suffix... (cached) no
checking for ar... (cached) /usr/bin/ar
checking for gawk... (cached) gawk
checking for gawk command line variable assignment... (cached) yes
with -v
checking for bison... (cached) bison -y
checking for cat... (cached) /bin/cat
checking for compress... (cached) /usr/bin/compress
checking for dd... (cached) /bin/dd
checking for egrep... (cached) /bin/egrep
checking for getconf... (cached) /usr/bin/getconf
checking for gnuplot... (cached) /usr/bin/gnuplot
checking for grep... (cached) /bin/grep
checking for gtar... (cached) /bin/tar
checking for smbclient... (cached) /usr/bin/smbclient
checking for gzip... (cached) /bin/gzip
checking for Mail... (cached) /usr/bin/Mail
checking for mt... (cached) /bin/mt
checking for chio... no
checking for chs... no
checking for mtx... (cached) /usr/sbin/mtx
checking for lpr... (cached) /usr/bin/lpr
checking which flag to use to select a printer... (cached) -P
checking for pcat... no
checking for perl5... (cached) /usr/bin/perl
checking for sh... (cached) /bin/sh
checking for ufsdump... (cached) /sbin/dump
checking for ufsrestore... (cached) /sbin/restore
checking whether /sbin/dump supports -E or -S for estimates...
(cached) S
checking for xfsdump... (cached) /sbin/xfsdump
checking for xfsrestore... (cached) /sbin/xfsrestore
checking for vxdump... no
checking for vxrestore... no
checking for vdump... no
checking for vrestore... no
checking for large file compilation CFLAGS... (cached)
-D_FILE_OFFSET_BITS=64
checking for large file compilation LDFLAGS... (cached)
checking for large file compilation LIBS... (cached)
checking how to run the C preprocessor... (cached) gcc -E
checking for ld used by GCC... (cached) /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... (cached) yes
checking for /usr/bin/ld option to reload object files... (cached) -r
checking for BSD-compatible nm... (cached) /usr/bin/nm -B
checking whether ln -s works... (cached) yes
checking how to recognise dependant libraries... (cached) pass_all
checking command to parse /usr/bin/nm -B output... (cached) ok
checking for dlfcn.h... (cached) yes
checking for ranlib... (cached) ranlib
checking for strip... (cached) strip
checking for objdir... .libs
checking for gcc option to produce PIC... (cached) -fPIC
checking if gcc PIC flag -fPIC works... (cached) yes
checking if gcc static flag -static works... (cached) yes
checking if gcc supports -c -o file.o... (cached) yes
checking if gcc supports -c -o file.lo... (cached) yes
checking if gcc supports -fno-rtti -fno-exceptions... yes
checking whether the linker (/usr/bin/ld) supports shared libraries...
yes
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
creating libtool
checking whether gcc needs -traditional... (cached) no
checking for working const... (cached) yes
checking for
chg-scsi: dev and scsitapedev on IRIX
Hello folks, What would be the difference between dev and scsitapedev? Would dev be the tape device name and scsitapedev be the scsi controller device name? I'm trying to use the chg-scsi from amanda-2.4.3 and am having no success. My chg-scsi.conf works fine with amanda-2.4.2. Would someone who has a working config on IRIX show me their config file please? Cheers, Luc.
Re: chg-scsi: dev and scsitapedev on IRIX
Luc Lalonde writes: - What would be the difference between dev and scsitapedev? Would - dev be the tape device name and scsitapedev be the scsi controller - device name? dev is the blocking tape device name (/dev/nrst0), while scstapedev is the non blocking (control) device name (/dev/enrst0 on NetBSD, I'm not sure there is something comparable on IRIX, there wasn't the last time I plaed with it, circa 1995) Try leaving scsitapedev unset. -- Eric Schnoebelen[EMAIL PROTECTED] http://www.cirr.com Beneath revolution (Linux) there's also evolution (*BSD) -- Andre Oppermann, Apr. 1998
Re: Fwd: Re: Fwd: help for compiling amanda to support xfsdump in RH 7.3
On Fri, Oct 11, 2002 at 04:01:06PM -0500, Zhen Liu wrote: Hi, Haven't got a solution for this issue...help...!!! Any feedback would be really appreciated!!! When you don't get a response from a group as helpful as this one it generally means no one had anything to contribute. In that case, publishing the same query with the same info is likely to elicit the same no response. A better approach is to ask the question differently, with different information, or a different question. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
chg-scsi and tape eject
I've noticed chg-scsi unloads ejected tape in the first available slot (no matter what startuse/enduse setup for the pool). Is it bug or feature ?
