Re: transferring Amanda database from one server to another
On 5/22/13 2:57 PM, Charles Curley wrote: On Wed, 22 May 2013 14:21:31 -0400 Chris Hoogendyk wrote: However, in this situation, what I'm trying to cope with is a *merge*. The transition from one server to the other was not an abrupt total switchover. Both servers were running. I got Amanda 3.3.3 installed and working for the server itself. Then I moved a couple of clients as a test. Then I moved the tape library and the remaining clients. So, the Amanda database on the new server has some unique data that it has created, and the Amanda database on the old server has the longer history. I'm trying to figure out how to merge the longer history from the old server onto the new server without stepping on the new information that is unique to the backups that have been done on the new server. Ah, sorry, I wasn't clear. I figured you could do your metadata backup, using your script or mine. Then you could have two configurations, one of which points to the new and ongoing data/metadata, the other of which points to the old data/metadata. This is not as elegant as doing a complete merge, which is what you asked for. On the other tentacle, it's probably a lot less work. And, Murphy willing, you won't need it. Once gotcha is that the metadata format may have changed from 2.5.1p3 to 3.3.3. I suspect other people may be interested in your results. Debian 7 (Wheezy [*Where* do the get these code names?]) was released early in May, with Amanda 3.3.1-4 on it. This replaces 6, (Squeeze), with Amanda 2.6.1p2-3. Debian admins tend to be a cautious lot, but I expect we'll see migrations over the next few months. I'm already running a 2.6.1p2 server with several 3.3.1 clients, and no problems so far. I should do some test restores, though. :-) I've typically done my Amanda builds from source rather than use what is packaged with the distribution. Partly that is coming from Solaris. But, my first Ubuntu systems were clients, and I didn't want to try backing up a client with a newer Amanda than the server. So I just took the source tar.gz from the server and built it on the Ubuntu client. Mostly, on Ubuntu, I have been sticking with the packages and aptitude. What's fun is dealing with custom configurations with upstart and apparmor (e.g. multiple instances of MySQL). Same order of fun as dealing with Solaris SMF. And then jumping back and forth between them. Recently, I jumped to Amanda 3.3.3 on my primary T5220 Solaris 10 servers. That worked nicely with the 2.5.1p3 clients. When I made the switch in another department from the E250 Amanda server to the SuperMicro Amanda server, I also jumped to Amanda 3.3.3. It again worked nicely with the 2.5.1p3 clients. Then I bumped a test client to 3.3.3 (also SuperMicro with Ubuntu). That went smoothly as well. So, I'll probably work towards building 3.3.3 for all the clients and then my other Amanda installations as well. -- --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology & Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst --- Erdös 4
Re: transferring Amanda database from one server to another
On Wed, 22 May 2013 14:21:31 -0400 Chris Hoogendyk wrote: > However, in this situation, what I'm trying to cope with is a > *merge*. The transition from one server to the other was not an > abrupt total switchover. Both servers were running. I got Amanda > 3.3.3 installed and working for the server itself. Then I moved a > couple of clients as a test. Then I moved the tape library and the > remaining clients. So, the Amanda database on the new server has some > unique data that it has created, and the Amanda database on the old > server has the longer history. I'm trying to figure out how to merge > the longer history from the old server onto the new server without > stepping on the new information that is unique to the backups that > have been done on the new server. Ah, sorry, I wasn't clear. I figured you could do your metadata backup, using your script or mine. Then you could have two configurations, one of which points to the new and ongoing data/metadata, the other of which points to the old data/metadata. This is not as elegant as doing a complete merge, which is what you asked for. On the other tentacle, it's probably a lot less work. And, Murphy willing, you won't need it. Once gotcha is that the metadata format may have changed from 2.5.1p3 to 3.3.3. I suspect other people may be interested in your results. Debian 7 (Wheezy [*Where* do the get these code names?]) was released early in May, with Amanda 3.3.1-4 on it. This replaces 6, (Squeeze), with Amanda 2.6.1p2-3. Debian admins tend to be a cautious lot, but I expect we'll see migrations over the next few months. I'm already running a 2.6.1p2 server with several 3.3.1 clients, and no problems so far. I should do some test restores, though. :-) -- Charles Curley /"\ASCII Ribbon Campaign Looking for fine software \ /Respect for open standards and/or writing? X No HTML/RTF in email http://www.charlescurley.com/ \No M$ Word docs in email Key fingerprint = CE5C 6645 A45A 64E4 94C0 809C FFF6 4C48 4ECD DFDB
Re: Fw: transferring Amanda database from one server to another
I actually do something very similar to that on all my Amanda servers. My crontab entries for amdump look like: 45 22 * * 1-5 su - amandabackup -c "/usr/local/sbin/amdump daily" > /dev/null; /usr/local/bin/amchanger daily unload; /usr/local/adm/backup/daily-amanda-con fig 45 0 * * 0-1 su - amandabackup -c "/usr/local/sbin/amdump --no-taper daily" > /dev/null; /usr/local/adm/backup/daily-amanda-config The script daily-amanda-config backs up all the configuration directories into a set of day name directories on a different drive that get cycled weekly. The script then tars up that day name directory and ships it over to another server. So, it is doubly backed up in case a drive fails or in case the whole server goes belly up. I think that script is in the amanda users list archives. That intermediate script is a simple one I wrote to unload the tape so that it is not sitting in the drive all day. We had some serious power outages one summer, and if the library lost power with a tape in the drive, there was a good chance it would jam the tape and ruin it. That also required taking out the drive and manually winding out the tape. Unloading the tape significantly reduced the risk of that occurring. However, in this situation, what I'm trying to cope with is a *merge*. The transition from one server to the other was not an abrupt total switchover. Both servers were running. I got Amanda 3.3.3 installed and working for the server itself. Then I moved a couple of clients as a test. Then I moved the tape library and the remaining clients. So, the Amanda database on the new server has some unique data that it has created, and the Amanda database on the old server has the longer history. I'm trying to figure out how to merge the longer history from the old server onto the new server without stepping on the new information that is unique to the backups that have been done on the new server. Chris On 5/22/13 1:40 PM, Charles Curley wrote: Forward to the list because I didn't check the "to:" line when I replied. Begin forwarded message: Date: Wed, 22 May 2013 10:58:48 -0600 From: Charles Curley To: Chris Hoogendyk Subject: Re: transferring Amanda database from one server to another On Wed, 22 May 2013 12:08:08 -0400 Chris Hoogendyk wrote: Is there a simple clean way of transferring that information from the E250 without stepping on the Amanda database entries that have been accumulated already on the SuperMicro? I faced a similar problem in making offsite backups of my Amanda data. See http://charlescurley.com/blog/articles/backing_up_amanda_metadata/index.html I doubt you need the scripts, but that should tell you everything you need to preserve. I believe most (all?) of the directories you need to preserve are specified in amanda.conf. -- --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology & Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst --- Erdös 4
Re: transferring Amanda database from one server to another
On 05/22/2013 01:08 PM, Chris Hoogendyk wrote: The index doesn't seem to be all that is needed. I looked at one of the index files, and it contained all the items in that backup for that dle, but it didn't indicate what tape it went to. That seems to be in the logs. I think I can see how to merge the index directories, because the indexes are all date/time/dle stamped. So, if I switched a client from the old server to the new server on, say May 16, then it's older indexes would be on the old server and it's newer ones would be on the new server. I could copy the old indexes over to the new server without stepping on anything. However, the log files don't seem to be so neatly segmented, and I'm not sure how I would merge them. Copy them all. I would love to do a complete merge for another reason. While I moved all the clients to the new server yesterday, I commented out many of the dle's to avoid having a complete set of full backups done all at once. I expected to add a portion of the dle's each day through the end of this week. However, if I could do a complete merge of the Amanda databases from the old server to the new server, then the planner would have that information and would take care of everything for me. You kind of get spoiled using Amanda and end up expecting that sort of thing. ;-) You should have copied the infodir (amgetconf CONFIG infofile). Some application use statefile on the client (gnutar-listdir), you should moved them if you changed the path. I think that's all you need: tapelist log files index directories info directories client state file Jean-Louis On 5/22/13 12:47 PM, Brian Cuttler wrote: Chris, You saved the index files? Just copy them to a separate directory and search them manually if necessary, or copy the entire amanda structure but give it a config name that is different than the functional one on the new server. On Wed, May 22, 2013 at 12:08:08PM -0400, Chris Hoogendyk wrote: With luck I won't need this, but one doesn't assume luck when planning backup systems. ;-) I've just successfully transferred a tape library from an old Sun E250 running Solaris 9 and Amanda 2.5.1p3 to a new SuperMicro running Ubuntu 12.04 LTS and Amanda 3.3.3. This has been a somewhat slow transition. We have moved that departments web services to the new server and set up Amanda on it. It was backing itself up, and then I moved a few other servers to it late last week. This was being held on a sizable pair of holding disks. The original E250 was still backing up a few other servers and itself. Then yesterday, after the Monday night backups flushed the weekend backups out to tape on the E250, we moved the tape library and switched all the servers to being backed up by the new server. That ran last night, and the tape library is working just fine. After transferring the tape library, I copied over /usr/local/amanda/etc/daily/tapelist so that the new server would continue with the correct tape sequence. That seems to have worked just fine. After this has been running for 6 weeks, all the tapes will have been cycled around and the Amanda database will reflect only what has been backed up on the new server. However, between now and then, if I need to recover something that was backed up on the E250, I have the tape library and tapes on the SuperMicro, but the Amanda database is on the E250 for part of the backups. Is there a simple clean way of transferring that information from the E250 without stepping on the Amanda database entries that have been accumulated already on the SuperMicro? TIA
Re: transferring Amanda database from one server to another
The index doesn't seem to be all that is needed. I looked at one of the index files, and it contained all the items in that backup for that dle, but it didn't indicate what tape it went to. That seems to be in the logs. I think I can see how to merge the index directories, because the indexes are all date/time/dle stamped. So, if I switched a client from the old server to the new server on, say May 16, then it's older indexes would be on the old server and it's newer ones would be on the new server. I could copy the old indexes over to the new server without stepping on anything. However, the log files don't seem to be so neatly segmented, and I'm not sure how I would merge them. I would love to do a complete merge for another reason. While I moved all the clients to the new server yesterday, I commented out many of the dle's to avoid having a complete set of full backups done all at once. I expected to add a portion of the dle's each day through the end of this week. However, if I could do a complete merge of the Amanda databases from the old server to the new server, then the planner would have that information and would take care of everything for me. You kind of get spoiled using Amanda and end up expecting that sort of thing. ;-) On 5/22/13 12:47 PM, Brian Cuttler wrote: Chris, You saved the index files? Just copy them to a separate directory and search them manually if necessary, or copy the entire amanda structure but give it a config name that is different than the functional one on the new server. On Wed, May 22, 2013 at 12:08:08PM -0400, Chris Hoogendyk wrote: With luck I won't need this, but one doesn't assume luck when planning backup systems. ;-) I've just successfully transferred a tape library from an old Sun E250 running Solaris 9 and Amanda 2.5.1p3 to a new SuperMicro running Ubuntu 12.04 LTS and Amanda 3.3.3. This has been a somewhat slow transition. We have moved that departments web services to the new server and set up Amanda on it. It was backing itself up, and then I moved a few other servers to it late last week. This was being held on a sizable pair of holding disks. The original E250 was still backing up a few other servers and itself. Then yesterday, after the Monday night backups flushed the weekend backups out to tape on the E250, we moved the tape library and switched all the servers to being backed up by the new server. That ran last night, and the tape library is working just fine. After transferring the tape library, I copied over /usr/local/amanda/etc/daily/tapelist so that the new server would continue with the correct tape sequence. That seems to have worked just fine. After this has been running for 6 weeks, all the tapes will have been cycled around and the Amanda database will reflect only what has been backed up on the new server. However, between now and then, if I need to recover something that was backed up on the E250, I have the tape library and tapes on the SuperMicro, but the Amanda database is on the E250 for part of the backups. Is there a simple clean way of transferring that information from the E250 without stepping on the Amanda database entries that have been accumulated already on the SuperMicro? TIA -- --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology & Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst --- Erdös 4
transferring Amanda database from one server to another
With luck I won't need this, but one doesn't assume luck when planning backup systems. ;-) I've just successfully transferred a tape library from an old Sun E250 running Solaris 9 and Amanda 2.5.1p3 to a new SuperMicro running Ubuntu 12.04 LTS and Amanda 3.3.3. This has been a somewhat slow transition. We have moved that departments web services to the new server and set up Amanda on it. It was backing itself up, and then I moved a few other servers to it late last week. This was being held on a sizable pair of holding disks. The original E250 was still backing up a few other servers and itself. Then yesterday, after the Monday night backups flushed the weekend backups out to tape on the E250, we moved the tape library and switched all the servers to being backed up by the new server. That ran last night, and the tape library is working just fine. After transferring the tape library, I copied over /usr/local/amanda/etc/daily/tapelist so that the new server would continue with the correct tape sequence. That seems to have worked just fine. After this has been running for 6 weeks, all the tapes will have been cycled around and the Amanda database will reflect only what has been backed up on the new server. However, between now and then, if I need to recover something that was backed up on the E250, I have the tape library and tapes on the SuperMicro, but the Amanda database is on the E250 for part of the backups. Is there a simple clean way of transferring that information from the E250 without stepping on the Amanda database entries that have been accumulated already on the SuperMicro? TIA -- --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology & Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst --- Erdös 4
