Re: [CentOS] Best way to duplicate a live Centos 5 server?
On Tue, Jul 10, 2012 at 11:59 PM, Emmanuel Noobadmin centos.ad...@gmail.com wrote: On 7/11/12, Les Mikesell lesmikes...@gmail.com wrote: Hours? This should happen in the time it takes to transfer a directory listing and read through it unless you used --ignore-times in the arguments. If you have many millions of files or not enough RAM to hold the list I suppose it could take hours. Not that many files definitely, more in the range of tens of thousands. But definitely more than an hour or two with small bursts of network traffic. Perhaps you have some very large files with small changes then (mailboxes, logfiles, db's, etc.). In that case the receiving rsync spends a lot of time copying the previous version of the file in addition to merging the changed bits. Rear 'might' be quick and easy. It is intended to be almost unattended and do everything for you. As for extra software - it is a 'yum install' from EPEL. The down side is that if it doesn't work, it isn't very well documented to help figure out how to fix it.I'd still recommend looking at it as a backup/restore solution with an option to clone. With a minimum amount of fiddling you can get it to generate a boot iso image that will re-create the source filesystem layout and bring up the network. Then, if you didn't want to let it handle the backup/restore part you could manually rsync to it from the live system. I'll look into it when I need to do this again. It just isn't something I expect to do with any regularity and unfortunately server admin isn't what directly goes into my salary so it has to take a second priority. ReaR's (Relax and Restore) real purpose is to be a full-auto restore to the existing hardware after replacing disks, etc., something that is relatively hard to do with complex filesystem layouts (lvm, raid, etc.) and something armchair sysadmins are likely to need when they least expect it. It does that function pretty well with a couple of lines of config setup (point to an NFS share to hold the backup) for anything where live tar backups are likely to work. The whole point of the tool is that you don't need to know what it is doing and pretty much anyone could do the restore on bare metal. Using it to clone or to move to a modified layout is sort of an afterthought at this point but it is still not unreasonable - it is just a bunch of shell scripts wrapping the native tools from the system but you have to figure out the content of the files where it stores the layout to build. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 7/9/12, Les Mikesell lesmikes...@gmail.com wrote: One thing that helps is to break it up into separate runs, at least per-filesystem and perhaps some of the larger subdirectories. Depending on the circumstances, you might be able to do an initial run ahead of time when speed doesn't matter so much, then just before the cutover shut down the services that will be changing files and databases and do a final rsync which will go much faster. I did try this but the time taken is pretty similar in the main delay is the part where rsync goes through all the files and spend a few hours trying to figure out what needs to be the updated on the second run after I shutdown the services. In hindsight, I might had been able to speed up things up considerably if I had generated a file list based on last modified time and passed it to rsync via the exclude/include parameters. Also, have you looked at clonezilla and ReaR? Yes, but due to time constraints, I figured it was safer to go with something simpler that I didn't have to learn as I go and could be done live without needed extra hardware on site. Plus it would be something that works at any site I needed it without extra software too. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 7/10/12, aurfalien aurfal...@gmail.com wrote: I do dump/restores fir this sort of thing. Thanks for this, I didn't know there was such a command until now! But it looks like it should work for me since bulk of the data are usually in /home which is a separate fs/mount usually. I can always resize the fs after transfer so I'll give this a try the next time I need to do a dup/migrate. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 07/09/12 11:39 PM, Emmanuel Noobadmin wrote: On 7/10/12, aurfalien aurfal...@gmail.com wrote: I do dump/restores fir this sort of thing. Thanks for this, I didn't know there was such a command until now! But it looks like it should work for me since bulk of the data are usually in /home which is a separate fs/mount usually. I can always resize the fs after transfer so I'll give this a try the next time I need to do a dup/migrate. dump should not be used on mounted file systems, except / in single user. restore can restore to any size file system of the same type (ext3, ext4) thats large enough to hold the files dumped. -- john r pierceN 37, W 122 santa cruz ca mid-left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 7/10/12, John R Pierce pie...@hogranch.com wrote: dump should not be used on mounted file systems, except / in single user. Aha, thanks for the warning! ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 07/10/12 12:06 AM, Emmanuel Noobadmin wrote: On 7/10/12, John R Pierce pie...@hogranch.com wrote: dump should not be used on mounted file systems, except / in single user. Aha, thanks for the warning! IF you're using LVM, you can take a file system snapshot, and dump the snapshot, however, as this is a point-in-time replica of the file system. -- john r pierceN 37, W 122 santa cruz ca mid-left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On Tue, Jul 10, 2012 at 1:36 AM, Emmanuel Noobadmin centos.ad...@gmail.com wrote: On 7/9/12, Les Mikesell lesmikes...@gmail.com wrote: One thing that helps is to break it up into separate runs, at least per-filesystem and perhaps some of the larger subdirectories. Depending on the circumstances, you might be able to do an initial run ahead of time when speed doesn't matter so much, then just before the cutover shut down the services that will be changing files and databases and do a final rsync which will go much faster. I did try this but the time taken is pretty similar in the main delay is the part where rsync goes through all the files and spend a few hours trying to figure out what needs to be the updated on the second run after I shutdown the services. In hindsight, I might had been able to speed up things up considerably if I had generated a file list based on last modified time and passed it to rsync via the exclude/include parameters. Hours? This should happen in the time it takes to transfer a directory listing and read through it unless you used --ignore-times in the arguments. If you have many millions of files or not enough RAM to hold the list I suppose it could take hours. Also, have you looked at clonezilla and ReaR? Yes, but due to time constraints, I figured it was safer to go with something simpler that I didn't have to learn as I go and could be done live without needed extra hardware on site. Plus it would be something that works at any site I needed it without extra software too. Rear 'might' be quick and easy. It is intended to be almost unattended and do everything for you. As for extra software - it is a 'yum install' from EPEL. The down side is that if it doesn't work, it isn't very well documented to help figure out how to fix it.I'd still recommend looking at it as a backup/restore solution with an option to clone. With a minimum amount of fiddling you can get it to generate a boot iso image that will re-create the source filesystem layout and bring up the network. Then, if you didn't want to let it handle the backup/restore part you could manually rsync to it from the live system. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 7/11/12, Les Mikesell lesmikes...@gmail.com wrote: Hours? This should happen in the time it takes to transfer a directory listing and read through it unless you used --ignore-times in the arguments. If you have many millions of files or not enough RAM to hold the list I suppose it could take hours. Not that many files definitely, more in the range of tens of thousands. But definitely more than an hour or two with small bursts of network traffic. Rear 'might' be quick and easy. It is intended to be almost unattended and do everything for you. As for extra software - it is a 'yum install' from EPEL. The down side is that if it doesn't work, it isn't very well documented to help figure out how to fix it.I'd still recommend looking at it as a backup/restore solution with an option to clone. With a minimum amount of fiddling you can get it to generate a boot iso image that will re-create the source filesystem layout and bring up the network. Then, if you didn't want to let it handle the backup/restore part you could manually rsync to it from the live system. I'll look into it when I need to do this again. It just isn't something I expect to do with any regularity and unfortunately server admin isn't what directly goes into my salary so it has to take a second priority. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
Phil Savoie wrote: On 07/08/2012 06:48 PM, Micky wrote: The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. It works always if you know how it's done. If you need detailed instructions, I can send you that! Yes, please! Could you either post here to the list, or to me personally? The list, if you please. (A link will do.) Thx T. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On Mon, Jul 9, 2012 at 12:34 AM, Emmanuel Noobadmin centos.ad...@gmail.com wrote: On 7/9/12, Micky mickylmar...@gmail.com wrote: The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. It works always if you know how it's done. The problem I found with rsync is that it is very slow when there are a lot of small files. Any idea how this can be improved on or is that a fundamental limit? One thing that helps is to break it up into separate runs, at least per-filesystem and perhaps some of the larger subdirectories. Depending on the circumstances, you might be able to do an initial run ahead of time when speed doesn't matter so much, then just before the cutover shut down the services that will be changing files and databases and do a final rsync which will go much faster. Also, have you looked at clonezilla and ReaR? -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On Jul 8, 2012, at 10:34 PM, Emmanuel Noobadmin wrote: On 7/9/12, Micky mickylmar...@gmail.com wrote: The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. It works always if you know how it's done. The problem I found with rsync is that it is very slow when there are a lot of small files. Any idea how this can be improved on or is that a fundamental limit? I do dump/restores fir this sort of thing. - aurf ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
aurfalien wrote: On Jul 8, 2012, at 10:34 PM, Emmanuel Noobadmin wrote: On 7/9/12, Micky mickylmar...@gmail.com wrote: The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. It works always if you know how it's done. The problem I found with rsync is that it is very slow when there are a lot of small files. Any idea how this can be improved on or is that a fundamental limit? I do dump/restores fir this sort of thing. Depends on if you want everything, and of course, if there's a hardware difference, you need to chroot (assuming you rsync'd to special directories, like /new and /boot/new), and do some mounts and chroot, and rebuild the initrd.img mark ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On Mon, 2012-07-09 at 09:00 -0700, aurfalien wrote: On Jul 8, 2012, at 10:34 PM, Emmanuel Noobadmin wrote: On 7/9/12, Micky mickylmar...@gmail.com wrote: The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. It works always if you know how it's done. The problem I found with rsync is that it is very slow when there are a lot of small files. Any idea how this can be improved on or is that a fundamental limit? I do dump/restores fir this sort of thing. +1 John ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. It works always if you know how it's done. If you need detailed instructions, I can send you that! On Tue, Jun 19, 2012 at 10:44 AM, Les Mikesell lesmikes...@gmail.comwrote: On Tue, Jun 19, 2012 at 12:03 PM, Cal Sawyer ca...@blue-bolt.com wrote: You're right - documentation is pretty dire. Guess i'm not alone in hating doing it. Yes, I really, really wish the stuff they are doing was documented, somewhere, anywhere. Not just how to use the program itself which is supposed to 'just work' unattended once you set it up, but the black magic of how they detect and reproduce all of the hardware, lvm, raid, filesystem, etc. across different distributions and versions. USB backup is broken due to the order in which path variables get set - sure is lot of fun trawling through the scripts to find out what gets set when. Hope the ReaR maintainers are interested in this and haven't gotten themselves mired in tape archive integration - i would have thought USB backup was the winner in terms of getting broad acceptance as a bare-metal DR solution. USB is sort of 'hands-on' for something that should be unattended, and adds a lot of unpredictable messiness in drive detection, boot order, etc.All you really have to do is export some NFS space and point ReaR to it. At least that is the easy way to get started. If you have another Linux box, just plug your USB drive in there and access it over NFS... problem solved. Clonezilla-live plays in this space too, but it doesn't do raid or multiple disks at once, and you have to shut down to take the image. My 'ideal' system would be to have ReaR generate a directory of what will be on the boot iso leaving that somewhere on the host without actually making the image. Then use backuppc to back up the whole host and its normal duplicate-pooling mechanism will keep the extra copies of the tools from taking extra space. Then when/if you need a bare-metal restore, you would first grab the directory of the iso contents, burn a boot CD, let that reconstruct the filesystem, then tell backuppc to restore to it. That way would be completely automatic and always be up to date, with the advantage of backuppc's efficient storage and easy online access to individual files and directories. If you don't mind wasting a small amount of space for the isos, I think that approach would already work if you tell ReaR to just make the boot image and to wait for an external program to do the restore after the filesystem has been rebuilt. However, when it works - wow. Just restored an HP dl360 w/HWRAID to a Presario desktop machine and it lives! No network, but that's small beans compared to the larger win. Yes, I've even modified the filesystem layout file to go from a software RAID to a non-RAID, and to change partition sizes during the restores. If it was documented, that capability by itself would be fantastic. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 07/08/2012 06:48 PM, Micky wrote: The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. It works always if you know how it's done. If you need detailed instructions, I can send you that! Yes, please! Could you either post here to the list, or to me personally? Thank you, Phil -- Carpe Aptenodytes! (Seize the Penguins!) ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On Jul 8, 2012, at 6:57 PM, Phil Savoie psavoie1...@rogers.com wrote: On 07/08/2012 06:48 PM, Micky wrote: The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. It works always if you know how it's done. If you need detailed instructions, I can send you that! Yes, please! Could you either post here to the list, or to me personally? Thank you, Phil -- What is running on the server? You might be able to get away with a dd, to build a duplicate disk. This disk can be directly attached or on another server tunneled through ssh. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 07/08/12 7:14 PM, Joseph Spenner wrote: What is running on the server? You might be able to get away with a dd, to build a duplicate disk. This disk can be directly attached or on another server tunneled through ssh. or setup a drbd replica, wait for it to replicate, then stop the replication. -- john r pierceN 37, W 122 santa cruz ca mid-left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 07/08/2012 10:14 PM, Joseph Spenner wrote: On Jul 8, 2012, at 6:57 PM, Phil Savoiepsavoie1...@rogers.com wrote: On 07/08/2012 06:48 PM, Micky wrote: The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. It works always if you know how it's done. If you need detailed instructions, I can send you that! Yes, please! Could you either post here to the list, or to me personally? Thank you, Phil -- What is running on the server? You might be able to get away with a dd, to build a duplicate disk. This disk can be directly attached or on another server tunneled through ssh. Centos 5.8 and Centos 6.2 servers. A duplicate disk is not what I am after as I cannot always replace with exact drives, i.e., same make, model, size, etc. But thank you anyway... Phil ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos Cen -- Carpe Aptenodytes! (Seize the Penguins!) ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 07/08/12 8:20 PM, Phil Savoie wrote: Centos 5.8 and Centos 6.2 servers. A duplicate disk is not what I am after as I cannot always replace with exact drives, i.e., same make, model, size, etc. note that there's a lot of things where file by file, or even sector by sector, duplicates are NOT valid if made while the system is online. for instance, relational databases such as PostgreSQL, Oracle, you can't just copy their files while the database server is running, as they rely on writes being made in a specific order. -- john r pierceN 37, W 122 santa cruz ca mid-left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 7/9/12, Micky mickylmar...@gmail.com wrote: The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. It works always if you know how it's done. The problem I found with rsync is that it is very slow when there are a lot of small files. Any idea how this can be improved on or is that a fundamental limit? ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 7/9/12, John R Pierce pie...@hogranch.com wrote: On 07/08/12 7:14 PM, Joseph Spenner wrote: What is running on the server? You might be able to get away with a dd, to build a duplicate disk. This disk can be directly attached or on another server tunneled through ssh. or setup a drbd replica, wait for it to replicate, then stop the replication. That requires drbd to be setup in advance doesn't it? I was trying this approach then ran into that wall. And given the amount of work required to get drbd working on a new setup, it seemed easier to use mdraid to do the same thing. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 9/7/2012 1:48 πμ, Micky wrote: The best and traditional way that has been there for decades is an rsync and then reinstallation of boot-loader. We are using mondorescue (mondoarchive and mondorestore). Works fine and supports many ways of archiving/restoring, LVM etc. I recommend it. Good both for backups and cloning. Nick ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
Subject: Re: [CentOS] Best way to duplicate a live Centos 5 server? To: CentOS mailing list centos@centos.org On Mon, Jun 18, 2012 at 11:21 AM, Cal Sawyer ca...@blue-bolt.com wrote: ReaR has suddenly become very interesting to me, probably explaining why it utterly fails to work properly (for me).I'm using 1.13 to pull a USB-based recovery image, but there's an error in the backup/NETFS/default/50_make_backup.sh script that doesn't mount the USB device after the mkrecovery step, so subsequent tar fails on write to the non-existent mountpoint. I fixed that, but on recovery it fails to mount the necessary directories on the restore drive as well, so rear recover quickly bombs out. Is anyone having any success actually using ReaR on CentOS 6.x? - csawyer It intentionally doesn't deal with drives the kernel has marked as removable. I had trouble with that with the main drives on a SAS hotswap backplane in an earlier version but I think that is fixed now. I'd recommend asking how to override this on the ReaR mail list. While the code seems usable (and yes, I have succeeded in using it on Centos 6x.), the documentation either doesn't exist yet or is very out of date. But, the authors are very responsive and it would be good to let them know about any bugs or usability problems. The mail list is still at sourceforge although the code has been moved to github and there is talk of moving the list elsewhere. -- Les Mikesell lesmikes...@gmail.com ReaR hasn't posed any insurmountable problems with USB removable media but i can see SATA/SAS needing some massaging in the device detection dep't. Yes, i've gotten myself on the ReaR mailing list now. You're right - documentation is pretty dire. Guess i'm not alone in hating doing it. USB backup is broken due to the order in which path variables get set - sure is lot of fun trawling through the scripts to find out what gets set when. Hope the ReaR maintainers are interested in this and haven't gotten themselves mired in tape archive integration - i would have thought USB backup was the winner in terms of getting broad acceptance as a bare-metal DR solution. However, when it works - wow. Just restored an HP dl360 w/HWRAID to a Presario desktop machine and it lives! No network, but that's small beans compared to the larger win. - csawyer ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On Tue, Jun 19, 2012 at 12:03 PM, Cal Sawyer ca...@blue-bolt.com wrote: You're right - documentation is pretty dire. Guess i'm not alone in hating doing it. Yes, I really, really wish the stuff they are doing was documented, somewhere, anywhere. Not just how to use the program itself which is supposed to 'just work' unattended once you set it up, but the black magic of how they detect and reproduce all of the hardware, lvm, raid, filesystem, etc. across different distributions and versions. USB backup is broken due to the order in which path variables get set - sure is lot of fun trawling through the scripts to find out what gets set when. Hope the ReaR maintainers are interested in this and haven't gotten themselves mired in tape archive integration - i would have thought USB backup was the winner in terms of getting broad acceptance as a bare-metal DR solution. USB is sort of 'hands-on' for something that should be unattended, and adds a lot of unpredictable messiness in drive detection, boot order, etc.All you really have to do is export some NFS space and point ReaR to it. At least that is the easy way to get started. If you have another Linux box, just plug your USB drive in there and access it over NFS... problem solved. Clonezilla-live plays in this space too, but it doesn't do raid or multiple disks at once, and you have to shut down to take the image. My 'ideal' system would be to have ReaR generate a directory of what will be on the boot iso leaving that somewhere on the host without actually making the image. Then use backuppc to back up the whole host and its normal duplicate-pooling mechanism will keep the extra copies of the tools from taking extra space. Then when/if you need a bare-metal restore, you would first grab the directory of the iso contents, burn a boot CD, let that reconstruct the filesystem, then tell backuppc to restore to it. That way would be completely automatic and always be up to date, with the advantage of backuppc's efficient storage and easy online access to individual files and directories. If you don't mind wasting a small amount of space for the isos, I think that approach would already work if you tell ReaR to just make the boot image and to wait for an external program to do the restore after the filesystem has been rebuilt. However, when it works - wow. Just restored an HP dl360 w/HWRAID to a Presario desktop machine and it lives! No network, but that's small beans compared to the larger win. Yes, I've even modified the filesystem layout file to go from a software RAID to a non-RAID, and to change partition sizes during the restores. If it was documented, that capability by itself would be fantastic. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
From: Les Mikesell lesmikes...@gmail.com Subject: Re: [CentOS] Best way to duplicate a live Centos 5 server? To: CentOS mailing list centos@centos.org On Fri, Jun 8, 2012 at 12:04 PM, Scott Silva ssi...@sgvwater.com wrote: Am I missing something glaringly obvious here, or is the only way I'm going be able to migrate is to shutdown the C5 server for a few hours while duping the old drives? Would greatly appreciate any pointers how best to do this. You could always rsync the old server to the new one... a few runs will get 99% of the files, and a quick run after the shutdown can get the rest... Have a tar file ready of the needed config changes ready and untar it and start up the new system... An interesting variation on this is to use 'ReaR' to back up and restore the machine, essentially cloning it but give the copy a different IP address as you bring it up. Then when the clone is close to ready to take over, shut down your apps for the time it takes a final rsync to fix up the differences (in the data areas only - avoid /etc/, (etc.), then switch the IP. ReaR is in active development now and is very usable. It is a set of shell scripts designed to run live backups that are capable of restoring to bare metal. It makes a new boot iso with tools from the running system to reconstruct the filesystem (including lvm/raid, etc.) and restore on top of that. Several backup methods are supported but tar to an nfs location is probably the easiest to set up. With a small amount of extra work you can tweak the filesystem layout, etc. if you don't want an exact clone. With hardware differences you might need to tweak modules and build a new initrd, too. ReaR is packaged in EPEL as rear. ReaR has suddenly become very interesting to me, probably explaining why it utterly fails to work properly (for me).I'm using 1.13 to pull a USB-based recovery image, but there's an error in the backup/NETFS/default/50_make_backup.sh script that doesn't mount the USB device after the mkrecovery step, so subsequent tar fails on write to the non-existent mountpoint. I fixed that, but on recovery it fails to mount the necessary directories on the restore drive as well, so rear recover quickly bombs out. Is anyone having any success actually using ReaR on CentOS 6.x? - csawyer ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On Mon, Jun 18, 2012 at 11:21 AM, Cal Sawyer ca...@blue-bolt.com wrote: ReaR has suddenly become very interesting to me, probably explaining why it utterly fails to work properly (for me).I'm using 1.13 to pull a USB-based recovery image, but there's an error in the backup/NETFS/default/50_make_backup.sh script that doesn't mount the USB device after the mkrecovery step, so subsequent tar fails on write to the non-existent mountpoint. I fixed that, but on recovery it fails to mount the necessary directories on the restore drive as well, so rear recover quickly bombs out. Is anyone having any success actually using ReaR on CentOS 6.x? - csawyer It intentionally doesn't deal with drives the kernel has marked as removable. I had trouble with that with the main drives on a SAS hotswap backplane in an earlier version but I think that is fixed now. I'd recommend asking how to override this on the ReaR mail list. While the code seems usable (and yes, I have succeeded in using it on Centos 6x.), the documentation either doesn't exist yet or is very out of date. But, the authors are very responsive and it would be good to let them know about any bugs or usability problems. The mail list is still at sourceforge although the code has been moved to github and there is talk of moving the list elsewhere. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 6/14/12, Smithies, Russell russell.smith...@agresearch.co.nz wrote: How about using one of the backup tools to image the server? We use Symantec System Recovery and image all the disks. We then have the option of restoring to different hardware (physical or virtual) which works very well. There's a 60-day evaluation period. http://www.symantec.com/products/trialware.jsp?pcid=pcat_business_contpvid=1602_1 Not an option for me unfortunately, the only Windows systems on location are at best Win7 Home Premium and SSR requires a Win Server OS according to their page. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On Thu, Jun 14, 2012 at 2:32 AM, Emmanuel Noobadmin centos.ad...@gmail.com wrote: On 6/14/12, Smithies, Russell russell.smith...@agresearch.co.nz wrote: How about using one of the backup tools to image the server? We use Symantec System Recovery and image all the disks. We then have the option of restoring to different hardware (physical or virtual) which works very well. There's a 60-day evaluation period. http://www.symantec.com/products/trialware.jsp?pcid=pcat_business_contpvid=1602_1 Not an option for me unfortunately, the only Windows systems on location are at best Win7 Home Premium and SSR requires a Win Server OS according to their page. Clonezilla-live is good for straight image copies, but you have to shut down the source while taking the copy and it doesn't do raid. It does handle most filesystems including windows and knows enough to only copy the used blocks. ReaR will make the copy with the source running and handles most linux disk layouts. There is not much documentation at this point and there are a lot of options, but if you have an NFS share to hold the intermediate backup copy it only takes a couple of lines in a conf file to set it up. However, since it is designed for backup/restore, the default is for the restore iso to use the same IP as the source which is awkward for live cloning. You can work around that but should probably try a test system first. It is definitely worth looking at as a simple backup solution in any case. If the target hardware is different, both clonezilla and rear may require you to build a new initrd with appropriate disk drivers included. Using the VMware converter tool (free) might work. I've done it with windows, but so far it has not worked with the disk layouts on the linux systems I have tried. When it works, it works very well - and you could probably do additional conversions from the vmware image. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On 08/06/2012 17:33, Emmanuel Noobadmin wrote: I've got a CentOS 5 server that I want to migrate over into a virtualized instance. The problem is I need to minimize downtime so was trying to figure out a way to live clone the original. Initially, I thought I could do this via exporting an iSCSI target from the virtual host, create a MD raid 1 array on the C5 server, wait for it to sync, then shutdown the physical server and switch to the virtual one. But after getting iSCSI working... I realize I could not create a md device on a mounted disk. Unfortunately this old C5 wasn't setup with md raid 1 originally so I can't just add a the iSCSI target as an additional member for a triplicate. So I remembered DRBD was supposed to be used for replication. But after getting things set up, running the drbd-admin create-md command gave me this scary warning it will destroy data on the disk. Apparently because drbd writes meta data to the drive. So that appears to be a no go too. Am I missing something glaringly obvious here, or is the only way I'm going be able to migrate is to shutdown the C5 server for a few hours while duping the old drives? Would greatly appreciate any pointers how best to do this. You don't say what virtualisation platform you are using is, but if it's VMware, then you can use VMware converter to do the migration. This can, if you want, clone the physical computer into VMware, shut down the physical computer and bring up the new virtual instance. All whilst the physical remained up. I've used it for a few Linux boxes, where I've wanted a quick dev version of an existing server and its been fine. I guess, you could try pulling it into an ESXi host, and then exporting that in a format whatever virtualisation program it is you use supports... Regards, Tris * This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify postmas...@bgfl.org The views expressed within this email are those of the individual, and not necessarily those of the organisation * ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
I'm using KVM so didn't have the tool. While Les' suggestion looked like it was going to be pretty useful for a variety of backup/restore situations, I didn't know if I had the time to go through the docs and get things working in time. So in the end I went with the repeated rSync method Scott mentioned. The advantage is, I also went and made the new system C6 first, then rsync the necessary data files instead of leaving it still on C5. Thankfully nothing broke, well, except SSL certs for some reason but that was easily fixed once people started complaining. On 6/13/12, Tris Hoar trish...@bgfl.org wrote: On 08/06/2012 17:33, Emmanuel Noobadmin wrote: I've got a CentOS 5 server that I want to migrate over into a virtualized instance. The problem is I need to minimize downtime so was trying to figure out a way to live clone the original. Initially, I thought I could do this via exporting an iSCSI target from the virtual host, create a MD raid 1 array on the C5 server, wait for it to sync, then shutdown the physical server and switch to the virtual one. But after getting iSCSI working... I realize I could not create a md device on a mounted disk. Unfortunately this old C5 wasn't setup with md raid 1 originally so I can't just add a the iSCSI target as an additional member for a triplicate. So I remembered DRBD was supposed to be used for replication. But after getting things set up, running the drbd-admin create-md command gave me this scary warning it will destroy data on the disk. Apparently because drbd writes meta data to the drive. So that appears to be a no go too. Am I missing something glaringly obvious here, or is the only way I'm going be able to migrate is to shutdown the C5 server for a few hours while duping the old drives? Would greatly appreciate any pointers how best to do this. You don't say what virtualisation platform you are using is, but if it's VMware, then you can use VMware converter to do the migration. This can, if you want, clone the physical computer into VMware, shut down the physical computer and bring up the new virtual instance. All whilst the physical remained up. I've used it for a few Linux boxes, where I've wanted a quick dev version of an existing server and its been fine. I guess, you could try pulling it into an ESXi host, and then exporting that in a format whatever virtualisation program it is you use supports... Regards, Tris * This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify postmas...@bgfl.org The views expressed within this email are those of the individual, and not necessarily those of the organisation * ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
How about using one of the backup tools to image the server? We use Symantec System Recovery and image all the disks. We then have the option of restoring to different hardware (physical or virtual) which works very well. There's a 60-day evaluation period. http://www.symantec.com/products/trialware.jsp?pcid=pcat_business_contpvid=1602_1 --Russell -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Emmanuel Noobadmin Sent: Thursday, 14 June 2012 2:36 a.m. To: CentOS mailing list Subject: Re: [CentOS] Best way to duplicate a live Centos 5 server? I'm using KVM so didn't have the tool. While Les' suggestion looked like it was going to be pretty useful for a variety of backup/restore situations, I didn't know if I had the time to go through the docs and get things working in time. So in the end I went with the repeated rSync method Scott mentioned. The advantage is, I also went and made the new system C6 first, then rsync the necessary data files instead of leaving it still on C5. Thankfully nothing broke, well, except SSL certs for some reason but that was easily fixed once people started complaining. On 6/13/12, Tris Hoar trish...@bgfl.org wrote: On 08/06/2012 17:33, Emmanuel Noobadmin wrote: I've got a CentOS 5 server that I want to migrate over into a virtualized instance. The problem is I need to minimize downtime so was trying to figure out a way to live clone the original. Initially, I thought I could do this via exporting an iSCSI target from the virtual host, create a MD raid 1 array on the C5 server, wait for it to sync, then shutdown the physical server and switch to the virtual one. But after getting iSCSI working... I realize I could not create a md device on a mounted disk. Unfortunately this old C5 wasn't setup with md raid 1 originally so I can't just add a the iSCSI target as an additional member for a triplicate. So I remembered DRBD was supposed to be used for replication. But after getting things set up, running the drbd-admin create-md command gave me this scary warning it will destroy data on the disk. Apparently because drbd writes meta data to the drive. So that appears to be a no go too. Am I missing something glaringly obvious here, or is the only way I'm going be able to migrate is to shutdown the C5 server for a few hours while duping the old drives? Would greatly appreciate any pointers how best to do this. You don't say what virtualisation platform you are using is, but if it's VMware, then you can use VMware converter to do the migration. This can, if you want, clone the physical computer into VMware, shut down the physical computer and bring up the new virtual instance. All whilst the physical remained up. I've used it for a few Linux boxes, where I've wanted a quick dev version of an existing server and its been fine. I guess, you could try pulling it into an ESXi host, and then exporting that in a format whatever virtualisation program it is you use supports... Regards, Tris * This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify postmas...@bgfl.org The views expressed within this email are those of the individual, and not necessarily those of the organisation * ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
on 6/8/2012 9:33 AM Emmanuel Noobadmin spake the following: I've got a CentOS 5 server that I want to migrate over into a virtualized instance. The problem is I need to minimize downtime so was trying to figure out a way to live clone the original. Initially, I thought I could do this via exporting an iSCSI target from the virtual host, create a MD raid 1 array on the C5 server, wait for it to sync, then shutdown the physical server and switch to the virtual one. But after getting iSCSI working... I realize I could not create a md device on a mounted disk. Unfortunately this old C5 wasn't setup with md raid 1 originally so I can't just add a the iSCSI target as an additional member for a triplicate. So I remembered DRBD was supposed to be used for replication. But after getting things set up, running the drbd-admin create-md command gave me this scary warning it will destroy data on the disk. Apparently because drbd writes meta data to the drive. So that appears to be a no go too. Am I missing something glaringly obvious here, or is the only way I'm going be able to migrate is to shutdown the C5 server for a few hours while duping the old drives? Would greatly appreciate any pointers how best to do this. You could always rsync the old server to the new one... a few runs will get 99% of the files, and a quick run after the shutdown can get the rest... Have a tar file ready of the needed config changes ready and untar it and start up the new system... ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Best way to duplicate a live Centos 5 server?
On Fri, Jun 8, 2012 at 12:04 PM, Scott Silva ssi...@sgvwater.com wrote: Am I missing something glaringly obvious here, or is the only way I'm going be able to migrate is to shutdown the C5 server for a few hours while duping the old drives? Would greatly appreciate any pointers how best to do this. You could always rsync the old server to the new one... a few runs will get 99% of the files, and a quick run after the shutdown can get the rest... Have a tar file ready of the needed config changes ready and untar it and start up the new system... An interesting variation on this is to use 'ReaR' to back up and restore the machine, essentially cloning it but give the copy a different IP address as you bring it up. Then when the clone is close to ready to take over, shut down your apps for the time it takes a final rsync to fix up the differences (in the data areas only - avoid /etc/, (etc.), then switch the IP. ReaR is in active development now and is very usable. It is a set of shell scripts designed to run live backups that are capable of restoring to bare metal. It makes a new boot iso with tools from the running system to reconstruct the filesystem (including lvm/raid, etc.) and restore on top of that. Several backup methods are supported but tar to an nfs location is probably the easiest to set up. With a small amount of extra work you can tweak the filesystem layout, etc. if you don't want an exact clone. With hardware differences you might need to tweak modules and build a new initrd, too. ReaR is packaged in EPEL as rear. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
