Re: Moving server to new server with tar
Op Mon, 08 Jun 2015 16:06:41 -0600, schreef Bob Proulx: > Linux4Bene wrote: > Since /dev is dynamic anything done to it will evaporate after a reboot. > After a reboot it will all be as if nothing had been overwritten there. > If you get to the point of a reboot then there is nothing lingering > afterward. You would be in the clear. I would still exclude it of > course. The possibility of archiving /dev/mem for example makes me > nervous. :-) Hehe indeed. I will change the tar command to exclude it completely. > http://marc.merlins.org/perso/linux/post_2014-01-06_My-Live-Upgrading- >Many-Thousands-of-Servers-ProdNG-talk-at-Linux_conf_au-2014.html > > Unfortunately the original paper is now 403 forbidden. I think that is > likely a mistake somewhere. But the Internet Archive Wayback Machine > has a copy if you want to browse it. > > https://web.archive.org/web/*/http://marc.merlins.org/linux/talks >ProdNG-LCA2014/Paper/ProdNG.pdf Thanks, the last links you posted worked. I'm very interested in reading about it. > As I recall one spot I don't think was as complete was the kernel both > running and otherwise. I think debtakeover required that the new system > run on the foreign system's kernel. Which is not always possible due to > system version differences. I think that is more of a problem now than > it was then. So for example replacing a RHEL 6 system with Jessie would > fail because jessie binaries require a newer kernel than the default > RHEL 6 kernel. But probably upgrading the kernel first with a native > backport and then doing the debtakeover process would get past that > problem. It still seems like software that can be useful today or are there other preferred (manual) ways of doing such a conversion? >> When I rebooted the system, it failed because the UUID was still the >> UUID of the main disk of the old system. > > Ah... Probably should 'grep -r $UUID /etc' for every mention of it. Another useful comment, thanks. > Are you using lvm or raid? If either of those then would probably want > to *avoid* overwriting the /etc/lvm or /etc/mdadm directories. Both of > those configs keep UUIDs in them. No, it's a fairly simple VPS, no raid or LVM. >> Bob, thanks for your thorough explanation & insight. > > It is an interesting problem. I am deep in the middle of installation > and setup all of the time. > > Bob Sounds very challenging and interesting at the same time :) Benedict -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/ml62l8$d52$1...@ger.gmane.org
Re: Moving server to new server with tar
On Mon, 8 Jun 2015 16:49:45 -0600 Bob Proulx wrote: > Lisi Reisz wrote: > > Bob Proulx wrote: > > > Every file. File by file. I liked this presentation and found it > > > quite interesting. > > > > > > http://marc.merlins.org/perso/linux/post_2014-01-06_My-Live-Upgrading-Many-Thousands-of-Servers-ProdNG-talk-at-Linux_conf_au-2014.html > > That one definitely works for me at this moment. I just tested it and > got a page okay. Since that is the origin I wanted to quote it. > However the links from that page all give me 403 forbidden. Sigh. > > There is a video of the presentation somewhere. I don't have a link. > I imagine that would be on youtube and would still be available. > Maybe someone will find it and post a link to it. > > > > Unfortunately the original paper is now 403 forbidden. I think > > > that is likely a mistake somewhere. But the Internet Archive > > > Wayback Machine has a copy if you want to browse it. > > > > > > https://web.archive.org/web/*/http://marc.merlins.org/linux/talks/ProdNG-LCA2014/Paper/ProdNG.pdf > > > > I got 403 forbidden on both. :-( > > That second one is 403 for me now too. The '*' means grab the newest > version and it has updated to the 403 version. But looking through > the older snapshots I find the specific version is here: > > > https://web.archive.org/web/20141016050915/http://marc.merlins.org/linux/talks/ProdNG-LCA2014/Paper/ProdNG.pdf > > That should get you the paper at least. I also found this just now: > > > https://www.usenix.org/conference/lisa13/technical-sessions/presentation/merlin > > Bob I got them straight from here, maybe something was temporarily down? http://marc.merlins.org/linux/talks/ProdNG-LCA2014/html/ Petter -- "I'm ionized" "Are you sure?" "I'm positive." pgpBEawtjcuTh.pgp Description: OpenPGP digital signature
Re: Moving server to new server with tar
Quoting Lisi Reisz (lisi.re...@gmail.com): > On Monday 08 June 2015 23:06:41 Bob Proulx wrote: > > Every file. File by file. I liked this presentation and found it > > quite interesting. > > http://marc.merlins.org/perso/linux/post_2014-01-06_My-Live-Upgrading-Many- > >Thousands-of-Servers-ProdNG-talk-at-Linux_conf_au-2014.html > > > > Unfortunately the original paper is now 403 forbidden. I think that > > is likely a mistake somewhere. But the Internet Archive Wayback > > Machine has a copy if you want to browse it. > > https://web.archive.org/web/*/http://marc.merlins.org/linux/talks/ProdNG-LC > >A2014/Paper/ProdNG.pdf > > I got 403 forbidden on both. :-( Googling 13457-lisa13-merlin.pdf should find a downloadable pdf link. Googling live upgrade redhat debian youtube gives two videos of a lecture. I haven't yet worked out how I found and downloaded my shorter video of a similar lecture. Cheers, David. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150609003552.GA18474@alum
Re: Moving server to new server with tar
Lisi Reisz wrote: > Bob Proulx wrote: > > Every file. File by file. I liked this presentation and found it > > quite interesting. > > > > http://marc.merlins.org/perso/linux/post_2014-01-06_My-Live-Upgrading-Many-Thousands-of-Servers-ProdNG-talk-at-Linux_conf_au-2014.html That one definitely works for me at this moment. I just tested it and got a page okay. Since that is the origin I wanted to quote it. However the links from that page all give me 403 forbidden. Sigh. There is a video of the presentation somewhere. I don't have a link. I imagine that would be on youtube and would still be available. Maybe someone will find it and post a link to it. > > Unfortunately the original paper is now 403 forbidden. I think that > > is likely a mistake somewhere. But the Internet Archive Wayback > > Machine has a copy if you want to browse it. > > > > https://web.archive.org/web/*/http://marc.merlins.org/linux/talks/ProdNG-LCA2014/Paper/ProdNG.pdf > > I got 403 forbidden on both. :-( That second one is 403 for me now too. The '*' means grab the newest version and it has updated to the 403 version. But looking through the older snapshots I find the specific version is here: https://web.archive.org/web/20141016050915/http://marc.merlins.org/linux/talks/ProdNG-LCA2014/Paper/ProdNG.pdf That should get you the paper at least. I also found this just now: https://www.usenix.org/conference/lisa13/technical-sessions/presentation/merlin Bob signature.asc Description: Digital signature
Re: Moving server to new server with tar
On Monday 08 June 2015 23:06:41 Bob Proulx wrote: > Every file. File by file. I liked this presentation and found it > quite interesting. > > > http://marc.merlins.org/perso/linux/post_2014-01-06_My-Live-Upgrading-Many- >Thousands-of-Servers-ProdNG-talk-at-Linux_conf_au-2014.html > > Unfortunately the original paper is now 403 forbidden. I think that > is likely a mistake somewhere. But the Internet Archive Wayback > Machine has a copy if you want to browse it. > > > https://web.archive.org/web/*/http://marc.merlins.org/linux/talks/ProdNG-LC >A2014/Paper/ProdNG.pdf I got 403 forbidden on both. :-( Lisi -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/201506082324.18731.lisi.re...@gmail.com
Re: Moving server to new server with tar
Linux4Bene wrote: > schreef Bob Proulx: > thanks for your reply and the time invested. Much appreciated. > It does indeed seem tricky unless you go the full monty and replace the > whole installation except for the special dirs like dev as you noted. > In my test, I didn't get any strange results in the end but I have > learned not to always trust what I see in IT. In the back of my mind, I > always suspect a problem popping up when the server is in production. Since /dev is dynamic anything done to it will evaporate after a reboot. After a reboot it will all be as if nothing had been overwritten there. If you get to the point of a reboot then there is nothing lingering afterward. You would be in the clear. I would still exclude it of course. The possibility of archiving /dev/mem for example makes me nervous. :-) > > But if you were overwriting *everything* on the system from the backup > > on one to the new system then after having done so then postfix would > > have been installed. Right? The binaries in /usr/sbin/postfix would > > The reason why postfix didn't want to start, even after untarring the > whole system was because exim was still running. After stopping exim, I > could start postfix without a hitch. Ah, yes, port 25 was still busy. You would have needed to kill exim first. Gotcha! > >> Might be better to start with: > >> Old server: dpkg --get-selections > packages > >> New server: dpkg --set-selections < packages > > > > Yes. That is what that was designed for along with some other things > > such as the dselect-upgrade and so forth. Those will be suggested and > > with identical versions (Stable, OldStable) they should be able to > > replicate the same packages installed on each machine. I wanted to add and emphasize for others that this doesn't work very wll when changing from one machine to a different system with a different version. Lots of packages change names and dependencies across major releases. It works fine when the version is identical. But across different releases there are a lot of packages that don't like up exactly from one version to the next. You are on the same version but I wanted to leave this comment in the archive for others reading it. > > And also because the user ids of system users and groups are dependent > > upon the order in which they are installed. When doing a system splat > > from one system on top of another all of /etc/passwd, group, shadow, > > gshadow being overwritten with new numbers for all of the system > > installed accounts. Of course that is matched by completely overwriting > > all of the files elsewhere too. And because it won't remove files on > > the new server that weren't on the old server and those files will > > become orphaned when /var/lib/dpkg is overwritten. As long as it is a > > complete overwrite of *everything* then it works. But for files either > > extra or missing it can cause problems. > > Indeed. And the users is one of the reasons why I did want to try a full > copy otherwise these subtleties could make the move harder. > Same with databases and the database users. > The fully copy means not having to worry with creating the users on the > system and database level. Then you will be set and good to go. > > Having said this it is something that gets done periodically and it does > > work. Google does upgrades across its data centers "one file at a time" > > using rsync. Pretty amazingly cool. > > Wow I didn't know that. They rsync their servers? That's binaries and > config I suppose? Every file. File by file. I liked this presentation and found it quite interesting. http://marc.merlins.org/perso/linux/post_2014-01-06_My-Live-Upgrading-Many-Thousands-of-Servers-ProdNG-talk-at-Linux_conf_au-2014.html Unfortunately the original paper is now 403 forbidden. I think that is likely a mistake somewhere. But the Internet Archive Wayback Machine has a copy if you want to browse it. https://web.archive.org/web/*/http://marc.merlins.org/linux/talks/ProdNG-LCA2014/Paper/ProdNG.pdf > > And then there is Guillem Jover's older 'debtakeover' project. > > http://www.hadrons.org/~guillem/debian/debtakeover/README > > The debtakeover project replaces a foreign non-Debian system (such as > > Red Hat or SuSE) with a Debian system in-place while the system is > > running. It basically replaces the system out from under itself with > > That's impressive. It sounds a bit like a Debian borg system :) As I recall one spot I don't think was as complete was the kernel both running and otherwise. I think debtakeover required that the new system run on the foreign system's kernel. Which is not always possible due to system version differences. I think that is more of a problem now than it was then. So for example replacing a RHEL 6 system with Jessie would fail because jessie binaries require a newer kernel than the default RHEL 6 kernel. But probably upgrading the kernel first with a nati
Re: Moving server to new server with tar
Op Sat, 06 Jun 2015 13:59:16 -0600, schreef Bob Proulx: > This is one of those hard topics. It seems easy enough. But the > reality is that there are many subtle problems. It comes up for > discussion every so often over the years. I don't think there has ever > been a completely satisfactory canonical one true answer that solves the > problem in one collection. There just isn't any perfect methods. > Instead all that we can suggest is to understand everything and deal > with it ad-hoc. This is exactly what you are doing. With what you have > written I can tell that you are very much aware of most of the traps and > pitfalls and are dealing with them as best as you can. Good job! > > It would be good if there were a better way to deal with this. There > are many different strategies. Some people favor one strategy. Other > people favor other strategies. You and I have already diverged from > favored strategy. Personally I prefer to build a pristine system and > then install the new services upon it. That allows me to be refreshed > in everything. (On the other hand I always upgrade servers in place. I > carry the upgrade history along with me. An irony of opposites for me I > know.) I would use this as an opportunity to clean and clean and clean. > But it is okay if you tell me you want to have the identical server, as > identical as possible, moved without doing that in this step. That is > fine too. Bob, thanks for your reply and the time invested. Much appreciated. It does indeed seem tricky unless you go the full monty and replace the whole installation except for the special dirs like dev as you noted. In my test, I didn't get any strange results in the end but I have learned not to always trust what I see in IT. In the back of my mind, I always suspect a problem popping up when the server is in production. >> Tar command from the backup script on the old server: >> EXCLUDE="--exclude=proc --exclude=sys --exclude=dev/pts >> --exclude=backups" >> tar -czpf /backups/full.tar.gz --directory=/ $EXCLUDE / 2>&1 > > Since /dev is dynamic I exclude /dev from the backup too. Your new > installation will already have a static copy of the minimum dev under > the udev mount point. Indeed, it should be excluded as well. Good point. > One sideways strategy that you might consider for risk management is to > untar a copy of your old system into what will become a chroot area on > the new system. That will give you a reference on the new system. You > can run services from there. But mainly it would give you a way to do > an A-B comparison between what you had before and what you are creating > new. I do that often. If something shows up being different then can > go investigate the way things were and find lost and forgotten tweaks > and revive them. That's a really good idea. Untar in a separate directory and manage the installation of the services from there or use it as a base for comparison. I like it. > Right. They intentionally confict with each other and push each other > out. It will sound obvious but postfix can't come up if it isn't > installed. :-) > > But if you were overwriting *everything* on the system from the backup > on one to the new system then after having done so then postfix would > have been installed. Right? The binaries in /usr/sbin/postfix would > have been copied into place and the package manager would think it had > installed it in /var/lib/dpkg too. The biggest issue being any daemon > that changed uids and was running would need to have been stopped before > this and restarted after this. Right? This is one of the issues that > makes doing it this way tricky. Not impossible. Just tricky. The reason why postfix didn't want to start, even after untarring the whole system was because exim was still running. After stopping exim, I could start postfix without a hitch. >> Might be better to start with: >> Old server: dpkg --get-selections > packages >> New server: dpkg --set-selections < packages > > Yes. That is what that was designed for along with some other things > such as the dselect-upgrade and so forth. Those will be suggested and > with identical versions (Stable, OldStable) they should be able to > replicate the same packages installed on each machine. > > There are some issues with doing it this way. You should read about > 'apt-mark' and the database flag that indicates whether the package has > been installed explicitly or automatically. Automatically installed > packages without anything depending upon them are candidates for > 'apt-get autoremove' to remove them. Explicitly installed packages are > not. You can dump the previous values from the old server with: > > apt-mark showauto > apt-mark showmanual > > And then use the lists to set the same values in the new server. > Here is one way. I will leave you to fiddle with this further. > > apt-mark auto $(cat list-of-auto-packages) > > You can
Re: Moving server to new server with tar
Linux4Bene wrote: > I am in the process of moving my server to another VPS. > The goal is to keep the old VPS around and convert it to backup MX & DNS > amongst other things. I will purchase the new VPS from another company so > I can't just copy the vm file/container. > > As a start, I would do a full tar archive of the old server and start > from there. A test on a local VM worked, with some adjustments. Both use > Debian 7.8. The services on the old server that need to me moved: > - Mail: Postfix, Dovecot, Spamassassin, Clamav, Postgresql, ... > - Web: nginx, supervisord, python, php5-fpm, Postgresql, ... > - DNS: PowerDNS This is one of those hard topics. It seems easy enough. But the reality is that there are many subtle problems. It comes up for discussion every so often over the years. I don't think there has ever been a completely satisfactory canonical one true answer that solves the problem in one collection. There just isn't any perfect methods. Instead all that we can suggest is to understand everything and deal with it ad-hoc. This is exactly what you are doing. With what you have written I can tell that you are very much aware of most of the traps and pitfalls and are dealing with them as best as you can. Good job! It would be good if there were a better way to deal with this. There are many different strategies. Some people favor one strategy. Other people favor other strategies. You and I have already diverged from favored strategy. Personally I prefer to build a pristine system and then install the new services upon it. That allows me to be refreshed in everything. (On the other hand I always upgrade servers in place. I carry the upgrade history along with me. An irony of opposites for me I know.) I would use this as an opportunity to clean and clean and clean. But it is okay if you tell me you want to have the identical server, as identical as possible, moved without doing that in this step. That is fine too. > Tar command from the backup script on the old server: > EXCLUDE="--exclude=proc --exclude=sys --exclude=dev/pts --exclude=backups" > tar -czpf /backups/full.tar.gz --directory=/ $EXCLUDE / 2>&1 Since /dev is dynamic I exclude /dev from the backup too. Your new installation will already have a static copy of the minimum dev under the udev mount point. > I know I can migrate by first installing all packages, and then copying > the config and data from one server to the other. But then you need to > pick all data to be moved. It takes longer and it's more prone to error > (forgetting something). I want this server to be exactly the same as the > first one. One sideways strategy that you might consider for risk management is to untar a copy of your old system into what will become a chroot area on the new system. That will give you a reference on the new system. You can run services from there. But mainly it would give you a way to do an A-B comparison between what you had before and what you are creating new. I do that often. If something shows up being different then can go investigate the way things were and find lost and forgotten tweaks and revive them. > What I've found so far in my test: > - It's a good thing to first install all the same packages on the new > machine first. I didn't do that in my first test and Postfix wouldn't > come up because of Exim that was installed on the base version of the new > OS. Simple to solve but this wouldn't have happened if I had installed > Postfix first as Exim would have been purged. Right. They intentionally confict with each other and push each other out. It will sound obvious but postfix can't come up if it isn't installed. :-) But if you were overwriting *everything* on the system from the backup on one to the new system then after having done so then postfix would have been installed. Right? The binaries in /usr/sbin/postfix would have been copied into place and the package manager would think it had installed it in /var/lib/dpkg too. The biggest issue being any daemon that changed uids and was running would need to have been stopped before this and restarted after this. Right? This is one of the issues that makes doing it this way tricky. Not impossible. Just tricky. > Might be better to start with: > Old server: dpkg --get-selections > packages > New server: dpkg --set-selections < packages Yes. That is what that was designed for along with some other things such as the dselect-upgrade and so forth. Those will be suggested and with identical versions (Stable, OldStable) they should be able to replicate the same packages installed on each machine. There are some issues with doing it this way. You should read about 'apt-mark' and the database flag that indicates whether the package has been installed explicitly or automatically. Automatically installed packages without anything depending upon them are candidates for 'apt-get autoremove' to remove them. Explicitly installe
Moving server to new server with tar
Hi, I am in the process of moving my server to another VPS. The goal is to keep the old VPS around and convert it to backup MX & DNS amongst other things. I will purchase the new VPS from another company so I can't just copy the vm file/container. As a start, I would do a full tar archive of the old server and start from there. A test on a local VM worked, with some adjustments. Both use Debian 7.8. The services on the old server that need to me moved: - Mail: Postfix, Dovecot, Spamassassin, Clamav, Postgresql, ... - Web: nginx, supervisord, python, php5-fpm, Postgresql, ... - DNS: PowerDNS Tar command from the backup script on the old server: EXCLUDE="--exclude=proc --exclude=sys --exclude=dev/pts --exclude=backups" tar -czpf /backups/full.tar.gz --directory=/ $EXCLUDE / 2>&1 I know I can migrate by first installing all packages, and then copying the config and data from one server to the other. But then you need to pick all data to be moved. It takes longer and it's more prone to error (forgetting something). I want this server to be exactly the same as the first one. What I've found so far in my test: - It's a good thing to first install all the same packages on the new machine first. I didn't do that in my first test and Postfix wouldn't come up because of Exim that was installed on the base version of the new OS. Simple to solve but this wouldn't have happened if I had installed Postfix first as Exim would have been purged. Might be better to start with: Old server: dpkg --get-selections > packages New server: dpkg --set-selections < packages - Extract the tar archive from the root on the new server - Adjust /etc/hostname, /etc/hosts, /etc/network/interfaces - Adjust PowerDNS settings on new server. If the new server is up I will need to change the PowerDNS settings on the old server as well and set up DNS synchronization. DNS entries at the domain registrars & reverse DNS will need to be changed. - Check configs of the services above for the old ip or hostname - Run update-grub as the id of the disk has changed. - Reboot This worked well, but I wonder if there are good reasons to not do move the server like this? Thanks for any info or insight Regards, Benedict -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/mkovvi$jc2$1...@ger.gmane.org
