Re: SOLVED - Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Sunday 08 Dec 2013 14:41:44 Tanstaafl wrote: On 2013-12-03 8:19 AM, Tanstaafl tansta...@libertytrek.org wrote: Current command I'll be using: rsync -avHP --numeric-ids /mnt/gentoo/oldusr/ /mnt/gentoo/usr/ Well, that was about as uneventful as it gets... Took all of 6 minutes (and almost all of that was rsyncing /usr)... Glad it went well. Now for the next hurdle, eh? :-) -- Regards Peter
SOLVED - Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 2013-12-03 8:19 AM, Tanstaafl tansta...@libertytrek.org wrote: Current command I'll be using: rsync -avHP --numeric-ids /mnt/gentoo/oldusr/ /mnt/gentoo/usr/ Well, that was about as uneventful as it gets... Took all of 6 minutes (and almost all of that was rsyncing /usr)... Made a forum post in case anyone else wants to do this and may be a little hesitant like I was... http://forums.gentoo.org/viewtopic-p-7457324.html#7457324 Thanks to all who patiently answered my questions... I now have /usr merged back into / and no longer have to choose between using an intramfs (which I vehemently did not want to do) and updating my system without fear of breakage. whew
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 2013-12-04 8:07 AM, Peter Humphrey pe...@prh.myzen.co.uk wrote: On Wednesday 04 Dec 2013 07:28:18 Tanstaafl wrote: I've never used the -x option with cp... what exactly is meant by 'stay on same filesystem’?Should Stay on same filesystem is for the case in which you have another partition mounted somewhere in the tree below the current working directory. It means that you want to omit everything in that second file system. If you haven’t any such complication you don’t need to specify -x. Ah, ok that makes perfect sense, thanks. And for the record (you didn't specifically say so), are you in agreement that cp -a /usr/. /usr.tmp/. will accomplish the exact same thing as the rsync command I was planning on using? ... I imagine I could use the cp command first on the live system to ‘prime’ it, then use the rsync command after booting to the liveCD to quickly confirm it - but if there were no issues during the initial cp, and nothing changes in between, there shouldn't really be any difference to copy anyway? Indeed. I hope you don’t have experience of cp failing to copy what it should. Right... if it did it would most likely indicate some kind of filesystem corruption... Thanks again Peter, Charles
Re: [gentoo-user] Merging separate /usr back into / - one last time...
Le 2013-12-06 11:07, Tanstaafl wrote : And for the record (you didn't specifically say so), are you in agreement that cp -a /usr/. /usr.tmp/. will accomplish the exact same thing as the rsync command I was planning on using? For me, it's best to use rsync, because rsync will not copy file if they are already existing, and are the same. It's quite usefull when using a copy over a network, or even locally when spurious error can occur and especially when the filesystem is live and file may be modified during the copy. To copy large file tree rsync is, for me, always the best choice, and I'm pretty sure that there are some cases that rsync will behave better than a simple recursive cp Cheers, Godzil
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Fri, Dec 06, 2013 at 06:07:01AM -0500, Tanstaafl wrote: And for the record (you didn't specifically say so), are you in agreement that cp -a /usr/. /usr.tmp/. will accomplish the exact same thing as the rsync command I was planning on using? There are reasons why rsync is better than cp. The best one IMO is that rysnc will not copy a corrupt file, while cp will. And rsync will tell you about the corrupt file. You have so many questions about rsync, and different opinions thrown at you. Why do you not read man rsync and learn about it? This man page is thorough, easy to read, and explains it's usage by examples. Cheers, Bruce -- Happy Penguin Computers ') 126 Fenco Drive ( \ Tupelo, MS 38801 ^^ supp...@happypenguincomputers.com 662-269-2706 662-205-6424 http://happypenguincomputers.com/ A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? Don't top-post: http://en.wikipedia.org/wiki/Top_post#Top-posting
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 2013-12-06 8:13 AM, Bruce Hill da...@happypenguincomputers.com wrote: There are reasons why rsync is better than cp. The best one IMO is that rysnc will not copy a corrupt file, while cp will. And rsync will tell you about the corrupt file. Interesting and a good reason to use rsync over cp... if true... are you certain of this?
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Fri, Dec 06, 2013 at 11:35:29AM -0500, Tanstaafl wrote: On 2013-12-06 8:13 AM, Bruce Hill da...@happypenguincomputers.com wrote: There are reasons why rsync is better than cp. The best one IMO is that rysnc will not copy a corrupt file, while cp will. And rsync will tell you about the corrupt file. Interesting and a good reason to use rsync over cp... if true... are you certain of this? Yes, absolutely. Which is one reason I suggested you rean man rsync. I don't mean to be unkind, but there have been so many conflicting opinions in your thread. -- Happy Penguin Computers ') 126 Fenco Drive ( \ Tupelo, MS 38801 ^^ supp...@happypenguincomputers.com 662-269-2706 662-205-6424 http://happypenguincomputers.com/ A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? Don't top-post: http://en.wikipedia.org/wiki/Top_post#Top-posting
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 2013-12-03 8:19 AM, Tanstaafl tansta...@libertytrek.org wrote: Current command I'll be using: rsync -avHP --numeric-ids /mnt/gentoo/oldusr/ /mnt/gentoo/usr/ I had initially been planning on just using cp, trying now to remember why I decided on using rsync - I think it was someone here who said it would be better. Anyway, not to start a holy war about which is better, rsync or cp, but someone on the rsync list just posted that using rsync for this was really not the best use of rsync, and that cp was much better suited (would be much faster for one thing, and would not suffer the same potential problem of running out of memory because of too many hard links for another), and I think I agree with them... sigh So, can someone confirm that this command: cp -ax /usr/. /usr.tmp/. would accomplish the same goal? I've never used the -x option with cp... what exactly is meant by 'stay on same filesystem'? Should I use this seeing as current /usr is reiserfs on LVM, and / is ext3 on simple partition - ie, *not* the 'same filesystem'? Is the only real advantage of using rsync to do this it's ability to pick up where it left off if there is a problem? If so, then I imagine I could use the cp command first on the live system to 'prime' it, then use the rsync command after booting to the liveCD to quickly confirm it - but if there were no issues during the initial cp, and nothing changes in between, there shouldn't really be any difference to copy anyway? Sorry for all the questions, I promise this will be the last one on this subject...
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Wednesday 04 Dec 2013 07:28:18 Tanstaafl wrote: I've never used the -x option with cp... what exactly is meant by 'stay on same filesystem’?Should Stay on same filesystem is for the case in which you have another partition mounted somewhere in the tree below the current working directory. It means that you want to omit everything in that second file system. If you haven’t any such complication you don’t need to specify -x. For instance, I have separate partitions for /usr/portage and /usr/portage/packages. If I wanted to cp everything in portage but not in packages I’d specify -x. I use this seeing as current /usr is reiserfs on LVM, and / is ext3 on simple partition - ie, *not* the 'same filesystem’? Doesn’t matter. The type of file system is not visible to the copying program: to it, a file is a file is a file. Well, for present purposes anyway. I think you can interpret file-system as identical to partition here. ... I imagine I could use the cp command first on the live system to ‘prime’ it, then use the rsync command after booting to the liveCD to quickly confirm it - but if there were no issues during the initial cp, and nothing changes in between, there shouldn't really be any difference to copy anyway? Indeed. I hope you don’t have experience of cp failing to copy what it should. Sorry for all the questions, I promise this will be the last one on this subject... It’s how we learn, so don’t worry about it. :-) -- Regards Peter
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 2013-12-02 8:02 PM, Neil Bothwick n...@digimed.co.uk wrote: On Tue, 03 Dec 2013 06:24:43 +0800, William Kenworthy wrote: You have got the disk space, so if you have a backup its reversible so don't be a wimp :) It's reversible even if there is no backup, because data it copied from /usr to /, not moved. If the new /usr doesn't work for any reason, just mount the old one on it. True, I had forgotten about that... thanks, that actually does make it much less risky, so definitely will be trying this soon (maybe even this weekend if I have time)... :) Yeah, when it comes to servers, I'm more of a wimp than not... but being careful and conservative on my servers has saved me more times than I can count, so I'm ok with it... ;)
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 2013-12-02 5:24 PM, William Kenworthy bi...@iinet.net.au wrote: You are looking far too deep just rsync -avP to /newusr But this would not copy the hardlinks... and there are a bunch on /usr, so... reboot to livecd rsync again with --delete to update ... takes a only few seconds this time - minimal downtime:) mv /usr /oldusr mv /newusr /usr reboot The --numeric-ids is a good idea but I've made my systems consistent with the standard gentoo id's so that's no longer a problem. Ok, so, last questions... When mounting /, do I need to specify the fstype and mount option showing in fstab, ie: mount -t ext3 -o noatime /dev/sda3 /mnt/gentoo/ Then when mounting the old /usr, since it is reiserfs: mount -t reiserfs -o noatime /dev/vg/usr /mnt/gentoo/oldusr Last - are there any concerns about the fact that /usr is currently on a reiserfs file system, moving to an ext3 filesystem? Current command I'll be using: rsync -avHP --numeric-ids /mnt/gentoo/oldusr/ /mnt/gentoo/usr/ Thanks again, Charles
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Tue, Dec 03, 2013 at 06:47:01AM -0500, Tanstaafl wrote: Yeah, when it comes to servers, I'm more of a wimp than not... but being careful and conservative on my servers has saved me more times than I can count, so I'm ok with it... ;) I have one server with separate /usr that's in LVM. It only gets rebooted for kernel changes, or power loss greater than USB. The latter condition happened two days ago, but everything worked when it was booted again. I've got an initramfs on my laptop with everything on one LVM, so writing one for that server wouldn't be impossible. My only issue is having initramfs forced on us because of other people's bad ideas. And my server is setup the way it is for a lot of reasons, security being one. It has: o@server ~ $ df -hT │link/ether a0:88:b4:54:33:04 brd ff:ff:ff:ff:ff:ff Filesystem Type Size Used Avail Use% Mounted on │inet 192.168.11.7/24 brd 192.168.11.255 scope global wlan0 rootfs rootfs2.0G 116M 1.9G 6% / │ valid_lft forever preferred_lft forever /dev/rootxfs 2.0G 116M 1.9G 6% / │baruch ~ # ip addr devtmpfs devtmpfs 3.0G 0 3.0G 0% /dev │1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN tmpfstmpfs 603M 464K 603M 1% /run │link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 shm tmpfs 3.0G 0 3.0G 0% /dev/shm │inet 127.0.0.1/8 brd 127.255.255.255 scope host lo /dev/mapper/system-var xfs10G 721M 9.3G 8% /var │ valid_lft forever preferred_lft forever /dev/mapper/system-usr xfs10G 4.8G 5.2G 49% /usr │2: dummy0: BROADCAST,NOARP mtu 1500 qdisc noop state DOWN /dev/mapper/system-home xfs 6.0G 5.5G 580M 91% /home │link/ether 6e:83:0f:ef:52:15 brd ff:ff:ff:ff:ff:ff /dev/mapper/storage-photos xfs 500G 19G 482G 4% /photos │3: eth0: NO-CARRIER,BROADCAST,MULTICAST,UP mtu 1500 qdisc pfifo_fast state DOWN qlen 1000 /dev/mapper/storage-backups xfs 500G 166G 335G 34% /backups │link/ether 00:21:cc:5e:c3:12 brd ff:ff:ff:ff:ff:ff /dev/mapper/storage-offload fuseblk 300G 341M 300G 1% /offload │4: wlan0: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc mq state UP qlen 1000 /dev/mapper/storage-peterxfs25G 1.7G 24G 7% /peter │link/ether a0:88:b4:54:33:04 brd ff:ff:ff:ff:ff:ff /dev/mapper/storage-jeremiah xfs10G 3.4G 6.7G 34% /jeremiah so moving /usr into / isn't even an option. So perhaps one day it will get an initramfs ... or go back to devfs. :D -- Happy Penguin Computers ') 126 Fenco Drive ( \ Tupelo, MS 38801 ^^ supp...@happypenguincomputers.com 662-269-2706 662-205-6424 http://happypenguincomputers.com/ A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? Don't top-post: http://en.wikipedia.org/wiki/Top_post#Top-posting
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Tue, Dec 03, 2013 at 11:06:16AM -0600, Bruce Hill wrote: mingdao@server ~ $ df -hT │link/ether a0:88:b4:54:33:04 brd ff:ff:ff:ff:ff:ff Filesystem Type Size Used Avail Use% Mounted on │inet 192.168.11.7/24 brd 192.168.11.255 scope global wlan0 rootfs rootfs2.0G 116M 1.9G 6% / │ valid_lft forever preferred_lft forever /dev/rootxfs 2.0G 116M 1.9G 6% / │baruch ~ # ip addr devtmpfs devtmpfs 3.0G 0 3.0G 0% /dev │1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN tmpfstmpfs 603M 464K 603M 1% /run │link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 shm tmpfs 3.0G 0 3.0G 0% /dev/shm │inet 127.0.0.1/8 brd 127.255.255.255 scope host lo /dev/mapper/system-var xfs10G 721M 9.3G 8% /var │ valid_lft forever preferred_lft forever /dev/mapper/system-usr xfs10G 4.8G 5.2G 49% /usr │2: dummy0: BROADCAST,NOARP mtu 1500 qdisc noop state DOWN /dev/mapper/system-home xfs 6.0G 5.5G 580M 91% /home │link/ether 6e:83:0f:ef:52:15 brd ff:ff:ff:ff:ff:ff /dev/mapper/storage-photos xfs 500G 19G 482G 4% /photos │3: eth0: NO-CARRIER,BROADCAST,MULTICAST,UP mtu 1500 qdisc pfifo_fast state DOWN qlen 1000 /dev/mapper/storage-backups xfs 500G 166G 335G 34% /backups │link/ether 00:21:cc:5e:c3:12 brd ff:ff:ff:ff:ff:ff /dev/mapper/storage-offload fuseblk 300G 341M 300G 1% /offload │4: wlan0: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc mq state UP qlen 1000 /dev/mapper/storage-peterxfs25G 1.7G 24G 7% /peter │link/ether a0:88:b4:54:33:04 brd ff:ff:ff:ff:ff:ff /dev/mapper/storage-jeremiah xfs10G 3.4G 6.7G 34% /jeremiah Note to self: cp/paste from tmux isn't working as expected ... read moar docs -- Happy Penguin Computers ') 126 Fenco Drive ( \ Tupelo, MS 38801 ^^ supp...@happypenguincomputers.com 662-269-2706 662-205-6424 http://happypenguincomputers.com/ A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? Don't top-post: http://en.wikipedia.org/wiki/Top_post#Top-posting
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Tue, Dec 3, 2013 at 12:06 AM, Canek Peláez Valdés can...@gmail.com wrote: On Mon, Dec 2, 2013 at 10:34 PM, Jc García jyo.gar...@gmail.com wrote: 2013/12/2 William Kenworthy bi...@iinet.net.au You are looking far too deep just rsync -avP to /newusr +1 I have done this more or less the same way reboot to livecd rsync again with --delete to update ... takes a only few seconds this time - minimal downtime :) mv /usr /oldusr mv /newusr /usr reboot Let's make this thread more interesting, would it be possible to do this without a reboot? ie: going single user mode, kill anything that might still be running from usr, umount /usr, mount it to /mnt, rsync -avP to usr, going again into runlevel 3 or 5. Obviously not possible if running systemd. I'm not so sure it's not possible. Perhaps it's even easier. So, yeah, I think it's easier with systemd. You just: 1. systemctl isolate emergency.target 2. log in again (all the normal gettys are killed with the above command) 3. rsync -PvasHA /usr/ /newusr/ 4. mv /usr /oldusr # mv is on /bin, so no problems here 5. mv /newusr /usr 6. rm -rf /oldusr (to make sure nothing uses it anymore) 7. systemctl isolate multi-user.target 8. You have your system again. I did this on a minimal QEMU virtual machine. However I think it should work with even the most complex setups, as long as your initramfs has the necessary tools, which is really easy with dracut. Regards. -- Canek Peláez Valdés Posgrado en Ciencia e Ingeniería de la Computación Universidad Nacional Autónoma de México
Re: [gentoo-user] Merging separate /usr back into / - one last time...
2013/12/3 Canek Peláez Valdés can...@gmail.com: On Tue, Dec 3, 2013 at 12:06 AM, Canek Peláez Valdés can...@gmail.com wrote: On Mon, Dec 2, 2013 at 10:34 PM, Jc García jyo.gar...@gmail.com wrote: 2013/12/2 William Kenworthy bi...@iinet.net.au You are looking far too deep just rsync -avP to /newusr +1 I have done this more or less the same way reboot to livecd rsync again with --delete to update ... takes a only few seconds this time - minimal downtime :) mv /usr /oldusr mv /newusr /usr reboot Let's make this thread more interesting, would it be possible to do this without a reboot? ie: going single user mode, kill anything that might still be running from usr, umount /usr, mount it to /mnt, rsync -avP to usr, going again into runlevel 3 or 5. Obviously not possible if running systemd. I'm not so sure it's not possible. Perhaps it's even easier. So, yeah, I think it's easier with systemd. You just: 1. systemctl isolate emergency.target 2. log in again (all the normal gettys are killed with the above command) 3. rsync -PvasHA /usr/ /newusr/ 4. mv /usr /oldusr # mv is on /bin, so no problems here 5. mv /newusr /usr 6. rm -rf /oldusr (to make sure nothing uses it anymore) 7. systemctl isolate multi-user.target 8. You have your system again. Nice, I thought systemd residing within /usr would be the limitation, i haven't used systemd very much , so i don't really know it's options, but later after thinking about it, i thought that using the initramfs would be a way to go, but as i understand here systemd already has an option to make use of it. I did this on a minimal QEMU virtual machine. However I think it should work with even the most complex setups, as long as your initramfs has the necessary tools, which is really easy with dracut. Regards. -- Canek Peláez Valdés Posgrado en Ciencia e Ingeniería de la Computación Universidad Nacional Autónoma de México
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Tue, Dec 3, 2013 at 5:22 PM, Jc García jyo.gar...@gmail.com wrote: 2013/12/3 Canek Peláez Valdés can...@gmail.com: On Tue, Dec 3, 2013 at 12:06 AM, Canek Peláez Valdés can...@gmail.com wrote: On Mon, Dec 2, 2013 at 10:34 PM, Jc García jyo.gar...@gmail.com wrote: 2013/12/2 William Kenworthy bi...@iinet.net.au You are looking far too deep just rsync -avP to /newusr +1 I have done this more or less the same way reboot to livecd rsync again with --delete to update ... takes a only few seconds this time - minimal downtime :) mv /usr /oldusr mv /newusr /usr reboot Let's make this thread more interesting, would it be possible to do this without a reboot? ie: going single user mode, kill anything that might still be running from usr, umount /usr, mount it to /mnt, rsync -avP to usr, going again into runlevel 3 or 5. Obviously not possible if running systemd. I'm not so sure it's not possible. Perhaps it's even easier. So, yeah, I think it's easier with systemd. You just: 1. systemctl isolate emergency.target 2. log in again (all the normal gettys are killed with the above command) 3. rsync -PvasHA /usr/ /newusr/ 4. mv /usr /oldusr # mv is on /bin, so no problems here 5. mv /newusr /usr 6. rm -rf /oldusr (to make sure nothing uses it anymore) 7. systemctl isolate multi-user.target 8. You have your system again. Nice, I thought systemd residing within /usr would be the limitation, i haven't used systemd very much , so i don't really know it's options, but later after thinking about it, i thought that using the initramfs would be a way to go, but as i understand here systemd already has an option to make use of it. Yeah, you can create an initramfs with dracut that uses systemd; the initramfs mounts /usr, and then handles back control to the systemd instance in it. When shutting down, the inverse occurs; the systemd in /usr handles back control to the systemd in the initramfs, which in turns shutdowns the machine. I don't know exactly how problematic would be for a complex setup (/usr using LVM+cryptfs+mdraid, for example), but if the necessary tools are available in the initramfs, then I think it could be done. Of course, the simplest and easiest thing to do is to use a live CD. Regards. -- Canek Peláez Valdés Posgrado en Ciencia e Ingeniería de la Computación Universidad Nacional Autónoma de México
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Mon, Dec 02 2013, tansta...@libertytrek.org wrote: Is rsync -a enough for my relatively simple system setup, or would using any or all of the other options suggested in those threads be safer/better? Specifically: -a, or -axAHX, or -apogXx, or -PvasHAX I am not an expert but here goes. -x would not hurt but should not be needed since i believe that your current /dev/vg/usr is just one partition. I didn't need -X -A because I don't have acls or extended attributes or should I go with a combined -apogsvxAHPX ? So, here's the plan, please check me... 1. Boot off of the latest gentoo LiveDVD 2. Mount / and create new /usr directory I am missing something. I would have thought your old / (dev/sda3) already has an empty /usr directory where you previously mounted /dev/vg/usr mount /dev/sda3 /mnt/gentoo/ mkdir /mnt/gentoo/usr 3. Mount old /usr to be moved/merged vgscan vgchange -a y mount /dev/vg/usr /mnt/gentoo/oldusr 4. Copy /oldusr to /usr This suggests that your current root (dev/sda3) is big enough to include the previous /usr (dev/vg/usr). That is indeed a simple case. Many of us had to move partitions around to get a big enough partition for / + /usr. rsync -a? /mnt/gentoo/oldusr/ /mnt/gentoo/usr/ Are the trailing slashes required/important/necessary? The first trailing slash (oldusr/) is important. Without it, you would be creating the directory /mnt/gentoo/usr/oldusr. With it (as you wrote) just the contents of /oldusr are copied not the directory itself. So yes you want that slash. I don't believe the 2nd trailing / (usr/) is needed, but doesn't hurt. The rsync man page shows both uses and I don't see any words saying anything about the difference. I must say I never noticed the two different uses in the man page can't remember what I used. But again, I believe the results are the same. Which arguments should I use? Discussed above 5. Edit /etc/fstab and comment/remove the /usr line nano -wc /mnt/gentoo/etc/fstab #/dev/vg/usr /usrreiserfsnoatime 0 0 6. Unmount mounted filesystems umount /mnt/gentoo/oldusr umount /mnt/gentoo 7. Reboot into new system Done? I'm pretty sure that: 1. There is no need to chroot into the real system during this process, and 2. I only need to mount / and the old /usr, no need to mount anything else (/proc, /sys, /var, /home, activating swap, etc) Correct? Both of these seem correct to me. Good luck! allan
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 2013-12-02 11:26 AM, gottl...@nyu.edu gottl...@nyu.edu wrote: On Mon, Dec 02 2013, tansta...@libertytrek.org wrote: So, here's the plan, please check me... 1. Boot off of the latest gentoo LiveDVD 2. Mount / and create new /usr directory I am missing something. I would have thought your old / (dev/sda3) already has an empty /usr directory where you previously mounted /dev/vg/usr Hmmm... I guess you're right, although I guess I'd have realized that as soon as I mounted / to /mnt/gentoo and did an ls... 4. Copy /oldusr to /usr This suggests that your current root (dev/sda3) is big enough to include the previous /usr (dev/vg/usr). Yep, plenty of room... Are you saying you went through this too? Hopefully a few others will chime in with more on the exact rsync arguments I should use... Thanks Allan... :)
Re: [gentoo-user] Merging separate /usr back into / - one last time...
on 12/02/2013 04:02 PM Tanstaafl wrote the following: So, here's the plan, please check me... 1. Boot off of the latest gentoo LiveDVD If you boot a different system to do the rsync, or, if you do it over ssh, add the option --numeric-ids I usually do rsync -aHvxW --numeric-ids --delete sourcedit/ targetdir/
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 2013-12-02 1:47 PM, Thanasis thana...@asyr.hopto.org wrote: on 12/02/2013 04:02 PM Tanstaafl wrote the following: So, here's the plan, please check me... 1. Boot off of the latest gentoo LiveDVD If you boot a different system to do the rsync, or, if you do it over ssh, add the option --numeric-ids Thanks, but no, like I said, I'll just boot that system to a LiveDVD and do it from there... So, I guess my main question is... Would it hurt anything to use all of the args: ie... -apogsvxAHPX I guess I may be overthinking this (again)...
Re: [gentoo-user] Merging separate /usr back into / - one last time...
An alternative to booting to external media, etc, would be a bind mount of / and /usr on separate temporary mount points, then dumping the data between them, leaving the existing system chugging along. A re-mount of the current /usr in -o ro mode might not be a terrible idea in that case. I had a good bit of luck going that route. A simple cp -a did the trick on the one system I've bothered with it on so far. I have a couple laptops that're rushing headlong into unsupported land right now though, so I'll be revisiting this soon enough. As for specifics: # Make the temporary working areas mkdir /tmp/a; mkdir /tmp/b # Make sure nothing changes in /usr while the copy is done mount -o remount,ro /usr # Mount a mirror of the source and destination filesystems mount --bind /usr/ /tmp/a mount --bind / /tmp/b # And now, copy. cd /tmp/a cp -a ./ /tmp/b/usr/ The one big point of what not to do would be mount --rbind. Very important (recursive bind would have the current /usr still visible in /tmp/b/usr/). After all that, comment out /usr in fstab and reboot. You *could* even just drop to a minimal runlevel that doesn't require /usr, unmount the old one and then jump back to your standard runlevel, but due to the reasons this is required now, I'm not entirely sure that option exists anymore (i.e. too much is dependent on /usr). That said, if you are booting to a LiveDVD -- On Mon, Dec 2, 2013 at 1:58 PM, Tanstaafl tansta...@libertytrek.org wrote: On 2013-12-02 1:47 PM, Thanasis thana...@asyr.hopto.org wrote: on 12/02/2013 04:02 PM Tanstaafl wrote the following: So, here's the plan, please check me... 1. Boot off of the latest gentoo LiveDVD If you boot a different system to do the rsync, or, if you do it over ssh, add the option --numeric-ids Thanks, but no, like I said, I'll just boot that system to a LiveDVD and do it from there... That actually does fall under boot a different system since the users won't line up between a LiveDVD and your actual system. -- Poison [BLX] Joshua M. Murphy
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 2013-12-02 2:25 PM, Poison BL. poiso...@gmail.com wrote: An alternative to booting to external media, etc, would be a bind mount of / and /usr on separate temporary mount points, then dumping the data between them, leaving the existing system chugging along. A re-mount of the current /usr in -o ro mode might not be a terrible idea in that case. snip Not comfortable doing that on a productions server at all... but thanks anyway... :) That said, if you are booting to a LiveDVD -- On Mon, Dec 2, 2013 at 1:58 PM, Tanstaafl tansta...@libertytrek.org wrote: Thanks, but no, like I said, I'll just boot that system to a LiveDVD and do it from there... That actually does fall under boot a different system since the users won't line up between a LiveDVD and your actual system. Hmmm, ok, but if I chroot'd into the system (ie, like I might do if I had an interrupted install, and wanted to pick it back up), that would solve that? Or, I could just figure out the right rsync arguments to use... ;) I just asked on the rsync list too, maybe someone there will chime in with a little more confidence...
Re: [gentoo-user] Merging separate /usr back into / - one last time...
on 12/02/2013 08:58 PM Tanstaafl wrote the following: On 2013-12-02 1:47 PM, Thanasis thana...@asyr.hopto.org wrote: on 12/02/2013 04:02 PM Tanstaafl wrote the following: So, here's the plan, please check me... 1. Boot off of the latest gentoo LiveDVD If you boot a different system to do the rsync, or, if you do it over ssh, add the option --numeric-ids Thanks, but no, like I said, I'll just boot that system to a LiveDVD and do it from there... Are you sure the user IDs of the LiveDVD are the same as the other system's users' IDs? That is why I recommend using the option --numeric-ids. And using it would not hurt anyway.
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 2013-12-02 2:41 PM, Thanasis thana...@asyr.hopto.org wrote: on 12/02/2013 08:58 PM Tanstaafl wrote the following: On 2013-12-02 1:47 PM, Thanasis thana...@asyr.hopto.org wrote: on 12/02/2013 04:02 PM Tanstaafl wrote the following: So, here's the plan, please check me... 1. Boot off of the latest gentoo LiveDVD If you boot a different system to do the rsync, or, if you do it over ssh, add the option --numeric-ids Thanks, but no, like I said, I'll just boot that system to a LiveDVD and do it from there... Are you sure the user IDs of the LiveDVD are the same as the other system's users' IDs? That is why I recommend using the option --numeric-ids. And using it would not hurt anyway. Right... poison pointed this out... This is why I asked for help about the arguments. I honestly don't care about superflous/unnecessary arguments, I just want to make sure I use at least the ones needed for this to work. Thanks...
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Monday 02 Dec 2013 20:40:28 Tanstaafl wrote: On 2013-12-02 2:41 PM, Thanasis thana...@asyr.hopto.org wrote: That is why I recommend using the option --numeric-ids. And using it would not hurt anyway. Right... poison pointed this out... This is why I asked for help about the arguments. I honestly don't care about superflous/unnecessary arguments, I just want to make sure I use at least the ones needed for this to work. Thanks... The comment about --numeric-ids that Thanasis made is valid. I messed up some fs of mine last time I used rsync, when I wasn't paying much attention! I made a mental note to always use it in the future. On the other hand, if you're not that comfortable with it, a quick trial run with a test filesystem will offer some assurance that your chosen command and options will work as you intended. BTW, you do not *have* to use rsync: cp -a will do the same. su - cd /old_usr tar --one-file-system -cf . | (cd /new_usr ; tar -xvpf - ) will also do the same. Finally, star -copy is my favourite faster alternative to copying directories, inc. respecting any acl's and the like if you specify it in the options: su - star -copy options -C /old_usr . /new_usr Then you can also add -diff to see if any file was not copied correctly (use star diffopts=!option to exclude things like ctime, or you'll drown in the noise of the output). Speaking from experience I suggest that you do not blast your old /usr away until you have booted with /usr mounted in the new location and have verified that ownership and access rights are as you expected. -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Merging separate /usr back into / - one last time...
You are looking far too deep just rsync -avP to /newusr reboot to livecd rsync again with --delete to update ... takes a only few seconds this time - minimal downtime :) mv /usr /oldusr mv /newusr /usr reboot The --numeric-ids is a good idea but I've made my systems consistent with the standard gentoo id's so that's no longer a problem. Ive done this many times over the years, and to the system I am writing this on many times (moving to lvm2, restoring from backups after disk failures, restoring from backups after user failure - rm -rf /usr !) If you need to practice, run up a vm and test/destroy :) You have got the disk space, so if you have a backup its reversible so don't be a wimp :) BillK On 03/12/13 05:36, Mick wrote: On Monday 02 Dec 2013 20:40:28 Tanstaafl wrote: On 2013-12-02 2:41 PM, Thanasis thana...@asyr.hopto.org wrote: That is why I recommend using the option --numeric-ids. And using it would not hurt anyway. Right... poison pointed this out... This is why I asked for help about the arguments. I honestly don't care about superflous/unnecessary arguments, I just want to make sure I use at least the ones needed for this to work. Thanks... The comment about --numeric-ids that Thanasis made is valid. I messed up some fs of mine last time I used rsync, when I wasn't paying much attention! I made a mental note to always use it in the future. On the other hand, if you're not that comfortable with it, a quick trial run with a test filesystem will offer some assurance that your chosen command and options will work as you intended. BTW, you do not *have* to use rsync: cp -a will do the same. su - cd /old_usr tar --one-file-system -cf . | (cd /new_usr ; tar -xvpf - ) will also do the same. Finally, star -copy is my favourite faster alternative to copying directories, inc. respecting any acl's and the like if you specify it in the options: su - star -copy options -C /old_usr . /new_usr Then you can also add -diff to see if any file was not copied correctly (use star diffopts=!option to exclude things like ctime, or you'll drown in the noise of the output). Speaking from experience I suggest that you do not blast your old /usr away until you have booted with /usr mounted in the new location and have verified that ownership and access rights are as you expected.
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Mon, Dec 02 2013, tansta...@libertytrek.org wrote: On 2013-12-02 11:26 AM, gottl...@nyu.edu gottl...@nyu.edu wrote: On Mon, Dec 02 2013, tansta...@libertytrek.org wrote: So, here's the plan, please check me... 1. Boot off of the latest gentoo LiveDVD 2. Mount / and create new /usr directory I am missing something. I would have thought your old / (dev/sda3) already has an empty /usr directory where you previously mounted /dev/vg/usr Hmmm... I guess you're right, although I guess I'd have realized that as soon as I mounted / to /mnt/gentoo and did an ls... 4. Copy /oldusr to /usr This suggests that your current root (dev/sda3) is big enough to include the previous /usr (dev/vg/usr). Yep, plenty of room... Are you saying you went through this too? Yes I did, but did not have the room to simply stick /usr into /. Like you I had lvm. Hopefully a few others will chime in with more on the exact rsync arguments I should use... Thanks Allan... :) My pleasure. allan
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Tue, 03 Dec 2013 06:24:43 +0800, William Kenworthy wrote: You have got the disk space, so if you have a backup its reversible so don't be a wimp :) It's reversible even if there is no backup, because data it copied from /usr to /, not moved. If the new /usr doesn't work for any reason, just mount the old one on it. -- Neil Bothwick Unthinking respect for authority is the greatest enemy of truth. (Albert Einstein) signature.asc Description: PGP signature
Re: [gentoo-user] Merging separate /usr back into / - one last time...
2013/12/2 William Kenworthy bi...@iinet.net.au You are looking far too deep just rsync -avP to /newusr +1 I have done this more or less the same way reboot to livecd rsync again with --delete to update ... takes a only few seconds this time - minimal downtime :) mv /usr /oldusr mv /newusr /usr reboot Let's make this thread more interesting, would it be possible to do this without a reboot? ie: going single user mode, kill anything that might still be running from usr, umount /usr, mount it to /mnt, rsync -avP to usr, going again into runlevel 3 or 5. Obviously not possible if running systemd. The --numeric-ids is a good idea but I've made my systems consistent with the standard gentoo id's so that's no longer a problem. Ive done this many times over the years, and to the system I am writing this on many times (moving to lvm2, restoring from backups after disk failures, restoring from backups after user failure - rm -rf /usr !) If you need to practice, run up a vm and test/destroy :) You have got the disk space, so if you have a backup its reversible so don't be a wimp :) BillK On 03/12/13 05:36, Mick wrote: On Monday 02 Dec 2013 20:40:28 Tanstaafl wrote: On 2013-12-02 2:41 PM, Thanasis thana...@asyr.hopto.org wrote: That is why I recommend using the option --numeric-ids. And using it would not hurt anyway. Right... poison pointed this out... This is why I asked for help about the arguments. I honestly don't care about superflous/unnecessary arguments, I just want to make sure I use at least the ones needed for this to work. Thanks... The comment about --numeric-ids that Thanasis made is valid. I messed up some fs of mine last time I used rsync, when I wasn't paying much attention! I made a mental note to always use it in the future. On the other hand, if you're not that comfortable with it, a quick trial run with a test filesystem will offer some assurance that your chosen command and options will work as you intended. BTW, you do not *have* to use rsync: cp -a will do the same. su - cd /old_usr tar --one-file-system -cf . | (cd /new_usr ; tar -xvpf - ) will also do the same. Finally, star -copy is my favourite faster alternative to copying directories, inc. respecting any acl's and the like if you specify it in the options: su - star -copy options -C /old_usr . /new_usr Then you can also add -diff to see if any file was not copied correctly (use star diffopts=!option to exclude things like ctime, or you'll drown in the noise of the output). Speaking from experience I suggest that you do not blast your old /usr away until you have booted with /usr mounted in the new location and have verified that ownership and access rights are as you expected. PD: sorry if my english is not so good
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On 03/12/13 12:34, Jc García wrote: 2013/12/2 William Kenworthy bi...@iinet.net.au You are looking far too deep just rsync -avP to /newusr +1 I have done this more or less the same way reboot to livecd rsync again with --delete to update ... takes a only few seconds this time - minimal downtime :) mv /usr /oldusr mv /newusr /usr reboot Let's make this thread more interesting, would it be possible to do this without a reboot? ie: going single user mode, kill anything that might still be running from usr, umount /usr, mount it to /mnt, rsync -avP to usr, going again into runlevel 3 or 5. Obviously not possible if running systemd. I did try it single user mode - it works but I rebooted when the pressure came off just in case. Wanted minimal downtime. BillK
Re: [gentoo-user] Merging separate /usr back into / - one last time...
On Mon, Dec 2, 2013 at 10:34 PM, Jc García jyo.gar...@gmail.com wrote: 2013/12/2 William Kenworthy bi...@iinet.net.au You are looking far too deep just rsync -avP to /newusr +1 I have done this more or less the same way reboot to livecd rsync again with --delete to update ... takes a only few seconds this time - minimal downtime :) mv /usr /oldusr mv /newusr /usr reboot Let's make this thread more interesting, would it be possible to do this without a reboot? ie: going single user mode, kill anything that might still be running from usr, umount /usr, mount it to /mnt, rsync -avP to usr, going again into runlevel 3 or 5. Obviously not possible if running systemd. I'm not so sure it's not possible. Perhaps it's even easier. If you do systemctl isolate emergency.target then remount / read/write, do the move, and then again isolate multi-user.target or graphical.target, I think is possible. I will try on a virtual machine; is an interesting question. You would need to use absolute pathnames when actually performing the move, but I think is possible. Regards. -- Canek Peláez Valdés Posgrado en Ciencia e Ingeniería de la Computación Universidad Nacional Autónoma de México
