Sounds like overlayfs is what you are looking for? # mount -t overlayfs overlayfs ~/rootfs/etc/ -o rw,upperdir=~/custom,lowerdir=/etc # rm ~/rootfs/etc/passwd # ls /etc/passwd /etc/passwd # ls custom/ -al total 8 drwxr-xr-x 2 root root 4096 Dec 15 16:29 . drwx------ 12 root root 4096 Dec 15 16:29 .. lrwxrwxrwx 1 root root 18 Dec 15 16:29 passwd -> (overlay-whiteout)
On Mon, Dec 15, 2014 at 3:30 PM, Barry Jaspan <[email protected]> wrote: > On Mon, Dec 15, 2014 at 2:52 PM, Serge Hallyn <[email protected]> > wrote: >> >> /other_file is bind-mounted *over* /empty. So /empty is busy. > > > Okay, I get it. The inode which is the host's /empty is being used as a > mount-point. It makes sense that being a mount-point makes the inode busy. > > My goal is to use the host's /etc inside the container, but to replace > /etc/passwd+group+shadow within the container with unique per-container > versions, while still allowing commands like useradd/chsh/etc, which unlink > /etc/passwd, to work. My current work-around is to create a hardlink farm > of /etc to another path and replace passwd+group+shadow with empty files: > > cp -al /etc /etc_farm > rm /etc_farm/{passwd,group,shadow} > touch /etc_farm/{passwd,group,shadow} > > I can then mount /etc_farm as the container's /etc, and mount a unique > per-container passwd+group+shadow over the container's /etc versions, and > the host's /etc/passwd+group+shadow are not EBUSY. This works fine. The > downside is that any changes to the host's /etc are not reflected in > /etc_farm unless I regenerate /etc_farm. > > Does anyone have another suggested approach? > > Barry > > >> >> >> > host# ls -li /empty /other_file >> > 57876 -rw-r--r-- 1 root root 0 Dec 15 19:26 /empty >> > 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /other_file >> > host# lxc-execute -n test -f test.config -- ls -li /empty /other_file >> > 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /empty >> > 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /other_file >> > >> > I would expect the host's /other_file to be busy, but not the host's >> > /empty. >> > >> > Barry >> > >> > >> > >> > >> > >> > >> > >> > > It is over-mounted. So it >> > > is busy. >> > > >> > > Quoting Barry Jaspan ([email protected]): >> > > > I have reduced the problem I described in my previous message to a >> > > > much >> > > > smaller test case. Here is an LXC config file that bind-mounts a >> > > > single >> > > > file, /other_file, from the host's root filesystem on top of another >> > > file, >> > > > /empty, also from the host's root filesystem: >> > > > >> > > > lxc.console = none >> > > > lxc.rootfs = / >> > > > lxc.mount.entry=/other_file empty none rw,bind 0 0 >> > > > >> > > > While the container is running, the host's /empty cannot be >> > > > unlinked: >> > > > >> > > > host# rm /empty /other_file >> > > > host# touch /empty /other_file >> > > > host# lxc-execute -n test -f test.config -- sleep 30 & >> > > > [1] 2419 >> > > > host# strace -o rm.out rm /empty >> > > > rm: cannot remove `/empty': Device or resource busy >> > > > host# grep EBUSY rm.out >> > > > unlinkat(AT_FDCWD, "/empty", 0) = -1 EBUSY (Device or >> > > > resource >> > > busy) >> > > > host# fg >> > > > lxc-execute -n test -f test.config -- sleep 30 >> > > > ^C >> > > > host# rm /empty >> > > > host# >> > > > >> > > > Can anyone explain why the host's /empty cannot be unlinked even >> > > > though >> > > it >> > > > is bind-mounted out of the running container? Why is it "busy"? >> > > > >> > > > Thanks, >> > > > Barry >> > > > >> > > > >> > > > >> > > > >> > > > On Mon, Dec 8, 2014 at 5:53 PM, Barry Jaspan >> > > > <[email protected]> >> > > > wrote: >> > > > > >> > > > > I'm using LXC on Ubuntu 12.04 (Precise) on EC2. I am creating a >> > > container >> > > > > that uses the host's root filesystem. In the lxc config, I am >> > > bind-mounting >> > > > > a different passwd file on top of /etc/passwd: >> > > > > >> > > > > lxc.mount.entry=/container/passwd etc/passwd none rw,bind 0 0 >> > > > > >> > > > > As expected, /etc/passwd on the host and /etc/passwd in the >> > > > > container >> > > are >> > > > > different inodes: >> > > > > >> > > > > host# ls -i /etc/passwd >> > > > > 58046 /etc/passwd >> > > > > host# ls -i /container/passwd >> > > > > 287145 /container/passwd >> > > > > >> > > > > container# ls -i /etc/passwd >> > > > > 287145 /etc/passwd >> > > > > >> > > > > What I did not expect is that now it is not possible to unlink >> > > /etc/passwd >> > > > > at the host level (or inside the container, although I don't care >> > > > > about >> > > > > that): >> > > > > >> > > > > host# rm /etc/passwd >> > > > > rm: cannot remove `/etc/passwd': Device or resource busy >> > > > > host# chsh -s /bin/false root >> > > > > chsh: failure while writing changes to /etc/passwd >> > > > > >> > > > > strace confirms that unlinkat() and rename() are returning EBUSY. >> > > > > (chsh >> > > > > creates a replacement for /etc/passwd and then rename()'s it into >> > > place.) >> > > > > >> > > > > So, somehow, the fact that the host's /etc/passwd is *not* present >> > > > > in >> > > the >> > > > > container marks its inode as busy. I do not think it is as simple >> > > > > as >> > > "an >> > > > > inode that has been bind-mounted out of a namespace is busy" >> > > > > because a >> > > > > simple bind-mount test case does not reproduce it: >> > > > > >> > > > > # echo foo > foo >> > > > > # touch bar >> > > > > # mount --bind ./foo ./bar >> > > > > # cat bar >> > > > > foo >> > > > > # touch new >> > > > > # mv new foo >> > > > > # >> > > > > >> > > > > This seems related to http://lwn.net/Articles/570338/. However, in >> > > that >> > > > > article, a file which is bind-mounted into another namespace >> > > > > causes >> > > EBUSY >> > > > > from unlink/rename(). In my case, a file which is *not* >> > > > > bind-mounted >> > > into >> > > > > another namespace is causing EBUSY. >> > > > > >> > > > > Can someone explain what is going on? >> > > > > >> > > > > Thanks, >> > > > > >> > > > > Barry >> > > > > >> > > > > -- >> > > > > Barry Jaspan >> > > > > Senior Architect | Acquia <http://acquia.com> >> > > > > [email protected] | (c) 617.905.2208 | (w) 781-313-8298 >> > > > > >> > > > > Acquia Dev Cloud: You build killer websites. We do the rest. >> > > > > <http://www.acquia.com/dev-cloud> <http://acquia.com/dev-cloud> >> > > > > Acquia ranked #1 Software Vendor on the 2012 Inc 500 >> > > > > < >> > > >> > > http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing >> > > > >> > > > > >> > > > > >> > > > >> > > > -- >> > > > Barry Jaspan >> > > > Senior Architect | Acquia <http://acquia.com> >> > > > [email protected] | (c) 617.905.2208 | (w) 781-313-8298 >> > > > >> > > > Acquia Dev Cloud: You build killer websites. We do the rest. >> > > > <http://www.acquia.com/dev-cloud> <http://acquia.com/dev-cloud> >> > > > Acquia ranked #1 Software Vendor on the 2012 Inc 500 >> > > > < >> > > >> > > http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing >> > > > >> > > >> > > > _______________________________________________ >> > > > lxc-users mailing list >> > > > [email protected] >> > > > http://lists.linuxcontainers.org/listinfo/lxc-users >> > > >> > > _______________________________________________ >> > > lxc-users mailing list >> > > [email protected] >> > > http://lists.linuxcontainers.org/listinfo/lxc-users >> > >> > >> > >> > -- >> > Barry Jaspan >> > Senior Architect | Acquia <http://acquia.com> >> > [email protected] | (c) 617.905.2208 | (w) 781-313-8298 >> > >> > Acquia Dev Cloud: You build killer websites. We do the rest. >> > <http://www.acquia.com/dev-cloud> <http://acquia.com/dev-cloud> >> > Acquia ranked #1 Software Vendor on the 2012 Inc 500 >> > >> > <http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing> >> >> > _______________________________________________ >> > lxc-users mailing list >> > [email protected] >> > http://lists.linuxcontainers.org/listinfo/lxc-users >> >> _______________________________________________ >> lxc-users mailing list >> [email protected] >> http://lists.linuxcontainers.org/listinfo/lxc-users > > > > -- > Barry Jaspan > Senior Architect | Acquia > [email protected] | (c) 617.905.2208 | (w) 781-313-8298 > > Acquia Dev Cloud: You build killer websites. We do the rest. > Acquia ranked #1 Software Vendor on the 2012 Inc 500 > > > _______________________________________________ > lxc-users mailing list > [email protected] > http://lists.linuxcontainers.org/listinfo/lxc-users -- S.Çağlar Onur <[email protected]> _______________________________________________ lxc-users mailing list [email protected] http://lists.linuxcontainers.org/listinfo/lxc-users
