Re: [fix] Undeletable Dirs with Layered ro+wh Mounts

[Kazuya Yokogawa]

   I have similar problem, and I hope that it is the same cause.
   I created a test script that reproduces my problem.
   
   #!/bin/bash
   set -u
   set -e
   MERGE=${PWD}/merge
   LAYER01=${PWD}/layer01
   LAYER02=${PWD}/layer02
   LAYER03=${PWD}/layer03
   LAYER04=${PWD}/layer04
   mountpoint ${PWD}/merge >/dev/null && umount ${PWD}/merge
   for i in layer0{1..4} merge; do
   Â  rm -rf ${PWD}/${i}
   Â  mkdir -p ${PWD}/${i}
   done
   # layer01 / mkdir
   mkdir -vp ${LAYER01}/bar
   # layer02 / rmdir
   mount -t aufs -o br:${LAYER02}=rw:${LAYER01}=ro+wh none ${MERGE}
   rm -vrf ${MERGE}/bar
   umount ${MERGE}
   tree -a ${PWD}/layer* ${MERGE}
   # layer03 / mkdir
   mount -t aufs -o br:${LAYER03}=rw:${LAYER02}=ro+wh:${LAYER01}=ro+wh none
   ${MERGE}
   mkdir -vp ${MERGE}/bar
   umount ${MERGE}
   tree -a ${PWD}/layer* ${MERGE}
   # layer04 / touch and rmdir
   mount -t aufs -o
   br:${LAYER04}=rw:${LAYER03}=ro+wh:${LAYER02}=ro+wh:${LAYER01}=ro+whnone
   ${MERGE}
   ls -la ${MERGE}
   touch ${MERGE}/bar
   ls -la ${MERGE}
   rm -vrf ${MERGE}/bar
   # deleted ${MARGE}/bar. But exists.
   ls -la ${MERGE}
   
   I checked with folloing kernels.
   Â - debian kernel(3.16.0.4) + aufs package
   Â - vanilla(4.2.1)+patch(aufs-4.2-20150928[standalone]) .
   The other information is listed in the following.
   I want to solve this problem.
   You tell me if that there is information required for the other .
   
   debian-jessie:~/aufs-test# cat /proc/mounts
   sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
   proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
   udev /dev devtmpfs rw,relatime,size=10240k,nr_inodes=61036,mode=755 0 0
   devpts/dev/ptsdevptsrw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
   0 0
   tmpfs /run tmpfs rw,nosuid,relatime,size=101112k,mode=755 0 0
   /dev/sda1 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
   securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0
   0
   tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
   tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
   tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
   cgroup /sys/fs/cgroup/systemd cgroup
   rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgr
   oups-agent,name=systemd 0 0
   pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
   cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0
   0
   cgroup /sys/fs/cgroup/cpu,cpuacct cgroup
   rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
   cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
   cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0
   0
   cgroup/sys/fs/cgroup/devicescgrouprw,nosuid,nodev,noexec,relatime,devices
   0 0
   cgroup/sys/fs/cgroup/freezercgrouprw,nosuid,nodev,noexec,relatime,freezer
   0 0
   cgroup  /sys/fs/cgroup/net_cls,net_prio cgroup
   rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
   cgroup /sys/fs/cgroup/perf_event cgroup
   rw,nosuid,nodev,noexec,relatime,perf_event 0 0
   cgroup/sys/fs/cgroup/hugetlbcgrouprw,nosuid,nodev,noexec,relatime,hugetlb
   0 0
   systemd-1 /proc/sys/fs/binfmt_misc autofs
   rw,relatime,fd=22,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
   hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
   debugfs /sys/kernel/debug debugfs rw,relatime 0 0
   mqueue /dev/mqueue mqueue rw,relatime 0 0
   rpc_pipefs /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
   none /root/aufs-test/merge aufs rw,relatime,si=281b3cc259ea3b9 0 0
   debian-jessie:~/aufs-test# find /sys/module/aufs/ -type f | xargs head
   ==> /sys/module/aufs/parameters/brs <==
   1
   ==> /sys/module/aufs/parameters/debug <==
   0
   ==> /sys/module/aufs/parameters/sysrq <==
   a
   ==> /sys/module/aufs/parameters/allow_userns <==
   N
   ==> /sys/module/aufs/version <==
   4.2-20150928
   head: cannot open â/sys/module/aufs/ueventâ for reading: Permission denied
   debian-jessie:~/aufs-test# cat /proc/mounts
   sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
   proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
   udev /dev devtmpfs rw,relatime,size=10240k,nr_inodes=61036,mode=755 0 0
   devpts/dev/ptsdevptsrw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
   0 0
   tmpfs /run tmpfs rw,nosuid,relatime,size=101112k,mode=755 0 0
   /dev/sda1 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
   securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0
   0
   tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
   tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
   tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
   cgroup /sys/fs/cgroup/systemd cgroup
   rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgr
   oups-agent,name=systemd 0 0
   pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
   cgroup /sys/fs/cgroup/cpuset cgroup 

Re: [fix] Undeletable Dirs with Layered ro+wh Mounts

[sfjro]

Hello Yokogawa,

Kazuya Yokogawa:
> I have similar problem, and I hope that it is the same cause.
>
> I created a test script that reproduces my problem.

Thank you for the script.
Now I think I could understand the problem.
I will apply Karl Semich's fix and release on next Monday.


J. R. Okajima

--

Re: [fix] Undeletable Dirs with Layered ro+wh Mounts

[sfjro]

Hello Karl,

Karl Semich:
> - 3 layers of +wh under the rw layer on top
> - Bottom layer has folder A with files inside it
> - Second-to-bottom layer has a whiteout deleting folder A on bottom layer
> - Second-to-top layer has folder A recreated with files inside it
> - Go into rw top mount, remove all files from A.  Works fine.
> - Rmdir A.  Immediately reappears containing the files again.

Let me make sure.
- /u = /rwTOP + /ro+whONE + /ro+whMID + /ro+whBOT
- /ro+whBOT/dirA exists with its children
- /ro+whMID/.wh.dirA exists
- /ro+whONE/dirA exists with its children
- both of /rwTOP/dirA and /rwTOP/.wh.dirA don't exist
Right?

Here trying 'rmdir /u/dirA' should fail with ENOTEMPTY.

Or do you mean
- both of /rwTOP/dirA and /rwTOP/.wh.dirA don't exist
is wrong and
- /rwTOP/dirA exists with its children
- by-passing aufs 'rm /rwTOP/dirA/*' executed
is correct?
If so, did you try 'udba=notify'?

Next time when you post, please provide these info.

(from aufs README file)
--
When you have any problems or strange behaviour in aufs, please let me
know with:
- /proc/mounts (instead of the output of mount(8))
- /sys/module/aufs/*
- /sys/fs/aufs/* (if you have them)
- /debug/aufs/* (if you have them)
- linux kernel version
  if your kernel is not plain, for example modified by distributor,
  the url where i can download its source is necessary too.
- aufs version which was printed at loading the module or booting the
  system, instead of the date you downloaded.
- configuration (define/undefine CONFIG_AUFS_xxx)
- kernel configuration or /proc/config.gz (if you have it)
- behaviour which you think to be incorrect
- actual operation, reproducible one is better
- mailto: aufs-users at lists.sourceforge.net
--


J. R. Okajima

--