I have similar problem, and I hope that it is the same cause.
I created a test script that reproduces my problem.
----
#!/bin/bash
set -u
set -e
MERGE=${PWD}/merge
LAYER01=${PWD}/layer01
LAYER02=${PWD}/layer02
LAYER03=${PWD}/layer03
LAYER04=${PWD}/layer04
mountpoint ${PWD}/merge >/dev/null && umount ${PWD}/merge
for i in layer0{1..4} merge; do
 rm -rf ${PWD}/${i}
 mkdir -p ${PWD}/${i}
done
# layer01 / mkdir
mkdir -vp ${LAYER01}/bar
# layer02 / rmdir
mount -t aufs -o br:${LAYER02}=rw:${LAYER01}=ro+wh none ${MERGE}
rm -vrf ${MERGE}/bar
umount ${MERGE}
tree -a ${PWD}/layer* ${MERGE}
# layer03 / mkdir
mount -t aufs -o br:${LAYER03}=rw:${LAYER02}=ro+wh:${LAYER01}=ro+wh none
${MERGE}
mkdir -vp ${MERGE}/bar
umount ${MERGE}
tree -a ${PWD}/layer* ${MERGE}
# layer04 / touch and rmdir
mount -t aufs -o
br:${LAYER04}=rw:${LAYER03}=ro+wh:${LAYER02}=ro+wh:${LAYER01}=ro+whnone
${MERGE}
ls -la ${MERGE}
touch ${MERGE}/bar
ls -la ${MERGE}
rm -vrf ${MERGE}/bar
# deleted ${MARGE}/bar. But exists.
ls -la ${MERGE}
----
I checked with folloing kernels.
 - debian kernel(3.16.0.4) + aufs package
 - vanilla(4.2.1)+patch(aufs-4.2-20150928[standalone]) .
The other information is listed in the following.
I want to solve this problem.
You tell me if that there is information required for the other .
----
debian-jessie:~/aufs-test# cat /proc/mounts
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=10240k,nr_inodes=61036,mode=755 0 0
devpts/dev/ptsdevptsrw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
0 0
tmpfs /run tmpfs rw,nosuid,relatime,size=101112k,mode=755 0 0
/dev/sda1 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0
0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup
rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgr
oups-agent,name=systemd 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0
0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup
rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0
0
cgroup/sys/fs/cgroup/devicescgrouprw,nosuid,nodev,noexec,relatime,devices
0 0
cgroup/sys/fs/cgroup/freezercgrouprw,nosuid,nodev,noexec,relatime,freezer
0 0
cgroup /sys/fs/cgroup/net_cls,net_prio cgroup
rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup
rw,nosuid,nodev,noexec,relatime,perf_event 0 0
cgroup/sys/fs/cgroup/hugetlbcgrouprw,nosuid,nodev,noexec,relatime,hugetlb
0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs
rw,relatime,fd=22,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
mqueue /dev/mqueue mqueue rw,relatime 0 0
rpc_pipefs /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
none /root/aufs-test/merge aufs rw,relatime,si=281b3cc259ea3b9 0 0
debian-jessie:~/aufs-test# find /sys/module/aufs/ -type f | xargs head
==> /sys/module/aufs/parameters/brs <==
1
==> /sys/module/aufs/parameters/debug <==
0
==> /sys/module/aufs/parameters/sysrq <==
a
==> /sys/module/aufs/parameters/allow_userns <==
N
==> /sys/module/aufs/version <==
4.2-20150928
head: cannot open â/sys/module/aufs/ueventâ for reading: Permission denied
debian-jessie:~/aufs-test# cat /proc/mounts
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=10240k,nr_inodes=61036,mode=755 0 0
devpts/dev/ptsdevptsrw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
0 0
tmpfs /run tmpfs rw,nosuid,relatime,size=101112k,mode=755 0 0
/dev/sda1 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0
0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup
rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgr
oups-agent,name=systemd 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0
0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup
rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0
0
cgroup/sys/fs/cgroup/devicescgrouprw,nosuid,nodev,noexec,relatime,devices
0 0
cgroup/sys/fs/cgroup/freezercgrouprw,nosuid,nodev,noexec,relatime,freezer
0 0
cgroup /sys/fs/cgroup/net_cls,net_prio cgroup
rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup
rw,nosuid,nodev,noexec,relatime,perf_event 0 0
cgroup/sys/fs/cgroup/hugetlbcgrouprw,nosuid,nodev,noexec,relatime,hugetlb
0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs
rw,relatime,fd=22,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
mqueue /dev/mqueue mqueue rw,relatime 0 0
rpc_pipefs /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
none /root/aufs-test/merge aufs rw,relatime,si=281b3cc259ea3b9 0 0
debian-jessie:~/aufs-test# find /sys/module/aufs/ -type f | xargs head
==> /sys/module/aufs/parameters/brs <==
1
==> /sys/module/aufs/parameters/debug <==
0
==> /sys/module/aufs/parameters/sysrq <==
a
==> /sys/module/aufs/parameters/allow_userns <==
N
==> /sys/module/aufs/version <==
4.2-20150928
head: cannot open â/sys/module/aufs/ueventâ for reading: Permission denied
debian-jessie:~/aufs-test# find /sys/fs/aufs/ -type f | xargs head
==> /sys/fs/aufs/si_281b3cc259ea3b9/br0 <==
/root/aufs-test/layer04=rw
==> /sys/fs/aufs/si_281b3cc259ea3b9/br1 <==
/root/aufs-test/layer03=ro+wh
==> /sys/fs/aufs/si_281b3cc259ea3b9/br2 <==
/root/aufs-test/layer02=ro+wh
==> /sys/fs/aufs/si_281b3cc259ea3b9/br3 <==
/root/aufs-test/layer01=ro+wh
==> /sys/fs/aufs/si_281b3cc259ea3b9/xi_path <==
/root/aufs-test/layer04/.aufs.xino
==> /sys/fs/aufs/si_281b3cc259ea3b9/brid0 <==
512
==> /sys/fs/aufs/si_281b3cc259ea3b9/brid1 <==
513
==> /sys/fs/aufs/si_281b3cc259ea3b9/brid2 <==
514
==> /sys/fs/aufs/si_281b3cc259ea3b9/brid3 <==
515
debian-jessie:~/aufs-test# uname -a
Linux debian-jessie 4.2.1-2-aufs #1 SMP Mon Sep 28 22:01:19 JST 2015 x86_64
GNU/Linux
debian-jessie:~/aufs-test# zgrep CONFIG_AUFS /proc/config.gz
CONFIG_AUFS_FS=y
# CONFIG_AUFS_BRANCH_MAX_127 is not set
# CONFIG_AUFS_BRANCH_MAX_511 is not set
CONFIG_AUFS_BRANCH_MAX_1023=y
# CONFIG_AUFS_BRANCH_MAX_32767 is not set
CONFIG_AUFS_SBILIST=y
CONFIG_AUFS_HNOTIFY=y
CONFIG_AUFS_HFSNOTIFY=y
CONFIG_AUFS_EXPORT=y
CONFIG_AUFS_INO_T_64=y
CONFIG_AUFS_XATTR=y
CONFIG_AUFS_FHSM=y
CONFIG_AUFS_RDU=y
CONFIG_AUFS_SHWH=y
CONFIG_AUFS_BR_RAMFS=y
CONFIG_AUFS_BR_FUSE=y
CONFIG_AUFS_POLL=y
CONFIG_AUFS_BR_HFSPLUS=y
CONFIG_AUFS_BDEV_LOOP=y
CONFIG_AUFS_DEBUG=y
CONFIG_AUFS_MAGIC_SYSRQ=y
----
debug_log
----
rm -vrf ${MERGE}/bar
kernel: aufs h_d_revalidate:853:rm[1086]: DEBUG: b0, bar
kernel: aufs h_d_revalidate:853:rm[1086]: DEBUG: b1, bar
kernel: aufs h_d_revalidate:853:rm[1086]: DEBUG: b0, bar
kernel: aufs h_d_revalidate:853:rm[1086]: DEBUG: b1, bar
kernel: aufs aufs_iterate:473:rm[1086]: DEBUG: bar, ctx{filldir, 0}
kernel: aufs vfsub_iterate_dir:543:rm[1086]: DEBUG: bar, ctx{fillvdir, 0}
kernel: aufs vfsub_iterate_dir:543:rm[1086]: DEBUG: bar, ctx{fillvdir,
9223372036854775807}
kernel: aufs vfsub_iterate_dir:543:rm[1086]: DEBUG: bar, ctx{fillvdir,
9223372036854775807}
kernel: aufs test_known:223:rm[1086]: DEBUG: ..
kernel: aufs test_known:223:rm[1086]: DEBUG: .
kernel: aufs vfsub_iterate_dir:543:rm[1086]: DEBUG: bar, ctx{fillvdir,
9223372036854775807}
kernel: aufs seek_vdir:801:rm[1086]: DEBUG: offset 0
kernel: aufs au_vdir_fill_de:863:rm[1086]: DEBUG: .., off0, i2, dt4
kernel: aufs au_vdir_fill_de:863:rm[1086]: DEBUG: ., off16, i12, dt4
kernel: aufs aufs_iterate:473:rm[1086]: DEBUG: bar, ctx{filldir, 32}
kernel: aufs seek_vdir:801:rm[1086]: DEBUG: offset 32
kernel: aufs h_d_revalidate:853:rm[1086]: DEBUG: b0, bar
kernel: aufs h_d_revalidate:853:rm[1086]: DEBUG: b1, bar
kernel: aufs vfsub_iterate_dir:543:rm[1086]: DEBUG: bar, ctx{test_empty_cb,
0}
kernel: aufs vfsub_iterate_dir:543:rm[1086]: DEBUG: bar, ctx{test_empty_cb,
9223372036854775807}
kernel: aufs vfsub_iterate_dir:543:rm[1086]: DEBUG: bar, ctx{test_empty_cb,
9223372036854775807}
kernel: aufs vfsub_iterate_dir:543:rm[1086]: DEBUG: bar, ctx{test_empty_cb,
9223372036854775807}
kernel: aufs au_wr_dir_need_wh:53:rm[1086]: DEBUG: bcpup 0, bstart 0
kernel: aufs au_wr_dir_need_wh:76:rm[1086]: DEBUG: need_wh 0
----
2015å¹´10æ6æ¥(ç«) 10:18 <[1][email protected]>:
Hello Karl,
Karl Semich:
> - 3 layers of +wh under the rw layer on top
> - Bottom layer has folder A with files inside it
> - Second-to-bottom layer has a whiteout deleting folder A on bottom
layer
> - Second-to-top layer has folder A recreated with files inside it
> - Go into rw top mount, remove all files from A. Works fine.
> - Rmdir A. Immediately reappears containing the files again.
Let me make sure.
- /u = /rwTOP + /ro+whONE + /ro+whMID + /ro+whBOT
- /ro+whBOT/dirA exists with its children
- /ro+whMID/.wh.dirA exists
- /ro+whONE/dirA exists with its children
- both of /rwTOP/dirA and /rwTOP/.wh.dirA don't exist
Right?
Here trying 'rmdir /u/dirA' should fail with ENOTEMPTY.
Or do you mean
- both of /rwTOP/dirA and /rwTOP/.wh.dirA don't exist
is wrong and
- /rwTOP/dirA exists with its children
- by-passing aufs 'rm /rwTOP/dirA/*' executed
is correct?
If so, did you try 'udba=notify'?
Next time when you post, please provide these info.
(from aufs README file)
----------------------------------------------------------------------
When you have any problems or strange behaviour in aufs, please let me
know with:
- /proc/mounts (instead of the output of mount(8))
- /sys/module/aufs/*
- /sys/fs/aufs/* (if you have them)
- /debug/aufs/* (if you have them)
- linux kernel version
 if your kernel is not plain, for example modified by distributor,
 the url where i can download its source is necessary too.
- aufs version which was printed at loading the module or booting the
 system, instead of the date you downloaded.
- configuration (define/undefine CONFIG_AUFS_xxx)
- kernel configuration or /proc/config.gz (if you have it)
- behaviour which you think to be incorrect
- actual operation, reproducible one is better
- mailto: aufs-users at [2]lists.sourceforge.net
----------------------------------------------------------------------
J. R. Okajima
--------------------------------------------------------------------------
----
References
1. mailto:[email protected]
2. http://lists.sourceforge.net/
------------------------------------------------------------------------------
