Am 22.02.2013 08:26, schrieb sf...@users.sourceforge.net: > Hello Eibo, > > Eibo Thieme: >> Sometimes I loose all mounted filesystems on the client, except for the >> aufs mounted root. It is reproducible by triggering a tab filename >> completion in bash, it can mostly be prevented by doing an ls on the >> aufs mount before switching root at startup. Sometimes, with no clear >> evidence on what provoked it, all mounts are lost anyways. Using X11, >> OpenOffice, Firefox and Thunderbird is apparently usually safe, >> especially work on the command line seems to trigger it. >> >> The Error: >> Pressing ls /pro followed by the tab key (or /hom or /us or /var) >> All mounted filesystems are empty afterwards
Correction: All mounted filesystems on mount points in root are empty afterwards. So /proc, /sys, /dev, /usr, /home, /var, /tmp, /swap are gone, but /aufs/xino, /aufs/private, ... are still there. > > Currently I have no idea about the cause. At least I am not alone. :) > As you know, procfs is outside of aufs. It means all the access to /proc > is not handled by aufs essentially. But the root dir is aufs. If aufs > tells you that /proc is brabra as a bogus result, then you will notice > something is wrong at once. > We may need to investigate focusing these points. > > - Is the root dir ok? In other words, is NFS ok? > All info returned by aufs is based upon its branch fs-es. You have 3 > NFS branches for the root dir. I'd suggest you to confirm them by > "ls -al /aufs/{private,clients,server}" which by-passes > aufs. Also "stat / /aufs/{private,clients,server}" and > "stat -f / /aufs/{private,clients,server}" may give us some hint. If I am not mistaken, NFS is ok. Everything on the root fs stays accessible after the error appears. You can even mount those lost directories again, if you happen to be root already. Here are the outputs of the commands, the output seems not to change after the error. ls -al /aufs/private === drwxr-xr-x 14 root root 4096 22. Feb 11:01 . drwxr-xr-x 8 root root 4096 28. Okt 11:02 .. drwxr-xr-x 3 root root 4096 27. Okt 19:10 boot drwxr-sr-x 4 root src 4096 28. Okt 14:46 dev drwxr-xr-x 15 root root 4096 22. Feb 10:56 etc drwxr-xr-x 2 root root 4096 19. Nov 14:23 export drwxr-xr-x 4 root root 4096 9. Jul 2007 lib drwxr-xr-x 2 root root 4096 22. Feb 10:52 media drwxr-xr-x 3 root root 4096 27. Okt 09:47 nfs drwxr-xr-x 12 root root 4096 21. Feb 16:24 root drwxr-xr-x 3 root root 4096 24. Jan 13:39 tmp drwxr-xr-x 3 root root 4096 28. Okt 15:32 var -r--r--r-- 25 root root 0 29. Feb 2008 .wh..wh.aufs drwx--x--x 2 root root 4096 8. Apr 2010 .wh..wh.orph drwxr-xr-x 2 root root 4096 21. Feb 15:44 .xino (actually I am wondering about those old whiteout files, must be leftovers of former installations) ls -al /aufs/clients === drwxr-xr-x 14 root root 4096 28. Okt 17:25 . drwxr-xr-x 8 root root 4096 28. Okt 11:02 .. drwxr-xr-x 8 root root 4096 28. Okt 11:02 aufs drwxr-xr-x 2 root root 4096 8. Apr 2008 .client-var drwxr-xr-x 2 root root 4096 8. Apr 2010 dev drwxr-xr-x 19 root root 4096 19. Nov 14:41 etc drwxr-xr-x 3 root root 4096 9. Jul 2007 lib drwxr-xr-x 3 root root 4096 30. Jun 2009 nfs lrwxrwxrwx 1 root root 8 24. Mär 2012 opt -> /usr/opt drwxr-xr-x 2 root root 4096 31. Mai 2008 sbin drwxr-xr-x 2 root root 4096 8. Apr 2008 .server-var drwxr-xr-x 2 root root 4096 7. Feb 2008 swap drwxr-xr-x 2 root root 4096 28. Okt 17:25 tmp drwxr-xr-x 4 root root 4096 28. Okt 14:42 var drwxr-xr-x 2 root root 4096 18. Feb 2008 .xino ls -al /aufs/server === drwxr-xr-x 24 root root 4096 22. Feb 11:17 . drwxr-xr-x 8 root root 4096 28. Okt 11:02 .. drwxrwxr-x 2 hti users 4096 8. Mär 2012 ablage-hti drwxrwxr-x 2 hti users 4096 1. Mär 2012 ablage-rtp drwxr-xr-x 2 root root 4096 24. Feb 2011 archiv drwxr-xr-x 2 root root 4096 10. Nov 12:07 bin drwxr-xr-x 4 root root 4096 11. Nov 12:50 boot drwxr-xr-x 8 root root 36864 2. Aug 2012 dev drwxr-xr-x 203 root root 20480 21. Feb 14:17 etc drwxr-xr-x 2 root root 4096 27. Mär 2011 export drwxr-xr-x 2 root root 4096 2. Feb 2012 home drwxr-xr-x 13 root root 12288 13. Dez 14:20 lib drwxr-xr-x 2 root root 16384 10. Jul 2003 lost+found drwxr-xr-x 2 root root 4096 24. Mär 2012 media drwxr-xr-x 2 root root 4096 18. Sep 2003 mnt lrwxrwxrwx 1 root root 8 24. Mär 2012 opt -> /usr/opt dr-xr-xr-x 2 root root 4096 24. Mär 2012 proc drwx--x--x 46 root root 4096 21. Feb 15:02 root drwxr-xr-x 2 root root 4096 10. Nov 12:07 sbin drwxr-xr-x 2 root root 4096 7. Mär 2007 selinux lrwxrwxrwx 1 root root 13 24. Mär 2012 share -> /export/share -rw-r--r-- 1 root root 0 2. Aug 2012 SQUEEZE drwxr-xr-x 3 root root 4096 25. Mär 2012 srv drwxr-xr-x 2 root root 4096 24. Mär 2012 sys drwxrwxrwt 7 root root 4096 10. Nov 17:41 tmp drwxr-xr-x 2 root root 4096 16. Sep 2010 usr drwxr-xr-x 4 root root 4096 2. Aug 2012 var stat / === File: „/“ Size: 4096 Blocks: 8 IO Block: 32768 Verzeichnis Device: 10h/16d Inode: 2 Links: 48 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2013-02-22 11:13:49.000000000 +0100 Modify: 2013-02-22 11:01:36.000000000 +0100 Change: 2013-02-22 11:01:36.000000000 +0100 stat /aufs/private === File: „/aufs/private“ Size: 4096 Blocks: 8 IO Block: 32768 Verzeichnis Device: dh/13d Inode: 944705 Links: 14 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2013-02-22 11:13:49.000000000 +0100 Modify: 2013-02-22 11:01:36.000000000 +0100 Change: 2013-02-22 11:01:36.000000000 +0100 stat /aufs/clients === File: „/aufs/clients“ Size: 4096 Blocks: 8 IO Block: 32768 Verzeichnis Device: eh/14d Inode: 146593 Links: 13 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2013-02-22 12:04:20.000000000 +0100 Modify: 2013-02-22 12:04:17.000000000 +0100 Change: 2013-02-22 12:04:17.000000000 +0100 stat /aufs/server === File: „/aufs/server“ Size: 4096 Blocks: 8 IO Block: 32768 Verzeichnis Device: fh/15d Inode: 2 Links: 24 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2013-02-22 11:18:00.000000000 +0100 Modify: 2013-02-22 11:17:41.000000000 +0100 Change: 2013-02-22 11:17:41.000000000 +0100 stat -f / === File: "/" ID: 0 Namelen: 242 Type: UNKNOWN (0x61756673) Block size: 32768 Fundamental block size: 32768 Blocks: Total: 600881 Free: 394835 Available: 364312 Inodes: Total: 1221600 Free: 903633 stat -f /aufs/private === File: "/aufs/private" ID: 0 Namelen: 255 Type: nfs Block size: 32768 Fundamental block size: 32768 Blocks: Total: 600881 Free: 394835 Available: 364312 Inodes: Total: 1221600 Free: 903633 stat -f /aufs/clients === File: "/aufs/clients" ID: 0 Namelen: 255 Type: nfs Block size: 32768 Fundamental block size: 32768 Blocks: Total: 600881 Free: 394835 Available: 364312 Inodes: Total: 1221600 Free: 903633 stat -f /aufs/server === File: "/aufs/server" ID: 0 Namelen: 255 Type: nfs Block size: 32768 Fundamental block size: 32768 Blocks: Total: 30138 Free: 21337 Available: 19806 Inodes: Total: 61312 Free: 44025 > - Is the mount tree traversal ok? > I guess there is nothing wrong in this point, but I'd just make sure. > Try "ls /proc" before and after "mount --move". > Also "cat /proc/mounts" before and after "mount --move" would be good. As I have no direct access to the machines at the moment, it is a bit difficult to provide this information at the moment, but I had someone to check. ls /proc in the initramfs is giving a plausible listing, ls /proc in runlevel 2 is also ok, as long as I don't trigger the error. cat /proc/mounts before "mount --move" === rootfs / rootfs rw 0 0 none /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 none /proc proc rw,nosuid,nodev,noexec,relatime 0 0 none /aufs/xino tmpfs rw,relatime,size=16384k 0 0 192.168.0.1:/target/export/hti13/ /aufs/private nfs rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,nolock,proto=udp,timeo=11,retrans=10,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=all,addr=192.168.0.1 0 0 192.168.0.1:/target/export/clients/ /aufs/clients nfs ro,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,nolock,proto=udp,timeo=11,retrans=10,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=all,addr=192.168.0.1 0 0 192.168.0.1:/target/ /aufs/server nfs ro,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,nolock,proto=udp,timeo=11,retrans=10,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=all,addr=192.168.0.1 0 0 none /root aufs rw,relatime,si=ae5bae97,noplink 0 0 cat /proc/mounts after booting to runlevel 2 (way past mount --move): === rootfs / rootfs rw 0 0 none /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 none /proc proc rw,nosuid,nodev,noexec,relatime 0 0 none /aufs/xino tmpfs rw,relatime,size=16384k 0 0 192.168.0.1:/target/export/hti13/ /aufs/private nfs rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,nolock,proto=udp,timeo=11,retrans=10,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=all,addr=192.168.0.1 0 0 192.168.0.1:/target/export/clients/ /aufs/clients nfs ro,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,nolock,proto=udp,timeo=11,retrans=10,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=all,addr=192.168.0.1 0 0 192.168.0.1:/target/ /aufs/server nfs ro,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,nolock,proto=udp,timeo=11,retrans=10,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=all,addr=192.168.0.1 0 0 none / aufs rw,relatime,si=ae5bae97,noplink 0 0 tmpfs /lib/init/rw tmpfs rw,nosuid,relatime,mode=755 0 0 varrun /var/run tmpfs rw,nosuid,relatime,mode=755 0 0 varlock /var/lock tmpfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev tmpfs rw,relatime,size=10240k,mode=755 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,relatime 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620 0 0 192.168.0.1:/target/usr/ /usr nfs rw,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=none,addr=192.168.0.1 0 0 192.168.0.1:/target/var/ /aufs/server-var nfs ro,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=none,addr=192.168.0.1 0 0 192.168.0.1:/target/export/var/hti13/ /aufs/client-var nfs rw,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=none,addr=192.168.0.1 0 0 192.168.0.1:/target/export/tmp/hti13/ /tmp nfs rw,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=none,addr=192.168.0.1 0 0 192.168.0.1:/target/export/swaps/hti13/ /swap nfs rw,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=none,addr=192.168.0.1 0 0 192.168.0.1:/home/ /home nfs rw,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.1,mountvers=3,mountport=58461,mountproto=udp,local_lock=none,addr=192.168.0.1 0 0 none /var aufs rw,relatime,si=af17bf97 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 > > - Is procfs ok? > This is just-to-make-sure point too. Try "stat /proc; stat -f /proc". Before error: ls -la /proc before error === dr-xr-xr-x 73 root root 0 22. Feb 12:32 . drwxr-xr-x 47 root root 4096 22. Feb 12:50 .. dr-xr-xr-x 7 root root 0 22. Feb 12:49 1 dr-xr-xr-x 7 root root 0 22. Feb 12:49 10 dr-xr-xr-x 7 daemon daemon 0 22. Feb 12:55 1014 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1039 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1079 dr-xr-xr-x 7 root root 0 22. Feb 12:50 1084 dr-xr-xr-x 7 root root 0 22. Feb 12:49 11 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1104 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1105 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1106 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1107 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1108 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1109 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1112 dr-xr-xr-x 7 root root 0 22. Feb 12:50 1113 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1117 dr-xr-xr-x 7 root root 0 22. Feb 12:55 1118 dr-xr-xr-x 7 root root 0 22. Feb 12:51 1126 dr-xr-xr-x 7 eibo root 0 22. Feb 12:55 1129 dr-xr-xr-x 7 root root 0 22. Feb 12:51 1131 dr-xr-xr-x 7 eibo users 0 22. Feb 12:55 1199 dr-xr-xr-x 7 root root 0 22. Feb 12:49 12 dr-xr-xr-x 7 eibo users 0 22. Feb 12:51 1200 dr-xr-xr-x 7 eibo users 0 22. Feb 13:02 1204 dr-xr-xr-x 7 root root 0 22. Feb 12:49 13 dr-xr-xr-x 7 root root 0 22. Feb 12:49 139 dr-xr-xr-x 7 root root 0 22. Feb 12:49 14 dr-xr-xr-x 7 root root 0 22. Feb 12:49 15 dr-xr-xr-x 7 root root 0 22. Feb 12:49 16 dr-xr-xr-x 7 root root 0 22. Feb 12:49 17 dr-xr-xr-x 7 root root 0 22. Feb 12:49 18 dr-xr-xr-x 7 root root 0 22. Feb 12:49 19 dr-xr-xr-x 7 root root 0 22. Feb 12:49 2 dr-xr-xr-x 7 root root 0 22. Feb 12:49 20 dr-xr-xr-x 7 root root 0 22. Feb 12:49 21 dr-xr-xr-x 7 root root 0 22. Feb 12:49 282 dr-xr-xr-x 7 root root 0 22. Feb 12:49 293 dr-xr-xr-x 7 root root 0 22. Feb 12:49 297 dr-xr-xr-x 7 root root 0 22. Feb 12:49 3 dr-xr-xr-x 7 root root 0 22. Feb 12:49 352 dr-xr-xr-x 7 root root 0 22. Feb 12:49 5 dr-xr-xr-x 7 daemon daemon 0 22. Feb 12:49 554 dr-xr-xr-x 7 root root 0 22. Feb 12:49 561 dr-xr-xr-x 7 root root 0 22. Feb 12:49 572 dr-xr-xr-x 7 root root 0 22. Feb 12:49 6 dr-xr-xr-x 7 root root 0 22. Feb 12:49 698 dr-xr-xr-x 7 root root 0 22. Feb 12:49 7 dr-xr-xr-x 7 root root 0 22. Feb 12:49 707 dr-xr-xr-x 7 messagebus messagebus 0 22. Feb 12:49 721 dr-xr-xr-x 7 root root 0 22. Feb 12:49 743 dr-xr-xr-x 7 root root 0 22. Feb 12:49 761 dr-xr-xr-x 7 root root 0 22. Feb 12:49 8 dr-xr-xr-x 7 root root 0 22. Feb 12:55 868 dr-xr-xr-x 7 postfix postfix 0 22. Feb 12:55 881 dr-xr-xr-x 7 postfix postfix 0 22. Feb 12:55 882 dr-xr-xr-x 7 root root 0 22. Feb 12:55 883 dr-xr-xr-x 7 ntp office 0 22. Feb 12:50 887 dr-xr-xr-x 7 root root 0 22. Feb 12:49 9 dr-xr-xr-x 7 haldaemon haldaemon 0 22. Feb 12:50 904 dr-xr-xr-x 7 root root 0 22. Feb 12:50 905 dr-xr-xr-x 7 haldaemon haldaemon 0 22. Feb 12:50 936 dr-xr-xr-x 7 root root 0 22. Feb 12:55 961 dr-xr-xr-x 2 root root 0 22. Feb 12:50 acpi dr-xr-xr-x 5 root root 0 22. Feb 13:02 asound -r--r--r-- 1 root root 0 22. Feb 13:02 buddyinfo dr-xr-xr-x 4 root root 0 22. Feb 13:02 bus -r--r--r-- 1 root root 0 22. Feb 13:02 cmdline -r--r--r-- 1 root root 13753 22. Feb 13:02 config.gz -r--r--r-- 1 root root 0 22. Feb 13:02 consoles -r--r--r-- 1 root root 0 22. Feb 13:02 cpuinfo -r--r--r-- 1 root root 0 22. Feb 13:02 devices -r--r--r-- 1 root root 0 22. Feb 13:02 diskstats -r--r--r-- 1 root root 0 22. Feb 13:02 dma dr-xr-xr-x 3 root root 0 22. Feb 13:02 driver -r--r--r-- 1 root root 0 22. Feb 13:02 execdomains -r--r--r-- 1 root root 0 22. Feb 13:02 filesystems dr-xr-xr-x 5 root root 0 22. Feb 13:02 fs -r--r--r-- 1 root root 0 22. Feb 13:02 interrupts -r--r--r-- 1 root root 0 22. Feb 13:02 iomem -r--r--r-- 1 root root 0 22. Feb 13:02 ioports dr-xr-xr-x 23 root root 0 22. Feb 13:02 irq -r--r--r-- 1 root root 0 22. Feb 13:02 kallsyms -r-------- 1 root root 0 22. Feb 12:49 kmsg -r-------- 1 root root 0 22. Feb 13:02 kpagecount -r-------- 1 root root 0 22. Feb 13:02 kpageflags -r--r--r-- 1 root root 0 22. Feb 13:02 loadavg -r--r--r-- 1 root root 0 22. Feb 13:02 locks -r--r--r-- 1 root root 0 22. Feb 13:02 meminfo -r--r--r-- 1 root root 0 22. Feb 13:02 misc -r--r--r-- 1 root root 0 22. Feb 13:02 modules lrwxrwxrwx 1 root root 11 22. Feb 13:02 mounts -> self/mounts -rw-r--r-- 1 root root 0 22. Feb 12:50 mtrr lrwxrwxrwx 1 root root 8 22. Feb 13:02 net -> self/net -r--r--r-- 1 root root 0 22. Feb 13:02 pagetypeinfo -r--r--r-- 1 root root 0 22. Feb 13:02 partitions lrwxrwxrwx 1 root root 64 22. Feb 12:32 self -> 1204 -r-------- 1 root root 0 22. Feb 13:02 slabinfo -r--r--r-- 1 root root 0 22. Feb 13:02 softirqs -r--r--r-- 1 root root 0 22. Feb 13:02 stat -r--r--r-- 1 root root 0 22. Feb 13:02 swaps dr-xr-xr-x 1 root root 0 22. Feb 12:49 sys dr-xr-xr-x 2 root root 0 22. Feb 13:02 sysvipc -r--r--r-- 1 root root 0 22. Feb 13:02 timer_list dr-xr-xr-x 4 root root 0 22. Feb 13:02 tty -r--r--r-- 1 root root 0 22. Feb 13:02 uptime -r--r--r-- 1 root root 0 22. Feb 13:02 version -r-------- 1 root root 0 22. Feb 13:02 vmallocinfo -r--r--r-- 1 root root 0 22. Feb 13:02 vmstat -r--r--r-- 1 root root 0 22. Feb 13:02 zoneinfo stat /proc before error === File: „/proc“ Size: 0 Blocks: 0 IO Block: 1024 Verzeichnis Device: 3h/3d Inode: 1 Links: 76 Access: (0555/dr-xr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2013-02-22 12:32:48.000999848 +0100 Modify: 2013-02-22 12:32:48.000999848 +0100 Change: 2013-02-22 12:32:48.000999848 +0100 stat -f /proc before error === File: "/proc" ID: 0 Namelen: 255 Type: proc Block size: 4096 Fundamental block size: 4096 Blocks: Total: 0 Free: 0 Available: 0 Inodes: Total: 0 Free: 0 After error: ls -la /proc after error === insgesamt 8 dr-xr-xr-x 2 root root 4096 24. Mär 2012 . drwxr-xr-x 48 root root 4096 22. Feb 11:01 .. stat /proc after error === File: „/proc“ Size: 4096 Blocks: 8 IO Block: 32768 Verzeichnis Device: 10h/16d Inode: 17 Links: 2 Access: (0555/dr-xr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2013-02-21 15:41:15.000000000 +0100 Modify: 2012-03-24 13:01:37.000000000 +0100 Change: 2012-03-24 14:48:40.000000000 +0100 stat -f /proc after error === File: "/proc" ID: 0 Namelen: 242 Type: UNKNOWN (0x61756673) Block size: 32768 Fundamental block size: 32768 Blocks: Total: 600881 Free: 394835 Available: 364312 Inodes: Total: 1221600 Free: 903632 > We may need to put some debug print into your kernel. If I write such > patch, will you apply and test it? Any such patches would be most welcome, I feel quite comfortable patching the kernel, even manually, it's only I am not advanced enough coming up with any sensible places to look at in this case. :) If possible I would prefer the testing on a kernel with an aufs.ko kernel module, other than that I could switch to any newer kernel, if that would be helpful. > And... > - your /proc/mounts shows there is only one aufs (the root dir), but > /sys/fs/aufs shows there two aufs mounts (root and var?) > Is this intentinal? Actually, I use two aufs mounts, the root mount is the setup you have seen with private on top of general client on top of server root. This gives me an amazingly simple way to administrate these machines, I can install anything on the server and it just works on the clients. Adding a client is a trivial. Over the years I have come to the conclusion, that to keep it simple, I have to use aufs for /var as well, reducing the usage a bit by remote logging to the server. The mounting of /var is done by an init script, which I should have given in my first post. There is one phenonmenon here, as both mounts /aufs/server-var and /aufs/client-var are still there, even after mounting. It might be that only mounts on mountpoints in the root directory are affected. I will try to check this out in more detail. =================================================== # # disklessmount.sh Mount all filesystems. # # Version: @(#)disklessmount.sh 0.9-9 19-Nov-2012 e...@plusgut.de # VERBOSE=yes [ -f /etc/default/rcS ] && . /etc/default/rcS # # /var/run /var/lock tmpfs # #mount -t tmpfs -omode=0755,nosuid none /var/run #mount -t tmpfs -omode=1777,nodev,noexec,nosuid none /var/lock # # starting portmap # needed by nfs (eibo) # #echo -n "Starting portmap daemon: portmap" #start-stop-daemon --start --quiet --exec /sbin/portmap #echo "." #sleep 1 #/sbin/rpc.statd -P /nfs mkdir /aufs/xino/run mount --move /var/run /aufs/xino/run mkdir /aufs/xino/lock mount --move /var/lock /aufs/xino/lock # # Mount remote file systems in /etc/fstab. # [ "$VERBOSE" != no ] && echo "Mounting remote filesystems..." # alte mtab ist falsch cat /proc/mounts > /etc/mtab.sav mv /etc/mtab.sav /etc/mtab mount -a -t nfs # # Var mit Overlay # mount -t aufs -o br:/aufs/client-var=rw:/aufs/server-var=ro,xino=/aufs/xino/var none /var mount --move /aufs/xino/run /var/run rmdir /aufs/xino/run mount --move /aufs/xino/lock /var/lock rmdir /aufs/xino/lock # # Swap over NFS # swapon -a : exit 0 =================================================== > - I could not get your debug output, > > https://docs.google.com/file/d/0B0s7uiDVoB3OV09wNXZrM0Uyc2s/edit?usp=sharing > since I don't have google drive account. You should not need an account, try this slightly modified link instead: https://docs.google.com/uc?id=0B0s7uiDVoB3OV09wNXZrM0Uyc2s&export=download Or try this link: https://www.dropbox.com/s/13vxwegryhp3iia/aufs.debug Eibo > > J. R. Okajima > ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb