- I'm unable to write via nfs to an aufs union of zfs branches. The file is created, but nothing is written to the file, and nfs reports errno -38 (ENOSYS). - Writing directly to the aufs union (ie, bypass the nfs mount) works. - Reading via the nfs mount seems to work. - Writing via nfs to aufs union of EXT4 branches works. - Writing via nfs to zfs (ie, without aufs union) works.
So it seems that there is something special in the mix of nfs + aufs + zfs that causes this to fail. Any idea what causes this? This is what I've done to reproduce it. # # create and mount zfs branches in /d1 and /d2 and union in /ds # dd if=/dev/zero of=/run/shm/d1 bs=1024768 count=100 # dd if=/dev/zero of=/run/shm/d2 bs=1024768 count=100 # zpool create d1 /run/shm/d1 # zpool create d2 /run/shm/d2 # mount -t aufs none /ds -o br:/d1=rw:/d2=rw,xino=/run/shm/ds.aufs.xino # chmod 777 /d[12s] # exportfs -v|grep /ds /ds <world>(rw,wdelay,root_squash,all_squash,no_subtree_check,fsid=4,anonuid=998,anongid=999) # mount -t nfs 127.0.0.1:/ds /mnt # echo helloworld > /mnt/abc # echo helloworld > /ds/def # ls -l /mnt total 2 -rw-r--r-- 1 share share 0 Sep 28 21:13 abc -rw-r--r-- 1 root root 11 Sep 28 21:14 def # ls -l /d? /d1: total 2 -rw-r--r-- 1 share share 0 Sep 28 21:13 abc -rw-r--r-- 1 root root 11 Sep 28 21:14 def /d2: total 0 /ds: total 2 -rw-r--r-- 1 share share 0 Sep 28 21:13 abc -rw-r--r-- 1 root root 11 Sep 28 21:14 def # tail -1 /var/log/syslog Sep 28 21:13:48 storage kernel: [75393.117716] nfsd: non-standard errno: -38 # grep /d /proc/mounts d1 /d1 zfs rw,relatime,xattr 0 0 d2 /d2 zfs rw,relatime,xattr 0 0 none /ds aufs rw,relatime,si=350e142b1cf01770 0 0 127.0.0.1:/ds /mnt nfs4 rw,relatime,vers=4,rsize=1048576,wsize=1048576,namlen=242,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=127.0.0.1,minorversion=0,local_lock=none,addr=127.0.0.1 0 0 /sys/module/aufs/* available on request, not sure what is useful /sys/module/aufs/parameters/brs: 1 /sys/module/aufs/srcversion: 523C2808903A77806346A06 /sys/module/aufs/version: 3.2-20130204 # cat /sys/fs/aufs/config CONFIG_AUFS_FS=m CONFIG_AUFS_BRANCH_MAX_127=y CONFIG_AUFS_SBILIST=y CONFIG_AUFS_EXPORT=y CONFIG_AUFS_INO_T_64=y CONFIG_AUFS_BR_HFSPLUS=y CONFIG_AUFS_BDEV_LOOP=y # cat /sys/fs/aufs/si_350e142b1cf01770/* /d1=rw /d2=rw /run/shm/ds.aufs.xino # cat $debugfs/aufs/si_350e142b1cf01770/* 1, 8x4096 104 1, 8x4096 40 8x4096 4096 8x4096 52 Debian wheezy kernel: 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux; sources available at http://packages.debian.org/wheezy/linux-image-3.2.0-4-amd64 aufs: module is from the staging directory, the quality is unknown, you have been warned. aufs 3.2-20130204 # grep -i aufs /boot/config-3.2.0-4-amd64 CONFIG_AUFS_FS=m CONFIG_AUFS_BRANCH_MAX_127=y # CONFIG_AUFS_BRANCH_MAX_511 is not set # CONFIG_AUFS_BRANCH_MAX_1023 is not set # CONFIG_AUFS_BRANCH_MAX_32767 is not set CONFIG_AUFS_SBILIST=y # CONFIG_AUFS_HNOTIFY is not set CONFIG_AUFS_EXPORT=y CONFIG_AUFS_INO_T_64=y # CONFIG_AUFS_RDU is not set # CONFIG_AUFS_PROC_MAP is not set # CONFIG_AUFS_SP_IATTR is not set # CONFIG_AUFS_SHWH is not set # CONFIG_AUFS_BR_RAMFS is not set # CONFIG_AUFS_BR_FUSE is not set CONFIG_AUFS_BR_HFSPLUS=y CONFIG_AUFS_BDEV_LOOP=y # CONFIG_AUFS_DEBUG is not set Other packages from debian wheezy: ii nfs-common 1:1.2.6-4 amd64 NFS support files common to client and server ii nfs-kernel-server 1:1.2.6-4 amd64 support for NFS kernel server ii aufs-tools 1:3.0+20120411-2 amd64 Tools to manage aufs filesystems zfs packages for debian wheezy from zfsonlinux: ii debian-zfs 7~wheezy amd64 Native ZFS filesystem metapackage for Debian. ii libzfs1 0.6.2-1 amd64 Native ZFS filesystem library for Linux ii zfs-dkms 0.6.2-1 all Native ZFS filesystem kernel modules for Linux ii zfsonlinux 2~wheezy all archive.zfsonlinux.org trust package ii zfsutils 0.6.2-1 amd64 command-line tools to manage ZFS filesystems ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
