Re: Permissions for root user
On Mon, Mar 30 2015, Yair Yarom ir...@cs.huji.ac.il wrote: On Sat, Mar 28 2015, sf...@users.sourceforge.net wrote: Are you using user-space NFS server too? If so, does this script surely reproduce the problem? This script prints 0 at the end when everything is fine. Our nfs servers are NetAPP and FreeBSD machines (not user space...). The script itself prints 0, however I currently don't have time to thoroughly check it. In a couple of days when I have time I'll try to see if I can make a script that reproduces the problem. It seems the script doesn't mount the nfs, so I'm a bit puzzled about what it was suppose to check. In any case, though I don't think it was my original problem (but I might be wrong), it appears that currently if the nfs is mounted with noacl, it might fail with Operation not supported even if the nfs mount is rw and can be changed directly on the mount. The attached script reproduces the Operation not supported when trying to move a file on the aufs (though moving it on the nfs works). It's not as nice as yours, it prints some stuff, and assumes nfs-kernel-server is already running. Yair. c.sh.bz2 Description: BZip2 compressed data -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
pmfsrr mode
Hello, Long time lurker, first time poster, hope I get things right. I have an AuFS mount that is 21TB in size. I’m trying to use AuFS in a way that will ensure my disks (branches) do not fill up and result in out of space issues (preferably without using create=mfs). There are currently 7 disks, and I’m attempting to use the pmfsrr mode without much success. Here is a list of my disks: /dev/sde1 3.6T 3.4T 51G 99% /media/disks/WD-WCC4E5AFUJZK /dev/sdf1 2.7T 2.6T 36G 99% /media/disks/WD-WMC4N0765691 /dev/sdg1 1.8T 1.8T 35G 99% /media/disks/WD-WMAY00153458 /dev/sdh1 1.8T 528G 1.2T 31% /media/disks/WD-WMAZA4509546 /dev/sdi1 2.7T 2.6T 32G 99% /media/disks/WD-WMC4N0751110 /dev/sdj1 2.7T 2.6T 34G 99% /media/disks/WD-WMC4N0855507 /dev/sdk1 2.7T 2.6T 37G 99% /media/disks/WD-WCC1T1479153 /dev/sdl1 2.7T 2.6T 35G 99% /media/disks/SG-W1F1Q1J4 Here is my mount command (I’m running the latest build): mount -t aufs -o br:/media/disks/WD-WMC4N0765691=rw:/media/disks/WD-WMAY00153458=rw:/media/di sks/WD-WCC4E5AFUJZK=rw:/media/disks/SG-W1F1Q1J4=rw:/media/disks/WD-WCC1T1479 153=rw:/media/disks/WD-WMC4N0751110=rw:/media/disks/WD-WMC4N0855507=rw:/medi a/disks/WD-WMAZA4509546=rw-osum,create=pmfsrr:53687091200,udba=notifynone /mnt/media Now, from my understanding of the manual, the way I’m mounting with pmfsrr, I am setting a [low] value of 50GB. Again, according to my understanding, if all branches are below the “low” value, it will just round robin to one of the other branches that has the most space (from man: If there are less than ’low’ bytes available on all branches where the parent dir exists, aufs selects the one which has the most free space regardless the parent dir). The way I’m interpreting that is if “parentdir” exists on 2 of the branches, but both those branches are below the “low” value, it will go and select the branch with has the most free space, even if the parentdir does not exist, including another branch where the parentdir does not exist. Here’s where I’m running into the issue: 1. Copy 2GB file to /mnt/media/music/artist/file.mp4 2. the “parentdir” /music/artist exists on 2 of the branches that fall below the low level (WD-WMC4N0751110 and WD-WMC4N0855507, both around 32-34GB left) 3. The file does not get copied to WD-WMAZA4509546 (which has 1.2TB left), but instead WD-WMC4N0855507, which is the branch where the parentdir exists that contains the most free space (out of the two). My understanding should be that seeing as both disks fall below the “low” marker, the file should be copied to the branch with 1.2TB left, regardless if the parentdir exists on that branch. It is fully possible that I am misinterpreting the manual. If I mount with mfs, all writes go to the 1.2TB disk as expected. However, I am trying to avoid using mfs because I’d like to reduce fragmentation of the files across branches (where pmfsrr comes in), but still avoid the dreaded “out of space” error that will surely happen if I continue to use pmfsrr. The other reason I’m not a big fan of mfs is that if you have 2x3TB drives, they will “ping pong” (copying one file one place, one file another, I usually deal with 2-25GB files) with each other, which isn’t ideal. Can someone shed some light on this? I’ve been able to easily recreate my situation in my test environment and I can confirm that if the parentdir does not exist on a branch with pmfsrr, it will not copy a file to that branch (which would include creating the directory structure up to the point where the file is to be copied). Many thanks! J. -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
Hello J. Jean-Ray Arseneau: Now, from my understanding of the manual, the way I=E2=80=99m mounting with= pmfsrr, I am setting a [low] value of 50GB. Again, according to my = understanding, if all branches are below the =E2=80=9Clow=E2=80=9D value, = it will just round robin to one of the other branches that has the most = space (from man: If there are less than =E2=80=99low=E2=80=99 bytes = available on all branches where the parent dir exists, aufs selects the one= which has the most free space regardless the parent dir). Yes, 'pmfsrr' should behave such like that. If CONFIG_AUFS_DEBUG is disabled, can you enable it and rebuild aufs module? If so, set the module parameter 'debug' ON, and then $ echo 1 /sys/fs/module/aufs/parameter/debug $ cp 2GB-file /mnt/media/music/artist/file.mp4 $ echo 0 /sys/fs/module/aufs/parameter/debug This will print many debug information including how aufs selected the branch. I may send you another debug patch to print more information and ask you to apply/rebuild if you can. I will start reproducing the problem on my side with much smaller size in a few days. J. R. Okajima (another 'J.') -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: Permissions for root user
Christoph Pleger: Now I found that the script was successful because the machine where I tested it has another Debian version installed than the other machine. I compared two Debian 7 nfs clients with two Debian 8 nfs clients, in Debian 7 file creation was successful, in Debian 8 it was not. I guess it is related to ACL. When CONFIG_NFS_V3_ACL is enabled, NFS client confirms whether NFS server supports ACL or not. Your userspace NFS server doesn't seem to be supporting ACL and it returns Operation not supported. As long as unfs3 doesn't support ACL, you need to disable CONFIG_NFS_V3_ACL I am afraid. J. R. Okajima -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: Permissions for root user
Yair Yarom: It seems the script doesn't mount the nfs, so I'm a bit puzzled about what it was suppose to check. Oops! I am afraid I remove the line accidentaly. Sorry. In any case, though I don't think it was my original problem (but I might be wrong), it appears that currently if the nfs is mounted with noacl, it might fail with Operation not supported even if the nfs mount is rw and can be changed directly on the mount. Thanx for the script. On my test system where NFS server is linux, the first mv back failed. ::: exporting localhost:/tmp/irush/export + mv -v /tmp/irush/nfs/a/b /tmp/irush/nfs/a/c `/tmp/irush/nfs/a/b' - `/tmp/irush/nfs/a/c' mv: cannot move `/tmp/irush/nfs/a/b' to `/tmp/irush/nfs/a/c': Permission denied ::: Anyway ACL is a good hint actually. While I cannot reproduce your problem on my side, I'd suggest you to try a branch attribute icexsys for the writable branch such as # mount -t aufs -o br:$tmp/rw=rw+icexsys:$tmp/nfs=ro aufs $tmp/union For details, refer to linux/Documentation/filesystems/aufs/design/xattr.txt and aufs manual in aufs-util.git. J. R. Okajima -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
