Re: pmfsrr mode
Hello Jean, Sorry for may late reaction. Are you still interested in pmfsrr mode? In my last mail, I described about the cache. But it might be wrong. Currently I think this behaviour correctly follows what it should be, but the description of 'pmfsrr' mode in aufs maunal is poor. By design, 'pmfsrr' means - try top-down-parent mode first - test the decided branch has the larger free space than the specified 'low' watermark. - if the free space is smaller, then try mfs-rr mode. And 'mfsrr' mode means - find the branch whose has the largest free space - test the decided branch has the larger free space than the specified 'low' watermark. - if the free space is smaller, then try round-robin mode. In 'round-robin' mode, all the free-space information are ignored. But the manual describes differently a little. -- .B create=mfsrr:low[:second] Selects a writable branch in most\-free\-space mode first, and then round\-robin mode. If the selected branch has less free space than the specified value `low' in bytes, then aufs re-tries in round\-robin mode. ::: .B create=pmfsrr:low[:second] Firstly selects a writable branch as the `pmfs mode.' If there are less than `low' bytes available on all branches where the parent dir exists, aufs selects the one which has the most free space regardless the parent dir. -- Note that mfsrr says "round-robin" explicitly but pmfsrr doesn't. The description of pmfsrr should be If there are less than `low' bytes available on all branches where the parent dir exists, then mfsrr mode is applied regardless the parent dir. Because you specified a very large value as the 'low' watermark, I guess your 2GB file was fallen to the simple 'rr' mode. So the one solution will be to specify the value lower than the free-space which your desired branch has. Is it a good solution for you? sf...@users.sourceforge.net: > > "Jean-Ray Arseneau": > > Here=E2=80=99s the updated log after applying the latest debug b.patch: > > Thank you very much. > I think I could see the scenario. > As you might know, the mfs policy caches the free-space value per > branch, and pmfsrr policy internally runs the mfs rourtine twice. The > first one is considering the parent dir, and the other is without > considering the parent. In this second call, the mfs routine doesn't > work expectedly since the result is cached and the routine dicide the > target bracnh based upon the last result. Obviously it is an aufs bug. > I will adress it in next week. If you can't wait, try 'pmfdrr::0' > which will force the mfs routine not to use the cached value. -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
Re: pmfsrr mode
"Jean-Ray Arseneau": > Here=E2=80=99s the updated log after applying the latest debug b.patch: Thank you very much. I think I could see the scenario. As you might know, the mfs policy caches the free-space value per branch, and pmfsrr policy internally runs the mfs rourtine twice. The first one is considering the parent dir, and the other is without considering the parent. In this second call, the mfs routine doesn't work expectedly since the result is cached and the routine dicide the target bracnh based upon the last result. Obviously it is an aufs bug. I will adress it in next week. If you can't wait, try 'pmfdrr::0' which will force the mfs routine not to use the cached value. J. R. Okajima -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
Hereâs the updated log after applying the latest debug b.patch: https://dl.dropboxusercontent.com/u/26943/syslog-20150404.zip Jean On Sat, Apr 4, 2015 at 11:53 AM, sf...@users.sourceforge.net <[1]sf...@users.sourceforge.net> wrote: "Jean-Ray Arseneau": > Absolutely Here is an additional debug patch. J. R. Okajima References 1. mailto:sf...@users.sourceforge.net -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
"Jean-Ray Arseneau": > Absolutely Here is an additional debug patch. J. R. Okajima b.patch.bz2 Description: BZip2 compressed data -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
Absolutely On Sat, Apr 4, 2015 at 1:30 AM, sf...@users.sourceforge.net <[1]sf...@users.sourceforge.net> wrote: "Jean-Ray Arseneau": > Apologies, I sent the wrong log file. > > Here is the correct one, I am quite sure the patch is installed, as I see = > lines like this one: Ok thanx, confirmed the patch was applied. The strange thing is that aufs considers there are only 6 writable branches, while you specified 8 writable branches. The missing two are - br2, WD-WCC4E5AFUJZK - br7, WD-WMAZA4509546 and br7 is what you expect as the target. If I write more patch to investigate this point, would you test again? J. R. Okajima References 1. mailto:sf...@users.sourceforge.net -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
"Jean-Ray Arseneau": > Apologies, I sent the wrong log file. > > Here is the correct one, I am quite sure the patch is installed, as I see = > lines like this one: Ok thanx, confirmed the patch was applied. The strange thing is that aufs considers there are only 6 writable branches, while you specified 8 writable branches. The missing two are - br2, WD-WCC4E5AFUJZK - br7, WD-WMAZA4509546 and br7 is what you expect as the target. If I write more patch to investigate this point, would you test again? J. R. Okajima -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
Hello, Apologies, I sent the wrong log file. Here is the correct one, I am quite sure the patch is installed, as I see lines like this one: aufs au_wbr_create_mfsrr:533:cp[3711]: DEBUG: mfsrr_bytes 42183135232, mfsrr_watermark 107374182400 Link to file: https://dl.dropboxusercontent.com/u/26943/syslog-20150403-2.zip Cheers, Jean On Fri, Apr 3, 2015 at 10:10 PM, sf...@users.sourceforge.net <[1]sf...@users.sourceforge.net> wrote: "Jean-Ray Arseneau": > I have applied the patch, at least I=E2=80=99m quite sure I did. The debug patch doesn't seem to be applied. Are you sure that you have loaded the newly compiled module? J. R. Okajima References 1. mailto:sf...@users.sourceforge.net -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
"Jean-Ray Arseneau": > I have applied the patch, at least I=E2=80=99m quite sure I did. The debug patch doesn't seem to be applied. Are you sure that you have loaded the newly compiled module? J. R. Okajima -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
Hello, I have applied the patch, at least Iâm quite sure I did. Here is a link to the new syslog: https://dl.dropboxusercontent.com/u/26943/syslog-20150403.zip Let me know what else I can do to help. Cheers, On Thu, Apr 2, 2015 at 11:55 PM, sf...@users.sourceforge.net <[1]sf...@users.sourceforge.net> wrote: "Jean-Ray Arseneau": > I have included linked to the files. They are: Ok thanx. The log shows the expected behaviour, but the result is wrong. To investigate more, would you apply this debug print patch and try again? This patch doesn't solve the problem, but help investigating. Original J. R. References 1. mailto:sf...@users.sourceforge.net -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
"Jean-Ray Arseneau": > I have included linked to the files. They are: Ok thanx. The log shows the expected behaviour, but the result is wrong. To investigate more, would you apply this debug print patch and try again? This patch doesn't solve the problem, but help investigating. Original J. R. a.patch.bz2 Description: BZip2 compressed data -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
sf...@users.sourceforge.net: > This will print many debug information including how aufs selected the > branch. I may send you another debug patch to print more information and > ask you to apply/rebuild if you can. I tried but could not reproduce the problem. Branches: /run/shm/11044.1=rw /run/shm/11044.2=rw /run/shm/rw=rw /run/shm/ro=ro Filesystem 1K-blocks Used Available Use% Mounted on tmpfs 20252416202508 1% /run/shm/rw tmpfs 40508 0 40508 0% /run/shm Option: create=pmfsrr:$((50*1024*1024)) Status: - the target parent dir is "a/b/c". let's call it $dir. - /run/shm/rw/$dir does not exist. - the dir exists both under /run/shm/11044.[12]. In other words, - there are three writable branches. - two of them have a parent dir, but they are smaller in free space. Test: cp /etc/hosts $dir/ I confirmed the file went to /run/shm/rw/$dir expectedly. Next time when you post, give me these info. (from aufs README) -- When you have any problems or strange behaviour in aufs, please let me know with: - /proc/mounts (instead of the output of mount(8)) - /sys/module/aufs/* - /sys/fs/aufs/* (if you have them) - /debug/aufs/* (if you have them) - linux kernel version if your kernel is not plain, for example modified by distributor, the url where i can download its source is necessary too. - aufs version which was printed at loading the module or booting the system, instead of the date you downloaded. - configuration (define/undefine CONFIG_AUFS_xxx) -- J. R. Okajima -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Re: pmfsrr mode
Hello J. "Jean-Ray Arseneau": > Now, from my understanding of the manual, the way I=E2=80=99m mounting with= > pmfsrr, I am setting a [low] value of 50GB. Again, according to my = > understanding, if all branches are below the =E2=80=9Clow=E2=80=9D value, = > it will just round robin to one of the other branches that has the most = > space (from man: If there are less than =E2=80=99low=E2=80=99 bytes = > available on all branches where the parent dir exists, aufs selects the one= > which has the most free space regardless the parent dir). Yes, 'pmfsrr' should behave such like that. If CONFIG_AUFS_DEBUG is disabled, can you enable it and rebuild aufs module? If so, set the module parameter 'debug' ON, and then $ echo 1 > /sys/fs/module/aufs/parameter/debug $ cp 2GB-file /mnt/media/music/artist/file.mp4 $ echo 0 > /sys/fs/module/aufs/parameter/debug This will print many debug information including how aufs selected the branch. I may send you another debug patch to print more information and ask you to apply/rebuild if you can. I will start reproducing the problem on my side with much smaller size in a few days. J. R. Okajima (another 'J.') -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
pmfsrr mode
Hello, Long time lurker, first time poster, hope I get things right. I have an AuFS mount that is 21TB in size. I’m trying to use AuFS in a way that will ensure my disks (branches) do not fill up and result in out of space issues (preferably without using create=mfs). There are currently 7 disks, and I’m attempting to use the pmfsrr mode without much success. Here is a list of my disks: /dev/sde1 3.6T 3.4T 51G 99% /media/disks/WD-WCC4E5AFUJZK /dev/sdf1 2.7T 2.6T 36G 99% /media/disks/WD-WMC4N0765691 /dev/sdg1 1.8T 1.8T 35G 99% /media/disks/WD-WMAY00153458 /dev/sdh1 1.8T 528G 1.2T 31% /media/disks/WD-WMAZA4509546 /dev/sdi1 2.7T 2.6T 32G 99% /media/disks/WD-WMC4N0751110 /dev/sdj1 2.7T 2.6T 34G 99% /media/disks/WD-WMC4N0855507 /dev/sdk1 2.7T 2.6T 37G 99% /media/disks/WD-WCC1T1479153 /dev/sdl1 2.7T 2.6T 35G 99% /media/disks/SG-W1F1Q1J4 Here is my mount command (I’m running the latest build): mount -t aufs -o br:/media/disks/WD-WMC4N0765691=rw:/media/disks/WD-WMAY00153458=rw:/media/di sks/WD-WCC4E5AFUJZK=rw:/media/disks/SG-W1F1Q1J4=rw:/media/disks/WD-WCC1T1479 153=rw:/media/disks/WD-WMC4N0751110=rw:/media/disks/WD-WMC4N0855507=rw:/medi a/disks/WD-WMAZA4509546=rw-osum,create=pmfsrr:53687091200,udba=notifynone /mnt/media Now, from my understanding of the manual, the way I’m mounting with pmfsrr, I am setting a [low] value of 50GB. Again, according to my understanding, if all branches are below the “low” value, it will just round robin to one of the other branches that has the most space (from man: If there are less than ’low’ bytes available on all branches where the parent dir exists, aufs selects the one which has the most free space regardless the parent dir). The way I’m interpreting that is if “parentdir” exists on 2 of the branches, but both those branches are below the “low” value, it will go and select the branch with has the most free space, even if the parentdir does not exist, including another branch where the parentdir does not exist. Here’s where I’m running into the issue: 1. Copy 2GB file to /mnt/media/music/artist/file.mp4 2. the “parentdir” /music/artist exists on 2 of the branches that fall below the low level (WD-WMC4N0751110 and WD-WMC4N0855507, both around 32-34GB left) 3. The file does not get copied to WD-WMAZA4509546 (which has 1.2TB left), but instead WD-WMC4N0855507, which is the branch where the parentdir exists that contains the most free space (out of the two). My understanding should be that seeing as both disks fall below the “low” marker, the file should be copied to the branch with 1.2TB left, regardless if the parentdir exists on that branch. It is fully possible that I am misinterpreting the manual. If I mount with mfs, all writes go to the 1.2TB disk as expected. However, I am trying to avoid using mfs because I’d like to reduce fragmentation of the files across branches (where pmfsrr comes in), but still avoid the dreaded “out of space” error that will surely happen if I continue to use pmfsrr. The other reason I’m not a big fan of mfs is that if you have 2x3TB drives, they will “ping pong” (copying one file one place, one file another, I usually deal with 2-25GB files) with each other, which isn’t ideal. Can someone shed some light on this? I’ve been able to easily recreate my situation in my test environment and I can confirm that if the parentdir does not exist on a branch with pmfsrr, it will not copy a file to that branch (which would include creating the directory structure up to the point where the file is to be copied). Many thanks! J. -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/