Aaron, brilliant! Your example is close to the worst case, where every file is 512K+1 bytes and the blocksize is 1024K. Yes, in the worse case 49.99999% of space is "lost" or wasted. Don't do that!
One can construct such a worst case for any system that allocates by blocks or sectors or whatever you want to call it. Just fill the system with files that are each 0.5*Block_Size+1 bytes and argue that 1/2 the space is wasted. From: Aaron Knister <[email protected]> To: <[email protected]> Date: 11/06/2017 12:10 AM Subject: Re: [gpfsug-discuss] file layout API + file fragmentation Sent by: [email protected] Thanks, Frank! That's truly fascinating and has some interesting implications that I hadn't thought of before. I just ran a test on an ~8G fs with a block size of 1M: for i in `seq 1 100000`; do dd if=/dev/zero of=foofile${i} bs=520K count=1 done The fs is "full" according to df/mmdf but there's 3.6G left in subblocks but yeah, I can't allocate any new files that wouldn't fit into the inode and I can't seem to allocate any new subblocks to existing files (e.g. append). What's interesting is if I do the same exercise but with a file size of 30K or even 260K I don't seem to run into the same issue. I'm not sure I understand that yet. I was curious about what this meant in the case of appending to a file where the last offset in the file was allocated to a fragment. By looking at "tsdbfs listda" and appending to a file I could see that the last DA would change (presumably to point to the DA of the start of a contiguous subblock) once the amount of data appended caused the file size to exceed the space available in the trailing subblocks. -Aaron On 11/5/17 7:57 PM, Frank Schmuck wrote: > In GPFS blocks within a file are never fragmented. For example, if you > have a file of size 7.3 MB and your file system block size is 1MB, then > this file will be made up of 7 full blocks and one fragment of size 320k > (10 subblocks). Each of the 7 full blocks will be contiguous on a singe > diks (LUN) behind a single NSD server. The fragment that makes up the > last part of the file will also be contiguous on a single disk, just > shorter than a full block. > > Frank Schmuck > IBM Almaden Research Center > > > > ----- Original message ----- > From: Aaron Knister <[email protected]> > Sent by: [email protected] > To: <[email protected]> > Cc: > Subject: Re: [gpfsug-discuss] file layout API + file fragmentation > Date: Sun, Nov 5, 2017 3:39 PM > > Thanks Marc, that helps. I can't easily use tsdbfs for what I'm working > on since it needs to be run as unprivileged users. > > Perhaps I'm not asking the right question. I'm wondering how the file > layout api behaves if a given "block"-aligned offset in a file is made > up of sub-blocks/fragments that are not all on the same NSD. The > assumption based on how I've seen the API used so far is that all > sub-blocks within a block at a given offset within a file are all on the > same NSD. > > -Aaron > > On 11/5/17 6:01 PM, Marc A Kaplan wrote: > > I googled GPFS_FCNTL_GET_DATABLKDISKIDX > > > > and found this discussion: > > > > > Â https://www.ibm.com/developerworks/community/forums/html/topic?id=db48b190-4f2f-4e24-a035-25d3e2b06b2d&ps=50 > > > > In general, GPFS files ARE deliberately "fragmented" but we don't say > > that - we say they are "striped" over many disks -- and that is > > generally a good thing for parallel performance. > > > > Also, in GPFS, if the last would-be block of a file is less than a > > block, then it is stored in a "fragment" of a block. Â > > So you see we use "fragment" to mean something different than > other file > > systems you may know. > > > > --marc > > > > > > > > From: Â Â Â Â Aaron Knister <[email protected]> > > To: Â Â Â Â gpfsug main discussion list > <[email protected]> > > Date: Â Â Â Â 11/04/2017 12:22 PM > > Subject: Â Â Â Â [gpfsug-discuss] file layout API + file > fragmentation > > Sent by: Â Â Â Â [email protected] > > > ------------------------------------------------------------------------ > > > > > > > > I've got a question about the file layout API and how it reacts in the > > case of fragmented files. > > > > I'm using the GPFS_FCNTL_GET_DATABLKDISKIDX structure and have > some code > > based on tsGetDataBlk.C. I'm basing the block size based off of what's > > returned by filemapOut.blockSize but that only seems to return a > value > > > 0 when filemapIn.startOffset is 0. > > > > In a case where a file were to be made up of a significant number of > > non-contiguous fragments (which... would be awful in of itself) how > > would this be reported by the file layout API? Does the interface > > technically just report the disk location information of the first > block > > of the $blockSize range and assume that it's contiguous? > > > > Thanks! > > > > -Aaron > > > > -- > > Aaron Knister > > NASA Center for Climate Simulation (Code 606.2) > > Goddard Space Flight Center > > (301) 286-2776 > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at spectrumscale.org > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=wnR7m6d4urZ_8dM4mkHQjMbFD9xJEeesmJyzt1osCnM&s=-dgGO6O5i1EqWj-8MmzjxJ1Iz2I5gT1aRmtyP44Cvdg&e= > > > > > > > > > > > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at spectrumscale.org > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e= > > > > -- > Aaron Knister > NASA Center for Climate Simulation (Code 606.2) > Goddard Space Flight Center > (301) 286-2776 > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e= > > > > > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=_xM9xVsqOuNiCqn3ikx6ZaaIHChTPhz_8iDmEKoteX4&s=uy462L5sxX_3Mm3Dh824ptJIxtah9LVRPMmyKz1lAdg&e= > -- Aaron Knister NASA Center for Climate Simulation (Code 606.2) Goddard Space Flight Center (301) 286-2776 _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=_xM9xVsqOuNiCqn3ikx6ZaaIHChTPhz_8iDmEKoteX4&s=uy462L5sxX_3Mm3Dh824ptJIxtah9LVRPMmyKz1lAdg&e=
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
