Re: [gpfsug-discuss] file layout API + file fragmentation

Mon, 06 Nov 2017 12:59:33 -0800

The general strategy is this: when you open a file and start writing, we'll allocate full blocks.  When you close the file we look at file size and replace the last block with a fragment just big enough to hold the data at the end of the file ("shrink-to-fit").
 
If you open the file again later and append new data that doesn't fit in the existing fragment, we'll first allocate another full block ("fragment expansion"), then shrink again when you close the file again.
 
Note that we don't shrink or expand "in place", but rather first allocate a whole new fragment or block somewhere else, then free the old block or fragment that's no longer needed.  That's done to minimize space fragmentation, because shrink-to-fit will be able to use left over free space in another block that holds fragments from other files already, rather than cutting up the full block that was originally allocated for the last block of the file.
 
In your example, where you only append once every 24 hours, that means your fragment will keep moving around once a day, as the fragment grows, until it has become a full block.  Normally, if writes/appends occur close enough in time, dirty data in the last block tends to stay in the buffer pool long enough that the final shrink-to-fit happens before dirty data is flushed to disk, so data is only written once, to its final location.
 
 -- Frank
 
 
----- Original message -----
From: "Daniel Kidger" <[email protected]>
Sent by: [email protected]
To: "gpfsug main discussion list" <[email protected]>
Cc:
Subject: Re: [gpfsug-discuss] file layout API + file fragmentation
Date: Mon, Nov 6, 2017 11:01 AM
 
Frank,
 
For clarity in the understanding the underlying mechanism in GPFS, could you describe what happens in the case say of a particular file that is appended to every 24 hours?  
ie. as that file gets to 7MB, it then writes to a new sub-block (1/32 of the next 1MB block). I guess that sub block could be 10th in a a block that already has 9 used. Later on, the file grows to need an 11th subblock and so on. So at what point does this growing file at 8MB occupy all 32 sunblocks of  8 full blocks?
 
Daniel
/spectrum_storage-banne

 
Spectrum Scale Logo
 
  
Dr Daniel Kidger 
IBM Technical Sales Specialist
Software Defined Solution Sales

+ 44-(0)7818 522 266 
[email protected]

On 6 Nov 2017, at 00:57, Frank Schmuck <[email protected]> wrote:
 
In GPFS blocks within a file are never fragmented.  For example, if you have a file of size 7.3 MB and your file system block size is 1MB, then this file will be made up of 7 full blocks and one fragment of size 320k (10 subblocks).  Each of the 7 full blocks will be contiguous on a singe diks (LUN) behind a single NSD server.  The fragment that makes up the last part of the file will also be contiguous on a single disk, just shorter than a full block.
 
Frank Schmuck
IBM Almaden Research Center
 
 
----- Original message -----
From: Aaron Knister <[email protected]>
Sent by: [email protected]
To: <[email protected]>
Cc:
Subject: Re: [gpfsug-discuss] file layout API + file fragmentation
Date: Sun, Nov 5, 2017 3:39 PM
 
Thanks Marc, that helps. I can't easily use tsdbfs for what I'm working
on since it needs to be run as unprivileged users.

Perhaps I'm not asking the right question. I'm wondering how the file
layout api behaves if a given "block"-aligned offset in a file is made
up of sub-blocks/fragments that are not all on the same NSD. The
assumption based on how I've seen the API used so far is that all
sub-blocks within a block at a given offset within a file are all on the
same NSD.

-Aaron

On 11/5/17 6:01 PM, Marc A Kaplan wrote:
> I googled GPFS_FCNTL_GET_DATABLKDISKIDX
>
> and found this discussion:
>
>  https://www.ibm.com/developerworks/community/forums/html/topic?id=db48b190-4f2f-4e24-a035-25d3e2b06b2d&ps=50
>
> In general, GPFS files ARE deliberately "fragmented" but we don't say
> that - we say they are "striped" over many disks -- and that is
> generally a good thing for parallel performance.
>
> Also, in GPFS, if the last would-be block of a file is less than a
> block, then it is stored in a "fragment" of a block.  
> So you see we use "fragment" to mean something different than other file
> systems you may know.
>
> --marc
>
>
>
> From:        Aaron Knister <[email protected]>
> To:        gpfsug main discussion list <[email protected]>
> Date:        11/04/2017 12:22 PM
> Subject:        [gpfsug-discuss] file layout API + file fragmentation
> Sent by:        [email protected]
> ------------------------------------------------------------------------
>
>
>
> I've got a question about the file layout API and how it reacts in the
> case of fragmented files.
>
> I'm using the GPFS_FCNTL_GET_DATABLKDISKIDX structure and have some code
> based on tsGetDataBlk.C. I'm basing the block size based off of what's
> returned by filemapOut.blockSize but that only seems to return a value >
> 0 when filemapIn.startOffset is 0.
>
> In a case where a file were to be made up of a significant number of
> non-contiguous fragments (which... would be awful in of itself) how
> would this be reported by the file layout API? Does the interface
> technically just report the disk location information of the first block
> of the $blockSize range and assume that it's contiguous?
>
> Thanks!
>
> -Aaron
>
> --
> Aaron Knister
> NASA Center for Climate Simulation (Code 606.2)
> Goddard Space Flight Center
> (301) 286-2776
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=wnR7m6d4urZ_8dM4mkHQjMbFD9xJEeesmJyzt1osCnM&s=-dgGO6O5i1EqWj-8MmzjxJ1Iz2I5gT1aRmtyP44Cvdg&e=
>
>
>
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e=
>

--
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e=
 
 
 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
 
 

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Reply via email to