Hi Daniel, It’s funny you mention compression & quotas – it’s something I’ve hit on a.n.other system. The supplier ended up providing different ways of presenting quotas as a mount option. Options are:
- Quotas are tracked using the post compression/dedupe actually used space. (i.e. users see the benefit/drawback of compression) - Quotas are tracked using the pre compression/dedupe filesize. (i.e. the sysadmin sees the benefit; users are unaware what’s happening underneath). Would something similar be possible in GPFS land? David From: [email protected] [mailto:[email protected]] On Behalf Of Daniel Kidger Sent: 04 May 2016 12:36 To: [email protected] Cc: [email protected] Subject: Re: [gpfsug-discuss] Data replication and fileset quotas As Simon says, Quota measures used blocks in the filesystem. Hence users can and should have behaviour that keeps within these limits. GPFS Replication though is a system-administrator level concept - to protect data access in the case of power outages or though gross hardware failures. So as such should be transparent to the end users. Unless users are enabled to choose 1 or 2 (or 3) way replication of their own files dependent on their importance (eg 1 copy for scratch files) then imho replication should not be measured in quota reporting. On a related note, compression is great new feature, but it may confuse users if they delete some older but big 100GB files then try and recreate them only to find they can't because their quota is now exceeded (as compression is not at file creation but driven later by policies. Thoughts? Daniel Dr Daniel Kidger IBM Technical Sales Specialist Software Defined Solution Sales +44-07818 522 266 [email protected]<mailto:[email protected]> ----- Original message ----- From: "Oesterlin, Robert" <[email protected]<mailto:[email protected]>> Sent by: [email protected]<mailto:[email protected]> To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Cc: Subject: Re: [gpfsug-discuss] Data replication and fileset quotas Date: Wed, May 4, 2016 12:19 PM From the documentation: "When setting quota limits for a file system, replication within the file system should be considered. GPFS quota management takes replication into account when reporting on and determining if quota limits have been exceeded for both block and file usage. In a file system that has either type of replication set to a value of two, the values reported on by both the mmlsquota command and the mmrepquota command are double the value reported by the ls command." Bob Oesterlin Sr Storage Engineer, Nuance HPC Grid 507-269-0413 From: <[email protected]<mailto:[email protected]>> on behalf of "Simon Thompson (Research Computing - IT Services)" <[email protected]<mailto:[email protected]>> Reply-To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Date: Wednesday, May 4, 2016 at 2:11 AM To: 'gpfsug main discussion list' <[email protected]<mailto:[email protected]>> Subject: [EXTERNAL] Re: [gpfsug-discuss] Data replication and fileset quotas Yes, this correct (I think there was some discussion on the mailing list a little while back). The quota is the space used on disk, so if you run compression its the compressed size, tape hsm don't count until you recall them. I assume mcstore is the same, I.e. Only counts on recall, but I haven't tested this. Simon -- Simon Thompson Research Computing Team, IT Services +44 121 415 8675 -----Original Message----- From: Bryan Banister [[email protected]<mailto:[email protected]>] Sent: Tuesday, May 03, 2016 09:33 PM GMT Standard Time To: gpfsug main discussion list Subject: Re: [gpfsug-discuss] Data replication and fileset quotas Yes, I believe that is the case due to the fact that every file is replicated therefore taking twice as much space within the file system. -Bryan From:[email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Buterbaugh, Kevin L Sent: Tuesday, May 03, 2016 3:27 PM To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Subject: [gpfsug-discuss] Data replication and fileset quotas Hi again all, I have another question on a completely different topic and therefore I decided it was better to send two separate e-mails. For a number of years now we have had a GPFS filesystem where we use filesets and set fileset quotas. Data replication is set to one. I understand how that all works. We are creating another GPFS filesystem where we intend to also use filesets and fileset quotas, but set data replication to two. Based on my experience with data replication on a filesystem that doesn’t use filesets, I am expecting that setting data replication to two means that I will need to double the quota for each fileset (i.e. if a group has bought 5 TB of space I’ll need to set their fileset quota to 10 TB) but haven’t found where that is explicitly documented. Is that correct? Thanks again, all… Kevin — Kevin Buterbaugh - Senior System Administrator Vanderbilt University - Advanced Computing Center for Research and Education [email protected]<mailto:[email protected]> - (615)875-9633 ________________________________ Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product. _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
