In addition, the in_doubt column is a function of the data turn-over
and the internal gpfs accounting synchronization period (beyond root
control). The higher the in_doubt values the less accurate the real
amount of space/inodes a user/group/fileset has in the filesystem.
What I noticed in practice is the the in_doubt values only get worst
overtime, and work against the quotas, making them hit the limits
sooner. Therefore, you may wish to run a 'mmcheckquota' crontab job
once or twice a day, to reset the in_doubt column to zero mover often.
GPFS has a very high lag to do this on its own in the most recent
versions, and seldom really catches up on a very active filesystem.
If your grace period is set to 7 days I can assure you that in an HPC
environment it's the equivalent of not having quotas effectively. You
should set it to 2 hours or 4 hours.
In an environment such as ours a runway process can easily generate
500TB of data or 1 billion inodes in few hours, and choke the file
system to all users/jobs.
Jaime
Quoting "Buterbaugh, Kevin L" <[email protected]>:
Hi Renar,
I?m sure this is the case, but I don?t see anywhere in this thread
where this is explicitly stated ? you?re not doing your tests as
root, are you? root, of course, is not bound by any quotas.
Kevin
On Jul 31, 2017, at 2:04 PM, Grunenberg, Renar
<[email protected]<mailto:[email protected]>>
wrote:
Hallo J. Eric, hallo Jaime,
Ok after we hit the softlimit we see that the graceperiod are go to
7 days. I think that?s the default. But was does it mean.
After we reach the ?hard?-limit. we see additionaly the gbytes in_doubt.
My interpretation now we can write many gb to the nospace-left event
in the filesystem.
But our intention is to restricted some application to write only to
the hardlimit in the fileset. Any hints to accomplish this?
Renar Grunenberg
Abteilung Informatik ? Betrieb
HUK-COBURG
Bahnhofsplatz
96444 Coburg
Telefon: 09561 96-44110
Telefax: 09561 96-44104
E-Mail: [email protected]<mailto:[email protected]>
Internet: www.huk.de<http://www.huk.de/>
________________________________
HUK-COBURG Haftpflicht-Unterstützungs-Kasse kraftfahrender Beamter
Deutschlands a. G. in Coburg
Reg.-Gericht Coburg HRB 100; St.-Nr. 9212/101/00021
Sitz der Gesellschaft: Bahnhofsplatz, 96444 Coburg
Vorsitzender des Aufsichtsrats: Prof. Dr. Heinrich R. Schradin.
Vorstand: Klaus-Jürgen Heitmann (Sprecher), Stefan Gronbach, Dr.
Hans Olav Herøy, Dr. Jörg Rheinländer (stv.), Sarah Rössler, Daniel
Thomas (stv.).
________________________________
Diese Nachricht enthält vertrauliche und/oder rechtlich geschützte
Informationen.
Wenn Sie nicht der richtige Adressat sind oder diese Nachricht
irrtümlich erhalten haben,
informieren Sie bitte sofort den Absender und vernichten Sie diese Nachricht.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser
Nachricht ist nicht gestattet.
This information may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this
information in error) please notify the
sender immediately and destroy this information.
Any unauthorized copying, disclosure or distribution of the material
in this information is strictly forbidden.
________________________________
Von:
[email protected]<mailto:[email protected]> [mailto:[email protected]] Im Auftrag von J. Eric
Wonderley
Gesendet: Montag, 31. Juli 2017 19:55
An: gpfsug main discussion list
<[email protected]<mailto:[email protected]>>
Betreff: Re: [gpfsug-discuss] Quota and hardlimit enforcement
Hi Renar:
What does 'mmlsquota -j fileset filesystem' report?
I did not think you would get a grace period of none unless the
hardlimit=softlimit.
On Mon, Jul 31, 2017 at 1:44 PM, Grunenberg, Renar
<[email protected]<mailto:[email protected]>>
wrote:
Hallo All,
we are on Version 4.2.3.2 and see some missunderstandig in the
enforcement of hardlimit definitions on a flieset quota. What we see
is we put some 200 GB files on following quota definitions: quota
150 GB Limit 250 GB Grace none.
After the creating of one 200 GB we hit the softquota limit, thats
ok. But After the the second file was created!! we expect an io
error but it don?t happen. We define all well know Parameters
(-Q,..) on the filesystem . Is this a bug or a Feature? mmcheckquota
are already running at first.
Regards Renar.
Renar Grunenberg
Abteilung Informatik ? Betrieb
HUK-COBURG
Bahnhofsplatz
96444 Coburg
Telefon:
09561 96-44110
Telefax:
09561 96-44104
E-Mail:
[email protected]<mailto:[email protected]>
Internet:
www.huk.de<http://www.huk.de/>
________________________________
HUK-COBURG Haftpflicht-Unterstützungs-Kasse kraftfahrender Beamter
Deutschlands a. G. in Coburg
Reg.-Gericht Coburg HRB 100; St.-Nr. 9212/101/00021
Sitz der Gesellschaft: Bahnhofsplatz, 96444 Coburg
Vorsitzender des Aufsichtsrats: Prof. Dr. Heinrich R. Schradin.
Vorstand: Klaus-Jürgen Heitmann (Sprecher), Stefan Gronbach, Dr.
Hans Olav Herøy, Dr. Jörg Rheinländer (stv.), Sarah Rössler, Daniel
Thomas (stv.).
________________________________
Diese Nachricht enthält vertrauliche und/oder rechtlich geschützte
Informationen.
Wenn Sie nicht der richtige Adressat sind oder diese Nachricht
irrtümlich erhalten haben,
informieren Sie bitte sofort den Absender und vernichten Sie diese Nachricht.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser
Nachricht ist nicht gestattet.
This information may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this
information in error) please notify the
sender immediately and destroy this information.
Any unauthorized copying, disclosure or distribution of the material
in this information is strictly forbidden.
________________________________
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org/>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org/>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
************************************
TELL US ABOUT YOUR SUCCESS STORIES
http://www.scinethpc.ca/testimonials
************************************
---
Jaime Pinto
SciNet HPC Consortium - Compute/Calcul Canada
www.scinet.utoronto.ca - www.computecanada.ca
University of Toronto
661 University Ave. (MaRS), Suite 1140
Toronto, ON, M5G1M1
P: 416-978-2755
C: 416-505-1477
----------------------------------------------------------------
This message was sent using IMP at SciNet Consortium, University of Toronto.
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
