Re: [gpfsug-discuss] Odd entries in quota listing

[Oesterlin, Robert]

Well - good idea, but these large numbers in no way reflect valid ID numbers in 
our environment. Wondering how they got there…


Bob Oesterlin
Sr Storage Engineer, Nuance HPC Grid


From:  on behalf of Jonathan Buzzard 

Reply-To: gpfsug main discussion list 
Date: Tuesday, August 23, 2016 at 9:06 AM
To: "gpfsug-discuss@spectrumscale.org" 
Subject: [EXTERNAL] Re: [gpfsug-discuss] Odd entries in quota listing

I am guessing they are quotas that have been set for users that are now
deleted. GPFS stores the quota for a user under their UID, and deleting
the user and all their data is not enough to remove the entry from the
quota reporting, you also have to delete their quota.


___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] Odd entries in quota listing

[Jonathan Buzzard]

On Tue, 2016-08-23 at 13:58 +, Oesterlin, Robert wrote:
> In one of my file systems, I have some odd entries that seem to not be
> associated with a user - any ideas on the cause or how to track these
> down? This is a snippet from mmprepquota:
> 
>  
> 
>  Block Limits
> | File Limits
> 
> Name   type KB  quota  limit   in_doubt
> grace |files   quotalimit in_doubtgrace
> 
> 2751555824 USR   0 1073741824 5368709120  0
> none |0   000 none
> 
> 2348898617 USR   0 1073741824 5368709120  0
> none |0   000 none
> 
> 2348895209 USR   0 1073741824 5368709120  0
> none |0   000 none
> 
> 1610682073 USR   0 1073741824 5368709120  0
> none |0   000 none
> 
> 536964752  USR   0 1073741824 5368709120  0
> none |0   000 none
> 
> 403325529  USR   0 1073741824 5368709120  0
> none |0   000 none
> 

I am guessing they are quotas that have been set for users that are now
deleted. GPFS stores the quota for a user under their UID, and deleting
the user and all their data is not enough to remove the entry from the
quota reporting, you also have to delete their quota.


JAB.

-- 
Jonathan A. Buzzard Email: jonathan (at) buzzard.me.uk
Fife, United Kingdom.


___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] CES and mmuserauth command

[Jan-Frode Myklebust]

Sorry to see no authoritative answers yet.. I'm doing lots of CES
installations, but have not quite yet gotten the full understanding of
this..

Simple stuff first:

--servers You can only have one with AD.

--enable-kerberos shouldn't be used, as that's only for LDAP according to
the documentation. Guess kerberos is implied with AD.

--idmap-role -- I've been using "master". Man-page says

ID map role of a stand‐alone or singular system deployment must be
selected "master"


What the idmap options seems to be doing is configure the idmap options for
Samba. Maybe best explained by:

  https://wiki.samba.org/index.php/Idmap_config_ad


Your suggested options will then give you the samba idmap configuration:

  idmap config * : rangesize = 100
  idmap config * : range = 300-350
  idmap config * : read only = no
  idmap:cache = no
  idmap config * : backend = autorid

  idmap config DOMAIN : schema_mode = rfc2307
  idmap config DOMAIN : range = 500-200
  idmap config DOMAIN : backend = ad

Most likely you want to replace DOMAIN by your AD domain name.. So the
--idmap options sets some defaults, that you probably won't care about,
since all your users are likely covered by the specific "idmap config
DOMAIN" config.

Hope this helps somewhat, now I'll follow up with something I'm wondering
myself...:

Is the netbios name just a name, without any connection to anything in AD?

Is the --user-name/--password a one-time used account that's only necessary
when executing the mmuserauth command, or will it also be for communication
between CES and AD while the services are running?



  -jf




On Mon, Aug 22, 2016 at 1:59 PM, Sobey, Richard A 
wrote:

> Hi all,
>
>
>
> We’re just about to start testing a new CES 4.2.0 cluster and at the stage
> of “joining” the cluster to our AD. What’s the bare minimum we need to get
> going with this? My Windows guy (who is more Linux but whatever) has
> suggested the following:
>
>
>
> mmuserauth service create --type ad --data-access-method file
>
> --netbios-name store --user-name USERNAME --password
>
> --enable-nfs-kerberos --enable-kerberos
>
> --servers list,of,servers
>
> --idmap-range-size 100 --idmap-range 300 - 350
> --unixmap-domains 'DOMAIN(500 - 200)'
>
>
>
> He has also asked what the following is:
>
>
>
> --idmap-role ???
>
> --idmap-range-size ??
>
>
>
> All our LDAP GID/UIDs are coming from a system outside of GPFS so do we
> leave this blank, or say master Or, now I’ve re-read and mmuserauth page,
> is this purely for when you have AFM relationships and one GPFS cluster
> (the subordinate / the second cluster) gets its UIDs and GIDs from another
> GPFS cluster (the master / the first one)?
>
>
>
> For idmap-range-size is this essentially the highest number of users and
> groups you can have defined within Spectrum Scale? (I love how I’m using
> GPFS and SS interchangeably.. forgive me!)
>
>
>
> Many thanks
>
>
>
> Richard
>
>
>
>
>
> Richard Sobey
>
> Storage Area Network (SAN) Analyst
> Technical Operations, ICT
> Imperial College London
> South Kensington
> 403, City & Guilds Building
> London SW7 2AZ
> Tel: +44 (0)20 7594 6915
> Email: r.so...@imperial.ac.uk
> http://www.imperial.ac.uk/admin-services/ict/
>
>
>
> ___
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] GPFS FPO

[Brian Marshall]

Aaron,

Do you have experience running this on native GPFS?  The docs say Lustre
and any NFS filesystem.

Thanks,
Brian

On Aug 22, 2016 10:37 PM, "Aaron Knister"  wrote:

> Yes, indeed. Note that these are my personal opinions.
>
> It seems to work quite well and it's not terribly hard to set up or get
> running. That said, if you've got a traditional HPC cluster with reasonably
> good bandwidth (and especially if your data is already on the HPC cluster)
> I wouldn't bother with FPO and just use something like magpie (
> https://github.com/LLNL/magpie) to run your hadoopy workload on GPFS on
> your traditional HPC cluster. I believe FPO (and by extension data
> locality) is important when the available bandwidth between your clients
> and servers/disks (in a traditional GPFS environment) is less than the
> bandwidth available within a node (e.g. between your local disks and the
> host CPU).
>
> -Aaron
>
> On 8/22/16 10:23 PM, Brian Marshall wrote:
>
>> Does anyone have any experiences to share (good or bad) about setting up
>> and utilizing FPO for hadoop compute on top of GPFS?
>>
>>
>> ___
>> gpfsug-discuss mailing list
>> gpfsug-discuss at spectrumscale.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>
>>
> --
> Aaron Knister
> NASA Center for Climate Simulation (Code 606.2)
> Goddard Space Flight Center
> (301) 286-2776
> ___
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss