Replying to a 3 year old message I sent, hoping that in the last couple of years that Scale has added some ILM extensions into the policy engine that I have missed, or somehow didn't notice? Just ran into a file with an 'unbalanced' flag and I REALLY don't want to have to mmlsattr everything. AGAIN. /facepalm
IBM? Bueller? Bueller? When everyone answers: "No", I'm guessing this needs to be a request for improvement/enhancement? Ed Wahl Ohio Supercomputer Center ________________________________ From: [email protected] <[email protected]> on behalf of Edward Wahl <[email protected]> Sent: Friday, February 2, 2018 3:23 PM To: John Hearns <[email protected]> Cc: gpfsug main discussion list <[email protected]> Subject: Re: [gpfsug-discuss] policy ilm features? Thanks John, this was the path I was HOPING to go down as I do similar things already, but there appears to be no extended attribute in ILM for what I want. Data block replication flag exists in the ILM, but not MetaData, or balance. Yet these states ARE reported by mmlsattr, so there must be a flag somewhere. bad MD replication & balance example: mmlsattr -L /fs/scratch/sysp/ed/180days.pol file name: /fs/scratch/sysp/ed/180days.pol metadata replication: 1 max 2 data replication: 1 max 2 flags: illreplicated,unbalanced Encrypted: yes File next to it for comparison. note proper MD replication and balance. mmlsattr -L /fs/scratch/sysp/ed/120days.pol file name: /fs/scratch/sysp/ed/120days.pol metadata replication: 2 max 2 data replication: 1 max 2 flags: Encrypted: yes misc_attributes flags from a policy run showing no difference in status: FJAEu -- /fs/scratch/sysp/ed/180days.pol FJAEu -- /fs/scratch/sysp/ed/120days.pol File system has MD replication enabled, but not Data, so ALL files show "J" ilm flag mmlsfs scratch -m flag value description ------------------- ------------------------ ----------------------------------- -m 2 Default number of metadata replicas mmlsfs scratch -r flag value description ------------------- ------------------------ ----------------------------------- -r 1 Default number of data replicas I poked around a little trying to find out if perhaps using GetXattr would work and show me what I wanted, it does not. All I sem to be able to get is the File Encryption Key. I was hoping perhaps someone had found a cheaper way for this to work rather than hundreds of millions of 'mmlsattr' execs. :-( On the plus side, I've only run across a few of these and all appear to be from before we did the MD replication and re-striping. On the minus, I have NO idea where they are, and they appears to be on both of our filesystems. So several hundred million files to check. Ed On Mon, 22 Jan 2018 08:29:42 +0000 John Hearns <[email protected]> wrote: > Ed, > This is not a perfect answer. You need to look at policies for this. I have > been doing something similar recently. > > Something like: > > RULE 'list_file' EXTERNAL LIST 'all-files' EXEC > '/var/mmfs/etc/mmpolicyExec-list' RULE 'listall' list 'all-files' > SHOW( varchar(kb_allocated) || ' ' || varchar(file_size) || ' ' || > varchar(misc_attributes) || ' ' || name || ' ' || fileset_name ) WHERE > REGEX(misc_attributes,'[J]') > > > So this policy shows the kbytes allocates, file size, the miscellaneous > attributes, name and fileset name For all files with miscellaneous > attributes of 'J' which means 'Some data blocks might be ill replicated' > > > > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Edward Wahl > Sent: Friday, January 19, 2018 10:38 PM To: [email protected] > Subject: [gpfsug-discuss] policy ilm features? > > > This one has been on my list a long time so I figured I'd ask here first > before I open an apar or request an enhancement (most likely). > > Is there a way using the policy engine to determine the following? > > -metadata replication total/current > -unbalanced file > > Looking to catch things like this that stand out on my filesystem without > having to run several hundred million 'mmlsattr's. > > metadata replication: 1 max 2 > flags: unbalanced > > Ed > > > > -- > > Ed Wahl > Ohio Supercomputer Center > 614-292-9302 > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=01%7C01%7Cjohn.hearns%40asml.com%7C056e34c5a8df4d8f10fd08d55f91e73c%7Caf73baa8f5944eb2a39d93e96cad61fc%7C1&sdata=dnt7vV4TCd68l7fSJnY35eyNM%2B8pNrZElImSZeZbit8%3D&reserved=0 > -- The information contained in this communication and any attachments is > confidential and may be privileged, and is for the sole use of the intended > recipient(s). Any unauthorized review, use, disclosure or distribution is > prohibited. Unless explicitly stated otherwise in the body of this > communication or the attachment thereto (if any), the information is provided > on an AS-IS basis without any express or implied warranties or liabilities. > To the extent you are relying on this information, you are doing so at your > own risk. If you are not the intended recipient, please notify the sender > immediately by replying to this message and destroy all copies of this > message and any attachments. Neither the sender nor the company/group of > companies he or she represents shall be liable for the proper and complete > transmission of the information contained in this communication, or for any > delay in its receipt. _______________________________________________ > gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss -- Ed Wahl Ohio Supercomputer Center 614-292-9302 _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
