Hello Ed. Jordan contacted me about the question you are posing so I am responding to your message. Could you please provide clarification as to why the existence of the unbalanced flag is of a concern, or why you would want to know all the files that have this flag set? The flag would be cleared once the file was rebalanced either through normal access or through the execution of the mmrestripefs/mmrestripefile commands.
Fred
__________________________________________________
Fred Stock | IBM Pittsburgh Lab | 720-430-8821
[email protected]
----- Original message -----
From: "Wahl, Edward" <[email protected]>
Sent by: [email protected]
To: gpfsug main discussion list <[email protected]>
Cc:
Subject: [EXTERNAL] Re: [gpfsug-discuss] policy ilm features?
Date: Tue, Feb 2, 2021 11:52 AM
Replying to a 3 year old message I sent, hoping that in the last couple of years that Scale has added some ILM extensions into the policy engine that I have missed, or somehow didn't notice?Just ran into a file with an 'unbalanced' flag and I REALLY don't want to have to mmlsattr everything. AGAIN. /facepalmIBM? Bueller? Bueller?When everyone answers: "No", I'm guessing this needs to be a request for improvement/enhancement?Ed WahlOhio Supercomputer CenterFrom: [email protected] <[email protected]> on behalf of Edward Wahl <[email protected]>
Sent: Friday, February 2, 2018 3:23 PM
To: John Hearns <[email protected]>
Cc: gpfsug main discussion list <[email protected]>
Subject: Re: [gpfsug-discuss] policy ilm features?
Thanks John, this was the path I was HOPING to go down as I do similar things
already, but there appears to be no extended attribute in ILM for what I want.
Data block replication flag exists in the ILM, but not MetaData, or balance.
Yet these states ARE reported by mmlsattr, so there must be a flag somewhere.
bad MD replication & balance example:
mmlsattr -L /fs/scratch/sysp/ed/180days.pol
file name: /fs/scratch/sysp/ed/180days.pol
metadata replication: 1 max 2
data replication: 1 max 2
flags: illreplicated,unbalanced
Encrypted: yes
File next to it for comparison. note proper MD replication and balance.
mmlsattr -L /fs/scratch/sysp/ed/120days.pol
file name: /fs/scratch/sysp/ed/120days.pol
metadata replication: 2 max 2
data replication: 1 max 2
flags:
Encrypted: yes
misc_attributes flags from a policy run showing no difference in status:
FJAEu -- /fs/scratch/sysp/ed/180days.pol
FJAEu -- /fs/scratch/sysp/ed/120days.pol
File system has MD replication enabled, but not Data, so ALL files show "J" ilm
flag
mmlsfs scratch -m
flag value description
------------------- ------------------------ -----------------------------------
-m 2 Default number of metadata replicas
mmlsfs scratch -r
flag value description
------------------- ------------------------ -----------------------------------
-r 1 Default number of data replicas
I poked around a little trying to find out if perhaps using GetXattr would
work and show me what I wanted, it does not. All I sem to be able to get is the
File Encryption Key.
I was hoping perhaps someone had found a cheaper way for this to work rather
than hundreds of millions of 'mmlsattr' execs. :-(
On the plus side, I've only run across a few of these and all appear to be
from before we did the MD replication and re-striping. On the minus, I have NO
idea where they are, and they appears to be on both of our filesystems. So
several hundred million files to check.
Ed
On Mon, 22 Jan 2018 08:29:42 +0000
John Hearns <[email protected]> wrote:
> Ed,
> This is not a perfect answer. You need to look at policies for this. I have
> been doing something similar recently.
>
> Something like:
>
> RULE 'list_file' EXTERNAL LIST 'all-files' EXEC
> '/var/mmfs/etc/mmpolicyExec-list' RULE 'listall' list 'all-files'
> SHOW( varchar(kb_allocated) || ' ' || varchar(file_size) || ' ' ||
> varchar(misc_attributes) || ' ' || name || ' ' || fileset_name ) WHERE
> REGEX(misc_attributes,'[J]')
>
>
> So this policy shows the kbytes allocates, file size, the miscellaneous
> attributes, name and fileset name For all files with miscellaneous
> attributes of 'J' which means 'Some data blocks might be ill replicated'
>
>
>
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Edward Wahl
> Sent: Friday, January 19, 2018 10:38 PM To: [email protected]
> Subject: [gpfsug-discuss] policy ilm features?
>
>
> This one has been on my list a long time so I figured I'd ask here first
> before I open an apar or request an enhancement (most likely).
>
> Is there a way using the policy engine to determine the following?
>
> -metadata replication total/current
> -unbalanced file
>
> Looking to catch things like this that stand out on my filesystem without
> having to run several hundred million 'mmlsattr's.
>
> metadata replication: 1 max 2
> flags: unbalanced
>
> Ed
>
>
>
> --
>
> Ed Wahl
> Ohio Supercomputer Center
> 614-292-9302
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://emea01.safelinks.protection.outlook.com/?url="">
> -- The information contained in this communication and any attachments is
> confidential and may be privileged, and is for the sole use of the intended
> recipient(s). Any unauthorized review, use, disclosure or distribution is
> prohibited. Unless explicitly stated otherwise in the body of this
> communication or the attachment thereto (if any), the information is provided
> on an AS-IS basis without any express or implied warranties or liabilities.
> To the extent you are relying on this information, you are doing so at your
> own risk. If you are not the intended recipient, please notify the sender
> immediately by replying to this message and destroy all copies of this
> message and any attachments. Neither the sender nor the company/group of
> companies he or she represents shall be liable for the proper and complete
> transmission of the information contained in this communication, or for any
> delay in its receipt. _______________________________________________
> gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
--
Ed Wahl
Ohio Supercomputer Center
614-292-9302
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
