Thanks Alec! On Thu, Mar 31, 2022 at 7:41 AM Alec <[email protected]> wrote:
> I'd like to offer a dissenting opinion... I believe this is a way to > limit the number of actual bytes that any execution must work on.... As in > only send 500gb worth of files to each invocation. Whereas the -B is the > actual number of files to be passed to each invocation. I would think that > size has no default limit as I've had a single invocation stuck with 30TB > of data in its 100 file grouping, lol. > > I however use a different mechanism to divy up my files because I need to > worry about my backup streams getting choked up, and I want to take a cross > section of each directory on different nights... > > I call mmapplypolicy with -M shares=<someincrementingnumber>, so let's say > I increase my number by 1 every day, starting at 0, over the course of a > week... > > In my policy I have this line: > WHERE MOD(INODE,6)<=shares > > So then I work on 1/7th of the files in my file system and cumulatively > pick up any INODES until I've hit all of the files... I could pick just the > matching segment by saying = instead of <=. > > This is happening at the file selection phase, so the application of -B is > after this selection, as my files are quite huge, I tend to use -B 1. > > Alec > > On Wed, Mar 30, 2022 at 3:44 PM IBM Spectrum Scale <[email protected]> > wrote: > >> Hi shao feng, >> >> It is controlled by -B option. >> >> *-B* *MaxFiles* >> Specifies how many files are passed for each invocation >> of the EXEC script. The default value is 100. >> >> If the number of files exceeds the value specified for >> MaxFiles, *mmapplypolicy* invokes the external >> program multiple times. >> >> For more information about file list records, refer to >> the IBM Spectrum Scale: Administration Guide. >> >> Regards, The Spectrum Scale (GPFS) team >> >> >> ------------------------------------------------------------------------------------------------------------------ >> If your query concerns a potential software error in Spectrum Scale >> (GPFS) and you have an IBM software maintenance contract please contact >> 1-800-237-5511 in the United States or your local IBM Service Center in >> other countries. >> >> [image: Inactive hide details for IBM Spectrum Scale---03/30/2022 >> 09:53:51 AM---Hi Diane, Can you please look at the below policy relat]IBM >> Spectrum Scale---03/30/2022 09:53:51 AM---Hi Diane, Can you please look at >> the below policy related query. >> >> From: IBM Spectrum Scale/Poughkeepsie/IBM@IBMUS >> To: "gpfsug main discussion list" <[email protected]>, Diane >> Chung/Poughkeepsie/IBM@IBMMail >> Cc: "gpfsug-discuss" <[email protected]> >> Date: 03/30/2022 09:53 AM >> Subject: [EXTERNAL] Re: [gpfsug-discuss] default value of SIZE in >> EXTERNAL POOL and EXTERNAL LIST rule >> Sent by: "gpfsug-discuss" <[email protected]> >> ------------------------------ >> >> >> >> Hi Diane, >> >> Can you please look at the below policy related query. >> >> Regards, The Spectrum Scale (GPFS) team >> >> >> ------------------------------------------------------------------------------------------------------------------ >> If you feel that your question can benefit other users of Spectrum Scale >> (GPFS), then please post it to the public IBM developerWroks Forum at >> *https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000479* >> <https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000479>. >> >> >> If your query concerns a potential software error in Spectrum Scale >> (GPFS) and you have an IBM software maintenance contract please contact >> 1-800-237-5511 in the United States or your local IBM Service Center in >> other countries. >> >> The forum is informally monitored as time permits and should not be used >> for priority messages to the Spectrum Scale (GPFS) team. >> >> [image: Inactive hide details for "shao feng" ---28-03-2022 03.21.39 >> PM---Hello, What is the default value of SIZE in EXTERNAL POOL in]"shao >> feng" ---28-03-2022 03.21.39 PM---Hello, What is the default value of SIZE >> in EXTERNAL POOL in filesystem policy? >> >> From: "shao feng" <[email protected]> >> To: "gpfsug main discussion list" <[email protected]> >> Date: 28-03-2022 03.21 PM >> Subject: [EXTERNAL] [gpfsug-discuss] default value of SIZE in EXTERNAL >> POOL and EXTERNAL LIST rule >> Sent by: "gpfsug-discuss" <[email protected]> >> ------------------------------ >> >> >> >> Hello, >> >> What is the default value of SIZE in EXTERNAL POOL in filesystem policy? >> >> The document did not mention that, but I found it is not infinity if not >> specified. >> >> Thank you! >> >> *SIZE **sum-number* >> Is an optional clause of the *EXTERNAL POOL* and *EXTERNAL LIST* rules. >> *sum-number* limits the total number of bytes in all of the files named >> in each list of files passed to your EXEC 'script'. If a single file is >> larger than *sum-number*, it is passed to your EXEC 'script' as the only >> entry in a “singleton” file list. >> >> Specify *sum-number* as a numeric constant or a floating-point value. >> >> *Note:* The value of *sum-number* is in kilobytes. >> *https://www.ibm.com/docs/en/spectrum-scale/5.1.2?topic=rules-policy-terms* >> <https://www.ibm.com/docs/en/spectrum-scale/5.1.2?topic=rules-policy-terms> >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> *http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org* >> <http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org> >> >> >> >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org >> >> >> >> >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org >> > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at gpfsug.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
