Khanh Could you tell us what the policy file name is or where to get it?
Thanks Jeff From: [email protected] <[email protected]> On Behalf Of Khanh V Ngo Sent: Thursday, May 3, 2018 10:30 AM To: [email protected] Subject: Re: [gpfsug-discuss] Recharging where HSM is used Specifically with IBM Spectrum Archive EE, there is a script (mmapplypolicy with list rules and python since it outputs many different tables) to provide the total size of user files by file states. This way you can charge more for files that remain on disk and charge less for files migrated to tape. I have seen various prices for the chargeback so it's probably better to calculate based on your environment. The script can easily be changed to output based on GID, filesets, etc. Here's a snippet of the output (in human-readable units): +-------+-----------+-------------+-------------+-----------+ | User | Migrated | Premigrated | Resident | TOTAL | +-------+-----------+-------------+-------------+-----------+ | 0 | 1.563 KB | 50.240 GB | 6.000 bytes | 50.240 GB | | 27338 | 9.338 TB | 1.566 TB | 63.555 GB | 10.965 TB | | 27887 | 58.341 GB | 191.653 KB | | 58.341 GB | | 27922 | 2.111 MB | | | 2.111 MB | | 24089 | 4.657 TB | 22.921 TB | 433.660 GB | 28.002 TB | | 29657 | 29.219 TB | 32.049 TB | | 61.268 TB | | 29210 | 3.057 PB | 399.908 TB | 47.448 TB | 3.494 PB | | 23326 | 7.793 GB | 257.005 MB | 166.364 MB | 8.207 GB | | TOTAL | 3.099 PB | 456.492 TB | 47.933 TB | 3.592 PB | +-------+-----------+-------------+-------------+-----------+ Thanks, Khanh Khanh Ngo, Tape Storage Test Architect Senior Technical Staff Member and Master Inventor Tie-Line 8-321-4802 External Phone: (520)799-4802 9042/1/1467 Tucson, AZ [email protected]<mailto:[email protected]> (internet) It's okay to not understand something. It's NOT okay to test something you do NOT understand. ----- Original message ----- From: [email protected]<mailto:[email protected]> Sent by: [email protected]<mailto:[email protected]> To: [email protected]<mailto:[email protected]> Cc: Subject: gpfsug-discuss Digest, Vol 76, Issue 7 Date: Thu, May 3, 2018 8:19 AM Send gpfsug-discuss mailing list submissions to [email protected]<mailto:[email protected]> To subscribe or unsubscribe via the World Wide Web, visit https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KV_9f1Z5cuLb_ISUqPwkvwc38LRXRLXwKd7w3_A1HS8&m=lrEfoX8I-iJKZPqhVJc0DTlof2GckqjbexR-HavAMOY&s=kOxgdZpL-0VRJL_Vcpst3jnumGGDVZ4mZ9JL7nW0_eA&e= or, via email, send a message with subject or body 'help' to [email protected]<mailto:[email protected]> You can reach the person managing the list at [email protected]<mailto:[email protected]> When replying, please edit your Subject line so it is more specific than "Re: Contents of gpfsug-discuss digest..." Today's Topics: 1. Re: Recharging where HSM is used (Sobey, Richard A) 2. Re: Spectrum Scale CES and remote file system mounts (Mathias Dietz) ---------------------------------------------------------------------- Message: 1 Date: Thu, 3 May 2018 15:02:51 +0000 From: "Sobey, Richard A" <[email protected]<mailto:[email protected]>> To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Subject: Re: [gpfsug-discuss] Recharging where HSM is used Message-ID: <vi1pr06mb320080bcab0499af166a59e4df...@vi1pr06mb3200.eurprd06.prod.outlook.com<mailto:vi1pr06mb320080bcab0499af166a59e4df...@vi1pr06mb3200.eurprd06.prod.outlook.com>> Content-Type: text/plain; charset="utf-8" Stephen, Bryan, Thanks for the input, it?s greatly appreciated. For us we?re trying ? as many people are ? to drive down the usage of under-the-desk NAS appliances and USB HDDs. We offer space on disk, but you can?t charge for 3TB of storage the same as you would down PC World and many customers don?t understand the difference between what we do, and what a USB disk offers. So, offering tape as a medium to store cold data, but not archive data, is one offering we?re just getting round to discussing. The solution is in place. To answer the specific question: for our customers that adopt HSM, how much less should/could/can we charge them per TB. We know how much a tape costs, but we don?t necessarily have the means (or knowledge?) to say that for a given fileset, 80% of the data is on tape. Then you get into 80% of 1TB is not the same as 80% of 10TB. Richard From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> On Behalf Of Stephen Ulmer Sent: 03 May 2018 14:03 To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Subject: Re: [gpfsug-discuss] Recharging where HSM is used I work for a partner, but I occasionally have to help customers work on cost justification that includes charge-back (or I encourage them to do show-back to alleviate some political concerns). I?d also like to see what people are doing around this. If I may ask a question, what is the goal for your site? Are you trying to figure out how to charge for the tape space, or to NOT charge the migrated files as resident? Would you (need to) charge for pre-migrated files twice? Are you trying to figure out how to have users pay for recalls? Basically, what costs are you trying to cover? I realize that was not ?a? question? :) Also, do you specifically mean TSM HSM, or do you mean GPFS policies and an external storage pool? -- Stephen On May 3, 2018, at 5:43 AM, Sobey, Richard A <[email protected]<mailto:[email protected]>> wrote: Hi all, I?d be interested to talk to anyone that is using HSM to move data to tape, (and stubbing the file(s)) specifically any strategies you?ve employed to figure out how to charge your customers (where you do charge anyway) based on usage. On-list or off is fine with me. Thanks Richard _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org_&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KV_9f1Z5cuLb_ISUqPwkvwc38LRXRLXwKd7w3_A1HS8&m=lrEfoX8I-iJKZPqhVJc0DTlof2GckqjbexR-HavAMOY&s=T_TgIy-09AYM6gPWskPSxePi3tM5MADG_NorJCiEjUI&e=> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KV_9f1Z5cuLb_ISUqPwkvwc38LRXRLXwKd7w3_A1HS8&m=lrEfoX8I-iJKZPqhVJc0DTlof2GckqjbexR-HavAMOY&s=kOxgdZpL-0VRJL_Vcpst3jnumGGDVZ4mZ9JL7nW0_eA&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_pipermail_gpfsug-2Ddiscuss_attachments_20180503_dcde0c3e_attachment-2D0001.html&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KV_9f1Z5cuLb_ISUqPwkvwc38LRXRLXwKd7w3_A1HS8&m=lrEfoX8I-iJKZPqhVJc0DTlof2GckqjbexR-HavAMOY&s=Kg8NWE_XWsVduKDWJAVO47iYJmhc9Z9kqdMoHkZ-xlE&e=> ------------------------------ Message: 2 Date: Thu, 3 May 2018 17:14:20 +0200 From: "Mathias Dietz" <[email protected]<mailto:[email protected]>> To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Subject: Re: [gpfsug-discuss] Spectrum Scale CES and remote file system mounts Message-ID: <ofe5d1e6a6.3395b08a-onc1258282.00538cbe-c1258282.0053b...@notes.na.collabserv.com<mailto:ofe5d1e6a6.3395b08a-onc1258282.00538cbe-c1258282.0053b...@notes.na.collabserv.com>> Content-Type: text/plain; charset="iso-8859-1" yes, deleting all NFS exports which point to a given file system would allow you to unmount it without bringing down the other file systems. Mit freundlichen Gr??en / Kind regards Mathias Dietz Spectrum Scale Development - Release Lead Architect (4.2.x) Spectrum Scale RAS Architect --------------------------------------------------------------------------- IBM Deutschland Am Weiher 24 65451 Kelsterbach Phone: +49 70342744105 Mobile: +49-15152801035 E-Mail: [email protected]<mailto:[email protected]> ----------------------------------------------------------------------------- IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina Koederitz, Gesch?ftsf?hrung: Dirk WittkoppSitz der Gesellschaft: B?blingen / Registergericht: Amtsgericht Stuttgart, HRB 243294 From: [email protected]<mailto:[email protected]> To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Date: 03/05/2018 16:41 Subject: Re: [gpfsug-discuss] Spectrum Scale CES and remote file system mounts Sent by: [email protected]<mailto:[email protected]> Thanks Mathiaz, Yes i do understand the concern, that if one of the remote file systems go down abruptly - the others will go down too. However, i suppose we could bring down one of the filesystems before a planned downtime? For example, by unexporting the filesystems on NFS/SMB before the downtime? I might not want to be in a situation, where i have to bring down all the remote filesystems because of planned downtime of one of the remote clusters. Regards, Lohit On May 3, 2018, 7:41 AM -0400, Mathias Dietz <[email protected]<mailto:[email protected]>>, wrote: Hi Lohit, >I am thinking of using a single CES protocol cluster, with remote mounts from 3 storage clusters. Technically this should work fine (assuming all 3 clusters use the same uids/guids). However this has not been tested in our Test lab. >One thing to watch, be careful if your CES root is on a remote fs, as if that goes away, so do all CES exports. Not only the ces root file system is a concern, the whole CES cluster will go down if any remote file systems with NFS exports is not available. e.g. if remote cluster 1 is not available, the CES cluster will unmount the corresponding file system which will lead to a NFS failure on all CES nodes. Mit freundlichen Gr??en / Kind regards Mathias Dietz Spectrum Scale Development - Release Lead Architect (4.2.x) Spectrum Scale RAS Architect --------------------------------------------------------------------------- IBM Deutschland Am Weiher 24 65451 Kelsterbach Phone: +49 70342744105 Mobile: +49-15152801035 E-Mail: [email protected]<mailto:[email protected]> ----------------------------------------------------------------------------- IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina Koederitz, Gesch?ftsf?hrung: Dirk WittkoppSitz der Gesellschaft: B?blingen / Registergericht: Amtsgericht Stuttgart, HRB 243294 From: [email protected]<mailto:[email protected]> To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Date: 01/05/2018 16:34 Subject: Re: [gpfsug-discuss] Spectrum Scale CES and remote file system mounts Sent by: [email protected]<mailto:[email protected]> Thanks Simon. I will make sure i am careful about the CES root and test nfs exporting more than 2 remote file systems. Regards, Lohit On Apr 30, 2018, 5:57 PM -0400, Simon Thompson (IT Research Support) <[email protected]<mailto:[email protected]>>, wrote: You have been able to do this for some time, though I think it's only just supported. We've been exporting remote mounts since CES was added. At some point we've had two storage clusters supplying data and at least 3 remote file-systems exported over NFS and SMB. One thing to watch, be careful if your CES root is on a remote fs, as if that goes away, so do all CES exports. We do have CES root on a remote fs and it works, just be aware... Simon ________________________________________ From: [email protected]<mailto:[email protected]> [[email protected]] on behalf of [email protected]<mailto:[email protected]> [[email protected]] Sent: 30 April 2018 22:11 To: gpfsug main discussion list Subject: [gpfsug-discuss] Spectrum Scale CES and remote file system mounts Hello All, I read from the below link, that it is now possible to export remote mounts over NFS/SMB. https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adv_protocoloverremoteclu.htm I am thinking of using a single CES protocol cluster, with remote mounts from 3 storage clusters. May i know, if i will be able to export the 3 remote mounts(from 3 storage clusters) over NFS/SMB from a single CES protocol cluster? Because according to the limitations as mentioned in the below link: https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adv_limitationofprotocolonRMT.htm It says ?You can configure one storage cluster and up to five protocol clusters (current limit).? Regards, Lohit _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KV_9f1Z5cuLb_ISUqPwkvwc38LRXRLXwKd7w3_A1HS8&m=lrEfoX8I-iJKZPqhVJc0DTlof2GckqjbexR-HavAMOY&s=kOxgdZpL-0VRJL_Vcpst3jnumGGDVZ4mZ9JL7nW0_eA&e= _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KV_9f1Z5cuLb_ISUqPwkvwc38LRXRLXwKd7w3_A1HS8&m=lrEfoX8I-iJKZPqhVJc0DTlof2GckqjbexR-HavAMOY&s=kOxgdZpL-0VRJL_Vcpst3jnumGGDVZ4mZ9JL7nW0_eA&e= _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KV_9f1Z5cuLb_ISUqPwkvwc38LRXRLXwKd7w3_A1HS8&m=lrEfoX8I-iJKZPqhVJc0DTlof2GckqjbexR-HavAMOY&s=kOxgdZpL-0VRJL_Vcpst3jnumGGDVZ4mZ9JL7nW0_eA&e= _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KV_9f1Z5cuLb_ISUqPwkvwc38LRXRLXwKd7w3_A1HS8&m=lrEfoX8I-iJKZPqhVJc0DTlof2GckqjbexR-HavAMOY&s=kOxgdZpL-0VRJL_Vcpst3jnumGGDVZ4mZ9JL7nW0_eA&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_pipermail_gpfsug-2Ddiscuss_attachments_20180503_32718637_attachment.html&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KV_9f1Z5cuLb_ISUqPwkvwc38LRXRLXwKd7w3_A1HS8&m=lrEfoX8I-iJKZPqhVJc0DTlof2GckqjbexR-HavAMOY&s=h50Fy4alAwKdwU5cRNk4JB3q-8f9aTA2mtAfTS3YuRc&e=> ------------------------------ _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=KV_9f1Z5cuLb_ISUqPwkvwc38LRXRLXwKd7w3_A1HS8&m=lrEfoX8I-iJKZPqhVJc0DTlof2GckqjbexR-HavAMOY&s=kOxgdZpL-0VRJL_Vcpst3jnumGGDVZ4mZ9JL7nW0_eA&e= End of gpfsug-discuss Digest, Vol 76, Issue 7 *********************************************
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
