Using a placement policy to an external pool seems an excellent idea.
However as I understand it every file has a GPFS attribute as to which *internal* pool it belongs.
If a file is transparently fetched back from an external pool such as S3 Object , TSM or LTFS tape then it is is to this pool that it returns.
Also it is this internal pool that the first block resides even if the rest of the file is HSMed somewhere else?
Daniel
| | Dr Daniel Kidger IBM Technical Sales Specialist Software Defined Solution Sales +44-(0)7818 522 266 [email protected] |
----- Original message -----
From: Edward Wahl <[email protected]>
Sent by: [email protected]
To: <[email protected]>
Cc:
Subject: Re: [gpfsug-discuss] Introduction/Question
Date: Mon, Nov 6, 2017 7:43 PM
On Mon, 6 Nov 2017 09:20:11 +0000
"Chase, Peter" <[email protected]> wrote:
> how can I automate sending files to a cloud object store as they arrive in
> GPFS and keep a copy of the file in GPFS?
Sounds like you already have an idea how to do this by using ILM policies.
Either quota based as you mention or 'placement' policies should work, though I
cannot speak to placement in an S3 environment, the policy engine has a way to
call external commands for that if necessary. Though if you create an external
pool, a placement policy may be much simpler and possibly faster as well as
data would be sent to S3 on write, rather than on a quota trigger.
If an external storage pool works properly for S3, I'd probably use a placement
policy myself. This also would depend on how/when I needed the data on S3 and
your mention of timeliness tells me placement rather than quota may be best.
Weighing the solutions for this may be better tested(and timed!) than anything.
EVERYONE wants a timely weather forecast. ^_-
Ed
--
Ed Wahl
Ohio Supercomputer Center
614-292-9302
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=lkq-3LnnOZn49mjQUkTsxZogZSU7Wlu9JgMssyfPGy8&s=mSz97ZpTbPWXyFjwhJi4mPF6S_dtloQOTmDWfR2UYfw&e=
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
