Peter, 1. to best exploit and integrate both Spectrum Scale and Cloud Storage, please consider: https://www.ibm.com/blogs/systems/spectrum-scale-transparent-cloud-tiering/
2. Yes, you can use mmapplypolicy to push copies of files to an "external" system. But you'll probably need a strategy or technique to avoid redundantly pushing the "next time" you run the command... 3. Regarding mmapplypolicy nitty-gritty: you can use the -N option to say exactly which nodes you want to run the command. And regarding using ... EXTERNAL ... EXEC 'myscript' You can further restrict which nodes will act as mmapplypolicy "helpers" -- If on a particular node x, 'myscript' does not exist OR myscript TEST returns a non-zero exit code then node x will be excluded.... You will see a message like this: [I] Messages tagged with <3> are from node n3. <3> [E:73] Error on system(/ghome/makaplan/policies/mynodes.sh TEST '/foo/bar5' 2>&1) <3> [W] EXEC '/ghome/makaplan/policies/mynodes.sh' of EXTERNAL POOL or LIST 'x' fails TEST with code 73 on this node. OR [I] Messages tagged with <5> are from node n4. <5> sh: /tmp/mynodes.sh: No such file or directory <5> [E:127] Error on system(/tmp/mynodes.sh TEST '/foo/bar5' 2>&1) <5> [W] EXEC '/tmp/mynodes.sh' of EXTERNAL POOL or LIST 'x' fails TEST with code 127 on this node. From: "Chase, Peter" <[email protected]> To: "'[email protected]'" <[email protected]> Date: 11/08/2017 10:51 AM Subject: [gpfsug-discuss] Default placement/External Pool Sent by: [email protected] Hello! A follow up to my previous question about automatically sending files to Amazon s3 as they arrive in GPFS. I have created an interface script to manage Amazon s3 storage as an external pool, I have created a migration policy that pre-migrates all files to the external pool and I have set that as the default policy for the file system. All good so far, but the problem I'm now facing is: Only some of the cluster nodes have access to Amazon due to network constraints. I read the statement "The mmapplypolicy command invokes the external pool script on all nodes in the cluster that have installed the script in its designated location."[1] and thought, 'Great! I'll only install the script on nodes that have access to Amazon' but that appears not to work for a placement policy/default policy and instead, the script runs on precisely no nodes. I assumed this happened because running the script on a non-Amazon facing node resulted in a horrible error (i.e. file not found), so I edited my script to return a non-zero response if being run on a node that isn't in my cloudNode class, then installed the script every where. But this appears to have had no effect what-so-ever.
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
