[gpfsug-discuss] verbsRdmaSend yes or no

2017-11-13 Thread Damir Krstic 
I am missing out on SC17 this year because of some instability with our 2
ESS storage arrays. We have just recently upgraded our ESS to 5.2 and we
have a question about verbRdmaSend setting.

Per IBM and GPFS guidelines for a large cluster, we have this setting off
on all compute nodes. We were able to turn it off on ESS 1 (IO1 and IO2).

However, IBM was unable to turn it off on ESS 2 (IO3 and IO4).

ESS 1 has following filesystem: projects (1PB)
ESS 2 has following filesystems: home and hpc

All our client nodes have this setting off.

So the question is, should we push through and get it disabled on IO3 and
IO4 so that we are consistent across the environment? I assume the answer
is yes. But I would also like to know what the impact is of leaving it
enabled on IO3 and IO4.

Thank you.
Damir
___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] Specifying nodes in commands

2017-11-13 Thread Marc A Kaplan 
My guess is you have some expectation of how things "ought to be" that 
does not match how things actually are.
If you haven't already done so, put some diagnostics into your script, 
such as 

env
hostname
echo "my args are: $*"

And run 

mmapplypolicy with an explicit node list:

   mmapplypolicy /some/small-set-of-files  -P /mypolicyfile -N 
node1,node2,node3   -I test -L 1 -d 1

And see how things go


Hmmm... reading your post again... It seems perhaps you've got some things 
out of order or again, incorrect expectations or model of how the this 
world works...

mmapplypolicy reads your policy rules and scans the files and calls the 
script(s) you've named in the EXEC options of your EXTERNAL rules

The scripts are expected to process file lists -- NOT call mmapplypolicy 
again...

Refer to examples in the documentation, and in samples/ilm  - and try 
them!

--marc





From:   "Chase, Peter" 
To: "'gpfsug-discuss@spectrumscale.org'" 

Date:   11/13/2017 06:15 AM
Subject:Re: [gpfsug-discuss] Specifying nodes in commands
Sent by:gpfsug-discuss-boun...@spectrumscale.org



Hi Marc,

Thanks for your response, there's some handy advice in there that I'll 
look at further.

I'm still struggling a bit with mmapplypolicy and it's -N option.
I've changed my external list command to point at a script, that script 
looks for "LIST" as the first argument, and runs 
"/usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -d 1 -N cloudNode -P 
/gpfs1/s3upload/policies/migration.policy 
>>/gpfs1/s3upload/external-list.log 2>&1".

If the script is run from the command line on a node that's not in 
cloudNode class it works without issue and uses nodes in the cloudNode 
class as helpers, but if the script is called from the active policy, 
mmapplypolicy runs, but seems to ignore the -N and doesn't use the 
cloudNode nodes as helpers and instead seems to run locally (from which 
ever node started the active policy).

So now my questions is: why does the -N option appear to be honoured when 
run from the command line, but not appear to be honoured when triggered by 
the active policy?

Regards,

Peter Chase
GPCS Team
Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United Kingdom
Tel: +44 (0)1392 886921
Email: peter.ch...@metoffice.gov.uk Website: www.metoffice.gov.uk 

___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=tNW4WqkmstX3B3t1dvbenDx32bw3S1FQ4BrpLrs1r4o&s=CBzS6KRLe_hQhI4zpeeuvNaYdraGbc7cCV-JTvCgDcM&e=






___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] Specifying nodes in commands

2017-11-13 Thread Chase, Peter 
Hi Marc,

Thanks for your response, there's some handy advice in there that I'll look at 
further.

I'm still struggling a bit with mmapplypolicy and it's -N option.
I've changed my external list command to point at a script, that script looks 
for "LIST" as the first argument, and runs "/usr/lpp/mmfs/bin/mmapplypolicy 
/gpfs1/aws -d 1 -N cloudNode -P /gpfs1/s3upload/policies/migration.policy 
>>/gpfs1/s3upload/external-list.log 2>&1".

If the script is run from the command line on a node that's not in cloudNode 
class it works without issue and uses nodes in the cloudNode class as helpers, 
but if the script is called from the active policy, mmapplypolicy runs, but 
seems to ignore the -N and doesn't use the cloudNode nodes as helpers and 
instead seems to run locally (from which ever node started the active policy).

So now my questions is: why does the -N option appear to be honoured when run 
from the command line, but not appear to be honoured when triggered by the 
active policy?

Regards,

Peter Chase
GPCS Team
Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United Kingdom
Tel: +44 (0)1392 886921
Email: peter.ch...@metoffice.gov.uk Website: www.metoffice.gov.uk 

___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss