Hi, Saula,
Can the expelled node and expelling node ping each other?
We expanded our gpfs IB network from /24 to /20 but some clients still used
/24, they cannot talk to the added new clients using /20 and expelled the new
clients persistently.
Changing the netmask all to /20 works out.
FYI.
Wei Guo
HPC Administartor
UT Southwestern Medical Center
wei1....@utsouthwestern.edu
________________________________________
From: gpfsug-discuss-boun...@spectrumscale.org
<gpfsug-discuss-boun...@spectrumscale.org> on behalf of
gpfsug-discuss-requ...@spectrumscale.org
<gpfsug-discuss-requ...@spectrumscale.org>
Sent: Thursday, March 8, 2018 11:37 AM
To: gpfsug-discuss@spectrumscale.org
Subject: gpfsug-discuss Digest, Vol 74, Issue 17
Send gpfsug-discuss mailing list submissions to
gpfsug-discuss@spectrumscale.org
To subscribe or unsubscribe via the World Wide Web, visit
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
or, via email, send a message with subject or body 'help' to
gpfsug-discuss-requ...@spectrumscale.org
You can reach the person managing the list at
gpfsug-discuss-ow...@spectrumscale.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of gpfsug-discuss digest..."
Today's Topics:
1. Thoughts on GPFS on IB & MTU sizes (Saula, Oluwasijibomi)
2. Re: wondering about outage free protocols upgrades
(Christof Schmitt)
----------------------------------------------------------------------
Message: 1
Date: Thu, 8 Mar 2018 15:06:03 +0000
From: "Saula, Oluwasijibomi" <oluwasijibomi.sa...@ndsu.edu>
To: "gpfsug-discuss@spectrumscale.org"
<gpfsug-discuss@spectrumscale.org>
Subject: [gpfsug-discuss] Thoughts on GPFS on IB & MTU sizes
Message-ID:
<cy4pr08mb2854ff1706f7b6c59d687be998...@cy4pr08mb2854.namprd08.prod.outlook.com>
Content-Type: text/plain; charset="windows-1252"
Hi Folks,
As this is my first post to the group, let me start by saying I applaud the
commentary from the user group as it has been a resource to those of us
watching from the sidelines.
That said, we have a GPFS layered on IPoIB, and recently, we started having
some issues on our IB FDR fabric which manifested when GPFS began sending
persistent expel messages to particular nodes.
Shortly after, we embarked on a tuning exercise using IBM tuning
recommendations<https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Welcome%20to%20High%20Performance%20Computing%20%28HPC%29%20Central/page/Linux%20System%20Tuning%20Recommendations>
but this page is quite old and we've run into some snags, specifically with
setting 4k MTUs using mlx4_core/mlx4_en module options.
While setting 4k MTUs as the guide recommends is our general inclination, I'd
like to solicit some advice as to whether 4k MTUs are a good idea and any
hitch-free steps to accomplishing this. I'm getting some conflicting remarks
from Mellanox support asking why we'd want to use 4k MTUs with Unreliable
Datagram mode.
Also, any pointers to best practices or resources for network configurations
for heavy I/O clusters would be much appreciated.
Thanks,
Siji Saula
HPC System Administrator
Center for Computationally Assisted Science & Technology
NORTH DAKOTA STATE UNIVERSITY
<https://www.ndsu.edu/alphaindex/buildings/Building::395>Research 2
Building<https://www.ndsu.edu/alphaindex/buildings/Building::396><https://www.ndsu.edu/alphaindex/buildings/Building::395>
? Room 220B
Dept 4100, PO Box 6050 / Fargo, ND 58108-6050
p:701.231.7749
www.ccast.ndsu.edu<file://composeviewinternalloadurl/www.ccast.ndsu.edu> |
www.ndsu.edu<file://composeviewinternalloadurl/www.ndsu.edu>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20180308/0f2fc16f/attachment-0001.html>
------------------------------
Message: 2
Date: Thu, 8 Mar 2018 17:37:12 +0000
From: "Christof Schmitt" <christof.schm...@us.ibm.com>
To: gpfsug-discuss@spectrumscale.org
Subject: Re: [gpfsug-discuss] wondering about outage free protocols
upgrades
Message-ID:
<of84aa7f39.23bfbfaf-on0025824a.005e7d99-0025824a.0060c...@notes.na.collabserv.com>
Content-Type: text/plain; charset="us-ascii"
An HTML attachment was scrubbed...
URL:
<http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20180308/89483e8a/attachment.html>
------------------------------
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
End of gpfsug-discuss Digest, Vol 74, Issue 17
**********************************************
________________________________
UT Southwestern
Medical Center
The future of medicine, today.
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
