Re: [gpfsug-discuss] gpfsug-discuss Digest, Vol 113, Issue 19

2021-06-22 Thread Saula, Oluwasijibomi
Today's Topics: 1. GPFS bad at memory-mapped files? (Saula, Oluwasijibomi) 2. Re: GPFS bad at memory-mapped files? (Simon Thompson) -- Message: 1 Date: Tue, 22 Jun 2021 15:17:16 +0000 From: "Saula, Oluwasijibomi&

[gpfsug-discuss] GPFS bad at memory-mapped files?

2021-06-22 Thread Saula, Oluwasijibomi
Hello, While reviewing AMS software suite for installation, I noticed this remark (https://www.scm.com/doc/Installation/Additional_Information_and_Known_Issues.html#gpfs-file-system): - GPFS file

[gpfsug-discuss] Long IO waiters and IBM Storwize V5030

2021-05-28 Thread Saula, Oluwasijibomi
Hi Folks, So, we are experiencing some very long IO waiters in our GPFS cluster: # mmdiag --waiters === mmdiag: waiters === Waiting 17.3823 sec since 10:41:01, monitored, thread 21761 NSDThread: for I/O completion Waiting 16.6140 sec since 10:41:02, monitored, thread 21730 NSDThread: for

Re: [gpfsug-discuss] gpfsug-discuss Digest, Vol 110, Issue 34

2021-03-30 Thread Saula, Oluwasijibomi
ode (Saula, Oluwasijibomi) 2. Re: Filesystem mount attempt hangs GPFS client node (Olaf Weiser) -- Message: 1 Date: Mon, 29 Mar 2021 18:38:00 +0000 From: "Saula, Oluwasijibomi" To: "gpfsug-discuss@spec

[gpfsug-discuss] Filesystem mount attempt hangs GPFS client node

2021-03-29 Thread Saula, Oluwasijibomi
Hello Folks, So we are experiencing a mind-boggling issue where just a couple of nodes in our cluster, at GPFS boot up, get hung so badly that the node must be power reset. These AMD client nodes are diskless in nature and have at least 256G of memory. We have other AMD nodes that are working

Re: [gpfsug-discuss] gpfsug-discuss Digest, Vol 106, Issue 3

2020-11-04 Thread Saula, Oluwasijibomi
Could someone share the password for the event today? Thanks! Thanks, Oluwasijibomi (Siji) Saula HPC Systems Administrator / Information Technology Research 2 Building 220B / Fargo ND 58108-6050 p: 701.231.7749 / www.ndsu.edu [cid:image001.gif@01D57DE0.91C300C0]

Re: [gpfsug-discuss] gpfsug-discuss Digest, Vol 104, Issue 19

2020-09-19 Thread Saula, Oluwasijibomi
ol 104, Issue 18 (Saula, Oluwasijibomi) 2. Re: gpfsug-discuss Digest, Vol 104, Issue 18 (Ryan Novosielski) -- Message: 1 Date: Sat, 19 Sep 2020 20:52:19 +0000 From: "Saula, Oluwasijibomi" To: "g

Re: [gpfsug-discuss] gpfsug-discuss Digest, Vol 104, Issue 18

2020-09-19 Thread Saula, Oluwasijibomi
CR errors (Saula, Oluwasijibomi) 2. Re: CCR errors (Ryan Novosielski) -- Message: 1 Date: Sat, 19 Sep 2020 20:11:31 +0000 From: "Saula, Oluwasijibomi" To: "gpfsug-discuss@spectrumscale.org" Subject:

[gpfsug-discuss] CCR errors

2020-09-19 Thread Saula, Oluwasijibomi
Hello, Anyone available to assist with CCR errors: [root@nsd02 ~]# mmchnode -N nsd04-ib --quorum --manager mmchnode: Unable to obtain the GPFS configuration file lock. Retrying ... Per IBM support's direction, I already followed the Manual Repair

[gpfsug-discuss] Short-term Deactivation of NSD server

2020-09-04 Thread Saula, Oluwasijibomi
Hello GPFS Experts, Say, is there any way to disable a particular NSD server outside of shutting down GPFS on the server, or shutting down the entire cluster and removing the NSD server from the list of NSD servers? I'm finding that TSM activity on one of our NSD servers is stifling IO traffic

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-05 Thread Saula, Oluwasijibomi
y and High NSD Server Load Average Message-ID: <309214.1591319828@turing-police> Content-Type: text/plain; charset="us-ascii" On Thu, 04 Jun 2020 15:33:18 -, "Saula, Oluwasijibomi" said: > However, I still can't understand why write IO operations are 5x more latent &

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-04 Thread Saula, Oluwasijibomi
path the clients are using to the NSDs, i.e. which server. See if you are surprised. :) -- Stephen > On Jun 3, 2020, at 6:03 PM, Saula, Oluwasijibomi > wrote: > > ? > Frederick, > > Yes on both counts! - mmdf is showing pretty uniform (ie 5 NSDs out of 30 > report 65% fr

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-03 Thread Saula, Oluwasijibomi
r)) 2. Client Latency and High NSD Server Load Average (Saula, Oluwasijibomi) 3. Re: Client Latency and High NSD Server Load Average (Frederick Stock) -- Message: 1 Date: Wed, 03 Jun 2020 20:11:17 +0100 From: "Sim

[gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-03 Thread Saula, Oluwasijibomi
Hello, Anyone faced a situation where a majority of NSDs have a high load average and a minority don't? Also, is 10x NSD server latency for write operations than for read operations expected in any circumstance? We are seeing client latency between 6 and 9 seconds and are wondering if some

[gpfsug-discuss] Reason for shutdown: Reset old shared segment

2019-05-23 Thread Saula, Oluwasijibomi
Hey Folks, I got a strange message one of my HPC cluster nodes that I'm hoping to understand better: "Reason for shutdown: Reset old shared segment" 2019-05-23_11:47:07.328-0500: [I] This node has a valid standard license 2019-05-23_11:47:07.327-0500: [I] Initializing the fast condition

Re: [gpfsug-discuss] gpfsug-discuss Digest, Vol 82, Issue 31

2018-11-21 Thread Saula, Oluwasijibomi
ny unused objects.??? If you can identify the user job that is causing this? you could monitor a system more closely. Jim On Wednesday, November 21, 2018, 2:10:45 AM EST, Saula, Oluwasijibomi wrote: Hello Scalers, First, let me say Happy Thanksgiving to those of us in the US and to

[gpfsug-discuss] mmfsd recording High CPU usage

2018-11-20 Thread Saula, Oluwasijibomi
Hello Scalers, First, let me say Happy Thanksgiving to those of us in the US and to those beyond, well, it's a still happy day seeing we're still above ground!  Now, what I have to discuss isn't anything extreme so don't skip the turkey for this, but lately, on a few of our compute GPFS

Re: [gpfsug-discuss] Thoughts on GPFS on IB & MTU sizes

2018-03-10 Thread Saula, Oluwasijibomi
Box 6050 / Fargo, ND 58108-6050 p:701.231.7749 www.ccast.ndsu.edu | www.ndsu.edu From: Edward Wahl <ew...@osc.edu> Sent: Friday, March 9, 2018 8:19:10 AM To: gpfsug-discuss@spectrumscale.org Cc: Saula, Oluwasijibomi Subject: Re: [gpfsug-discuss] Though

[gpfsug-discuss] Thoughts on GPFS on IB & MTU sizes

2018-03-08 Thread Saula, Oluwasijibomi
Hi Folks, As this is my first post to the group, let me start by saying I applaud the commentary from the user group as it has been a resource to those of us watching from the sidelines. That said, we have a GPFS layered on IPoIB, and recently, we started having some issues on our IB FDR