Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-06 Thread Luis Bolinches
Hi   On top of what has been mentioned here (RAID <-> BS aligment, and many other things) I would suggest to look at the last 4/5 slides of a 2018 (disclaimer: my own) London UG presentation http://files.gpfsug.org/presentations/2018/London/14_LuisBolinches_GPFSUG.pdf   It gives a start on

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-05 Thread Valdis Klētnieks
On Fri, 05 Jun 2020 14:24:27 -, "Saula, Oluwasijibomi" said: > But with the RAID 6 writing costs Vladis explained, it now makes sense why > the write IO was badly affected... > Action [1,2,3,4,A] : The only valid responses are characters from this set: > [1, 2, 3, 4, A] > Action

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-05 Thread Saula, Oluwasijibomi
(Valdis Kl=?utf-8?Q?=c4=93?=tnieks) -- Message: 1 Date: Thu, 04 Jun 2020 21:17:08 -0400 From: "Valdis Kl=?utf-8?Q?=c4=93?=tnieks" To: gpfsug main discussion list Subject: Re: [gpfsug-discuss] Client Latenc

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-04 Thread Valdis Klētnieks
On Thu, 04 Jun 2020 15:33:18 -, "Saula, Oluwasijibomi" said: > However, I still can't understand why write IO operations are 5x more latent > than ready operations to the same class of disks. Two things that may be biting you: First, on a RAID 5 or 6 LUN, most of the time you only need to

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-04 Thread Saula, Oluwasijibomi
(Stephen Ulmer) -- Message: 1 Date: Wed, 3 Jun 2020 22:19:49 -0400 From: Stephen Ulmer To: gpfsug main discussion list Subject: Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average Message-ID: Content-Type: text/plain; chars

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-04 Thread Kumaran Rajaram
chments/20200603/e839fc73/attachment-0001.html > -- Message: 2 Date: Wed, 3 Jun 2020 21:45:05 + From: "Saula, Oluwasijibomi" To: "gpfsug-discuss@spectrumscale.org" Subject: [gpfsug-discuss] Client Latency and High NSD Server L

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-04 Thread Frederick Stock
From the waiters you provided I would guess there is something amiss with some of your storage systems.  Since those waiters are on NSD servers they are waiting for IO requests to the kernel to complete.  Generally IOs are expected to complete in milliseconds, not seconds.  You could look at the

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-04 Thread Stephen Ulmer
hanks to Ulf, Kristy, Bill, Bob and Ted for their help and guidance in > getting this going. > > > > We?re keen to include some user talks and site updates later in the series, > so please let me know if you might be interested in presenting in this format. > > > > Simon Thompson > &

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-03 Thread Wahl, Edward
, IF the hardware is healthy... Ed From: gpfsug-discuss-boun...@spectrumscale.org on behalf of Saula, Oluwasijibomi Sent: Wednesday, June 3, 2020 5:45 PM To: gpfsug-discuss@spectrumscale.org Subject: [gpfsug-discuss] Client Latency and High NSD Server Load

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-03 Thread Uwe Falke
rg" Date: 03/06/2020 23:45 Subject: [EXTERNAL] [gpfsug-discuss] Client Latency and High NSD Server Load Average Sent by:gpfsug-discuss-boun...@spectrumscale.org Hello, Anyone faced a situation where a majority of NSDs have a high load average and a minority don't? Also, i

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-03 Thread Kristy Kallback-Rose
la, Oluwasijibomi" > Sent by: gpfsug-discuss-boun...@spectrumscale.org > To: "gpfsug-discuss@spectrumscale.org" > Cc: > Subject: [EXTERNAL] [gpfsug-discuss] Client Latency and High NSD Server Load > Average > Date: Wed, Jun 3, 2020 5:45 PM > >

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-03 Thread Saula, Oluwasijibomi
2 Date: Wed, 3 Jun 2020 21:45:05 + From: "Saula, Oluwasijibomi" To: "gpfsug-discuss@spectrumscale.org" Subject: [gpfsug-discuss] Client Latency and High NSD Server Load Average Message-ID: Content-Type: text/plain; charset="iso-8859-1"

Re: [gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-03 Thread Frederick Stock
Does the output of mmdf show that data is evenly distributed across your NSDs?  If not that could be contributing to your problem.  Also, are your NSDs evenly distributed across your NSD servers, and the NSD configured so the first NSD server for each is not the same one?

[gpfsug-discuss] Client Latency and High NSD Server Load Average

2020-06-03 Thread Saula, Oluwasijibomi
Hello, Anyone faced a situation where a majority of NSDs have a high load average and a minority don't? Also, is 10x NSD server latency for write operations than for read operations expected in any circumstance? We are seeing client latency between 6 and 9 seconds and are wondering if some