Re: [CentOS] NFS Server Centos7

[Barbara Krašovec]

NFS4.x works best with newer kernels. Maybe you could try with an elrepo 
kernel.


Increase the number of NFS threads on the NFS server (default is 8, 
which is too little).


|RPCNFSDCOUNT=32|

This is an example of kernel tunning for 10Gb network, adjust to your setup:

vm.min_free_kbytes = 1048576

  vm.swappiness = 10

  net.core.somaxconn = 4096

  # set max buffers to 64MB
  net.core.rmem_max = 67108864
  net.core.wmem_max = 67108864

  # increase Linux autotuning TCP buffer size to 32MB
  net.ipv4.tcp_rmem = 4096 87380 33554432
  net.ipv4.tcp_wmem = 4096 65536 33554432

  net.core.netdev_max_backlog = 25

  net.ipv4.neigh.default.unres_qlen = 100

  net.ipv4.neigh.ens3f0.unres_qlen = 100

  net.ipv4.neigh.ens3f1.unres_qlen = 100

  fs.file-max = 98584540

You can calculate the buffer size with this tool:

https://www.switch.ch/network/tools/tcp_throughput/?do+new+calculation=do+new+calculation 
 



Enable readahead for the filesystem for instance:

blockdev --setra 16384 /dev/mapper/mpatha1

for x in sd[a-d]; do blockdev --setra 16384 /dev/$x ; done



It also helps if you disable flowcontrol on the network cards (and 
switches).


You could also try to reduce the maximum flow rate on the NFS host (set 
max rate to 8gbit for example)


It would be best to test the actual speed by iperf3 test as someone else 
already suggested. But since the performance it so bad, I would suspect 
that the source of the problem is in the the network, not the server.


Cheers,

Barbara


On 5/29/22 01:52, Valeri Galtsev wrote:



On 5/28/22 9:43 AM, Erik Frangež via CentOS wrote:

Hi guys,

we are setting NFS server on CentOS7 system. Everything working OK 
except speed, speed over NFS very drop... if we run dd command 
directly on server we are getting speed around 1,4Gbps, if we run 
from client connected to NFS is 200Mbps.


Do you have maybe some advice what we need to check?



Make sure you have good speed of physical connection along the whole 
path from server to client(s), enable Jumbo packets on all switches 
along the path. NFS experts will add NFS specific tuning.


Valeri


Thank you!

Best, Erik

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

[CentOS] NFS Server Centos7

[Erik Frangež via CentOS]

Hi guys,

 

Thank you all for replys and sorry I was having enabled digest mode and I did 
not get all messages in time.

 

Here are answers:

 

Message 2:

Client and server are virtual machines inside vmware environment which is 
connected to 10gbps. Network is devided between virtual machines. Physical 
servers are connected to Nexus switch with 10gbps.

 

Message 3:

We have try speeds with iperf, they are between 1 and 6 gbps, sometimes goes to 
8gbps - depends of the load of networks. All checks was done with sync mode 
(NFS will be used for important data and we do not want to lose something).

 

Message 4:

Config is default, just installed and tested with NFSv3 NFSv4.

 

Message 5:

We did not enable jumbo frames on network. 

 

For all: We are testing speed with this script (PHP7.4):

 

 1, 'over' => 1.25, 'fail' => 
BENCHFAIL_SLOWHARDDRIVE);

 

 

?>

 

On 29/05/2022, 14:00, "centos-boun...@centos.org on behalf of 
centos-requ...@centos.org"  wrote:

 

 

 

    --

 

    Message: 1

    Date: Sat, 28 May 2022 15:43:12 +0200

    From: Erik Frange? 

    To: centos@centos.org

    Subject: [CentOS] NFS Server Centos7

    Message-ID: <6dfea2b2-1a90-059c-8ffe-f8f7f0775...@frangez.net>

    Content-Type: text/plain; charset=UTF-8; format=flowed

 

    Hi guys,

 

    we are setting NFS server on CentOS7 system. Everything working OK 

except speed, speed over NFS very drop... if we run dd command directly 

on server we are getting speed around 1,4Gbps, if we run from client 

connected to NFS is 200Mbps.

 

    Do you have maybe some advice what we need to check?

 

    Thank you!

 

    Best, Erik

 

 

 

    --

 

    Message: 2

    Date: Sat, 28 May 2022 10:40:10 -0600

    From: Frank Cox 

    To: centos@centos.org

    Subject: Re: [CentOS] NFS Server Centos7

    Message-ID: <20220528104010.b28a86a9d8ac9186f2fc3...@sasktel.net>

    Content-Type: text/plain; charset=UTF-8

 

    On Sat, 28 May 2022 15:43:12 +0200

    Erik Frange? via CentOS wrote:

 

    > we are setting NFS server on CentOS7 system. Everything working OK 

> except speed, speed over NFS very drop... if we run dd command directly 

> on server we are getting speed around 1,4Gbps, if we run from client 

> connected to NFS is 200Mbps.

    > 

> Do you have maybe some advice what we need to check?

 

    Speed of network card in the server.

    Speed of network card in the client.

    Speed of any and all routers and switches between the server and the client.

 

    -- 

MELVILLE THEATRE ~ Real D 3D Digital Cinema ~ www.melvilletheatre.com

 

 

    --

 

    Message: 3

    Date: Sat, 28 May 2022 12:07:17 -0700

    From: Skylar Thompson 

    To: centos@centos.org

    Subject: Re: [CentOS] NFS Server Centos7

    Message-ID: <20220528190717.ibbohtdtx4bzkl2w@hithlum>

    Content-Type: text/plain; charset=utf-8

 

    I would start with something like iperf to measure the actual network

    throughput b/w the client and server. Once you have a baseline for that,

    we'd have to know things like read and write buffer sizes, and sync vs

    async mode.

 

    On Sat, May 28, 2022 at 03:43:12PM +0200, Erik Frange? via CentOS wrote:

    > Hi guys,

    > 

> we are setting NFS server on CentOS7 system. Everything working OK except

    > speed, speed over NFS very drop... if we run dd command directly on server

    > we are getting speed around 1,4Gbps, if we run from client connected to 
NFS

    > is 200Mbps.

    > 

> Do you have maybe some advice what we need to check?

    > 

> Thank you!

    > 

> Best, Erik

 

    -- 

-- Skylar Thompson (skyl...@u.washington.edu)

    -- Genome Sciences Department (UW Medicine), System Administrator

    -- Foege Building S046, (206)-685-7354

    -- Pronouns: He/Him/His

 

 

    --

 

    Message: 4

    Date: Sat, 28 May 2022 21:08:14 +0200

    From: Leon Fauster 

    To: centos@centos.org

    Subject: Re: [CentOS] NFS Server Centos7

    Message-ID: <9f619570-23b6-eb0a-c68a-dfd744fbd...@gmail.com>

    Content-Type: text/plain; charset=UTF-8; format=flowed

 

    Am 28.05.22 um 18:40 schrieb Frank Cox:

    > On Sat, 28 May 2022 15:43:12 +0200

    > Erik Frange? via CentOS wrote:

    > 

>> we are setting NFS server on CentOS7 system. Everything working OK

    >> except speed, speed over NFS very drop... if we run dd command directly

    >> on server we are getting speed around 1,4Gbps, if we run from client

    >> connected to NFS is 200Mbps.

    >>

    >> Do you have maybe some advice what we need to check?

    > 

> Speed of network card in the server.

    > Sp

Re: [CentOS] NFS Server Centos7

[Valeri Galtsev]

On 5/28/22 9:43 AM, Erik Frangež via CentOS wrote:

Hi guys,

we are setting NFS server on CentOS7 system. Everything working OK 
except speed, speed over NFS very drop... if we run dd command directly 
on server we are getting speed around 1,4Gbps, if we run from client 
connected to NFS is 200Mbps.


Do you have maybe some advice what we need to check?



Make sure you have good speed of physical connection along the whole 
path from server to client(s), enable Jumbo packets on all switches 
along the path. NFS experts will add NFS specific tuning.


Valeri


Thank you!

Best, Erik

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] NFS Server Centos7

[Leon Fauster via CentOS]

Am 28.05.22 um 18:40 schrieb Frank Cox:

On Sat, 28 May 2022 15:43:12 +0200
Erik Frangež via CentOS wrote:


we are setting NFS server on CentOS7 system. Everything working OK
except speed, speed over NFS very drop... if we run dd command directly
on server we are getting speed around 1,4Gbps, if we run from client
connected to NFS is 200Mbps.

Do you have maybe some advice what we need to check?


Speed of network card in the server.
Speed of network card in the client.
Speed of any and all routers and switches between the server and the client.



and what config is in place ?

--
Leon
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] NFS Server Centos7

[Skylar Thompson]

I would start with something like iperf to measure the actual network
throughput b/w the client and server. Once you have a baseline for that,
we'd have to know things like read and write buffer sizes, and sync vs
async mode.

On Sat, May 28, 2022 at 03:43:12PM +0200, Erik Frangež via CentOS wrote:
> Hi guys,
> 
> we are setting NFS server on CentOS7 system. Everything working OK except
> speed, speed over NFS very drop... if we run dd command directly on server
> we are getting speed around 1,4Gbps, if we run from client connected to NFS
> is 200Mbps.
> 
> Do you have maybe some advice what we need to check?
> 
> Thank you!
> 
> Best, Erik

-- 
-- Skylar Thompson (skyl...@u.washington.edu)
-- Genome Sciences Department (UW Medicine), System Administrator
-- Foege Building S046, (206)-685-7354
-- Pronouns: He/Him/His
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] NFS Server Centos7

[Frank Cox]

On Sat, 28 May 2022 15:43:12 +0200
Erik Frangež via CentOS wrote:

> we are setting NFS server on CentOS7 system. Everything working OK 
> except speed, speed over NFS very drop... if we run dd command directly 
> on server we are getting speed around 1,4Gbps, if we run from client 
> connected to NFS is 200Mbps.
> 
> Do you have maybe some advice what we need to check?

Speed of network card in the server.
Speed of network card in the client.
Speed of any and all routers and switches between the server and the client.

-- 
MELVILLE THEATRE ~ Real D 3D Digital Cinema ~ www.melvilletheatre.com
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

[CentOS] NFS Server Centos7

[Erik Frangež via CentOS]

Hi guys,

we are setting NFS server on CentOS7 system. Everything working OK 
except speed, speed over NFS very drop... if we run dd command directly 
on server we are getting speed around 1,4Gbps, if we run from client 
connected to NFS is 200Mbps.


Do you have maybe some advice what we need to check?

Thank you!

Best, Erik

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos