Re: [lustre-discuss] Lnet Self Test

0 S   2.0  0.0  30:56.32 socknal_sd00_04 64334 root      20   0       0  0  0 D   1.3  0.0   7:19.46 ll_ost_io01_010 64329 root  20   0   0  0  0 S   1.0  0.0   7:46.48 ll_ost_io01_005 From: "Moreno Diego (ID SIS)" Date: Wednesday, December 4, 2019 at 1

Re: [lustre-discuss] Lnet Self Test

To: Jongwoo Han Cc: "lustre-discuss@lists.lustre.org" Subject: Re: [lustre-discuss] Lnet Self Test Thanks Jongwoo. I have the MTU set for 9000 and also ring buffer setting set to max. ip link set dev $primaryNICInterface mtu 9000 ethtool -G $primaryNICInterface rx 2047 tx 2047 rx-

Re: [lustre-discuss] Lnet Self Test

:07 PM To: Pinkesh Valdria Cc: Andreas Dilger , "lustre-discuss@lists.lustre.org" Subject: Re: [lustre-discuss] Lnet Self Test Have you tried MTU >= 9000 bytes (AKA jumbo frame) on the 25G ethernet and the switch? If it is set to 1500 bytes, ethernet + IP + TCP frame heade

Re: [lustre-discuss] Lnet Self Test

50= ) due to overheads on > ethernet with Lnet? > > > > > > Thanks, > > Pinkesh Valdria > > Oracle Cloud Infrastructure > > > > > > > > > > *From: *Andreas Dilger > *Date: *Wednesday, November 27, 2019 at 1:25 AM > *To: *Pin

Re: [lustre-discuss] Lnet Self Test

with Lnet? Thanks, Pinkesh Valdria Oracle Cloud Infrastructure From: Andreas Dilger Date: Wednesday, November 27, 2019 at 1:25 AM To: Pinkesh Valdria Cc: "lustre-discuss@lists.lustre.org" Subject: Re: [lustre-discuss] Lnet Self Test The first thing to note i

Re: [lustre-discuss] Lnet Self Test

The first thing to note is that lst reports results in binary units (MiB/s) while iperf reports results in decimal units (Gbps). If you do the conversion you get 2055.31 MiB/s = 2155 MB/s. The other thing to check is the CPU usage. For TCP the CPU usage can be high. You should try RoCE+o2iblnd

[lustre-discuss] Lnet Self Test

Hello All, I created a new Lustre cluster on CentOS7.6 and I am running lnet_selftest_wrapper.sh to measure throughput on the network.  The nodes are connected to each other using 25Gbps ethernet, so theoretical max is 25 Gbps * 125 = 3125 MB/s.    Using iperf3,  I get 22Gbps (2750 MB/s)

Re: [lustre-discuss] LNET Self-test

Thanks a lot! A related question: is it possible to use the result from the "ping" test to verify the latency obtained from openmpi? Or, how do I know it the result from the "ping" test is "acceptable"? /jon On 02/07/2017 06:38 PM, Oucharek, Doug S wrote: Because the stat command is “lst

Re: [lustre-discuss] LNET Self-test

Because the stat command is “lst stat servers”, the statistics you are seeing are from the perspective of the server. The “from” and “to” parameters can get quite confusing for the read case. When reading, you are transferring the bulk data from the “to” group to the “from” group (yes, seems

Re: [lustre-discuss] LNET Self-test

Probably doing something wrong here, but I tried to test only READING with the following: #!/bin/bash export LST_SESSION=$$ lst new_session read lst add_group servers 10.0.12.12@o2ib lst add_group readers 10.0.12.11@o2ib lst add_batch bulk_read lst add_test --batch bulk_read --concurrency 12

Re: [lustre-discuss] LNET Self-test

Try running just a read test and then just a write test rather than having both at the same time and see if the performance goes up. Doug > On Feb 6, 2017, at 4:40 AM, Jon Tegner wrote: > > Hi, > > I used the following script: > > #!/bin/bash > export LST_SESSION=$$ > lst

Re: [lustre-discuss] LNET Self-test

ilto:doug.s.oucha...@intel.com>> Sent: Sunday, February 5, 2017 3:18:10 PM To: Jeff Johnson Cc: lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> Subject: Re: [lustre-discuss] LNET Self-test Yes, you can bump your concurrency. Size caps out at 1M because that is

Re: [lustre-discuss] LNET Self-test

Hi, I used the following script: #!/bin/bash export LST_SESSION=$$ lst new_session read/write lst add_group servers 10.0.12.12@o2ib lst add_group readers 10.0.12.11@o2ib lst add_group writers 10.0.12.11@o2ib lst add_batch bulk_rw lst add_test --batch bulk_rw --concurrency 12 --from readers --to

Re: [lustre-discuss] LNET Self-test

com> Sent: Sunday, February 5, 2017 3:18:10 PM To: Jeff Johnson Cc: lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] LNET Self-test Yes, you can bump your concurrency. Size caps out at 1M because that is how LNet is setup to work. Going over 1M size would result in an unrealistic

Re: [lustre-discuss] LNET Self-test

Yes, you can bump your concurrency. Size caps out at 1M because that is how LNet is setup to work. Going over 1M size would result in an unrealistic Lustre test. Doug > On Feb 5, 2017, at 11:55 AM, Jeff Johnson > wrote: > > Without seeing your entire

Re: [lustre-discuss] LNET Self-test

Without seeing your entire command it is hard to say for sure but I would make sure your concurrency option is set to 8 for starters. --Jeff Sent from my iPhone > On Feb 5, 2017, at 11:30, Jon Tegner wrote: > > Hi, > > I'm trying to use lnet selftest to evaluate network

Re: [lustre-discuss] LNET Self-test

You should be able to do concurrent streams using --concurrency option. I would try with 2/4/8. -RG On Sun, Feb 5, 2017 at 1:30 PM Jon Tegner wrote: > Hi, > > I'm trying to use lnet selftest to evaluate network performance on a > test setup (only two machines). Using e.g., iperf

[lustre-discuss] LNET Self-test

Hi, I'm trying to use lnet selftest to evaluate network performance on a test setup (only two machines). Using e.g., iperf or Netpipe I've managed to demonstrate the bandwidth of the underlying 10 Gbits/s network (and typically you reach the expected bandwidth as the packet size increases).