Re: [gpfsug-discuss] RDMA data from Zimon

[Kristy Kallback-Rose]

Thanks Eric. No one who is a ZIMon developer has jumped up to contradict this, 
so I’ll go with it :-)

Many thanks. This is helpful to understand where the data is coming from and 
would be a welcome addition to the documentation.

Cheers,
Kristy

> On Feb 15, 2018, at 9:08 AM, Eric Agar  wrote:
> 
> Kristy,
> 
> I experimented a bit with this some months ago and looked at the ZIMon source 
> code. I came to the conclusion that ZIMon is reporting values obtained from 
> the IB counters (actually, delta values adjusted for time) and that yes, for 
> port_xmit_data and port_rcv_data, one would need to multiply the values by 4 
> to make sense of them.
> 
> To obtain a port_xmit_data value, the ZIMon sensor first looks for 
> /sys/class/infiniband//ports//counters_ext/port_xmit_data_64, 
> and if that is not found then looks for 
> /sys/class/infiniband//ports//counters/port_xmit_data. Similarly 
> for other counters/metrics.
> 
> Full disclosure: I am not an IB expert nor a ZIMon developer.
> 
> I hope this helps.
> 
> 
> Eric M. Agar
> a...@us.ibm.com
> 
> 
> Kristy Kallback-Rose ---02/14/2018 08:47:59 PM---Hi, Can one of 
> the IBMers tell me if port_xmit_data and port_rcv_data from Zimon can be 
> interpreted
> 
> From: Kristy Kallback-Rose 
> To: gpfsug main discussion list 
> Date: 02/14/2018 08:47 PM
> Subject: [gpfsug-discuss] RDMA data from Zimon
> Sent by: gpfsug-discuss-boun...@spectrumscale.org
> 
> 
> 
> 
> Hi,
> 
> Can one of the IBMers tell me if port_xmit_data and port_rcv_data from Zimon 
> can be interpreted as RDMA Bytes/sec? Ideally, also how this data is being 
> collected? I’m looking here: 
> https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1hlp_monnetworksmetrics.htm
>  
> 
> 
> But then I also look here: https://community.mellanox.com/docs/DOC-2751 
> 
> 
> and see "Total number of data octets, divided by 4 (lanes), received on all 
> VLs. This is 64 bit counter.” So I wasn’t sure if some multiplication by 4 
> was in order.
> 
> Please advise.
> 
> Cheers,
> Kristy___
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss=DwICAg=jf_iaSHvJObTbx-siA1ZOg=IbxtjdkPAM2Sbon4Lbbi4w=zIRb70L9sx_FvvC9IcWVKLOSOOFnx-hIGfjw0kUN7bw=D1g4YTG5WeUiHI3rCPr_kkPxbG9V9E-18UGXBeCvfB8=
>  
> 
> 
> 
> 
> ___
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss

___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] RDMA data from Zimon

[Eric Agar]

Kristy,

I experimented a bit with this some months ago and looked at the ZIMon
source code.  I came to the conclusion that ZIMon is reporting values
obtained from the IB counters (actually, delta values adjusted for time)
and that yes, for port_xmit_data and port_rcv_data, one would need to
multiply the values by 4 to make sense of them.

To obtain a port_xmit_data value, the ZIMon sensor first looks
for /sys/class/infiniband//ports//counters_ext/port_xmit_data_64,
 and if that is not found then looks
for /sys/class/infiniband//ports//counters/port_xmit_data.
Similarly for other counters/metrics.

Full disclosure: I am not an IB expert nor a ZIMon developer.

I hope this helps.


Eric M. Agar
a...@us.ibm.com




From:   Kristy Kallback-Rose 
To: gpfsug main discussion list 
Date:   02/14/2018 08:47 PM
Subject:[gpfsug-discuss] RDMA data from Zimon
Sent by:gpfsug-discuss-boun...@spectrumscale.org



Hi,

Can one of the IBMers tell me if port_xmit_data and port_rcv_data from
Zimon can be interpreted as RDMA Bytes/sec? Ideally, also how this data is
being collected? I’m looking here:
https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1hlp_monnetworksmetrics.htm

But then I also look here: https://community.mellanox.com/docs/DOC-2751

and see "Total number of data octets, divided by 4 (lanes), received on all
VLs. This is 64 bit counter.” So I wasn’t sure if some multiplication by 4
was in order.

Please advise.

Cheers,
Kristy___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss=DwICAg=jf_iaSHvJObTbx-siA1ZOg=IbxtjdkPAM2Sbon4Lbbi4w=zIRb70L9sx_FvvC9IcWVKLOSOOFnx-hIGfjw0kUN7bw=D1g4YTG5WeUiHI3rCPr_kkPxbG9V9E-18UGXBeCvfB8=



___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss