Hello,
I have a lustre cluster using 25gbps ethernet network. (no infinitiband). I see lot of examples online for infiniband and what tunables to use for it, but I am struggling to find recommendations when using ethernet networks. Appreciate if someone can share their experience and settings when using ethernet or if there is any details online for ethernet recommended values. >From my lustre client node. I don’t see any LND tunables below like >(peercredits_hiw: 64, map_on_demand: 32, concurrent_sends: 256, fmr_pool_size: >2048, fmr_flush_trigger: 512, fmr_cache: 1). [root@lustre-client-1 ~]# lnetctl net show --verbose net: - net type: lo local NI(s): - nid: 0@lo status: up statistics: send_count: 0 recv_count: 0 drop_count: 0 tunables: peer_timeout: 0 peer_credits: 0 peer_buffer_credits: 0 credits: 0 dev cpt: 0 tcp bonding: 0 CPT: "[0,1,2,3,4,5,6,7,8,9,10,11]" - net type: tcp1 local NI(s): - nid: 10.0.3.4@tcp1 status: up interfaces: 0: ens3 statistics: send_count: 657209676 recv_count: 657208330 drop_count: 0 tunables: peer_timeout: 180 peer_credits: 8 peer_buffer_credits: 0 credits: 256 dev cpt: -1 tcp bonding: 0 CPT: "[0,1,2,3,4,5,6,7,8,9,10,11]" [root@lustre-client-1 ~]# OSS Server [root@lustre-oss-server-nic0-1 ~]# lnetctl net show --verbose net: - net type: lo local NI(s): - nid: 0@lo status: up statistics: send_count: 0 recv_count: 0 drop_count: 0 tunables: peer_timeout: 0 peer_credits: 0 peer_buffer_credits: 0 credits: 0 dev cpt: 0 tcp bonding: 0 CPT: "[0,1]" - net type: tcp1 local NI(s): - nid: 10.0.3.6@tcp1 status: up interfaces: 0: eno3d1 statistics: send_count: 232650108 recv_count: 232650019 drop_count: 0 tunables: peer_timeout: 180 peer_credits: 8 peer_buffer_credits: 0 credits: 256 dev cpt: 0 tcp bonding: 0 CPT: "[0,1]" [root@lustre-oss-server-nic0-1 ~]#
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
