I’ve tried to debug a little further into why the Intel X520 MTU can’t be set by comparing 2 log files (/var/svc/log/network-physical:default.log) from separate boots as I enabled and disabled my two 10GbE NICs. I have a log file where my Myricom 10GbE adapter MTU is set to 9000 and where the X520 fails to set the MTU. The Myricom has the “feature” that the MTU can only be set once at the start of a boot. Here are the two relevant excerpts from the log files. All lines prior in both log files are more or less identical (differences being MAC addresses and NIC tag names).
X520: + link=ixgbe0 + [[ -z ixgbe0 ]] + /usr/sbin/dladm show-linkprop -c -o value -p mtu ixgbe0 + curmtu=1500 + [[ 0 -eq 0 ]] + [[ 1500 -eq 9000 ]] + /usr/sbin/dladm set-linkprop -p mtu=9000 ixgbe0 /usr/sbin/dladm: warning: cannot set link property 'mtu' on 'ixgbe0': link busy + echo 'Failed to set mtu to 9000 for link ixgbe0' Failed to set mtu to 9000 for link ixgbe0 + exit 95 Myricom: + link=myri10ge0 + [[ -z myri10ge0 ]] + /usr/sbin/dladm show-linkprop -c -o value -p mtu myri10ge0 + curmtu=1500 + [[ 0 -eq 0 ]] + [[ 1500 -eq 9000 ]] + /usr/sbin/dladm set-linkprop -p mtu=9000 myri10ge0 + [[ true == true ]] Unfortunately I don’t know enough about Illumos internals to go much further. Any suggestions for further experiments? Thanks in advance, John > On Aug 15, 2017, at 3:06 PM, John Croix <[email protected] > <mailto:[email protected]>> wrote: > > In my quest for better connectivity between my Mac and my iSCSI volume on my > SmartOS server, I’ve picked up an Intel X520-SR2 10GbE NIC and put it into my > server. The system recognized the card without an issue, and I was able to > use nictagadm to add the NIC to the system. The problem is that it won’t > accept a MTU parameter of 9000. I had to comment the MTU line out of > /usbkey/config in order to get the server to fully boot. > > Here are the relevant lines in the /usbkey/config file: > ixgbe0_nic=0:1b:21:bc:51:7a > ixgbe0_ip=192.168.2.2 > ixgbe0_netmask=255.255.255.0 > ixgbe0_mtu=9000 > > From “svcs -x”, I see this: > svc:/network/physical:default (physical network interfaces) > State: maintenance since August 15, 2017 at 07:52:33 PM UTC > Reason: Start method exited with $SMF_EXIT_ERR_FATAL. > See: http://illumos.org/msg/SMF-8000-KS > <http://illumos.org/msg/SMF-8000-KS> > See: ifconfig(1M) > See: /var/svc/log/network-physical:default.log > Impact: 22 dependent services are not running. (Use -v for list.) > > Here’s the log file content, with the error at the end: > [ Aug 15 19:52:31 Executing start method ("/lib/svc/method/net-physical"). ] > [ Aug 15 19:52:31 Timeout override by svc.startd. Using infinite timeout. ] > + smf_configure_ip > + /sbin/zonename -t > + [ global = global -o shared = exclusive ] > + return 0 > + LD_LIBRARY_PATH=/lib > + export LD_LIBRARY_PATH > + ADMIN_DHCP_TIMEOUT=300 > + ActiveAggrLinks='' > + typeset -A ActiveAggrLinks > + smf_netstrategy > + smf_is_nonglobalzone > + [ global != global ] > + return 1 > + /sbin/netstrategy > + set -- ufs none none > + [ 0 -eq 0 ] > + [ ufs = nfs ] > + _INIT_NET_STRATEGY=none > + export _INIT_NET_STRATEGY > + typeset -A plumbedifs > + smf_is_globalzone > + [ global = global ] > + return 0 > + /usr/sbin/dladm init-phys > + log_if_state before > + echo '== debug start: before ==' > == debug start: before == > + /usr/sbin/dladm show-phys > LINK MEDIA STATE SPEED DUPLEX DEVICE > myri10ge0 Ethernet unknown 10000 full myri10ge0 > e1000g0 Ethernet unknown 0 half e1000g0 > e1000g1 Ethernet unknown 0 half e1000g1 > ixgbe0 Ethernet down 0 unknown ixgbe0 > ixgbe1 Ethernet down 0 unknown ixgbe1 > + /sbin/ifconfig -a > lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 > index 1 > inet 127.0.0.1 netmask ff000000 > lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 > index 1 > inet6 ::1/128 > + echo '== debug end: before ==' > == debug end: before == > + load_sdc_sysinfo > + boot_file_config_enabled > + load_sdc_config > + load_sdc_bootparams > + sed -e 's/,/ /g' > + echo '' > + create_aggrs > + typeset links macs mode mtu > + [[ -z '' ]] > + return 0 > + setup_mtu > + typeset tag oldifs val mac link curmtu > + typeset -A mtus > + typeset -A tagmap > + set -o xtrace > + oldifs=$' \t\n' > + IFS=, > + eval val='${CONFIG_secondary_mtu}' > + val='' > + eval mac='${CONFIG_secondary_nic}' > + mac=00:25:90:35:BB:AB > + [[ -z '' ]] > + continue > + eval val='${CONFIG_admin_mtu}' > + val='' > + eval mac='${CONFIG_admin_nic}' > + mac=00:25:90:35:BB:AA > + [[ -z '' ]] > + continue > + eval val='${CONFIG_ixgbe0_mtu}' > + val=9000 > + eval mac='${CONFIG_ixgbe0_nic}' > + mac=0:1b:21:bc:51:7a > + [[ -z 9000 ]] > + valid_mtu ixgbe0 9000 > + typeset tag mtu > + tag=ixgbe0 > + mtu=9000 > + [[ 9000 == ~(E)[1-9][0-9][0-9][0-9] ]] > + [[ 9000 -gt 9000 ]] > + [[ 9000 -lt 1500 ]] > + [[ -z '' ]] > + tagmap[0:1b:21:bc:51:7a]=ixgbe0 > + [[ -z '' ]] > + mtus[0:1b:21:bc:51:7a]=9000 > + IFS=$' \t\n' > + tag=ixgbe0 > + eval link='${SYSINFO_NIC_ixgbe0}' > + link=ixgbe0 > + [[ -z ixgbe0 ]] > + /usr/sbin/dladm show-linkprop -c -o value -p mtu ixgbe0 > + curmtu=1500 > + [[ 0 -eq 0 ]] > + [[ 1500 -eq 9000 ]] > + /usr/sbin/dladm set-linkprop -p mtu=9000 ixgbe0 > /usr/sbin/dladm: warning: cannot set link property 'mtu' on 'ixgbe0': link > busy > + echo 'Failed to set mtu to 9000 for link ixgbe0' > Failed to set mtu to 9000 for link ixgbe0 > + exit 95 > [ Aug 15 19:52:33 Method "start" exited with status 95. ] > > The diadm command shows that the card easily supports 9000: > [root@smartos /opt]# dladm show-linkprop -p mtu > LINK PROPERTY PERM VALUE DEFAULT POSSIBLE > myri10ge0 mtu rw 1500 1500 1500-9000 > e1000g0 mtu rw 1500 1500 1500-9216 > e1000g1 mtu rw 1500 1500 1500-9216 > ixgbe0 mtu rw 1500 1500 1500-15500 > ixgbe1 mtu rw 1500 1500 1500-15500 > eth0 mtu rw 1500 1500 1500 > eth0 mtu rw 1500 1500 1500 > net0 mtu rw 1500 1500 1500 > net0 mtu rw 1500 1500 1500 > eth0 mtu rw 1500 1500 1500 > net0 mtu rw 1500 1500 1500 > > So I’m not sure what’s wrong at the moment. Does anybody have any experience > with this card and a 9000 MTU? > > Regards, > John > > smartos-discuss | Archives > <https://www.listbox.com/member/archive/184463/=now> > <https://www.listbox.com/member/archive/rss/184463/26313201-b436b91a> | > Modify <https://www.listbox.com/member/?&;> Your Subscription > <http://www.listbox.com/> ------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
