Hi Robert,
Changing the name led to a really interesting discovery. I changed my choice of
names to “abcdef", and now the MTU setting works for the X520. However, when my
Myricom card was initialized, the MTU couldn’t be set to 9000. Previously the
Myricom card was being set first, and the Intel X520 was being set second,
which is when the failure occurred. Now the X520 is being successfully set
first, and the Myricom card is failing.
Command trace:
- SmartOS (build: 20170720T001051Z)
[root@smartos ~]# cd /usbkey
[root@smartos /usbkey]# grep ixgbe config
[root@smartos /usbkey]# nictagadm add -p mtu=9000 abcdef 0:1b:21:bc:51:7a
MTU changes will not take effect until next reboot
[root@smartos /usbkey]# grep abc config
abcdef_nic=00:1b:21:bc:51:7a
abcdef_mtu=9000
[root@smartos /usbkey]# reboot
Here’s the log:
[ Aug 19 01:43:23 Executing start method ("/lib/svc/method/net-physical"). ]
[ Aug 19 01:43:23 Timeout override by svc.startd. Using infinite timeout. ]
+ smf_configure_ip
+ /sbin/zonename -t
+ [ global = global -o shared = exclusive ]
+ return 0
+ LD_LIBRARY_PATH=/lib
+ export LD_LIBRARY_PATH
+ ADMIN_DHCP_TIMEOUT=300
+ ActiveAggrLinks=''
+ typeset -A ActiveAggrLinks
+ smf_netstrategy
+ smf_is_nonglobalzone
+ [ global != global ]
+ return 1
+ /sbin/netstrategy
+ set -- ufs none none
+ [ 0 -eq 0 ]
+ [ ufs = nfs ]
+ _INIT_NET_STRATEGY=none
+ export _INIT_NET_STRATEGY
+ typeset -A plumbedifs
+ smf_is_globalzone
+ [ global = global ]
+ return 0
+ /usr/sbin/dladm init-phys
+ log_if_state before
+ echo '== debug start: before =='
== debug start: before ==
+ /usr/sbin/dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
myri10ge0 Ethernet unknown 10000 full myri10ge0
e1000g1 Ethernet unknown 0 half e1000g1
e1000g0 Ethernet unknown 0 half e1000g0
ixgbe0 Ethernet down 0 unknown ixgbe0
ixgbe1 Ethernet down 0 unknown ixgbe1
+ /sbin/ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232
index 1
inet 127.0.0.1 netmask ff000000
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252
index 1
inet6 ::1/128
+ echo '== debug end: before =='
== debug end: before ==
+ load_sdc_sysinfo
+ boot_file_config_enabled
+ load_sdc_config
+ load_sdc_bootparams
+ sed -e 's/,/ /g'
+ echo ''
+ create_aggrs
+ typeset links macs mode mtu
+ [[ -z '' ]]
+ return 0
+ setup_mtu
+ typeset tag oldifs val mac link curmtu
+ typeset -A mtus
+ typeset -A tagmap
+ set -o xtrace
+ oldifs=$' \t\n'
+ IFS=,
+ eval val='${CONFIG_secondary_mtu}'
+ val=''
+ eval mac='${CONFIG_secondary_nic}'
+ mac=00:25:90:35:BB:AB
+ [[ -z '' ]]
+ continue
+ eval val='${CONFIG_abcdef_mtu}'
+ val=9000
+ eval mac='${CONFIG_abcdef_nic}'
+ mac=00:1b:21:bc:51:7a
+ [[ -z 9000 ]]
+ valid_mtu abcdef 9000
+ typeset tag mtu
+ tag=abcdef
+ mtu=9000
+ [[ 9000 == ~(E)[1-9][0-9][0-9][0-9] ]]
+ [[ 9000 -gt 9000 ]]
+ [[ 9000 -lt 1500 ]]
+ [[ -z '' ]]
+ tagmap[00:1b:21:bc:51:7a]=abcdef
+ [[ -z '' ]]
+ mtus[00:1b:21:bc:51:7a]=9000
+ eval val='${CONFIG_myricom_mtu}'
+ val=9000
+ eval mac='${CONFIG_myricom_nic}'
+ mac=0:60:dd:45:73:22
+ [[ -z 9000 ]]
+ valid_mtu myricom 9000
+ typeset tag mtu
+ tag=myricom
+ mtu=9000
+ [[ 9000 == ~(E)[1-9][0-9][0-9][0-9] ]]
+ [[ 9000 -gt 9000 ]]
+ [[ 9000 -lt 1500 ]]
+ [[ -z '' ]]
+ tagmap[0:60:dd:45:73:22]=myricom
+ [[ -z '' ]]
+ mtus[0:60:dd:45:73:22]=9000
+ eval val='${CONFIG_admin_mtu}'
+ val=''
+ eval mac='${CONFIG_admin_nic}'
+ mac=00:25:90:35:BB:AA
+ [[ -z '' ]]
+ continue
+ IFS=$' \t\n'
+ tag=abcdef
+ eval link='${SYSINFO_NIC_abcdef}'
+ link=ixgbe0
+ [[ -z ixgbe0 ]]
+ /usr/sbin/dladm show-linkprop -c -o value -p mtu ixgbe0
+ curmtu=1500
+ [[ 0 -eq 0 ]]
+ [[ 1500 -eq 9000 ]]
+ /usr/sbin/dladm set-linkprop -p mtu=9000 ixgbe0
+ tag=myricom
+ eval link='${SYSINFO_NIC_myricom}'
+ link=myri10ge0
+ [[ -z myri10ge0 ]]
+ /usr/sbin/dladm show-linkprop -c -o value -p mtu myri10ge0
+ curmtu=1500
+ [[ 0 -eq 0 ]]
+ [[ 1500 -eq 9000 ]]
+ /usr/sbin/dladm set-linkprop -p mtu=9000 myri10ge0
/usr/sbin/dladm: warning: cannot set link property 'mtu' on 'myri10ge0': link
busy
+ echo 'Failed to set mtu to 9000 for link myri10ge0'
Failed to set mtu to 9000 for link myri10ge0
+ exit 95
[ Aug 19 01:43:25 Method "start" exited with status 95. ]
Regards,
John
> On Aug 18, 2017, at 6:42 PM, Robert Mustacchi <[email protected]> wrote:
>
> On 8/18/17 16:38 , John Croix wrote:
>> Hi Robert,
>>
>> Per instructions, I took out every single reference to “ixgbe0” from my
>> /usbkey/config file. I then rebooted the system. Next, I issued the
>> nictagadm command:
>>
>> - SmartOS (build: 20170720T001051Z)
>> [root@smartos ~]# cd /usbkey
>> [root@smartos /usbkey]# grep ixgbe0 config
>> [root@smartos /usbkey]# nictagadm add -p mtu=9000 ixgbe0 0:1b:21:bc:51:7a
>> MTU changes will not take effect until next reboot
>> [root@smartos /usbkey]# grep ixgbe0 config
>> ixgbe0_nic=00:1b:21:bc:51:7a
>> ixgbe0_mtu=9000
>> [root@smartos /usbkey]# reboot
>>
>> Upon rebooting, I had the very same issue, with the same error being
>> displayed in the log file about the MTU. Once I commented out the MTU
>> setting in /usbkey/config, I was able to boot the system normally, but at
>> the default MTU of 1500.
>
> That's a bit surprising. I guess we never name our nic tags with the
> same name as the device. Would you mind just making up some random name
> that doesn't match the device for the tag? If that fails, then can you
> share the failed service log?
>
>> Could this have anything to do with the fact that this is an OEM X520
>> instead of Intel-branded? It doesn’t matter for Linux, but could it make a
>> difference for SmartOS? Here’s the output from lspci on Linux for this
>> device:
>>
>> jcroix@ubuntu-server:~$ lspci -nn -vvv | grep Ethernet
>> 00:19.0 Ethernet controller [0200]: Intel Corporation 82579V Gigabit Network
>> Connection [8086:1503] (rev 06)
>> 02:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit
>> SFI/SFP+ Network Connection [8086:10fb] (rev 01)
>> Subsystem: Intel Corporation Ethernet Server Adapter X520-1 [8086:000a]
>> 02:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit
>> SFI/SFP+ Network Connection [8086:10fb] (rev 01)
>> Subsystem: Intel Corporation Ethernet Server Adapter X520-1 [8086:000a]
>>
>> Notice that the subsystem identifier is [8086:000a], which is NOT the same
>> subsystem ID for Intel-branded boards. Dell uses one subsystem ID, Small
>> Tree another, etc. It’s still an Intel X520, and it runs perfectly as such
>> under Linux using the Intel driver. And, of course, it works fine under
>> SmartOS at a MTU of 1500. Thought I’d mention it, “just in case”.
>
> That shouldn't make a difference, though thanks for mentioning it. From
> my memory we don't use the subdevice ids in the driver at all.
>
> Robert
>
-------------------------------------------
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription:
https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com
